2024-05-10T05:01:33.627 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-10T05:01:33.627 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} --archive /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605 --verbose -- /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605/orig.config.yaml 2024-05-10T05:01:33.681 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T05:01:33.776 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605 branch: wip-pdonnell-testing-20240509.171335-reef-debug description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} email: pdonnell@redhat.com first_in_suite: false job_id: '7700605' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240509.171335-reef-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 50174015a7e79f951b988949abe236791f52d4da subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 50174015a7e79f951b988949abe236791f52d4da kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240509.171335-reef-debug sha1: 50174015a7e79f951b988949abe236791f52d4da owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4933 sha1: 50174015a7e79f951b988949abe236791f52d4da sleep_before_teardown: 0 subset: 420/512 suite: fs suite_branch: wip-pdonnell-testing-20240509.171335-reef-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 50174015a7e79f951b988949abe236791f52d4da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO5EKF7TBei+eRPGMVwuBHPF6QhLH2JRopWs+IJtyl/xi47+WjVoX3ZBNDcMvseFe4nIdJdq1EUc7GpcQkWdFiQ= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBtSuA7cJ3vkaVLejTqaBBmMNUQRhq+6V+KPJ0vpwE1ihqzHlJl/DVlNNcyFbt2GCnAtO6I8/hLFj6MPCtWNgmY= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG4WEsA0WhE+Bb8atWFxUXNDME3D9QvolsUx5CsLl+vIxdbE+rd/ok68OZ1JLXzH9FQauan7vyfMiIu93XCieYQ= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-10_02:33:39 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-10T05:01:33.776 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa; will attempt to use it 2024-05-10T05:01:33.777 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa/tasks 2024-05-10T05:01:33.777 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-10T05:01:33.779 INFO:teuthology.task.internal:Checking packages... 2024-05-10T05:01:33.803 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '50174015a7e79f951b988949abe236791f52d4da' 2024-05-10T05:01:33.803 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-10T05:01:33.803 INFO:teuthology.packaging:ref: None 2024-05-10T05:01:33.803 INFO:teuthology.packaging:tag: None 2024-05-10T05:01:33.803 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T05:01:33.804 INFO:teuthology.packaging:sha1: 50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:01:33.804 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T05:01:34.115 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1330.g50174015 2024-05-10T05:01:34.116 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-10T05:01:34.123 INFO:teuthology.task.internal:no buildpackages task found 2024-05-10T05:01:34.124 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-10T05:01:34.289 INFO:teuthology.task.internal:Saving configuration 2024-05-10T05:01:34.309 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-10T05:01:34.344 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T05:01:34.371 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 04:55:44.751230', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO5EKF7TBei+eRPGMVwuBHPF6QhLH2JRopWs+IJtyl/xi47+WjVoX3ZBNDcMvseFe4nIdJdq1EUc7GpcQkWdFiQ='} 2024-05-10T05:01:34.397 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 04:55:44.749909', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBtSuA7cJ3vkaVLejTqaBBmMNUQRhq+6V+KPJ0vpwE1ihqzHlJl/DVlNNcyFbt2GCnAtO6I8/hLFj6MPCtWNgmY='} 2024-05-10T05:01:34.416 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 04:55:44.752417', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG4WEsA0WhE+Bb8atWFxUXNDME3D9QvolsUx5CsLl+vIxdbE+rd/ok68OZ1JLXzH9FQauan7vyfMiIu93XCieYQ='} 2024-05-10T05:01:34.416 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-10T05:01:34.467 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-10T05:01:34.467 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-10T05:01:34.467 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-10T05:01:34.467 INFO:teuthology.run_tasks:Running task console_log... 2024-05-10T05:01:34.615 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2b118cd0d0>, signals=[15]) 2024-05-10T05:01:34.616 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-10T05:01:34.622 INFO:teuthology.task.internal:Opening connections... 2024-05-10T05:01:34.622 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-05-10T05:01:34.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T05:01:34.699 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-05-10T05:01:34.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T05:01:34.772 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-05-10T05:01:34.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T05:01:34.844 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-10T05:01:34.851 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-05-10T05:01:34.872 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-05-10T05:01:34.873 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-05-10T05:01:34.932 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="Red Hat Enterprise Linux" 2024-05-10T05:01:34.932 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8.6 (Ootpa)" 2024-05-10T05:01:34.932 INFO:teuthology.orchestra.run.smithi131.stdout:ID="rhel" 2024-05-10T05:01:34.932 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="fedora" 2024-05-10T05:01:34.932 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8.6" 2024-05-10T05:01:34.932 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2024-05-10T05:01:34.932 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-10T05:01:34.933 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-05-10T05:01:34.933 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-10T05:01:34.933 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://www.redhat.com/" 2024-05-10T05:01:34.933 INFO:teuthology.orchestra.run.smithi131.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-10T05:01:34.933 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T05:01:34.933 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:01:34.933 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-10T05:01:34.933 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-10T05:01:34.933 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-10T05:01:34.933 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-10T05:01:34.934 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-05-10T05:01:34.953 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-05-10T05:01:34.973 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-05-10T05:01:34.973 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-05-10T05:01:35.033 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Red Hat Enterprise Linux" 2024-05-10T05:01:35.033 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8.6 (Ootpa)" 2024-05-10T05:01:35.033 INFO:teuthology.orchestra.run.smithi142.stdout:ID="rhel" 2024-05-10T05:01:35.033 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="fedora" 2024-05-10T05:01:35.033 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8.6" 2024-05-10T05:01:35.033 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8" 2024-05-10T05:01:35.033 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-10T05:01:35.033 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-05-10T05:01:35.033 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-10T05:01:35.033 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.redhat.com/" 2024-05-10T05:01:35.034 INFO:teuthology.orchestra.run.smithi142.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-10T05:01:35.034 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T05:01:35.034 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:01:35.034 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-10T05:01:35.034 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-10T05:01:35.034 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-10T05:01:35.034 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-10T05:01:35.034 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-05-10T05:01:35.053 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-05-10T05:01:35.075 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-05-10T05:01:35.075 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-05-10T05:01:35.137 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux" 2024-05-10T05:01:35.137 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.6 (Ootpa)" 2024-05-10T05:01:35.137 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel" 2024-05-10T05:01:35.138 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora" 2024-05-10T05:01:35.138 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.6" 2024-05-10T05:01:35.138 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2024-05-10T05:01:35.138 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-10T05:01:35.138 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-05-10T05:01:35.138 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-10T05:01:35.138 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/" 2024-05-10T05:01:35.138 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-10T05:01:35.139 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T05:01:35.139 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:01:35.139 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-10T05:01:35.139 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-10T05:01:35.139 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-10T05:01:35.139 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-10T05:01:35.139 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-05-10T05:01:35.161 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-10T05:01:35.173 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-10T05:01:35.261 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-10T05:01:35.262 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-05-10T05:01:35.264 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-05-10T05:01:35.266 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-05-10T05:01:35.285 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-10T05:01:35.399 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-10T05:01:35.399 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-05-10T05:01:35.401 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-05-10T05:01:35.403 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-05-10T05:01:35.433 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-10T05:01:35.503 INFO:teuthology.run_tasks:Running task kernel... 2024-05-10T05:01:35.517 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-10T05:01:35.518 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-10T05:01:35.518 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-10T05:01:35.518 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T05:01:35.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:01:35.538 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2024-05-10T05:01:35.717 DEBUG:teuthology.task.kernel:sha1 for client.0 is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T05:01:35.717 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-10T05:01:35.717 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-05-10T05:01:35.737 INFO:teuthology.orchestra.run.smithi131.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-10T05:01:35.737 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T05:01:35.738 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-10T05:01:35.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum upgrade -y linux-firmware 2024-05-10T05:01:37.272 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-05-10T05:01:37.273 INFO:teuthology.orchestra.run.smithi131.stdout:Unable to read consumer identity 2024-05-10T05:01:37.532 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:01:37.532 INFO:teuthology.orchestra.run.smithi131.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-10T05:01:37.532 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:01:39.472 INFO:teuthology.orchestra.run.smithi131.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-05-10T05:01:40.731 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 11 MB/s | 14 MB 00:01 2024-05-10T05:01:44.015 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 456 kB/s | 24 kB 00:00 2024-05-10T05:01:44.856 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-10T05:01:44.857 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-05-10T05:01:44.857 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-10T05:01:45.007 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.0... 2024-05-10T05:01:45.007 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T05:01:45.236 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/8/flavors/default/x86_64/ 2024-05-10T05:01:45.236 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/8/flavors/default/x86_64/ --input-file=- 2024-05-10T05:01:45.238 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi131.front.sepia.ceph.com to complete... 2024-05-10T05:01:59.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10 05:01:59 URL:https://1.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/8/flavors/default/x86_64/kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm [425100552/425100552] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-10T05:01:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:FINISHED --2024-05-10 05:01:59-- 2024-05-10T05:01:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:Total wall clock time: 14s 2024-05-10T05:01:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:Downloaded: 1 files, 405M in 14s (28.9 MB/s) 2024-05-10T05:01:59.476 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.0... 2024-05-10T05:01:59.476 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-10T05:02:01.414 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying... ######################################## 2024-05-10T05:02:01.496 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing... ######################################## 2024-05-10T05:02:01.638 INFO:teuthology.orchestra.run.smithi131.stdout:Updating / installing... 2024-05-10T05:02:44.399 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-6.9.0_rc3_gfb9a6a44107e-1 ######################################## 2024-05-10T05:02:48.820 INFO:teuthology.orchestra.run.smithi131.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-05-10T05:03:09.183 INFO:teuthology.orchestra.run.smithi131.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-05-10T05:05:28.340 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-10T05:05:28.341 DEBUG:teuthology.orchestra.run.smithi131:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-10T05:05:28.397 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-10T05:05:28.397 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-10T05:05:28.397 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/System.map 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/config 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-10T05:05:28.398 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-10T05:05:28.399 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-10T05:05:28.400 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-10T05:05:28.401 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-10T05:05:28.402 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-10T05:05:28.403 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-10T05:05:28.404 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ata_generic.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-10T05:05:28.405 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-10T05:05:28.406 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-10T05:05:28.407 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sis.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-10T05:05:28.408 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap 2024-05-10T05:05:28.409 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/brd.ko 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/loop.ko 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx 2024-05-10T05:05:28.410 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/virtio_blk.ko 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkfront.ko 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char 2024-05-10T05:05:28.411 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-10T05:05:28.412 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/uv_mmtimer.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/virtio_console.ko 2024-05-10T05:05:28.413 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat 2024-05-10T05:05:28.414 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw 2024-05-10T05:05:28.415 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-10T05:05:28.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat 2024-05-10T05:05:28.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-10T05:05:28.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac 2024-05-10T05:05:28.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-10T05:05:28.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-10T05:05:28.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-10T05:05:28.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-10T05:05:28.416 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/edd.ko 2024-05-10T05:05:28.417 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-10T05:05:28.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio 2024-05-10T05:05:28.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-10T05:05:28.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-10T05:05:28.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid 2024-05-10T05:05:28.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-10T05:05:28.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-10T05:05:28.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-10T05:05:28.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-10T05:05:28.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-10T05:05:28.418 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-10T05:05:28.419 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-10T05:05:28.420 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-10T05:05:28.421 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-10T05:05:28.422 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-10T05:05:28.423 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-10T05:05:28.424 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-10T05:05:28.425 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-10T05:05:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-10T05:05:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-10T05:05:28.428 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-10T05:05:28.429 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-10T05:05:28.430 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-10T05:05:28.431 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-10T05:05:28.432 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-10T05:05:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-10T05:05:28.434 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-dev.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband 2024-05-10T05:05:28.435 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-10T05:05:28.436 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt 2024-05-10T05:05:28.437 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-10T05:05:28.438 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-10T05:05:28.439 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-10T05:05:28.440 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-10T05:05:28.441 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion 2024-05-10T05:05:28.442 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mfd-core.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-10T05:05:28.443 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-10T05:05:28.444 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp 2024-05-10T05:05:28.445 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa 2024-05-10T05:05:28.446 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-10T05:05:28.447 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-10T05:05:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-10T05:05:28.449 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio 2024-05-10T05:05:28.450 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-10T05:05:28.451 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex 2024-05-10T05:05:28.452 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e 2024-05-10T05:05:28.453 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell 2024-05-10T05:05:28.454 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-10T05:05:28.455 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-10T05:05:28.456 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc 2024-05-10T05:05:28.457 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-10T05:05:28.458 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-10T05:05:28.459 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-10T05:05:28.460 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/net_failover.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-10T05:05:28.461 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/dp83640.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-10T05:05:28.462 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-10T05:05:28.463 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slhc.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-10T05:05:28.464 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tun.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/virtio_net.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netfront.ko 2024-05-10T05:05:28.465 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug 2024-05-10T05:05:28.466 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-10T05:05:28.467 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-10T05:05:28.468 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps 2024-05-10T05:05:28.469 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-10T05:05:28.470 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-10T05:05:28.471 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-10T05:05:28.472 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-10T05:05:28.473 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-10T05:05:28.474 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-10T05:05:28.475 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-10T05:05:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-10T05:05:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-10T05:05:28.478 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-10T05:05:28.479 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-10T05:05:28.480 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-10T05:05:28.481 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty 2024-05-10T05:05:28.482 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-10T05:05:28.483 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-10T05:05:28.484 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-10T05:05:28.485 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-10T05:05:28.486 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-10T05:05:28.487 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-10T05:05:28.488 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-10T05:05:28.489 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-10T05:05:28.490 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-10T05:05:28.491 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio.ko 2024-05-10T05:05:28.492 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_balloon.ko 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_mmio.ko 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci.ko 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_ring.ko 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-10T05:05:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-10T05:05:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-10T05:05:28.495 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-10T05:05:28.496 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-10T05:05:28.497 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-10T05:05:28.498 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/fat.ko 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/vfat.ko 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/fuse.ko 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs 2024-05-10T05:05:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs 2024-05-10T05:05:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-10T05:05:28.501 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-10T05:05:28.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-10T05:05:28.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-10T05:05:28.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster 2024-05-10T05:05:28.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs 2024-05-10T05:05:28.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf 2024-05-10T05:05:28.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu/rcutorture.ko 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/asn1_encoder.ko 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-10T05:05:28.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6 2024-05-10T05:05:28.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd/zstd_compress.ko 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-10T05:05:28.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-10T05:05:28.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter 2024-05-10T05:05:28.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-10T05:05:28.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core 2024-05-10T05:05:28.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/failover.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver/dns_resolver.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-10T05:05:28.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-10T05:05:28.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-10T05:05:28.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-10T05:05:28.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-10T05:05:28.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-10T05:05:28.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6 2024-05-10T05:05:28.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-10T05:05:28.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-10T05:05:28.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-10T05:05:28.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-10T05:05:28.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-10T05:05:28.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-10T05:05:28.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-10T05:05:28.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter 2024-05-10T05:05:28.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-10T05:05:28.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-10T05:05:28.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-10T05:05:28.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-10T05:05:28.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-10T05:05:28.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-10T05:05:28.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-10T05:05:28.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-10T05:05:28.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-10T05:05:28.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_quota.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tunnel.ko 2024-05-10T05:05:28.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-10T05:05:28.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-10T05:05:28.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-10T05:05:28.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-10T05:05:28.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-10T05:05:28.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-10T05:05:28.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-10T05:05:28.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-10T05:05:28.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-10T05:05:28.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma 2024-05-10T05:05:28.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-10T05:05:28.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys/trusted.ko 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-10T05:05:28.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-10T05:05:28.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-10T05:05:28.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-10T05:05:28.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-10T05:05:28.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-10T05:05:28.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-10T05:05:28.548 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-10T05:05:28.548 DEBUG:teuthology.orchestra.run.smithi131:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-10T05:05:28.572 INFO:teuthology.orchestra.run.smithi131.stdout:postinstall scriptlet (using /bin/sh): 2024-05-10T05:05:28.572 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-10T05:05:28.572 INFO:teuthology.orchestra.run.smithi131.stdout: /usr/bin/kernel-install add 6.9.0-rc3-gfb9a6a44107e /lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-10T05:05:28.572 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-10T05:05:28.572 INFO:teuthology.orchestra.run.smithi131.stdout:for file in vmlinuz System.map config; do 2024-05-10T05:05:28.572 INFO:teuthology.orchestra.run.smithi131.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e"; then 2024-05-10T05:05:28.572 INFO:teuthology.orchestra.run.smithi131.stdout: cp "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T05:05:28.572 INFO:teuthology.orchestra.run.smithi131.stdout: fi 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout:done 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -d "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" ] && \ 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" >/dev/null 2>&1; then 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout: rm -rf "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout: cp -r "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep" ]; then 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout: /usr/sbin/depmod 6.9.0-rc3-gfb9a6a44107e 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-install remove 6.9.0-rc3-gfb9a6a44107e 2024-05-10T05:05:28.573 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-10T05:05:28.574 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -qi grub2-tools 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:Name : grub2-tools 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:Epoch : 1 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:Version : 2.02 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:Release : 123.el8 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: x86_64 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:Group : System Environment/Base 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:Size : 9265450 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:License : GPLv3+ 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-05-10T05:05:28.660 INFO:teuthology.orchestra.run.smithi131.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:Relocations : (not relocatable) 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:Packager : Red Hat, Inc. 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:Vendor : Red Hat, Inc. 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:Summary : Support tools for GRUB. 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:Description : 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout:hardware devices. 2024-05-10T05:05:28.661 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:05:28.662 INFO:teuthology.orchestra.run.smithi131.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T05:05:28.662 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T05:05:28.662 INFO:teuthology.task.kernel:Updating grub on smithi131 to boot 6.9.0-rc3-gfb9a6a44107e 2024-05-10T05:05:28.663 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T05:05:29.450 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2024-05-10T05:05:31.652 INFO:teuthology.orchestra.run.smithi131.stderr:done 2024-05-10T05:05:31.654 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-05-10T05:05:31.671 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.HzkwyEocRW 2024-05-10T05:05:31.671 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HzkwyEocRW 2024-05-10T05:05:31.745 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.HzkwyEocRW 2024-05-10T05:05:31.915 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.HzkwyEocRW is 6KB 2024-05-10T05:05:31.963 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.HzkwyEocRW 2024-05-10T05:05:31.979 DEBUG:teuthology.orchestra.run.smithi131:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T05:05:32.056 INFO:teuthology.orchestra.run.smithi131.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-05-10T05:05:32.057 INFO:teuthology.orchestra.run.smithi131.stdout:2db68f91c8f90721a6596efe657a5324-6.9.0-rc3-gfb9a6a44107e.conf 2024-05-10T05:05:32.057 INFO:teuthology.orchestra.run.smithi131.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-05-10T05:05:32.057 INFO:teuthology.orchestra.run.smithi131.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-05-10T05:05:32.057 INFO:teuthology.orchestra.run.smithi131.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-05-10T05:05:32.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.9.0-rc3-gfb9a6a44107e 2024-05-10T05:05:32.218 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2024-05-10T05:06:02.222 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-10T05:06:02.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-10T05:06:02.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T05:06:20.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-05-10T05:06:29.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-10T05:06:29.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T05:06:47.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-05-10T05:06:59.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-10T05:06:59.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T05:07:03.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-05-10T05:07:18.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-10T05:07:18.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T05:07:18.596 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-05-10T05:07:19.126 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-10T05:07:19.126 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-10T05:07:19.126 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-05-10T05:07:19.166 INFO:teuthology.orchestra.run.smithi131.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-10T05:07:19.167 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-10T05:07:19.167 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T05:07:19.167 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-05-10T05:07:20.168 DEBUG:teuthology.parallel:result is None 2024-05-10T05:07:20.168 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-10T05:07:20.175 INFO:teuthology.task.internal:Creating test directory... 2024-05-10T05:07:20.176 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T05:07:20.180 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T05:07:20.183 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T05:07:20.219 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-10T05:07:20.225 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-10T05:07:20.261 INFO:teuthology.task.internal:Creating archive directory... 2024-05-10T05:07:20.261 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T05:07:20.265 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T05:07:20.267 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T05:07:20.318 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-10T05:07:20.325 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-10T05:07:20.325 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T05:07:20.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:07:20.358 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T05:07:20.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:07:20.377 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T05:07:20.381 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T05:07:20.402 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T05:07:20.454 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T05:07:20.481 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T05:07:20.503 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T05:07:20.506 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T05:07:20.552 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T05:07:20.587 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T05:07:20.591 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-10T05:07:20.598 INFO:teuthology.task.internal:Configuring sudo... 2024-05-10T05:07:20.599 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T05:07:20.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T05:07:20.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T05:07:20.672 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-10T05:07:20.682 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-10T05:07:20.682 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T05:07:20.686 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T05:07:20.688 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T05:07:20.724 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T05:07:20.837 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T05:07:20.959 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T05:07:21.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T05:07:21.108 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:07:21.108 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T05:07:21.191 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T05:07:21.239 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T05:07:21.290 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T05:07:21.377 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T05:07:21.422 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:07:21.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T05:07:21.491 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T05:07:21.540 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T05:07:21.593 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T05:07:21.679 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T05:07:21.724 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:07:21.725 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T05:07:21.793 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-05-10T05:07:21.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-05-10T05:07:21.799 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-05-10T05:07:21.858 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T05:07:21.937 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T05:07:22.081 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T05:07:22.378 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-10T05:07:22.389 INFO:teuthology.task.internal:Starting timer... 2024-05-10T05:07:22.389 INFO:teuthology.run_tasks:Running task pcp... 2024-05-10T05:07:22.412 INFO:teuthology.run_tasks:Running task selinux... 2024-05-10T05:07:22.445 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-05-10T05:07:22.445 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-05-10T05:07:22.574 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-05-10T05:07:22.578 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-05-10T05:07:22.653 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-05-10T05:07:22.655 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-05-10T05:07:22.736 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-05-10T05:07:22.738 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-10T05:07:22.738 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-05-10T05:07:22.789 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-05-10T05:07:22.789 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-05-10T05:07:22.816 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-05-10T05:07:22.817 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-05-10T05:07:22.844 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-05-10T05:07:22.844 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-05-10T05:07:22.845 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T05:07:22.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:07:22.911 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T05:07:22.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:07:22.939 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T05:07:22.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:07:22.969 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-10T05:07:22.969 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-05-10T05:07:23.034 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-05-10T05:07:23.065 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-05-10T05:07:23.092 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-10T05:07:23.103 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-10T05:07:23.108 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-10T05:07:23.255 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-10T05:07:23.274 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-10T05:07:23.275 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi142.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-10T05:19:18.964 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-05-10T05:19:18.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-10T05:19:18.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T05:19:19.059 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-05-10T05:19:19.185 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-10T05:19:19.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-05-10T05:19:19.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T05:19:19.257 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-05-10T05:19:19.329 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-05-10T05:19:19.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-05-10T05:19:19.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T05:19:19.405 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-05-10T05:19:19.480 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-05-10T05:19:19.481 INFO:teuthology.run_tasks:Running task clock... 2024-05-10T05:19:19.494 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-10T05:19:19.494 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T05:19:19.495 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T05:19:19.499 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T05:19:19.499 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T05:19:19.501 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T05:19:19.501 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T05:19:19.537 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T05:19:19.553 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T05:19:19.557 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T05:19:19.567 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T05:19:19.573 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T05:19:19.600 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T05:19:19.637 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-05-10T05:19:19.644 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-05-10T05:19:19.650 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-05-10T05:19:19.656 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-05-10T05:19:19.666 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T05:19:19.672 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T05:19:19.681 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T05:19:19.686 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T05:19:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-05-10T05:19:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-05-10T05:19:19.752 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2024-05-10T05:19:19.753 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T05:19:19.756 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T05:19:19.756 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-05-10T05:19:19.756 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.756 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.756 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.756 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:bash: ntpq: command not found 2024-05-10T05:19:19.770 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T05:19:19.770 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-05-10T05:19:19.770 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.770 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.770 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.770 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.784 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T05:19:19.877 INFO:teuthology.orchestra.run.smithi131.stderr:bash: ntpq: command not found 2024-05-10T05:19:19.883 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T05:19:19.884 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-05-10T05:19:19.884 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.884 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.884 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.884 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:19:19.885 INFO:teuthology.run_tasks:Running task pexec... 2024-05-10T05:19:19.894 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-10T05:19:19.894 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi131.front.sepia.ceph.com 2024-05-10T05:19:19.895 DEBUG:teuthology.orchestra.run.smithi131:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-10T05:19:19.895 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi142.front.sepia.ceph.com 2024-05-10T05:19:19.895 DEBUG:teuthology.orchestra.run.smithi142:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-10T05:19:19.895 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi157.front.sepia.ceph.com 2024-05-10T05:19:19.896 DEBUG:teuthology.orchestra.run.smithi157:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-10T05:19:20.273 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-05-10T05:19:20.273 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2024-05-10T05:19:20.382 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-05-10T05:19:22.279 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-05-10T05:19:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-05-10T05:19:22.499 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-05-10T05:19:22.641 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-05-10T05:19:22.955 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-05-10T05:19:22.959 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-05-10T05:19:25.573 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-05-10T05:19:25.574 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-05-10T05:19:25.574 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-05-10T05:19:25.574 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-05-10T05:19:25.574 INFO:teuthology.orchestra.run.smithi157.stdout:Resetting modules: 2024-05-10T05:19:25.574 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools 2024-05-10T05:19:25.574 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:19:25.574 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-05-10T05:19:25.575 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-05-10T05:19:25.575 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:19:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-05-10T05:19:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-05-10T05:19:25.780 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-05-10T05:19:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-05-10T05:19:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:Resetting modules: 2024-05-10T05:19:25.780 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools 2024-05-10T05:19:25.780 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:19:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-05-10T05:19:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-05-10T05:19:25.781 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:19:26.156 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-10T05:19:26.187 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-05-10T05:19:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-05-10T05:19:26.187 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-05-10T05:19:26.187 INFO:teuthology.orchestra.run.smithi131.stdout:Resetting modules: 2024-05-10T05:19:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools 2024-05-10T05:19:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:19:26.188 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-05-10T05:19:26.188 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-05-10T05:19:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:19:38.640 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-10T05:19:39.287 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-05-10T05:19:39.404 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-05-10T05:19:39.883 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2024-05-10T05:19:41.178 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-05-10T05:19:41.562 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 kB/s | 4.5 kB 00:00 2024-05-10T05:19:41.746 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-05-10T05:19:41.755 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 kB/s | 4.1 kB 00:00 2024-05-10T05:19:42.091 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-05-10T05:19:42.231 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-05-10T05:19:44.133 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-05-10T05:19:44.599 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-10T05:19:44.772 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================================== 2024-05-10T05:19:44.772 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-05-10T05:19:44.772 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================================== 2024-05-10T05:19:44.772 INFO:teuthology.orchestra.run.smithi131.stdout:Installing group/module packages: 2024-05-10T05:19:44.772 INFO:teuthology.orchestra.run.smithi131.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-10T05:19:44.772 INFO:teuthology.orchestra.run.smithi131.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-10T05:19:44.772 INFO:teuthology.orchestra.run.smithi131.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-10T05:19:44.772 INFO:teuthology.orchestra.run.smithi131.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout:Installing module profiles: 2024-05-10T05:19:44.773 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools/common 2024-05-10T05:19:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:19:44.774 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling module streams: 2024-05-10T05:19:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: container-tools rhel8 2024-05-10T05:19:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:19:44.774 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-05-10T05:19:44.774 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================================== 2024-05-10T05:19:44.774 INFO:teuthology.orchestra.run.smithi131.stdout:Install 13 Packages 2024-05-10T05:19:44.774 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:19:44.775 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 26 M 2024-05-10T05:19:44.775 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 87 M 2024-05-10T05:19:44.775 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-05-10T05:19:44.783 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 kB/s | 4.1 kB 00:00 2024-05-10T05:19:44.943 INFO:teuthology.orchestra.run.smithi131.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 561 kB/s | 94 kB 00:00 2024-05-10T05:19:44.947 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-05-10T05:19:45.002 INFO:teuthology.orchestra.run.smithi131.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 110 kB/s | 25 kB 00:00 2024-05-10T05:19:45.044 INFO:teuthology.orchestra.run.smithi131.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 147 kB/s | 39 kB 00:00 2024-05-10T05:19:45.111 INFO:teuthology.orchestra.run.smithi131.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 245 kB/s | 41 kB 00:00 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================================== 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================================== 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout:Installing group/module packages: 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-10T05:19:45.115 INFO:teuthology.orchestra.run.smithi157.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout:Installing module profiles: 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools/common 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout:Enabling module streams: 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools rhel8 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-05-10T05:19:45.116 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================================== 2024-05-10T05:19:45.117 INFO:teuthology.orchestra.run.smithi157.stdout:Install 13 Packages 2024-05-10T05:19:45.117 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:19:45.117 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 26 M 2024-05-10T05:19:45.117 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 87 M 2024-05-10T05:19:45.117 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-05-10T05:19:45.194 INFO:teuthology.orchestra.run.smithi131.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 5.3 MB/s | 1.0 MB 00:00 2024-05-10T05:19:45.301 INFO:teuthology.orchestra.run.smithi157.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 136 kB/s | 25 kB 00:00 2024-05-10T05:19:45.311 INFO:teuthology.orchestra.run.smithi131.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 3.6 MB/s | 739 kB 00:00 2024-05-10T05:19:45.326 INFO:teuthology.orchestra.run.smithi157.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 189 kB/s | 39 kB 00:00 2024-05-10T05:19:45.345 INFO:teuthology.orchestra.run.smithi131.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.6 MB/s | 239 kB 00:00 2024-05-10T05:19:45.393 INFO:teuthology.orchestra.run.smithi157.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 341 kB/s | 94 kB 00:00 2024-05-10T05:19:45.418 INFO:teuthology.orchestra.run.smithi157.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 348 kB/s | 41 kB 00:00 2024-05-10T05:19:45.469 INFO:teuthology.orchestra.run.smithi157.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 7.1 MB/s | 1.0 MB 00:00 2024-05-10T05:19:45.537 INFO:teuthology.orchestra.run.smithi131.stdout:(8/13): buildah-1.31.5-1.module+el8.9.0+21697+6 18 MB/s | 8.8 MB 00:00 2024-05-10T05:19:45.562 INFO:teuthology.orchestra.run.smithi131.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 708 kB/s | 154 kB 00:00 2024-05-10T05:19:45.577 INFO:teuthology.orchestra.run.smithi157.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 4.6 MB/s | 739 kB 00:00 2024-05-10T05:19:45.628 INFO:teuthology.orchestra.run.smithi157.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.5 MB/s | 239 kB 00:00 2024-05-10T05:19:45.663 INFO:teuthology.orchestra.run.smithi131.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 11 MB/s | 3.7 MB 00:00 2024-05-10T05:19:45.745 INFO:teuthology.orchestra.run.smithi157.stdout:(8/13): buildah-1.31.5-1.module+el8.9.0+21697+6 25 MB/s | 8.8 MB 00:00 2024-05-10T05:19:45.761 INFO:teuthology.orchestra.run.smithi157.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.1 MB/s | 154 kB 00:00 2024-05-10T05:19:45.762 INFO:teuthology.orchestra.run.smithi131.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 13 MB/s | 2.5 MB 00:00 2024-05-10T05:19:45.788 INFO:teuthology.orchestra.run.smithi131.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 395 kB/s | 49 kB 00:00 2024-05-10T05:19:45.853 INFO:teuthology.orchestra.run.smithi157.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 13 MB/s | 3.7 MB 00:00 2024-05-10T05:19:45.963 INFO:teuthology.orchestra.run.smithi131.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 19 MB/s | 8.1 MB 00:00 2024-05-10T05:19:45.964 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-05-10T05:19:45.964 INFO:teuthology.orchestra.run.smithi131.stdout:Total 22 MB/s | 26 MB 00:01 2024-05-10T05:19:45.979 INFO:teuthology.orchestra.run.smithi157.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 12 MB/s | 2.5 MB 00:00 2024-05-10T05:19:46.004 INFO:teuthology.orchestra.run.smithi157.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 329 kB/s | 49 kB 00:00 2024-05-10T05:19:46.121 INFO:teuthology.orchestra.run.smithi157.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 22 MB/s | 8.1 MB 00:00 2024-05-10T05:19:46.121 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-05-10T05:19:46.121 INFO:teuthology.orchestra.run.smithi157.stdout:Total 25 MB/s | 26 MB 00:01 2024-05-10T05:19:46.401 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-05-10T05:19:46.422 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-05-10T05:19:46.422 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-05-10T05:19:46.440 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-05-10T05:19:46.465 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-05-10T05:19:46.466 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-05-10T05:19:46.651 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-05-10T05:19:46.660 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-05-10T05:19:46.873 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-05-10T05:19:46.883 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-05-10T05:19:47.406 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-05-10T05:19:47.583 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-05-10T05:19:47.713 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout:========================================================================================================== 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout:========================================================================================================== 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout:Installing group/module packages: 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-10T05:19:47.748 INFO:teuthology.orchestra.run.smithi142.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout:Installing module profiles: 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools/common 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling module streams: 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools rhel8 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:19:47.749 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-05-10T05:19:47.750 INFO:teuthology.orchestra.run.smithi142.stdout:========================================================================================================== 2024-05-10T05:19:47.750 INFO:teuthology.orchestra.run.smithi142.stdout:Install 13 Packages 2024-05-10T05:19:47.750 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:19:47.750 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 26 M 2024-05-10T05:19:47.750 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 87 M 2024-05-10T05:19:47.750 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-05-10T05:19:47.893 INFO:teuthology.orchestra.run.smithi142.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 658 kB/s | 94 kB 00:00 2024-05-10T05:19:47.895 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-10T05:19:47.927 INFO:teuthology.orchestra.run.smithi142.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 223 kB/s | 39 kB 00:00 2024-05-10T05:19:47.952 INFO:teuthology.orchestra.run.smithi142.stdout:(3/13): python3-pytoml-0.1.14-5.git7dea353.el8. 123 kB/s | 25 kB 00:00 2024-05-10T05:19:47.977 INFO:teuthology.orchestra.run.smithi142.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 488 kB/s | 41 kB 00:00 2024-05-10T05:19:48.066 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-10T05:19:48.078 INFO:teuthology.orchestra.run.smithi142.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.7 MB/s | 1.0 MB 00:00 2024-05-10T05:19:48.145 INFO:teuthology.orchestra.run.smithi142.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 4.3 MB/s | 739 kB 00:00 2024-05-10T05:19:48.252 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-10T05:19:48.257 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-10T05:19:48.295 INFO:teuthology.orchestra.run.smithi142.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 26 MB/s | 8.8 MB 00:00 2024-05-10T05:19:48.320 INFO:teuthology.orchestra.run.smithi142.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 988 kB/s | 239 kB 00:00 2024-05-10T05:19:48.395 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-10T05:19:48.437 INFO:teuthology.orchestra.run.smithi142.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 13 MB/s | 3.7 MB 00:00 2024-05-10T05:19:48.462 INFO:teuthology.orchestra.run.smithi142.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 921 kB/s | 154 kB 00:00 2024-05-10T05:19:48.501 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-10T05:19:48.557 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-10T05:19:48.563 INFO:teuthology.orchestra.run.smithi142.stdout:(11/13): udica-0.2.6-20.module+el8.9.0+21697+6a 494 kB/s | 49 kB 00:00 2024-05-10T05:19:48.630 INFO:teuthology.orchestra.run.smithi142.stdout:(12/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 13 MB/s | 2.5 MB 00:00 2024-05-10T05:19:48.672 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-10T05:19:48.779 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-10T05:19:48.792 INFO:teuthology.orchestra.run.smithi142.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 17 MB/s | 8.1 MB 00:00 2024-05-10T05:19:48.793 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-05-10T05:19:48.793 INFO:teuthology.orchestra.run.smithi142.stdout:Total 25 MB/s | 26 MB 00:01 2024-05-10T05:19:48.850 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-10T05:19:49.019 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-10T05:19:49.043 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-10T05:19:49.077 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-05-10T05:19:49.098 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-05-10T05:19:49.099 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-05-10T05:19:49.290 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-10T05:19:49.326 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-05-10T05:19:49.335 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-05-10T05:19:49.562 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-10T05:19:49.917 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T05:19:50.118 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-05-10T05:19:50.171 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T05:19:50.507 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T05:19:50.511 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-10T05:19:50.620 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-10T05:19:50.742 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T05:19:50.847 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-10T05:19:51.007 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-10T05:19:51.159 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-10T05:19:51.184 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-10T05:19:51.355 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-10T05:19:51.377 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-10T05:19:51.597 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-10T05:19:51.606 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-10T05:19:51.747 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T05:19:51.858 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-10T05:19:52.126 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-10T05:19:52.182 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T05:19:52.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-10T05:19:52.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-10T05:19:52.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-10T05:19:52.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-10T05:19:52.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-10T05:19:52.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-10T05:19:52.183 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-10T05:19:52.183 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T05:19:52.183 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T05:19:52.183 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-10T05:19:52.183 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-10T05:19:52.183 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-10T05:19:52.394 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-10T05:19:52.564 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T05:19:52.738 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T05:19:52.977 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T05:19:53.234 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-10T05:19:53.235 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-10T05:19:53.344 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-10T05:19:54.263 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-10T05:19:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-10T05:19:54.703 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T05:19:55.134 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T05:19:55.134 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-10T05:19:55.134 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-10T05:19:55.134 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-10T05:19:55.134 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-10T05:19:55.134 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-10T05:19:55.134 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-10T05:19:55.134 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-10T05:19:55.135 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T05:19:55.135 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T05:19:55.135 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-10T05:19:55.135 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-10T05:19:55.135 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-10T05:20:04.804 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-10T05:20:04.804 INFO:teuthology.orchestra.run.smithi157.stdout:Installed products updated. 2024-05-10T05:20:05.364 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:20:05.364 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-05-10T05:20:05.364 INFO:teuthology.orchestra.run.smithi157.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:05.364 INFO:teuthology.orchestra.run.smithi157.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:05.364 INFO:teuthology.orchestra.run.smithi157.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T05:20:05.364 INFO:teuthology.orchestra.run.smithi157.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-10T05:20:05.364 INFO:teuthology.orchestra.run.smithi157.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:05.364 INFO:teuthology.orchestra.run.smithi157.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:05.364 INFO:teuthology.orchestra.run.smithi157.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T05:20:05.364 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-10T05:20:05.365 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-10T05:20:05.365 INFO:teuthology.orchestra.run.smithi157.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:05.365 INFO:teuthology.orchestra.run.smithi157.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:05.365 INFO:teuthology.orchestra.run.smithi157.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T05:20:05.365 INFO:teuthology.orchestra.run.smithi157.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-10T05:20:05.365 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:20:05.365 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-05-10T05:20:05.538 DEBUG:teuthology.parallel:result is None 2024-05-10T05:20:06.903 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-10T05:20:06.903 INFO:teuthology.orchestra.run.smithi131.stdout:Installed products updated. 2024-05-10T05:20:07.549 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:20:07.550 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-05-10T05:20:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T05:20:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-10T05:20:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T05:20:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-10T05:20:07.550 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-10T05:20:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T05:20:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-10T05:20:07.551 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:20:07.551 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-10T05:20:07.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-10T05:20:07.660 INFO:teuthology.orchestra.run.smithi142.stdout:Installed products updated. 2024-05-10T05:20:07.808 DEBUG:teuthology.parallel:result is None 2024-05-10T05:20:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:20:08.225 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-05-10T05:20:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T05:20:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-10T05:20:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:08.226 INFO:teuthology.orchestra.run.smithi142.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:08.226 INFO:teuthology.orchestra.run.smithi142.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T05:20:08.226 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-10T05:20:08.226 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-10T05:20:08.226 INFO:teuthology.orchestra.run.smithi142.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:08.226 INFO:teuthology.orchestra.run.smithi142.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T05:20:08.226 INFO:teuthology.orchestra.run.smithi142.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T05:20:08.226 INFO:teuthology.orchestra.run.smithi142.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-10T05:20:08.226 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:20:08.226 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-05-10T05:20:08.397 DEBUG:teuthology.parallel:result is None 2024-05-10T05:20:08.397 INFO:teuthology.run_tasks:Running task install... 2024-05-10T05:20:08.407 DEBUG:teuthology.task.install:project ceph 2024-05-10T05:20:08.407 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'}} 2024-05-10T05:20:08.407 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'} 2024-05-10T05:20:08.407 INFO:teuthology.task.install:Using flavor: default 2024-05-10T05:20:08.413 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-10T05:20:08.413 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-10T05:20:08.414 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T05:20:08.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:20:08.417 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T05:20:08.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:20:08.418 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T05:20:08.418 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:20:08.563 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/ 2024-05-10T05:20:08.563 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T05:20:08.599 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/ 2024-05-10T05:20:08.599 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T05:20:08.639 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/ 2024-05-10T05:20:08.639 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T05:20:08.917 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T05:20:08.918 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:20:08.918 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T05:20:08.926 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T05:20:08.926 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:20:08.926 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T05:20:08.931 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T05:20:08.931 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:20:08.932 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T05:20:08.947 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T05:20:08.948 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T05:20:08.960 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T05:20:08.960 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T05:20:09.005 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T05:20:09.006 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T05:20:09.016 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T05:20:09.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T05:20:09.095 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T05:20:09.114 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T05:20:09.115 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T05:20:09.160 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-05-10T05:20:09.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-05-10T05:20:09.182 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-05-10T05:20:09.183 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-05-10T05:20:09.206 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T05:20:09.271 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-05-10T05:20:09.275 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-05-10T05:20:09.537 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-05-10T05:20:09.559 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2024-05-10T05:20:09.723 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-05-10T05:20:10.924 INFO:teuthology.orchestra.run.smithi157.stdout:61 files removed 2024-05-10T05:20:10.926 INFO:teuthology.orchestra.run.smithi142.stdout:61 files removed 2024-05-10T05:20:10.976 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T05:20:10.978 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T05:20:11.068 INFO:teuthology.orchestra.run.smithi131.stdout:61 files removed 2024-05-10T05:20:11.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T05:20:11.315 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2024-05-10T05:20:11.317 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-05-10T05:20:11.579 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2024-05-10T05:20:13.541 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 97 kB/s | 80 kB 00:00 2024-05-10T05:20:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 100 kB/s | 80 kB 00:00 2024-05-10T05:20:13.929 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 98 kB/s | 80 kB 00:00 2024-05-10T05:20:14.233 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-05-10T05:20:14.244 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-05-10T05:20:14.513 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 21 kB/s | 11 kB 00:00 2024-05-10T05:20:14.759 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 3.0 kB/s | 1.5 kB 00:00 2024-05-10T05:20:14.795 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 2.8 kB/s | 1.5 kB 00:00 2024-05-10T05:20:14.926 INFO:teuthology.orchestra.run.smithi157.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-05-10T05:20:14.954 INFO:teuthology.orchestra.run.smithi142.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-05-10T05:20:15.047 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 2.9 kB/s | 1.5 kB 00:00 2024-05-10T05:20:15.190 INFO:teuthology.orchestra.run.smithi131.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2024-05-10T05:20:16.063 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 13 MB/s | 14 MB 00:01 2024-05-10T05:20:16.387 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 14 MB 00:01 2024-05-10T05:20:16.766 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 14 MB 00:01 2024-05-10T05:20:19.405 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 339 kB/s | 24 kB 00:00 2024-05-10T05:20:19.727 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 392 kB/s | 24 kB 00:00 2024-05-10T05:20:20.757 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 MB/s | 61 MB 00:01 2024-05-10T05:20:21.113 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 6.7 MB/s | 8.8 MB 00:01 2024-05-10T05:20:21.133 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 48 kB/s | 24 kB 00:00 2024-05-10T05:20:22.135 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 9.6 MB/s | 8.8 MB 00:00 2024-05-10T05:20:24.064 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 67 MB 00:01 2024-05-10T05:20:25.503 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 MB/s | 67 MB 00:01 2024-05-10T05:20:31.800 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 11 MB/s | 8.8 MB 00:00 2024-05-10T05:20:34.908 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 67 MB 00:01 2024-05-10T05:20:35.056 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 61 MB 00:01 2024-05-10T05:20:37.321 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 MB/s | 61 MB 00:01 2024-05-10T05:20:50.363 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T05:20:50.364 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T05:20:50.383 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-10T05:20:50.384 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-10T05:20:50.386 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-10T05:20:50.387 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-10T05:20:50.390 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-10T05:20:50.394 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-10T05:20:50.395 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-10T05:20:50.589 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T05:20:50.590 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T05:20:50.608 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-10T05:20:50.610 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-10T05:20:50.611 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-10T05:20:50.613 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-10T05:20:50.615 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-10T05:20:50.619 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-10T05:20:50.621 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-10T05:20:50.621 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-05-10T05:20:50.635 INFO:teuthology.orchestra.run.smithi142.stdout:=================================================================================================================================== 2024-05-10T05:20:50.636 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-05-10T05:20:50.636 INFO:teuthology.orchestra.run.smithi142.stdout:=================================================================================================================================== 2024-05-10T05:20:50.636 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-05-10T05:20:50.636 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-10T05:20:50.636 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-10T05:20:50.636 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el8 ceph 6.4 k 2024-05-10T05:20:50.636 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.7 M 2024-05-10T05:20:50.636 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 829 k 2024-05-10T05:20:50.636 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el8 ceph 157 k 2024-05-10T05:20:50.636 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el8 ceph 1.5 M 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 138 k 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 3.5 M 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 7.4 M 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 49 k 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 14 M 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el8 ceph 49 M 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 263 k 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 173 k 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 246 k 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 18 k 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-10T05:20:50.637 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-10T05:20:50.638 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-10T05:20:50.638 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 31 k 2024-05-10T05:20:50.638 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 756 k 2024-05-10T05:20:50.638 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T05:20:50.638 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-10T05:20:50.638 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-10T05:20:50.638 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-10T05:20:50.638 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-10T05:20:50.638 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-10T05:20:50.638 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el8 ceph 197 k 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el8 ceph 371 k 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 364 k 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 107 k 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 91 k 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.3 M 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 179 k 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-10T05:20:50.639 INFO:teuthology.orchestra.run.smithi142.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-10T05:20:50.640 INFO:teuthology.orchestra.run.smithi142.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 23 M 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 24 k 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el8 ceph 2.2 M 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 246 k 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.1 M 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el8 ceph 19 M 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 16 k 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el8 ceph 24 k 2024-05-10T05:20:50.641 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-10T05:20:50.642 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 461 k 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 7.2 M 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-10T05:20:50.643 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el8 ceph 45 k 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 129 k 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-10T05:20:50.644 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-10T05:20:50.645 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-10T05:20:50.646 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-10T05:20:50.647 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-10T05:20:50.647 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-10T05:20:50.647 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-10T05:20:50.647 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-10T05:20:50.647 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T05:20:50.647 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-10T05:20:50.647 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-10T05:20:50.647 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-10T05:20:50.647 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-10T05:20:50.647 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-10T05:20:50.648 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling module streams: 2024-05-10T05:20:50.649 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql 10 2024-05-10T05:20:50.649 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:20:50.649 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-05-10T05:20:50.649 INFO:teuthology.orchestra.run.smithi142.stdout:=================================================================================================================================== 2024-05-10T05:20:50.649 INFO:teuthology.orchestra.run.smithi142.stdout:Install 130 Packages 2024-05-10T05:20:50.649 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 16 Packages 2024-05-10T05:20:50.649 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:20:50.650 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 221 M 2024-05-10T05:20:50.650 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-05-10T05:20:50.840 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-05-10T05:20:50.854 INFO:teuthology.orchestra.run.smithi157.stdout:=================================================================================================================================== 2024-05-10T05:20:50.854 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-05-10T05:20:50.854 INFO:teuthology.orchestra.run.smithi157.stdout:=================================================================================================================================== 2024-05-10T05:20:50.854 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-05-10T05:20:50.854 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el8 ceph 6.4 k 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.7 M 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 829 k 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el8 ceph 157 k 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el8 ceph 1.5 M 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 138 k 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 3.5 M 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 7.4 M 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 49 k 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 14 M 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el8 ceph 49 M 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 263 k 2024-05-10T05:20:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 173 k 2024-05-10T05:20:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 246 k 2024-05-10T05:20:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 18 k 2024-05-10T05:20:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-10T05:20:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-10T05:20:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-10T05:20:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 31 k 2024-05-10T05:20:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 756 k 2024-05-10T05:20:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T05:20:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el8 ceph 197 k 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el8 ceph 371 k 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 364 k 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 107 k 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 91 k 2024-05-10T05:20:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.3 M 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 179 k 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T05:20:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 23 M 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 24 k 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el8 ceph 2.2 M 2024-05-10T05:20:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 246 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.1 M 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el8 ceph 19 M 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 16 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el8 ceph 24 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-10T05:20:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 461 k 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 7.2 M 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-10T05:20:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-10T05:20:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-10T05:20:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-10T05:20:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-10T05:20:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-10T05:20:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-10T05:20:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-10T05:20:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-10T05:20:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-10T05:20:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-10T05:20:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el8 ceph 45 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 129 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-10T05:20:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-10T05:20:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-10T05:20:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T05:20:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-10T05:20:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-10T05:20:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-10T05:20:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T05:20:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-10T05:20:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-10T05:20:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T05:20:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-10T05:20:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-10T05:20:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-10T05:20:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-10T05:20:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-10T05:20:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-10T05:20:50.866 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-10T05:20:50.866 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-10T05:20:50.866 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-10T05:20:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-10T05:20:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-10T05:20:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-10T05:20:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T05:20:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-10T05:20:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-10T05:20:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-10T05:20:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-10T05:20:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-10T05:20:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-10T05:20:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-10T05:20:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-10T05:20:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-10T05:20:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-10T05:20:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-10T05:20:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-10T05:20:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-10T05:20:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-10T05:20:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-10T05:20:50.870 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-10T05:20:50.870 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-10T05:20:50.870 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-10T05:20:50.870 INFO:teuthology.orchestra.run.smithi157.stdout:Enabling module streams: 2024-05-10T05:20:50.870 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql 10 2024-05-10T05:20:50.871 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:20:50.871 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-05-10T05:20:50.871 INFO:teuthology.orchestra.run.smithi157.stdout:=================================================================================================================================== 2024-05-10T05:20:50.871 INFO:teuthology.orchestra.run.smithi157.stdout:Install 130 Packages 2024-05-10T05:20:50.871 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 16 Packages 2024-05-10T05:20:50.871 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:20:50.872 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 221 M 2024-05-10T05:20:50.872 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-05-10T05:20:50.967 INFO:teuthology.orchestra.run.smithi142.stdout:(1/146): ceph-18.2.2-1330.g50174015.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2024-05-10T05:20:51.159 INFO:teuthology.orchestra.run.smithi142.stdout:(2/146): ceph-fuse-18.2.2-1330.g50174015.el8.x8 4.2 MB/s | 829 kB 00:00 2024-05-10T05:20:51.226 INFO:teuthology.orchestra.run.smithi142.stdout:(3/146): ceph-immutable-object-cache-18.2.2-133 2.3 MB/s | 157 kB 00:00 2024-05-10T05:20:51.526 INFO:teuthology.orchestra.run.smithi142.stdout:(4/146): ceph-mds-18.2.2-1330.g50174015.el8.x86 7.2 MB/s | 2.2 MB 00:00 2024-05-10T05:20:51.776 INFO:teuthology.orchestra.run.smithi142.stdout:(5/146): ceph-mgr-18.2.2-1330.g50174015.el8.x86 6.0 MB/s | 1.5 MB 00:00 2024-05-10T05:20:51.951 INFO:teuthology.orchestra.run.smithi142.stdout:(6/146): ceph-base-18.2.2-1330.g50174015.el8.x8 4.5 MB/s | 5.7 MB 00:01 2024-05-10T05:20:52.435 INFO:teuthology.orchestra.run.smithi142.stdout:(7/146): ceph-mon-18.2.2-1330.g50174015.el8.x86 7.8 MB/s | 5.1 MB 00:00 2024-05-10T05:20:52.672 INFO:teuthology.orchestra.run.smithi157.stdout:(1/146): ceph-18.2.2-1330.g50174015.el8.x86_64. 30 kB/s | 6.4 kB 00:00 2024-05-10T05:20:52.752 INFO:teuthology.orchestra.run.smithi142.stdout:(8/146): ceph-common-18.2.2-1330.g50174015.el8. 11 MB/s | 23 MB 00:02 2024-05-10T05:20:52.810 INFO:teuthology.orchestra.run.smithi142.stdout:(9/146): ceph-selinux-18.2.2-1330.g50174015.el8 420 kB/s | 24 kB 00:00 2024-05-10T05:20:53.372 INFO:teuthology.orchestra.run.smithi157.stdout:(2/146): ceph-fuse-18.2.2-1330.g50174015.el8.x8 1.2 MB/s | 829 kB 00:00 2024-05-10T05:20:53.568 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T05:20:53.569 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T05:20:53.572 INFO:teuthology.orchestra.run.smithi157.stdout:(3/146): ceph-immutable-object-cache-18.2.2-133 786 kB/s | 157 kB 00:00 2024-05-10T05:20:53.588 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-10T05:20:53.590 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-10T05:20:53.591 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-10T05:20:53.593 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-10T05:20:53.596 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-10T05:20:53.600 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-10T05:20:53.602 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-10T05:20:53.826 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-10T05:20:53.843 INFO:teuthology.orchestra.run.smithi131.stdout:=================================================================================================================================== 2024-05-10T05:20:53.843 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-05-10T05:20:53.843 INFO:teuthology.orchestra.run.smithi131.stdout:=================================================================================================================================== 2024-05-10T05:20:53.843 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-05-10T05:20:53.843 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-10T05:20:53.843 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el8 ceph 6.4 k 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.7 M 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 829 k 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el8 ceph 157 k 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el8 ceph 1.5 M 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 138 k 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 3.5 M 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 7.4 M 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 49 k 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 14 M 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el8 ceph 49 M 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 263 k 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 173 k 2024-05-10T05:20:53.844 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 246 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 18 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 31 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 756 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-10T05:20:53.845 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el8 ceph 197 k 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el8 ceph 371 k 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 364 k 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 107 k 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 91 k 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.3 M 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 179 k 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-10T05:20:53.846 INFO:teuthology.orchestra.run.smithi131.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-10T05:20:53.847 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 23 M 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 24 k 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el8 ceph 2.2 M 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 246 k 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.1 M 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el8 ceph 19 M 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 16 k 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el8 ceph 24 k 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-10T05:20:53.848 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 461 k 2024-05-10T05:20:53.849 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 7.2 M 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-10T05:20:53.850 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el8 ceph 45 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 129 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-10T05:20:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-10T05:20:53.852 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T05:20:53.853 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-10T05:20:53.854 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling module streams: 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql 10 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout:=================================================================================================================================== 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout:Install 130 Packages 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 16 Packages 2024-05-10T05:20:53.855 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:20:53.856 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 221 M 2024-05-10T05:20:53.856 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-05-10T05:20:54.312 INFO:teuthology.orchestra.run.smithi131.stdout:(1/146): ceph-18.2.2-1330.g50174015.el8.x86_64. 15 kB/s | 6.4 kB 00:00 2024-05-10T05:20:54.527 INFO:teuthology.orchestra.run.smithi142.stdout:(10/146): ceph-radosgw-18.2.2-1330.g50174015.el 6.9 MB/s | 14 MB 00:02 2024-05-10T05:20:54.611 INFO:teuthology.orchestra.run.smithi142.stdout:(11/146): cephfs-mirror-18.2.2-1330.g50174015.e 2.9 MB/s | 246 kB 00:00 2024-05-10T05:20:54.689 INFO:teuthology.orchestra.run.smithi157.stdout:(4/146): ceph-mds-18.2.2-1330.g50174015.el8.x86 1.9 MB/s | 2.2 MB 00:01 2024-05-10T05:20:54.722 INFO:teuthology.orchestra.run.smithi142.stdout:(12/146): libcephfs-devel-18.2.2-1330.g50174015 278 kB/s | 31 kB 00:00 2024-05-10T05:20:54.969 INFO:teuthology.orchestra.run.smithi142.stdout:(13/146): libcephfs2-18.2.2-1330.g50174015.el8. 3.0 MB/s | 756 kB 00:00 2024-05-10T05:20:55.045 INFO:teuthology.orchestra.run.smithi131.stdout:(2/146): ceph-fuse-18.2.2-1330.g50174015.el8.x8 1.1 MB/s | 829 kB 00:00 2024-05-10T05:20:55.056 INFO:teuthology.orchestra.run.smithi157.stdout:(5/146): ceph-base-18.2.2-1330.g50174015.el8.x8 2.2 MB/s | 5.7 MB 00:02 2024-05-10T05:20:55.094 INFO:teuthology.orchestra.run.smithi142.stdout:(14/146): libcephsqlite-18.2.2-1330.g50174015.e 1.1 MB/s | 139 kB 00:00 2024-05-10T05:20:55.189 INFO:teuthology.orchestra.run.smithi157.stdout:(6/146): ceph-mgr-18.2.2-1330.g50174015.el8.x86 3.0 MB/s | 1.5 MB 00:00 2024-05-10T05:20:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:(3/146): ceph-immutable-object-cache-18.2.2-133 898 kB/s | 157 kB 00:00 2024-05-10T05:20:55.236 INFO:teuthology.orchestra.run.smithi142.stdout:(15/146): librados-devel-18.2.2-1330.g50174015. 980 kB/s | 139 kB 00:00 2024-05-10T05:20:55.545 INFO:teuthology.orchestra.run.smithi142.stdout:(16/146): libradosstriper1-18.2.2-1330.g5017401 1.5 MB/s | 461 kB 00:00 2024-05-10T05:20:56.348 INFO:teuthology.orchestra.run.smithi157.stdout:(7/146): ceph-common-18.2.2-1330.g50174015.el8. 5.9 MB/s | 23 MB 00:03 2024-05-10T05:20:56.646 INFO:teuthology.orchestra.run.smithi131.stdout:(4/146): ceph-mds-18.2.2-1330.g50174015.el8.x86 1.5 MB/s | 2.2 MB 00:01 2024-05-10T05:20:56.970 INFO:teuthology.orchestra.run.smithi142.stdout:(17/146): ceph-osd-18.2.2-1330.g50174015.el8.x8 3.7 MB/s | 19 MB 00:05 2024-05-10T05:20:57.112 INFO:teuthology.orchestra.run.smithi142.stdout:(18/146): python3-ceph-argparse-18.2.2-1330.g50 315 kB/s | 45 kB 00:00 2024-05-10T05:20:57.296 INFO:teuthology.orchestra.run.smithi142.stdout:(19/146): python3-ceph-common-18.2.2-1330.g5017 704 kB/s | 129 kB 00:00 2024-05-10T05:20:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:(5/146): ceph-mgr-18.2.2-1330.g50174015.el8.x86 1.9 MB/s | 1.5 MB 00:00 2024-05-10T05:20:57.496 INFO:teuthology.orchestra.run.smithi142.stdout:(20/146): python3-cephfs-18.2.2-1330.g50174015. 986 kB/s | 197 kB 00:00 2024-05-10T05:20:57.805 INFO:teuthology.orchestra.run.smithi142.stdout:(21/146): python3-rados-18.2.2-1330.g50174015.e 1.2 MB/s | 371 kB 00:00 2024-05-10T05:20:57.931 INFO:teuthology.orchestra.run.smithi157.stdout:(8/146): ceph-mon-18.2.2-1330.g50174015.el8.x86 1.8 MB/s | 5.1 MB 00:02 2024-05-10T05:20:58.023 INFO:teuthology.orchestra.run.smithi157.stdout:(9/146): ceph-selinux-18.2.2-1330.g50174015.el8 265 kB/s | 24 kB 00:00 2024-05-10T05:20:58.063 INFO:teuthology.orchestra.run.smithi142.stdout:(22/146): python3-rbd-18.2.2-1330.g50174015.el8 1.4 MB/s | 364 kB 00:00 2024-05-10T05:20:58.163 INFO:teuthology.orchestra.run.smithi142.stdout:(23/146): python3-rgw-18.2.2-1330.g50174015.el8 1.0 MB/s | 107 kB 00:00 2024-05-10T05:20:58.247 INFO:teuthology.orchestra.run.smithi142.stdout:(24/146): rbd-fuse-18.2.2-1330.g50174015.el8.x8 1.1 MB/s | 91 kB 00:00 2024-05-10T05:20:58.265 INFO:teuthology.orchestra.run.smithi157.stdout:(10/146): ceph-radosgw-18.2.2-1330.g50174015.el 7.5 MB/s | 14 MB 00:01 2024-05-10T05:20:58.458 INFO:teuthology.orchestra.run.smithi157.stdout:(11/146): ceph-osd-18.2.2-1330.g50174015.el8.x8 5.7 MB/s | 19 MB 00:03 2024-05-10T05:20:58.483 INFO:teuthology.orchestra.run.smithi157.stdout:(12/146): cephfs-mirror-18.2.2-1330.g50174015.e 1.1 MB/s | 246 kB 00:00 2024-05-10T05:20:58.533 INFO:teuthology.orchestra.run.smithi157.stdout:(13/146): libcephfs-devel-18.2.2-1330.g50174015 413 kB/s | 31 kB 00:00 2024-05-10T05:20:58.633 INFO:teuthology.orchestra.run.smithi157.stdout:(14/146): libcephsqlite-18.2.2-1330.g50174015.e 1.4 MB/s | 139 kB 00:00 2024-05-10T05:20:58.658 INFO:teuthology.orchestra.run.smithi157.stdout:(15/146): libcephfs2-18.2.2-1330.g50174015.el8. 4.2 MB/s | 756 kB 00:00 2024-05-10T05:20:58.736 INFO:teuthology.orchestra.run.smithi157.stdout:(16/146): librados-devel-18.2.2-1330.g50174015. 1.3 MB/s | 139 kB 00:00 2024-05-10T05:20:58.767 INFO:teuthology.orchestra.run.smithi157.stdout:(17/146): libradosstriper1-18.2.2-1330.g5017401 4.1 MB/s | 461 kB 00:00 2024-05-10T05:20:58.876 INFO:teuthology.orchestra.run.smithi157.stdout:(18/146): python3-ceph-argparse-18.2.2-1330.g50 412 kB/s | 45 kB 00:00 2024-05-10T05:20:58.976 INFO:teuthology.orchestra.run.smithi157.stdout:(19/146): python3-ceph-common-18.2.2-1330.g5017 1.3 MB/s | 129 kB 00:00 2024-05-10T05:20:59.085 INFO:teuthology.orchestra.run.smithi157.stdout:(20/146): python3-cephfs-18.2.2-1330.g50174015. 1.8 MB/s | 197 kB 00:00 2024-05-10T05:20:59.218 INFO:teuthology.orchestra.run.smithi157.stdout:(21/146): python3-rados-18.2.2-1330.g50174015.e 2.7 MB/s | 371 kB 00:00 2024-05-10T05:20:59.335 INFO:teuthology.orchestra.run.smithi157.stdout:(22/146): python3-rbd-18.2.2-1330.g50174015.el8 3.1 MB/s | 364 kB 00:00 2024-05-10T05:20:59.411 INFO:teuthology.orchestra.run.smithi157.stdout:(23/146): python3-rgw-18.2.2-1330.g50174015.el8 1.4 MB/s | 107 kB 00:00 2024-05-10T05:20:59.478 INFO:teuthology.orchestra.run.smithi157.stdout:(24/146): rbd-fuse-18.2.2-1330.g50174015.el8.x8 1.3 MB/s | 91 kB 00:00 2024-05-10T05:20:59.573 INFO:teuthology.orchestra.run.smithi131.stdout:(6/146): ceph-mon-18.2.2-1330.g50174015.el8.x86 2.4 MB/s | 5.1 MB 00:02 2024-05-10T05:20:59.686 INFO:teuthology.orchestra.run.smithi157.stdout:(25/146): librgw2-18.2.2-1330.g50174015.el8.x86 7.6 MB/s | 7.2 MB 00:00 2024-05-10T05:20:59.747 INFO:teuthology.orchestra.run.smithi142.stdout:(25/146): ceph-test-18.2.2-1330.g50174015.el8.x 7.1 MB/s | 49 MB 00:06 2024-05-10T05:20:59.761 INFO:teuthology.orchestra.run.smithi157.stdout:(26/146): rbd-nbd-18.2.2-1330.g50174015.el8.x86 2.3 MB/s | 179 kB 00:00 2024-05-10T05:20:59.820 INFO:teuthology.orchestra.run.smithi157.stdout:(27/146): rbd-mirror-18.2.2-1330.g50174015.el8. 9.7 MB/s | 3.3 MB 00:00 2024-05-10T05:20:59.839 INFO:teuthology.orchestra.run.smithi142.stdout:(26/146): librgw2-18.2.2-1330.g50174015.el8.x86 1.7 MB/s | 7.2 MB 00:04 2024-05-10T05:20:59.845 INFO:teuthology.orchestra.run.smithi157.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1330.g 291 kB/s | 24 kB 00:00 2024-05-10T05:20:59.864 INFO:teuthology.orchestra.run.smithi142.stdout:(27/146): rbd-nbd-18.2.2-1330.g50174015.el8.x86 1.5 MB/s | 179 kB 00:00 2024-05-10T05:20:59.890 INFO:teuthology.orchestra.run.smithi142.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1330.g 487 kB/s | 24 kB 00:00 2024-05-10T05:20:59.912 INFO:teuthology.orchestra.run.smithi157.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1330.g5017401 1.5 MB/s | 138 kB 00:00 2024-05-10T05:20:59.923 INFO:teuthology.orchestra.run.smithi142.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1330.g5017401 2.3 MB/s | 138 kB 00:00 2024-05-10T05:21:00.090 INFO:teuthology.orchestra.run.smithi142.stdout:(30/146): rbd-mirror-18.2.2-1330.g50174015.el8. 1.8 MB/s | 3.3 MB 00:01 2024-05-10T05:21:00.307 INFO:teuthology.orchestra.run.smithi142.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1330.g50 1.1 MB/s | 246 kB 00:00 2024-05-10T05:21:00.449 INFO:teuthology.orchestra.run.smithi142.stdout:(32/146): ceph-mgr-rook-18.2.2-1330.g50174015.e 347 kB/s | 49 kB 00:00 2024-05-10T05:21:00.541 INFO:teuthology.orchestra.run.smithi142.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1330.g5 178 kB/s | 16 kB 00:00 2024-05-10T05:21:00.621 INFO:teuthology.orchestra.run.smithi157.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1330.g50174 4.6 MB/s | 3.5 MB 00:00 2024-05-10T05:21:00.691 INFO:teuthology.orchestra.run.smithi142.stdout:(34/146): ceph-volume-18.2.2-1330.g50174015.el8 1.7 MB/s | 263 kB 00:00 2024-05-10T05:21:00.729 INFO:teuthology.orchestra.run.smithi157.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1330.g50 2.2 MB/s | 246 kB 00:00 2024-05-10T05:21:00.821 INFO:teuthology.orchestra.run.smithi157.stdout:(32/146): ceph-mgr-rook-18.2.2-1330.g50174015.e 537 kB/s | 49 kB 00:00 2024-05-10T05:21:00.850 INFO:teuthology.orchestra.run.smithi142.stdout:(35/146): cephadm-18.2.2-1330.g50174015.el8.noa 1.1 MB/s | 173 kB 00:00 2024-05-10T05:21:00.896 INFO:teuthology.orchestra.run.smithi157.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1330.g5 218 kB/s | 16 kB 00:00 2024-05-10T05:21:00.934 INFO:teuthology.orchestra.run.smithi142.stdout:(36/146): ceph-mgr-diskprediction-local-18.2.2- 7.3 MB/s | 7.4 MB 00:01 2024-05-10T05:21:00.950 INFO:teuthology.orchestra.run.smithi142.stdout:(37/146): cephfs-top-18.2.2-1330.g50174015.el8. 181 kB/s | 18 kB 00:00 2024-05-10T05:21:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:(34/146): ceph-mgr-diskprediction-local-18.2.2- 6.5 MB/s | 7.4 MB 00:01 2024-05-10T05:21:01.081 INFO:teuthology.orchestra.run.smithi157.stdout:(35/146): ceph-volume-18.2.2-1330.g50174015.el8 1.4 MB/s | 263 kB 00:00 2024-05-10T05:21:01.106 INFO:teuthology.orchestra.run.smithi157.stdout:(36/146): cephadm-18.2.2-1330.g50174015.el8.noa 2.9 MB/s | 173 kB 00:00 2024-05-10T05:21:01.126 INFO:teuthology.orchestra.run.smithi142.stdout:(38/146): ceph-mgr-dashboard-18.2.2-1330.g50174 2.9 MB/s | 3.5 MB 00:01 2024-05-10T05:21:01.131 INFO:teuthology.orchestra.run.smithi157.stdout:(37/146): cephfs-top-18.2.2-1330.g50174015.el8. 361 kB/s | 18 kB 00:00 2024-05-10T05:21:01.201 INFO:teuthology.orchestra.run.smithi142.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 235 kB/s | 59 kB 00:00 2024-05-10T05:21:01.335 INFO:teuthology.orchestra.run.smithi142.stdout:(40/146): gperftools-libs-2.7-9.el8.x86_64.rpm 764 kB/s | 306 kB 00:00 2024-05-10T05:21:01.393 INFO:teuthology.orchestra.run.smithi142.stdout:(41/146): libunwind-1.3.1-3.el8.x86_64.rpm 281 kB/s | 75 kB 00:00 2024-05-10T05:21:01.419 INFO:teuthology.orchestra.run.smithi142.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 523 kB/s | 44 kB 00:00 2024-05-10T05:21:01.448 INFO:teuthology.orchestra.run.smithi157.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 186 kB/s | 59 kB 00:00 2024-05-10T05:21:01.582 INFO:teuthology.orchestra.run.smithi157.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 644 kB/s | 306 kB 00:00 2024-05-10T05:21:01.624 INFO:teuthology.orchestra.run.smithi157.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 427 kB/s | 75 kB 00:00 2024-05-10T05:21:01.671 INFO:teuthology.orchestra.run.smithi142.stdout:(43/146): python3-asyncssh-2.7.0-2.el8.noarch.r 1.0 MB/s | 476 kB 00:00 2024-05-10T05:21:01.716 INFO:teuthology.orchestra.run.smithi157.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 476 kB/s | 44 kB 00:00 2024-05-10T05:21:01.766 INFO:teuthology.orchestra.run.smithi157.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.5 MB/s | 476 kB 00:00 2024-05-10T05:21:01.806 INFO:teuthology.orchestra.run.smithi131.stdout:(7/146): ceph-base-18.2.2-1330.g50174015.el8.x8 733 kB/s | 5.7 MB 00:07 2024-05-10T05:21:01.866 INFO:teuthology.orchestra.run.smithi157.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2024-05-10T05:21:01.880 INFO:teuthology.orchestra.run.smithi142.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 71 kB/s | 33 kB 00:00 2024-05-10T05:21:01.891 INFO:teuthology.orchestra.run.smithi157.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2024-05-10T05:21:01.911 INFO:teuthology.orchestra.run.smithi142.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 68 kB/s | 16 kB 00:00 2024-05-10T05:21:01.936 INFO:teuthology.orchestra.run.smithi142.stdout:(46/146): python3-beautifulsoup4-4.6.3-2.el8.1. 341 kB/s | 185 kB 00:00 2024-05-10T05:21:01.986 INFO:teuthology.orchestra.run.smithi142.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2024-05-10T05:21:01.992 INFO:teuthology.orchestra.run.smithi157.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2024-05-10T05:21:02.017 INFO:teuthology.orchestra.run.smithi157.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2024-05-10T05:21:02.020 INFO:teuthology.orchestra.run.smithi142.stdout:(48/146): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2024-05-10T05:21:02.046 INFO:teuthology.orchestra.run.smithi142.stdout:(49/146): python3-google-auth-1.1.1-10.el8.noar 746 kB/s | 81 kB 00:00 2024-05-10T05:21:02.079 INFO:teuthology.orchestra.run.smithi142.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 120 kB/s | 11 kB 00:00 2024-05-10T05:21:02.105 INFO:teuthology.orchestra.run.smithi142.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2024-05-10T05:21:02.126 INFO:teuthology.orchestra.run.smithi157.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2024-05-10T05:21:02.142 INFO:teuthology.orchestra.run.smithi157.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 648 kB/s | 81 kB 00:00 2024-05-10T05:21:02.171 INFO:teuthology.orchestra.run.smithi142.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 530 kB/s | 49 kB 00:00 2024-05-10T05:21:02.226 INFO:teuthology.orchestra.run.smithi157.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-05-10T05:21:02.226 INFO:teuthology.orchestra.run.smithi142.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 484 kB/s | 59 kB 00:00 2024-05-10T05:21:02.251 INFO:teuthology.orchestra.run.smithi157.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2024-05-10T05:21:02.272 INFO:teuthology.orchestra.run.smithi142.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 601 kB/s | 60 kB 00:00 2024-05-10T05:21:02.331 INFO:teuthology.orchestra.run.smithi142.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.7 MB/s | 283 kB 00:00 2024-05-10T05:21:02.360 INFO:teuthology.orchestra.run.smithi157.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2024-05-10T05:21:02.364 INFO:teuthology.orchestra.run.smithi142.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2024-05-10T05:21:02.406 INFO:teuthology.orchestra.run.smithi142.stdout:(57/146): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2024-05-10T05:21:02.423 INFO:teuthology.orchestra.run.smithi142.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 364 kB/s | 34 kB 00:00 2024-05-10T05:21:02.460 INFO:teuthology.orchestra.run.smithi157.stdout:(52/146): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2024-05-10T05:21:02.465 INFO:teuthology.orchestra.run.smithi142.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2024-05-10T05:21:02.485 INFO:teuthology.orchestra.run.smithi157.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 469 kB/s | 59 kB 00:00 2024-05-10T05:21:02.490 INFO:teuthology.orchestra.run.smithi142.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 764 kB/s | 64 kB 00:00 2024-05-10T05:21:02.515 INFO:teuthology.orchestra.run.smithi142.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 203 kB/s | 19 kB 00:00 2024-05-10T05:21:02.561 INFO:teuthology.orchestra.run.smithi157.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 602 kB/s | 60 kB 00:00 2024-05-10T05:21:02.582 INFO:teuthology.orchestra.run.smithi131.stdout:(8/146): ceph-common-18.2.2-1330.g50174015.el8. 2.7 MB/s | 23 MB 00:08 2024-05-10T05:21:02.582 INFO:teuthology.orchestra.run.smithi142.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2024-05-10T05:21:02.608 INFO:teuthology.orchestra.run.smithi142.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2024-05-10T05:21:02.611 INFO:teuthology.orchestra.run.smithi157.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2024-05-10T05:21:02.633 INFO:teuthology.orchestra.run.smithi142.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2024-05-10T05:21:02.640 INFO:teuthology.orchestra.run.smithi131.stdout:(9/146): ceph-selinux-18.2.2-1330.g50174015.el8 419 kB/s | 24 kB 00:00 2024-05-10T05:21:02.675 INFO:teuthology.orchestra.run.smithi142.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2024-05-10T05:21:02.700 INFO:teuthology.orchestra.run.smithi142.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 919 kB/s | 61 kB 00:00 2024-05-10T05:21:02.725 INFO:teuthology.orchestra.run.smithi142.stdout:(67/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2024-05-10T05:21:02.751 INFO:teuthology.orchestra.run.smithi142.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2024-05-10T05:21:02.776 INFO:teuthology.orchestra.run.smithi142.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2024-05-10T05:21:02.793 INFO:teuthology.orchestra.run.smithi142.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-05-10T05:21:02.935 INFO:teuthology.orchestra.run.smithi142.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 8.3 MB/s | 1.7 MB 00:00 2024-05-10T05:21:02.944 INFO:teuthology.orchestra.run.smithi157.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 41 kB/s | 16 kB 00:00 2024-05-10T05:21:02.960 INFO:teuthology.orchestra.run.smithi142.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 109 kB/s | 20 kB 00:00 2024-05-10T05:21:02.978 INFO:teuthology.orchestra.run.smithi157.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 92 kB/s | 34 kB 00:00 2024-05-10T05:21:02.994 INFO:teuthology.orchestra.run.smithi142.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 226 kB/s | 44 kB 00:00 2024-05-10T05:21:03.062 INFO:teuthology.orchestra.run.smithi157.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2024-05-10T05:21:03.077 INFO:teuthology.orchestra.run.smithi142.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 775 kB/s | 110 kB 00:00 2024-05-10T05:21:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 592 kB/s | 64 kB 00:00 2024-05-10T05:21:03.103 INFO:teuthology.orchestra.run.smithi142.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 194 kB/s | 27 kB 00:00 2024-05-10T05:21:03.128 INFO:teuthology.orchestra.run.smithi142.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 945 kB/s | 126 kB 00:00 2024-05-10T05:21:03.162 INFO:teuthology.orchestra.run.smithi157.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2024-05-10T05:21:03.178 INFO:teuthology.orchestra.run.smithi142.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 4.5 MB/s | 457 kB 00:00 2024-05-10T05:21:03.188 INFO:teuthology.orchestra.run.smithi157.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2024-05-10T05:21:03.262 INFO:teuthology.orchestra.run.smithi142.stdout:(78/146): bison-3.0.4-10.el8.x86_64.rpm 4.2 MB/s | 688 kB 00:00 2024-05-10T05:21:03.263 INFO:teuthology.orchestra.run.smithi157.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 292 kB/s | 29 kB 00:00 2024-05-10T05:21:03.288 INFO:teuthology.orchestra.run.smithi157.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2024-05-10T05:21:03.295 INFO:teuthology.orchestra.run.smithi142.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 1.9 MB/s | 320 kB 00:00 2024-05-10T05:21:03.329 INFO:teuthology.orchestra.run.smithi142.stdout:(80/146): libtool-2.4.6-25.el8.x86_64.rpm 4.6 MB/s | 709 kB 00:00 2024-05-10T05:21:03.396 INFO:teuthology.orchestra.run.smithi142.stdout:(81/146): python3-markupsafe-0.23-19.el8.x86_64 290 kB/s | 39 kB 00:00 2024-05-10T05:21:03.397 INFO:teuthology.orchestra.run.smithi157.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2024-05-10T05:21:03.421 INFO:teuthology.orchestra.run.smithi142.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2024-05-10T05:21:03.422 INFO:teuthology.orchestra.run.smithi157.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2024-05-10T05:21:03.496 INFO:teuthology.orchestra.run.smithi142.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 383 kB/s | 64 kB 00:00 2024-05-10T05:21:03.497 INFO:teuthology.orchestra.run.smithi157.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 611 kB/s | 61 kB 00:00 2024-05-10T05:21:03.530 INFO:teuthology.orchestra.run.smithi142.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 774 kB/s | 103 kB 00:00 2024-05-10T05:21:03.531 INFO:teuthology.orchestra.run.smithi157.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 787 kB/s | 85 kB 00:00 2024-05-10T05:21:03.599 INFO:teuthology.orchestra.run.smithi157.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 228 kB/s | 23 kB 00:00 2024-05-10T05:21:03.614 INFO:teuthology.orchestra.run.smithi142.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 4.5 MB/s | 538 kB 00:00 2024-05-10T05:21:03.615 INFO:teuthology.orchestra.run.smithi157.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2024-05-10T05:21:03.681 INFO:teuthology.orchestra.run.smithi142.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2024-05-10T05:21:03.766 INFO:teuthology.orchestra.run.smithi157.stdout:(70/146): python3-packaging-16.8-9.el8.noarch.r 322 kB/s | 48 kB 00:00 2024-05-10T05:21:03.781 INFO:teuthology.orchestra.run.smithi142.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 15 MB/s | 3.7 MB 00:00 2024-05-10T05:21:03.806 INFO:teuthology.orchestra.run.smithi142.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2024-05-10T05:21:03.848 INFO:teuthology.orchestra.run.smithi142.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2024-05-10T05:21:03.858 INFO:teuthology.orchestra.run.smithi157.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 5.3 MB/s | 1.7 MB 00:00 2024-05-10T05:21:03.900 INFO:teuthology.orchestra.run.smithi157.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 134 kB/s | 18 kB 00:00 2024-05-10T05:21:03.940 INFO:teuthology.orchestra.run.smithi142.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2024-05-10T05:21:04.049 INFO:teuthology.orchestra.run.smithi142.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 784 kB/s | 157 kB 00:00 2024-05-10T05:21:04.075 INFO:teuthology.orchestra.run.smithi157.stdout:(73/146): python3-oauthlib-2.1.0-1.el8.noarch.r 915 kB/s | 155 kB 00:00 2024-05-10T05:21:04.092 INFO:teuthology.orchestra.run.smithi157.stdout:(74/146): python3-jwt-1.6.1-2.el8.noarch.rpm 182 kB/s | 43 kB 00:00 2024-05-10T05:21:04.174 INFO:teuthology.orchestra.run.smithi142.stdout:(92/146): openblas-0.3.15-6.el8.x86_64.rpm 22 MB/s | 5.0 MB 00:00 2024-05-10T05:21:04.207 INFO:teuthology.orchestra.run.smithi142.stdout:(93/146): automake-1.16.1-8.el8.noarch.rpm 4.4 MB/s | 714 kB 00:00 2024-05-10T05:21:04.297 INFO:teuthology.orchestra.run.smithi157.stdout:(75/146): python3-pycparser-2.14-14.el8.noarch. 492 kB/s | 109 kB 00:00 2024-05-10T05:21:04.299 INFO:teuthology.orchestra.run.smithi142.stdout:(94/146): openblas-threads-0.3.15-6.el8.x86_64. 11 MB/s | 5.2 MB 00:00 2024-05-10T05:21:04.317 INFO:teuthology.orchestra.run.smithi142.stdout:(95/146): jq-1.6-7.el8.x86_64.rpm 1.8 MB/s | 202 kB 00:00 2024-05-10T05:21:04.326 INFO:teuthology.orchestra.run.smithi157.stdout:(76/146): python3-requests-oauthlib-1.0.0-1.el8 183 kB/s | 43 kB 00:00 2024-05-10T05:21:04.384 INFO:teuthology.orchestra.run.smithi142.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 947 kB/s | 198 kB 00:00 2024-05-10T05:21:04.409 INFO:teuthology.orchestra.run.smithi142.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 498 kB/s | 54 kB 00:00 2024-05-10T05:21:04.426 INFO:teuthology.orchestra.run.smithi157.stdout:(77/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 145 kB/s | 19 kB 00:00 2024-05-10T05:21:04.451 INFO:teuthology.orchestra.run.smithi157.stdout:(78/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 350 kB/s | 44 kB 00:00 2024-05-10T05:21:04.475 INFO:teuthology.orchestra.run.smithi142.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-10T05:21:04.585 INFO:teuthology.orchestra.run.smithi157.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.7 MB/s | 605 kB 00:00 2024-05-10T05:21:04.610 INFO:teuthology.orchestra.run.smithi157.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2024-05-10T05:21:04.617 INFO:teuthology.orchestra.run.smithi142.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 7.2 MB/s | 1.5 MB 00:00 2024-05-10T05:21:04.660 INFO:teuthology.orchestra.run.smithi142.stdout:(100/146): postgresql-contrib-10.23-4.module+el 4.3 MB/s | 812 kB 00:00 2024-05-10T05:21:04.711 INFO:teuthology.orchestra.run.smithi157.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 2.0 MB/s | 200 kB 00:00 2024-05-10T05:21:04.768 INFO:teuthology.orchestra.run.smithi157.stdout:(82/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2024-05-10T05:21:04.826 INFO:teuthology.orchestra.run.smithi142.stdout:(101/146): python3-scipy-1.0.0-21.module+el8.9. 27 MB/s | 14 MB 00:00 2024-05-10T05:21:04.836 INFO:teuthology.orchestra.run.smithi157.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 385 kB/s | 48 kB 00:00 2024-05-10T05:21:04.851 INFO:teuthology.orchestra.run.smithi142.stdout:(102/146): python3-packaging-16.8-9.el8.noarch. 251 kB/s | 48 kB 00:00 2024-05-10T05:21:04.920 INFO:teuthology.orchestra.run.smithi157.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 574 kB/s | 87 kB 00:00 2024-05-10T05:21:04.977 INFO:teuthology.orchestra.run.smithi142.stdout:(103/146): postgresql-server-10.23-4.module+el8 14 MB/s | 5.1 MB 00:00 2024-05-10T05:21:05.002 INFO:teuthology.orchestra.run.smithi142.stdout:(104/146): libverto-devel-0.3.0-5.el8.x86_64.rp 102 kB/s | 18 kB 00:00 2024-05-10T05:21:05.020 INFO:teuthology.orchestra.run.smithi157.stdout:(85/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 314 kB/s | 58 kB 00:00 2024-05-10T05:21:05.027 INFO:teuthology.orchestra.run.smithi142.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 242 kB/s | 43 kB 00:00 2024-05-10T05:21:05.053 INFO:teuthology.orchestra.run.smithi157.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 289 kB/s | 39 kB 00:00 2024-05-10T05:21:05.086 INFO:teuthology.orchestra.run.smithi142.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.4 MB/s | 155 kB 00:00 2024-05-10T05:21:05.128 INFO:teuthology.orchestra.run.smithi142.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 870 kB/s | 109 kB 00:00 2024-05-10T05:21:05.153 INFO:teuthology.orchestra.run.smithi142.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 340 kB/s | 43 kB 00:00 2024-05-10T05:21:05.154 INFO:teuthology.orchestra.run.smithi157.stdout:(87/146): libgfortran-8.5.0-20.el8.x86_64.rpm 6.3 MB/s | 645 kB 00:00 2024-05-10T05:21:05.179 INFO:teuthology.orchestra.run.smithi157.stdout:(88/146): fuse-2.9.7-17.el8.x86_64.rpm 523 kB/s | 83 kB 00:00 2024-05-10T05:21:05.203 INFO:teuthology.orchestra.run.smithi142.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 159 kB/s | 19 kB 00:00 2024-05-10T05:21:05.258 INFO:teuthology.orchestra.run.smithi142.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 337 kB/s | 44 kB 00:00 2024-05-10T05:21:05.296 INFO:teuthology.orchestra.run.smithi157.stdout:(89/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.2 MB/s | 172 kB 00:00 2024-05-10T05:21:05.330 INFO:teuthology.orchestra.run.smithi157.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-05-10T05:21:05.337 INFO:teuthology.orchestra.run.smithi142.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2024-05-10T05:21:05.363 INFO:teuthology.orchestra.run.smithi142.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2024-05-10T05:21:05.379 INFO:teuthology.orchestra.run.smithi142.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2024-05-10T05:21:05.396 INFO:teuthology.orchestra.run.smithi157.stdout:(91/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 725 kB/s | 73 kB 00:00 2024-05-10T05:21:05.463 INFO:teuthology.orchestra.run.smithi157.stdout:(92/146): elfutils-libelf-devel-0.189-3.el8.x86 462 kB/s | 62 kB 00:00 2024-05-10T05:21:05.471 INFO:teuthology.orchestra.run.smithi142.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.5 MB/s | 200 kB 00:00 2024-05-10T05:21:05.496 INFO:teuthology.orchestra.run.smithi142.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 744 kB/s | 87 kB 00:00 2024-05-10T05:21:05.530 INFO:teuthology.orchestra.run.smithi142.stdout:(116/146): keyutils-libs-devel-1.5.10-9.el8.x86 287 kB/s | 48 kB 00:00 2024-05-10T05:21:05.565 INFO:teuthology.orchestra.run.smithi131.stdout:(10/146): ceph-osd-18.2.2-1330.g50174015.el8.x8 3.1 MB/s | 19 MB 00:05 2024-05-10T05:21:05.614 INFO:teuthology.orchestra.run.smithi142.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 405 kB/s | 58 kB 00:00 2024-05-10T05:21:05.638 INFO:teuthology.orchestra.run.smithi157.stdout:(93/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-10T05:21:05.681 INFO:teuthology.orchestra.run.smithi142.stdout:(118/146): fuse-2.9.7-17.el8.x86_64.rpm 553 kB/s | 83 kB 00:00 2024-05-10T05:21:05.705 INFO:teuthology.orchestra.run.smithi157.stdout:(94/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 1.8 MB/s | 562 kB 00:00 2024-05-10T05:21:05.709 INFO:teuthology.orchestra.run.smithi131.stdout:(11/146): cephfs-mirror-18.2.2-1330.g50174015.e 1.7 MB/s | 246 kB 00:00 2024-05-10T05:21:05.715 INFO:teuthology.orchestra.run.smithi142.stdout:(119/146): libcom_err-devel-1.45.6-4.el8.x86_64 177 kB/s | 39 kB 00:00 2024-05-10T05:21:05.731 INFO:teuthology.orchestra.run.smithi157.stdout:(95/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 513 kB/s | 47 kB 00:00 2024-05-10T05:21:05.774 INFO:teuthology.orchestra.run.smithi131.stdout:(12/146): libcephfs-devel-18.2.2-1330.g50174015 481 kB/s | 31 kB 00:00 2024-05-10T05:21:05.781 INFO:teuthology.orchestra.run.smithi142.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.8 MB/s | 645 kB 00:00 2024-05-10T05:21:05.815 INFO:teuthology.orchestra.run.smithi142.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-05-10T05:21:05.856 INFO:teuthology.orchestra.run.smithi157.stdout:(96/146): NetworkManager-initscripts-updown-1.4 1.1 MB/s | 144 kB 00:00 2024-05-10T05:21:05.865 INFO:teuthology.orchestra.run.smithi142.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-05-10T05:21:05.891 INFO:teuthology.orchestra.run.smithi142.stdout:(123/146): elfutils-libelf-devel-0.189-3.el8.x8 820 kB/s | 62 kB 00:00 2024-05-10T05:21:05.923 INFO:teuthology.orchestra.run.smithi157.stdout:(97/146): python3-cryptography-3.2.1-7.el8_9.x8 2.5 MB/s | 559 kB 00:00 2024-05-10T05:21:05.949 INFO:teuthology.orchestra.run.smithi142.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 433 kB/s | 73 kB 00:00 2024-05-10T05:21:06.016 INFO:teuthology.orchestra.run.smithi142.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.7 MB/s | 562 kB 00:00 2024-05-10T05:21:06.041 INFO:teuthology.orchestra.run.smithi142.stdout:(126/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 513 kB/s | 47 kB 00:00 2024-05-10T05:21:06.075 INFO:teuthology.orchestra.run.smithi142.stdout:(127/146): python3-cryptography-3.2.1-7.el8_9.x 9.3 MB/s | 559 kB 00:00 2024-05-10T05:21:06.091 INFO:teuthology.orchestra.run.smithi131.stdout:(13/146): ceph-radosgw-18.2.2-1330.g50174015.el 3.4 MB/s | 14 MB 00:04 2024-05-10T05:21:06.124 INFO:teuthology.orchestra.run.smithi131.stdout:(14/146): libcephfs2-18.2.2-1330.g50174015.el8. 2.1 MB/s | 756 kB 00:00 2024-05-10T05:21:06.125 INFO:teuthology.orchestra.run.smithi142.stdout:(128/146): openssl-devel-1.1.1k-12.el8_9.x86_64 9.9 MB/s | 2.3 MB 00:00 2024-05-10T05:21:06.150 INFO:teuthology.orchestra.run.smithi131.stdout:(15/146): libcephsqlite-18.2.2-1330.g50174015.e 2.3 MB/s | 139 kB 00:00 2024-05-10T05:21:06.151 INFO:teuthology.orchestra.run.smithi142.stdout:(129/146): NetworkManager-initscripts-updown-1. 1.3 MB/s | 144 kB 00:00 2024-05-10T05:21:06.208 INFO:teuthology.orchestra.run.smithi131.stdout:(16/146): librados-devel-18.2.2-1330.g50174015. 1.6 MB/s | 139 kB 00:00 2024-05-10T05:21:06.226 INFO:teuthology.orchestra.run.smithi142.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 664 kB/s | 100 kB 00:00 2024-05-10T05:21:06.275 INFO:teuthology.orchestra.run.smithi131.stdout:(17/146): libradosstriper1-18.2.2-1330.g5017401 3.6 MB/s | 461 kB 00:00 2024-05-10T05:21:06.342 INFO:teuthology.orchestra.run.smithi131.stdout:(18/146): python3-ceph-argparse-18.2.2-1330.g50 673 kB/s | 45 kB 00:00 2024-05-10T05:21:06.393 INFO:teuthology.orchestra.run.smithi142.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.4 MB/s | 232 kB 00:00 2024-05-10T05:21:06.409 INFO:teuthology.orchestra.run.smithi131.stdout:(19/146): python3-ceph-common-18.2.2-1330.g5017 1.9 MB/s | 129 kB 00:00 2024-05-10T05:21:06.424 INFO:teuthology.orchestra.run.smithi157.stdout:(98/146): ceph-test-18.2.2-1330.g50174015.el8.x 5.9 MB/s | 49 MB 00:08 2024-05-10T05:21:06.449 INFO:teuthology.orchestra.run.smithi157.stdout:(99/146): python3-jsonpointer-1.10-11.el8.noarc 38 kB/s | 20 kB 00:00 2024-05-10T05:21:06.476 INFO:teuthology.orchestra.run.smithi131.stdout:(20/146): python3-cephfs-18.2.2-1330.g50174015. 2.9 MB/s | 197 kB 00:00 2024-05-10T05:21:06.482 INFO:teuthology.orchestra.run.smithi157.stdout:(100/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 159 kB/s | 100 kB 00:00 2024-05-10T05:21:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2024-05-10T05:21:06.585 INFO:teuthology.orchestra.run.smithi131.stdout:(21/146): python3-rados-18.2.2-1330.g50174015.e 3.4 MB/s | 371 kB 00:00 2024-05-10T05:21:06.624 INFO:teuthology.orchestra.run.smithi157.stdout:(101/146): python3-jsonpatch-1.21-2.el8.noarch. 188 kB/s | 27 kB 00:00 2024-05-10T05:21:06.658 INFO:teuthology.orchestra.run.smithi157.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 542 kB/s | 110 kB 00:00 2024-05-10T05:21:06.702 INFO:teuthology.orchestra.run.smithi142.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2024-05-10T05:21:06.708 INFO:teuthology.orchestra.run.smithi157.stdout:(103/146): python3-prettytable-0.7.2-14.el8.noa 155 kB/s | 44 kB 00:00 2024-05-10T05:21:06.725 INFO:teuthology.orchestra.run.smithi157.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.2 MB/s | 126 kB 00:00 2024-05-10T05:21:06.811 INFO:teuthology.orchestra.run.smithi142.stdout:(134/146): librados2-18.2.2-1330.g50174015.el8. 5.2 MB/s | 3.5 MB 00:00 2024-05-10T05:21:06.844 INFO:teuthology.orchestra.run.smithi142.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.9 MB/s | 711 kB 00:00 2024-05-10T05:21:06.867 INFO:teuthology.orchestra.run.smithi157.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 2.1 MB/s | 457 kB 00:00 2024-05-10T05:21:06.900 INFO:teuthology.orchestra.run.smithi157.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2024-05-10T05:21:06.928 INFO:teuthology.orchestra.run.smithi131.stdout:(22/146): python3-rbd-18.2.2-1330.g50174015.el8 1.0 MB/s | 364 kB 00:00 2024-05-10T05:21:06.934 INFO:teuthology.orchestra.run.smithi157.stdout:(107/146): bison-3.0.4-10.el8.x86_64.rpm 3.0 MB/s | 688 kB 00:00 2024-05-10T05:21:06.978 INFO:teuthology.orchestra.run.smithi142.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.8 MB/s | 1.5 MB 00:00 2024-05-10T05:21:06.993 INFO:teuthology.orchestra.run.smithi157.stdout:(108/146): python3-markupsafe-0.23-19.el8.x86_6 422 kB/s | 39 kB 00:00 2024-05-10T05:21:06.994 INFO:teuthology.orchestra.run.smithi131.stdout:(23/146): python3-rgw-18.2.2-1330.g50174015.el8 1.6 MB/s | 107 kB 00:00 2024-05-10T05:21:07.011 INFO:teuthology.orchestra.run.smithi142.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.0 MB/s | 333 kB 00:00 2024-05-10T05:21:07.043 INFO:teuthology.orchestra.run.smithi157.stdout:(109/146): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2024-05-10T05:21:07.052 INFO:teuthology.orchestra.run.smithi131.stdout:(24/146): rbd-fuse-18.2.2-1330.g50174015.el8.x8 1.5 MB/s | 91 kB 00:00 2024-05-10T05:21:07.085 INFO:teuthology.orchestra.run.smithi157.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2024-05-10T05:21:07.118 INFO:teuthology.orchestra.run.smithi157.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 510 kB/s | 64 kB 00:00 2024-05-10T05:21:07.145 INFO:teuthology.orchestra.run.smithi142.stdout:(138/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.6 MB/s | 221 kB 00:00 2024-05-10T05:21:07.169 INFO:teuthology.orchestra.run.smithi157.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 824 kB/s | 103 kB 00:00 2024-05-10T05:21:07.187 INFO:teuthology.orchestra.run.smithi142.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.1 MB/s | 1.1 MB 00:00 2024-05-10T05:21:07.219 INFO:teuthology.orchestra.run.smithi157.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 5.2 MB/s | 538 kB 00:00 2024-05-10T05:21:07.287 INFO:teuthology.orchestra.run.smithi142.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.8 MB/s | 254 kB 00:00 2024-05-10T05:21:07.321 INFO:teuthology.orchestra.run.smithi142.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 237 kB 00:00 2024-05-10T05:21:07.344 INFO:teuthology.orchestra.run.smithi157.stdout:(114/146): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2024-05-10T05:21:07.370 INFO:teuthology.orchestra.run.smithi157.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.1 MB/s | 323 kB 00:00 2024-05-10T05:21:07.371 INFO:teuthology.orchestra.run.smithi142.stdout:(142/146): librbd1-18.2.2-1330.g50174015.el8.x8 2.9 MB/s | 3.5 MB 00:01 2024-05-10T05:21:07.396 INFO:teuthology.orchestra.run.smithi142.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.6 MB/s | 180 kB 00:00 2024-05-10T05:21:07.438 INFO:teuthology.orchestra.run.smithi142.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 845 kB/s | 99 kB 00:00 2024-05-10T05:21:07.462 INFO:teuthology.orchestra.run.smithi157.stdout:(116/146): python3-numpy-1.14.3-10.el8.x86_64.r 13 MB/s | 3.7 MB 00:00 2024-05-10T05:21:07.487 INFO:teuthology.orchestra.run.smithi157.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.6 MB/s | 225 kB 00:00 2024-05-10T05:21:07.521 INFO:teuthology.orchestra.run.smithi142.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 16 MB/s | 2.5 MB 00:00 2024-05-10T05:21:07.537 INFO:teuthology.orchestra.run.smithi157.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.1 MB/s | 354 kB 00:00 2024-05-10T05:21:07.555 INFO:teuthology.orchestra.run.smithi142.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 648 kB/s | 103 kB 00:00 2024-05-10T05:21:07.558 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-05-10T05:21:07.558 INFO:teuthology.orchestra.run.smithi142.stdout:Total 13 MB/s | 221 MB 00:16 2024-05-10T05:21:07.622 INFO:teuthology.orchestra.run.smithi157.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2024-05-10T05:21:07.655 INFO:teuthology.orchestra.run.smithi131.stdout:(25/146): rbd-mirror-18.2.2-1330.g50174015.el8. 5.5 MB/s | 3.3 MB 00:00 2024-05-10T05:21:07.728 INFO:teuthology.orchestra.run.smithi131.stdout:(26/146): rbd-nbd-18.2.2-1330.g50174015.el8.x86 2.4 MB/s | 179 kB 00:00 2024-05-10T05:21:07.748 INFO:teuthology.orchestra.run.smithi157.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 18 MB/s | 5.2 MB 00:00 2024-05-10T05:21:07.788 INFO:teuthology.orchestra.run.smithi157.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 4.2 MB/s | 714 kB 00:00 2024-05-10T05:21:07.795 INFO:teuthology.orchestra.run.smithi131.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1330.g 366 kB/s | 24 kB 00:00 2024-05-10T05:21:07.846 INFO:teuthology.orchestra.run.smithi157.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 2.0 MB/s | 198 kB 00:00 2024-05-10T05:21:07.862 INFO:teuthology.orchestra.run.smithi131.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1330.g5017401 2.0 MB/s | 138 kB 00:00 2024-05-10T05:21:07.913 INFO:teuthology.orchestra.run.smithi157.stdout:(123/146): openblas-0.3.15-6.el8.x86_64.rpm 13 MB/s | 5.0 MB 00:00 2024-05-10T05:21:07.938 INFO:teuthology.orchestra.run.smithi157.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2024-05-10T05:21:07.962 INFO:teuthology.orchestra.run.smithi131.stdout:(29/146): librgw2-18.2.2-1330.g50174015.el8.x86 4.1 MB/s | 7.2 MB 00:01 2024-05-10T05:21:07.964 INFO:teuthology.orchestra.run.smithi157.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 464 kB/s | 54 kB 00:00 2024-05-10T05:21:08.089 INFO:teuthology.orchestra.run.smithi157.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-10T05:21:08.148 INFO:teuthology.orchestra.run.smithi157.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 8.2 MB/s | 1.5 MB 00:00 2024-05-10T05:21:08.249 INFO:teuthology.orchestra.run.smithi157.stdout:(128/146): postgresql-contrib-10.23-4.module+el 5.0 MB/s | 812 kB 00:00 2024-05-10T05:21:08.388 INFO:teuthology.orchestra.run.smithi131.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1330.g50174 6.7 MB/s | 3.5 MB 00:00 2024-05-10T05:21:08.454 INFO:teuthology.orchestra.run.smithi131.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1330.g50 3.6 MB/s | 246 kB 00:00 2024-05-10T05:21:08.473 INFO:teuthology.orchestra.run.smithi157.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 24 MB/s | 14 MB 00:00 2024-05-10T05:21:08.522 INFO:teuthology.orchestra.run.smithi131.stdout:(32/146): ceph-mgr-rook-18.2.2-1330.g50174015.e 736 kB/s | 49 kB 00:00 2024-05-10T05:21:08.582 INFO:teuthology.orchestra.run.smithi157.stdout:(130/146): postgresql-server-10.23-4.module+el8 12 MB/s | 5.1 MB 00:00 2024-05-10T05:21:08.588 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-05-10T05:21:08.589 INFO:teuthology.orchestra.run.smithi131.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1330.g5 246 kB/s | 16 kB 00:00 2024-05-10T05:21:08.672 INFO:teuthology.orchestra.run.smithi131.stdout:(34/146): ceph-volume-18.2.2-1330.g50174015.el8 3.1 MB/s | 263 kB 00:00 2024-05-10T05:21:08.733 INFO:teuthology.orchestra.run.smithi157.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.5 MB/s | 232 kB 00:00 2024-05-10T05:21:08.747 INFO:teuthology.orchestra.run.smithi131.stdout:(35/146): cephadm-18.2.2-1330.g50174015.el8.noa 2.3 MB/s | 173 kB 00:00 2024-05-10T05:21:08.814 INFO:teuthology.orchestra.run.smithi131.stdout:(36/146): cephfs-top-18.2.2-1330.g50174015.el8. 272 kB/s | 18 kB 00:00 2024-05-10T05:21:08.891 INFO:teuthology.orchestra.run.smithi157.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.9 MB/s | 303 kB 00:00 2024-05-10T05:21:08.966 INFO:teuthology.orchestra.run.smithi157.stdout:(133/146): librbd1-18.2.2-1330.g50174015.el8.x8 7.2 MB/s | 3.5 MB 00:00 2024-05-10T05:21:08.981 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-05-10T05:21:08.982 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-05-10T05:21:09.059 INFO:teuthology.orchestra.run.smithi157.stdout:(134/146): librados2-18.2.2-1330.g50174015.el8. 4.4 MB/s | 3.5 MB 00:00 2024-05-10T05:21:09.100 INFO:teuthology.orchestra.run.smithi157.stdout:(135/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.0 MB/s | 854 kB 00:00 2024-05-10T05:21:09.167 INFO:teuthology.orchestra.run.smithi157.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.5 MB/s | 711 kB 00:00 2024-05-10T05:21:09.234 INFO:teuthology.orchestra.run.smithi157.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.4 MB/s | 1.5 MB 00:00 2024-05-10T05:21:09.267 INFO:teuthology.orchestra.run.smithi157.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2024-05-10T05:21:09.309 INFO:teuthology.orchestra.run.smithi157.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.5 MB/s | 1.1 MB 00:00 2024-05-10T05:21:09.360 INFO:teuthology.orchestra.run.smithi157.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 221 kB 00:00 2024-05-10T05:21:09.365 INFO:teuthology.orchestra.run.smithi131.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 557 kB/s | 306 kB 00:00 2024-05-10T05:21:09.385 INFO:teuthology.orchestra.run.smithi157.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 254 kB 00:00 2024-05-10T05:21:09.418 INFO:teuthology.orchestra.run.smithi157.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 237 kB 00:00 2024-05-10T05:21:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 2.1 MB/s | 180 kB 00:00 2024-05-10T05:21:09.448 INFO:teuthology.orchestra.run.smithi131.stdout:(38/146): ceph-mgr-diskprediction-local-18.2.2- 5.0 MB/s | 7.4 MB 00:01 2024-05-10T05:21:09.473 INFO:teuthology.orchestra.run.smithi131.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 544 kB/s | 59 kB 00:00 2024-05-10T05:21:09.511 INFO:teuthology.orchestra.run.smithi157.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 789 kB/s | 99 kB 00:00 2024-05-10T05:21:09.619 INFO:teuthology.orchestra.run.smithi157.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 586 kB/s | 103 kB 00:00 2024-05-10T05:21:09.669 INFO:teuthology.orchestra.run.smithi157.stdout:(146/146): util-linux-2.32.1-44.el8_9.1.x86_64. 9.9 MB/s | 2.5 MB 00:00 2024-05-10T05:21:09.672 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-05-10T05:21:09.672 INFO:teuthology.orchestra.run.smithi157.stdout:Total 12 MB/s | 221 MB 00:18 2024-05-10T05:21:09.674 INFO:teuthology.orchestra.run.smithi131.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.3 MB/s | 476 kB 00:00 2024-05-10T05:21:09.774 INFO:teuthology.orchestra.run.smithi131.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 437 kB/s | 44 kB 00:00 2024-05-10T05:21:09.816 INFO:teuthology.orchestra.run.smithi131.stdout:(42/146): libunwind-1.3.1-3.el8.x86_64.rpm 204 kB/s | 75 kB 00:00 2024-05-10T05:21:09.883 INFO:teuthology.orchestra.run.smithi131.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2024-05-10T05:21:09.933 INFO:teuthology.orchestra.run.smithi131.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2024-05-10T05:21:09.975 INFO:teuthology.orchestra.run.smithi131.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 177 kB/s | 16 kB 00:00 2024-05-10T05:21:10.109 INFO:teuthology.orchestra.run.smithi131.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2024-05-10T05:21:10.168 INFO:teuthology.orchestra.run.smithi131.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 737 kB/s | 173 kB 00:00 2024-05-10T05:21:10.214 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-05-10T05:21:10.224 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-05-10T05:21:10.268 INFO:teuthology.orchestra.run.smithi131.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-05-10T05:21:10.293 INFO:teuthology.orchestra.run.smithi131.stdout:(49/146): python3-google-auth-1.1.1-10.el8.noar 442 kB/s | 81 kB 00:00 2024-05-10T05:21:10.369 INFO:teuthology.orchestra.run.smithi131.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2024-05-10T05:21:10.469 INFO:teuthology.orchestra.run.smithi131.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 490 kB/s | 49 kB 00:00 2024-05-10T05:21:10.578 INFO:teuthology.orchestra.run.smithi131.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 542 kB/s | 59 kB 00:00 2024-05-10T05:21:10.611 INFO:teuthology.orchestra.run.smithi131.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 3.7 MB/s | 1.2 MB 00:00 2024-05-10T05:21:10.677 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-05-10T05:21:10.678 INFO:teuthology.orchestra.run.smithi131.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 604 kB/s | 60 kB 00:00 2024-05-10T05:21:10.720 INFO:teuthology.orchestra.run.smithi131.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2024-05-10T05:21:10.779 INFO:teuthology.orchestra.run.smithi131.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2024-05-10T05:21:10.821 INFO:teuthology.orchestra.run.smithi131.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 335 kB/s | 34 kB 00:00 2024-05-10T05:21:10.887 INFO:teuthology.orchestra.run.smithi131.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2024-05-10T05:21:10.929 INFO:teuthology.orchestra.run.smithi131.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 593 kB/s | 64 kB 00:00 2024-05-10T05:21:10.988 INFO:teuthology.orchestra.run.smithi131.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 186 kB/s | 19 kB 00:00 2024-05-10T05:21:11.047 INFO:teuthology.orchestra.run.smithi131.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2024-05-10T05:21:11.071 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-05-10T05:21:11.071 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-05-10T05:21:11.089 INFO:teuthology.orchestra.run.smithi131.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 292 kB/s | 29 kB 00:00 2024-05-10T05:21:11.148 INFO:teuthology.orchestra.run.smithi131.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2024-05-10T05:21:11.206 INFO:teuthology.orchestra.run.smithi131.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 2.1 MB/s | 255 kB 00:00 2024-05-10T05:21:11.256 INFO:teuthology.orchestra.run.smithi131.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2024-05-10T05:21:11.310 INFO:teuthology.orchestra.run.smithi131.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 611 kB/s | 61 kB 00:00 2024-05-10T05:21:11.357 INFO:teuthology.orchestra.run.smithi131.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 853 kB/s | 85 kB 00:00 2024-05-10T05:21:11.916 INFO:teuthology.orchestra.run.smithi131.stdout:(68/146): ceph-test-18.2.2-1330.g50174015.el8.x 5.3 MB/s | 49 MB 00:09 2024-05-10T05:21:11.941 INFO:teuthology.orchestra.run.smithi131.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 36 kB/s | 23 kB 00:00 2024-05-10T05:21:12.000 INFO:teuthology.orchestra.run.smithi131.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 837 kB/s | 70 kB 00:00 2024-05-10T05:21:12.058 INFO:teuthology.orchestra.run.smithi131.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 2.5 MB/s | 1.7 MB 00:00 2024-05-10T05:21:12.184 INFO:teuthology.orchestra.run.smithi131.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 101 kB/s | 18 kB 00:00 2024-05-10T05:21:12.250 INFO:teuthology.orchestra.run.smithi131.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 229 kB/s | 43 kB 00:00 2024-05-10T05:21:12.284 INFO:teuthology.orchestra.run.smithi131.stdout:(74/146): python3-packaging-16.8-9.el8.noarch.r 141 kB/s | 48 kB 00:00 2024-05-10T05:21:12.295 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-05-10T05:21:12.305 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-05-10T05:21:12.342 INFO:teuthology.orchestra.run.smithi131.stdout:(75/146): python3-pycparser-2.14-14.el8.noarch. 1.2 MB/s | 109 kB 00:00 2024-05-10T05:21:12.368 INFO:teuthology.orchestra.run.smithi131.stdout:(76/146): python3-oauthlib-2.1.0-1.el8.noarch.r 842 kB/s | 155 kB 00:00 2024-05-10T05:21:12.471 INFO:teuthology.orchestra.run.smithi131.stdout:(77/146): python3-requests-oauthlib-1.0.0-1.el8 228 kB/s | 43 kB 00:00 2024-05-10T05:21:12.502 INFO:teuthology.orchestra.run.smithi131.stdout:(78/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 118 kB/s | 19 kB 00:00 2024-05-10T05:21:12.610 INFO:teuthology.orchestra.run.smithi131.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2024-05-10T05:21:12.644 INFO:teuthology.orchestra.run.smithi131.stdout:(80/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 159 kB/s | 44 kB 00:00 2024-05-10T05:21:12.761 INFO:teuthology.orchestra.run.smithi131.stdout:(81/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.0 MB/s | 605 kB 00:00 2024-05-10T05:21:12.795 INFO:teuthology.orchestra.run.smithi131.stdout:(82/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.3 MB/s | 200 kB 00:00 2024-05-10T05:21:12.820 INFO:teuthology.orchestra.run.smithi131.stdout:(83/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.0 MB/s | 220 kB 00:00 2024-05-10T05:21:12.878 INFO:teuthology.orchestra.run.smithi131.stdout:(84/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 411 kB/s | 48 kB 00:00 2024-05-10T05:21:12.904 INFO:teuthology.orchestra.run.smithi131.stdout:(85/146): libsepol-devel-2.9-3.el8.x86_64.rpm 801 kB/s | 87 kB 00:00 2024-05-10T05:21:12.929 INFO:teuthology.orchestra.run.smithi131.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 772 kB/s | 39 kB 00:00 2024-05-10T05:21:12.988 INFO:teuthology.orchestra.run.smithi131.stdout:(87/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 344 kB/s | 58 kB 00:00 2024-05-10T05:21:13.063 INFO:teuthology.orchestra.run.smithi131.stdout:(88/146): fuse-2.9.7-17.el8.x86_64.rpm 523 kB/s | 83 kB 00:00 2024-05-10T05:21:13.113 INFO:teuthology.orchestra.run.smithi131.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 645 kB 00:00 2024-05-10T05:21:13.155 INFO:teuthology.orchestra.run.smithi131.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2024-05-10T05:21:13.189 INFO:teuthology.orchestra.run.smithi131.stdout:(91/146): libquadmath-8.5.0-20.el8.x86_64.rpm 858 kB/s | 172 kB 00:00 2024-05-10T05:21:13.214 INFO:teuthology.orchestra.run.smithi131.stdout:(92/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 721 kB/s | 73 kB 00:00 2024-05-10T05:21:13.264 INFO:teuthology.orchestra.run.smithi131.stdout:(93/146): elfutils-libelf-devel-0.189-3.el8.x86 567 kB/s | 62 kB 00:00 2024-05-10T05:21:13.356 INFO:teuthology.orchestra.run.smithi131.stdout:(94/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-05-10T05:21:13.398 INFO:teuthology.orchestra.run.smithi131.stdout:(95/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 2.6 MB/s | 562 kB 00:00 2024-05-10T05:21:13.482 INFO:teuthology.orchestra.run.smithi131.stdout:(96/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 8.7 MB/s | 2.3 MB 00:00 2024-05-10T05:21:13.515 INFO:teuthology.orchestra.run.smithi131.stdout:(97/146): python3-cryptography-3.2.1-7.el8_9.x8 3.4 MB/s | 559 kB 00:00 2024-05-10T05:21:13.541 INFO:teuthology.orchestra.run.smithi131.stdout:(98/146): NetworkManager-initscripts-updown-1.4 1.0 MB/s | 144 kB 00:00 2024-05-10T05:21:13.591 INFO:teuthology.orchestra.run.smithi131.stdout:(99/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 919 kB/s | 100 kB 00:00 2024-05-10T05:21:13.625 INFO:teuthology.orchestra.run.smithi131.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 184 kB/s | 20 kB 00:00 2024-05-10T05:21:13.650 INFO:teuthology.orchestra.run.smithi131.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 404 kB/s | 44 kB 00:00 2024-05-10T05:21:13.683 INFO:teuthology.orchestra.run.smithi131.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 1.2 MB/s | 110 kB 00:00 2024-05-10T05:21:13.742 INFO:teuthology.orchestra.run.smithi131.stdout:(103/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.3 MB/s | 126 kB 00:00 2024-05-10T05:21:13.768 INFO:teuthology.orchestra.run.smithi131.stdout:(104/146): python3-jsonpatch-1.21-2.el8.noarch. 186 kB/s | 27 kB 00:00 2024-05-10T05:21:13.818 INFO:teuthology.orchestra.run.smithi131.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 3.3 MB/s | 457 kB 00:00 2024-05-10T05:21:13.876 INFO:teuthology.orchestra.run.smithi131.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2024-05-10T05:21:13.940 INFO:teuthology.orchestra.run.smithi131.stdout:(107/146): bison-3.0.4-10.el8.x86_64.rpm 3.4 MB/s | 688 kB 00:00 2024-05-10T05:21:13.986 INFO:teuthology.orchestra.run.smithi131.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 4.1 MB/s | 709 kB 00:00 2024-05-10T05:21:14.010 INFO:teuthology.orchestra.run.smithi131.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 290 kB/s | 39 kB 00:00 2024-05-10T05:21:14.044 INFO:teuthology.orchestra.run.smithi131.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.4 MB/s | 259 kB 00:00 2024-05-10T05:21:14.094 INFO:teuthology.orchestra.run.smithi131.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 590 kB/s | 64 kB 00:00 2024-05-10T05:21:14.119 INFO:teuthology.orchestra.run.smithi131.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 953 kB/s | 103 kB 00:00 2024-05-10T05:21:14.203 INFO:teuthology.orchestra.run.smithi131.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 4.8 MB/s | 538 kB 00:00 2024-05-10T05:21:14.362 INFO:teuthology.orchestra.run.smithi131.stdout:(114/146): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2024-05-10T05:21:14.462 INFO:teuthology.orchestra.run.smithi131.stdout:(115/146): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2024-05-10T05:21:14.487 INFO:teuthology.orchestra.run.smithi131.stdout:(116/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.1 MB/s | 323 kB 00:00 2024-05-10T05:21:14.529 INFO:teuthology.orchestra.run.smithi131.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.3 MB/s | 225 kB 00:00 2024-05-10T05:21:14.588 INFO:teuthology.orchestra.run.smithi131.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2024-05-10T05:21:14.722 INFO:teuthology.orchestra.run.smithi131.stdout:(119/146): openblas-threads-0.3.15-6.el8.x86_64 22 MB/s | 5.2 MB 00:00 2024-05-10T05:21:14.747 INFO:teuthology.orchestra.run.smithi131.stdout:(120/146): python3-mako-1.0.6-14.el8.noarch.rpm 722 kB/s | 157 kB 00:00 2024-05-10T05:21:14.847 INFO:teuthology.orchestra.run.smithi131.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 5.6 MB/s | 714 kB 00:00 2024-05-10T05:21:14.947 INFO:teuthology.orchestra.run.smithi131.stdout:(122/146): openblas-0.3.15-6.el8.x86_64.rpm 14 MB/s | 5.0 MB 00:00 2024-05-10T05:21:14.972 INFO:teuthology.orchestra.run.smithi131.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 878 kB/s | 198 kB 00:00 2024-05-10T05:21:14.998 INFO:teuthology.orchestra.run.smithi131.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2024-05-10T05:21:15.090 INFO:teuthology.orchestra.run.smithi131.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 382 kB/s | 54 kB 00:00 2024-05-10T05:21:15.173 INFO:teuthology.orchestra.run.smithi131.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-05-10T05:21:15.282 INFO:teuthology.orchestra.run.smithi131.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 7.8 MB/s | 1.5 MB 00:00 2024-05-10T05:21:15.324 INFO:teuthology.orchestra.run.smithi131.stdout:(128/146): postgresql-contrib-10.23-4.module+el 5.3 MB/s | 812 kB 00:00 2024-05-10T05:21:15.666 INFO:teuthology.orchestra.run.smithi131.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 20 MB/s | 14 MB 00:00 2024-05-10T05:21:15.766 INFO:teuthology.orchestra.run.smithi131.stdout:(130/146): postgresql-server-10.23-4.module+el8 10 MB/s | 5.1 MB 00:00 2024-05-10T05:21:15.933 INFO:teuthology.orchestra.run.smithi131.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.4 MB/s | 232 kB 00:00 2024-05-10T05:21:16.039 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-05-10T05:21:16.050 INFO:teuthology.orchestra.run.smithi131.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2024-05-10T05:21:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-10T05:21:16.225 INFO:teuthology.orchestra.run.smithi131.stdout:(133/146): librados2-18.2.2-1330.g50174015.el8. 3.9 MB/s | 3.5 MB 00:00 2024-05-10T05:21:16.284 INFO:teuthology.orchestra.run.smithi131.stdout:(134/146): librbd1-18.2.2-1330.g50174015.el8.x8 5.7 MB/s | 3.5 MB 00:00 2024-05-10T05:21:16.334 INFO:teuthology.orchestra.run.smithi131.stdout:(135/146): iproute-6.2.0-5.el8_9.x86_64.rpm 2.9 MB/s | 854 kB 00:00 2024-05-10T05:21:16.409 INFO:teuthology.orchestra.run.smithi131.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 12 MB/s | 1.5 MB 00:00 2024-05-10T05:21:16.434 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T05:21:16.442 INFO:teuthology.orchestra.run.smithi131.stdout:(137/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.2 MB/s | 711 kB 00:00 2024-05-10T05:21:16.476 INFO:teuthology.orchestra.run.smithi131.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.3 MB/s | 333 kB 00:00 2024-05-10T05:21:16.576 INFO:teuthology.orchestra.run.smithi131.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.6 MB/s | 221 kB 00:00 2024-05-10T05:21:16.610 INFO:teuthology.orchestra.run.smithi131.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.9 MB/s | 254 kB 00:00 2024-05-10T05:21:16.652 INFO:teuthology.orchestra.run.smithi131.stdout:(141/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 4.4 MB/s | 1.1 MB 00:00 2024-05-10T05:21:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T05:21:16.728 INFO:teuthology.orchestra.run.smithi131.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 237 kB 00:00 2024-05-10T05:21:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.2 MB/s | 180 kB 00:00 2024-05-10T05:21:16.786 INFO:teuthology.orchestra.run.smithi131.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 741 kB/s | 99 kB 00:00 2024-05-10T05:21:16.886 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-10T05:21:16.903 INFO:teuthology.orchestra.run.smithi131.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 14 MB/s | 2.5 MB 00:00 2024-05-10T05:21:17.078 INFO:teuthology.orchestra.run.smithi131.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 316 kB/s | 103 kB 00:00 2024-05-10T05:21:17.082 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-05-10T05:21:17.082 INFO:teuthology.orchestra.run.smithi131.stdout:Total 9.5 MB/s | 221 MB 00:23 2024-05-10T05:21:17.082 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T05:21:17.198 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T05:21:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T05:21:17.693 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T05:21:17.898 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-10T05:21:18.074 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-10T05:21:18.144 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-10T05:21:18.183 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-05-10T05:21:18.317 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-10T05:21:18.369 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-10T05:21:18.540 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T05:21:18.586 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T05:21:18.801 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T05:21:18.991 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T05:21:18.995 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-10T05:21:19.190 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T05:21:19.228 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-05-10T05:21:19.233 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T05:21:19.312 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T05:21:19.517 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T05:21:19.599 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T05:21:19.640 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T05:21:19.672 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-05-10T05:21:19.672 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-05-10T05:21:19.813 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-10T05:21:19.823 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T05:21:19.951 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T05:21:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-10T05:21:20.140 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T05:21:20.150 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T05:21:20.237 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T05:21:20.248 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T05:21:20.458 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T05:21:20.549 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T05:21:20.831 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T05:21:20.872 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T05:21:20.920 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T05:21:21.049 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T05:21:21.146 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-10T05:21:21.324 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-05-10T05:21:21.337 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-05-10T05:21:21.426 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-10T05:21:21.463 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T05:21:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-10T05:21:21.649 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T05:21:21.774 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T05:21:21.958 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T05:21:21.979 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-10T05:21:22.035 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T05:21:22.235 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-10T05:21:22.351 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T05:21:22.419 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T05:21:22.523 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T05:21:22.649 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T05:21:22.687 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T05:21:22.734 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T05:21:22.752 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T05:21:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-10T05:21:22.947 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-10T05:21:23.170 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-10T05:21:23.236 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-10T05:21:23.386 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-10T05:21:23.506 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-10T05:21:23.769 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-10T05:21:24.020 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-10T05:21:24.128 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-10T05:21:24.332 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-10T05:21:24.491 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T05:21:24.596 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T05:21:24.740 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T05:21:24.817 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T05:21:24.934 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-10T05:21:24.994 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-10T05:21:25.106 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-10T05:21:25.249 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-10T05:21:25.496 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-10T05:21:25.657 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T05:21:25.793 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T05:21:25.821 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-10T05:21:25.957 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T05:21:26.078 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-10T05:21:26.252 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-10T05:21:26.408 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-10T05:21:26.577 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T05:21:26.705 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-10T05:21:26.739 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T05:21:27.010 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-10T05:21:27.098 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-05-10T05:21:27.284 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-10T05:21:27.290 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-10T05:21:27.326 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T05:21:27.462 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 31/162 2024-05-10T05:21:27.515 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T05:21:27.519 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T05:21:27.641 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 32/162 2024-05-10T05:21:27.815 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T05:21:27.883 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 33/162 2024-05-10T05:21:28.037 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-10T05:21:28.181 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T05:21:28.232 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T05:21:28.341 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T05:21:28.454 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-10T05:21:28.523 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T05:21:28.637 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-10T05:21:28.646 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T05:21:28.854 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-10T05:21:28.865 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-10T05:21:28.883 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-10T05:21:29.046 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 31/162 2024-05-10T05:21:29.071 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-10T05:21:29.209 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 32/162 2024-05-10T05:21:29.218 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-10T05:21:29.274 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T05:21:29.430 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T05:21:29.461 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 33/162 2024-05-10T05:21:29.492 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-10T05:21:29.657 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T05:21:29.698 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T05:21:29.747 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-10T05:21:30.124 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T05:21:30.298 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T05:21:30.758 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T05:21:30.940 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T05:21:31.128 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T05:21:31.307 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T05:21:31.417 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T05:21:31.439 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T05:21:31.484 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-10T05:21:31.775 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-10T05:21:31.824 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T05:21:31.830 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-10T05:21:32.107 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-10T05:21:32.119 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T05:21:32.236 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T05:21:32.352 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-10T05:21:32.483 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-10T05:21:32.585 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 41/162 2024-05-10T05:21:32.762 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 42/162 2024-05-10T05:21:32.769 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-10T05:21:32.818 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-10T05:21:32.946 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T05:21:33.010 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T05:21:33.063 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-10T05:21:33.103 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-10T05:21:33.238 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T05:21:33.298 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-10T05:21:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T05:21:33.478 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-10T05:21:33.525 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-10T05:21:33.559 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-10T05:21:33.720 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-10T05:21:33.798 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-10T05:21:33.815 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-10T05:21:33.909 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T05:21:33.994 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 41/162 2024-05-10T05:21:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T05:21:34.061 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T05:21:34.139 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 42/162 2024-05-10T05:21:34.231 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T05:21:34.332 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T05:21:34.339 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T05:21:34.417 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T05:21:34.448 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T05:21:34.521 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-10T05:21:34.606 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T05:21:34.705 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T05:21:34.797 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T05:21:34.803 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-10T05:21:34.813 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T05:21:34.919 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-10T05:21:34.991 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-10T05:21:35.085 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-10T05:21:35.133 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-10T05:21:35.216 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-10T05:21:35.222 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T05:21:35.347 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T05:21:35.398 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T05:21:35.399 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-10T05:21:35.581 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T05:21:35.615 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-10T05:21:35.808 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T05:21:35.841 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-10T05:21:36.139 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-10T05:21:36.165 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T05:21:36.192 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T05:21:36.272 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T05:21:36.389 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-10T05:21:36.401 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-10T05:21:36.465 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-10T05:21:36.489 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T05:21:36.659 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-10T05:21:36.757 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-10T05:21:36.825 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-10T05:21:36.972 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-10T05:21:37.065 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-10T05:21:37.160 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T05:21:37.172 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-10T05:21:37.227 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-10T05:21:37.356 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T05:21:37.424 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-10T05:21:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-10T05:21:37.604 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-10T05:21:37.763 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-10T05:21:37.806 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-10T05:21:38.026 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/162 2024-05-10T05:21:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-10T05:21:38.212 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/162 2024-05-10T05:21:38.227 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T05:21:38.336 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-10T05:21:38.385 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/162 2024-05-10T05:21:38.522 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-10T05:21:38.531 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-10T05:21:38.600 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/162 2024-05-10T05:21:38.715 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-10T05:21:38.776 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/162 2024-05-10T05:21:38.782 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-10T05:21:39.026 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-10T05:21:39.092 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/162 2024-05-10T05:21:39.099 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-10T05:21:39.205 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 31/162 2024-05-10T05:21:39.334 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/162 2024-05-10T05:21:39.368 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-10T05:21:39.368 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 32/162 2024-05-10T05:21:39.572 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-10T05:21:39.576 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/162 2024-05-10T05:21:39.659 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 33/162 2024-05-10T05:21:39.725 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/162 2024-05-10T05:21:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-10T05:21:39.881 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T05:21:39.965 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/162 2024-05-10T05:21:40.048 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-10T05:21:40.240 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-16.8-9.el8.noarch 70/162 2024-05-10T05:21:40.348 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/162 2024-05-10T05:21:40.462 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-10T05:21:40.541 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-10T05:21:40.632 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 67/162 2024-05-10T05:21:40.854 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-05-10T05:21:40.936 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/162 2024-05-10T05:21:41.129 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 69/162 2024-05-10T05:21:41.137 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-05-10T05:21:41.311 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-05-10T05:21:41.351 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 70/162 2024-05-10T05:21:41.426 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-05-10T05:21:41.574 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 71/162 2024-05-10T05:21:41.594 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-05-10T05:21:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-05-10T05:21:41.833 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/162 2024-05-10T05:21:41.963 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-05-10T05:21:41.985 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T05:21:42.063 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/162 2024-05-10T05:21:42.137 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-10T05:21:42.307 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/162 2024-05-10T05:21:42.317 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-10T05:21:42.371 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-10T05:21:42.484 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/162 2024-05-10T05:21:42.650 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-10T05:21:42.789 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/162 2024-05-10T05:21:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-10T05:21:42.947 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-10T05:21:43.095 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-10T05:21:43.124 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-10T05:21:43.281 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-10T05:21:43.291 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-10T05:21:43.482 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-10T05:21:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-10T05:21:43.558 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-10T05:21:43.691 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-10T05:21:43.811 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-10T05:21:43.833 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-10T05:21:43.961 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-10T05:21:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-10T05:21:44.105 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-10T05:21:44.164 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-10T05:21:44.173 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-10T05:21:44.300 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-10T05:21:44.349 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-10T05:21:44.355 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-10T05:21:44.531 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-10T05:21:44.577 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-10T05:21:44.592 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-10T05:21:44.732 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-10T05:21:44.787 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 41/162 2024-05-10T05:21:44.816 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-10T05:21:44.917 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-10T05:21:44.989 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 42/162 2024-05-10T05:21:45.024 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-10T05:21:45.165 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-10T05:21:45.168 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-10T05:21:45.230 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T05:21:45.330 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T05:21:45.391 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-10T05:21:45.393 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-10T05:21:45.546 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T05:21:45.593 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-10T05:21:45.615 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-10T05:21:45.640 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T05:21:45.760 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-10T05:21:45.868 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-10T05:21:45.881 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-10T05:21:45.982 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-10T05:21:46.117 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-10T05:21:46.219 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-10T05:21:46.371 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T05:21:46.385 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-10T05:21:46.653 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-10T05:21:46.940 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-10T05:21:46.976 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T05:21:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-10T05:21:47.355 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T05:21:47.400 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T05:21:47.476 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-10T05:21:47.571 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-10T05:21:47.702 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-10T05:21:47.786 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-10T05:21:47.800 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-10T05:21:47.989 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-10T05:21:48.102 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-10T05:21:48.115 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-10T05:21:48.265 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-10T05:21:48.326 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-10T05:21:48.361 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-10T05:21:48.454 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-10T05:21:48.530 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-10T05:21:48.636 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-10T05:21:48.729 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-10T05:21:48.896 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-10T05:21:48.912 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-10T05:21:49.090 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-10T05:21:49.111 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-10T05:21:49.282 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-10T05:21:49.301 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/162 2024-05-10T05:21:49.435 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T05:21:49.499 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/162 2024-05-10T05:21:49.512 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T05:21:49.702 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 104/162 2024-05-10T05:21:49.846 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/162 2024-05-10T05:21:50.026 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/162 2024-05-10T05:21:50.051 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-10T05:21:50.285 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/162 2024-05-10T05:21:50.384 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T05:21:50.444 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T05:21:50.600 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/162 2024-05-10T05:21:50.629 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 104/162 2024-05-10T05:21:50.915 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/162 2024-05-10T05:21:51.185 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/162 2024-05-10T05:21:51.356 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/162 2024-05-10T05:21:51.564 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/162 2024-05-10T05:21:52.001 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/162 2024-05-10T05:21:52.343 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 67/162 2024-05-10T05:21:52.637 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/162 2024-05-10T05:21:52.880 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 69/162 2024-05-10T05:21:53.042 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 70/162 2024-05-10T05:21:53.323 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 71/162 2024-05-10T05:21:53.563 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/162 2024-05-10T05:21:53.750 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T05:21:53.814 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/162 2024-05-10T05:21:54.027 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T05:21:54.031 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/162 2024-05-10T05:21:54.264 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/162 2024-05-10T05:21:54.543 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/162 2024-05-10T05:21:54.688 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T05:21:54.710 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T05:21:54.764 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-10T05:21:54.929 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-10T05:21:54.969 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T05:21:55.120 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-10T05:21:55.462 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-10T05:21:55.607 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T05:21:55.711 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-10T05:21:55.867 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-10T05:21:55.979 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T05:21:56.032 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-10T05:21:56.069 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T05:21:56.234 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T05:21:56.275 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-10T05:21:56.510 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-10T05:21:56.783 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-10T05:21:56.863 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T05:21:56.952 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T05:21:56.957 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-10T05:21:57.077 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T05:21:57.160 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-10T05:21:57.402 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-10T05:21:57.659 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-10T05:21:57.826 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-10T05:21:58.575 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-10T05:21:58.795 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-10T05:21:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-10T05:21:59.227 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-10T05:22:00.545 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-10T05:22:00.898 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-10T05:22:01.132 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-10T05:22:01.518 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-10T05:22:01.818 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-10T05:22:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-10T05:22:03.064 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-10T05:22:03.276 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T05:22:03.368 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T05:22:04.020 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 104/162 2024-05-10T05:22:09.284 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T05:22:09.727 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T05:22:10.458 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T05:22:10.796 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T05:22:10.931 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T05:22:11.127 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T05:22:15.823 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T05:22:15.823 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-05-10T05:22:15.823 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-05-10T05:22:15.823 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /dev 2024-05-10T05:22:15.823 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /run 2024-05-10T05:22:15.823 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-05-10T05:22:15.823 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-05-10T05:22:15.823 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-05-10T05:22:15.823 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-05-10T05:22:15.824 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /dev 2024-05-10T05:22:15.824 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:22:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T05:22:16.757 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T05:22:16.757 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-05-10T05:22:16.757 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-05-10T05:22:16.758 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2024-05-10T05:22:16.758 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /run 2024-05-10T05:22:16.758 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-05-10T05:22:16.758 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-05-10T05:22:16.758 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-05-10T05:22:16.758 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-05-10T05:22:16.758 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2024-05-10T05:22:16.758 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:22:17.141 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T05:22:17.939 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T05:22:18.117 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T05:22:18.336 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T05:22:18.522 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T05:22:18.867 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T05:22:19.042 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T05:22:19.092 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T05:22:19.222 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T05:22:19.239 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T05:22:19.311 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T05:22:19.498 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T05:22:20.017 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T05:22:20.190 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T05:22:20.207 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T05:22:21.068 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T05:22:21.261 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T05:22:21.378 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 113/162 2024-05-10T05:22:21.564 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 114/162 2024-05-10T05:22:21.805 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T05:22:21.974 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T05:22:22.186 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T05:22:22.321 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 113/162 2024-05-10T05:22:22.517 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 114/162 2024-05-10T05:22:22.764 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T05:22:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T05:22:23.107 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T05:22:23.795 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T05:22:24.057 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T05:22:24.075 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T05:22:24.246 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T05:22:24.339 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T05:22:24.706 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 118/162 2024-05-10T05:22:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T05:22:24.904 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T05:22:25.026 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T05:22:25.096 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T05:22:25.108 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T05:22:25.188 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T05:22:25.282 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T05:22:25.657 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 118/162 2024-05-10T05:22:25.849 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T05:22:25.888 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T05:22:26.064 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T05:22:26.075 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T05:22:26.615 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el8.x86_64 121/162 2024-05-10T05:22:26.832 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T05:22:26.867 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T05:22:27.079 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T05:22:27.123 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T05:22:27.279 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T05:22:27.632 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T05:22:27.692 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el8.x86_64 121/162 2024-05-10T05:22:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T05:22:27.918 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T05:22:27.950 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T05:22:28.114 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T05:22:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T05:22:28.347 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T05:22:28.699 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T05:22:28.872 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T05:22:32.631 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T05:22:32.631 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-05-10T05:22:32.632 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-05-10T05:22:32.632 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /dev 2024-05-10T05:22:32.632 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /run 2024-05-10T05:22:32.632 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-05-10T05:22:32.632 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-05-10T05:22:32.632 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-05-10T05:22:32.632 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-05-10T05:22:32.632 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /dev 2024-05-10T05:22:32.632 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:22:32.748 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T05:22:33.047 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T05:22:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 126/162 2024-05-10T05:22:33.538 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-10T05:22:33.753 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-10T05:22:33.767 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T05:22:34.444 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-10T05:22:34.750 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 126/162 2024-05-10T05:22:34.765 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-10T05:22:34.919 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T05:22:35.051 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-10T05:22:35.070 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-10T05:22:35.117 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T05:22:35.317 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T05:22:35.330 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-10T05:22:35.393 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T05:22:35.538 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-10T05:22:35.555 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T05:22:35.690 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 129/162 2024-05-10T05:22:35.770 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-10T05:22:35.971 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 134/162 2024-05-10T05:22:36.014 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-10T05:22:36.144 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T05:22:36.254 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T05:22:36.267 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T05:22:36.339 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T05:22:36.381 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T05:22:36.456 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-10T05:22:36.663 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-10T05:22:36.822 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 134/162 2024-05-10T05:22:38.237 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T05:22:38.553 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T05:22:38.592 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T05:22:39.042 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-10T05:22:39.100 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T05:22:39.328 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T05:22:39.404 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T05:22:39.456 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T05:22:39.491 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-10T05:22:39.495 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T05:22:39.605 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 113/162 2024-05-10T05:22:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-10T05:22:39.823 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 114/162 2024-05-10T05:22:39.849 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-10T05:22:39.969 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 139/162 2024-05-10T05:22:40.046 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T05:22:40.212 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 140/162 2024-05-10T05:22:40.277 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-10T05:22:40.522 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 141/162 2024-05-10T05:22:40.550 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-10T05:22:40.728 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 139/162 2024-05-10T05:22:40.812 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-10T05:22:40.970 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 140/162 2024-05-10T05:22:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-10T05:22:40.988 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-10T05:22:41.214 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-10T05:22:41.240 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 141/162 2024-05-10T05:22:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T05:22:41.460 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-10T05:22:41.499 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T05:22:41.522 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-10T05:22:41.540 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-10T05:22:41.769 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T05:22:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-10T05:22:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-10T05:22:41.849 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T05:22:42.001 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-10T05:22:42.080 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T05:22:42.091 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T05:22:42.212 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-10T05:22:42.294 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T05:22:42.304 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T05:22:42.389 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-10T05:22:42.389 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T05:22:42.455 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-10T05:22:42.609 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T05:22:42.659 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-10T05:22:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T05:22:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T05:22:42.787 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T05:22:42.838 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-10T05:22:42.921 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T05:22:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T05:22:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T05:22:42.990 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-10T05:22:42.990 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T05:22:43.216 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T05:22:43.273 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T05:22:43.273 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T05:22:43.318 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 118/162 2024-05-10T05:22:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-10T05:22:43.473 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T05:22:43.473 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T05:22:43.516 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T05:22:43.586 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T05:22:43.645 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-10T05:22:43.726 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T05:22:43.762 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T05:22:43.792 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T05:22:43.920 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-10T05:22:43.920 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T05:22:43.946 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-10T05:22:44.138 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-10T05:22:44.162 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T05:22:44.163 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T05:22:44.371 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-10T05:22:44.371 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T05:22:44.421 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T05:22:44.421 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T05:22:44.630 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T05:22:44.630 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T05:22:44.662 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T05:22:44.662 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T05:22:44.839 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T05:22:44.839 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T05:22:44.970 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T05:22:44.970 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T05:22:45.080 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T05:22:45.080 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T05:22:45.195 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T05:22:45.313 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T05:22:45.313 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T05:22:45.373 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el8.x86_64 121/162 2024-05-10T05:22:45.553 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-10T05:22:45.554 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T05:22:45.563 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T05:22:45.616 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T05:22:45.678 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T05:22:45.696 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-10T05:22:45.696 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T05:22:45.846 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T05:22:45.918 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T05:22:46.121 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T05:22:46.509 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T05:22:46.509 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el8.x86_64 1/162 2024-05-10T05:22:46.509 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2/162 2024-05-10T05:22:46.509 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 3/162 2024-05-10T05:22:46.509 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 4/162 2024-05-10T05:22:46.509 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/162 2024-05-10T05:22:46.509 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 6/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 7/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 8/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 13/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 14/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 15/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 16/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 17/162 2024-05-10T05:22:46.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 18/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 21/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 22/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 23/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 24/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 25/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 26/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 27/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 28/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 30/162 2024-05-10T05:22:46.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 31/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 34/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 36/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el8.noarch 37/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 38/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-10T05:22:46.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-10T05:22:46.513 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-10T05:22:46.514 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-05-10T05:22:46.515 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-05-10T05:22:46.516 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 103/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-10T05:22:46.517 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-10T05:22:46.518 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 133/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-10T05:22:46.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-10T05:22:46.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-10T05:22:46.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-10T05:22:46.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-10T05:22:46.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-10T05:22:46.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-10T05:22:46.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-10T05:22:46.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-10T05:22:46.531 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T05:22:46.592 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T05:22:46.592 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el8.x86_64 1/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 3/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 4/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 6/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 7/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 8/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 13/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 14/162 2024-05-10T05:22:46.593 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 15/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 16/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 17/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 18/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 21/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 22/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 23/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 24/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 25/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 26/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 27/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 28/162 2024-05-10T05:22:46.594 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/162 2024-05-10T05:22:46.595 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 30/162 2024-05-10T05:22:46.606 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 31/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 34/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 36/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el8.noarch 37/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 38/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-10T05:22:46.607 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-10T05:22:46.608 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-10T05:22:46.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/162 2024-05-10T05:22:46.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 93/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 94/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 95/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 96/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 97/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 98/162 2024-05-10T05:22:46.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 99/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-10T05:22:46.612 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-10T05:22:46.613 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 133/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-10T05:22:46.614 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-10T05:22:46.615 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-10T05:22:46.616 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-10T05:22:46.616 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-10T05:22:46.616 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-10T05:22:46.616 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-10T05:22:46.616 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-10T05:22:46.616 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-10T05:22:46.720 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T05:22:52.048 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T05:22:53.398 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 126/162 2024-05-10T05:22:53.723 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-10T05:22:53.991 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-10T05:22:54.481 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 129/162 2024-05-10T05:22:54.834 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-10T05:22:55.035 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T05:22:55.066 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T05:22:55.315 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-10T05:22:55.545 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-10T05:22:55.803 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 134/162 2024-05-10T05:22:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T05:22:59.611 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T05:22:59.698 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T05:23:00.100 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-10T05:23:00.229 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-10T05:23:00.229 INFO:teuthology.orchestra.run.smithi157.stdout:Installed products updated. 2024-05-10T05:23:00.499 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-10T05:23:00.784 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-10T05:23:01.095 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:23:01.095 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-05-10T05:23:01.095 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-10T05:23:01.095 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:01.096 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.1-8.el8.noarch 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.0.4-10.el8.x86_64 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.097 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:01.098 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.1-9.el8.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.6-7.el8.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.099 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.100 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-10T05:23:01.100 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-10T05:23:01.100 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-10T05:23:01.100 INFO:teuthology.orchestra.run.smithi157.stdout: libpq-13.11-1.el8.x86_64 2024-05-10T05:23:01.100 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-10T05:23:01.100 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-10T05:23:01.100 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.100 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.100 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-10T05:23:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-10T05:23:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-10T05:23:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-10T05:23:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-10T05:23:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T05:23:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T05:23:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T05:23:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-10T05:23:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-10T05:23:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-10T05:23:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-10T05:23:01.103 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-10T05:23:01.103 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.103 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.103 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.103 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-10T05:23:01.103 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-10T05:23:01.103 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-10T05:23:01.103 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-10T05:23:01.103 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-10T05:23:01.103 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-10T05:23:01.104 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-2.6-1.el8.noarch 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.105 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-10T05:23:01.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-10T05:23:01.107 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-10T05:23:01.107 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.107 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.107 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:01.107 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-10T05:23:01.107 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-10T05:23:01.107 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-10T05:23:01.107 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-10T05:23:01.107 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-10T05:23:01.108 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-10T05:23:01.108 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:23:01.108 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-05-10T05:23:01.328 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 139/162 2024-05-10T05:23:01.350 DEBUG:teuthology.parallel:result is None 2024-05-10T05:23:01.595 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 140/162 2024-05-10T05:23:01.914 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 141/162 2024-05-10T05:23:02.264 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-10T05:23:02.475 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T05:23:02.599 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T05:23:02.712 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-10T05:23:02.713 INFO:teuthology.orchestra.run.smithi142.stdout:Installed products updated. 2024-05-10T05:23:02.809 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T05:23:02.841 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T05:23:03.034 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T05:23:03.060 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T05:23:03.178 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-10T05:23:03.360 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-10T05:23:03.360 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:23:03.360 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-05-10T05:23:03.360 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:03.361 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.1-8.el8.noarch 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.0.4-10.el8.x86_64 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:03.362 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.363 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.1-9.el8.x86_64 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: jq-1.6-7.el8.x86_64 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-10T05:23:03.364 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: libpq-13.11-1.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-10T05:23:03.365 INFO:teuthology.orchestra.run.smithi142.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-10T05:23:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-10T05:23:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-10T05:23:03.368 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-10T05:23:03.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-10T05:23:03.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-10T05:23:03.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-10T05:23:03.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-10T05:23:03.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-10T05:23:03.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-10T05:23:03.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-2.6-1.el8.noarch 2024-05-10T05:23:03.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-10T05:23:03.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-10T05:23:03.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-10T05:23:03.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-10T05:23:03.371 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-10T05:23:03.372 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-10T05:23:03.372 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-10T05:23:03.372 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-10T05:23:03.372 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T05:23:03.372 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-05-10T05:23:03.589 DEBUG:teuthology.parallel:result is None 2024-05-10T05:23:03.620 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-10T05:23:03.798 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-10T05:23:03.798 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T05:23:04.122 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T05:23:04.123 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T05:23:04.399 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T05:23:04.399 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T05:23:04.702 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T05:23:04.911 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-10T05:23:05.129 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-10T05:23:05.369 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-10T05:23:05.370 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T05:23:05.662 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T05:23:05.662 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T05:23:05.912 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T05:23:05.912 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T05:23:07.304 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T05:23:07.304 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T05:23:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T05:23:07.562 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T05:23:07.845 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T05:23:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-10T05:23:08.012 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T05:23:08.612 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T05:23:08.612 INFO:teuthology.orchestra.run.smithi131.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-10T05:23:08.612 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:23:09.313 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el8.x86_64 1/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 3/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 4/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 6/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 7/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 8/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T05:23:09.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 13/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 14/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 15/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 16/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 17/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 18/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 21/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 22/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 23/162 2024-05-10T05:23:09.315 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 24/162 2024-05-10T05:23:09.316 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 25/162 2024-05-10T05:23:09.316 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 26/162 2024-05-10T05:23:09.316 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 27/162 2024-05-10T05:23:09.316 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 28/162 2024-05-10T05:23:09.316 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/162 2024-05-10T05:23:09.316 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 30/162 2024-05-10T05:23:09.329 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 31/162 2024-05-10T05:23:09.329 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/162 2024-05-10T05:23:09.329 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/162 2024-05-10T05:23:09.329 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 34/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 36/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el8.noarch 37/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 38/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-10T05:23:09.330 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-10T05:23:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-10T05:23:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-10T05:23:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-10T05:23:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-10T05:23:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-10T05:23:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-10T05:23:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-10T05:23:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-10T05:23:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-10T05:23:09.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-10T05:23:09.332 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/162 2024-05-10T05:23:09.333 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-05-10T05:23:09.334 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 93/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 94/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 95/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 96/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 97/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 98/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 99/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-10T05:23:09.335 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-10T05:23:09.336 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T05:23:09.337 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 133/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-10T05:23:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-10T05:23:09.339 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-10T05:23:09.340 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-10T05:23:09.340 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-10T05:23:09.340 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-10T05:23:09.340 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-10T05:23:24.721 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-10T05:23:24.721 INFO:teuthology.orchestra.run.smithi131.stdout:Installed products updated. 2024-05-10T05:23:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.1-8.el8.noarch 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.0.4-10.el8.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.1-9.el8.x86_64 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-10T05:23:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: jq-1.6-7.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: libpq-13.11-1.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-10T05:23:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-10T05:23:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-10T05:23:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-10T05:23:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-10T05:23:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-10T05:23:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-10T05:23:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-10T05:23:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-10T05:23:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-10T05:23:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-10T05:23:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-2.6-1.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-10T05:23:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-10T05:23:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-10T05:23:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T05:23:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-10T05:23:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-10T05:23:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-10T05:23:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-10T05:23:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-10T05:23:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-10T05:23:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:23:25.387 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-10T05:23:25.749 DEBUG:teuthology.parallel:result is None 2024-05-10T05:23:25.750 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:23:25.976 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T05:23:26.029 INFO:teuthology.orchestra.run.smithi131.stdout:18.2.2-1330.g50174015.el8 2024-05-10T05:23:26.030 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el8 2024-05-10T05:23:26.030 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T05:23:26.032 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:23:26.252 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T05:23:26.281 INFO:teuthology.orchestra.run.smithi142.stdout:18.2.2-1330.g50174015.el8 2024-05-10T05:23:26.282 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el8 2024-05-10T05:23:26.282 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T05:23:26.284 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:23:26.504 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T05:23:26.533 INFO:teuthology.orchestra.run.smithi157.stdout:18.2.2-1330.g50174015.el8 2024-05-10T05:23:26.533 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el8 2024-05-10T05:23:26.533 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T05:23:26.535 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-10T05:23:26.535 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:23:26.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T05:23:26.623 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:23:26.623 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T05:23:26.672 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:23:26.672 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T05:23:26.723 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-10T05:23:26.723 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:23:26.724 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T05:23:26.788 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T05:23:26.892 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:23:26.892 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T05:23:26.926 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T05:23:26.997 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:23:26.997 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T05:23:27.031 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T05:23:27.106 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-10T05:23:27.106 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:23:27.106 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T05:23:27.171 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T05:23:27.276 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:23:27.276 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T05:23:27.311 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T05:23:27.383 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:23:27.383 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T05:23:27.416 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T05:23:27.487 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-10T05:23:27.487 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:23:27.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T05:23:27.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T05:23:27.658 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:23:27.658 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T05:23:27.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T05:23:27.760 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:23:27.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T05:23:27.793 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T05:23:27.864 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-10T05:23:27.973 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-10T05:23:27.974 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:23:27.974 INFO:tasks.cephadm:Cluster fsid is 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:23:27.974 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-10T05:23:27.975 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.131', 'mon.b': '172.21.15.142', 'mon.c': '172.21.15.157'} 2024-05-10T05:23:27.975 INFO:tasks.cephadm:First mon is mon.a on smithi131 2024-05-10T05:23:27.975 INFO:tasks.cephadm:First mgr is x 2024-05-10T05:23:27.975 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-10T05:23:27.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2024-05-10T05:23:28.042 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-05-10T05:23:28.075 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-05-10T05:23:28.108 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-10T05:23:28.109 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:23:28.250 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79270/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1330-g50174015', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1330.g50174015'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-10 02:06:05.327930', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240509.171335-reef-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-10T05:23:28.386 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240509.171335-reef-debug, sha1 50174015a7e79f951b988949abe236791f52d4da from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:23:28.387 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm 2024-05-10T05:23:28.388 DEBUG:teuthology.orchestra.run.smithi131:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T05:23:28.955 INFO:teuthology.orchestra.run.smithi131.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 10 05:23 /home/ubuntu/cephtest/cephadm 2024-05-10T05:23:28.956 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T05:23:29.328 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 10 05:23 /home/ubuntu/cephtest/cephadm 2024-05-10T05:23:29.328 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T05:23:29.834 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 10 05:23 /home/ubuntu/cephtest/cephadm 2024-05-10T05:23:29.835 DEBUG:teuthology.orchestra.run.smithi131:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T05:23:29.884 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T05:23:29.907 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T05:23:29.960 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da on all hosts... 2024-05-10T05:23:29.960 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da pull 2024-05-10T05:23:29.965 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da pull 2024-05-10T05:23:29.967 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da pull 2024-05-10T05:23:30.909 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T05:23:30.930 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T05:23:31.446 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T05:24:30.215 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:24:30.215 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T05:24:30.215 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b", 2024-05-10T05:24:30.215 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2024-05-10T05:24:30.216 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:042b1f8e1a7496c1ea8ed86a601031b3e73743a9905239589edc685eef0b229c" 2024-05-10T05:24:30.216 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-10T05:24:30.216 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:24:33.820 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-10T05:24:33.821 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T05:24:33.821 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b", 2024-05-10T05:24:33.821 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-05-10T05:24:33.821 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:042b1f8e1a7496c1ea8ed86a601031b3e73743a9905239589edc685eef0b229c" 2024-05-10T05:24:33.821 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-05-10T05:24:33.821 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-10T05:25:10.403 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-10T05:25:10.403 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T05:25:10.403 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b", 2024-05-10T05:25:10.403 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-05-10T05:25:10.403 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:042b1f8e1a7496c1ea8ed86a601031b3e73743a9905239589edc685eef0b229c" 2024-05-10T05:25:10.403 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-05-10T05:25:10.403 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-10T05:25:10.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2024-05-10T05:25:10.500 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-05-10T05:25:10.538 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-05-10T05:25:10.569 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2024-05-10T05:25:10.636 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-05-10T05:25:10.667 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-05-10T05:25:10.694 INFO:tasks.cephadm:Writing seed config... 2024-05-10T05:25:10.696 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-10T05:25:10.696 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-10T05:25:10.696 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-10T05:25:10.696 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-10T05:25:10.697 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-10T05:25:10.697 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-10T05:25:10.697 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-10T05:25:10.697 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-10T05:25:10.697 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-10T05:25:10.697 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-10T05:25:10.697 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-10T05:25:10.697 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-10T05:25:10.698 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-10T05:25:10.698 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-10T05:25:10.698 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-10T05:25:10.698 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-10T05:25:10.698 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-10T05:25:10.698 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-10T05:25:10.698 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-10T05:25:10.698 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-10T05:25:10.699 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-10T05:25:10.699 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-10T05:25:10.699 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-10T05:25:10.699 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-10T05:25:10.699 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-10T05:25:10.699 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-10T05:25:10.699 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-10T05:25:10.700 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-10T05:25:10.700 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-10T05:25:10.700 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-10T05:25:10.700 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-10T05:25:10.700 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-10T05:25:10.700 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-10T05:25:10.700 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-10T05:25:10.700 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-10T05:25:10.700 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-10T05:25:10.701 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-10T05:25:10.701 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-10T05:25:10.701 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-10T05:25:10.701 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-10T05:25:10.701 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-10T05:25:10.701 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-10T05:25:10.702 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-10T05:25:10.703 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:25:10.703 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-10T05:25:10.745 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 6e8de474-0e8d-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-10T05:25:10.745 DEBUG:teuthology.orchestra.run.smithi131:mon.a> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.a.service 2024-05-10T05:25:10.790 DEBUG:teuthology.orchestra.run.smithi131:mgr.x> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.x.service 2024-05-10T05:25:10.836 INFO:tasks.cephadm:Bootstrapping... 2024-05-10T05:25:10.836 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da -v bootstrap --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.131 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-10T05:25:10.854 INFO:journalctl@ceph.mon.a.smithi131.stdout:-- Logs begin at Fri 2024-05-10 05:06:53 UTC. -- 2024-05-10T05:25:10.900 INFO:journalctl@ceph.mgr.x.smithi131.stdout:-- Logs begin at Fri 2024-05-10 05:06:53 UTC. -- 2024-05-10T05:25:11.053 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-05-10T05:25:11.053 INFO:teuthology.orchestra.run.smithi131.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da', '-v', 'bootstrap', '--fsid', '6e8de474-0e8d-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.131', '--skip-admin-label'] 2024-05-10T05:25:11.102 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.6.1 2024-05-10T05:25:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-10T05:25:11.102 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying podman|docker is present... 2024-05-10T05:25:11.146 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.6.1 2024-05-10T05:25:11.146 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying lvm2 is present... 2024-05-10T05:25:11.146 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying time synchronization is in place... 2024-05-10T05:25:11.160 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T05:25:11.160 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T05:25:11.172 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T05:25:11.172 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-10T05:25:11.184 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-05-10T05:25:11.197 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-05-10T05:25:11.197 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-05-10T05:25:11.197 INFO:teuthology.orchestra.run.smithi131.stdout:Repeating the final host check... 2024-05-10T05:25:11.240 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.6.1 2024-05-10T05:25:11.240 INFO:teuthology.orchestra.run.smithi131.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-10T05:25:11.241 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl is present 2024-05-10T05:25:11.241 INFO:teuthology.orchestra.run.smithi131.stdout:lvcreate is present 2024-05-10T05:25:11.253 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T05:25:11.253 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T05:25:11.265 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T05:25:11.265 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-10T05:25:11.277 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-05-10T05:25:11.290 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-05-10T05:25:11.290 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-05-10T05:25:11.290 INFO:teuthology.orchestra.run.smithi131.stdout:Host looks OK 2024-05-10T05:25:11.290 INFO:teuthology.orchestra.run.smithi131.stdout:Cluster fsid: 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:11.290 INFO:teuthology.orchestra.run.smithi131.stdout:Acquiring lock 140589711370728 on /run/cephadm/6e8de474-0e8d-11ef-bc98-c7b262605968.lock 2024-05-10T05:25:11.290 INFO:teuthology.orchestra.run.smithi131.stdout:Lock 140589711370728 acquired on /run/cephadm/6e8de474-0e8d-11ef-bc98-c7b262605968.lock 2024-05-10T05:25:11.291 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying IP 172.21.15.131 port 3300 ... 2024-05-10T05:25:11.291 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying IP 172.21.15.131 port 6789 ... 2024-05-10T05:25:11.291 INFO:teuthology.orchestra.run.smithi131.stdout:Base mon IP(s) is [172.21.15.131:3300, 172.21.15.131:6789], mon addrv is [v2:172.21.15.131:3300,v1:172.21.15.131:6789] 2024-05-10T05:25:11.299 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2024-05-10T05:25:11.299 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.131 metric 100 2024-05-10T05:25:11.306 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-10T05:25:11.306 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-10T05:25:11.306 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1772sec hoplimit 64 pref medium 2024-05-10T05:25:11.312 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-10T05:25:11.312 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-10T05:25:11.312 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T05:25:11.312 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-10T05:25:11.312 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-10T05:25:11.313 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T05:25:11.313 INFO:teuthology.orchestra.run.smithi131.stdout:Mon IP `172.21.15.131` is in CIDR network `172.21.0.0/20` 2024-05-10T05:25:11.313 INFO:teuthology.orchestra.run.smithi131.stdout:Mon IP `172.21.15.131` is in CIDR network `172.21.0.0/20` 2024-05-10T05:25:11.313 INFO:teuthology.orchestra.run.smithi131.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-10T05:25:11.314 INFO:teuthology.orchestra.run.smithi131.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-10T05:25:11.315 INFO:teuthology.orchestra.run.smithi131.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T05:25:11.756 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b 2024-05-10T05:25:11.756 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T05:25:11.756 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Getting image source signatures 2024-05-10T05:25:11.756 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:5889b1d3c79fb904da685578f1acb24d7c4091d50b19c3ba92279423a786328a 2024-05-10T05:25:11.756 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-10T05:25:11.756 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying config sha256:0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b 2024-05-10T05:25:11.757 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-10T05:25:12.569 INFO:teuthology.orchestra.run.smithi131.stdout:ceph: stdout ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable) 2024-05-10T05:25:12.569 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph version: ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable) 2024-05-10T05:25:12.569 INFO:teuthology.orchestra.run.smithi131.stdout:Extracting ceph user uid/gid from container image... 2024-05-10T05:25:13.363 INFO:teuthology.orchestra.run.smithi131.stdout:stat: stdout 167 167 2024-05-10T05:25:13.363 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial keys... 2024-05-10T05:25:14.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQC5rz1msi63KhAAkKfqCI6OmNVNaphtsCllCA== 2024-05-10T05:25:14.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQC6rz1mSArsGhAAa/fy1lg4lvzFe9Qd1Go/ig== 2024-05-10T05:25:15.506 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQC7rz1msP86ChAAwTyZnaSJ4XvDn30TMffWiA== 2024-05-10T05:25:15.507 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial monmap... 2024-05-10T05:25:16.227 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T05:25:16.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-10T05:25:16.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:16.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T05:25:16.228 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool for a [v2:172.21.15.131:3300,v1:172.21.15.131:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T05:25:16.228 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = pacific 2024-05-10T05:25:16.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: set fsid to 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:16.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T05:25:16.228 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:25:16.228 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mon... 2024-05-10T05:25:16.960 INFO:teuthology.orchestra.run.smithi131.stdout:create mon.a on 2024-05-10T05:25:17.210 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-10T05:25:17.407 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-10T05:25:17.641 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-6e8de474-0e8d-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-6e8de474-0e8d-11ef-bc98-c7b262605968.target. 2024-05-10T05:25:17.641 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-6e8de474-0e8d-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-6e8de474-0e8d-11ef-bc98-c7b262605968.target. 2024-05-10T05:25:17.979 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.a 2024-05-10T05:25:17.979 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-10T05:25:18.176 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6e8de474-0e8d-11ef-bc98-c7b262605968.target.wants/ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@.service. 2024-05-10T05:25:18.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:18 smithi131 systemd[1]: Starting Ceph mon.a for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:25:18.727 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T05:25:18.727 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-05-10T05:25:18.743 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T05:25:18.743 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-10T05:25:18.743 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-05-10T05:25:18.743 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T05:25:18.744 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon to start... 2024-05-10T05:25:18.744 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon... 2024-05-10T05:25:18.818 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:18 smithi131 bash[36729]: c2203778b4d231edb8601f437ba6cde5b95080c8c8a9e7e6d5aa94f8178e3d18 2024-05-10T05:25:18.818 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:18 smithi131 systemd[1]: Started Ceph mon.a for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:25:18.818 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:18 smithi131 ceph-mon[36767]: mkfs 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:18.955 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:18 smithi131 ceph-mon[36767]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T05:25:19.713 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout cluster: 2024-05-10T05:25:19.719 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout id: 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:19.719 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-10T05:25:19.719 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:19.719 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout services: 2024-05-10T05:25:19.719 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.536232s) 2024-05-10T05:25:19.719 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-10T05:25:19.719 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout data: 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pgs: 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.257+0000 7fab58000700 1 Processor -- start 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.258+0000 7fab58000700 1 -- start start 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.258+0000 7fab58000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab50107630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.258+0000 7fab58000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab50107c20 con 0x7fab50107220 2024-05-10T05:25:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.259+0000 7fab55600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab50107630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:19.721 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.259+0000 7fab55600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab50107630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41214/0 (socket says 172.21.15.131:41214) 2024-05-10T05:25:19.721 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.259+0000 7fab55600700 1 -- 172.21.15.131:0/1505330968 learned_addr learned my addr 172.21.15.131:0/1505330968 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:19.721 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.260+0000 7fab55600700 1 -- 172.21.15.131:0/1505330968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab501083f0 con 0x7fab50107220 2024-05-10T05:25:19.721 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.260+0000 7fab55600700 1 --2- 172.21.15.131:0/1505330968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab50107630 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fab440098a0 tx=0x7fab44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcf74335271b568 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:19.721 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.260+0000 7fab4fe00700 1 -- 172.21.15.131:0/1505330968 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab44004160 con 0x7fab50107220 2024-05-10T05:25:19.721 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.260+0000 7fab4fe00700 1 -- 172.21.15.131:0/1505330968 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab440042c0 con 0x7fab50107220 2024-05-10T05:25:19.721 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.260+0000 7fab4fe00700 1 -- 172.21.15.131:0/1505330968 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab440045a0 con 0x7fab50107220 2024-05-10T05:25:19.721 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.261+0000 7fab58000700 1 -- 172.21.15.131:0/1505330968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 msgr2=0x7fab50107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:19.721 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.261+0000 7fab58000700 1 --2- 172.21.15.131:0/1505330968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab50107630 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fab440098a0 tx=0x7fab44009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:19.721 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.261+0000 7fab58000700 1 -- 172.21.15.131:0/1505330968 shutdown_connections 2024-05-10T05:25:19.722 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.261+0000 7fab58000700 1 --2- 172.21.15.131:0/1505330968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab50107630 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:19.722 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.261+0000 7fab58000700 1 -- 172.21.15.131:0/1505330968 >> 172.21.15.131:0/1505330968 conn(0x7fab50102840 msgr2=0x7fab50104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:19.722 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.261+0000 7fab58000700 1 -- 172.21.15.131:0/1505330968 shutdown_connections 2024-05-10T05:25:19.722 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.261+0000 7fab58000700 1 -- 172.21.15.131:0/1505330968 wait complete. 2024-05-10T05:25:19.722 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.262+0000 7fab58000700 1 Processor -- start 2024-05-10T05:25:19.722 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.262+0000 7fab58000700 1 -- start start 2024-05-10T05:25:19.722 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.263+0000 7fab58000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab5019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:19.723 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.263+0000 7fab58000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5019d2a0 con 0x7fab50107220 2024-05-10T05:25:19.723 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.263+0000 7fab55600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab5019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:19.723 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.263+0000 7fab55600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab5019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41226/0 (socket says 172.21.15.131:41226) 2024-05-10T05:25:19.723 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.263+0000 7fab55600700 1 -- 172.21.15.131:0/3906241163 learned_addr learned my addr 172.21.15.131:0/3906241163 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:19.723 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.263+0000 7fab55600700 1 -- 172.21.15.131:0/3906241163 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab44009550 con 0x7fab50107220 2024-05-10T05:25:19.723 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.263+0000 7fab55600700 1 --2- 172.21.15.131:0/3906241163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab5019cd60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab44004000 tx=0x7fab44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:19.723 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.264+0000 7fab4e000700 1 -- 172.21.15.131:0/3906241163 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab44011630 con 0x7fab50107220 2024-05-10T05:25:19.723 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.264+0000 7fab4e000700 1 -- 172.21.15.131:0/3906241163 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab44011790 con 0x7fab50107220 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.264+0000 7fab58000700 1 -- 172.21.15.131:0/3906241163 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5019d4a0 con 0x7fab50107220 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.264+0000 7fab4e000700 1 -- 172.21.15.131:0/3906241163 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab44011a30 con 0x7fab50107220 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.264+0000 7fab58000700 1 -- 172.21.15.131:0/3906241163 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5019d940 con 0x7fab50107220 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.264+0000 7fab4e000700 1 -- 172.21.15.131:0/3906241163 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fab44022070 con 0x7fab50107220 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.264+0000 7fab4e000700 1 -- 172.21.15.131:0/3906241163 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fab4401c790 con 0x7fab50107220 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.265+0000 7fab58000700 1 -- 172.21.15.131:0/3906241163 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab50108760 con 0x7fab50107220 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.268+0000 7fab4e000700 1 -- 172.21.15.131:0/3906241163 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fab4401bcf0 con 0x7fab50107220 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.343+0000 7fab58000700 1 -- 172.21.15.131:0/3906241163 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fab501a0310 con 0x7fab50107220 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.343+0000 7fab4e000700 1 -- 172.21.15.131:0/3906241163 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fab4401b450 con 0x7fab50107220 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.344+0000 7fab58000700 1 -- 172.21.15.131:0/3906241163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 msgr2=0x7fab5019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:19.724 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.344+0000 7fab58000700 1 --2- 172.21.15.131:0/3906241163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab5019cd60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab44004000 tx=0x7fab44004750 comp rx=0 tx=0).stop 2024-05-10T05:25:19.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.344+0000 7fab58000700 1 -- 172.21.15.131:0/3906241163 shutdown_connections 2024-05-10T05:25:19.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.344+0000 7fab58000700 1 --2- 172.21.15.131:0/3906241163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab50107220 0x7fab5019cd60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:19.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.344+0000 7fab58000700 1 -- 172.21.15.131:0/3906241163 >> 172.21.15.131:0/3906241163 conn(0x7fab50102840 msgr2=0x7fab5007b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:19.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.344+0000 7fab58000700 1 -- 172.21.15.131:0/3906241163 shutdown_connections 2024-05-10T05:25:19.725 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:19.344+0000 7fab58000700 1 -- 172.21.15.131:0/3906241163 wait complete. 2024-05-10T05:25:19.725 INFO:teuthology.orchestra.run.smithi131.stdout:mon is available 2024-05-10T05:25:19.725 INFO:teuthology.orchestra.run.smithi131.stdout:Assimilating anything we can from ceph.conf... 2024-05-10T05:25:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:19 smithi131 ceph-mon[36767]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T05:25:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:19 smithi131 ceph-mon[36767]: monmap e1: 1 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:25:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:19 smithi131 ceph-mon[36767]: fsmap 2024-05-10T05:25:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:19 smithi131 ceph-mon[36767]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T05:25:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:19 smithi131 ceph-mon[36767]: mgrmap e1: no daemons active 2024-05-10T05:25:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:19 smithi131 ceph-mon[36767]: from='client.? 172.21.15.131:0/3906241163' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-10T05:25:20.747 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:20.747 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T05:25:20.747 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:20.747 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.131:3300,v1:172.21.15.131:6789] 2024-05-10T05:25:20.747 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T05:25:20.747 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T05:25:20.747 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T05:25:20.747 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T05:25:20.748 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.239+0000 7f9e71e00700 1 Processor -- start 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.239+0000 7f9e71e00700 1 -- start start 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.240+0000 7f9e71e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c07a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.240+0000 7f9e71e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c07a770 con 0x7f9e6c07be00 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.240+0000 7f9e6aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c07a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.240+0000 7f9e6aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c07a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41238/0 (socket says 172.21.15.131:41238) 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.240+0000 7f9e6aa00700 1 -- 172.21.15.131:0/2943949821 learned_addr learned my addr 172.21.15.131:0/2943949821 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.240+0000 7f9e6aa00700 1 -- 172.21.15.131:0/2943949821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c07a8b0 con 0x7f9e6c07be00 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.241+0000 7f9e6aa00700 1 --2- 172.21.15.131:0/2943949821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c07a230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0098a0 tx=0x7f9e5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddf333299b8a882 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.241+0000 7f9e69600700 1 -- 172.21.15.131:0/2943949821 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e5c004160 con 0x7f9e6c07be00 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.241+0000 7f9e69600700 1 -- 172.21.15.131:0/2943949821 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c0042c0 con 0x7f9e6c07be00 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.241+0000 7f9e69600700 1 -- 172.21.15.131:0/2943949821 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e5c0045a0 con 0x7f9e6c07be00 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.242+0000 7f9e71e00700 1 -- 172.21.15.131:0/2943949821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 msgr2=0x7f9e6c07a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.242+0000 7f9e71e00700 1 --2- 172.21.15.131:0/2943949821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c07a230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0098a0 tx=0x7f9e5c009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:20.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.242+0000 7f9e71e00700 1 -- 172.21.15.131:0/2943949821 shutdown_connections 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.242+0000 7f9e71e00700 1 --2- 172.21.15.131:0/2943949821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c07a230 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.242+0000 7f9e71e00700 1 -- 172.21.15.131:0/2943949821 >> 172.21.15.131:0/2943949821 conn(0x7f9e6c102840 msgr2=0x7f9e6c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.242+0000 7f9e71e00700 1 -- 172.21.15.131:0/2943949821 shutdown_connections 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.242+0000 7f9e71e00700 1 -- 172.21.15.131:0/2943949821 wait complete. 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.243+0000 7f9e71e00700 1 Processor -- start 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.243+0000 7f9e71e00700 1 -- start start 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.243+0000 7f9e71e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.243+0000 7f9e71e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c19d2b0 con 0x7f9e6c07be00 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.244+0000 7f9e6aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.244+0000 7f9e6aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41254/0 (socket says 172.21.15.131:41254) 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.244+0000 7f9e6aa00700 1 -- 172.21.15.131:0/1526982964 learned_addr learned my addr 172.21.15.131:0/1526982964 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.244+0000 7f9e6aa00700 1 -- 172.21.15.131:0/1526982964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c009550 con 0x7f9e6c07be00 2024-05-10T05:25:20.750 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.244+0000 7f9e6aa00700 1 --2- 172.21.15.131:0/1526982964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c19cd70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c004000 tx=0x7f9e5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.244+0000 7f9e70a00700 1 -- 172.21.15.131:0/1526982964 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e5c011630 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.244+0000 7f9e70a00700 1 -- 172.21.15.131:0/1526982964 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c011790 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.245+0000 7f9e70a00700 1 -- 172.21.15.131:0/1526982964 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e5c011a30 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.245+0000 7f9e71e00700 1 -- 172.21.15.131:0/1526982964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c19d4b0 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.245+0000 7f9e71e00700 1 -- 172.21.15.131:0/1526982964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c19d950 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.245+0000 7f9e70a00700 1 -- 172.21.15.131:0/1526982964 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9e5c022070 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.245+0000 7f9e70a00700 1 -- 172.21.15.131:0/1526982964 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9e5c01c790 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.246+0000 7f9e71e00700 1 -- 172.21.15.131:0/1526982964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6c1967c0 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.248+0000 7f9e70a00700 1 -- 172.21.15.131:0/1526982964 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f9e5c01bca0 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.319+0000 7f9e71e00700 1 -- 172.21.15.131:0/1526982964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f9e6c07ae20 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.325+0000 7f9e70a00700 1 -- 172.21.15.131:0/1526982964 <== mon.0 v2:172.21.15.131:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e5c01be40 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.325+0000 7f9e70a00700 1 -- 172.21.15.131:0/1526982964 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f9e5c037720 con 0x7f9e6c07be00 2024-05-10T05:25:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.326+0000 7f9e71e00700 1 -- 172.21.15.131:0/1526982964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 msgr2=0x7f9e6c19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:20.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.326+0000 7f9e71e00700 1 --2- 172.21.15.131:0/1526982964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c19cd70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c004000 tx=0x7f9e5c004750 comp rx=0 tx=0).stop 2024-05-10T05:25:20.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.326+0000 7f9e71e00700 1 -- 172.21.15.131:0/1526982964 shutdown_connections 2024-05-10T05:25:20.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.326+0000 7f9e71e00700 1 --2- 172.21.15.131:0/1526982964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e6c07be00 0x7f9e6c19cd70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:20.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.326+0000 7f9e71e00700 1 -- 172.21.15.131:0/1526982964 >> 172.21.15.131:0/1526982964 conn(0x7f9e6c102840 msgr2=0x7f9e6c104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:20.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.326+0000 7f9e71e00700 1 -- 172.21.15.131:0/1526982964 shutdown_connections 2024-05-10T05:25:20.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:20.327+0000 7f9e71e00700 1 -- 172.21.15.131:0/1526982964 wait complete. 2024-05-10T05:25:20.752 INFO:teuthology.orchestra.run.smithi131.stdout:Generating new minimal ceph.conf... 2024-05-10T05:25:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:20 smithi131 ceph-mon[36767]: from='client.? 172.21.15.131:0/1526982964' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T05:25:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:20 smithi131 ceph-mon[36767]: from='client.? 172.21.15.131:0/1526982964' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.252+0000 7fda3a800700 1 Processor -- start 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.252+0000 7fda3a800700 1 -- start start 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.253+0000 7fda3a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3407a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.253+0000 7fda3a800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3407a770 con 0x7fda3407be00 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.253+0000 7fda33400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3407a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.253+0000 7fda33400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3407a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41264/0 (socket says 172.21.15.131:41264) 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.253+0000 7fda33400700 1 -- 172.21.15.131:0/2958408271 learned_addr learned my addr 172.21.15.131:0/2958408271 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.254+0000 7fda33400700 1 -- 172.21.15.131:0/2958408271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3407a8b0 con 0x7fda3407be00 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.254+0000 7fda33400700 1 --2- 172.21.15.131:0/2958408271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3407a230 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fda240098a0 tx=0x7fda24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1259cfe5c3591ef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.254+0000 7fda32a00700 1 -- 172.21.15.131:0/2958408271 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda24004160 con 0x7fda3407be00 2024-05-10T05:25:21.751 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.254+0000 7fda32a00700 1 -- 172.21.15.131:0/2958408271 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fda240042c0 con 0x7fda3407be00 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.254+0000 7fda32a00700 1 -- 172.21.15.131:0/2958408271 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda24011a00 con 0x7fda3407be00 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.255+0000 7fda3a800700 1 -- 172.21.15.131:0/2958408271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 msgr2=0x7fda3407a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.255+0000 7fda3a800700 1 --2- 172.21.15.131:0/2958408271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3407a230 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fda240098a0 tx=0x7fda24009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.255+0000 7fda3a800700 1 -- 172.21.15.131:0/2958408271 shutdown_connections 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.255+0000 7fda3a800700 1 --2- 172.21.15.131:0/2958408271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3407a230 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.255+0000 7fda3a800700 1 -- 172.21.15.131:0/2958408271 >> 172.21.15.131:0/2958408271 conn(0x7fda34102840 msgr2=0x7fda34104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.255+0000 7fda3a800700 1 -- 172.21.15.131:0/2958408271 shutdown_connections 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.255+0000 7fda3a800700 1 -- 172.21.15.131:0/2958408271 wait complete. 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.256+0000 7fda3a800700 1 Processor -- start 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.256+0000 7fda3a800700 1 -- start start 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.257+0000 7fda3a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.257+0000 7fda3a800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419db70 con 0x7fda3407be00 2024-05-10T05:25:21.752 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.257+0000 7fda33400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.257+0000 7fda33400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41274/0 (socket says 172.21.15.131:41274) 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.257+0000 7fda33400700 1 -- 172.21.15.131:0/509711586 learned_addr learned my addr 172.21.15.131:0/509711586 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.257+0000 7fda33400700 1 -- 172.21.15.131:0/509711586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda24009550 con 0x7fda3407be00 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.258+0000 7fda33400700 1 --2- 172.21.15.131:0/509711586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3419d630 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fda24011710 tx=0x7fda24011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.258+0000 7fda30c00700 1 -- 172.21.15.131:0/509711586 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda24003aa0 con 0x7fda3407be00 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.258+0000 7fda30c00700 1 -- 172.21.15.131:0/509711586 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fda24003c00 con 0x7fda3407be00 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.258+0000 7fda3a800700 1 -- 172.21.15.131:0/509711586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3419dd70 con 0x7fda3407be00 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.258+0000 7fda30c00700 1 -- 172.21.15.131:0/509711586 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda24024800 con 0x7fda3407be00 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.258+0000 7fda3a800700 1 -- 172.21.15.131:0/509711586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3419e210 con 0x7fda3407be00 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.258+0000 7fda30c00700 1 -- 172.21.15.131:0/509711586 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fda24024d30 con 0x7fda3407be00 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.259+0000 7fda30c00700 1 -- 172.21.15.131:0/509711586 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fda240193c0 con 0x7fda3407be00 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.259+0000 7fda3a800700 1 -- 172.21.15.131:0/509711586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3410d610 con 0x7fda3407be00 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.262+0000 7fda30c00700 1 -- 172.21.15.131:0/509711586 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fda2401abd0 con 0x7fda3407be00 2024-05-10T05:25:21.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.334+0000 7fda3a800700 1 -- 172.21.15.131:0/509711586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fda341a0c50 con 0x7fda3407be00 2024-05-10T05:25:21.754 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.334+0000 7fda30c00700 1 -- 172.21.15.131:0/509711586 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fda2401ad70 con 0x7fda3407be00 2024-05-10T05:25:21.754 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.335+0000 7fda3a800700 1 -- 172.21.15.131:0/509711586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 msgr2=0x7fda3419d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:21.754 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.335+0000 7fda3a800700 1 --2- 172.21.15.131:0/509711586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3419d630 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fda24011710 tx=0x7fda24011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:21.754 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.335+0000 7fda3a800700 1 -- 172.21.15.131:0/509711586 shutdown_connections 2024-05-10T05:25:21.754 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.335+0000 7fda3a800700 1 --2- 172.21.15.131:0/509711586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda3407be00 0x7fda3419d630 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:21.754 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.335+0000 7fda3a800700 1 -- 172.21.15.131:0/509711586 >> 172.21.15.131:0/509711586 conn(0x7fda34102840 msgr2=0x7fda34103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:21.754 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.335+0000 7fda3a800700 1 -- 172.21.15.131:0/509711586 shutdown_connections 2024-05-10T05:25:21.754 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:21.335+0000 7fda3a800700 1 -- 172.21.15.131:0/509711586 wait complete. 2024-05-10T05:25:21.754 INFO:teuthology.orchestra.run.smithi131.stdout:Restarting the monitor... 2024-05-10T05:25:22.051 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:21 smithi131 systemd[1]: Stopping Ceph mon.a for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:25:22.051 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:21 smithi131 ceph-mon[36767]: from='client.? 172.21.15.131:0/509711586' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:25:22.442 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:22 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-a[36744]: 2024-05-10T05:25:22.048+0000 7fbc08600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:25:22.442 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:22 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-a[36744]: 2024-05-10T05:25:22.048+0000 7fbc08600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-10T05:25:22.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:22 smithi131 bash[37103]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-a 2024-05-10T05:25:22.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:22 smithi131 systemd[1]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.a.service: Succeeded. 2024-05-10T05:25:22.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:22 smithi131 systemd[1]: Stopped Ceph mon.a for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:25:22.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:22 smithi131 systemd[1]: Starting Ceph mon.a for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:25:23.298 INFO:teuthology.orchestra.run.smithi131.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-10T05:25:23.544 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 bash[37225]: 33ae1f0f9704c4c53a5094290bc805eb4ebfef1cd22d9a9cddf9b81c41c1b7ad 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 systemd[1]: Started Ceph mon.a for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable), process ceph-mon, pid 7 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: pidfile_write: ignore empty --pid-file 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: load: jerasure load: lrc 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: RocksDB version: 7.9.2 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Git sha 0 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Compile date 2024-05-10 00:51:33 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: DB SUMMARY 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: DB Session ID: W18KMUJG17P388KZP2G0 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: CURRENT file: CURRENT 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T05:25:23.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98437 ; 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.error_if_exists: 0 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.create_if_missing: 0 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.paranoid_checks: 1 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.env: 0x556fdc048460 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.info_log: 0x556ffe0857a0 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T05:25:23.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.statistics: (nil) 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.use_fsync: 0 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_log_file_size: 0 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.allow_fallocate: 1 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.use_direct_reads: 0 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.db_log_dir: 2024-05-10T05:25:23.547 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.wal_dir: 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.write_buffer_manager: 0x556ffdfc90e0 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T05:25:23.548 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.unordered_write: 0 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.row_cache: None 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.wal_filter: None 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.two_write_queues: 0 2024-05-10T05:25:23.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.wal_compression: 0 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.atomic_flush: 0 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.log_readahead_size: 0 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T05:25:23.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_background_jobs: 2 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_background_compactions: -1 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_subcompactions: 1 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_open_files: -1 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T05:25:23.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_background_flushes: -1 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Compression algorithms supported: 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: kZSTD supported: 0 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: kXpressCompression supported: 0 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: kZlibCompression supported: 1 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: kSnappyCompression supported: 1 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: kLZ4Compression supported: 1 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: kBZip2Compression supported: 0 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T05:25:23.552 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.merge_operator: 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_filter: None 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556ffe0858c0) 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-05-10T05:25:23.553 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_type: 0 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_index_type: 0 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_shortening: 1 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: checksum: 4 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: no_block_cache: 0 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache: 0x556ffe04b1f0 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_options: 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: capacity : 536870912 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: num_shard_bits : 4 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: strict_capacity_limit : 0 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-05-10T05:25:23.554 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_compressed: (nil) 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: persistent_cache: (nil) 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size: 4096 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size_deviation: 10 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_restart_interval: 16 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_block_restart_interval: 1 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: metadata_block_size: 4096 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: partition_filters: 0 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: use_delta_encoding: 1 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: filter_policy: bloomfilter 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: whole_key_filtering: 1 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: verify_compression: 0 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: format_version: 5 2024-05-10T05:25:23.555 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_index_compression: 1 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_align: 0 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout: max_auto_readahead_size: 262144 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout: prepopulate_block_cache: 0 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout: initial_auto_readahead_size: 8192 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compression: NoCompression 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.num_levels: 7 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T05:25:23.556 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T05:25:23.557 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T05:25:23.557 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T05:25:23.557 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T05:25:23.557 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T05:25:23.557 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T05:25:23.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T05:25:23.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T05:25:23.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T05:25:23.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T05:25:23.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T05:25:23.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T05:25:23.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T05:25:23.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T05:25:23.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T05:25:23.559 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T05:25:23.560 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.table_properties_collectors: 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.inplace_update_support: 0 2024-05-10T05:25:23.561 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.bloom_locality: 0 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.max_successive_merges: 0 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.ttl: 2592000 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T05:25:23.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.enable_blob_files: false 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.min_blob_size: 0 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e7eaf429-2227-44fb-8f51-f83ffa2b059e 2024-05-10T05:25:23.563 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715318723388701, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715318723390480, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93767, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 312, "table_properties": {"data_size": 91762, "index_size": 287, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13529, "raw_average_key_size": 49, "raw_value_size": 84349, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715318723, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e7eaf429-2227-44fb-8f51-f83ffa2b059e", "db_session_id": "W18KMUJG17P388KZP2G0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715318723390612, "job": 1, "event": "recovery_finished"} 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556ffdfddc00 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: DB pointer 0x557002dcc000 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** DB Stats ** 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T05:25:23.564 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: L0 2/0 93.40 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: Sum 2/0 93.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-10T05:25:23.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 96.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative compaction: 0.00 GB write, 13.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T05:25:23.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval compaction: 0.00 GB write, 13.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout: Block cache BinnedLRUCache@0x556ffe04b1f0#7 capacity: 512.00 MB usage: 1.38 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: starting mon.a rank 0 at public addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] at bind addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: mon.a@-1(???) e1 preinit fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: mon.a@-1(???).mds e1 new map 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: mon.a@-1(???).mds e1 print_map 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout: e1 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T05:25:23.567 INFO:journalctl@ceph.mon.a.smithi131.stdout: legacy client fscid: -1 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout: No filesystems configured 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: monmap e1: 1 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: fsmap 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T05:25:23.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:23 smithi131 ceph-mon[37264]: mgrmap e1: no daemons active 2024-05-10T05:25:24.279 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.818+0000 7ff72de00700 1 Processor -- start 2024-05-10T05:25:24.279 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.819+0000 7ff72de00700 1 -- start start 2024-05-10T05:25:24.279 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.819+0000 7ff72de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72807a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:24.279 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.819+0000 7ff72de00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72807a750 con 0x7ff72807bde0 2024-05-10T05:25:24.279 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.820+0000 7ff726a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72807a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:24.279 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.820+0000 7ff726a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72807a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42070/0 (socket says 172.21.15.131:42070) 2024-05-10T05:25:24.279 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.820+0000 7ff726a00700 1 -- 172.21.15.131:0/3792784107 learned_addr learned my addr 172.21.15.131:0/3792784107 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:24.279 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.820+0000 7ff726a00700 1 -- 172.21.15.131:0/3792784107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72807a890 con 0x7ff72807bde0 2024-05-10T05:25:24.279 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.820+0000 7ff726a00700 1 --2- 172.21.15.131:0/3792784107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72807a210 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff7180098a0 tx=0x7ff718009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd5d2081e4c860c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:24.279 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.821+0000 7ff725600700 1 -- 172.21.15.131:0/3792784107 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff718004160 con 0x7ff72807bde0 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.821+0000 7ff725600700 1 -- 172.21.15.131:0/3792784107 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff7180042c0 con 0x7ff72807bde0 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.821+0000 7ff725600700 1 -- 172.21.15.131:0/3792784107 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff718011a00 con 0x7ff72807bde0 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.821+0000 7ff72de00700 1 -- 172.21.15.131:0/3792784107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 msgr2=0x7ff72807a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.821+0000 7ff72de00700 1 --2- 172.21.15.131:0/3792784107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72807a210 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff7180098a0 tx=0x7ff718009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.821+0000 7ff72de00700 1 -- 172.21.15.131:0/3792784107 shutdown_connections 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.821+0000 7ff72de00700 1 --2- 172.21.15.131:0/3792784107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72807a210 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.821+0000 7ff72de00700 1 -- 172.21.15.131:0/3792784107 >> 172.21.15.131:0/3792784107 conn(0x7ff728102840 msgr2=0x7ff728104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.822+0000 7ff72de00700 1 -- 172.21.15.131:0/3792784107 shutdown_connections 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.822+0000 7ff72de00700 1 -- 172.21.15.131:0/3792784107 wait complete. 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.823+0000 7ff72de00700 1 Processor -- start 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.823+0000 7ff72de00700 1 -- start start 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.823+0000 7ff72de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:24.280 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.823+0000 7ff72de00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72819db70 con 0x7ff72807bde0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.823+0000 7ff726a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.823+0000 7ff726a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42072/0 (socket says 172.21.15.131:42072) 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.823+0000 7ff726a00700 1 -- 172.21.15.131:0/4073130366 learned_addr learned my addr 172.21.15.131:0/4073130366 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.824+0000 7ff726a00700 1 -- 172.21.15.131:0/4073130366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff718009550 con 0x7ff72807bde0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.824+0000 7ff726a00700 1 --2- 172.21.15.131:0/4073130366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72819d630 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff718009130 tx=0x7ff718011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.824+0000 7ff72ca00700 1 -- 172.21.15.131:0/4073130366 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7180038b0 con 0x7ff72807bde0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.824+0000 7ff72ca00700 1 -- 172.21.15.131:0/4073130366 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff718003a10 con 0x7ff72807bde0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.824+0000 7ff72ca00700 1 -- 172.21.15.131:0/4073130366 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff718019550 con 0x7ff72807bde0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.824+0000 7ff72de00700 1 -- 172.21.15.131:0/4073130366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72819dd70 con 0x7ff72807bde0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.824+0000 7ff72de00700 1 -- 172.21.15.131:0/4073130366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72819e210 con 0x7ff72807bde0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.825+0000 7ff72ca00700 1 -- 172.21.15.131:0/4073130366 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff718019ac0 con 0x7ff72807bde0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.825+0000 7ff72ca00700 1 -- 172.21.15.131:0/4073130366 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff71802de20 con 0x7ff72807bde0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.825+0000 7ff72de00700 1 -- 172.21.15.131:0/4073130366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72810d630 con 0x7ff72807bde0 2024-05-10T05:25:24.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.829+0000 7ff72ca00700 1 -- 172.21.15.131:0/4073130366 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff718003b80 con 0x7ff72807bde0 2024-05-10T05:25:24.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.899+0000 7ff72de00700 1 -- 172.21.15.131:0/4073130366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7ff72810d820 con 0x7ff72807bde0 2024-05-10T05:25:24.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.903+0000 7ff72ca00700 1 -- 172.21.15.131:0/4073130366 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff71801a960 con 0x7ff72807bde0 2024-05-10T05:25:24.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.904+0000 7ff72de00700 1 -- 172.21.15.131:0/4073130366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 msgr2=0x7ff72819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:24.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.904+0000 7ff72de00700 1 --2- 172.21.15.131:0/4073130366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72819d630 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff718009130 tx=0x7ff718011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:24.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.904+0000 7ff72de00700 1 -- 172.21.15.131:0/4073130366 shutdown_connections 2024-05-10T05:25:24.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.904+0000 7ff72de00700 1 --2- 172.21.15.131:0/4073130366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff72807bde0 0x7ff72819d630 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:24.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.904+0000 7ff72de00700 1 -- 172.21.15.131:0/4073130366 >> 172.21.15.131:0/4073130366 conn(0x7ff728102840 msgr2=0x7ff7281032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:24.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.904+0000 7ff72de00700 1 -- 172.21.15.131:0/4073130366 shutdown_connections 2024-05-10T05:25:24.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:23.904+0000 7ff72de00700 1 -- 172.21.15.131:0/4073130366 wait complete. 2024-05-10T05:25:24.282 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-10T05:25:24.343 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-10T05:25:24.343 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mgr... 2024-05-10T05:25:24.343 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-10T05:25:24.344 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-10T05:25:24.703 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.x 2024-05-10T05:25:24.704 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-10T05:25:24.908 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6e8de474-0e8d-11ef-bc98-c7b262605968.target.wants/ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@.service. 2024-05-10T05:25:25.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:24 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4073130366' entity='client.admin' 2024-05-10T05:25:25.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:24 smithi131 systemd[1]: Starting Ceph mgr.x for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:25:25.445 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T05:25:25.445 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-05-10T05:25:25.461 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T05:25:25.461 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-10T05:25:25.461 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-05-10T05:25:25.461 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T05:25:25.480 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T05:25:25.480 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-05-10T05:25:25.498 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T05:25:25.498 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-10T05:25:25.498 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-05-10T05:25:25.498 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-10T05:25:25.498 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr to start... 2024-05-10T05:25:25.498 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr... 2024-05-10T05:25:25.690 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:25 smithi131 bash[37499]: ea4ddccd927597f0830d32bf3e24a0a44d986d8b12895cf6d5240229c6deac90 2024-05-10T05:25:25.691 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:25 smithi131 systemd[1]: Started Ceph mgr.x for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:25:26.299 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:26 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2245770720' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T05:25:26.300 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:26 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:26.034+0000 7fdb3a6fe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T05:25:26.300 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:26 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:26.159+0000 7fdb3a6fe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T05:25:26.300 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:26 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:26.281+0000 7fdb3a6fe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T05:25:26.449 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:26.451 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-10T05:25:26.451 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:25:26.451 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T05:25:26.451 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:26.452 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T05:25:26.453 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T05:25:26.453 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:26.453 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T05:25:26.453 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:26.453 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T05:25:26.461 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T05:25:26.462 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T05:25:18.810847+0000", 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T05:25:26.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.019+0000 7ff683e00700 1 Processor -- start 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.020+0000 7ff683e00700 1 -- start start 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.020+0000 7ff683e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c07a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.020+0000 7ff683e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c07a750 con 0x7ff67c07bde0 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.021+0000 7ff681400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c07a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.021+0000 7ff681400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c07a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42086/0 (socket says 172.21.15.131:42086) 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.021+0000 7ff681400700 1 -- 172.21.15.131:0/1114161846 learned_addr learned my addr 172.21.15.131:0/1114161846 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.021+0000 7ff681400700 1 -- 172.21.15.131:0/1114161846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c07a890 con 0x7ff67c07bde0 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.022+0000 7ff681400700 1 --2- 172.21.15.131:0/1114161846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c07a210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff6700098a0 tx=0x7ff670009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a466d220268d00 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.022+0000 7ff67be00700 1 -- 172.21.15.131:0/1114161846 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff670004160 con 0x7ff67c07bde0 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.022+0000 7ff67be00700 1 -- 172.21.15.131:0/1114161846 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6700042c0 con 0x7ff67c07bde0 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.022+0000 7ff67be00700 1 -- 172.21.15.131:0/1114161846 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff670011a00 con 0x7ff67c07bde0 2024-05-10T05:25:26.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.023+0000 7ff683e00700 1 -- 172.21.15.131:0/1114161846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 msgr2=0x7ff67c07a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.023+0000 7ff683e00700 1 --2- 172.21.15.131:0/1114161846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c07a210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff6700098a0 tx=0x7ff670009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.023+0000 7ff683e00700 1 -- 172.21.15.131:0/1114161846 shutdown_connections 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.023+0000 7ff683e00700 1 --2- 172.21.15.131:0/1114161846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c07a210 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.023+0000 7ff683e00700 1 -- 172.21.15.131:0/1114161846 >> 172.21.15.131:0/1114161846 conn(0x7ff67c102840 msgr2=0x7ff67c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.023+0000 7ff683e00700 1 -- 172.21.15.131:0/1114161846 shutdown_connections 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.023+0000 7ff683e00700 1 -- 172.21.15.131:0/1114161846 wait complete. 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.024+0000 7ff683e00700 1 Processor -- start 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.024+0000 7ff683e00700 1 -- start start 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.024+0000 7ff683e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.024+0000 7ff683e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c19db80 con 0x7ff67c07bde0 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.025+0000 7ff681400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.025+0000 7ff681400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42094/0 (socket says 172.21.15.131:42094) 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.025+0000 7ff681400700 1 -- 172.21.15.131:0/2245770720 learned_addr learned my addr 172.21.15.131:0/2245770720 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:26.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.025+0000 7ff681400700 1 -- 172.21.15.131:0/2245770720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff670009550 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.025+0000 7ff681400700 1 --2- 172.21.15.131:0/2245770720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c19d640 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff670011710 tx=0x7ff670011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.025+0000 7ff67a000700 1 -- 172.21.15.131:0/2245770720 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff670003aa0 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.026+0000 7ff67a000700 1 -- 172.21.15.131:0/2245770720 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff670003c00 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.026+0000 7ff683e00700 1 -- 172.21.15.131:0/2245770720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c19dd80 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.026+0000 7ff67a000700 1 -- 172.21.15.131:0/2245770720 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff670024800 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.026+0000 7ff683e00700 1 -- 172.21.15.131:0/2245770720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c19e220 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.026+0000 7ff67a000700 1 -- 172.21.15.131:0/2245770720 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff670024c60 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.026+0000 7ff67a000700 1 -- 172.21.15.131:0/2245770720 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff6700193c0 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.027+0000 7ff683e00700 1 -- 172.21.15.131:0/2245770720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c10d630 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.029+0000 7ff67a000700 1 -- 172.21.15.131:0/2245770720 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff67001abd0 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.102+0000 7ff683e00700 1 -- 172.21.15.131:0/2245770720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff67c10d820 con 0x7ff67c07bde0 2024-05-10T05:25:26.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.103+0000 7ff67a000700 1 -- 172.21.15.131:0/2245770720 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7ff67001ad70 con 0x7ff67c07bde0 2024-05-10T05:25:26.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.104+0000 7ff683e00700 1 -- 172.21.15.131:0/2245770720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 msgr2=0x7ff67c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:26.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.104+0000 7ff683e00700 1 --2- 172.21.15.131:0/2245770720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c19d640 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff670011710 tx=0x7ff670011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:26.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.104+0000 7ff683e00700 1 -- 172.21.15.131:0/2245770720 shutdown_connections 2024-05-10T05:25:26.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.104+0000 7ff683e00700 1 --2- 172.21.15.131:0/2245770720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c07bde0 0x7ff67c19d640 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:26.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.104+0000 7ff683e00700 1 -- 172.21.15.131:0/2245770720 >> 172.21.15.131:0/2245770720 conn(0x7ff67c102840 msgr2=0x7ff67c103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:26.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.104+0000 7ff683e00700 1 -- 172.21.15.131:0/2245770720 shutdown_connections 2024-05-10T05:25:26.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:26.104+0000 7ff683e00700 1 -- 172.21.15.131:0/2245770720 wait complete. 2024-05-10T05:25:26.467 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (1/15)... 2024-05-10T05:25:26.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:26 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:26.394+0000 7fdb3a6fe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T05:25:27.575 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:27 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:27.260+0000 7fdb3a6fe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T05:25:27.829 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:27 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:27.574+0000 7fdb3a6fe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T05:25:27.830 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:27 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:27.687+0000 7fdb3a6fe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T05:25:28.202 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:27 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:27.828+0000 7fdb3a6fe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T05:25:28.203 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:28 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:28.029+0000 7fdb3a6fe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T05:25:28.460 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:28 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:28.201+0000 7fdb3a6fe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T05:25:28.460 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:28 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:28.314+0000 7fdb3a6fe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T05:25:28.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:28 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:28.811+0000 7fdb3a6fe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T05:25:29.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:29 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/943379024' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T05:25:29.460 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:29 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:29.337+0000 7fdb3a6fe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T05:25:29.470 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:29.471 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T05:25:29.472 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:29.473 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:29.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T05:25:18.810847+0000", 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.991+0000 7f0139600700 1 Processor -- start 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.991+0000 7f0139600700 1 -- start start 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.991+0000 7f0139600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013407a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.991+0000 7f0139600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013407a840 con 0x7f013407be40 2024-05-10T05:25:29.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.992+0000 7f0133e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013407a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.992+0000 7f0133e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013407a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42110/0 (socket says 172.21.15.131:42110) 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.992+0000 7f0133e00700 1 -- 172.21.15.131:0/197941969 learned_addr learned my addr 172.21.15.131:0/197941969 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.992+0000 7f0133e00700 1 -- 172.21.15.131:0/197941969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013407a980 con 0x7f013407be40 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.993+0000 7f0133e00700 1 --2- 172.21.15.131:0/197941969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013407a270 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01240098a0 tx=0x7f0124009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce3868ce79fbd1a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.993+0000 7f0132a00700 1 -- 172.21.15.131:0/197941969 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0124004160 con 0x7f013407be40 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.993+0000 7f0132a00700 1 -- 172.21.15.131:0/197941969 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01240042c0 con 0x7f013407be40 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.993+0000 7f0132a00700 1 -- 172.21.15.131:0/197941969 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0124011a00 con 0x7f013407be40 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.994+0000 7f0139600700 1 -- 172.21.15.131:0/197941969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 msgr2=0x7f013407a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.994+0000 7f0139600700 1 --2- 172.21.15.131:0/197941969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013407a270 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01240098a0 tx=0x7f0124009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.994+0000 7f0139600700 1 -- 172.21.15.131:0/197941969 shutdown_connections 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.994+0000 7f0139600700 1 --2- 172.21.15.131:0/197941969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013407a270 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.994+0000 7f0139600700 1 -- 172.21.15.131:0/197941969 >> 172.21.15.131:0/197941969 conn(0x7f01341026b0 msgr2=0x7f0134104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.994+0000 7f0139600700 1 -- 172.21.15.131:0/197941969 shutdown_connections 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.994+0000 7f0139600700 1 -- 172.21.15.131:0/197941969 wait complete. 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.995+0000 7f0139600700 1 Processor -- start 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.995+0000 7f0139600700 1 -- start start 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.996+0000 7f0139600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.996+0000 7f0139600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013419da10 con 0x7f013407be40 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.996+0000 7f0133e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.996+0000 7f0133e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013419d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42114/0 (socket says 172.21.15.131:42114) 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.996+0000 7f0133e00700 1 -- 172.21.15.131:0/943379024 learned_addr learned my addr 172.21.15.131:0/943379024 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.996+0000 7f0133e00700 1 -- 172.21.15.131:0/943379024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0124009550 con 0x7f013407be40 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.996+0000 7f0133e00700 1 --2- 172.21.15.131:0/943379024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013419d4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0124011710 tx=0x7f0124011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.997+0000 7f0130c00700 1 -- 172.21.15.131:0/943379024 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0124003af0 con 0x7f013407be40 2024-05-10T05:25:29.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.997+0000 7f0130c00700 1 -- 172.21.15.131:0/943379024 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0124003c50 con 0x7f013407be40 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.997+0000 7f0139600700 1 -- 172.21.15.131:0/943379024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013419dc10 con 0x7f013407be40 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.997+0000 7f0130c00700 1 -- 172.21.15.131:0/943379024 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0124024780 con 0x7f013407be40 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.997+0000 7f0139600700 1 -- 172.21.15.131:0/943379024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013419e0b0 con 0x7f013407be40 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.997+0000 7f0130c00700 1 -- 172.21.15.131:0/943379024 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0124024cc0 con 0x7f013407be40 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.997+0000 7f0130c00700 1 -- 172.21.15.131:0/943379024 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f01240193c0 con 0x7f013407be40 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:28.998+0000 7f0139600700 1 -- 172.21.15.131:0/943379024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013410d4b0 con 0x7f013407be40 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:29.001+0000 7f0130c00700 1 -- 172.21.15.131:0/943379024 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f012401ab40 con 0x7f013407be40 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:29.079+0000 7f0139600700 1 -- 172.21.15.131:0/943379024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f01341a0af0 con 0x7f013407be40 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:29.080+0000 7f0130c00700 1 -- 172.21.15.131:0/943379024 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f012401ace0 con 0x7f013407be40 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:29.081+0000 7f0139600700 1 -- 172.21.15.131:0/943379024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 msgr2=0x7f013419d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:29.081+0000 7f0139600700 1 --2- 172.21.15.131:0/943379024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013419d4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0124011710 tx=0x7f0124011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:29.081+0000 7f0139600700 1 -- 172.21.15.131:0/943379024 shutdown_connections 2024-05-10T05:25:29.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:29.081+0000 7f0139600700 1 --2- 172.21.15.131:0/943379024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f013407be40 0x7f013419d4d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:29.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:29.081+0000 7f0139600700 1 -- 172.21.15.131:0/943379024 >> 172.21.15.131:0/943379024 conn(0x7f01341026b0 msgr2=0x7f01341031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:29.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:29.081+0000 7f0139600700 1 -- 172.21.15.131:0/943379024 shutdown_connections 2024-05-10T05:25:29.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:29.081+0000 7f0139600700 1 -- 172.21.15.131:0/943379024 wait complete. 2024-05-10T05:25:29.479 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (2/15)... 2024-05-10T05:25:29.720 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:29 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:29.597+0000 7fdb3a6fe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T05:25:30.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:29 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:29.719+0000 7fdb3a6fe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T05:25:30.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:30 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:30.678+0000 7fdb3a6fe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T05:25:30.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:30 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:30.803+0000 7fdb3a6fe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T05:25:31.460 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:30 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:30.994+0000 7fdb3a6fe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T05:25:31.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:31 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:31.847+0000 7fdb3a6fe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T05:25:32.376 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:32 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/803729657' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T05:25:32.376 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:32 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:32.103+0000 7fdb3a6fe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T05:25:32.418 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:32.421 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-10T05:25:32.421 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:25:32.421 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T05:25:32.421 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T05:25:32.421 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T05:25:32.421 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T05:25:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T05:25:32.423 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:32.423 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T05:25:32.423 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:32.423 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T05:25:32.423 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T05:25:32.431 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T05:25:32.431 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T05:25:32.431 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T05:25:32.431 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T05:25:32.431 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:32.431 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T05:25:32.432 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T05:25:18.810847+0000", 2024-05-10T05:25:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.981+0000 7fb749600700 1 Processor -- start 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.982+0000 7fb749600700 1 -- start start 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.982+0000 7fb749600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb744107630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.982+0000 7fb749600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb744107c20 con 0x7fb744107220 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.982+0000 7fb742a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb744107630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.983+0000 7fb742a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb744107630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42120/0 (socket says 172.21.15.131:42120) 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.983+0000 7fb742a00700 1 -- 172.21.15.131:0/297063723 learned_addr learned my addr 172.21.15.131:0/297063723 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.983+0000 7fb742a00700 1 -- 172.21.15.131:0/297063723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7441083f0 con 0x7fb744107220 2024-05-10T05:25:32.434 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.983+0000 7fb742a00700 1 --2- 172.21.15.131:0/297063723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb744107630 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb7340098a0 tx=0x7fb734009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f53e03d9a20962 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.983+0000 7fb741600700 1 -- 172.21.15.131:0/297063723 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb734004160 con 0x7fb744107220 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.983+0000 7fb741600700 1 -- 172.21.15.131:0/297063723 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7340042c0 con 0x7fb744107220 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.983+0000 7fb741600700 1 -- 172.21.15.131:0/297063723 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb734011a00 con 0x7fb744107220 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.984+0000 7fb749600700 1 -- 172.21.15.131:0/297063723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 msgr2=0x7fb744107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.984+0000 7fb749600700 1 --2- 172.21.15.131:0/297063723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb744107630 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb7340098a0 tx=0x7fb734009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.984+0000 7fb749600700 1 -- 172.21.15.131:0/297063723 shutdown_connections 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.984+0000 7fb749600700 1 --2- 172.21.15.131:0/297063723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb744107630 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.984+0000 7fb749600700 1 -- 172.21.15.131:0/297063723 >> 172.21.15.131:0/297063723 conn(0x7fb744102840 msgr2=0x7fb744104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.984+0000 7fb749600700 1 -- 172.21.15.131:0/297063723 shutdown_connections 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.984+0000 7fb749600700 1 -- 172.21.15.131:0/297063723 wait complete. 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.985+0000 7fb749600700 1 Processor -- start 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.986+0000 7fb749600700 1 -- start start 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.986+0000 7fb749600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb74419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:32.435 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.986+0000 7fb749600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74419dbc0 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.986+0000 7fb742a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb74419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.986+0000 7fb742a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb74419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42122/0 (socket says 172.21.15.131:42122) 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.986+0000 7fb742a00700 1 -- 172.21.15.131:0/803729657 learned_addr learned my addr 172.21.15.131:0/803729657 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.987+0000 7fb742a00700 1 -- 172.21.15.131:0/803729657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb734009550 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.987+0000 7fb742a00700 1 --2- 172.21.15.131:0/803729657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb74419d680 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb734011a20 tx=0x7fb734011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.987+0000 7fb73b400700 1 -- 172.21.15.131:0/803729657 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb734003aa0 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.987+0000 7fb73b400700 1 -- 172.21.15.131:0/803729657 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb734003c00 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.987+0000 7fb73b400700 1 -- 172.21.15.131:0/803729657 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7340196a0 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.987+0000 7fb749600700 1 -- 172.21.15.131:0/803729657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74419ddc0 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.987+0000 7fb749600700 1 -- 172.21.15.131:0/803729657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74419e260 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.988+0000 7fb73b400700 1 -- 172.21.15.131:0/803729657 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb7340198d0 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.988+0000 7fb73b400700 1 -- 172.21.15.131:0/803729657 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb73400b3e0 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.988+0000 7fb749600700 1 -- 172.21.15.131:0/803729657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb744197490 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:31.991+0000 7fb73b400700 1 -- 172.21.15.131:0/803729657 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fb73402a060 con 0x7fb744107220 2024-05-10T05:25:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:32.065+0000 7fb749600700 1 -- 172.21.15.131:0/803729657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb74410b620 con 0x7fb744107220 2024-05-10T05:25:32.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:32.066+0000 7fb73b400700 1 -- 172.21.15.131:0/803729657 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fb734019cc0 con 0x7fb744107220 2024-05-10T05:25:32.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:32.067+0000 7fb749600700 1 -- 172.21.15.131:0/803729657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 msgr2=0x7fb74419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:32.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:32.067+0000 7fb749600700 1 --2- 172.21.15.131:0/803729657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb74419d680 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb734011a20 tx=0x7fb734011b00 comp rx=0 tx=0).stop 2024-05-10T05:25:32.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:32.067+0000 7fb749600700 1 -- 172.21.15.131:0/803729657 shutdown_connections 2024-05-10T05:25:32.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:32.067+0000 7fb749600700 1 --2- 172.21.15.131:0/803729657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb744107220 0x7fb74419d680 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:32.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:32.067+0000 7fb749600700 1 -- 172.21.15.131:0/803729657 >> 172.21.15.131:0/803729657 conn(0x7fb744102840 msgr2=0x7fb74407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:32.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:32.067+0000 7fb749600700 1 -- 172.21.15.131:0/803729657 shutdown_connections 2024-05-10T05:25:32.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:32.067+0000 7fb749600700 1 -- 172.21.15.131:0/803729657 wait complete. 2024-05-10T05:25:32.437 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (3/15)... 2024-05-10T05:25:32.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:32 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:32.375+0000 7fdb3a6fe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T05:25:33.908 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:33 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:33.535+0000 7fdb3a6fe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T05:25:34.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:33 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:33.907+0000 7fdb3a6fe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T05:25:34.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:34 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:34.025+0000 7fdb3a6fe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T05:25:34.582 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:34 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:34.276+0000 7fdb3a6fe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T05:25:34.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: Activating manager daemon x 2024-05-10T05:25:34.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: mgrmap e2: x(active, starting, since 0.00466557s) 2024-05-10T05:25:34.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: from='mgr.14100 172.21.15.131:0/3978019798' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T05:25:34.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: from='mgr.14100 172.21.15.131:0/3978019798' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T05:25:34.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: from='mgr.14100 172.21.15.131:0/3978019798' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T05:25:34.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: from='mgr.14100 172.21.15.131:0/3978019798' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:25:34.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: from='mgr.14100 172.21.15.131:0/3978019798' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T05:25:34.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: Manager daemon x is now available 2024-05-10T05:25:34.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: from='mgr.14100 172.21.15.131:0/3978019798' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T05:25:34.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: from='mgr.14100 172.21.15.131:0/3978019798' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T05:25:34.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: from='mgr.14100 172.21.15.131:0/3978019798' entity='mgr.x' 2024-05-10T05:25:34.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: from='mgr.14100 172.21.15.131:0/3978019798' entity='mgr.x' 2024-05-10T05:25:34.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:34 smithi131 ceph-mon[37264]: from='mgr.14100 172.21.15.131:0/3978019798' entity='mgr.x' 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-05-10T05:25:35.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T05:25:35.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T05:25:35.359 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T05:25:35.359 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T05:25:35.359 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T05:25:35.359 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:35.359 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T05:25:35.359 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T05:25:35.359 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T05:25:35.359 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T05:25:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T05:25:18.810847+0000", 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T05:25:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.929+0000 7fdfe2a00700 1 Processor -- start 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.930+0000 7fdfe2a00700 1 -- start start 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.930+0000 7fdfe2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc07a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.930+0000 7fdfe2a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc07a820 con 0x7fdfdc07be20 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.930+0000 7fdfdbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc07a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.930+0000 7fdfdbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc07a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41744/0 (socket says 172.21.15.131:41744) 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.930+0000 7fdfdbe00700 1 -- 172.21.15.131:0/3140400285 learned_addr learned my addr 172.21.15.131:0/3140400285 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.931+0000 7fdfdbe00700 1 -- 172.21.15.131:0/3140400285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc07a960 con 0x7fdfdc07be20 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.931+0000 7fdfdbe00700 1 --2- 172.21.15.131:0/3140400285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc07a250 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0098a0 tx=0x7fdfcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96ba5daaf00d449 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.931+0000 7fdfdaa00700 1 -- 172.21.15.131:0/3140400285 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfcc004160 con 0x7fdfdc07be20 2024-05-10T05:25:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.932+0000 7fdfdaa00700 1 -- 172.21.15.131:0/3140400285 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdfcc0042c0 con 0x7fdfdc07be20 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.932+0000 7fdfdaa00700 1 -- 172.21.15.131:0/3140400285 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfcc011a00 con 0x7fdfdc07be20 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.932+0000 7fdfe2a00700 1 -- 172.21.15.131:0/3140400285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 msgr2=0x7fdfdc07a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.932+0000 7fdfe2a00700 1 --2- 172.21.15.131:0/3140400285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc07a250 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0098a0 tx=0x7fdfcc009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.932+0000 7fdfe2a00700 1 -- 172.21.15.131:0/3140400285 shutdown_connections 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.932+0000 7fdfe2a00700 1 --2- 172.21.15.131:0/3140400285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc07a250 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.932+0000 7fdfe2a00700 1 -- 172.21.15.131:0/3140400285 >> 172.21.15.131:0/3140400285 conn(0x7fdfdc102840 msgr2=0x7fdfdc104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.932+0000 7fdfe2a00700 1 -- 172.21.15.131:0/3140400285 shutdown_connections 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.932+0000 7fdfe2a00700 1 -- 172.21.15.131:0/3140400285 wait complete. 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.934+0000 7fdfe2a00700 1 Processor -- start 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.934+0000 7fdfe2a00700 1 -- start start 2024-05-10T05:25:35.363 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.934+0000 7fdfe2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.935+0000 7fdfe2a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc19dba0 con 0x7fdfdc07be20 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.935+0000 7fdfdbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.935+0000 7fdfdbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41754/0 (socket says 172.21.15.131:41754) 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.935+0000 7fdfdbe00700 1 -- 172.21.15.131:0/1792546134 learned_addr learned my addr 172.21.15.131:0/1792546134 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.935+0000 7fdfdbe00700 1 -- 172.21.15.131:0/1792546134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc009550 con 0x7fdfdc07be20 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.935+0000 7fdfdbe00700 1 --2- 172.21.15.131:0/1792546134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc19d660 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc011710 tx=0x7fdfcc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.936+0000 7fdfd8c00700 1 -- 172.21.15.131:0/1792546134 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfcc003aa0 con 0x7fdfdc07be20 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.936+0000 7fdfd8c00700 1 -- 172.21.15.131:0/1792546134 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdfcc003c00 con 0x7fdfdc07be20 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.936+0000 7fdfd8c00700 1 -- 172.21.15.131:0/1792546134 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfcc024800 con 0x7fdfdc07be20 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.936+0000 7fdfe2a00700 1 -- 172.21.15.131:0/1792546134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfdc19dda0 con 0x7fdfdc07be20 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.936+0000 7fdfe2a00700 1 -- 172.21.15.131:0/1792546134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfdc19e240 con 0x7fdfdc07be20 2024-05-10T05:25:35.364 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.937+0000 7fdfd8c00700 1 -- 172.21.15.131:0/1792546134 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 2) v1 ==== 45297+0+0 (secure 0 0 0) 0x7fdfcc024960 con 0x7fdfdc07be20 2024-05-10T05:25:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.937+0000 7fdfe2a00700 1 -- 172.21.15.131:0/1792546134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfdc197270 con 0x7fdfdc07be20 2024-05-10T05:25:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.937+0000 7fdfd8c00700 1 -- 172.21.15.131:0/1792546134 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdfcc04bfb0 con 0x7fdfdc07be20 2024-05-10T05:25:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:34.939+0000 7fdfd8c00700 1 -- 172.21.15.131:0/1792546134 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fdfcc018530 con 0x7fdfdc07be20 2024-05-10T05:25:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:35.012+0000 7fdfe2a00700 1 -- 172.21.15.131:0/1792546134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fdfdc10d600 con 0x7fdfdc07be20 2024-05-10T05:25:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:35.013+0000 7fdfd8c00700 1 -- 172.21.15.131:0/1792546134 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fdfcc018350 con 0x7fdfdc07be20 2024-05-10T05:25:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:35.013+0000 7fdfe2a00700 1 -- 172.21.15.131:0/1792546134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 msgr2=0x7fdfdc19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:35.013+0000 7fdfe2a00700 1 --2- 172.21.15.131:0/1792546134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc19d660 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc011710 tx=0x7fdfcc011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:35.014+0000 7fdfe2a00700 1 -- 172.21.15.131:0/1792546134 shutdown_connections 2024-05-10T05:25:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:35.014+0000 7fdfe2a00700 1 --2- 172.21.15.131:0/1792546134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc07be20 0x7fdfdc19d660 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:35.014+0000 7fdfe2a00700 1 -- 172.21.15.131:0/1792546134 >> 172.21.15.131:0/1792546134 conn(0x7fdfdc102840 msgr2=0x7fdfdc103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:35.366 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:35.014+0000 7fdfe2a00700 1 -- 172.21.15.131:0/1792546134 shutdown_connections 2024-05-10T05:25:35.366 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:35.014+0000 7fdfe2a00700 1 -- 172.21.15.131:0/1792546134 wait complete. 2024-05-10T05:25:35.366 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (4/15)... 2024-05-10T05:25:35.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:35 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1792546134' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T05:25:35.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:35 smithi131 ceph-mon[37264]: mgrmap e3: x(active, since 1.00883s) 2024-05-10T05:25:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:37 smithi131 ceph-mon[37264]: mgrmap e4: x(active, since 2s) 2024-05-10T05:25:38.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:38 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/408451613' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T05:25:38.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:38.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-10T05:25:38.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:25:38.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T05:25:38.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T05:25:38.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T05:25:38.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:38.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T05:25:38.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:38.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T05:25:38.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T05:25:38.484 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T05:25:38.485 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T05:25:38.486 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T05:25:18.810847+0000", 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.831+0000 7f777a600700 1 Processor -- start 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.832+0000 7f777a600700 1 -- start start 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.832+0000 7f777a600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f7774107630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.832+0000 7f777a600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7774107c20 con 0x7f7774107220 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.832+0000 7f7773400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f7774107630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:38.487 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.832+0000 7f7773400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f7774107630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41764/0 (socket says 172.21.15.131:41764) 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.832+0000 7f7773400700 1 -- 172.21.15.131:0/3845903676 learned_addr learned my addr 172.21.15.131:0/3845903676 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.833+0000 7f7773400700 1 -- 172.21.15.131:0/3845903676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77741083f0 con 0x7f7774107220 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.833+0000 7f7773400700 1 --2- 172.21.15.131:0/3845903676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f7774107630 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f77640098a0 tx=0x7f7764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962338876c1e1337 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.833+0000 7f7772000700 1 -- 172.21.15.131:0/3845903676 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7764004160 con 0x7f7774107220 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.833+0000 7f7772000700 1 -- 172.21.15.131:0/3845903676 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f77640042c0 con 0x7f7774107220 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.833+0000 7f7772000700 1 -- 172.21.15.131:0/3845903676 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7764011a00 con 0x7f7774107220 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.834+0000 7f777a600700 1 -- 172.21.15.131:0/3845903676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 msgr2=0x7f7774107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.834+0000 7f777a600700 1 --2- 172.21.15.131:0/3845903676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f7774107630 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f77640098a0 tx=0x7f7764009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.834+0000 7f777a600700 1 -- 172.21.15.131:0/3845903676 shutdown_connections 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.834+0000 7f777a600700 1 --2- 172.21.15.131:0/3845903676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f7774107630 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.834+0000 7f777a600700 1 -- 172.21.15.131:0/3845903676 >> 172.21.15.131:0/3845903676 conn(0x7f7774102840 msgr2=0x7f7774104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.834+0000 7f777a600700 1 -- 172.21.15.131:0/3845903676 shutdown_connections 2024-05-10T05:25:38.488 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.834+0000 7f777a600700 1 -- 172.21.15.131:0/3845903676 wait complete. 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.835+0000 7f777a600700 1 Processor -- start 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.835+0000 7f777a600700 1 -- start start 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.836+0000 7f777a600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f777419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.836+0000 7f777a600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777419b970 con 0x7f7774107220 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.836+0000 7f7773400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f777419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.836+0000 7f7773400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f777419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41766/0 (socket says 172.21.15.131:41766) 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.836+0000 7f7773400700 1 -- 172.21.15.131:0/408451613 learned_addr learned my addr 172.21.15.131:0/408451613 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.836+0000 7f7773400700 1 -- 172.21.15.131:0/408451613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7764009550 con 0x7f7774107220 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.836+0000 7f7773400700 1 --2- 172.21.15.131:0/408451613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f777419b430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7764011a20 tx=0x7f7764011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.837+0000 7f776be00700 1 -- 172.21.15.131:0/408451613 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7764003aa0 con 0x7f7774107220 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.837+0000 7f776be00700 1 -- 172.21.15.131:0/408451613 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7764003c00 con 0x7f7774107220 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.837+0000 7f776be00700 1 -- 172.21.15.131:0/408451613 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77640196a0 con 0x7f7774107220 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.837+0000 7f777a600700 1 -- 172.21.15.131:0/408451613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777419bb70 con 0x7f7774107220 2024-05-10T05:25:38.489 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.837+0000 7f777a600700 1 -- 172.21.15.131:0/408451613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777419c010 con 0x7f7774107220 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.838+0000 7f777a600700 1 -- 172.21.15.131:0/408451613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777410b620 con 0x7f7774107220 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.838+0000 7f776be00700 1 -- 172.21.15.131:0/408451613 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f77640198d0 con 0x7f7774107220 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.838+0000 7f776be00700 1 --2- 172.21.15.131:0/408451613 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7758038700 0x7f775803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.839+0000 7f776be00700 1 -- 172.21.15.131:0/408451613 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f776404c8a0 con 0x7f7774107220 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.839+0000 7f7772a00700 1 --2- 172.21.15.131:0/408451613 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7758038700 0x7f775803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.839+0000 7f7772a00700 1 --2- 172.21.15.131:0/408451613 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7758038700 0x7f775803ac10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7760006fd0 tx=0x7f7760006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:37.844+0000 7f776be00700 1 -- 172.21.15.131:0/408451613 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7764018db0 con 0x7f7774107220 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.123+0000 7f777a600700 1 -- 172.21.15.131:0/408451613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f777410b830 con 0x7f7774107220 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.123+0000 7f776be00700 1 -- 172.21.15.131:0/408451613 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f7764018350 con 0x7f7774107220 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.126+0000 7f777a600700 1 -- 172.21.15.131:0/408451613 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7758038700 msgr2=0x7f775803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.126+0000 7f777a600700 1 --2- 172.21.15.131:0/408451613 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7758038700 0x7f775803ac10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7760006fd0 tx=0x7f7760006e40 comp rx=0 tx=0).stop 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.126+0000 7f777a600700 1 -- 172.21.15.131:0/408451613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 msgr2=0x7f777419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:38.490 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.126+0000 7f777a600700 1 --2- 172.21.15.131:0/408451613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f777419b430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7764011a20 tx=0x7f7764011b00 comp rx=0 tx=0).stop 2024-05-10T05:25:38.491 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.126+0000 7f777a600700 1 -- 172.21.15.131:0/408451613 shutdown_connections 2024-05-10T05:25:38.491 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.126+0000 7f777a600700 1 --2- 172.21.15.131:0/408451613 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7758038700 0x7f775803ac10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:38.491 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.126+0000 7f777a600700 1 --2- 172.21.15.131:0/408451613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7774107220 0x7f777419b430 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:38.491 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.126+0000 7f777a600700 1 -- 172.21.15.131:0/408451613 >> 172.21.15.131:0/408451613 conn(0x7f7774102840 msgr2=0x7f77741032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:38.491 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.126+0000 7f777a600700 1 -- 172.21.15.131:0/408451613 shutdown_connections 2024-05-10T05:25:38.491 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.126+0000 7f777a600700 1 -- 172.21.15.131:0/408451613 wait complete. 2024-05-10T05:25:38.491 INFO:teuthology.orchestra.run.smithi131.stdout:mgr is available 2024-05-10T05:25:39.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:39 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2837876289' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-05-10T05:25:39.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.951+0000 7f826f000700 1 Processor -- start 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.951+0000 7f826f000700 1 -- start start 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.951+0000 7f826f000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826807a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.951+0000 7f826f000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826807a770 con 0x7f826807be00 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.952+0000 7f8267e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826807a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.952+0000 7f8267e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826807a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41778/0 (socket says 172.21.15.131:41778) 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.952+0000 7f8267e00700 1 -- 172.21.15.131:0/1287958503 learned_addr learned my addr 172.21.15.131:0/1287958503 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.952+0000 7f8267e00700 1 -- 172.21.15.131:0/1287958503 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826807a8b0 con 0x7f826807be00 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.952+0000 7f8267e00700 1 --2- 172.21.15.131:0/1287958503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826807a230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f82580098a0 tx=0x7f8258009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7382102af1e06c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.953+0000 7f8266a00700 1 -- 172.21.15.131:0/1287958503 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8258004160 con 0x7f826807be00 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.953+0000 7f8266a00700 1 -- 172.21.15.131:0/1287958503 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f82580042c0 con 0x7f826807be00 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.953+0000 7f8266a00700 1 -- 172.21.15.131:0/1287958503 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8258011a00 con 0x7f826807be00 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.953+0000 7f826f000700 1 -- 172.21.15.131:0/1287958503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 msgr2=0x7f826807a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.953+0000 7f826f000700 1 --2- 172.21.15.131:0/1287958503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826807a230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f82580098a0 tx=0x7f8258009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.953+0000 7f826f000700 1 -- 172.21.15.131:0/1287958503 shutdown_connections 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.953+0000 7f826f000700 1 --2- 172.21.15.131:0/1287958503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826807a230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.953+0000 7f826f000700 1 -- 172.21.15.131:0/1287958503 >> 172.21.15.131:0/1287958503 conn(0x7f8268102830 msgr2=0x7f8268104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.954+0000 7f826f000700 1 -- 172.21.15.131:0/1287958503 shutdown_connections 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.954+0000 7f826f000700 1 -- 172.21.15.131:0/1287958503 wait complete. 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.955+0000 7f826f000700 1 Processor -- start 2024-05-10T05:25:39.524 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.955+0000 7f826f000700 1 -- start start 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.955+0000 7f826f000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.955+0000 7f826f000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826819dba0 con 0x7f826807be00 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.955+0000 7f8267e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.955+0000 7f8267e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41782/0 (socket says 172.21.15.131:41782) 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.955+0000 7f8267e00700 1 -- 172.21.15.131:0/2837876289 learned_addr learned my addr 172.21.15.131:0/2837876289 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.956+0000 7f8267e00700 1 -- 172.21.15.131:0/2837876289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258009550 con 0x7f826807be00 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.956+0000 7f8267e00700 1 --2- 172.21.15.131:0/2837876289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826819d660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8258009130 tx=0x7f8258011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.956+0000 7f8264c00700 1 -- 172.21.15.131:0/2837876289 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82580038b0 con 0x7f826807be00 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.956+0000 7f8264c00700 1 -- 172.21.15.131:0/2837876289 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8258003a10 con 0x7f826807be00 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.956+0000 7f826f000700 1 -- 172.21.15.131:0/2837876289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826819dda0 con 0x7f826807be00 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.956+0000 7f8264c00700 1 -- 172.21.15.131:0/2837876289 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8258019550 con 0x7f826807be00 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.956+0000 7f826f000700 1 -- 172.21.15.131:0/2837876289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826819e240 con 0x7f826807be00 2024-05-10T05:25:39.525 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.957+0000 7f826f000700 1 -- 172.21.15.131:0/2837876289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826810d600 con 0x7f826807be00 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.957+0000 7f8264c00700 1 -- 172.21.15.131:0/2837876289 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f8258019810 con 0x7f826807be00 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.958+0000 7f8264c00700 1 --2- 172.21.15.131:0/2837876289 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f824c038640 0x7f824c03ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.958+0000 7f8264c00700 1 -- 172.21.15.131:0/2837876289 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f825804c670 con 0x7f826807be00 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.958+0000 7f8267400700 1 --2- 172.21.15.131:0/2837876289 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f824c038640 0x7f824c03ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.959+0000 7f8267400700 1 --2- 172.21.15.131:0/2837876289 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f824c038640 0x7f824c03ab50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8254006fd0 tx=0x7f8254006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:38.963+0000 7f8264c00700 1 -- 172.21.15.131:0/2837876289 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8258019ac0 con 0x7f826807be00 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.163+0000 7f826f000700 1 -- 172.21.15.131:0/2837876289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f826810d840 con 0x7f826807be00 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.164+0000 7f8264c00700 1 -- 172.21.15.131:0/2837876289 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f8258018b40 con 0x7f826807be00 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.167+0000 7f826f000700 1 -- 172.21.15.131:0/2837876289 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f824c038640 msgr2=0x7f824c03ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.167+0000 7f826f000700 1 --2- 172.21.15.131:0/2837876289 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f824c038640 0x7f824c03ab50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8254006fd0 tx=0x7f8254006e40 comp rx=0 tx=0).stop 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.167+0000 7f826f000700 1 -- 172.21.15.131:0/2837876289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 msgr2=0x7f826819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.167+0000 7f826f000700 1 --2- 172.21.15.131:0/2837876289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826819d660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8258009130 tx=0x7f8258011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:39.526 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.167+0000 7f826f000700 1 -- 172.21.15.131:0/2837876289 shutdown_connections 2024-05-10T05:25:39.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.167+0000 7f826f000700 1 --2- 172.21.15.131:0/2837876289 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f824c038640 0x7f824c03ab50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:39.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.167+0000 7f826f000700 1 --2- 172.21.15.131:0/2837876289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826807be00 0x7f826819d660 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:39.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.167+0000 7f826f000700 1 -- 172.21.15.131:0/2837876289 >> 172.21.15.131:0/2837876289 conn(0x7f8268102830 msgr2=0x7f8268103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:39.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.167+0000 7f826f000700 1 -- 172.21.15.131:0/2837876289 shutdown_connections 2024-05-10T05:25:39.527 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.167+0000 7f826f000700 1 -- 172.21.15.131:0/2837876289 wait complete. 2024-05-10T05:25:39.527 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling cephadm module... 2024-05-10T05:25:40.689 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:40 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/274497733' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-10T05:25:40.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:40 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: ignoring --setuser ceph since I am not root 2024-05-10T05:25:40.689 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:40 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: ignoring --setgroup ceph since I am not root 2024-05-10T05:25:40.694 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.968+0000 7f7e32c00700 1 Processor -- start 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.968+0000 7f7e32c00700 1 -- start start 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.968+0000 7f7e32c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c07a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.968+0000 7f7e32c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c07a770 con 0x7f7e2c07be00 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.969+0000 7f7e2be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c07a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.969+0000 7f7e2be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c07a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41798/0 (socket says 172.21.15.131:41798) 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.969+0000 7f7e2be00700 1 -- 172.21.15.131:0/3198355290 learned_addr learned my addr 172.21.15.131:0/3198355290 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.969+0000 7f7e2be00700 1 -- 172.21.15.131:0/3198355290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c07a8b0 con 0x7f7e2c07be00 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.970+0000 7f7e2be00700 1 --2- 172.21.15.131:0/3198355290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c07a230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0098a0 tx=0x7f7e1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80820586ae6bbbbd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.970+0000 7f7e2aa00700 1 -- 172.21.15.131:0/3198355290 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e1c004160 con 0x7f7e2c07be00 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.970+0000 7f7e2aa00700 1 -- 172.21.15.131:0/3198355290 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e1c0042c0 con 0x7f7e2c07be00 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.970+0000 7f7e2aa00700 1 -- 172.21.15.131:0/3198355290 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e1c011a00 con 0x7f7e2c07be00 2024-05-10T05:25:40.695 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.971+0000 7f7e32c00700 1 -- 172.21.15.131:0/3198355290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 msgr2=0x7f7e2c07a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.971+0000 7f7e32c00700 1 --2- 172.21.15.131:0/3198355290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c07a230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0098a0 tx=0x7f7e1c009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.971+0000 7f7e32c00700 1 -- 172.21.15.131:0/3198355290 shutdown_connections 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.971+0000 7f7e32c00700 1 --2- 172.21.15.131:0/3198355290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c07a230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.971+0000 7f7e32c00700 1 -- 172.21.15.131:0/3198355290 >> 172.21.15.131:0/3198355290 conn(0x7f7e2c102840 msgr2=0x7f7e2c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.971+0000 7f7e32c00700 1 -- 172.21.15.131:0/3198355290 shutdown_connections 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.971+0000 7f7e32c00700 1 -- 172.21.15.131:0/3198355290 wait complete. 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.972+0000 7f7e32c00700 1 Processor -- start 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.972+0000 7f7e32c00700 1 -- start start 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.972+0000 7f7e32c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.972+0000 7f7e32c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c19dbb0 con 0x7f7e2c07be00 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.973+0000 7f7e2be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.973+0000 7f7e2be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41806/0 (socket says 172.21.15.131:41806) 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.973+0000 7f7e2be00700 1 -- 172.21.15.131:0/274497733 learned_addr learned my addr 172.21.15.131:0/274497733 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:40.696 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.973+0000 7f7e2be00700 1 -- 172.21.15.131:0/274497733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1c009550 con 0x7f7e2c07be00 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.973+0000 7f7e2be00700 1 --2- 172.21.15.131:0/274497733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c19d670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c011710 tx=0x7f7e1c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.973+0000 7f7e28c00700 1 -- 172.21.15.131:0/274497733 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e1c003aa0 con 0x7f7e2c07be00 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.973+0000 7f7e28c00700 1 -- 172.21.15.131:0/274497733 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e1c003c00 con 0x7f7e2c07be00 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.973+0000 7f7e28c00700 1 -- 172.21.15.131:0/274497733 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e1c024800 con 0x7f7e2c07be00 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.973+0000 7f7e32c00700 1 -- 172.21.15.131:0/274497733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c19ddb0 con 0x7f7e2c07be00 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.974+0000 7f7e32c00700 1 -- 172.21.15.131:0/274497733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c19e250 con 0x7f7e2c07be00 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.975+0000 7f7e32c00700 1 -- 172.21.15.131:0/274497733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c197280 con 0x7f7e2c07be00 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.975+0000 7f7e28c00700 1 -- 172.21.15.131:0/274497733 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f7e1c024a30 con 0x7f7e2c07be00 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.975+0000 7f7e28c00700 1 --2- 172.21.15.131:0/274497733 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7e100386b0 0x7f7e1003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.975+0000 7f7e28c00700 1 -- 172.21.15.131:0/274497733 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7e1c04c460 con 0x7f7e2c07be00 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.976+0000 7f7e2b400700 1 --2- 172.21.15.131:0/274497733 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7e100386b0 0x7f7e1003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.976+0000 7f7e2b400700 1 --2- 172.21.15.131:0/274497733 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7e100386b0 0x7f7e1003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006fd0 tx=0x7f7e18006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:40.697 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:39.981+0000 7f7e28c00700 1 -- 172.21.15.131:0/274497733 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7e1c010790 con 0x7f7e2c07be00 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.236+0000 7f7e32c00700 1 -- 172.21.15.131:0/274497733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f7e2c10d610 con 0x7f7e2c07be00 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.343+0000 7f7e28c00700 1 -- 172.21.15.131:0/274497733 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f7e1c01abd0 con 0x7f7e2c07be00 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.345+0000 7f7e28c00700 1 -- 172.21.15.131:0/274497733 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f7e1c04c1c0 con 0x7f7e2c07be00 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.348+0000 7f7e32c00700 1 -- 172.21.15.131:0/274497733 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7e100386b0 msgr2=0x7f7e1003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.348+0000 7f7e32c00700 1 --2- 172.21.15.131:0/274497733 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7e100386b0 0x7f7e1003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006fd0 tx=0x7f7e18006e40 comp rx=0 tx=0).stop 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.348+0000 7f7e32c00700 1 -- 172.21.15.131:0/274497733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 msgr2=0x7f7e2c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.348+0000 7f7e32c00700 1 --2- 172.21.15.131:0/274497733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c19d670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c011710 tx=0x7f7e1c011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.348+0000 7f7e32c00700 1 -- 172.21.15.131:0/274497733 shutdown_connections 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.348+0000 7f7e32c00700 1 --2- 172.21.15.131:0/274497733 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f7e100386b0 0x7f7e1003abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.348+0000 7f7e32c00700 1 --2- 172.21.15.131:0/274497733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e2c07be00 0x7f7e2c19d670 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.348+0000 7f7e32c00700 1 -- 172.21.15.131:0/274497733 >> 172.21.15.131:0/274497733 conn(0x7f7e2c102840 msgr2=0x7f7e2c103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.348+0000 7f7e32c00700 1 -- 172.21.15.131:0/274497733 shutdown_connections 2024-05-10T05:25:40.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:40.348+0000 7f7e32c00700 1 -- 172.21.15.131:0/274497733 wait complete. 2024-05-10T05:25:41.344 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:41 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:41.076+0000 7faf83787200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T05:25:41.344 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:41 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:41.203+0000 7faf83787200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T05:25:41.344 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:41 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:41.322+0000 7faf83787200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T05:25:41.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:41 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/274497733' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-10T05:25:41.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:41 smithi131 ceph-mon[37264]: mgrmap e5: x(active, since 6s) 2024-05-10T05:25:41.697 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:41 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:41.436+0000 7faf83787200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T05:25:41.854 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-10T05:25:41.854 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-10T05:25:41.854 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T05:25:41.854 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T05:25:41.854 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T05:25:41.854 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-10T05:25:41.854 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.223+0000 7f3b79000700 1 Processor -- start 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.223+0000 7f3b79000700 1 -- start start 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.223+0000 7f3b79000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7407a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.223+0000 7f3b79000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7407a770 con 0x7f3b7407be00 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.224+0000 7f3b73400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7407a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.224+0000 7f3b73400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7407a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41822/0 (socket says 172.21.15.131:41822) 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.224+0000 7f3b73400700 1 -- 172.21.15.131:0/3551670132 learned_addr learned my addr 172.21.15.131:0/3551670132 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.224+0000 7f3b73400700 1 -- 172.21.15.131:0/3551670132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7407a8b0 con 0x7f3b7407be00 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.224+0000 7f3b73400700 1 --2- 172.21.15.131:0/3551670132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7407a230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3b640098a0 tx=0x7f3b64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c66567553d4e650 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.225+0000 7f3b72000700 1 -- 172.21.15.131:0/3551670132 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b64004160 con 0x7f3b7407be00 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.225+0000 7f3b72000700 1 -- 172.21.15.131:0/3551670132 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b640042c0 con 0x7f3b7407be00 2024-05-10T05:25:41.855 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.225+0000 7f3b72000700 1 -- 172.21.15.131:0/3551670132 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b64011a00 con 0x7f3b7407be00 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.225+0000 7f3b79000700 1 -- 172.21.15.131:0/3551670132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 msgr2=0x7f3b7407a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.225+0000 7f3b79000700 1 --2- 172.21.15.131:0/3551670132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7407a230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3b640098a0 tx=0x7f3b64009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.226+0000 7f3b79000700 1 -- 172.21.15.131:0/3551670132 shutdown_connections 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.226+0000 7f3b79000700 1 --2- 172.21.15.131:0/3551670132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7407a230 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.226+0000 7f3b79000700 1 -- 172.21.15.131:0/3551670132 >> 172.21.15.131:0/3551670132 conn(0x7f3b741026b0 msgr2=0x7f3b74104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.226+0000 7f3b79000700 1 -- 172.21.15.131:0/3551670132 shutdown_connections 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.226+0000 7f3b79000700 1 -- 172.21.15.131:0/3551670132 wait complete. 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.227+0000 7f3b79000700 1 Processor -- start 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.227+0000 7f3b79000700 1 -- start start 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.227+0000 7f3b79000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:41.856 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.227+0000 7f3b79000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7419da90 con 0x7f3b7407be00 2024-05-10T05:25:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.228+0000 7f3b73400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.228+0000 7f3b73400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7419d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41834/0 (socket says 172.21.15.131:41834) 2024-05-10T05:25:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.228+0000 7f3b73400700 1 -- 172.21.15.131:0/233383472 learned_addr learned my addr 172.21.15.131:0/233383472 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.228+0000 7f3b73400700 1 -- 172.21.15.131:0/233383472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b64009550 con 0x7f3b7407be00 2024-05-10T05:25:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.229+0000 7f3b73400700 1 --2- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7419d550 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3b64011710 tx=0x7f3b64011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.229+0000 7f3b6be00700 1 -- 172.21.15.131:0/233383472 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b64003af0 con 0x7f3b7407be00 2024-05-10T05:25:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.229+0000 7f3b6be00700 1 -- 172.21.15.131:0/233383472 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b64003c50 con 0x7f3b7407be00 2024-05-10T05:25:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.229+0000 7f3b79000700 1 -- 172.21.15.131:0/233383472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7419dc90 con 0x7f3b7407be00 2024-05-10T05:25:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.229+0000 7f3b6be00700 1 -- 172.21.15.131:0/233383472 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b64024780 con 0x7f3b7407be00 2024-05-10T05:25:41.857 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.229+0000 7f3b79000700 1 -- 172.21.15.131:0/233383472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7419e130 con 0x7f3b7407be00 2024-05-10T05:25:41.858 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.230+0000 7f3b79000700 1 -- 172.21.15.131:0/233383472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b741970f0 con 0x7f3b7407be00 2024-05-10T05:25:41.858 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.230+0000 7f3b6be00700 1 -- 172.21.15.131:0/233383472 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3b64024970 con 0x7f3b7407be00 2024-05-10T05:25:41.858 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.231+0000 7f3b6be00700 1 --2- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f3b58038750 0x7f3b5803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:41.858 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.231+0000 7f3b6be00700 1 -- 172.21.15.131:0/233383472 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3b6404c680 con 0x7f3b7407be00 2024-05-10T05:25:41.858 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.231+0000 7f3b72a00700 1 -- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f3b58038750 msgr2=0x7f3b5803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2225571046 2024-05-10T05:25:41.858 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.231+0000 7f3b72a00700 1 --2- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f3b58038750 0x7f3b5803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T05:25:41.858 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.236+0000 7f3b6be00700 1 -- 172.21.15.131:0/233383472 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3b64018d20 con 0x7f3b7407be00 2024-05-10T05:25:41.858 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.432+0000 7f3b72a00700 1 -- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f3b58038750 msgr2=0x7f3b5803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2225571046 2024-05-10T05:25:41.858 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.432+0000 7f3b72a00700 1 --2- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f3b58038750 0x7f3b5803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T05:25:41.859 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.503+0000 7f3b79000700 1 -- 172.21.15.131:0/233383472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3b7410d6c0 con 0x7f3b7407be00 2024-05-10T05:25:41.859 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.503+0000 7f3b6be00700 1 -- 172.21.15.131:0/233383472 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3b64018b40 con 0x7f3b7407be00 2024-05-10T05:25:41.859 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.506+0000 7f3b79000700 1 -- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f3b58038750 msgr2=0x7f3b5803ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:25:41.859 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.506+0000 7f3b79000700 1 --2- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f3b58038750 0x7f3b5803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:41.859 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.506+0000 7f3b79000700 1 -- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 msgr2=0x7f3b7419d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:41.859 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.506+0000 7f3b79000700 1 --2- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7419d550 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3b64011710 tx=0x7f3b64011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:41.859 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.506+0000 7f3b79000700 1 -- 172.21.15.131:0/233383472 shutdown_connections 2024-05-10T05:25:41.859 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.506+0000 7f3b79000700 1 --2- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7f3b58038750 0x7f3b5803ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:41.859 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.506+0000 7f3b79000700 1 --2- 172.21.15.131:0/233383472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7407be00 0x7f3b7419d550 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:41.860 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.506+0000 7f3b79000700 1 -- 172.21.15.131:0/233383472 >> 172.21.15.131:0/233383472 conn(0x7f3b741026b0 msgr2=0x7f3b741032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:41.860 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.506+0000 7f3b79000700 1 -- 172.21.15.131:0/233383472 shutdown_connections 2024-05-10T05:25:41.860 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:41.506+0000 7f3b79000700 1 -- 172.21.15.131:0/233383472 wait complete. 2024-05-10T05:25:41.860 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-05-10T05:25:41.860 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 5... 2024-05-10T05:25:42.634 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:42 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/233383472' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T05:25:42.634 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:42 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:42.310+0000 7faf83787200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T05:25:42.634 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:42 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:42.633+0000 7faf83787200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T05:25:42.889 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:42 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:42.748+0000 7faf83787200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T05:25:42.889 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:42 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:42.888+0000 7faf83787200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T05:25:43.212 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:43 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:43.086+0000 7faf83787200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T05:25:43.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:43 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:43.253+0000 7faf83787200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T05:25:43.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:43 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:43.364+0000 7faf83787200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T05:25:44.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:43 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:43.832+0000 7faf83787200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T05:25:44.585 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:44 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:44.331+0000 7faf83787200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T05:25:44.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:44 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:44.584+0000 7faf83787200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T05:25:44.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:44 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:44.707+0000 7faf83787200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T05:25:45.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:45 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:45.664+0000 7faf83787200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T05:25:45.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:45 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:45.791+0000 7faf83787200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T05:25:46.460 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:45 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:45.984+0000 7faf83787200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T05:25:47.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:46 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:46.798+0000 7faf83787200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T05:25:47.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:47 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:47.040+0000 7faf83787200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T05:25:47.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:47 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:47.296+0000 7faf83787200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T05:25:48.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:48 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:48.438+0000 7faf83787200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T05:25:49.182 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:48 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:48.810+0000 7faf83787200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T05:25:49.183 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:48 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:48.928+0000 7faf83787200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T05:25:49.183 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:49 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:25:49.181+0000 7faf83787200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T05:25:49.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: Active manager daemon x restarted 2024-05-10T05:25:49.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: Activating manager daemon x 2024-05-10T05:25:49.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: osdmap e2: 0 total, 0 up, 0 in 2024-05-10T05:25:49.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: mgrmap e6: x(active, starting, since 0.00493329s) 2024-05-10T05:25:49.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:25:49.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T05:25:49.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T05:25:49.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T05:25:49.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T05:25:49.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: Manager daemon x is now available 2024-05-10T05:25:49.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:25:49.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:25:49.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:49 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:25:50.576 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-10T05:25:50.576 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-10T05:25:50.576 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T05:25:50.576 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-10T05:25:50.576 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.358+0000 7ff195000700 1 Processor -- start 2024-05-10T05:25:50.576 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.358+0000 7ff195000700 1 -- start start 2024-05-10T05:25:50.576 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.359+0000 7ff195000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19007a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:50.576 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.359+0000 7ff195000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19007a820 con 0x7ff19007be20 2024-05-10T05:25:50.577 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.359+0000 7ff18de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19007a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:50.577 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.359+0000 7ff18de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19007a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41848/0 (socket says 172.21.15.131:41848) 2024-05-10T05:25:50.577 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.359+0000 7ff18de00700 1 -- 172.21.15.131:0/12897488 learned_addr learned my addr 172.21.15.131:0/12897488 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:50.577 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.359+0000 7ff18de00700 1 -- 172.21.15.131:0/12897488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19007a960 con 0x7ff19007be20 2024-05-10T05:25:50.577 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.360+0000 7ff18de00700 1 --2- 172.21.15.131:0/12897488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19007a250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff1800098a0 tx=0x7ff180009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121163fe9f327740 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:50.577 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.360+0000 7ff18ca00700 1 -- 172.21.15.131:0/12897488 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff180004160 con 0x7ff19007be20 2024-05-10T05:25:50.577 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.360+0000 7ff18ca00700 1 -- 172.21.15.131:0/12897488 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1800042c0 con 0x7ff19007be20 2024-05-10T05:25:50.577 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.360+0000 7ff18ca00700 1 -- 172.21.15.131:0/12897488 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff180011a00 con 0x7ff19007be20 2024-05-10T05:25:50.577 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.361+0000 7ff195000700 1 -- 172.21.15.131:0/12897488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 msgr2=0x7ff19007a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:50.577 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.361+0000 7ff195000700 1 --2- 172.21.15.131:0/12897488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19007a250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff1800098a0 tx=0x7ff180009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.361+0000 7ff195000700 1 -- 172.21.15.131:0/12897488 shutdown_connections 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.361+0000 7ff195000700 1 --2- 172.21.15.131:0/12897488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19007a250 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.361+0000 7ff195000700 1 -- 172.21.15.131:0/12897488 >> 172.21.15.131:0/12897488 conn(0x7ff190102840 msgr2=0x7ff190104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.361+0000 7ff195000700 1 -- 172.21.15.131:0/12897488 shutdown_connections 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.361+0000 7ff195000700 1 -- 172.21.15.131:0/12897488 wait complete. 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.362+0000 7ff195000700 1 Processor -- start 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.362+0000 7ff195000700 1 -- start start 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.363+0000 7ff195000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.363+0000 7ff195000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19019dba0 con 0x7ff19007be20 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.363+0000 7ff18de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.363+0000 7ff18de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41850/0 (socket says 172.21.15.131:41850) 2024-05-10T05:25:50.578 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.363+0000 7ff18de00700 1 -- 172.21.15.131:0/1664941237 learned_addr learned my addr 172.21.15.131:0/1664941237 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.363+0000 7ff18de00700 1 -- 172.21.15.131:0/1664941237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180009550 con 0x7ff19007be20 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.364+0000 7ff18de00700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19019d660 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff180011710 tx=0x7ff180011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.364+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff180003aa0 con 0x7ff19007be20 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.364+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff180003c00 con 0x7ff19007be20 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.364+0000 7ff195000700 1 -- 172.21.15.131:0/1664941237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19019dda0 con 0x7ff19007be20 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.364+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff180024800 con 0x7ff19007be20 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.364+0000 7ff195000700 1 -- 172.21.15.131:0/1664941237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19019e240 con 0x7ff19007be20 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.365+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7ff180024960 con 0x7ff19007be20 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.366+0000 7ff186a00700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 0x7ff17403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.366+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 --> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff17403b370 con 0x7ff174038750 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.366+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff18004c370 con 0x7ff19007be20 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.366+0000 7ff18d400700 1 -- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 msgr2=0x7ff17403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2225571046 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.366+0000 7ff18d400700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 0x7ff17403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T05:25:50.579 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.567+0000 7ff18d400700 1 -- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 msgr2=0x7ff17403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2225571046 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.567+0000 7ff18d400700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 0x7ff17403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.968+0000 7ff18d400700 1 -- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 msgr2=0x7ff17403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2225571046 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:42.968+0000 7ff18d400700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 0x7ff17403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:43.769+0000 7ff18d400700 1 -- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 msgr2=0x7ff17403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2225571046 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:43.769+0000 7ff18d400700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 0x7ff17403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:45.371+0000 7ff18d400700 1 -- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 msgr2=0x7ff17403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2225571046 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:45.371+0000 7ff18d400700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 0x7ff17403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:48.574+0000 7ff18d400700 1 -- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 msgr2=0x7ff17403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2225571046 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:48.574+0000 7ff18d400700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 0x7ff17403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:49.186+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7ff18002d960 con 0x7ff19007be20 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:49.186+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 msgr2=0x7ff17403ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:49.186+0000 7ff186a00700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 0x7ff17403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:50.580 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.188+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7ff180016cc0 con 0x7ff19007be20 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.188+0000 7ff186a00700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7ff17403bfe0 0x7ff17403e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.188+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 --> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff17403b370 con 0x7ff17403bfe0 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.189+0000 7ff18d400700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7ff17403bfe0 0x7ff17403e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.190+0000 7ff18d400700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7ff17403bfe0 0x7ff17403e400 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff17c003a10 tx=0x7ff17c0092b0 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.191+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 <== mgr.14122 v2:172.21.15.131:6800/3754674139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7ff17403b370 con 0x7ff17403bfe0 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.198+0000 7ff195000700 1 -- 172.21.15.131:0/1664941237 --> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff19010d600 con 0x7ff17403bfe0 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.198+0000 7ff186a00700 1 -- 172.21.15.131:0/1664941237 <== mgr.14122 v2:172.21.15.131:6800/3754674139 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff19010d600 con 0x7ff17403bfe0 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 -- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7ff17403bfe0 msgr2=0x7ff17403e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7ff17403bfe0 0x7ff17403e400 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff17c003a10 tx=0x7ff17c0092b0 comp rx=0 tx=0).stop 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 -- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 msgr2=0x7ff19019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19019d660 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff180011710 tx=0x7ff180011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:50.581 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 -- 172.21.15.131:0/1664941237 shutdown_connections 2024-05-10T05:25:50.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7ff17403bfe0 0x7ff17403e400 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:50.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff19007be20 0x7ff19019d660 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:50.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 --2- 172.21.15.131:0/1664941237 >> [v2:172.21.15.131:6800/2225571046,v1:172.21.15.131:6801/2225571046] conn(0x7ff174038750 0x7ff17403ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:50.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 -- 172.21.15.131:0/1664941237 >> 172.21.15.131:0/1664941237 conn(0x7ff190102840 msgr2=0x7ff190103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:50.582 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 -- 172.21.15.131:0/1664941237 shutdown_connections 2024-05-10T05:25:50.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:50 smithi131 ceph-mon[37264]: Found migration_current of "None". Setting to last migration. 2024-05-10T05:25:50.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:50 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:25:50.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:50 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T05:25:50.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:50 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T05:25:50.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:50 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:25:50.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:50 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:25:50.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:50 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:25:50.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:50 smithi131 ceph-mon[37264]: mgrmap e7: x(active, since 1.00801s) 2024-05-10T05:25:50.584 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:50.199+0000 7ff195000700 1 -- 172.21.15.131:0/1664941237 wait complete. 2024-05-10T05:25:50.584 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 5 is available 2024-05-10T05:25:50.584 INFO:teuthology.orchestra.run.smithi131.stdout:Setting orchestrator backend to cephadm... 2024-05-10T05:25:51.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:51 smithi131 ceph-mon[37264]: [10/May/2024:05:25:49] ENGINE Bus STARTING 2024-05-10T05:25:51.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:51 smithi131 ceph-mon[37264]: [10/May/2024:05:25:49] ENGINE Serving on http://172.21.15.131:8765 2024-05-10T05:25:51.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:51 smithi131 ceph-mon[37264]: [10/May/2024:05:25:50] ENGINE Serving on https://172.21.15.131:7150 2024-05-10T05:25:51.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:51 smithi131 ceph-mon[37264]: [10/May/2024:05:25:50] ENGINE Bus STARTED 2024-05-10T05:25:51.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:51 smithi131 ceph-mon[37264]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T05:25:51.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:51 smithi131 ceph-mon[37264]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T05:25:51.675 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.074+0000 7f1974000700 1 Processor -- start 2024-05-10T05:25:51.675 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.075+0000 7f1974000700 1 -- start start 2024-05-10T05:25:51.675 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.075+0000 7f1974000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c106540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:51.675 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.075+0000 7f1974000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c106b30 con 0x7f196c106130 2024-05-10T05:25:51.675 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.075+0000 7f1971600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c106540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:51.675 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.075+0000 7f1971600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c106540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56622/0 (socket says 172.21.15.131:56622) 2024-05-10T05:25:51.675 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.075+0000 7f1971600700 1 -- 172.21.15.131:0/2388495587 learned_addr learned my addr 172.21.15.131:0/2388495587 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:51.675 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.076+0000 7f1971600700 1 -- 172.21.15.131:0/2388495587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c107300 con 0x7f196c106130 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.076+0000 7f1971600700 1 --2- 172.21.15.131:0/2388495587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c106540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19600098a0 tx=0x7f1960009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b67d9f6f08fba58 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.076+0000 7f196be00700 1 -- 172.21.15.131:0/2388495587 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1960004160 con 0x7f196c106130 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.076+0000 7f196be00700 1 -- 172.21.15.131:0/2388495587 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f19600042c0 con 0x7f196c106130 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.076+0000 7f196be00700 1 -- 172.21.15.131:0/2388495587 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1960011a00 con 0x7f196c106130 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.077+0000 7f1974000700 1 -- 172.21.15.131:0/2388495587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 msgr2=0x7f196c106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.077+0000 7f1974000700 1 --2- 172.21.15.131:0/2388495587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c106540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19600098a0 tx=0x7f1960009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.077+0000 7f1974000700 1 -- 172.21.15.131:0/2388495587 shutdown_connections 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.077+0000 7f1974000700 1 --2- 172.21.15.131:0/2388495587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c106540 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.077+0000 7f1974000700 1 -- 172.21.15.131:0/2388495587 >> 172.21.15.131:0/2388495587 conn(0x7f196c101790 msgr2=0x7f196c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.077+0000 7f1974000700 1 -- 172.21.15.131:0/2388495587 shutdown_connections 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.077+0000 7f1974000700 1 -- 172.21.15.131:0/2388495587 wait complete. 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.079+0000 7f1974000700 1 Processor -- start 2024-05-10T05:25:51.676 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.079+0000 7f1974000700 1 -- start start 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.079+0000 7f1974000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.079+0000 7f1974000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c19a8d0 con 0x7f196c106130 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.080+0000 7f1971600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.080+0000 7f1971600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c19a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56630/0 (socket says 172.21.15.131:56630) 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.080+0000 7f1971600700 1 -- 172.21.15.131:0/44016540 learned_addr learned my addr 172.21.15.131:0/44016540 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.080+0000 7f1971600700 1 -- 172.21.15.131:0/44016540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960009550 con 0x7f196c106130 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.080+0000 7f1971600700 1 --2- 172.21.15.131:0/44016540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c19a390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1960011a20 tx=0x7f1960011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.080+0000 7f196a000700 1 -- 172.21.15.131:0/44016540 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1960003aa0 con 0x7f196c106130 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.080+0000 7f196a000700 1 -- 172.21.15.131:0/44016540 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1960003c00 con 0x7f196c106130 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.080+0000 7f1974000700 1 -- 172.21.15.131:0/44016540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c19aa70 con 0x7f196c106130 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.080+0000 7f196a000700 1 -- 172.21.15.131:0/44016540 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19600196a0 con 0x7f196c106130 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.081+0000 7f1974000700 1 -- 172.21.15.131:0/44016540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c19af70 con 0x7f196c106130 2024-05-10T05:25:51.677 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.082+0000 7f196a000700 1 -- 172.21.15.131:0/44016540 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f1960019920 con 0x7f196c106130 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.082+0000 7f1974000700 1 -- 172.21.15.131:0/44016540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c10a530 con 0x7f196c106130 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.082+0000 7f196a000700 1 --2- 172.21.15.131:0/44016540 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f19540385d0 0x7f195403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.082+0000 7f196a000700 1 -- 172.21.15.131:0/44016540 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f196004c930 con 0x7f196c106130 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.082+0000 7f1970c00700 1 --2- 172.21.15.131:0/44016540 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f19540385d0 0x7f195403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.083+0000 7f1970c00700 1 --2- 172.21.15.131:0/44016540 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f19540385d0 0x7f195403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1958006fd0 tx=0x7f1958006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.088+0000 7f196a000700 1 -- 172.21.15.131:0/44016540 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1960018db0 con 0x7f196c106130 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.240+0000 7f196a000700 1 -- 172.21.15.131:0/44016540 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f196000b900 con 0x7f196c106130 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.309+0000 7f1974000700 1 -- 172.21.15.131:0/44016540 --> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f196c102950 con 0x7f19540385d0 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.323+0000 7f196a000700 1 -- 172.21.15.131:0/44016540 <== mgr.14122 v2:172.21.15.131:6800/3754674139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f196c102950 con 0x7f19540385d0 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.325+0000 7f1974000700 1 -- 172.21.15.131:0/44016540 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f19540385d0 msgr2=0x7f195403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.325+0000 7f1974000700 1 --2- 172.21.15.131:0/44016540 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f19540385d0 0x7f195403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1958006fd0 tx=0x7f1958006e40 comp rx=0 tx=0).stop 2024-05-10T05:25:51.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.325+0000 7f1974000700 1 -- 172.21.15.131:0/44016540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 msgr2=0x7f196c19a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:51.679 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.325+0000 7f1974000700 1 --2- 172.21.15.131:0/44016540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c19a390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1960011a20 tx=0x7f1960011b00 comp rx=0 tx=0).stop 2024-05-10T05:25:51.679 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.325+0000 7f1974000700 1 -- 172.21.15.131:0/44016540 shutdown_connections 2024-05-10T05:25:51.679 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.325+0000 7f1974000700 1 --2- 172.21.15.131:0/44016540 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f19540385d0 0x7f195403aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:51.679 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.325+0000 7f1974000700 1 --2- 172.21.15.131:0/44016540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f196c106130 0x7f196c19a390 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:51.679 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.325+0000 7f1974000700 1 -- 172.21.15.131:0/44016540 >> 172.21.15.131:0/44016540 conn(0x7f196c101790 msgr2=0x7f196c1021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:51.679 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.325+0000 7f1974000700 1 -- 172.21.15.131:0/44016540 shutdown_connections 2024-05-10T05:25:51.679 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:51.326+0000 7f1974000700 1 -- 172.21.15.131:0/44016540 wait complete. 2024-05-10T05:25:52.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:52 smithi131 ceph-mon[37264]: mgrmap e8: x(active, since 2s) 2024-05-10T05:25:52.553 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:52 smithi131 ceph-mon[37264]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:25:52.554 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:52 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:25:52.554 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:52 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:25:52.707 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-10T05:25:52.707 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.141+0000 7f1206000700 1 Processor -- start 2024-05-10T05:25:52.707 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.141+0000 7f1206000700 1 -- start start 2024-05-10T05:25:52.707 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.141+0000 7f1206000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f1200107630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:52.708 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.141+0000 7f1206000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200107c20 con 0x7f1200107220 2024-05-10T05:25:52.708 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.141+0000 7f11ff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f1200107630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:52.708 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.142+0000 7f11ff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f1200107630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56632/0 (socket says 172.21.15.131:56632) 2024-05-10T05:25:52.708 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.142+0000 7f11ff400700 1 -- 172.21.15.131:0/811772358 learned_addr learned my addr 172.21.15.131:0/811772358 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:52.708 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.142+0000 7f11ff400700 1 -- 172.21.15.131:0/811772358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12001083f0 con 0x7f1200107220 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.142+0000 7f11ff400700 1 --2- 172.21.15.131:0/811772358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f1200107630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11f00098a0 tx=0x7f11f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e481846e5441cb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.142+0000 7f11fe000700 1 -- 172.21.15.131:0/811772358 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11f0004160 con 0x7f1200107220 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.143+0000 7f11fe000700 1 -- 172.21.15.131:0/811772358 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11f00042c0 con 0x7f1200107220 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.143+0000 7f11fe000700 1 -- 172.21.15.131:0/811772358 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11f0011a00 con 0x7f1200107220 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.143+0000 7f1206000700 1 -- 172.21.15.131:0/811772358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 msgr2=0x7f1200107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.143+0000 7f1206000700 1 --2- 172.21.15.131:0/811772358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f1200107630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11f00098a0 tx=0x7f11f0009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.143+0000 7f1206000700 1 -- 172.21.15.131:0/811772358 shutdown_connections 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.143+0000 7f1206000700 1 --2- 172.21.15.131:0/811772358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f1200107630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.143+0000 7f1206000700 1 -- 172.21.15.131:0/811772358 >> 172.21.15.131:0/811772358 conn(0x7f1200102840 msgr2=0x7f1200104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.143+0000 7f1206000700 1 -- 172.21.15.131:0/811772358 shutdown_connections 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.143+0000 7f1206000700 1 -- 172.21.15.131:0/811772358 wait complete. 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.144+0000 7f1206000700 1 Processor -- start 2024-05-10T05:25:52.709 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.145+0000 7f1206000700 1 -- start start 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.145+0000 7f1206000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f120019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.145+0000 7f1206000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120019db90 con 0x7f1200107220 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.145+0000 7f11ff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f120019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.145+0000 7f11ff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f120019d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56640/0 (socket says 172.21.15.131:56640) 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.145+0000 7f11ff400700 1 -- 172.21.15.131:0/676809450 learned_addr learned my addr 172.21.15.131:0/676809450 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.146+0000 7f11ff400700 1 -- 172.21.15.131:0/676809450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0009550 con 0x7f1200107220 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.146+0000 7f11ff400700 1 --2- 172.21.15.131:0/676809450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f120019d650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f11f0011a20 tx=0x7f11f0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.146+0000 7f11f7e00700 1 -- 172.21.15.131:0/676809450 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11f0003aa0 con 0x7f1200107220 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.146+0000 7f11f7e00700 1 -- 172.21.15.131:0/676809450 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11f0003c00 con 0x7f1200107220 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.146+0000 7f11f7e00700 1 -- 172.21.15.131:0/676809450 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11f00196a0 con 0x7f1200107220 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.146+0000 7f1206000700 1 -- 172.21.15.131:0/676809450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120019dd90 con 0x7f1200107220 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.146+0000 7f1206000700 1 -- 172.21.15.131:0/676809450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120019e230 con 0x7f1200107220 2024-05-10T05:25:52.710 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.147+0000 7f1206000700 1 -- 172.21.15.131:0/676809450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120010b620 con 0x7f1200107220 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.147+0000 7f11f7e00700 1 -- 172.21.15.131:0/676809450 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f11f00198d0 con 0x7f1200107220 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.148+0000 7f11f7e00700 1 --2- 172.21.15.131:0/676809450 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f11e4038750 0x7f11e403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.148+0000 7f11f7e00700 1 -- 172.21.15.131:0/676809450 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f11f004cab0 con 0x7f1200107220 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.148+0000 7f11fea00700 1 --2- 172.21.15.131:0/676809450 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f11e4038750 0x7f11e403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.149+0000 7f11fea00700 1 --2- 172.21.15.131:0/676809450 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f11e4038750 0x7f11e403ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11ec006fd0 tx=0x7f11ec006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.154+0000 7f11f7e00700 1 -- 172.21.15.131:0/676809450 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f11f0018db0 con 0x7f1200107220 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.358+0000 7f1206000700 1 -- 172.21.15.131:0/676809450 --> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f12001a0a90 con 0x7f11e4038750 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.359+0000 7f11f7e00700 1 -- 172.21.15.131:0/676809450 <== mgr.14122 v2:172.21.15.131:6800/3754674139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f12001a0a90 con 0x7f11e4038750 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.361+0000 7f1206000700 1 -- 172.21.15.131:0/676809450 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f11e4038750 msgr2=0x7f11e403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.361+0000 7f1206000700 1 --2- 172.21.15.131:0/676809450 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f11e4038750 0x7f11e403ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11ec006fd0 tx=0x7f11ec006e40 comp rx=0 tx=0).stop 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.361+0000 7f1206000700 1 -- 172.21.15.131:0/676809450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 msgr2=0x7f120019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.361+0000 7f1206000700 1 --2- 172.21.15.131:0/676809450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f120019d650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f11f0011a20 tx=0x7f11f0011b00 comp rx=0 tx=0).stop 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.361+0000 7f1206000700 1 -- 172.21.15.131:0/676809450 shutdown_connections 2024-05-10T05:25:52.711 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.361+0000 7f1206000700 1 --2- 172.21.15.131:0/676809450 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f11e4038750 0x7f11e403ac60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:52.712 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.361+0000 7f1206000700 1 --2- 172.21.15.131:0/676809450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1200107220 0x7f120019d650 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:52.712 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.361+0000 7f1206000700 1 -- 172.21.15.131:0/676809450 >> 172.21.15.131:0/676809450 conn(0x7f1200102840 msgr2=0x7f12001032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:52.712 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.361+0000 7f1206000700 1 -- 172.21.15.131:0/676809450 shutdown_connections 2024-05-10T05:25:52.712 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:52.361+0000 7f1206000700 1 -- 172.21.15.131:0/676809450 wait complete. 2024-05-10T05:25:52.712 INFO:teuthology.orchestra.run.smithi131.stdout:Generating ssh key... 2024-05-10T05:25:53.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:53 smithi131 ceph-mon[37264]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:25:54.154 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: Generating public/private rsa key pair. 2024-05-10T05:25:54.154 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: Your identification has been saved in /tmp/tmpuvy29ot4/key. 2024-05-10T05:25:54.154 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: Your public key has been saved in /tmp/tmpuvy29ot4/key.pub. 2024-05-10T05:25:54.154 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: The key fingerprint is: 2024-05-10T05:25:54.154 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: SHA256:nxFgyg5Zbpi+t5INSq1eL6TzbppeZcOY3OXyFQK76FE ceph-6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:54.154 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: The key's randomart image is: 2024-05-10T05:25:54.154 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: +---[RSA 3072]----+ 2024-05-10T05:25:54.154 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: | o o | 2024-05-10T05:25:54.154 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: | B = . | 2024-05-10T05:25:54.155 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: | = E o o | 2024-05-10T05:25:54.155 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: | o @ + . o | 2024-05-10T05:25:54.155 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: | .B X S o | 2024-05-10T05:25:54.155 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: | ..+= + o o | 2024-05-10T05:25:54.155 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: | . ==+. . o | 2024-05-10T05:25:54.155 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: | =+=o.. | 2024-05-10T05:25:54.155 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: | o=*ooo | 2024-05-10T05:25:54.155 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:25:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: +----[SHA256]-----+ 2024-05-10T05:25:54.160 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.214+0000 7f032ec00700 1 Processor -- start 2024-05-10T05:25:54.160 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.214+0000 7f032ec00700 1 -- start start 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.214+0000 7f032ec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032807a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.214+0000 7f032ec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032807a770 con 0x7f032807be00 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.215+0000 7f032d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032807a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.215+0000 7f032d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032807a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40320/0 (socket says 172.21.15.131:40320) 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.215+0000 7f032d800700 1 -- 172.21.15.131:0/1068817039 learned_addr learned my addr 172.21.15.131:0/1068817039 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.215+0000 7f032d800700 1 -- 172.21.15.131:0/1068817039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032807a8b0 con 0x7f032807be00 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.215+0000 7f032d800700 1 --2- 172.21.15.131:0/1068817039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032807a230 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f031c0098a0 tx=0x7f031c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4875dca142357d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.216+0000 7f0327e00700 1 -- 172.21.15.131:0/1068817039 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f031c004160 con 0x7f032807be00 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.216+0000 7f0327e00700 1 -- 172.21.15.131:0/1068817039 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f031c0042c0 con 0x7f032807be00 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.216+0000 7f0327e00700 1 -- 172.21.15.131:0/1068817039 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f031c011a00 con 0x7f032807be00 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.216+0000 7f032ec00700 1 -- 172.21.15.131:0/1068817039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 msgr2=0x7f032807a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.216+0000 7f032ec00700 1 --2- 172.21.15.131:0/1068817039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032807a230 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f031c0098a0 tx=0x7f031c009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:54.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.216+0000 7f032ec00700 1 -- 172.21.15.131:0/1068817039 shutdown_connections 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.216+0000 7f032ec00700 1 --2- 172.21.15.131:0/1068817039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032807a230 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.216+0000 7f032ec00700 1 -- 172.21.15.131:0/1068817039 >> 172.21.15.131:0/1068817039 conn(0x7f03281026b0 msgr2=0x7f0328104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.217+0000 7f032ec00700 1 -- 172.21.15.131:0/1068817039 shutdown_connections 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.217+0000 7f032ec00700 1 -- 172.21.15.131:0/1068817039 wait complete. 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.218+0000 7f032ec00700 1 Processor -- start 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.218+0000 7f032ec00700 1 -- start start 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.218+0000 7f032ec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.218+0000 7f032ec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032819da90 con 0x7f032807be00 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.218+0000 7f032d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.218+0000 7f032d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032819d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40330/0 (socket says 172.21.15.131:40330) 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.218+0000 7f032d800700 1 -- 172.21.15.131:0/3837674745 learned_addr learned my addr 172.21.15.131:0/3837674745 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.219+0000 7f032d800700 1 -- 172.21.15.131:0/3837674745 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c009550 con 0x7f032807be00 2024-05-10T05:25:54.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.219+0000 7f032d800700 1 --2- 172.21.15.131:0/3837674745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032819d550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f031c009130 tx=0x7f031c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.219+0000 7f0326000700 1 -- 172.21.15.131:0/3837674745 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f031c0038b0 con 0x7f032807be00 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.219+0000 7f0326000700 1 -- 172.21.15.131:0/3837674745 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f031c003a10 con 0x7f032807be00 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.219+0000 7f032ec00700 1 -- 172.21.15.131:0/3837674745 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032819dc90 con 0x7f032807be00 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.219+0000 7f0326000700 1 -- 172.21.15.131:0/3837674745 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f031c019550 con 0x7f032807be00 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.219+0000 7f032ec00700 1 -- 172.21.15.131:0/3837674745 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032819e130 con 0x7f032807be00 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.220+0000 7f032ec00700 1 -- 172.21.15.131:0/3837674745 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03281970f0 con 0x7f032807be00 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.220+0000 7f0326000700 1 -- 172.21.15.131:0/3837674745 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f031c0197d0 con 0x7f032807be00 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.221+0000 7f0326000700 1 --2- 172.21.15.131:0/3837674745 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f03100386b0 0x7f031003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.221+0000 7f0326000700 1 -- 172.21.15.131:0/3837674745 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f031c04c7a0 con 0x7f032807be00 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.221+0000 7f032ce00700 1 --2- 172.21.15.131:0/3837674745 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f03100386b0 0x7f031003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.222+0000 7f032ce00700 1 --2- 172.21.15.131:0/3837674745 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f03100386b0 0x7f031003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0314006fd0 tx=0x7f0314006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.226+0000 7f0326000700 1 -- 172.21.15.131:0/3837674745 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f031c018b80 con 0x7f032807be00 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.431+0000 7f032ec00700 1 -- 172.21.15.131:0/3837674745 --> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f032802d010 con 0x7f03100386b0 2024-05-10T05:25:54.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.795+0000 7f0326000700 1 -- 172.21.15.131:0/3837674745 <== mgr.14122 v2:172.21.15.131:6800/3754674139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f032802d010 con 0x7f03100386b0 2024-05-10T05:25:54.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.798+0000 7f032ec00700 1 -- 172.21.15.131:0/3837674745 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f03100386b0 msgr2=0x7f031003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:54.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.798+0000 7f032ec00700 1 --2- 172.21.15.131:0/3837674745 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f03100386b0 0x7f031003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0314006fd0 tx=0x7f0314006e40 comp rx=0 tx=0).stop 2024-05-10T05:25:54.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.798+0000 7f032ec00700 1 -- 172.21.15.131:0/3837674745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 msgr2=0x7f032819d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:54.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.798+0000 7f032ec00700 1 --2- 172.21.15.131:0/3837674745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032819d550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f031c009130 tx=0x7f031c011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:54.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.798+0000 7f032ec00700 1 -- 172.21.15.131:0/3837674745 shutdown_connections 2024-05-10T05:25:54.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.798+0000 7f032ec00700 1 --2- 172.21.15.131:0/3837674745 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f03100386b0 0x7f031003abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:54.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.798+0000 7f032ec00700 1 --2- 172.21.15.131:0/3837674745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032807be00 0x7f032819d550 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:54.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.798+0000 7f032ec00700 1 -- 172.21.15.131:0/3837674745 >> 172.21.15.131:0/3837674745 conn(0x7f03281026b0 msgr2=0x7f03281032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:54.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.798+0000 7f032ec00700 1 -- 172.21.15.131:0/3837674745 shutdown_connections 2024-05-10T05:25:54.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:53.798+0000 7f032ec00700 1 -- 172.21.15.131:0/3837674745 wait complete. 2024-05-10T05:25:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:54 smithi131 ceph-mon[37264]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:25:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:54 smithi131 ceph-mon[37264]: Generating ssh key... 2024-05-10T05:25:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:54 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:25:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:54 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:25:55.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB/MuxBgoRT1qdg22IKrFYT6x3n5JtBRY8wljoMMQ/2yiP059gZUmhirjz7427QjaBM0+Afi/k8DE6Cs+Sgvdyl52G21FuC1rAVVASIv6N+Wn5SCCc1LRHHYHrYiqM5MTKOkyWqbOAP2u0gWopqIvD58U80XARmJTXwxRZj39hWowWO8OUemwrICzX+xjz7OTuUKFaL6hyzlNiSoGPawL97EqEGc6eYypc3F5WEPMscpiMNLlFxHQErHX0J6ap+31yyEQiQDY3o4KQj4DnLp3OXQaROYSDcXXTEt74gIUyvMOVCyoRxIDV8LqLTzw7v1dFiUGIGSy01iF/Jw0vPKBSuKgc5jm3MZinJ0I2IhI7z+OXGpahOlnZhM3Ya7RabRK1hT0LJ+Av51RboKe3tLakZcsb4OGzaK+4PirZIkTNvZjhyVghaAZvqljj7+lW1cSqkFoDS4MPR6+Jw7ukPRyozZQmXh0D9HTemVVvMu46xsNobzqtv3bMAazLDf+Mxc= ceph-6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:25:55.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.647+0000 7f9f2d800700 1 Processor -- start 2024-05-10T05:25:55.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.647+0000 7f9f2d800700 1 -- start start 2024-05-10T05:25:55.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.647+0000 7f9f2d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2807a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.647+0000 7f9f2d800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2807a820 con 0x7f9f2807be20 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.648+0000 7f9f26a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2807a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.648+0000 7f9f26a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2807a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40340/0 (socket says 172.21.15.131:40340) 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.648+0000 7f9f26a00700 1 -- 172.21.15.131:0/798677378 learned_addr learned my addr 172.21.15.131:0/798677378 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.648+0000 7f9f26a00700 1 -- 172.21.15.131:0/798677378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2807a960 con 0x7f9f2807be20 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.648+0000 7f9f26a00700 1 --2- 172.21.15.131:0/798677378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2807a250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f180098a0 tx=0x7f9f18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e6165b09f5ba90 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.649+0000 7f9f25600700 1 -- 172.21.15.131:0/798677378 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f18004160 con 0x7f9f2807be20 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.649+0000 7f9f25600700 1 -- 172.21.15.131:0/798677378 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9f180042c0 con 0x7f9f2807be20 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.649+0000 7f9f25600700 1 -- 172.21.15.131:0/798677378 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f18011a00 con 0x7f9f2807be20 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.649+0000 7f9f2d800700 1 -- 172.21.15.131:0/798677378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 msgr2=0x7f9f2807a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.649+0000 7f9f2d800700 1 --2- 172.21.15.131:0/798677378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2807a250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f180098a0 tx=0x7f9f18009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:55.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.650+0000 7f9f2d800700 1 -- 172.21.15.131:0/798677378 shutdown_connections 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.650+0000 7f9f2d800700 1 --2- 172.21.15.131:0/798677378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2807a250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.650+0000 7f9f2d800700 1 -- 172.21.15.131:0/798677378 >> 172.21.15.131:0/798677378 conn(0x7f9f28102830 msgr2=0x7f9f28104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.650+0000 7f9f2d800700 1 -- 172.21.15.131:0/798677378 shutdown_connections 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.650+0000 7f9f2d800700 1 -- 172.21.15.131:0/798677378 wait complete. 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.651+0000 7f9f2d800700 1 Processor -- start 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.651+0000 7f9f2d800700 1 -- start start 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.651+0000 7f9f2d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.651+0000 7f9f2d800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2819db90 con 0x7f9f2807be20 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.651+0000 7f9f26a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.651+0000 7f9f26a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2819d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40356/0 (socket says 172.21.15.131:40356) 2024-05-10T05:25:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.651+0000 7f9f26a00700 1 -- 172.21.15.131:0/137549330 learned_addr learned my addr 172.21.15.131:0/137549330 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.652+0000 7f9f26a00700 1 -- 172.21.15.131:0/137549330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f18009550 con 0x7f9f2807be20 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.652+0000 7f9f26a00700 1 --2- 172.21.15.131:0/137549330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2819d650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f18011710 tx=0x7f9f18011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.652+0000 7f9f1f400700 1 -- 172.21.15.131:0/137549330 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f18003aa0 con 0x7f9f2807be20 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.652+0000 7f9f1f400700 1 -- 172.21.15.131:0/137549330 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9f18003c00 con 0x7f9f2807be20 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.652+0000 7f9f2d800700 1 -- 172.21.15.131:0/137549330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2819dd90 con 0x7f9f2807be20 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.652+0000 7f9f1f400700 1 -- 172.21.15.131:0/137549330 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f18024800 con 0x7f9f2807be20 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.652+0000 7f9f2d800700 1 -- 172.21.15.131:0/137549330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2819e230 con 0x7f9f2807be20 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.654+0000 7f9f2d800700 1 -- 172.21.15.131:0/137549330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2810d5f0 con 0x7f9f2807be20 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.654+0000 7f9f1f400700 1 -- 172.21.15.131:0/137549330 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f9f18024a30 con 0x7f9f2807be20 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.654+0000 7f9f1f400700 1 --2- 172.21.15.131:0/137549330 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f9f0c038700 0x7f9f0c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:55.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.654+0000 7f9f1f400700 1 -- 172.21.15.131:0/137549330 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9f1804c8f0 con 0x7f9f2807be20 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.654+0000 7f9f26000700 1 --2- 172.21.15.131:0/137549330 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f9f0c038700 0x7f9f0c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.655+0000 7f9f26000700 1 --2- 172.21.15.131:0/137549330 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f9f0c038700 0x7f9f0c03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f14006fd0 tx=0x7f9f14006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.660+0000 7f9f1f400700 1 -- 172.21.15.131:0/137549330 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9f180189a0 con 0x7f9f2807be20 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.866+0000 7f9f2d800700 1 -- 172.21.15.131:0/137549330 --> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f281a0a30 con 0x7f9f0c038700 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.867+0000 7f9f1f400700 1 -- 172.21.15.131:0/137549330 <== mgr.14122 v2:172.21.15.131:6800/3754674139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f9f281a0a30 con 0x7f9f0c038700 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.869+0000 7f9f2d800700 1 -- 172.21.15.131:0/137549330 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f9f0c038700 msgr2=0x7f9f0c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.869+0000 7f9f2d800700 1 --2- 172.21.15.131:0/137549330 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f9f0c038700 0x7f9f0c03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f14006fd0 tx=0x7f9f14006e40 comp rx=0 tx=0).stop 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.869+0000 7f9f2d800700 1 -- 172.21.15.131:0/137549330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 msgr2=0x7f9f2819d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.869+0000 7f9f2d800700 1 --2- 172.21.15.131:0/137549330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2819d650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f18011710 tx=0x7f9f18011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.869+0000 7f9f2d800700 1 -- 172.21.15.131:0/137549330 shutdown_connections 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.869+0000 7f9f2d800700 1 --2- 172.21.15.131:0/137549330 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f9f0c038700 0x7f9f0c03ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.869+0000 7f9f2d800700 1 --2- 172.21.15.131:0/137549330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2807be20 0x7f9f2819d650 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:55.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.869+0000 7f9f2d800700 1 -- 172.21.15.131:0/137549330 >> 172.21.15.131:0/137549330 conn(0x7f9f28102830 msgr2=0x7f9f28103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:55.224 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.869+0000 7f9f2d800700 1 -- 172.21.15.131:0/137549330 shutdown_connections 2024-05-10T05:25:55.224 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:54.869+0000 7f9f2d800700 1 -- 172.21.15.131:0/137549330 wait complete. 2024-05-10T05:25:55.224 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-10T05:25:55.224 INFO:teuthology.orchestra.run.smithi131.stdout:Adding key to root@localhost authorized_keys... 2024-05-10T05:25:55.224 INFO:teuthology.orchestra.run.smithi131.stdout:Adding host smithi131... 2024-05-10T05:25:55.938 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:55 smithi131 ceph-mon[37264]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:25:56.941 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:56 smithi131 ceph-mon[37264]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:25:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:57 smithi131 ceph-mon[37264]: Deploying cephadm binary to smithi131 2024-05-10T05:25:59.319 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Added host 'smithi131' with addr '172.21.15.131' 2024-05-10T05:25:59.319 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.694+0000 7fec16200700 1 Processor -- start 2024-05-10T05:25:59.319 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.694+0000 7fec16200700 1 -- start start 2024-05-10T05:25:59.319 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.694+0000 7fec16200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1007a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:59.319 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.694+0000 7fec16200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1007a840 con 0x7fec1007be40 2024-05-10T05:25:59.319 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.694+0000 7fec14e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1007a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:59.319 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.694+0000 7fec14e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1007a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40372/0 (socket says 172.21.15.131:40372) 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.695+0000 7fec14e00700 1 -- 172.21.15.131:0/301135011 learned_addr learned my addr 172.21.15.131:0/301135011 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.695+0000 7fec14e00700 1 -- 172.21.15.131:0/301135011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1007a980 con 0x7fec1007be40 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.695+0000 7fec14e00700 1 --2- 172.21.15.131:0/301135011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1007a270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fec040098a0 tx=0x7fec04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467e1205e3ab54fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.695+0000 7fec0f400700 1 -- 172.21.15.131:0/301135011 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec04004160 con 0x7fec1007be40 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.695+0000 7fec0f400700 1 -- 172.21.15.131:0/301135011 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fec040042c0 con 0x7fec1007be40 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.695+0000 7fec0f400700 1 -- 172.21.15.131:0/301135011 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec04011a00 con 0x7fec1007be40 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.696+0000 7fec16200700 1 -- 172.21.15.131:0/301135011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 msgr2=0x7fec1007a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.696+0000 7fec16200700 1 --2- 172.21.15.131:0/301135011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1007a270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fec040098a0 tx=0x7fec04009bb0 comp rx=0 tx=0).stop 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.696+0000 7fec16200700 1 -- 172.21.15.131:0/301135011 shutdown_connections 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.696+0000 7fec16200700 1 --2- 172.21.15.131:0/301135011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1007a270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.696+0000 7fec16200700 1 -- 172.21.15.131:0/301135011 >> 172.21.15.131:0/301135011 conn(0x7fec10102690 msgr2=0x7fec10104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:59.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.696+0000 7fec16200700 1 -- 172.21.15.131:0/301135011 shutdown_connections 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.696+0000 7fec16200700 1 -- 172.21.15.131:0/301135011 wait complete. 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.697+0000 7fec16200700 1 Processor -- start 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.697+0000 7fec16200700 1 -- start start 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.698+0000 7fec16200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.698+0000 7fec16200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1019d9f0 con 0x7fec1007be40 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.698+0000 7fec14e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.698+0000 7fec14e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1019d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40388/0 (socket says 172.21.15.131:40388) 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.698+0000 7fec14e00700 1 -- 172.21.15.131:0/1024839211 learned_addr learned my addr 172.21.15.131:0/1024839211 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.698+0000 7fec14e00700 1 -- 172.21.15.131:0/1024839211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec04009550 con 0x7fec1007be40 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.699+0000 7fec14e00700 1 --2- 172.21.15.131:0/1024839211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1019d4b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fec04011710 tx=0x7fec04011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.699+0000 7fec0d600700 1 -- 172.21.15.131:0/1024839211 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec04003af0 con 0x7fec1007be40 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.699+0000 7fec0d600700 1 -- 172.21.15.131:0/1024839211 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fec04003c50 con 0x7fec1007be40 2024-05-10T05:25:59.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.699+0000 7fec16200700 1 -- 172.21.15.131:0/1024839211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1019dbf0 con 0x7fec1007be40 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.699+0000 7fec0d600700 1 -- 172.21.15.131:0/1024839211 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec04024780 con 0x7fec1007be40 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.699+0000 7fec16200700 1 -- 172.21.15.131:0/1024839211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1019e090 con 0x7fec1007be40 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.700+0000 7fec0d600700 1 -- 172.21.15.131:0/1024839211 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fec040249c0 con 0x7fec1007be40 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.700+0000 7fec16200700 1 -- 172.21.15.131:0/1024839211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1010d490 con 0x7fec1007be40 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.701+0000 7fec0d600700 1 --2- 172.21.15.131:0/1024839211 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7febf8038700 0x7febf803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.701+0000 7fec0d600700 1 -- 172.21.15.131:0/1024839211 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fec0404c730 con 0x7fec1007be40 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.701+0000 7fec0fe00700 1 --2- 172.21.15.131:0/1024839211 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7febf8038700 0x7febf803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.701+0000 7fec0fe00700 1 --2- 172.21.15.131:0/1024839211 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7febf8038700 0x7febf803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7febfc006fd0 tx=0x7febfc006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.706+0000 7fec0d600700 1 -- 172.21.15.131:0/1024839211 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fec04018d20 con 0x7fec1007be40 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:55.912+0000 7fec16200700 1 -- 172.21.15.131:0/1024839211 --> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}) v1 -- 0x7fec101a0890 con 0x7febf8038700 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.927+0000 7fec0d600700 1 -- 172.21.15.131:0/1024839211 <== mgr.14122 v2:172.21.15.131:6800/3754674139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fec101a0890 con 0x7febf8038700 2024-05-10T05:25:59.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.930+0000 7fec16200700 1 -- 172.21.15.131:0/1024839211 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7febf8038700 msgr2=0x7febf803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:59.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.930+0000 7fec16200700 1 --2- 172.21.15.131:0/1024839211 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7febf8038700 0x7febf803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7febfc006fd0 tx=0x7febfc006e40 comp rx=0 tx=0).stop 2024-05-10T05:25:59.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.930+0000 7fec16200700 1 -- 172.21.15.131:0/1024839211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 msgr2=0x7fec1019d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:25:59.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.930+0000 7fec16200700 1 --2- 172.21.15.131:0/1024839211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1019d4b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fec04011710 tx=0x7fec04011ad0 comp rx=0 tx=0).stop 2024-05-10T05:25:59.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.930+0000 7fec16200700 1 -- 172.21.15.131:0/1024839211 shutdown_connections 2024-05-10T05:25:59.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.930+0000 7fec16200700 1 --2- 172.21.15.131:0/1024839211 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7febf8038700 0x7febf803ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:59.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.930+0000 7fec16200700 1 --2- 172.21.15.131:0/1024839211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec1007be40 0x7fec1019d4b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:25:59.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.930+0000 7fec16200700 1 -- 172.21.15.131:0/1024839211 >> 172.21.15.131:0/1024839211 conn(0x7fec10102690 msgr2=0x7fec10103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:25:59.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.930+0000 7fec16200700 1 -- 172.21.15.131:0/1024839211 shutdown_connections 2024-05-10T05:25:59.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:58.930+0000 7fec16200700 1 -- 172.21.15.131:0/1024839211 wait complete. 2024-05-10T05:25:59.323 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mon service... 2024-05-10T05:26:00.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:59 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:26:00.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:59 smithi131 ceph-mon[37264]: Added host smithi131 2024-05-10T05:26:00.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:25:59 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:00.614 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-10T05:26:00.614 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.834+0000 7f9308000700 1 Processor -- start 2024-05-10T05:26:00.614 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.834+0000 7f9308000700 1 -- start start 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.834+0000 7f9308000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930007a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.834+0000 7f9308000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930007a860 con 0x7f930007be60 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.835+0000 7f9305600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930007a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.835+0000 7f9305600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930007a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40404/0 (socket says 172.21.15.131:40404) 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.835+0000 7f9305600700 1 -- 172.21.15.131:0/3874773826 learned_addr learned my addr 172.21.15.131:0/3874773826 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.835+0000 7f9305600700 1 -- 172.21.15.131:0/3874773826 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930007a9a0 con 0x7f930007be60 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.835+0000 7f9305600700 1 --2- 172.21.15.131:0/3874773826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930007a290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f92f40098a0 tx=0x7f92f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6e16b15970b6c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.836+0000 7f92ffe00700 1 -- 172.21.15.131:0/3874773826 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92f4004160 con 0x7f930007be60 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.836+0000 7f92ffe00700 1 -- 172.21.15.131:0/3874773826 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f92f40042c0 con 0x7f930007be60 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.836+0000 7f92ffe00700 1 -- 172.21.15.131:0/3874773826 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92f4011a00 con 0x7f930007be60 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.837+0000 7f9308000700 1 -- 172.21.15.131:0/3874773826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 msgr2=0x7f930007a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.837+0000 7f9308000700 1 --2- 172.21.15.131:0/3874773826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930007a290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f92f40098a0 tx=0x7f92f4009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:00.615 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.837+0000 7f9308000700 1 -- 172.21.15.131:0/3874773826 shutdown_connections 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.837+0000 7f9308000700 1 --2- 172.21.15.131:0/3874773826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930007a290 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.837+0000 7f9308000700 1 -- 172.21.15.131:0/3874773826 >> 172.21.15.131:0/3874773826 conn(0x7f9300102840 msgr2=0x7f9300104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.837+0000 7f9308000700 1 -- 172.21.15.131:0/3874773826 shutdown_connections 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.837+0000 7f9308000700 1 -- 172.21.15.131:0/3874773826 wait complete. 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.838+0000 7f9308000700 1 Processor -- start 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.838+0000 7f9308000700 1 -- start start 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.838+0000 7f9308000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.838+0000 7f9308000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930019db60 con 0x7f930007be60 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.839+0000 7f9305600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.839+0000 7f9305600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40412/0 (socket says 172.21.15.131:40412) 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.839+0000 7f9305600700 1 -- 172.21.15.131:0/2443678480 learned_addr learned my addr 172.21.15.131:0/2443678480 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.839+0000 7f9305600700 1 -- 172.21.15.131:0/2443678480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f4009550 con 0x7f930007be60 2024-05-10T05:26:00.616 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.839+0000 7f9305600700 1 --2- 172.21.15.131:0/2443678480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930019d620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f92f4011710 tx=0x7f92f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.840+0000 7f92fe000700 1 -- 172.21.15.131:0/2443678480 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92f4003aa0 con 0x7f930007be60 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.840+0000 7f92fe000700 1 -- 172.21.15.131:0/2443678480 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f92f4003c00 con 0x7f930007be60 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.840+0000 7f92fe000700 1 -- 172.21.15.131:0/2443678480 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92f4024800 con 0x7f930007be60 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.840+0000 7f9308000700 1 -- 172.21.15.131:0/2443678480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930019dd60 con 0x7f930007be60 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.840+0000 7f9308000700 1 -- 172.21.15.131:0/2443678480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930019e200 con 0x7f930007be60 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.841+0000 7f9308000700 1 -- 172.21.15.131:0/2443678480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930010d670 con 0x7f930007be60 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.841+0000 7f92fe000700 1 -- 172.21.15.131:0/2443678480 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f92f4024960 con 0x7f930007be60 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.842+0000 7f92fe000700 1 --2- 172.21.15.131:0/2443678480 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f92e8038750 0x7f92e803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.842+0000 7f92fe000700 1 -- 172.21.15.131:0/2443678480 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f92f404c850 con 0x7f930007be60 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.842+0000 7f9304c00700 1 --2- 172.21.15.131:0/2443678480 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f92e8038750 0x7f92e803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.843+0000 7f9304c00700 1 --2- 172.21.15.131:0/2443678480 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f92e8038750 0x7f92e803ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f92ec006fd0 tx=0x7f92ec006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:25:59.848+0000 7f92fe000700 1 -- 172.21.15.131:0/2443678480 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f92f4018710 con 0x7f930007be60 2024-05-10T05:26:00.617 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.076+0000 7f9308000700 1 -- 172.21.15.131:0/2443678480 --> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f93001a0a00 con 0x7f92e8038750 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.084+0000 7f92fe000700 1 -- 172.21.15.131:0/2443678480 <== mgr.14122 v2:172.21.15.131:6800/3754674139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f93001a0a00 con 0x7f92e8038750 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.087+0000 7f9308000700 1 -- 172.21.15.131:0/2443678480 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f92e8038750 msgr2=0x7f92e803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.087+0000 7f9308000700 1 --2- 172.21.15.131:0/2443678480 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f92e8038750 0x7f92e803ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f92ec006fd0 tx=0x7f92ec006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.087+0000 7f9308000700 1 -- 172.21.15.131:0/2443678480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 msgr2=0x7f930019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.087+0000 7f9308000700 1 --2- 172.21.15.131:0/2443678480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930019d620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f92f4011710 tx=0x7f92f4011ad0 comp rx=0 tx=0).stop 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.087+0000 7f9308000700 1 -- 172.21.15.131:0/2443678480 shutdown_connections 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.087+0000 7f9308000700 1 --2- 172.21.15.131:0/2443678480 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f92e8038750 0x7f92e803ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.087+0000 7f9308000700 1 --2- 172.21.15.131:0/2443678480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f930007be60 0x7f930019d620 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.087+0000 7f9308000700 1 -- 172.21.15.131:0/2443678480 >> 172.21.15.131:0/2443678480 conn(0x7f9300102840 msgr2=0x7f93001032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.088+0000 7f9308000700 1 -- 172.21.15.131:0/2443678480 shutdown_connections 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:00.088+0000 7f9308000700 1 -- 172.21.15.131:0/2443678480 wait complete. 2024-05-10T05:26:00.618 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mgr service... 2024-05-10T05:26:01.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:01 smithi131 ceph-mon[37264]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:01.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:01 smithi131 ceph-mon[37264]: Saving service mon spec with placement count:5 2024-05-10T05:26:01.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:01 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:26:01.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:01 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:26:01.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:01 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:26:01.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-10T05:26:01.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.126+0000 7fb53a400700 1 Processor -- start 2024-05-10T05:26:01.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.126+0000 7fb53a400700 1 -- start start 2024-05-10T05:26:01.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.126+0000 7fb53a400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb534107630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:01.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.126+0000 7fb53a400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb534107c20 con 0x7fb534107220 2024-05-10T05:26:01.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.127+0000 7fb533400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb534107630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:01.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.127+0000 7fb533400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb534107630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40426/0 (socket says 172.21.15.131:40426) 2024-05-10T05:26:01.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.127+0000 7fb533400700 1 -- 172.21.15.131:0/2654735239 learned_addr learned my addr 172.21.15.131:0/2654735239 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:01.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.127+0000 7fb533400700 1 -- 172.21.15.131:0/2654735239 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5341083f0 con 0x7fb534107220 2024-05-10T05:26:01.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.128+0000 7fb533400700 1 --2- 172.21.15.131:0/2654735239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb534107630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb5240098a0 tx=0x7fb524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69f7592434570e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:01.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.128+0000 7fb532000700 1 -- 172.21.15.131:0/2654735239 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb524004160 con 0x7fb534107220 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.128+0000 7fb532000700 1 -- 172.21.15.131:0/2654735239 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5240042c0 con 0x7fb534107220 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.128+0000 7fb532000700 1 -- 172.21.15.131:0/2654735239 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5240119b0 con 0x7fb534107220 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.129+0000 7fb53a400700 1 -- 172.21.15.131:0/2654735239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 msgr2=0x7fb534107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.129+0000 7fb53a400700 1 --2- 172.21.15.131:0/2654735239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb534107630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb5240098a0 tx=0x7fb524009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.129+0000 7fb53a400700 1 -- 172.21.15.131:0/2654735239 shutdown_connections 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.129+0000 7fb53a400700 1 --2- 172.21.15.131:0/2654735239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb534107630 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.129+0000 7fb53a400700 1 -- 172.21.15.131:0/2654735239 >> 172.21.15.131:0/2654735239 conn(0x7fb534102840 msgr2=0x7fb534104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.129+0000 7fb53a400700 1 -- 172.21.15.131:0/2654735239 shutdown_connections 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.129+0000 7fb53a400700 1 -- 172.21.15.131:0/2654735239 wait complete. 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.130+0000 7fb53a400700 1 Processor -- start 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.130+0000 7fb53a400700 1 -- start start 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.130+0000 7fb53a400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb53419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:01.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.130+0000 7fb53a400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53419dbd0 con 0x7fb534107220 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.131+0000 7fb533400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb53419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.131+0000 7fb533400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb53419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40428/0 (socket says 172.21.15.131:40428) 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.131+0000 7fb533400700 1 -- 172.21.15.131:0/1207446227 learned_addr learned my addr 172.21.15.131:0/1207446227 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.131+0000 7fb533400700 1 -- 172.21.15.131:0/1207446227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524009550 con 0x7fb534107220 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.131+0000 7fb533400700 1 --2- 172.21.15.131:0/1207446227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb53419d690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb524011980 tx=0x7fb524011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.131+0000 7fb52be00700 1 -- 172.21.15.131:0/1207446227 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb524003aa0 con 0x7fb534107220 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.131+0000 7fb52be00700 1 -- 172.21.15.131:0/1207446227 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb524003c00 con 0x7fb534107220 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.132+0000 7fb53a400700 1 -- 172.21.15.131:0/1207446227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53419ddd0 con 0x7fb534107220 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.132+0000 7fb52be00700 1 -- 172.21.15.131:0/1207446227 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5240196c0 con 0x7fb534107220 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.132+0000 7fb53a400700 1 -- 172.21.15.131:0/1207446227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53419e270 con 0x7fb534107220 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.133+0000 7fb52be00700 1 -- 172.21.15.131:0/1207446227 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fb524019930 con 0x7fb534107220 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.133+0000 7fb53a400700 1 -- 172.21.15.131:0/1207446227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53410b620 con 0x7fb534107220 2024-05-10T05:26:01.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.133+0000 7fb52be00700 1 --2- 172.21.15.131:0/1207446227 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fb518038750 0x7fb51803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.133+0000 7fb52be00700 1 -- 172.21.15.131:0/1207446227 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb52401aa80 con 0x7fb534107220 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.133+0000 7fb532a00700 1 --2- 172.21.15.131:0/1207446227 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fb518038750 0x7fb51803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.134+0000 7fb532a00700 1 --2- 172.21.15.131:0/1207446227 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fb518038750 0x7fb51803ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb520006fd0 tx=0x7fb520006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.138+0000 7fb52be00700 1 -- 172.21.15.131:0/1207446227 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb524018db0 con 0x7fb534107220 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.378+0000 7fb53a400700 1 -- 172.21.15.131:0/1207446227 --> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb5341a0a70 con 0x7fb518038750 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.384+0000 7fb52be00700 1 -- 172.21.15.131:0/1207446227 <== mgr.14122 v2:172.21.15.131:6800/3754674139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb5341a0a70 con 0x7fb518038750 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.387+0000 7fb53a400700 1 -- 172.21.15.131:0/1207446227 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fb518038750 msgr2=0x7fb51803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.387+0000 7fb53a400700 1 --2- 172.21.15.131:0/1207446227 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fb518038750 0x7fb51803ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb520006fd0 tx=0x7fb520006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.387+0000 7fb53a400700 1 -- 172.21.15.131:0/1207446227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 msgr2=0x7fb53419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.387+0000 7fb53a400700 1 --2- 172.21.15.131:0/1207446227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb53419d690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb524011980 tx=0x7fb524011a60 comp rx=0 tx=0).stop 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.387+0000 7fb53a400700 1 -- 172.21.15.131:0/1207446227 shutdown_connections 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.387+0000 7fb53a400700 1 --2- 172.21.15.131:0/1207446227 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fb518038750 0x7fb51803ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:01.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.387+0000 7fb53a400700 1 --2- 172.21.15.131:0/1207446227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb534107220 0x7fb53419d690 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:01.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.387+0000 7fb53a400700 1 -- 172.21.15.131:0/1207446227 >> 172.21.15.131:0/1207446227 conn(0x7fb534102840 msgr2=0x7fb5341032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:01.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.387+0000 7fb53a400700 1 -- 172.21.15.131:0/1207446227 shutdown_connections 2024-05-10T05:26:01.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:01.388+0000 7fb53a400700 1 -- 172.21.15.131:0/1207446227 wait complete. 2024-05-10T05:26:02.645 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:02 smithi131 ceph-mon[37264]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:02.645 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:02 smithi131 ceph-mon[37264]: Saving service mgr spec with placement count:2 2024-05-10T05:26:02.645 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:02 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:26:03.032 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.372+0000 7f4be5800700 1 Processor -- start 2024-05-10T05:26:03.032 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.373+0000 7f4be5800700 1 -- start start 2024-05-10T05:26:03.032 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.373+0000 7f4be5800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be010a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.373+0000 7f4be5800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be0079e30 con 0x7f4be0107d50 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.373+0000 7f4bdea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be010a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.373+0000 7f4bdea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be010a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40440/0 (socket says 172.21.15.131:40440) 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.373+0000 7f4bdea00700 1 -- 172.21.15.131:0/2211879433 learned_addr learned my addr 172.21.15.131:0/2211879433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.374+0000 7f4bdea00700 1 -- 172.21.15.131:0/2211879433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be010a6b0 con 0x7f4be0107d50 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.374+0000 7f4bdea00700 1 --2- 172.21.15.131:0/2211879433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be010a170 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00098a0 tx=0x7f4bd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee04394b768df89b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.374+0000 7f4bdd600700 1 -- 172.21.15.131:0/2211879433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bd0004160 con 0x7f4be0107d50 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.374+0000 7f4bdd600700 1 -- 172.21.15.131:0/2211879433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bd00042c0 con 0x7f4be0107d50 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.374+0000 7f4bdd600700 1 -- 172.21.15.131:0/2211879433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bd00119b0 con 0x7f4be0107d50 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.375+0000 7f4be5800700 1 -- 172.21.15.131:0/2211879433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 msgr2=0x7f4be010a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.375+0000 7f4be5800700 1 --2- 172.21.15.131:0/2211879433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be010a170 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00098a0 tx=0x7f4bd0009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.375+0000 7f4be5800700 1 -- 172.21.15.131:0/2211879433 shutdown_connections 2024-05-10T05:26:03.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.375+0000 7f4be5800700 1 --2- 172.21.15.131:0/2211879433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be010a170 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:03.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.375+0000 7f4be5800700 1 -- 172.21.15.131:0/2211879433 >> 172.21.15.131:0/2211879433 conn(0x7f4be0101790 msgr2=0x7f4be0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:03.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.375+0000 7f4be5800700 1 -- 172.21.15.131:0/2211879433 shutdown_connections 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.375+0000 7f4be5800700 1 -- 172.21.15.131:0/2211879433 wait complete. 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.376+0000 7f4be5800700 1 Processor -- start 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.376+0000 7f4be5800700 1 -- start start 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.377+0000 7f4be5800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.377+0000 7f4be5800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be019b990 con 0x7f4be0107d50 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.377+0000 7f4bdea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.377+0000 7f4bdea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40448/0 (socket says 172.21.15.131:40448) 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.377+0000 7f4bdea00700 1 -- 172.21.15.131:0/2924263809 learned_addr learned my addr 172.21.15.131:0/2924263809 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.377+0000 7f4bdea00700 1 -- 172.21.15.131:0/2924263809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0009550 con 0x7f4be0107d50 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.378+0000 7f4bdea00700 1 --2- 172.21.15.131:0/2924263809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be019b450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0011670 tx=0x7f4bd0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.378+0000 7f4bd7400700 1 -- 172.21.15.131:0/2924263809 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bd0003af0 con 0x7f4be0107d50 2024-05-10T05:26:03.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.378+0000 7f4bd7400700 1 -- 172.21.15.131:0/2924263809 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bd0003c50 con 0x7f4be0107d50 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.378+0000 7f4be5800700 1 -- 172.21.15.131:0/2924263809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be019bb90 con 0x7f4be0107d50 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.378+0000 7f4bd7400700 1 -- 172.21.15.131:0/2924263809 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bd0024820 con 0x7f4be0107d50 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.378+0000 7f4be5800700 1 -- 172.21.15.131:0/2924263809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be019c030 con 0x7f4be0107d50 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.379+0000 7f4be5800700 1 -- 172.21.15.131:0/2924263809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be010b490 con 0x7f4be0107d50 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.380+0000 7f4bd7400700 1 -- 172.21.15.131:0/2924263809 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f4bd0024a50 con 0x7f4be0107d50 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.380+0000 7f4bd7400700 1 --2- 172.21.15.131:0/2924263809 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f4bc4038750 0x7f4bc403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.380+0000 7f4bd7400700 1 -- 172.21.15.131:0/2924263809 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4bd004c7f0 con 0x7f4be0107d50 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.381+0000 7f4bde000700 1 --2- 172.21.15.131:0/2924263809 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f4bc4038750 0x7f4bc403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.381+0000 7f4bde000700 1 --2- 172.21.15.131:0/2924263809 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f4bc4038750 0x7f4bc403ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc006fd0 tx=0x7f4bcc006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.386+0000 7f4bd7400700 1 -- 172.21.15.131:0/2924263809 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4bd0010790 con 0x7f4be0107d50 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.604+0000 7f4be5800700 1 -- 172.21.15.131:0/2924263809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f4be010b6a0 con 0x7f4be0107d50 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.610+0000 7f4bd7400700 1 -- 172.21.15.131:0/2924263809 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f4bd002c350 con 0x7f4be0107d50 2024-05-10T05:26:03.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.613+0000 7f4be5800700 1 -- 172.21.15.131:0/2924263809 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f4bc4038750 msgr2=0x7f4bc403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:03.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.613+0000 7f4be5800700 1 --2- 172.21.15.131:0/2924263809 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f4bc4038750 0x7f4bc403ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc006fd0 tx=0x7f4bcc006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:03.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.613+0000 7f4be5800700 1 -- 172.21.15.131:0/2924263809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 msgr2=0x7f4be019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:03.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.613+0000 7f4be5800700 1 --2- 172.21.15.131:0/2924263809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be019b450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0011670 tx=0x7f4bd0011a30 comp rx=0 tx=0).stop 2024-05-10T05:26:03.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.613+0000 7f4be5800700 1 -- 172.21.15.131:0/2924263809 shutdown_connections 2024-05-10T05:26:03.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.613+0000 7f4be5800700 1 --2- 172.21.15.131:0/2924263809 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f4bc4038750 0x7f4bc403ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:03.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.613+0000 7f4be5800700 1 --2- 172.21.15.131:0/2924263809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4be0107d50 0x7f4be019b450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:03.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.613+0000 7f4be5800700 1 -- 172.21.15.131:0/2924263809 >> 172.21.15.131:0/2924263809 conn(0x7f4be0101790 msgr2=0x7f4be0102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:03.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.613+0000 7f4be5800700 1 -- 172.21.15.131:0/2924263809 shutdown_connections 2024-05-10T05:26:03.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:02.613+0000 7f4be5800700 1 -- 172.21.15.131:0/2924263809 wait complete. 2024-05-10T05:26:03.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:03 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:26:03.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:03 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2924263809' entity='client.admin' 2024-05-10T05:26:04.218 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.605+0000 7fe2ab600700 1 Processor -- start 2024-05-10T05:26:04.218 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.606+0000 7fe2ab600700 1 -- start start 2024-05-10T05:26:04.218 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.606+0000 7fe2ab600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a4107630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:04.218 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.606+0000 7fe2ab600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4107c20 con 0x7fe2a4107220 2024-05-10T05:26:04.218 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.606+0000 7fe2a8c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a4107630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:04.218 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.607+0000 7fe2a8c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a4107630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59706/0 (socket says 172.21.15.131:59706) 2024-05-10T05:26:04.218 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.607+0000 7fe2a8c00700 1 -- 172.21.15.131:0/1463509783 learned_addr learned my addr 172.21.15.131:0/1463509783 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:04.218 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.607+0000 7fe2a8c00700 1 -- 172.21.15.131:0/1463509783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a41083f0 con 0x7fe2a4107220 2024-05-10T05:26:04.218 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.607+0000 7fe2a8c00700 1 --2- 172.21.15.131:0/1463509783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a4107630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe2980098a0 tx=0x7fe298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9544554d1d6973e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.607+0000 7fe2a3400700 1 -- 172.21.15.131:0/1463509783 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe298004160 con 0x7fe2a4107220 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.607+0000 7fe2a3400700 1 -- 172.21.15.131:0/1463509783 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2980042c0 con 0x7fe2a4107220 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.607+0000 7fe2a3400700 1 -- 172.21.15.131:0/1463509783 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2980119b0 con 0x7fe2a4107220 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.608+0000 7fe2ab600700 1 -- 172.21.15.131:0/1463509783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 msgr2=0x7fe2a4107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.608+0000 7fe2ab600700 1 --2- 172.21.15.131:0/1463509783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a4107630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe2980098a0 tx=0x7fe298009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.608+0000 7fe2ab600700 1 -- 172.21.15.131:0/1463509783 shutdown_connections 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.608+0000 7fe2ab600700 1 --2- 172.21.15.131:0/1463509783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a4107630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.608+0000 7fe2ab600700 1 -- 172.21.15.131:0/1463509783 >> 172.21.15.131:0/1463509783 conn(0x7fe2a4102840 msgr2=0x7fe2a4104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.609+0000 7fe2ab600700 1 -- 172.21.15.131:0/1463509783 shutdown_connections 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.609+0000 7fe2ab600700 1 -- 172.21.15.131:0/1463509783 wait complete. 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.610+0000 7fe2ab600700 1 Processor -- start 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.610+0000 7fe2ab600700 1 -- start start 2024-05-10T05:26:04.219 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.610+0000 7fe2ab600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.610+0000 7fe2ab600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a419dbc0 con 0x7fe2a4107220 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.610+0000 7fe2a8c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.610+0000 7fe2a8c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59722/0 (socket says 172.21.15.131:59722) 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.610+0000 7fe2a8c00700 1 -- 172.21.15.131:0/3308893604 learned_addr learned my addr 172.21.15.131:0/3308893604 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.611+0000 7fe2a8c00700 1 -- 172.21.15.131:0/3308893604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe298009550 con 0x7fe2a4107220 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.611+0000 7fe2a8c00700 1 --2- 172.21.15.131:0/3308893604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a419d680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe298011980 tx=0x7fe298011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.611+0000 7fe2a1600700 1 -- 172.21.15.131:0/3308893604 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe298003aa0 con 0x7fe2a4107220 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.611+0000 7fe2a1600700 1 -- 172.21.15.131:0/3308893604 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe298003c00 con 0x7fe2a4107220 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.611+0000 7fe2ab600700 1 -- 172.21.15.131:0/3308893604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a419ddc0 con 0x7fe2a4107220 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.612+0000 7fe2a1600700 1 -- 172.21.15.131:0/3308893604 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2980196c0 con 0x7fe2a4107220 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.612+0000 7fe2ab600700 1 -- 172.21.15.131:0/3308893604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a419e260 con 0x7fe2a4107220 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.613+0000 7fe2a1600700 1 -- 172.21.15.131:0/3308893604 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fe298019930 con 0x7fe2a4107220 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.613+0000 7fe2ab600700 1 -- 172.21.15.131:0/3308893604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a41974d0 con 0x7fe2a4107220 2024-05-10T05:26:04.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.613+0000 7fe2a1600700 1 --2- 172.21.15.131:0/3308893604 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fe28c038750 0x7fe28c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:04.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.613+0000 7fe2a1600700 1 -- 172.21.15.131:0/3308893604 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe29804c9b0 con 0x7fe2a4107220 2024-05-10T05:26:04.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.614+0000 7fe2a3e00700 1 --2- 172.21.15.131:0/3308893604 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fe28c038750 0x7fe28c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:04.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.614+0000 7fe2a3e00700 1 --2- 172.21.15.131:0/3308893604 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fe28c038750 0x7fe28c03ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe290006fd0 tx=0x7fe290006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:04.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.619+0000 7fe2a1600700 1 -- 172.21.15.131:0/3308893604 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe298018db0 con 0x7fe2a4107220 2024-05-10T05:26:04.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.842+0000 7fe2ab600700 1 -- 172.21.15.131:0/3308893604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe2a410b620 con 0x7fe2a4107220 2024-05-10T05:26:04.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.847+0000 7fe2a1600700 1 -- 172.21.15.131:0/3308893604 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe298018350 con 0x7fe2a4107220 2024-05-10T05:26:04.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.850+0000 7fe2ab600700 1 -- 172.21.15.131:0/3308893604 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fe28c038750 msgr2=0x7fe28c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:04.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.850+0000 7fe2ab600700 1 --2- 172.21.15.131:0/3308893604 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fe28c038750 0x7fe28c03ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe290006fd0 tx=0x7fe290006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:04.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.850+0000 7fe2ab600700 1 -- 172.21.15.131:0/3308893604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 msgr2=0x7fe2a419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:04.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.850+0000 7fe2ab600700 1 --2- 172.21.15.131:0/3308893604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a419d680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe298011980 tx=0x7fe298011a60 comp rx=0 tx=0).stop 2024-05-10T05:26:04.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.850+0000 7fe2ab600700 1 -- 172.21.15.131:0/3308893604 shutdown_connections 2024-05-10T05:26:04.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.850+0000 7fe2ab600700 1 --2- 172.21.15.131:0/3308893604 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fe28c038750 0x7fe28c03ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:04.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.850+0000 7fe2ab600700 1 --2- 172.21.15.131:0/3308893604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a4107220 0x7fe2a419d680 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:04.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.850+0000 7fe2ab600700 1 -- 172.21.15.131:0/3308893604 >> 172.21.15.131:0/3308893604 conn(0x7fe2a4102840 msgr2=0x7fe2a407a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:04.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.850+0000 7fe2ab600700 1 -- 172.21.15.131:0/3308893604 shutdown_connections 2024-05-10T05:26:04.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:03.850+0000 7fe2ab600700 1 -- 172.21.15.131:0/3308893604 wait complete. 2024-05-10T05:26:04.222 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling the dashboard module... 2024-05-10T05:26:05.117 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:04 smithi131 ceph-mon[37264]: from='mgr.14122 172.21.15.131:0/3819102524' entity='mgr.x' 2024-05-10T05:26:05.118 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:04 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3308893604' entity='client.admin' 2024-05-10T05:26:05.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:05 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2201863382' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-10T05:26:05.999 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:05 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: ignoring --setuser ceph since I am not root 2024-05-10T05:26:05.999 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:05 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: ignoring --setgroup ceph since I am not root 2024-05-10T05:26:06.126 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.736+0000 7fc782200700 1 Processor -- start 2024-05-10T05:26:06.132 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.736+0000 7fc782200700 1 -- start start 2024-05-10T05:26:06.132 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.736+0000 7fc782200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c10a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:06.132 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.736+0000 7fc782200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c079e70 con 0x7fc77c107d10 2024-05-10T05:26:06.132 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.737+0000 7fc77b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c10a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:06.132 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.737+0000 7fc77b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c10a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59724/0 (socket says 172.21.15.131:59724) 2024-05-10T05:26:06.132 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.737+0000 7fc77b400700 1 -- 172.21.15.131:0/1627275946 learned_addr learned my addr 172.21.15.131:0/1627275946 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:06.132 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.737+0000 7fc77b400700 1 -- 172.21.15.131:0/1627275946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c10a670 con 0x7fc77c107d10 2024-05-10T05:26:06.132 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.737+0000 7fc77b400700 1 --2- 172.21.15.131:0/1627275946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c10a130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0098a0 tx=0x7fc76c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2742cc1487be26ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:06.132 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.738+0000 7fc77a000700 1 -- 172.21.15.131:0/1627275946 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc76c004160 con 0x7fc77c107d10 2024-05-10T05:26:06.132 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.738+0000 7fc77a000700 1 -- 172.21.15.131:0/1627275946 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc76c0042c0 con 0x7fc77c107d10 2024-05-10T05:26:06.133 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.738+0000 7fc77a000700 1 -- 172.21.15.131:0/1627275946 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc76c0119b0 con 0x7fc77c107d10 2024-05-10T05:26:06.133 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.738+0000 7fc782200700 1 -- 172.21.15.131:0/1627275946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 msgr2=0x7fc77c10a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:06.133 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.738+0000 7fc782200700 1 --2- 172.21.15.131:0/1627275946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c10a130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0098a0 tx=0x7fc76c009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:06.133 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.738+0000 7fc782200700 1 -- 172.21.15.131:0/1627275946 shutdown_connections 2024-05-10T05:26:06.133 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.738+0000 7fc782200700 1 --2- 172.21.15.131:0/1627275946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c10a130 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:06.133 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.738+0000 7fc782200700 1 -- 172.21.15.131:0/1627275946 >> 172.21.15.131:0/1627275946 conn(0x7fc77c101790 msgr2=0x7fc77c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:06.134 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.739+0000 7fc782200700 1 -- 172.21.15.131:0/1627275946 shutdown_connections 2024-05-10T05:26:06.134 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.739+0000 7fc782200700 1 -- 172.21.15.131:0/1627275946 wait complete. 2024-05-10T05:26:06.134 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.740+0000 7fc782200700 1 Processor -- start 2024-05-10T05:26:06.134 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.740+0000 7fc782200700 1 -- start start 2024-05-10T05:26:06.134 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.740+0000 7fc782200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:06.134 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.740+0000 7fc782200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19b9a0 con 0x7fc77c107d10 2024-05-10T05:26:06.135 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.741+0000 7fc77b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:06.135 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.741+0000 7fc77b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59728/0 (socket says 172.21.15.131:59728) 2024-05-10T05:26:06.135 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.741+0000 7fc77b400700 1 -- 172.21.15.131:0/2201863382 learned_addr learned my addr 172.21.15.131:0/2201863382 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:06.135 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.741+0000 7fc77b400700 1 -- 172.21.15.131:0/2201863382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c009550 con 0x7fc77c107d10 2024-05-10T05:26:06.135 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.741+0000 7fc77b400700 1 --2- 172.21.15.131:0/2201863382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c19b460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc76c011670 tx=0x7fc76c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:06.135 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.741+0000 7fc773e00700 1 -- 172.21.15.131:0/2201863382 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc76c003af0 con 0x7fc77c107d10 2024-05-10T05:26:06.136 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.741+0000 7fc773e00700 1 -- 172.21.15.131:0/2201863382 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc76c003c50 con 0x7fc77c107d10 2024-05-10T05:26:06.136 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.741+0000 7fc773e00700 1 -- 172.21.15.131:0/2201863382 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc76c024820 con 0x7fc77c107d10 2024-05-10T05:26:06.136 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.742+0000 7fc782200700 1 -- 172.21.15.131:0/2201863382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c19bba0 con 0x7fc77c107d10 2024-05-10T05:26:06.136 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.742+0000 7fc782200700 1 -- 172.21.15.131:0/2201863382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c19c040 con 0x7fc77c107d10 2024-05-10T05:26:06.136 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.743+0000 7fc773e00700 1 -- 172.21.15.131:0/2201863382 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fc76c01a430 con 0x7fc77c107d10 2024-05-10T05:26:06.136 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.743+0000 7fc782200700 1 -- 172.21.15.131:0/2201863382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c10b450 con 0x7fc77c107d10 2024-05-10T05:26:06.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.743+0000 7fc773e00700 1 --2- 172.21.15.131:0/2201863382 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fc760038700 0x7fc76003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:06.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.743+0000 7fc773e00700 1 -- 172.21.15.131:0/2201863382 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc76c04ca20 con 0x7fc77c107d10 2024-05-10T05:26:06.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.744+0000 7fc77aa00700 1 --2- 172.21.15.131:0/2201863382 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fc760038700 0x7fc76003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:06.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.744+0000 7fc77aa00700 1 --2- 172.21.15.131:0/2201863382 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fc760038700 0x7fc76003ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc768006fd0 tx=0x7fc768006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:06.137 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:04.757+0000 7fc773e00700 1 -- 172.21.15.131:0/2201863382 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc76c018530 con 0x7fc77c107d10 2024-05-10T05:26:06.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.035+0000 7fc782200700 1 -- 172.21.15.131:0/2201863382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fc77c10b660 con 0x7fc77c107d10 2024-05-10T05:26:06.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.750+0000 7fc773e00700 1 -- 172.21.15.131:0/2201863382 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fc76c01ac10 con 0x7fc77c107d10 2024-05-10T05:26:06.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.752+0000 7fc773e00700 1 -- 172.21.15.131:0/2201863382 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fc76c02d4e0 con 0x7fc77c107d10 2024-05-10T05:26:06.138 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.755+0000 7fc782200700 1 -- 172.21.15.131:0/2201863382 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fc760038700 msgr2=0x7fc76003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:06.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.755+0000 7fc782200700 1 --2- 172.21.15.131:0/2201863382 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fc760038700 0x7fc76003ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc768006fd0 tx=0x7fc768006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:06.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.755+0000 7fc782200700 1 -- 172.21.15.131:0/2201863382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 msgr2=0x7fc77c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:06.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.755+0000 7fc782200700 1 --2- 172.21.15.131:0/2201863382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c19b460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc76c011670 tx=0x7fc76c011a30 comp rx=0 tx=0).stop 2024-05-10T05:26:06.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.755+0000 7fc782200700 1 -- 172.21.15.131:0/2201863382 shutdown_connections 2024-05-10T05:26:06.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.755+0000 7fc782200700 1 --2- 172.21.15.131:0/2201863382 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7fc760038700 0x7fc76003ac10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:06.139 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.755+0000 7fc782200700 1 --2- 172.21.15.131:0/2201863382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c107d10 0x7fc77c19b460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:06.140 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.755+0000 7fc782200700 1 -- 172.21.15.131:0/2201863382 >> 172.21.15.131:0/2201863382 conn(0x7fc77c101790 msgr2=0x7fc77c1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:06.140 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.755+0000 7fc782200700 1 -- 172.21.15.131:0/2201863382 shutdown_connections 2024-05-10T05:26:06.140 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:05.755+0000 7fc782200700 1 -- 172.21.15.131:0/2201863382 wait complete. 2024-05-10T05:26:06.627 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:06 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:06.499+0000 7fd103ca1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T05:26:06.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:06 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2201863382' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-10T05:26:06.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:06 smithi131 ceph-mon[37264]: mgrmap e9: x(active, since 16s) 2024-05-10T05:26:06.961 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:06 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:06.626+0000 7fd103ca1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T05:26:06.961 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:06 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:06.746+0000 7fd103ca1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T05:26:06.961 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:06 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:06.862+0000 7fd103ca1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T05:26:07.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-10T05:26:07.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-10T05:26:07.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.672+0000 7f3f90800700 1 Processor -- start 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.672+0000 7f3f90800700 1 -- start start 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.672+0000 7f3f90800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f8807a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.672+0000 7f3f90800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8807a860 con 0x7f3f8807be60 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.673+0000 7f3f8de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f8807a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.673+0000 7f3f8de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f8807a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59760/0 (socket says 172.21.15.131:59760) 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.673+0000 7f3f8de00700 1 -- 172.21.15.131:0/447022694 learned_addr learned my addr 172.21.15.131:0/447022694 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.673+0000 7f3f8de00700 1 -- 172.21.15.131:0/447022694 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8807a9a0 con 0x7f3f8807be60 2024-05-10T05:26:07.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.673+0000 7f3f8de00700 1 --2- 172.21.15.131:0/447022694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f8807a290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0098a0 tx=0x7f3f7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66811289ae00b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.674+0000 7f3f8ca00700 1 -- 172.21.15.131:0/447022694 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f7c004160 con 0x7f3f8807be60 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.674+0000 7f3f8ca00700 1 -- 172.21.15.131:0/447022694 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7c0042c0 con 0x7f3f8807be60 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.674+0000 7f3f8ca00700 1 -- 172.21.15.131:0/447022694 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f7c0119b0 con 0x7f3f8807be60 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.674+0000 7f3f90800700 1 -- 172.21.15.131:0/447022694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 msgr2=0x7f3f8807a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.674+0000 7f3f90800700 1 --2- 172.21.15.131:0/447022694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f8807a290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0098a0 tx=0x7f3f7c009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.674+0000 7f3f90800700 1 -- 172.21.15.131:0/447022694 shutdown_connections 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.674+0000 7f3f90800700 1 --2- 172.21.15.131:0/447022694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f8807a290 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.674+0000 7f3f90800700 1 -- 172.21.15.131:0/447022694 >> 172.21.15.131:0/447022694 conn(0x7f3f88102840 msgr2=0x7f3f88104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.675+0000 7f3f90800700 1 -- 172.21.15.131:0/447022694 shutdown_connections 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.675+0000 7f3f90800700 1 -- 172.21.15.131:0/447022694 wait complete. 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.676+0000 7f3f90800700 1 Processor -- start 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.676+0000 7f3f90800700 1 -- start start 2024-05-10T05:26:07.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.676+0000 7f3f90800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f881a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.676+0000 7f3f90800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881a1f90 con 0x7f3f8807be60 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.676+0000 7f3f8de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f881a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.676+0000 7f3f8de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f881a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59762/0 (socket says 172.21.15.131:59762) 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.676+0000 7f3f8de00700 1 -- 172.21.15.131:0/418179100 learned_addr learned my addr 172.21.15.131:0/418179100 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.677+0000 7f3f8de00700 1 -- 172.21.15.131:0/418179100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c009550 con 0x7f3f8807be60 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.677+0000 7f3f8de00700 1 --2- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f881a1a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c011670 tx=0x7f3f7c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.677+0000 7f3f82a00700 1 -- 172.21.15.131:0/418179100 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f7c003af0 con 0x7f3f8807be60 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.677+0000 7f3f82a00700 1 -- 172.21.15.131:0/418179100 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7c003c50 con 0x7f3f8807be60 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.677+0000 7f3f90800700 1 -- 172.21.15.131:0/418179100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f881a2190 con 0x7f3f8807be60 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.677+0000 7f3f82a00700 1 -- 172.21.15.131:0/418179100 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f7c024820 con 0x7f3f8807be60 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.677+0000 7f3f90800700 1 -- 172.21.15.131:0/418179100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f881a2630 con 0x7f3f8807be60 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.678+0000 7f3f82a00700 1 -- 172.21.15.131:0/418179100 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f3f7c024a50 con 0x7f3f8807be60 2024-05-10T05:26:07.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.679+0000 7f3f90800700 1 -- 172.21.15.131:0/418179100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8819b440 con 0x7f3f8807be60 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.679+0000 7f3f82a00700 1 --2- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f3f700387a0 0x7f3f7003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.679+0000 7f3f82a00700 1 -- 172.21.15.131:0/418179100 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3f7c04c8b0 con 0x7f3f8807be60 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.679+0000 7f3f8d400700 1 -- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f3f700387a0 msgr2=0x7f3f7003acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3754674139 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.679+0000 7f3f8d400700 1 --2- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f3f700387a0 0x7f3f7003acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.684+0000 7f3f82a00700 1 -- 172.21.15.131:0/418179100 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3f7c010790 con 0x7f3f8807be60 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.880+0000 7f3f8d400700 1 -- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f3f700387a0 msgr2=0x7f3f7003acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3754674139 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.880+0000 7f3f8d400700 1 --2- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f3f700387a0 0x7f3f7003acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.952+0000 7f3f90800700 1 -- 172.21.15.131:0/418179100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3f8802cf20 con 0x7f3f8807be60 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.952+0000 7f3f82a00700 1 -- 172.21.15.131:0/418179100 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3f7c02c350 con 0x7f3f8807be60 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.955+0000 7f3f90800700 1 -- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f3f700387a0 msgr2=0x7f3f7003acb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.955+0000 7f3f90800700 1 --2- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f3f700387a0 0x7f3f7003acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:07.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.955+0000 7f3f90800700 1 -- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 msgr2=0x7f3f881a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:07.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.955+0000 7f3f90800700 1 --2- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f881a1a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c011670 tx=0x7f3f7c011a30 comp rx=0 tx=0).stop 2024-05-10T05:26:07.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.955+0000 7f3f90800700 1 -- 172.21.15.131:0/418179100 shutdown_connections 2024-05-10T05:26:07.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.955+0000 7f3f90800700 1 --2- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f3f700387a0 0x7f3f7003acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:07.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.955+0000 7f3f90800700 1 --2- 172.21.15.131:0/418179100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f8807be60 0x7f3f881a1a50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:07.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.955+0000 7f3f90800700 1 -- 172.21.15.131:0/418179100 >> 172.21.15.131:0/418179100 conn(0x7f3f88102840 msgr2=0x7f3f88198580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:07.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.955+0000 7f3f90800700 1 -- 172.21.15.131:0/418179100 shutdown_connections 2024-05-10T05:26:07.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:06.955+0000 7f3f90800700 1 -- 172.21.15.131:0/418179100 wait complete. 2024-05-10T05:26:07.312 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-05-10T05:26:07.312 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 9... 2024-05-10T05:26:08.084 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:07 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/418179100' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T05:26:08.085 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:07 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:07.754+0000 7fd103ca1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T05:26:08.347 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:08.083+0000 7fd103ca1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T05:26:08.347 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:08.201+0000 7fd103ca1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T05:26:08.348 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:08.346+0000 7fd103ca1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T05:26:08.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:08.546+0000 7fd103ca1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T05:26:09.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:08.718+0000 7fd103ca1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T05:26:09.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:08.831+0000 7fd103ca1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T05:26:09.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:09 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:09.311+0000 7fd103ca1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T05:26:10.079 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:09 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:09.820+0000 7fd103ca1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T05:26:10.460 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:10 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:10.078+0000 7fd103ca1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T05:26:10.460 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:10 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:10.205+0000 7fd103ca1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T05:26:11.460 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:11 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:11.174+0000 7fd103ca1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T05:26:11.460 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:11 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:11.304+0000 7fd103ca1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T05:26:11.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:11 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:11.500+0000 7fd103ca1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T05:26:12.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:12 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:12.323+0000 7fd103ca1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T05:26:12.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:12 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:12.571+0000 7fd103ca1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T05:26:13.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:12 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:12.833+0000 7fd103ca1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T05:26:14.372 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:13 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:13.988+0000 7fd103ca1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T05:26:14.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:14.371+0000 7fd103ca1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T05:26:14.710 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:14.491+0000 7fd103ca1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T05:26:15.210 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:26:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:26:14.750+0000 7fd103ca1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T05:26:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: Active manager daemon x restarted 2024-05-10T05:26:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: Activating manager daemon x 2024-05-10T05:26:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: osdmap e3: 0 total, 0 up, 0 in 2024-05-10T05:26:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: mgrmap e10: x(active, starting, since 0.00456924s) 2024-05-10T05:26:15.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:26:15.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T05:26:15.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T05:26:15.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T05:26:15.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T05:26:15.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: Manager daemon x is now available 2024-05-10T05:26:15.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:16.091 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T05:26:16.091 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T05:26:16.091 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:15 smithi131 ceph-mon[37264]: [10/May/2024:05:26:14] ENGINE Bus STARTING 2024-05-10T05:26:16.091 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:15 smithi131 ceph-mon[37264]: mgrmap e11: x(active, since 1.00831s) 2024-05-10T05:26:16.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-05-10T05:26:16.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-10T05:26:16.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T05:26:16.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-05-10T05:26:16.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.849+0000 7f7b84200700 1 Processor -- start 2024-05-10T05:26:16.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.849+0000 7f7b84200700 1 -- start start 2024-05-10T05:26:16.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.850+0000 7f7b84200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c07a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:16.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.850+0000 7f7b84200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c07a770 con 0x7f7b7c07be00 2024-05-10T05:26:16.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.850+0000 7f7b81800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c07a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:16.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.850+0000 7f7b81800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c07a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59776/0 (socket says 172.21.15.131:59776) 2024-05-10T05:26:16.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.850+0000 7f7b81800700 1 -- 172.21.15.131:0/2215129428 learned_addr learned my addr 172.21.15.131:0/2215129428 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:16.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.850+0000 7f7b81800700 1 -- 172.21.15.131:0/2215129428 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c07a8b0 con 0x7f7b7c07be00 2024-05-10T05:26:16.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.851+0000 7f7b81800700 1 --2- 172.21.15.131:0/2215129428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c07a230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7b700098a0 tx=0x7f7b70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe91944c94fc1c85 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:16.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.851+0000 7f7b7be00700 1 -- 172.21.15.131:0/2215129428 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b70004160 con 0x7f7b7c07be00 2024-05-10T05:26:16.255 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.851+0000 7f7b7be00700 1 -- 172.21.15.131:0/2215129428 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b700042c0 con 0x7f7b7c07be00 2024-05-10T05:26:16.255 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.851+0000 7f7b7be00700 1 -- 172.21.15.131:0/2215129428 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b700119b0 con 0x7f7b7c07be00 2024-05-10T05:26:16.255 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.852+0000 7f7b84200700 1 -- 172.21.15.131:0/2215129428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 msgr2=0x7f7b7c07a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:16.255 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.852+0000 7f7b84200700 1 --2- 172.21.15.131:0/2215129428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c07a230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7b700098a0 tx=0x7f7b70009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:16.255 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.852+0000 7f7b84200700 1 -- 172.21.15.131:0/2215129428 shutdown_connections 2024-05-10T05:26:16.255 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.852+0000 7f7b84200700 1 --2- 172.21.15.131:0/2215129428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c07a230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:16.256 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.852+0000 7f7b84200700 1 -- 172.21.15.131:0/2215129428 >> 172.21.15.131:0/2215129428 conn(0x7f7b7c102840 msgr2=0x7f7b7c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:16.256 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.852+0000 7f7b84200700 1 -- 172.21.15.131:0/2215129428 shutdown_connections 2024-05-10T05:26:16.256 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.852+0000 7f7b84200700 1 -- 172.21.15.131:0/2215129428 wait complete. 2024-05-10T05:26:16.256 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.853+0000 7f7b84200700 1 Processor -- start 2024-05-10T05:26:16.256 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.853+0000 7f7b84200700 1 -- start start 2024-05-10T05:26:16.257 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.854+0000 7f7b84200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:16.257 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.854+0000 7f7b84200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c19db80 con 0x7f7b7c07be00 2024-05-10T05:26:16.257 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.854+0000 7f7b81800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:16.257 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.854+0000 7f7b81800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59782/0 (socket says 172.21.15.131:59782) 2024-05-10T05:26:16.257 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.854+0000 7f7b81800700 1 -- 172.21.15.131:0/1318571551 learned_addr learned my addr 172.21.15.131:0/1318571551 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:16.257 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.854+0000 7f7b81800700 1 -- 172.21.15.131:0/1318571551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70009550 con 0x7f7b7c07be00 2024-05-10T05:26:16.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.855+0000 7f7b81800700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c19d640 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7b70003e60 tx=0x7f7b70011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:16.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.855+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b700038b0 con 0x7f7b7c07be00 2024-05-10T05:26:16.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.855+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b70003a10 con 0x7f7b7c07be00 2024-05-10T05:26:16.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.855+0000 7f7b84200700 1 -- 172.21.15.131:0/1318571551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7c19dd80 con 0x7f7b7c07be00 2024-05-10T05:26:16.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.855+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b70019530 con 0x7f7b7c07be00 2024-05-10T05:26:16.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.855+0000 7f7b84200700 1 -- 172.21.15.131:0/1318571551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7c19e220 con 0x7f7b7c07be00 2024-05-10T05:26:16.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.856+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f7b700197f0 con 0x7f7b7c07be00 2024-05-10T05:26:16.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.857+0000 7f7b7a000700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 0x7f7b6403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:16.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.857+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 --> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b6403b320 con 0x7f7b64038700 2024-05-10T05:26:16.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.857+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7b7004d920 con 0x7f7b7c07be00 2024-05-10T05:26:16.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.857+0000 7f7b80e00700 1 -- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 msgr2=0x7f7b6403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3754674139 2024-05-10T05:26:16.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:07.857+0000 7f7b80e00700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 0x7f7b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T05:26:16.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:08.058+0000 7f7b80e00700 1 -- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 msgr2=0x7f7b6403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3754674139 2024-05-10T05:26:16.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:08.058+0000 7f7b80e00700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 0x7f7b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T05:26:16.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:08.459+0000 7f7b80e00700 1 -- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 msgr2=0x7f7b6403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3754674139 2024-05-10T05:26:16.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:08.459+0000 7f7b80e00700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 0x7f7b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T05:26:16.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:09.260+0000 7f7b80e00700 1 -- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 msgr2=0x7f7b6403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3754674139 2024-05-10T05:26:16.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:09.260+0000 7f7b80e00700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 0x7f7b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T05:26:16.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:10.861+0000 7f7b80e00700 1 -- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 msgr2=0x7f7b6403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3754674139 2024-05-10T05:26:16.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:10.862+0000 7f7b80e00700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 0x7f7b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T05:26:16.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:14.065+0000 7f7b80e00700 1 -- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 msgr2=0x7f7b6403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/3754674139 2024-05-10T05:26:16.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:14.065+0000 7f7b80e00700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 0x7f7b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-10T05:26:16.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:14.755+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7f7b70019d20 con 0x7f7b7c07be00 2024-05-10T05:26:16.261 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:14.755+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 msgr2=0x7f7b6403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:14.755+0000 7f7b7a000700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 0x7f7b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.757+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f7b7004e8f0 con 0x7f7b7c07be00 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.758+0000 7f7b7a000700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b7004d4b0 0x7f7b6403e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.758+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b6403b320 con 0x7f7b7004d4b0 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.759+0000 7f7b80e00700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b7004d4b0 0x7f7b6403e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.760+0000 7f7b80e00700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b7004d4b0 0x7f7b6403e000 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7b68003a10 tx=0x7f7b680092b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.761+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f7b6403b320 con 0x7f7b7004d4b0 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.768+0000 7f7b84200700 1 -- 172.21.15.131:0/1318571551 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7b7c10d660 con 0x7f7b7004d4b0 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.769+0000 7f7b7a000700 1 -- 172.21.15.131:0/1318571551 <== mgr.14154 v2:172.21.15.131:6800/2109111580 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f7b7c10d660 con 0x7f7b7004d4b0 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.769+0000 7f7b84200700 1 -- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b7004d4b0 msgr2=0x7f7b6403e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.769+0000 7f7b84200700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b7004d4b0 0x7f7b6403e000 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7b68003a10 tx=0x7f7b680092b0 comp rx=0 tx=0).stop 2024-05-10T05:26:16.262 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.769+0000 7f7b84200700 1 -- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 msgr2=0x7f7b7c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.769+0000 7f7b84200700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c19d640 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7b70003e60 tx=0x7f7b70011a30 comp rx=0 tx=0).stop 2024-05-10T05:26:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.769+0000 7f7b84200700 1 -- 172.21.15.131:0/1318571551 shutdown_connections 2024-05-10T05:26:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.769+0000 7f7b84200700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b7004d4b0 0x7f7b6403e000 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.769+0000 7f7b84200700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b7c07be00 0x7f7b7c19d640 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.769+0000 7f7b84200700 1 --2- 172.21.15.131:0/1318571551 >> [v2:172.21.15.131:6800/3754674139,v1:172.21.15.131:6801/3754674139] conn(0x7f7b64038700 0x7f7b6403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.769+0000 7f7b84200700 1 -- 172.21.15.131:0/1318571551 >> 172.21.15.131:0/1318571551 conn(0x7f7b7c102840 msgr2=0x7f7b7c103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:16.263 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.770+0000 7f7b84200700 1 -- 172.21.15.131:0/1318571551 shutdown_connections 2024-05-10T05:26:16.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:15.770+0000 7f7b84200700 1 -- 172.21.15.131:0/1318571551 wait complete. 2024-05-10T05:26:16.264 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 9 is available 2024-05-10T05:26:16.264 INFO:teuthology.orchestra.run.smithi131.stdout:Generating a dashboard self-signed certificate... 2024-05-10T05:26:17.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:16 smithi131 ceph-mon[37264]: [10/May/2024:05:26:15] ENGINE Serving on https://172.21.15.131:7150 2024-05-10T05:26:17.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:16 smithi131 ceph-mon[37264]: [10/May/2024:05:26:15] ENGINE Serving on http://172.21.15.131:8765 2024-05-10T05:26:17.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:16 smithi131 ceph-mon[37264]: [10/May/2024:05:26:15] ENGINE Bus STARTED 2024-05-10T05:26:17.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:16 smithi131 ceph-mon[37264]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T05:26:17.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:16 smithi131 ceph-mon[37264]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T05:26:17.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.778+0000 7fe709600700 1 Processor -- start 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.779+0000 7fe709600700 1 -- start start 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.779+0000 7fe709600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe704107720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.779+0000 7fe709600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe704107d10 con 0x7fe704107310 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.780+0000 7fe702a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe704107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.780+0000 7fe702a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe704107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57184/0 (socket says 172.21.15.131:57184) 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.780+0000 7fe702a00700 1 -- 172.21.15.131:0/2041672831 learned_addr learned my addr 172.21.15.131:0/2041672831 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.780+0000 7fe702a00700 1 -- 172.21.15.131:0/2041672831 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe704108440 con 0x7fe704107310 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.780+0000 7fe702a00700 1 --2- 172.21.15.131:0/2041672831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe704107720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40098a0 tx=0x7fe6f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832e121ffe99ef5a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:17.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.781+0000 7fe701600700 1 -- 172.21.15.131:0/2041672831 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6f4004160 con 0x7fe704107310 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.781+0000 7fe701600700 1 -- 172.21.15.131:0/2041672831 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f40042c0 con 0x7fe704107310 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.781+0000 7fe701600700 1 -- 172.21.15.131:0/2041672831 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6f40119b0 con 0x7fe704107310 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.781+0000 7fe709600700 1 -- 172.21.15.131:0/2041672831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 msgr2=0x7fe704107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.781+0000 7fe709600700 1 --2- 172.21.15.131:0/2041672831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe704107720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40098a0 tx=0x7fe6f4009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.781+0000 7fe709600700 1 -- 172.21.15.131:0/2041672831 shutdown_connections 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.781+0000 7fe709600700 1 --2- 172.21.15.131:0/2041672831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe704107720 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.781+0000 7fe709600700 1 -- 172.21.15.131:0/2041672831 >> 172.21.15.131:0/2041672831 conn(0x7fe704102930 msgr2=0x7fe704104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.782+0000 7fe709600700 1 -- 172.21.15.131:0/2041672831 shutdown_connections 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.782+0000 7fe709600700 1 -- 172.21.15.131:0/2041672831 wait complete. 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.783+0000 7fe709600700 1 Processor -- start 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.783+0000 7fe709600700 1 -- start start 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.783+0000 7fe709600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe70407ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:17.520 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.783+0000 7fe709600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70407bfa0 con 0x7fe704107310 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.783+0000 7fe702a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe70407ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.783+0000 7fe702a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe70407ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57200/0 (socket says 172.21.15.131:57200) 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.783+0000 7fe702a00700 1 -- 172.21.15.131:0/2137526562 learned_addr learned my addr 172.21.15.131:0/2137526562 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.784+0000 7fe702a00700 1 -- 172.21.15.131:0/2137526562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f4009550 con 0x7fe704107310 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.784+0000 7fe702a00700 1 --2- 172.21.15.131:0/2137526562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe70407ba60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4011980 tx=0x7fe6f4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.784+0000 7fe6fb400700 1 -- 172.21.15.131:0/2137526562 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6f4003aa0 con 0x7fe704107310 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.784+0000 7fe6fb400700 1 -- 172.21.15.131:0/2137526562 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f4003c00 con 0x7fe704107310 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.784+0000 7fe709600700 1 -- 172.21.15.131:0/2137526562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70407a1f0 con 0x7fe704107310 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.784+0000 7fe6fb400700 1 -- 172.21.15.131:0/2137526562 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6f4019670 con 0x7fe704107310 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.784+0000 7fe709600700 1 -- 172.21.15.131:0/2137526562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70407a690 con 0x7fe704107310 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.786+0000 7fe6fb400700 1 -- 172.21.15.131:0/2137526562 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fe6f40198e0 con 0x7fe704107310 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.786+0000 7fe6f8c00700 1 -- 172.21.15.131:0/2137526562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70407b0a0 con 0x7fe704107310 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.786+0000 7fe6fb400700 1 --2- 172.21.15.131:0/2137526562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe6e8038670 0x7fe6e803ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.786+0000 7fe6fb400700 1 -- 172.21.15.131:0/2137526562 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe6f404c930 con 0x7fe704107310 2024-05-10T05:26:17.521 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.786+0000 7fe702000700 1 --2- 172.21.15.131:0/2137526562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe6e8038670 0x7fe6e803ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.787+0000 7fe702000700 1 --2- 172.21.15.131:0/2137526562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe6e8038670 0x7fe6e803ab80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0006fd0 tx=0x7fe6f0006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:16.792+0000 7fe6fb400700 1 -- 172.21.15.131:0/2137526562 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe6f4018db0 con 0x7fe704107310 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.011+0000 7fe6f8c00700 1 -- 172.21.15.131:0/2137526562 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fe70407a8d0 con 0x7fe6e8038670 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.125+0000 7fe6fb400700 1 -- 172.21.15.131:0/2137526562 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fe70407a8d0 con 0x7fe6e8038670 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.127+0000 7fe6f8c00700 1 -- 172.21.15.131:0/2137526562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe6e8038670 msgr2=0x7fe6e803ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.127+0000 7fe6f8c00700 1 --2- 172.21.15.131:0/2137526562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe6e8038670 0x7fe6e803ab80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0006fd0 tx=0x7fe6f0006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.128+0000 7fe6f8c00700 1 -- 172.21.15.131:0/2137526562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 msgr2=0x7fe70407ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.128+0000 7fe6f8c00700 1 --2- 172.21.15.131:0/2137526562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe70407ba60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4011980 tx=0x7fe6f4011a60 comp rx=0 tx=0).stop 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.128+0000 7fe6f8c00700 1 -- 172.21.15.131:0/2137526562 shutdown_connections 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.128+0000 7fe6f8c00700 1 --2- 172.21.15.131:0/2137526562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe6e8038670 0x7fe6e803ab80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.128+0000 7fe6f8c00700 1 --2- 172.21.15.131:0/2137526562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe704107310 0x7fe70407ba60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:17.522 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.128+0000 7fe6f8c00700 1 -- 172.21.15.131:0/2137526562 >> 172.21.15.131:0/2137526562 conn(0x7fe704102930 msgr2=0x7fe7041033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:17.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.128+0000 7fe6f8c00700 1 -- 172.21.15.131:0/2137526562 shutdown_connections 2024-05-10T05:26:17.523 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:17.128+0000 7fe6f8c00700 1 -- 172.21.15.131:0/2137526562 wait complete. 2024-05-10T05:26:17.523 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial admin user... 2024-05-10T05:26:18.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:18 smithi131 ceph-mon[37264]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:18.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:18 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:18.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:18 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:18.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:18 smithi131 ceph-mon[37264]: mgrmap e12: x(active, since 2s) 2024-05-10T05:26:18.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:18 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:19.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Ji8Qt.z6x44s3Ot781559.jqFfYOxjdLWZn8xekg2fcT9gQ6ElNbm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715318778, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-10T05:26:19.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.010+0000 7f7e68000700 1 Processor -- start 2024-05-10T05:26:19.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.011+0000 7f7e68000700 1 -- start start 2024-05-10T05:26:19.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.011+0000 7f7e68000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e60107720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.011+0000 7f7e68000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60107d10 con 0x7f7e60107310 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.011+0000 7f7e65600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e60107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.011+0000 7f7e65600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e60107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57212/0 (socket says 172.21.15.131:57212) 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.011+0000 7f7e65600700 1 -- 172.21.15.131:0/1563389992 learned_addr learned my addr 172.21.15.131:0/1563389992 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.012+0000 7f7e65600700 1 -- 172.21.15.131:0/1563389992 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e60108440 con 0x7f7e60107310 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.012+0000 7f7e65600700 1 --2- 172.21.15.131:0/1563389992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e60107720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e540098a0 tx=0x7f7e54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641617afbc835623 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.012+0000 7f7e5fe00700 1 -- 172.21.15.131:0/1563389992 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e54004160 con 0x7f7e60107310 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.012+0000 7f7e5fe00700 1 -- 172.21.15.131:0/1563389992 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e540042c0 con 0x7f7e60107310 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.012+0000 7f7e5fe00700 1 -- 172.21.15.131:0/1563389992 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e540119b0 con 0x7f7e60107310 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.014+0000 7f7e68000700 1 -- 172.21.15.131:0/1563389992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 msgr2=0x7f7e60107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.014+0000 7f7e68000700 1 --2- 172.21.15.131:0/1563389992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e60107720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e540098a0 tx=0x7f7e54009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.014+0000 7f7e68000700 1 -- 172.21.15.131:0/1563389992 shutdown_connections 2024-05-10T05:26:19.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.014+0000 7f7e68000700 1 --2- 172.21.15.131:0/1563389992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e60107720 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.014+0000 7f7e68000700 1 -- 172.21.15.131:0/1563389992 >> 172.21.15.131:0/1563389992 conn(0x7f7e60102930 msgr2=0x7f7e60104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.014+0000 7f7e68000700 1 -- 172.21.15.131:0/1563389992 shutdown_connections 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.014+0000 7f7e68000700 1 -- 172.21.15.131:0/1563389992 wait complete. 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.015+0000 7f7e68000700 1 Processor -- start 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.015+0000 7f7e68000700 1 -- start start 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.015+0000 7f7e68000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e6019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.015+0000 7f7e68000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6019b9c0 con 0x7f7e60107310 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.016+0000 7f7e65600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e6019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.016+0000 7f7e65600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e6019b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57224/0 (socket says 172.21.15.131:57224) 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.016+0000 7f7e65600700 1 -- 172.21.15.131:0/2384651261 learned_addr learned my addr 172.21.15.131:0/2384651261 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:19.080 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.016+0000 7f7e65600700 1 -- 172.21.15.131:0/2384651261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54009550 con 0x7f7e60107310 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.016+0000 7f7e65600700 1 --2- 172.21.15.131:0/2384651261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e6019b480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e54011980 tx=0x7f7e54011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.016+0000 7f7e5e000700 1 -- 172.21.15.131:0/2384651261 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e54003aa0 con 0x7f7e60107310 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.017+0000 7f7e5e000700 1 -- 172.21.15.131:0/2384651261 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e54003c00 con 0x7f7e60107310 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.017+0000 7f7e68000700 1 -- 172.21.15.131:0/2384651261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6019bbc0 con 0x7f7e60107310 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.017+0000 7f7e5e000700 1 -- 172.21.15.131:0/2384651261 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e54019670 con 0x7f7e60107310 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.017+0000 7f7e68000700 1 -- 172.21.15.131:0/2384651261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6019c060 con 0x7f7e60107310 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.018+0000 7f7e5e000700 1 -- 172.21.15.131:0/2384651261 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f7e540198e0 con 0x7f7e60107310 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.018+0000 7f7e68000700 1 -- 172.21.15.131:0/2384651261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6010b670 con 0x7f7e60107310 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.018+0000 7f7e5e000700 1 --2- 172.21.15.131:0/2384651261 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7e480387a0 0x7f7e4803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.019+0000 7f7e5e000700 1 -- 172.21.15.131:0/2384651261 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7e5404c9c0 con 0x7f7e60107310 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.019+0000 7f7e64c00700 1 --2- 172.21.15.131:0/2384651261 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7e480387a0 0x7f7e4803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.020+0000 7f7e64c00700 1 --2- 172.21.15.131:0/2384651261 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7e480387a0 0x7f7e4803acb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c006fd0 tx=0x7f7e4c006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.028+0000 7f7e5e000700 1 -- 172.21.15.131:0/2384651261 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7e54018db0 con 0x7f7e60107310 2024-05-10T05:26:19.081 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.266+0000 7f7e68000700 1 -- 172.21.15.131:0/2384651261 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7e6010b880 con 0x7f7e480387a0 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.496+0000 7f7e5e000700 1 -- 172.21.15.131:0/2384651261 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f7e6010b880 con 0x7f7e480387a0 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.498+0000 7f7e68000700 1 -- 172.21.15.131:0/2384651261 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7e480387a0 msgr2=0x7f7e4803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.498+0000 7f7e68000700 1 --2- 172.21.15.131:0/2384651261 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7e480387a0 0x7f7e4803acb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c006fd0 tx=0x7f7e4c006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.498+0000 7f7e68000700 1 -- 172.21.15.131:0/2384651261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 msgr2=0x7f7e6019b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.498+0000 7f7e68000700 1 --2- 172.21.15.131:0/2384651261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e6019b480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e54011980 tx=0x7f7e54011a60 comp rx=0 tx=0).stop 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.499+0000 7f7e68000700 1 -- 172.21.15.131:0/2384651261 shutdown_connections 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.499+0000 7f7e68000700 1 --2- 172.21.15.131:0/2384651261 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7e480387a0 0x7f7e4803acb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.499+0000 7f7e68000700 1 --2- 172.21.15.131:0/2384651261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60107310 0x7f7e6019b480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.499+0000 7f7e68000700 1 -- 172.21.15.131:0/2384651261 >> 172.21.15.131:0/2384651261 conn(0x7f7e60102930 msgr2=0x7f7e60103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.499+0000 7f7e68000700 1 -- 172.21.15.131:0/2384651261 shutdown_connections 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:18.499+0000 7f7e68000700 1 -- 172.21.15.131:0/2384651261 wait complete. 2024-05-10T05:26:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:Fetching dashboard port number... 2024-05-10T05:26:19.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:19 smithi131 ceph-mon[37264]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:19.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:19 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:20.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 8443 2024-05-10T05:26:20.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.566+0000 7f04d2200700 1 Processor -- start 2024-05-10T05:26:20.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.567+0000 7f04d2200700 1 -- start start 2024-05-10T05:26:20.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.567+0000 7f04d2200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc07a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:20.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.567+0000 7f04d2200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc07a820 con 0x7f04cc07be20 2024-05-10T05:26:20.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.567+0000 7f04cb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc07a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:20.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.567+0000 7f04cb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc07a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57230/0 (socket says 172.21.15.131:57230) 2024-05-10T05:26:20.232 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.567+0000 7f04cb400700 1 -- 172.21.15.131:0/3348659186 learned_addr learned my addr 172.21.15.131:0/3348659186 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.568+0000 7f04cb400700 1 -- 172.21.15.131:0/3348659186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc07a960 con 0x7f04cc07be20 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.568+0000 7f04cb400700 1 --2- 172.21.15.131:0/3348659186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc07a250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0098a0 tx=0x7f04bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213390097b9d249a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.568+0000 7f04caa00700 1 -- 172.21.15.131:0/3348659186 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04bc004160 con 0x7f04cc07be20 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.568+0000 7f04caa00700 1 -- 172.21.15.131:0/3348659186 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04bc0042c0 con 0x7f04cc07be20 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.568+0000 7f04caa00700 1 -- 172.21.15.131:0/3348659186 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04bc0119b0 con 0x7f04cc07be20 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.569+0000 7f04d2200700 1 -- 172.21.15.131:0/3348659186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 msgr2=0x7f04cc07a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.569+0000 7f04d2200700 1 --2- 172.21.15.131:0/3348659186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc07a250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0098a0 tx=0x7f04bc009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.569+0000 7f04d2200700 1 -- 172.21.15.131:0/3348659186 shutdown_connections 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.569+0000 7f04d2200700 1 --2- 172.21.15.131:0/3348659186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc07a250 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.569+0000 7f04d2200700 1 -- 172.21.15.131:0/3348659186 >> 172.21.15.131:0/3348659186 conn(0x7f04cc102830 msgr2=0x7f04cc104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.569+0000 7f04d2200700 1 -- 172.21.15.131:0/3348659186 shutdown_connections 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.569+0000 7f04d2200700 1 -- 172.21.15.131:0/3348659186 wait complete. 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.570+0000 7f04d2200700 1 Processor -- start 2024-05-10T05:26:20.233 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.570+0000 7f04d2200700 1 -- start start 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.571+0000 7f04d2200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.571+0000 7f04d2200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc19dbc0 con 0x7f04cc07be20 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.571+0000 7f04cb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.571+0000 7f04cb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57244/0 (socket says 172.21.15.131:57244) 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.571+0000 7f04cb400700 1 -- 172.21.15.131:0/2344244341 learned_addr learned my addr 172.21.15.131:0/2344244341 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.571+0000 7f04cb400700 1 -- 172.21.15.131:0/2344244341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc009550 con 0x7f04cc07be20 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.572+0000 7f04cb400700 1 --2- 172.21.15.131:0/2344244341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc19d680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04bc011670 tx=0x7f04bc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.572+0000 7f04c8c00700 1 -- 172.21.15.131:0/2344244341 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04bc003af0 con 0x7f04cc07be20 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.572+0000 7f04c8c00700 1 -- 172.21.15.131:0/2344244341 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04bc003c50 con 0x7f04cc07be20 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.572+0000 7f04d2200700 1 -- 172.21.15.131:0/2344244341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc19ddc0 con 0x7f04cc07be20 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.572+0000 7f04c8c00700 1 -- 172.21.15.131:0/2344244341 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04bc024820 con 0x7f04cc07be20 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.572+0000 7f04d2200700 1 -- 172.21.15.131:0/2344244341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04cc19e260 con 0x7f04cc07be20 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.573+0000 7f04c8c00700 1 -- 172.21.15.131:0/2344244341 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f04bc024a50 con 0x7f04cc07be20 2024-05-10T05:26:20.234 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.573+0000 7f04d2200700 1 -- 172.21.15.131:0/2344244341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04cc197290 con 0x7f04cc07be20 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.574+0000 7f04c8c00700 1 --2- 172.21.15.131:0/2344244341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04b00387a0 0x7f04b003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.574+0000 7f04c8c00700 1 -- 172.21.15.131:0/2344244341 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f04bc04ca90 con 0x7f04cc07be20 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.574+0000 7f04c3e00700 1 --2- 172.21.15.131:0/2344244341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04b00387a0 0x7f04b003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.575+0000 7f04c3e00700 1 --2- 172.21.15.131:0/2344244341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04b00387a0 0x7f04b003acb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f04b4006fd0 tx=0x7f04b4006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.580+0000 7f04c8c00700 1 -- 172.21.15.131:0/2344244341 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f04bc0189a0 con 0x7f04cc07be20 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.781+0000 7f04d2200700 1 -- 172.21.15.131:0/2344244341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f04cc10d640 con 0x7f04cc07be20 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.781+0000 7f04c8c00700 1 -- 172.21.15.131:0/2344244341 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f04bc029210 con 0x7f04cc07be20 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.784+0000 7f04d2200700 1 -- 172.21.15.131:0/2344244341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04b00387a0 msgr2=0x7f04b003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.784+0000 7f04d2200700 1 --2- 172.21.15.131:0/2344244341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04b00387a0 0x7f04b003acb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f04b4006fd0 tx=0x7f04b4006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.784+0000 7f04d2200700 1 -- 172.21.15.131:0/2344244341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 msgr2=0x7f04cc19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.784+0000 7f04d2200700 1 --2- 172.21.15.131:0/2344244341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc19d680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04bc011670 tx=0x7f04bc011a30 comp rx=0 tx=0).stop 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.784+0000 7f04d2200700 1 -- 172.21.15.131:0/2344244341 shutdown_connections 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.784+0000 7f04d2200700 1 --2- 172.21.15.131:0/2344244341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04b00387a0 0x7f04b003acb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.784+0000 7f04d2200700 1 --2- 172.21.15.131:0/2344244341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04cc07be20 0x7f04cc19d680 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:20.235 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.784+0000 7f04d2200700 1 -- 172.21.15.131:0/2344244341 >> 172.21.15.131:0/2344244341 conn(0x7f04cc102830 msgr2=0x7f04cc103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:20.236 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.784+0000 7f04d2200700 1 -- 172.21.15.131:0/2344244341 shutdown_connections 2024-05-10T05:26:20.236 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:19.784+0000 7f04d2200700 1 -- 172.21.15.131:0/2344244341 wait complete. 2024-05-10T05:26:20.248 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T05:26:20.248 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-05-10T05:26:20.263 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T05:26:20.263 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-05-10T05:26:20.263 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-05-10T05:26:20.263 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-10T05:26:20.266 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph Dashboard is now available at: 2024-05-10T05:26:20.266 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:20.266 INFO:teuthology.orchestra.run.smithi131.stdout: URL: https://smithi131.front.sepia.ceph.com:8443/ 2024-05-10T05:26:20.266 INFO:teuthology.orchestra.run.smithi131.stdout: User: admin 2024-05-10T05:26:20.266 INFO:teuthology.orchestra.run.smithi131.stdout: Password: oqiahhdq1g 2024-05-10T05:26:20.266 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:20.266 INFO:teuthology.orchestra.run.smithi131.stdout:Saving cluster configuration to /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config directory 2024-05-10T05:26:20.268 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling autotune for osd_memory_target 2024-05-10T05:26:20.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:20 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2344244341' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-10T05:26:21.382 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.797+0000 7f999cc00700 1 Processor -- start 2024-05-10T05:26:21.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.797+0000 7f999cc00700 1 -- start start 2024-05-10T05:26:21.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.798+0000 7f999cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f9998109f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:21.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.798+0000 7f999cc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998079e70 con 0x7f9998107b50 2024-05-10T05:26:21.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.798+0000 7f9997400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f9998109f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:21.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.798+0000 7f9997400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f9998109f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57260/0 (socket says 172.21.15.131:57260) 2024-05-10T05:26:21.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.798+0000 7f9997400700 1 -- 172.21.15.131:0/245842422 learned_addr learned my addr 172.21.15.131:0/245842422 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:21.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.798+0000 7f9997400700 1 -- 172.21.15.131:0/245842422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999810a4b0 con 0x7f9998107b50 2024-05-10T05:26:21.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.799+0000 7f9997400700 1 --2- 172.21.15.131:0/245842422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f9998109f70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99880098a0 tx=0x7f9988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a77b4b3883b8b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:21.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.799+0000 7f9996000700 1 -- 172.21.15.131:0/245842422 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9988004160 con 0x7f9998107b50 2024-05-10T05:26:21.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.799+0000 7f9996000700 1 -- 172.21.15.131:0/245842422 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99880042c0 con 0x7f9998107b50 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.799+0000 7f9996000700 1 -- 172.21.15.131:0/245842422 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99880119b0 con 0x7f9998107b50 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.800+0000 7f999cc00700 1 -- 172.21.15.131:0/245842422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 msgr2=0x7f9998109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.800+0000 7f999cc00700 1 --2- 172.21.15.131:0/245842422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f9998109f70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99880098a0 tx=0x7f9988009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.800+0000 7f999cc00700 1 -- 172.21.15.131:0/245842422 shutdown_connections 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.800+0000 7f999cc00700 1 --2- 172.21.15.131:0/245842422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f9998109f70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.800+0000 7f999cc00700 1 -- 172.21.15.131:0/245842422 >> 172.21.15.131:0/245842422 conn(0x7f9998101590 msgr2=0x7f9998103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.800+0000 7f999cc00700 1 -- 172.21.15.131:0/245842422 shutdown_connections 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.800+0000 7f999cc00700 1 -- 172.21.15.131:0/245842422 wait complete. 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.801+0000 7f999cc00700 1 Processor -- start 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.802+0000 7f999cc00700 1 -- start start 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.802+0000 7f999cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f999819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:21.389 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.802+0000 7f999cc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999819b7f0 con 0x7f9998107b50 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.802+0000 7f9997400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f999819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.802+0000 7f9997400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f999819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57272/0 (socket says 172.21.15.131:57272) 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.802+0000 7f9997400700 1 -- 172.21.15.131:0/3616511562 learned_addr learned my addr 172.21.15.131:0/3616511562 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.803+0000 7f9997400700 1 -- 172.21.15.131:0/3616511562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9988009550 con 0x7f9998107b50 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.803+0000 7f9997400700 1 --2- 172.21.15.131:0/3616511562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f999819b2b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9988011670 tx=0x7f9988011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.803+0000 7f998fe00700 1 -- 172.21.15.131:0/3616511562 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9988003af0 con 0x7f9998107b50 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.803+0000 7f998fe00700 1 -- 172.21.15.131:0/3616511562 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9988003c50 con 0x7f9998107b50 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.803+0000 7f999cc00700 1 -- 172.21.15.131:0/3616511562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999819b9f0 con 0x7f9998107b50 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.803+0000 7f998fe00700 1 -- 172.21.15.131:0/3616511562 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9988024730 con 0x7f9998107b50 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.803+0000 7f999cc00700 1 -- 172.21.15.131:0/3616511562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999819be90 con 0x7f9998107b50 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.804+0000 7f999cc00700 1 -- 172.21.15.131:0/3616511562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999810b290 con 0x7f9998107b50 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.805+0000 7f998fe00700 1 -- 172.21.15.131:0/3616511562 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f998801a430 con 0x7f9998107b50 2024-05-10T05:26:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.805+0000 7f998fe00700 1 --2- 172.21.15.131:0/3616511562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f997c0387a0 0x7f997c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.805+0000 7f998fe00700 1 -- 172.21.15.131:0/3616511562 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f998804c830 con 0x7f9998107b50 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.805+0000 7f9996a00700 1 --2- 172.21.15.131:0/3616511562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f997c0387a0 0x7f997c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.806+0000 7f9996a00700 1 --2- 172.21.15.131:0/3616511562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f997c0387a0 0x7f997c03acb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9984006fd0 tx=0x7f9984006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:20.810+0000 7f998fe00700 1 -- 172.21.15.131:0/3616511562 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9988024e40 con 0x7f9998107b50 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.009+0000 7f999cc00700 1 -- 172.21.15.131:0/3616511562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f999810b4a0 con 0x7f9998107b50 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.010+0000 7f998fe00700 1 -- 172.21.15.131:0/3616511562 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f9988018350 con 0x7f9998107b50 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.012+0000 7f999cc00700 1 -- 172.21.15.131:0/3616511562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f997c0387a0 msgr2=0x7f997c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.012+0000 7f999cc00700 1 --2- 172.21.15.131:0/3616511562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f997c0387a0 0x7f997c03acb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9984006fd0 tx=0x7f9984006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.012+0000 7f999cc00700 1 -- 172.21.15.131:0/3616511562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 msgr2=0x7f999819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.012+0000 7f999cc00700 1 --2- 172.21.15.131:0/3616511562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f999819b2b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9988011670 tx=0x7f9988011a30 comp rx=0 tx=0).stop 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.012+0000 7f999cc00700 1 -- 172.21.15.131:0/3616511562 shutdown_connections 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.012+0000 7f999cc00700 1 --2- 172.21.15.131:0/3616511562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f997c0387a0 0x7f997c03acb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:21.391 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.012+0000 7f999cc00700 1 --2- 172.21.15.131:0/3616511562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9998107b50 0x7f999819b2b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:21.392 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.012+0000 7f999cc00700 1 -- 172.21.15.131:0/3616511562 >> 172.21.15.131:0/3616511562 conn(0x7f9998101590 msgr2=0x7f9998102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:21.392 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.013+0000 7f999cc00700 1 -- 172.21.15.131:0/3616511562 shutdown_connections 2024-05-10T05:26:21.392 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.013+0000 7f999cc00700 1 -- 172.21.15.131:0/3616511562 wait complete. 2024-05-10T05:26:22.550 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.854+0000 7fd37b200700 1 Processor -- start 2024-05-10T05:26:22.550 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.854+0000 7fd37b200700 1 -- start start 2024-05-10T05:26:22.550 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.855+0000 7fd37b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c107480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:22.550 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.855+0000 7fd37b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c107a70 con 0x7fd37c107070 2024-05-10T05:26:22.550 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.855+0000 7fd379e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c107480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:22.550 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.855+0000 7fd379e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c107480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57288/0 (socket says 172.21.15.131:57288) 2024-05-10T05:26:22.550 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.855+0000 7fd379e00700 1 -- 172.21.15.131:0/1036449307 learned_addr learned my addr 172.21.15.131:0/1036449307 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:22.550 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.856+0000 7fd379e00700 1 -- 172.21.15.131:0/1036449307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c108240 con 0x7fd37c107070 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.856+0000 7fd379e00700 1 --2- 172.21.15.131:0/1036449307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c107480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0098a0 tx=0x7fd36c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b77d330c1b57cfe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.856+0000 7fd378a00700 1 -- 172.21.15.131:0/1036449307 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd36c004160 con 0x7fd37c107070 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.856+0000 7fd378a00700 1 -- 172.21.15.131:0/1036449307 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd36c0042c0 con 0x7fd37c107070 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.856+0000 7fd378a00700 1 -- 172.21.15.131:0/1036449307 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd36c0119b0 con 0x7fd37c107070 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.857+0000 7fd37b200700 1 -- 172.21.15.131:0/1036449307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 msgr2=0x7fd37c107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.857+0000 7fd37b200700 1 --2- 172.21.15.131:0/1036449307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c107480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0098a0 tx=0x7fd36c009bb0 comp rx=0 tx=0).stop 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.857+0000 7fd37b200700 1 -- 172.21.15.131:0/1036449307 shutdown_connections 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.857+0000 7fd37b200700 1 --2- 172.21.15.131:0/1036449307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c107480 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.857+0000 7fd37b200700 1 -- 172.21.15.131:0/1036449307 >> 172.21.15.131:0/1036449307 conn(0x7fd37c102690 msgr2=0x7fd37c104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.857+0000 7fd37b200700 1 -- 172.21.15.131:0/1036449307 shutdown_connections 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.857+0000 7fd37b200700 1 -- 172.21.15.131:0/1036449307 wait complete. 2024-05-10T05:26:22.551 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.858+0000 7fd37b200700 1 Processor -- start 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.859+0000 7fd37b200700 1 -- start start 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.859+0000 7fd37b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.859+0000 7fd37b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c19da80 con 0x7fd37c107070 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.859+0000 7fd379e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.859+0000 7fd379e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c19d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57298/0 (socket says 172.21.15.131:57298) 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.859+0000 7fd379e00700 1 -- 172.21.15.131:0/93078954 learned_addr learned my addr 172.21.15.131:0/93078954 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.860+0000 7fd379e00700 1 -- 172.21.15.131:0/93078954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c009550 con 0x7fd37c107070 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.860+0000 7fd379e00700 1 --2- 172.21.15.131:0/93078954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c19d540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd36c011980 tx=0x7fd36c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.860+0000 7fd372a00700 1 -- 172.21.15.131:0/93078954 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd36c003aa0 con 0x7fd37c107070 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.860+0000 7fd372a00700 1 -- 172.21.15.131:0/93078954 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd36c003c00 con 0x7fd37c107070 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.860+0000 7fd37b200700 1 -- 172.21.15.131:0/93078954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c19dc80 con 0x7fd37c107070 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.860+0000 7fd372a00700 1 -- 172.21.15.131:0/93078954 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd36c019670 con 0x7fd37c107070 2024-05-10T05:26:22.552 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.860+0000 7fd37b200700 1 -- 172.21.15.131:0/93078954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c19e120 con 0x7fd37c107070 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.861+0000 7fd372a00700 1 -- 172.21.15.131:0/93078954 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fd36c0198e0 con 0x7fd37c107070 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.862+0000 7fd37b200700 1 -- 172.21.15.131:0/93078954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c10b470 con 0x7fd37c107070 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.862+0000 7fd372a00700 1 --2- 172.21.15.131:0/93078954 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd360038840 0x7fd36003ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.862+0000 7fd372a00700 1 -- 172.21.15.131:0/93078954 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd36c04cb40 con 0x7fd37c107070 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.862+0000 7fd379400700 1 --2- 172.21.15.131:0/93078954 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd360038840 0x7fd36003ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.863+0000 7fd379400700 1 --2- 172.21.15.131:0/93078954 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd360038840 0x7fd36003ad50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd368006fd0 tx=0x7fd368006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:21.867+0000 7fd372a00700 1 -- 172.21.15.131:0/93078954 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd36c018db0 con 0x7fd37c107070 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.177+0000 7fd37b200700 1 -- 172.21.15.131:0/93078954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fd37c10b680 con 0x7fd37c107070 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.180+0000 7fd372a00700 1 -- 172.21.15.131:0/93078954 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fd36c018350 con 0x7fd37c107070 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.183+0000 7fd37b200700 1 -- 172.21.15.131:0/93078954 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd360038840 msgr2=0x7fd36003ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.183+0000 7fd37b200700 1 --2- 172.21.15.131:0/93078954 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd360038840 0x7fd36003ad50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd368006fd0 tx=0x7fd368006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.183+0000 7fd37b200700 1 -- 172.21.15.131:0/93078954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 msgr2=0x7fd37c19d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:22.553 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.183+0000 7fd37b200700 1 --2- 172.21.15.131:0/93078954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c19d540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd36c011980 tx=0x7fd36c011a60 comp rx=0 tx=0).stop 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.183+0000 7fd37b200700 1 -- 172.21.15.131:0/93078954 shutdown_connections 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.183+0000 7fd37b200700 1 --2- 172.21.15.131:0/93078954 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd360038840 0x7fd36003ad50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.183+0000 7fd37b200700 1 --2- 172.21.15.131:0/93078954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd37c107070 0x7fd37c19d540 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.183+0000 7fd37b200700 1 -- 172.21.15.131:0/93078954 >> 172.21.15.131:0/93078954 conn(0x7fd37c102690 msgr2=0x7fd37c07a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.183+0000 7fd37b200700 1 -- 172.21.15.131:0/93078954 shutdown_connections 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-05-10T05:26:22.183+0000 7fd37b200700 1 -- 172.21.15.131:0/93078954 wait complete. 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout:Or, if you are only running a single cluster on this host: 2024-05-10T05:26:22.554 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout: ceph telemetry on 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout:For more information see: 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:22.555 INFO:teuthology.orchestra.run.smithi131.stdout:Bootstrap complete. 2024-05-10T05:26:22.605 INFO:tasks.cephadm:Fetching config... 2024-05-10T05:26:22.605 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:26:22.605 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-10T05:26:22.688 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-10T05:26:22.688 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:26:22.688 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-10T05:26:22.772 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-10T05:26:22.772 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:26:22.772 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-10T05:26:22.823 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:22 smithi131 ceph-mon[37264]: mgrmap e13: x(active, since 6s) 2024-05-10T05:26:22.823 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:22 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/93078954' entity='client.admin' 2024-05-10T05:26:22.887 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-10T05:26:22.887 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:26:22.887 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-10T05:26:22.970 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-10T05:26:22.971 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB/MuxBgoRT1qdg22IKrFYT6x3n5JtBRY8wljoMMQ/2yiP059gZUmhirjz7427QjaBM0+Afi/k8DE6Cs+Sgvdyl52G21FuC1rAVVASIv6N+Wn5SCCc1LRHHYHrYiqM5MTKOkyWqbOAP2u0gWopqIvD58U80XARmJTXwxRZj39hWowWO8OUemwrICzX+xjz7OTuUKFaL6hyzlNiSoGPawL97EqEGc6eYypc3F5WEPMscpiMNLlFxHQErHX0J6ap+31yyEQiQDY3o4KQj4DnLp3OXQaROYSDcXXTEt74gIUyvMOVCyoRxIDV8LqLTzw7v1dFiUGIGSy01iF/Jw0vPKBSuKgc5jm3MZinJ0I2IhI7z+OXGpahOlnZhM3Ya7RabRK1hT0LJ+Av51RboKe3tLakZcsb4OGzaK+4PirZIkTNvZjhyVghaAZvqljj7+lW1cSqkFoDS4MPR6+Jw7ukPRyozZQmXh0D9HTemVVvMu46xsNobzqtv3bMAazLDf+Mxc= ceph-6e8de474-0e8d-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T05:26:23.101 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB/MuxBgoRT1qdg22IKrFYT6x3n5JtBRY8wljoMMQ/2yiP059gZUmhirjz7427QjaBM0+Afi/k8DE6Cs+Sgvdyl52G21FuC1rAVVASIv6N+Wn5SCCc1LRHHYHrYiqM5MTKOkyWqbOAP2u0gWopqIvD58U80XARmJTXwxRZj39hWowWO8OUemwrICzX+xjz7OTuUKFaL6hyzlNiSoGPawL97EqEGc6eYypc3F5WEPMscpiMNLlFxHQErHX0J6ap+31yyEQiQDY3o4KQj4DnLp3OXQaROYSDcXXTEt74gIUyvMOVCyoRxIDV8LqLTzw7v1dFiUGIGSy01iF/Jw0vPKBSuKgc5jm3MZinJ0I2IhI7z+OXGpahOlnZhM3Ya7RabRK1hT0LJ+Av51RboKe3tLakZcsb4OGzaK+4PirZIkTNvZjhyVghaAZvqljj7+lW1cSqkFoDS4MPR6+Jw7ukPRyozZQmXh0D9HTemVVvMu46xsNobzqtv3bMAazLDf+Mxc= ceph-6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:26:23.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB/MuxBgoRT1qdg22IKrFYT6x3n5JtBRY8wljoMMQ/2yiP059gZUmhirjz7427QjaBM0+Afi/k8DE6Cs+Sgvdyl52G21FuC1rAVVASIv6N+Wn5SCCc1LRHHYHrYiqM5MTKOkyWqbOAP2u0gWopqIvD58U80XARmJTXwxRZj39hWowWO8OUemwrICzX+xjz7OTuUKFaL6hyzlNiSoGPawL97EqEGc6eYypc3F5WEPMscpiMNLlFxHQErHX0J6ap+31yyEQiQDY3o4KQj4DnLp3OXQaROYSDcXXTEt74gIUyvMOVCyoRxIDV8LqLTzw7v1dFiUGIGSy01iF/Jw0vPKBSuKgc5jm3MZinJ0I2IhI7z+OXGpahOlnZhM3Ya7RabRK1hT0LJ+Av51RboKe3tLakZcsb4OGzaK+4PirZIkTNvZjhyVghaAZvqljj7+lW1cSqkFoDS4MPR6+Jw7ukPRyozZQmXh0D9HTemVVvMu46xsNobzqtv3bMAazLDf+Mxc= ceph-6e8de474-0e8d-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T05:26:23.175 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB/MuxBgoRT1qdg22IKrFYT6x3n5JtBRY8wljoMMQ/2yiP059gZUmhirjz7427QjaBM0+Afi/k8DE6Cs+Sgvdyl52G21FuC1rAVVASIv6N+Wn5SCCc1LRHHYHrYiqM5MTKOkyWqbOAP2u0gWopqIvD58U80XARmJTXwxRZj39hWowWO8OUemwrICzX+xjz7OTuUKFaL6hyzlNiSoGPawL97EqEGc6eYypc3F5WEPMscpiMNLlFxHQErHX0J6ap+31yyEQiQDY3o4KQj4DnLp3OXQaROYSDcXXTEt74gIUyvMOVCyoRxIDV8LqLTzw7v1dFiUGIGSy01iF/Jw0vPKBSuKgc5jm3MZinJ0I2IhI7z+OXGpahOlnZhM3Ya7RabRK1hT0LJ+Av51RboKe3tLakZcsb4OGzaK+4PirZIkTNvZjhyVghaAZvqljj7+lW1cSqkFoDS4MPR6+Jw7ukPRyozZQmXh0D9HTemVVvMu46xsNobzqtv3bMAazLDf+Mxc= ceph-6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:26:23.188 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB/MuxBgoRT1qdg22IKrFYT6x3n5JtBRY8wljoMMQ/2yiP059gZUmhirjz7427QjaBM0+Afi/k8DE6Cs+Sgvdyl52G21FuC1rAVVASIv6N+Wn5SCCc1LRHHYHrYiqM5MTKOkyWqbOAP2u0gWopqIvD58U80XARmJTXwxRZj39hWowWO8OUemwrICzX+xjz7OTuUKFaL6hyzlNiSoGPawL97EqEGc6eYypc3F5WEPMscpiMNLlFxHQErHX0J6ap+31yyEQiQDY3o4KQj4DnLp3OXQaROYSDcXXTEt74gIUyvMOVCyoRxIDV8LqLTzw7v1dFiUGIGSy01iF/Jw0vPKBSuKgc5jm3MZinJ0I2IhI7z+OXGpahOlnZhM3Ya7RabRK1hT0LJ+Av51RboKe3tLakZcsb4OGzaK+4PirZIkTNvZjhyVghaAZvqljj7+lW1cSqkFoDS4MPR6+Jw7ukPRyozZQmXh0D9HTemVVvMu46xsNobzqtv3bMAazLDf+Mxc= ceph-6e8de474-0e8d-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T05:26:23.234 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB/MuxBgoRT1qdg22IKrFYT6x3n5JtBRY8wljoMMQ/2yiP059gZUmhirjz7427QjaBM0+Afi/k8DE6Cs+Sgvdyl52G21FuC1rAVVASIv6N+Wn5SCCc1LRHHYHrYiqM5MTKOkyWqbOAP2u0gWopqIvD58U80XARmJTXwxRZj39hWowWO8OUemwrICzX+xjz7OTuUKFaL6hyzlNiSoGPawL97EqEGc6eYypc3F5WEPMscpiMNLlFxHQErHX0J6ap+31yyEQiQDY3o4KQj4DnLp3OXQaROYSDcXXTEt74gIUyvMOVCyoRxIDV8LqLTzw7v1dFiUGIGSy01iF/Jw0vPKBSuKgc5jm3MZinJ0I2IhI7z+OXGpahOlnZhM3Ya7RabRK1hT0LJ+Av51RboKe3tLakZcsb4OGzaK+4PirZIkTNvZjhyVghaAZvqljj7+lW1cSqkFoDS4MPR6+Jw7ukPRyozZQmXh0D9HTemVVvMu46xsNobzqtv3bMAazLDf+Mxc= ceph-6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:26:23.247 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-10T05:26:23.605 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:26:25.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.224+0000 7f2aeec00700 1 -- 172.21.15.131:0/3214177147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80751d0 msgr2=0x7f2ae8073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:25.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.224+0000 7f2aeec00700 1 --2- 172.21.15.131:0/3214177147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80751d0 0x7f2ae8073600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009910 tx=0x7f2adc009c20 comp rx=0 tx=0).stop 2024-05-10T05:26:25.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.224+0000 7f2aeec00700 1 -- 172.21.15.131:0/3214177147 shutdown_connections 2024-05-10T05:26:25.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.224+0000 7f2aeec00700 1 --2- 172.21.15.131:0/3214177147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80751d0 0x7f2ae8073600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:25.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.224+0000 7f2aeec00700 1 -- 172.21.15.131:0/3214177147 >> 172.21.15.131:0/3214177147 conn(0x7f2ae80fca00 msgr2=0x7f2ae80fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:25.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.224+0000 7f2aeec00700 1 -- 172.21.15.131:0/3214177147 shutdown_connections 2024-05-10T05:26:25.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.224+0000 7f2aeec00700 1 -- 172.21.15.131:0/3214177147 wait complete. 2024-05-10T05:26:25.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.225+0000 7f2aeec00700 1 Processor -- start 2024-05-10T05:26:25.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.225+0000 7f2aeec00700 1 -- start start 2024-05-10T05:26:25.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.226+0000 7f2aeec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80751d0 0x7f2ae819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:25.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.226+0000 7f2aeec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819d7e0 con 0x7f2ae80751d0 2024-05-10T05:26:25.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.226+0000 7f2aed800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80751d0 0x7f2ae819d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:25.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.226+0000 7f2aed800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80751d0 0x7f2ae819d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48966/0 (socket says 172.21.15.131:48966) 2024-05-10T05:26:25.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.226+0000 7f2aed800700 1 -- 172.21.15.131:0/203974994 learned_addr learned my addr 172.21.15.131:0/203974994 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:25.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.226+0000 7f2aed800700 1 -- 172.21.15.131:0/203974994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc0095f0 con 0x7f2ae80751d0 2024-05-10T05:26:25.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.227+0000 7f2aed800700 1 --2- 172.21.15.131:0/203974994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80751d0 0x7f2ae819d2a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006010 tx=0x7f2adc005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:25.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.227+0000 7f2ae6000700 1 -- 172.21.15.131:0/203974994 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2adc01d070 con 0x7f2ae80751d0 2024-05-10T05:26:25.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.227+0000 7f2ae6000700 1 -- 172.21.15.131:0/203974994 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2adc00bc10 con 0x7f2ae80751d0 2024-05-10T05:26:25.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.227+0000 7f2aeec00700 1 -- 172.21.15.131:0/203974994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae819d9e0 con 0x7f2ae80751d0 2024-05-10T05:26:25.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.227+0000 7f2ae6000700 1 -- 172.21.15.131:0/203974994 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2adc003930 con 0x7f2ae80751d0 2024-05-10T05:26:25.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.227+0000 7f2aeec00700 1 -- 172.21.15.131:0/203974994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae819de80 con 0x7f2ae80751d0 2024-05-10T05:26:25.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.228+0000 7f2ae6000700 1 -- 172.21.15.131:0/203974994 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f2adc003bf0 con 0x7f2ae80751d0 2024-05-10T05:26:25.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.228+0000 7f2aeec00700 1 -- 172.21.15.131:0/203974994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae8196750 con 0x7f2ae80751d0 2024-05-10T05:26:25.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.229+0000 7f2ae6000700 1 --2- 172.21.15.131:0/203974994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2ad00387a0 0x7f2ad003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:25.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.229+0000 7f2ae6000700 1 -- 172.21.15.131:0/203974994 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2adc04c7a0 con 0x7f2ae80751d0 2024-05-10T05:26:25.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.229+0000 7f2aece00700 1 --2- 172.21.15.131:0/203974994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2ad00387a0 0x7f2ad003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:25.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.230+0000 7f2aece00700 1 --2- 172.21.15.131:0/203974994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2ad00387a0 0x7f2ad003acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4006fd0 tx=0x7f2ad4006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:25.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.235+0000 7f2ae6000700 1 -- 172.21.15.131:0/203974994 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2adc029930 con 0x7f2ae80751d0 2024-05-10T05:26:25.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.444+0000 7f2aeec00700 1 -- 172.21.15.131:0/203974994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f2ae8062840 con 0x7f2ae80751d0 2024-05-10T05:26:25.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.448+0000 7f2ae6000700 1 -- 172.21.15.131:0/203974994 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2adc026070 con 0x7f2ae80751d0 2024-05-10T05:26:25.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.451+0000 7f2aeec00700 1 -- 172.21.15.131:0/203974994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2ad00387a0 msgr2=0x7f2ad003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:25.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.451+0000 7f2aeec00700 1 --2- 172.21.15.131:0/203974994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2ad00387a0 0x7f2ad003acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4006fd0 tx=0x7f2ad4006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:25.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.451+0000 7f2aeec00700 1 -- 172.21.15.131:0/203974994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80751d0 msgr2=0x7f2ae819d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:25.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.451+0000 7f2aeec00700 1 --2- 172.21.15.131:0/203974994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80751d0 0x7f2ae819d2a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006010 tx=0x7f2adc005af0 comp rx=0 tx=0).stop 2024-05-10T05:26:25.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.451+0000 7f2aeec00700 1 -- 172.21.15.131:0/203974994 shutdown_connections 2024-05-10T05:26:25.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.451+0000 7f2aeec00700 1 --2- 172.21.15.131:0/203974994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2ad00387a0 0x7f2ad003acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:25.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.451+0000 7f2aeec00700 1 --2- 172.21.15.131:0/203974994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae80751d0 0x7f2ae819d2a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:25.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.451+0000 7f2aeec00700 1 -- 172.21.15.131:0/203974994 >> 172.21.15.131:0/203974994 conn(0x7f2ae80fca00 msgr2=0x7f2ae80fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:25.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.451+0000 7f2aeec00700 1 -- 172.21.15.131:0/203974994 shutdown_connections 2024-05-10T05:26:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:25.451+0000 7f2aeec00700 1 -- 172.21.15.131:0/203974994 wait complete. 2024-05-10T05:26:25.881 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-10T05:26:25.881 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-10T05:26:26.253 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:26:26.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:26 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/203974994' entity='client.admin' 2024-05-10T05:26:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.753+0000 7f6e60e00700 1 -- 172.21.15.131:0/3584337006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c101380 msgr2=0x7f6e5c101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.753+0000 7f6e60e00700 1 --2- 172.21.15.131:0/3584337006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c101380 0x7f6e5c101790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009910 tx=0x7f6e4c009c20 comp rx=0 tx=0).stop 2024-05-10T05:26:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.753+0000 7f6e60e00700 1 -- 172.21.15.131:0/3584337006 shutdown_connections 2024-05-10T05:26:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.753+0000 7f6e60e00700 1 --2- 172.21.15.131:0/3584337006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c101380 0x7f6e5c101790 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.753+0000 7f6e60e00700 1 -- 172.21.15.131:0/3584337006 >> 172.21.15.131:0/3584337006 conn(0x7f6e5c0fca00 msgr2=0x7f6e5c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.753+0000 7f6e60e00700 1 -- 172.21.15.131:0/3584337006 shutdown_connections 2024-05-10T05:26:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.753+0000 7f6e60e00700 1 -- 172.21.15.131:0/3584337006 wait complete. 2024-05-10T05:26:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.754+0000 7f6e60e00700 1 Processor -- start 2024-05-10T05:26:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.754+0000 7f6e60e00700 1 -- start start 2024-05-10T05:26:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.754+0000 7f6e60e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c101380 0x7f6e5c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.754+0000 7f6e60e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c199450 con 0x7f6e5c101380 2024-05-10T05:26:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.755+0000 7f6e5b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c101380 0x7f6e5c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.755+0000 7f6e5b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c101380 0x7f6e5c198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48986/0 (socket says 172.21.15.131:48986) 2024-05-10T05:26:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.755+0000 7f6e5b400700 1 -- 172.21.15.131:0/2543749351 learned_addr learned my addr 172.21.15.131:0/2543749351 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.755+0000 7f6e5b400700 1 -- 172.21.15.131:0/2543749351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c0095f0 con 0x7f6e5c101380 2024-05-10T05:26:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.755+0000 7f6e5b400700 1 --2- 172.21.15.131:0/2543749351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c101380 0x7f6e5c198f10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006010 tx=0x7f6e4c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.755+0000 7f6e53e00700 1 -- 172.21.15.131:0/2543749351 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e4c01d070 con 0x7f6e5c101380 2024-05-10T05:26:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.755+0000 7f6e53e00700 1 -- 172.21.15.131:0/2543749351 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4c00bc10 con 0x7f6e5c101380 2024-05-10T05:26:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.755+0000 7f6e60e00700 1 -- 172.21.15.131:0/2543749351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5c199650 con 0x7f6e5c101380 2024-05-10T05:26:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.755+0000 7f6e53e00700 1 -- 172.21.15.131:0/2543749351 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e4c003970 con 0x7f6e5c101380 2024-05-10T05:26:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.756+0000 7f6e60e00700 1 -- 172.21.15.131:0/2543749351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c199af0 con 0x7f6e5c101380 2024-05-10T05:26:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.757+0000 7f6e53e00700 1 -- 172.21.15.131:0/2543749351 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f6e4c00b6f0 con 0x7f6e5c101380 2024-05-10T05:26:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.757+0000 7f6e60e00700 1 -- 172.21.15.131:0/2543749351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5c192530 con 0x7f6e5c101380 2024-05-10T05:26:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.757+0000 7f6e53e00700 1 --2- 172.21.15.131:0/2543749351 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6e400387f0 0x7f6e4003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.757+0000 7f6e53e00700 1 -- 172.21.15.131:0/2543749351 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e4c04c5a0 con 0x7f6e5c101380 2024-05-10T05:26:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.757+0000 7f6e5aa00700 1 --2- 172.21.15.131:0/2543749351 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6e400387f0 0x7f6e4003ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:27.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.758+0000 7f6e5aa00700 1 --2- 172.21.15.131:0/2543749351 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6e400387f0 0x7f6e4003ad00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6e48006fd0 tx=0x7f6e48006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.762+0000 7f6e53e00700 1 -- 172.21.15.131:0/2543749351 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6e4c003ad0 con 0x7f6e5c101380 2024-05-10T05:26:27.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.972+0000 7f6e60e00700 1 -- 172.21.15.131:0/2543749351 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e5c061650 con 0x7f6e400387f0 2024-05-10T05:26:27.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.976+0000 7f6e53e00700 1 -- 172.21.15.131:0/2543749351 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6e5c061650 con 0x7f6e400387f0 2024-05-10T05:26:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.979+0000 7f6e60e00700 1 -- 172.21.15.131:0/2543749351 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6e400387f0 msgr2=0x7f6e4003ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.979+0000 7f6e60e00700 1 --2- 172.21.15.131:0/2543749351 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6e400387f0 0x7f6e4003ad00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6e48006fd0 tx=0x7f6e48006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.979+0000 7f6e60e00700 1 -- 172.21.15.131:0/2543749351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c101380 msgr2=0x7f6e5c198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.979+0000 7f6e60e00700 1 --2- 172.21.15.131:0/2543749351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c101380 0x7f6e5c198f10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006010 tx=0x7f6e4c005800 comp rx=0 tx=0).stop 2024-05-10T05:26:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.979+0000 7f6e60e00700 1 -- 172.21.15.131:0/2543749351 shutdown_connections 2024-05-10T05:26:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.979+0000 7f6e60e00700 1 --2- 172.21.15.131:0/2543749351 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6e400387f0 0x7f6e4003ad00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.979+0000 7f6e60e00700 1 --2- 172.21.15.131:0/2543749351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c101380 0x7f6e5c198f10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.979+0000 7f6e60e00700 1 -- 172.21.15.131:0/2543749351 >> 172.21.15.131:0/2543749351 conn(0x7f6e5c0fca00 msgr2=0x7f6e5c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.979+0000 7f6e60e00700 1 -- 172.21.15.131:0/2543749351 shutdown_connections 2024-05-10T05:26:27.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:27.979+0000 7f6e60e00700 1 -- 172.21.15.131:0/2543749351 wait complete. 2024-05-10T05:26:28.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:27 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:28.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:27 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:28.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:27 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:26:28.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:27 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:28.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:27 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:28.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:27 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:28.434 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-05-10T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-05-10T05:26:28.455 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:26:28.456 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T05:26:28.515 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-05-10T05:26:28.515 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch host add smithi142 2024-05-10T05:26:28.881 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:26:29.238 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:28 smithi131 ceph-mon[37264]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:29.238 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:29.238 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:29.239 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:29.239 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:26:30.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:29 smithi131 ceph-mon[37264]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-10T05:26:30.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:29 smithi131 ceph-mon[37264]: Updating smithi131:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T05:26:30.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.278+0000 7fdc61600700 1 -- 172.21.15.131:0/4051598110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c107590 msgr2=0x7fdc5c1079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:30.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.278+0000 7fdc61600700 1 --2- 172.21.15.131:0/4051598110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c107590 0x7fdc5c1079a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009910 tx=0x7fdc4c009c20 comp rx=0 tx=0).stop 2024-05-10T05:26:30.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.278+0000 7fdc61600700 1 -- 172.21.15.131:0/4051598110 shutdown_connections 2024-05-10T05:26:30.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.278+0000 7fdc61600700 1 --2- 172.21.15.131:0/4051598110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c107590 0x7fdc5c1079a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:30.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.278+0000 7fdc61600700 1 -- 172.21.15.131:0/4051598110 >> 172.21.15.131:0/4051598110 conn(0x7fdc5c0fcfd0 msgr2=0x7fdc5c0ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:30.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.278+0000 7fdc61600700 1 -- 172.21.15.131:0/4051598110 shutdown_connections 2024-05-10T05:26:30.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.278+0000 7fdc61600700 1 -- 172.21.15.131:0/4051598110 wait complete. 2024-05-10T05:26:30.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.279+0000 7fdc61600700 1 Processor -- start 2024-05-10T05:26:30.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.280+0000 7fdc61600700 1 -- start start 2024-05-10T05:26:30.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.280+0000 7fdc61600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c107590 0x7fdc5c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:30.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.280+0000 7fdc61600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c199700 con 0x7fdc5c107590 2024-05-10T05:26:30.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.280+0000 7fdc5aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c107590 0x7fdc5c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:30.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.280+0000 7fdc5aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c107590 0x7fdc5c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49000/0 (socket says 172.21.15.131:49000) 2024-05-10T05:26:30.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.280+0000 7fdc5aa00700 1 -- 172.21.15.131:0/2671628170 learned_addr learned my addr 172.21.15.131:0/2671628170 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:30.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.281+0000 7fdc5aa00700 1 -- 172.21.15.131:0/2671628170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c0095f0 con 0x7fdc5c107590 2024-05-10T05:26:30.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.281+0000 7fdc5aa00700 1 --2- 172.21.15.131:0/2671628170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c107590 0x7fdc5c1991c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0092e0 tx=0x7fdc4c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:30.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.281+0000 7fdc53400700 1 -- 172.21.15.131:0/2671628170 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc4c01d070 con 0x7fdc5c107590 2024-05-10T05:26:30.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.281+0000 7fdc53400700 1 -- 172.21.15.131:0/2671628170 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc4c00b750 con 0x7fdc5c107590 2024-05-10T05:26:30.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.281+0000 7fdc53400700 1 -- 172.21.15.131:0/2671628170 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc4c0038a0 con 0x7fdc5c107590 2024-05-10T05:26:30.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.281+0000 7fdc61600700 1 -- 172.21.15.131:0/2671628170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5c199960 con 0x7fdc5c107590 2024-05-10T05:26:30.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.281+0000 7fdc61600700 1 -- 172.21.15.131:0/2671628170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5c199e60 con 0x7fdc5c107590 2024-05-10T05:26:30.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.282+0000 7fdc53400700 1 -- 172.21.15.131:0/2671628170 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fdc4c004260 con 0x7fdc5c107590 2024-05-10T05:26:30.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.283+0000 7fdc61600700 1 -- 172.21.15.131:0/2671628170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5c102bd0 con 0x7fdc5c107590 2024-05-10T05:26:30.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.283+0000 7fdc53400700 1 --2- 172.21.15.131:0/2671628170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdc400387a0 0x7fdc4003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:30.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.283+0000 7fdc53400700 1 -- 172.21.15.131:0/2671628170 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdc4c04c080 con 0x7fdc5c107590 2024-05-10T05:26:30.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.283+0000 7fdc5a000700 1 --2- 172.21.15.131:0/2671628170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdc400387a0 0x7fdc4003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:30.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.284+0000 7fdc5a000700 1 --2- 172.21.15.131:0/2671628170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdc400387a0 0x7fdc4003acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdc48006fd0 tx=0x7fdc48006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:30.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.289+0000 7fdc53400700 1 -- 172.21.15.131:0/2671628170 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdc4c00b8c0 con 0x7fdc5c107590 2024-05-10T05:26:30.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:30.495+0000 7fdc61600700 1 -- 172.21.15.131:0/2671628170 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc5c1080b0 con 0x7fdc400387a0 2024-05-10T05:26:31.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:30 smithi131 ceph-mon[37264]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-05-10T05:26:31.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:30 smithi131 ceph-mon[37264]: Updating smithi131:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T05:26:31.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:30 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:31.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:30 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:31.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:30 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:32.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:31 smithi131 ceph-mon[37264]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:32.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:31 smithi131 ceph-mon[37264]: Deploying cephadm binary to smithi142 2024-05-10T05:26:32.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.627+0000 7fdc53400700 1 -- 172.21.15.131:0/2671628170 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fdc5c1080b0 con 0x7fdc400387a0 2024-05-10T05:26:32.628 INFO:teuthology.orchestra.run.smithi131.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-05-10T05:26:32.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.630+0000 7fdc61600700 1 -- 172.21.15.131:0/2671628170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdc400387a0 msgr2=0x7fdc4003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:32.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.630+0000 7fdc61600700 1 --2- 172.21.15.131:0/2671628170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdc400387a0 0x7fdc4003acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdc48006fd0 tx=0x7fdc48006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:32.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.630+0000 7fdc61600700 1 -- 172.21.15.131:0/2671628170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c107590 msgr2=0x7fdc5c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:32.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.630+0000 7fdc61600700 1 --2- 172.21.15.131:0/2671628170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c107590 0x7fdc5c1991c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0092e0 tx=0x7fdc4c004ff0 comp rx=0 tx=0).stop 2024-05-10T05:26:32.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.630+0000 7fdc61600700 1 -- 172.21.15.131:0/2671628170 shutdown_connections 2024-05-10T05:26:32.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.630+0000 7fdc61600700 1 --2- 172.21.15.131:0/2671628170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdc400387a0 0x7fdc4003acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.630+0000 7fdc61600700 1 --2- 172.21.15.131:0/2671628170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc5c107590 0x7fdc5c1991c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.630+0000 7fdc61600700 1 -- 172.21.15.131:0/2671628170 >> 172.21.15.131:0/2671628170 conn(0x7fdc5c0fcfd0 msgr2=0x7fdc5c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.630+0000 7fdc61600700 1 -- 172.21.15.131:0/2671628170 shutdown_connections 2024-05-10T05:26:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:32.630+0000 7fdc61600700 1 -- 172.21.15.131:0/2671628170 wait complete. 2024-05-10T05:26:33.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T05:26:33.373 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:26:33.847 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:33.847 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:33 smithi131 ceph-mon[37264]: Added host smithi142 2024-05-10T05:26:33.847 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:33.847 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:33.847 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.550+0000 7fac8b200700 1 -- 172.21.15.131:0/3846561327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac84101560 msgr2=0x7fac84101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.550+0000 7fac8b200700 1 --2- 172.21.15.131:0/3846561327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac84101560 0x7fac84101970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fac78009910 tx=0x7fac78009c20 comp rx=0 tx=0).stop 2024-05-10T05:26:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.550+0000 7fac8b200700 1 -- 172.21.15.131:0/3846561327 shutdown_connections 2024-05-10T05:26:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.550+0000 7fac8b200700 1 --2- 172.21.15.131:0/3846561327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac84101560 0x7fac84101970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.550+0000 7fac8b200700 1 -- 172.21.15.131:0/3846561327 >> 172.21.15.131:0/3846561327 conn(0x7fac840fce10 msgr2=0x7fac840ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.550+0000 7fac8b200700 1 -- 172.21.15.131:0/3846561327 shutdown_connections 2024-05-10T05:26:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.550+0000 7fac8b200700 1 -- 172.21.15.131:0/3846561327 wait complete. 2024-05-10T05:26:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.551+0000 7fac8b200700 1 Processor -- start 2024-05-10T05:26:34.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.552+0000 7fac8b200700 1 -- start start 2024-05-10T05:26:34.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.552+0000 7fac8b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac84101560 0x7fac84196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:34.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.552+0000 7fac8b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac84197460 con 0x7fac84101560 2024-05-10T05:26:34.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.552+0000 7fac89e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac84101560 0x7fac84196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:34.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.552+0000 7fac89e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac84101560 0x7fac84196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37514/0 (socket says 172.21.15.131:37514) 2024-05-10T05:26:34.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.552+0000 7fac89e00700 1 -- 172.21.15.131:0/2572689460 learned_addr learned my addr 172.21.15.131:0/2572689460 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:34.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.553+0000 7fac89e00700 1 -- 172.21.15.131:0/2572689460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac780095f0 con 0x7fac84101560 2024-05-10T05:26:34.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.553+0000 7fac89e00700 1 --2- 172.21.15.131:0/2572689460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac84101560 0x7fac84196f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fac78006010 tx=0x7fac78004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:34.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.553+0000 7fac7ea00700 1 -- 172.21.15.131:0/2572689460 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac7801d070 con 0x7fac84101560 2024-05-10T05:26:34.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.553+0000 7fac7ea00700 1 -- 172.21.15.131:0/2572689460 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac7800bbb0 con 0x7fac84101560 2024-05-10T05:26:34.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.553+0000 7fac8b200700 1 -- 172.21.15.131:0/2572689460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac841976c0 con 0x7fac84101560 2024-05-10T05:26:34.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.553+0000 7fac7ea00700 1 -- 172.21.15.131:0/2572689460 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac78003810 con 0x7fac84101560 2024-05-10T05:26:34.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.553+0000 7fac8b200700 1 -- 172.21.15.131:0/2572689460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac84197b40 con 0x7fac84101560 2024-05-10T05:26:34.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.554+0000 7fac8b200700 1 -- 172.21.15.131:0/2572689460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac84102a10 con 0x7fac84101560 2024-05-10T05:26:34.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.555+0000 7fac7ea00700 1 -- 172.21.15.131:0/2572689460 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fac7800bd20 con 0x7fac84101560 2024-05-10T05:26:34.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.555+0000 7fac7ea00700 1 --2- 172.21.15.131:0/2572689460 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac6c0387f0 0x7fac6c03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:34.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.555+0000 7fac7ea00700 1 -- 172.21.15.131:0/2572689460 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac7804c8b0 con 0x7fac84101560 2024-05-10T05:26:34.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.556+0000 7fac89400700 1 --2- 172.21.15.131:0/2572689460 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac6c0387f0 0x7fac6c03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:34.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.556+0000 7fac89400700 1 --2- 172.21.15.131:0/2572689460 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac6c0387f0 0x7fac6c03ad00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac74006fd0 tx=0x7fac74006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:34.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.560+0000 7fac7ea00700 1 -- 172.21.15.131:0/2572689460 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fac7800e3e0 con 0x7fac84101560 2024-05-10T05:26:34.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.770+0000 7fac8b200700 1 -- 172.21.15.131:0/2572689460 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac8410b6e0 con 0x7fac6c0387f0 2024-05-10T05:26:34.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.772+0000 7fac7ea00700 1 -- 172.21.15.131:0/2572689460 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7fac8410b6e0 con 0x7fac6c0387f0 2024-05-10T05:26:34.772 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:34.773 INFO:teuthology.orchestra.run.smithi131.stdout:[{"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-05-10T05:26:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.774+0000 7fac8b200700 1 -- 172.21.15.131:0/2572689460 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac6c0387f0 msgr2=0x7fac6c03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.774+0000 7fac8b200700 1 --2- 172.21.15.131:0/2572689460 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac6c0387f0 0x7fac6c03ad00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac74006fd0 tx=0x7fac74006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.774+0000 7fac8b200700 1 -- 172.21.15.131:0/2572689460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac84101560 msgr2=0x7fac84196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.774+0000 7fac8b200700 1 --2- 172.21.15.131:0/2572689460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac84101560 0x7fac84196f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fac78006010 tx=0x7fac78004ff0 comp rx=0 tx=0).stop 2024-05-10T05:26:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.774+0000 7fac8b200700 1 -- 172.21.15.131:0/2572689460 shutdown_connections 2024-05-10T05:26:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.774+0000 7fac8b200700 1 --2- 172.21.15.131:0/2572689460 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac6c0387f0 0x7fac6c03ad00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.774+0000 7fac8b200700 1 --2- 172.21.15.131:0/2572689460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac84101560 0x7fac84196f20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.774+0000 7fac8b200700 1 -- 172.21.15.131:0/2572689460 >> 172.21.15.131:0/2572689460 conn(0x7fac840fce10 msgr2=0x7fac84107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.774+0000 7fac8b200700 1 -- 172.21.15.131:0/2572689460 shutdown_connections 2024-05-10T05:26:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:34.774+0000 7fac8b200700 1 -- 172.21.15.131:0/2572689460 wait complete. 2024-05-10T05:26:35.122 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:35.122 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:35 smithi131 ceph-mon[37264]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:35.122 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:35 smithi131 ceph-mon[37264]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:26:35.150 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-05-10T05:26:35.150 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:26:35.150 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-05-10T05:26:35.171 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:26:35.171 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T05:26:35.231 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-05-10T05:26:35.231 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch host add smithi157 2024-05-10T05:26:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:26:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.868+0000 7ff676800700 1 -- 172.21.15.131:0/4250629945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff670107580 msgr2=0x7ff670107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.868+0000 7ff676800700 1 --2- 172.21.15.131:0/4250629945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff670107580 0x7ff670107990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff660009910 tx=0x7ff660009c20 comp rx=0 tx=0).stop 2024-05-10T05:26:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.869+0000 7ff676800700 1 -- 172.21.15.131:0/4250629945 shutdown_connections 2024-05-10T05:26:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.869+0000 7ff676800700 1 --2- 172.21.15.131:0/4250629945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff670107580 0x7ff670107990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.869+0000 7ff676800700 1 -- 172.21.15.131:0/4250629945 >> 172.21.15.131:0/4250629945 conn(0x7ff6700fcfc0 msgr2=0x7ff6700ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.869+0000 7ff676800700 1 -- 172.21.15.131:0/4250629945 shutdown_connections 2024-05-10T05:26:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.869+0000 7ff676800700 1 -- 172.21.15.131:0/4250629945 wait complete. 2024-05-10T05:26:36.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.870+0000 7ff676800700 1 Processor -- start 2024-05-10T05:26:36.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.870+0000 7ff676800700 1 -- start start 2024-05-10T05:26:36.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.871+0000 7ff676800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff670107580 0x7ff670073510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:36.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.871+0000 7ff676800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff670075380 con 0x7ff670107580 2024-05-10T05:26:36.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.871+0000 7ff66f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff670107580 0x7ff670073510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:36.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.871+0000 7ff66f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff670107580 0x7ff670073510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37524/0 (socket says 172.21.15.131:37524) 2024-05-10T05:26:36.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.871+0000 7ff66f400700 1 -- 172.21.15.131:0/2368706592 learned_addr learned my addr 172.21.15.131:0/2368706592 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:36.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.871+0000 7ff66f400700 1 -- 172.21.15.131:0/2368706592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600095f0 con 0x7ff670107580 2024-05-10T05:26:36.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.872+0000 7ff66f400700 1 --2- 172.21.15.131:0/2368706592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff670107580 0x7ff670073510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff6600092e0 tx=0x7ff660004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:36.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.872+0000 7ff66cc00700 1 -- 172.21.15.131:0/2368706592 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff66001d070 con 0x7ff670107580 2024-05-10T05:26:36.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.872+0000 7ff676800700 1 -- 172.21.15.131:0/2368706592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff670073a50 con 0x7ff670107580 2024-05-10T05:26:36.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.872+0000 7ff66cc00700 1 -- 172.21.15.131:0/2368706592 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff66000b750 con 0x7ff670107580 2024-05-10T05:26:36.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.872+0000 7ff66cc00700 1 -- 172.21.15.131:0/2368706592 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6600038a0 con 0x7ff670107580 2024-05-10T05:26:36.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.872+0000 7ff676800700 1 -- 172.21.15.131:0/2368706592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff670073f50 con 0x7ff670107580 2024-05-10T05:26:36.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.873+0000 7ff676800700 1 -- 172.21.15.131:0/2368706592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff670100580 con 0x7ff670107580 2024-05-10T05:26:36.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.873+0000 7ff66cc00700 1 -- 172.21.15.131:0/2368706592 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7ff660004260 con 0x7ff670107580 2024-05-10T05:26:36.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.874+0000 7ff66cc00700 1 --2- 172.21.15.131:0/2368706592 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6540387f0 0x7ff65403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:36.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.874+0000 7ff66cc00700 1 -- 172.21.15.131:0/2368706592 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff66004c240 con 0x7ff670107580 2024-05-10T05:26:36.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.874+0000 7ff667e00700 1 --2- 172.21.15.131:0/2368706592 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6540387f0 0x7ff65403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:36.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.874+0000 7ff667e00700 1 --2- 172.21.15.131:0/2368706592 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6540387f0 0x7ff65403ad00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff658006fd0 tx=0x7ff658006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:36.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:36.879+0000 7ff66cc00700 1 -- 172.21.15.131:0/2368706592 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff66000b8c0 con 0x7ff670107580 2024-05-10T05:26:37.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:37.091+0000 7ff676800700 1 -- 172.21.15.131:0/2368706592 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6701080a0 con 0x7ff6540387f0 2024-05-10T05:26:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:26:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:26:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-10T05:26:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: Updating smithi142:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T05:26:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-05-10T05:26:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: Updating smithi142:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T05:26:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:38 smithi131 ceph-mon[37264]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:38 smithi131 ceph-mon[37264]: Deploying cephadm binary to smithi157 2024-05-10T05:26:39.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.270+0000 7ff66cc00700 1 -- 172.21.15.131:0/2368706592 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff6701080a0 con 0x7ff6540387f0 2024-05-10T05:26:39.271 INFO:teuthology.orchestra.run.smithi131.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-05-10T05:26:39.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.272+0000 7ff676800700 1 -- 172.21.15.131:0/2368706592 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6540387f0 msgr2=0x7ff65403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:39.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.272+0000 7ff676800700 1 --2- 172.21.15.131:0/2368706592 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6540387f0 0x7ff65403ad00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff658006fd0 tx=0x7ff658006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.272+0000 7ff676800700 1 -- 172.21.15.131:0/2368706592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff670107580 msgr2=0x7ff670073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.272+0000 7ff676800700 1 --2- 172.21.15.131:0/2368706592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff670107580 0x7ff670073510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff6600092e0 tx=0x7ff660004ff0 comp rx=0 tx=0).stop 2024-05-10T05:26:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.272+0000 7ff676800700 1 -- 172.21.15.131:0/2368706592 shutdown_connections 2024-05-10T05:26:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.272+0000 7ff676800700 1 --2- 172.21.15.131:0/2368706592 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6540387f0 0x7ff65403ad00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.272+0000 7ff676800700 1 --2- 172.21.15.131:0/2368706592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff670107580 0x7ff670073510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.272+0000 7ff676800700 1 -- 172.21.15.131:0/2368706592 >> 172.21.15.131:0/2368706592 conn(0x7ff6700fcfc0 msgr2=0x7ff6700fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.273+0000 7ff676800700 1 -- 172.21.15.131:0/2368706592 shutdown_connections 2024-05-10T05:26:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:39.273+0000 7ff676800700 1 -- 172.21.15.131:0/2368706592 wait complete. 2024-05-10T05:26:39.504 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:39 smithi131 ceph-mon[37264]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:39.692 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T05:26:40.042 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:26:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:40 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:40 smithi131 ceph-mon[37264]: Added host smithi157 2024-05-10T05:26:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:40 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:40 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:40 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.253+0000 7f04ca800700 1 -- 172.21.15.131:0/199073489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c4107590 msgr2=0x7f04c41079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.253+0000 7f04ca800700 1 --2- 172.21.15.131:0/199073489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c4107590 0x7f04c41079a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009910 tx=0x7f04b4009c20 comp rx=0 tx=0).stop 2024-05-10T05:26:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.254+0000 7f04ca800700 1 -- 172.21.15.131:0/199073489 shutdown_connections 2024-05-10T05:26:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.254+0000 7f04ca800700 1 --2- 172.21.15.131:0/199073489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c4107590 0x7f04c41079a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.254+0000 7f04ca800700 1 -- 172.21.15.131:0/199073489 >> 172.21.15.131:0/199073489 conn(0x7f04c40fcfd0 msgr2=0x7f04c40ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.254+0000 7f04ca800700 1 -- 172.21.15.131:0/199073489 shutdown_connections 2024-05-10T05:26:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.254+0000 7f04ca800700 1 -- 172.21.15.131:0/199073489 wait complete. 2024-05-10T05:26:41.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.255+0000 7f04ca800700 1 Processor -- start 2024-05-10T05:26:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.255+0000 7f04ca800700 1 -- start start 2024-05-10T05:26:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.255+0000 7f04ca800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c4107590 0x7f04c4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.255+0000 7f04ca800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c41996d0 con 0x7f04c4107590 2024-05-10T05:26:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.256+0000 7f04c3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c4107590 0x7f04c4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.256+0000 7f04c3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c4107590 0x7f04c4199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37542/0 (socket says 172.21.15.131:37542) 2024-05-10T05:26:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.256+0000 7f04c3400700 1 -- 172.21.15.131:0/3974042567 learned_addr learned my addr 172.21.15.131:0/3974042567 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.256+0000 7f04c3400700 1 -- 172.21.15.131:0/3974042567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b40095f0 con 0x7f04c4107590 2024-05-10T05:26:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.256+0000 7f04c3400700 1 --2- 172.21.15.131:0/3974042567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c4107590 0x7f04c4199190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f04b40092e0 tx=0x7f04b4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.256+0000 7f04bbe00700 1 -- 172.21.15.131:0/3974042567 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04b401d070 con 0x7f04c4107590 2024-05-10T05:26:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.256+0000 7f04bbe00700 1 -- 172.21.15.131:0/3974042567 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b400b750 con 0x7f04c4107590 2024-05-10T05:26:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.256+0000 7f04ca800700 1 -- 172.21.15.131:0/3974042567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c4199930 con 0x7f04c4107590 2024-05-10T05:26:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.256+0000 7f04bbe00700 1 -- 172.21.15.131:0/3974042567 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04b40038a0 con 0x7f04c4107590 2024-05-10T05:26:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.257+0000 7f04ca800700 1 -- 172.21.15.131:0/3974042567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c4199e30 con 0x7f04c4107590 2024-05-10T05:26:41.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.258+0000 7f04bbe00700 1 -- 172.21.15.131:0/3974042567 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f04b4003a00 con 0x7f04c4107590 2024-05-10T05:26:41.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.258+0000 7f04ca800700 1 -- 172.21.15.131:0/3974042567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c4102bd0 con 0x7f04c4107590 2024-05-10T05:26:41.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.258+0000 7f04bbe00700 1 --2- 172.21.15.131:0/3974042567 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04a80387f0 0x7f04a803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:41.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.258+0000 7f04bbe00700 1 -- 172.21.15.131:0/3974042567 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f04b404c100 con 0x7f04c4107590 2024-05-10T05:26:41.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.258+0000 7f04c2a00700 1 --2- 172.21.15.131:0/3974042567 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04a80387f0 0x7f04a803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:41.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.259+0000 7f04c2a00700 1 --2- 172.21.15.131:0/3974042567 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04a80387f0 0x7f04a803ad00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f04b0006fd0 tx=0x7f04b0006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.264+0000 7f04bbe00700 1 -- 172.21.15.131:0/3974042567 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f04b402a590 con 0x7f04c4107590 2024-05-10T05:26:41.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.471+0000 7f04ca800700 1 -- 172.21.15.131:0/3974042567 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04c41080b0 con 0x7f04a80387f0 2024-05-10T05:26:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.473+0000 7f04bbe00700 1 -- 172.21.15.131:0/3974042567 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f04c41080b0 con 0x7f04a80387f0 2024-05-10T05:26:41.473 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:26:41.474 INFO:teuthology.orchestra.run.smithi131.stdout:[{"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-05-10T05:26:41.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.475+0000 7f04ca800700 1 -- 172.21.15.131:0/3974042567 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04a80387f0 msgr2=0x7f04a803ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:41.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.475+0000 7f04ca800700 1 --2- 172.21.15.131:0/3974042567 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04a80387f0 0x7f04a803ad00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f04b0006fd0 tx=0x7f04b0006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:41.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.475+0000 7f04ca800700 1 -- 172.21.15.131:0/3974042567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c4107590 msgr2=0x7f04c4199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.475+0000 7f04ca800700 1 --2- 172.21.15.131:0/3974042567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c4107590 0x7f04c4199190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f04b40092e0 tx=0x7f04b4004ff0 comp rx=0 tx=0).stop 2024-05-10T05:26:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.475+0000 7f04ca800700 1 -- 172.21.15.131:0/3974042567 shutdown_connections 2024-05-10T05:26:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.475+0000 7f04ca800700 1 --2- 172.21.15.131:0/3974042567 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04a80387f0 0x7f04a803ad00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.475+0000 7f04ca800700 1 --2- 172.21.15.131:0/3974042567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c4107590 0x7f04c4199190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.475+0000 7f04ca800700 1 -- 172.21.15.131:0/3974042567 >> 172.21.15.131:0/3974042567 conn(0x7f04c40fcfd0 msgr2=0x7f04c40fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.475+0000 7f04ca800700 1 -- 172.21.15.131:0/3974042567 shutdown_connections 2024-05-10T05:26:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:41.475+0000 7f04ca800700 1 -- 172.21.15.131:0/3974042567 wait complete. 2024-05-10T05:26:41.863 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:41 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:41.863 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:41 smithi131 ceph-mon[37264]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:41.889 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-10T05:26:41.889 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-10T05:26:42.239 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:26:42.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:42 smithi131 ceph-mon[37264]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:26:43.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.481+0000 7f52caa00700 1 -- 172.21.15.131:0/3530247063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c4101560 msgr2=0x7f52c4101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:43.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.481+0000 7f52caa00700 1 --2- 172.21.15.131:0/3530247063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c4101560 0x7f52c4101970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009910 tx=0x7f52b8009c20 comp rx=0 tx=0).stop 2024-05-10T05:26:43.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.481+0000 7f52caa00700 1 -- 172.21.15.131:0/3530247063 shutdown_connections 2024-05-10T05:26:43.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.481+0000 7f52caa00700 1 --2- 172.21.15.131:0/3530247063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c4101560 0x7f52c4101970 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:43.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.481+0000 7f52caa00700 1 -- 172.21.15.131:0/3530247063 >> 172.21.15.131:0/3530247063 conn(0x7f52c40fce10 msgr2=0x7f52c40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:43.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.482+0000 7f52caa00700 1 -- 172.21.15.131:0/3530247063 shutdown_connections 2024-05-10T05:26:43.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.482+0000 7f52caa00700 1 -- 172.21.15.131:0/3530247063 wait complete. 2024-05-10T05:26:43.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.483+0000 7f52caa00700 1 Processor -- start 2024-05-10T05:26:43.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.483+0000 7f52caa00700 1 -- start start 2024-05-10T05:26:43.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.483+0000 7f52caa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c4101560 0x7f52c4196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:43.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.483+0000 7f52caa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4197460 con 0x7f52c4101560 2024-05-10T05:26:43.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.483+0000 7f52c9600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c4101560 0x7f52c4196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:43.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.483+0000 7f52c9600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c4101560 0x7f52c4196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46872/0 (socket says 172.21.15.131:46872) 2024-05-10T05:26:43.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.483+0000 7f52c9600700 1 -- 172.21.15.131:0/153329554 learned_addr learned my addr 172.21.15.131:0/153329554 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:26:43.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.484+0000 7f52c9600700 1 -- 172.21.15.131:0/153329554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b80095f0 con 0x7f52c4101560 2024-05-10T05:26:43.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.484+0000 7f52c9600700 1 --2- 172.21.15.131:0/153329554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c4101560 0x7f52c4196f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52b8004f40 tx=0x7f52b8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:43.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.484+0000 7f52c2000700 1 -- 172.21.15.131:0/153329554 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52b801d070 con 0x7f52c4101560 2024-05-10T05:26:43.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.484+0000 7f52c2000700 1 -- 172.21.15.131:0/153329554 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b800bbb0 con 0x7f52c4101560 2024-05-10T05:26:43.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.484+0000 7f52caa00700 1 -- 172.21.15.131:0/153329554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c41976c0 con 0x7f52c4101560 2024-05-10T05:26:43.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.484+0000 7f52c2000700 1 -- 172.21.15.131:0/153329554 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52b8003810 con 0x7f52c4101560 2024-05-10T05:26:43.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.484+0000 7f52caa00700 1 -- 172.21.15.131:0/153329554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c4197b40 con 0x7f52c4101560 2024-05-10T05:26:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.485+0000 7f52c2000700 1 -- 172.21.15.131:0/153329554 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f52b8003a40 con 0x7f52c4101560 2024-05-10T05:26:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.486+0000 7f52caa00700 1 -- 172.21.15.131:0/153329554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c4102a10 con 0x7f52c4101560 2024-05-10T05:26:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.486+0000 7f52c2000700 1 --2- 172.21.15.131:0/153329554 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f52ac0387f0 0x7f52ac03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.486+0000 7f52c2000700 1 -- 172.21.15.131:0/153329554 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f52b804c4a0 con 0x7f52c4101560 2024-05-10T05:26:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.486+0000 7f52c8c00700 1 --2- 172.21.15.131:0/153329554 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f52ac0387f0 0x7f52ac03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:43.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.487+0000 7f52c8c00700 1 --2- 172.21.15.131:0/153329554 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f52ac0387f0 0x7f52ac03ad00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f52b0006fd0 tx=0x7f52b0006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:43.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.492+0000 7f52c2000700 1 -- 172.21.15.131:0/153329554 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f52b802ae20 con 0x7f52c4101560 2024-05-10T05:26:43.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:43.745+0000 7f52caa00700 1 -- 172.21.15.131:0/153329554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f52c410b670 con 0x7f52c4101560 2024-05-10T05:26:43.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:43 smithi131 ceph-mon[37264]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:43.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:43.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:43.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:43.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:43.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:26:43.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:43.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:26:44.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.667+0000 7f52c2000700 1 -- 172.21.15.131:0/153329554 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f52b8026020 con 0x7f52c4101560 2024-05-10T05:26:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.669+0000 7f52caa00700 1 -- 172.21.15.131:0/153329554 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f52ac0387f0 msgr2=0x7f52ac03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.669+0000 7f52caa00700 1 --2- 172.21.15.131:0/153329554 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f52ac0387f0 0x7f52ac03ad00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f52b0006fd0 tx=0x7f52b0006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.669+0000 7f52caa00700 1 -- 172.21.15.131:0/153329554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c4101560 msgr2=0x7f52c4196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.669+0000 7f52caa00700 1 --2- 172.21.15.131:0/153329554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c4101560 0x7f52c4196f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52b8004f40 tx=0x7f52b8005020 comp rx=0 tx=0).stop 2024-05-10T05:26:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.669+0000 7f52caa00700 1 -- 172.21.15.131:0/153329554 shutdown_connections 2024-05-10T05:26:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.669+0000 7f52caa00700 1 --2- 172.21.15.131:0/153329554 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f52ac0387f0 0x7f52ac03ad00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.669+0000 7f52caa00700 1 --2- 172.21.15.131:0/153329554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52c4101560 0x7f52c4196f20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.670+0000 7f52caa00700 1 -- 172.21.15.131:0/153329554 >> 172.21.15.131:0/153329554 conn(0x7f52c40fce10 msgr2=0x7f52c4107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.670+0000 7f52caa00700 1 -- 172.21.15.131:0/153329554 shutdown_connections 2024-05-10T05:26:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:26:44.670+0000 7f52caa00700 1 -- 172.21.15.131:0/153329554 wait complete. 2024-05-10T05:26:44.671 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2024-05-10T05:26:44.916 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:44 smithi131 ceph-mon[37264]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-10T05:26:44.917 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:44 smithi131 ceph-mon[37264]: Updating smithi157:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T05:26:44.917 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:44 smithi131 ceph-mon[37264]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-10T05:26:44.917 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:44 smithi131 ceph-mon[37264]: Updating smithi157:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T05:26:44.917 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:44 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/153329554' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T05:26:44.917 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:44 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:44.917 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:44 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:44.917 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:44 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:45.072 INFO:tasks.cephadm:Adding mon.a on smithi131 2024-05-10T05:26:45.073 INFO:tasks.cephadm:Adding mon.b on smithi142 2024-05-10T05:26:45.073 INFO:tasks.cephadm:Adding mon.c on smithi157 2024-05-10T05:26:45.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi131:172.21.15.131=a;smithi142:172.21.15.142=b;smithi157:172.21.15.157=c' 2024-05-10T05:26:45.328 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T05:26:45.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:45 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/153329554' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T05:26:45.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:45 smithi131 ceph-mon[37264]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:26:45.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:45 smithi131 ceph-mon[37264]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:46.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.232+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/3719258766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb19c100610 msgr2=0x7fb19c100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:46.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.232+0000 7fb1a2c9c700 1 --2- 172.21.15.157:0/3719258766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb19c100610 0x7fb19c100a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb190009910 tx=0x7fb190009c20 comp rx=0 tx=0).stop 2024-05-10T05:26:46.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.232+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/3719258766 shutdown_connections 2024-05-10T05:26:46.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.232+0000 7fb1a2c9c700 1 --2- 172.21.15.157:0/3719258766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb19c100610 0x7fb19c100a20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:46.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.232+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/3719258766 >> 172.21.15.157:0/3719258766 conn(0x7fb19c075f10 msgr2=0x7fb19c076350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:46.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.232+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/3719258766 shutdown_connections 2024-05-10T05:26:46.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.232+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/3719258766 wait complete. 2024-05-10T05:26:46.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.232+0000 7fb1a2c9c700 1 Processor -- start 2024-05-10T05:26:46.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.232+0000 7fb1a2c9c700 1 -- start start 2024-05-10T05:26:46.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.233+0000 7fb1a2c9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb19c100610 0x7fb19c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:46.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.233+0000 7fb1a2c9c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c069480 con 0x7fb19c100610 2024-05-10T05:26:46.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.234+0000 7fb1a0a38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb19c100610 0x7fb19c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:46.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.234+0000 7fb1a0a38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb19c100610 0x7fb19c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.157:35538/0 (socket says 172.21.15.157:35538) 2024-05-10T05:26:46.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.234+0000 7fb1a0a38700 1 -- 172.21.15.157:0/4028790692 learned_addr learned my addr 172.21.15.157:0/4028790692 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-10T05:26:46.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.234+0000 7fb1a0a38700 1 -- 172.21.15.157:0/4028790692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1900095f0 con 0x7fb19c100610 2024-05-10T05:26:46.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.234+0000 7fb1a0a38700 1 --2- 172.21.15.157:0/4028790692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb19c100610 0x7fb19c197ec0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb190004f40 tx=0x7fb190005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:46.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.235+0000 7fb199ffb700 1 -- 172.21.15.157:0/4028790692 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb19001d070 con 0x7fb19c100610 2024-05-10T05:26:46.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.235+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/4028790692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c198460 con 0x7fb19c100610 2024-05-10T05:26:46.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.235+0000 7fb199ffb700 1 -- 172.21.15.157:0/4028790692 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb19000bc10 con 0x7fb19c100610 2024-05-10T05:26:46.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.235+0000 7fb199ffb700 1 -- 172.21.15.157:0/4028790692 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1900037d0 con 0x7fb19c100610 2024-05-10T05:26:46.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.235+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/4028790692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c1988e0 con 0x7fb19c100610 2024-05-10T05:26:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.236+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/4028790692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c101ac0 con 0x7fb19c100610 2024-05-10T05:26:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.236+0000 7fb199ffb700 1 -- 172.21.15.157:0/4028790692 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fb190003a90 con 0x7fb19c100610 2024-05-10T05:26:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.236+0000 7fb199ffb700 1 --2- 172.21.15.157:0/4028790692 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb18c0387a0 0x7fb18c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.236+0000 7fb199ffb700 1 -- 172.21.15.157:0/4028790692 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb19004c510 con 0x7fb19c100610 2024-05-10T05:26:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.237+0000 7fb19bfff700 1 --2- 172.21.15.157:0/4028790692 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb18c0387a0 0x7fb18c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.237+0000 7fb19bfff700 1 --2- 172.21.15.157:0/4028790692 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb18c0387a0 0x7fb18c03acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb188006fd0 tx=0x7fb188006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:46.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.242+0000 7fb199ffb700 1 -- 172.21.15.157:0/4028790692 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb19002a820 con 0x7fb19c100610 2024-05-10T05:26:46.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.442+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/4028790692 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi142:172.21.15.142=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fb19c198c40 con 0x7fb18c0387a0 2024-05-10T05:26:46.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.452+0000 7fb199ffb700 1 -- 172.21.15.157:0/4028790692 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb19c198c40 con 0x7fb18c0387a0 2024-05-10T05:26:46.452 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-05-10T05:26:46.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.453+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/4028790692 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb18c0387a0 msgr2=0x7fb18c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:46.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.453+0000 7fb1a2c9c700 1 --2- 172.21.15.157:0/4028790692 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb18c0387a0 0x7fb18c03acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb188006fd0 tx=0x7fb188006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.453+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/4028790692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb19c100610 msgr2=0x7fb19c197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.453+0000 7fb1a2c9c700 1 --2- 172.21.15.157:0/4028790692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb19c100610 0x7fb19c197ec0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb190004f40 tx=0x7fb190005020 comp rx=0 tx=0).stop 2024-05-10T05:26:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.453+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/4028790692 shutdown_connections 2024-05-10T05:26:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.453+0000 7fb1a2c9c700 1 --2- 172.21.15.157:0/4028790692 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb18c0387a0 0x7fb18c03acb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.453+0000 7fb1a2c9c700 1 --2- 172.21.15.157:0/4028790692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb19c100610 0x7fb19c197ec0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.453+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/4028790692 >> 172.21.15.157:0/4028790692 conn(0x7fb19c075f10 msgr2=0x7fb19c101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.453+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/4028790692 shutdown_connections 2024-05-10T05:26:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:46.453+0000 7fb1a2c9c700 1 -- 172.21.15.157:0/4028790692 wait complete. 2024-05-10T05:26:46.865 DEBUG:teuthology.orchestra.run.smithi142:mon.b> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.b.service 2024-05-10T05:26:46.867 DEBUG:teuthology.orchestra.run.smithi157:mon.c> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.c.service 2024-05-10T05:26:46.869 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T05:26:46.870 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T05:26:46.899 INFO:journalctl@ceph.mon.c.smithi157.stdout:-- Logs begin at Fri 2024-05-10 05:00:45 UTC. -- 2024-05-10T05:26:46.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:-- Logs begin at Fri 2024-05-10 04:59:48 UTC. -- 2024-05-10T05:26:47.332 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T05:26:47.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi142:172.21.15.142=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:47.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: Saving service mon spec with placement smithi131:172.21.15.131=a;smithi142:172.21.15.142=b;smithi157:172.21.15.157=c;count:3 2024-05-10T05:26:47.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:47.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:47.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:47.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:26:47.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:47.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:26:47.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:47.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: Deploying daemon mon.c on smithi157 2024-05-10T05:26:47.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:47 smithi131 ceph-mon[37264]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:48.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.696+0000 7fe71b33c700 1 -- 172.21.15.157:0/827962642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71410fcc0 msgr2=0x7fe714108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:48.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.696+0000 7fe71b33c700 1 --2- 172.21.15.157:0/827962642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71410fcc0 0x7fe714108210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe710009910 tx=0x7fe710009c20 comp rx=0 tx=0).stop 2024-05-10T05:26:48.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.696+0000 7fe71b33c700 1 -- 172.21.15.157:0/827962642 shutdown_connections 2024-05-10T05:26:48.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.696+0000 7fe71b33c700 1 --2- 172.21.15.157:0/827962642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71410fcc0 0x7fe714108210 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:48.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.696+0000 7fe71b33c700 1 -- 172.21.15.157:0/827962642 >> 172.21.15.157:0/827962642 conn(0x7fe71406c5c0 msgr2=0x7fe71406ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:48.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.696+0000 7fe71b33c700 1 -- 172.21.15.157:0/827962642 shutdown_connections 2024-05-10T05:26:48.701 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.696+0000 7fe71b33c700 1 -- 172.21.15.157:0/827962642 wait complete. 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.697+0000 7fe71b33c700 1 Processor -- start 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.697+0000 7fe71b33c700 1 -- start start 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.697+0000 7fe71b33c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71410fcc0 0x7fe7141a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.697+0000 7fe71b33c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7141a6430 con 0x7fe71410fcc0 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.698+0000 7fe7190d8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71410fcc0 0x7fe7141a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.698+0000 7fe7190d8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71410fcc0 0x7fe7141a5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.157:35562/0 (socket says 172.21.15.157:35562) 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.698+0000 7fe7190d8700 1 -- 172.21.15.157:0/109503987 learned_addr learned my addr 172.21.15.157:0/109503987 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.699+0000 7fe7190d8700 1 -- 172.21.15.157:0/109503987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7100095f0 con 0x7fe71410fcc0 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.699+0000 7fe7190d8700 1 --2- 172.21.15.157:0/109503987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71410fcc0 0x7fe7141a5ef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe710006010 tx=0x7fe710004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.699+0000 7fe70a7fc700 1 -- 172.21.15.157:0/109503987 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe71001d070 con 0x7fe71410fcc0 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.699+0000 7fe71b33c700 1 -- 172.21.15.157:0/109503987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7141a6630 con 0x7fe71410fcc0 2024-05-10T05:26:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.699+0000 7fe71b33c700 1 -- 172.21.15.157:0/109503987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7141a6a50 con 0x7fe71410fcc0 2024-05-10T05:26:48.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.699+0000 7fe70a7fc700 1 -- 172.21.15.157:0/109503987 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe71000bc10 con 0x7fe71410fcc0 2024-05-10T05:26:48.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.699+0000 7fe70a7fc700 1 -- 172.21.15.157:0/109503987 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7100038f0 con 0x7fe71410fcc0 2024-05-10T05:26:48.703 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.700+0000 7fe71b33c700 1 -- 172.21.15.157:0/109503987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f80052f0 con 0x7fe71410fcc0 2024-05-10T05:26:48.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.701+0000 7fe70a7fc700 1 -- 172.21.15.157:0/109503987 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fe710003b60 con 0x7fe71410fcc0 2024-05-10T05:26:48.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.701+0000 7fe70a7fc700 1 --2- 172.21.15.157:0/109503987 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe7000387a0 0x7fe70003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:48.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.701+0000 7fe70a7fc700 1 -- 172.21.15.157:0/109503987 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe71004c2b0 con 0x7fe71410fcc0 2024-05-10T05:26:48.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.701+0000 7fe7188d7700 1 --2- 172.21.15.157:0/109503987 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe7000387a0 0x7fe70003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:48.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.702+0000 7fe7188d7700 1 --2- 172.21.15.157:0/109503987 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe7000387a0 0x7fe70003acb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe704006fd0 tx=0x7fe704006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:48.708 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:48.707+0000 7fe70a7fc700 1 -- 172.21.15.157:0/109503987 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe710015760 con 0x7fe71410fcc0 2024-05-10T05:26:49.013 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.011+0000 7fe71b33c700 1 -- 172.21.15.157:0/109503987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe6f80055a0 con 0x7fe71410fcc0 2024-05-10T05:26:49.013 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.012+0000 7fe70a7fc700 1 -- 172.21.15.157:0/109503987 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fe710026030 con 0x7fe71410fcc0 2024-05-10T05:26:49.014 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:26:49.014 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","modified":"2024-05-10T05:25:15.890090Z","created":"2024-05-10T05:25:15.890090Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-10T05:26:49.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.014+0000 7fe71b33c700 1 -- 172.21.15.157:0/109503987 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe7000387a0 msgr2=0x7fe70003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:49.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.014+0000 7fe71b33c700 1 --2- 172.21.15.157:0/109503987 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe7000387a0 0x7fe70003acb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe704006fd0 tx=0x7fe704006e40 comp rx=0 tx=0).stop 2024-05-10T05:26:49.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.014+0000 7fe71b33c700 1 -- 172.21.15.157:0/109503987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71410fcc0 msgr2=0x7fe7141a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:49.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.014+0000 7fe71b33c700 1 --2- 172.21.15.157:0/109503987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71410fcc0 0x7fe7141a5ef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe710006010 tx=0x7fe710004ff0 comp rx=0 tx=0).stop 2024-05-10T05:26:49.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.014+0000 7fe71b33c700 1 -- 172.21.15.157:0/109503987 shutdown_connections 2024-05-10T05:26:49.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.014+0000 7fe71b33c700 1 --2- 172.21.15.157:0/109503987 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe7000387a0 0x7fe70003acb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:49.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.014+0000 7fe71b33c700 1 --2- 172.21.15.157:0/109503987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe71410fcc0 0x7fe7141a5ef0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:49.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.014+0000 7fe71b33c700 1 -- 172.21.15.157:0/109503987 >> 172.21.15.157:0/109503987 conn(0x7fe71406c5c0 msgr2=0x7fe71406fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:49.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.015+0000 7fe71b33c700 1 -- 172.21.15.157:0/109503987 shutdown_connections 2024-05-10T05:26:49.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:49.015+0000 7fe71b33c700 1 -- 172.21.15.157:0/109503987 wait complete. 2024-05-10T05:26:49.018 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-05-10T05:26:50.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 systemd[1]: Started Ceph mon.c for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:26:50.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T05:26:50.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable), process ceph-mon, pid 7 2024-05-10T05:26:50.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: pidfile_write: ignore empty --pid-file 2024-05-10T05:26:50.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: load: jerasure load: lrc 2024-05-10T05:26:50.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: RocksDB version: 7.9.2 2024-05-10T05:26:50.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Git sha 0 2024-05-10T05:26:50.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Compile date 2024-05-10 00:51:33 2024-05-10T05:26:50.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: DB SUMMARY 2024-05-10T05:26:50.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: DB Session ID: ZMPNU62GK0ZWIKLCOKZH 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: CURRENT file: CURRENT 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.error_if_exists: 0 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.create_if_missing: 0 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.paranoid_checks: 1 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T05:26:50.056 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.env: 0x555f9b6f0460 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.info_log: 0x555f9de65760 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.statistics: (nil) 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.use_fsync: 0 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_log_file_size: 0 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.allow_fallocate: 1 2024-05-10T05:26:50.057 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.use_direct_reads: 0 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.db_log_dir: 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.wal_dir: 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T05:26:50.058 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.write_buffer_manager: 0x555f9dda90e0 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.unordered_write: 0 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T05:26:50.059 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.row_cache: None 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.wal_filter: None 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.two_write_queues: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.wal_compression: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.atomic_flush: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.log_readahead_size: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T05:26:50.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_background_jobs: 2 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_background_compactions: -1 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_subcompactions: 1 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T05:26:50.061 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_open_files: -1 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_background_flushes: -1 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Compression algorithms supported: 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: kZSTD supported: 0 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: kXpressCompression supported: 0 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T05:26:50.062 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: kZlibCompression supported: 1 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: kSnappyCompression supported: 1 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: kLZ4Compression supported: 1 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: kBZip2Compression supported: 0 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.merge_operator: 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_filter: None 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T05:26:50.063 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555f9de65880) 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks: 1 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_top_level_index_and_filter: 1 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_type: 0 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_index_type: 0 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_shortening: 1 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: checksum: 4 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: no_block_cache: 0 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache: 0x555f9de2b1f0 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_name: BinnedLRUCache 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_options: 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: capacity : 536870912 2024-05-10T05:26:50.064 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_shard_bits : 4 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: strict_capacity_limit : 0 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: high_pri_pool_ratio: 0.000 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_compressed: (nil) 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: persistent_cache: (nil) 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size: 4096 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size_deviation: 10 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_restart_interval: 16 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_block_restart_interval: 1 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: metadata_block_size: 4096 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: partition_filters: 0 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: use_delta_encoding: 1 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: filter_policy: bloomfilter 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: whole_key_filtering: 1 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: verify_compression: 0 2024-05-10T05:26:50.065 INFO:journalctl@ceph.mon.c.smithi157.stdout: read_amp_bytes_per_bit: 0 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout: format_version: 5 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_index_compression: 1 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_align: 0 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout: max_auto_readahead_size: 262144 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout: prepopulate_block_cache: 0 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout: initial_auto_readahead_size: 8192 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compression: NoCompression 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T05:26:50.066 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.num_levels: 7 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T05:26:50.067 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T05:26:50.068 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.table_properties_collectors: 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.inplace_update_support: 0 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.bloom_locality: 0 2024-05-10T05:26:50.070 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.max_successive_merges: 0 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.ttl: 2592000 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.enable_blob_files: false 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.min_blob_size: 0 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T05:26:50.071 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8e7fcb10-4c01-43d8-912f-7d7172562f91 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715318809707905, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715318809708344, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715318809, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8e7fcb10-4c01-43d8-912f-7d7172562f91", "db_session_id": "ZMPNU62GK0ZWIKLCOKZH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715318809708422, "job": 1, "event": "recovery_finished"} 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T05:26:50.072 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555f9ddbdc00 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: DB pointer 0x555fa2bac000 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** DB Stats ** 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T05:26:50.073 INFO:journalctl@ceph.mon.c.smithi157.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T05:26:50.074 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache BinnedLRUCache@0x555f9de2b1f0#7 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: using public_addr v2:172.21.15.157:0/0 -> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: starting mon.c rank -1 at public addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] at bind addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:26:50.075 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(???) e0 preinit fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).mds e1 new map 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout: e1 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout: legacy client fscid: -1 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout: No filesystems configured 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T05:26:50.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: Deploying cephadm binary to smithi157 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: Added host smithi157 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.077 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: Updating smithi157:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T05:26:50.078 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: Updating smithi157:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/153329554' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/153329554' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi142:172.21.15.142=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: Saving service mon spec with placement smithi131:172.21.15.131=a;smithi142:172.21.15.142=b;smithi157:172.21.15.157=c;count:3 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:26:50.079 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:50.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:26:50.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:50.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: Deploying daemon mon.c on smithi157 2024-05-10T05:26:50.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:50.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:49 smithi157 ceph-mon[78427]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T05:26:50.379 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T05:26:50.379 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T05:26:50.605 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:26:54.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.857+0000 7f3f8ad12700 1 -- 172.21.15.157:0/4043387901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f740035c0 msgr2=0x7f3f74005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:54.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.857+0000 7f3f8ad12700 1 --2- 172.21.15.157:0/4043387901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f740035c0 0x7f3f74005aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f84068d20 tx=0x7f3f78013e90 comp rx=0 tx=0).stop 2024-05-10T05:26:54.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.857+0000 7f3f8ad12700 1 -- 172.21.15.157:0/4043387901 shutdown_connections 2024-05-10T05:26:54.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.857+0000 7f3f8ad12700 1 --2- 172.21.15.157:0/4043387901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f740035c0 0x7f3f74005aa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:54.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.857+0000 7f3f8ad12700 1 -- 172.21.15.157:0/4043387901 >> 172.21.15.157:0/4043387901 conn(0x7f3f840757e0 msgr2=0x7f3f84075c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:54.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.857+0000 7f3f8ad12700 1 -- 172.21.15.157:0/4043387901 shutdown_connections 2024-05-10T05:26:54.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.857+0000 7f3f8ad12700 1 -- 172.21.15.157:0/4043387901 wait complete. 2024-05-10T05:26:54.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.858+0000 7f3f8ad12700 1 Processor -- start 2024-05-10T05:26:54.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.858+0000 7f3f8ad12700 1 -- start start 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f8ad12700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f740035c0 0x7f3f841971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f8ad12700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f841976e0 0x7f3f8419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f8ad12700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84197c60 con 0x7f3f841976e0 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f8ad12700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84197da0 con 0x7f3f740035c0 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f88aae700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f740035c0 0x7f3f841971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f88aae700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f740035c0 0x7f3f841971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48730/0 (socket says 172.21.15.157:48730) 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f88aae700 1 -- 172.21.15.157:0/1739571312 learned_addr learned my addr 172.21.15.157:0/1739571312 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f88aae700 1 -- 172.21.15.157:0/1739571312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f740035c0 msgr2=0x7f3f841971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f88aae700 1 -- 172.21.15.157:0/1739571312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f740035c0 msgr2=0x7f3f841971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f88aae700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f740035c0 0x7f3f841971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f83fff700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f841976e0 0x7f3f8419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:54.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.859+0000 7f3f88aae700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f740035c0 0x7f3f841971a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T05:26:54.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.860+0000 7f3f83fff700 1 -- 172.21.15.157:0/1739571312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f740035c0 msgr2=0x7f3f841971a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:26:54.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.860+0000 7f3f83fff700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f740035c0 0x7f3f841971a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:54.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.860+0000 7f3f83fff700 1 -- 172.21.15.157:0/1739571312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f780138e0 con 0x7f3f841976e0 2024-05-10T05:26:54.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.860+0000 7f3f83fff700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f841976e0 0x7f3f8419ba40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000b4d0 tx=0x7f3f7000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:54.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.860+0000 7f3f81ffb700 1 -- 172.21.15.157:0/1739571312 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f3f70011630 con 0x7f3f841976e0 2024-05-10T05:26:54.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.860+0000 7f3f81ffb700 1 -- 172.21.15.157:0/1739571312 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f70011c50 con 0x7f3f841976e0 2024-05-10T05:26:54.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.860+0000 7f3f8ad12700 1 -- 172.21.15.157:0/1739571312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8419c040 con 0x7f3f841976e0 2024-05-10T05:26:54.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.860+0000 7f3f81ffb700 1 -- 172.21.15.157:0/1739571312 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f3f70010d90 con 0x7f3f841976e0 2024-05-10T05:26:54.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.860+0000 7f3f8ad12700 1 -- 172.21.15.157:0/1739571312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8419c610 con 0x7f3f841976e0 2024-05-10T05:26:54.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.861+0000 7f3f81ffb700 1 -- 172.21.15.157:0/1739571312 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f3f7001a400 con 0x7f3f841976e0 2024-05-10T05:26:54.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.862+0000 7f3f8ad12700 1 -- 172.21.15.157:0/1739571312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f84190c50 con 0x7f3f841976e0 2024-05-10T05:26:54.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.862+0000 7f3f81ffb700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3f68038620 0x7f3f6803ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:26:54.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.862+0000 7f3f81ffb700 1 -- 172.21.15.157:0/1739571312 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3f7004aee0 con 0x7f3f841976e0 2024-05-10T05:26:54.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.862+0000 7f3f88aae700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3f68038620 0x7f3f6803ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:26:54.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.863+0000 7f3f88aae700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3f68038620 0x7f3f6803ab30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3f78001eb0 tx=0x7f3f78013460 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:26:54.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:54.868+0000 7f3f81ffb700 1 -- 172.21.15.157:0/1739571312 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3f70019730 con 0x7f3f841976e0 2024-05-10T05:26:55.140 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.139+0000 7f3f8ad12700 1 -- 172.21.15.157:0/1739571312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3f84198500 con 0x7f3f841976e0 2024-05-10T05:26:55.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.140+0000 7f3f81ffb700 1 -- 172.21.15.157:0/1739571312 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f3f700172c0 con 0x7f3f841976e0 2024-05-10T05:26:55.141 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:26:55.141 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","modified":"2024-05-10T05:26:49.737026Z","created":"2024-05-10T05:25:15.890090Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-10T05:26:55.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 -- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3f68038620 msgr2=0x7f3f6803ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:55.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3f68038620 0x7f3f6803ab30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3f78001eb0 tx=0x7f3f78013460 comp rx=0 tx=0).stop 2024-05-10T05:26:55.143 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 -- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f841976e0 msgr2=0x7f3f8419ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:26:55.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f841976e0 0x7f3f8419ba40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000b4d0 tx=0x7f3f7000b890 comp rx=0 tx=0).stop 2024-05-10T05:26:55.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 -- 172.21.15.157:0/1739571312 shutdown_connections 2024-05-10T05:26:55.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3f68038620 0x7f3f6803ab30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:55.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f740035c0 0x7f3f841971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:55.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 --2- 172.21.15.157:0/1739571312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f841976e0 0x7f3f8419ba40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:26:55.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 -- 172.21.15.157:0/1739571312 >> 172.21.15.157:0/1739571312 conn(0x7f3f840757e0 msgr2=0x7f3f84100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:26:55.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 -- 172.21.15.157:0/1739571312 shutdown_connections 2024-05-10T05:26:55.144 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:26:55.143+0000 7f3f8ad12700 1 -- 172.21.15.157:0/1739571312 wait complete. 2024-05-10T05:26:55.144 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-05-10T05:26:55.167 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:26:55.167 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.167 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: mon.a calling monitor election 2024-05-10T05:26:55.167 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.167 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:55.167 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.167 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: mon.c calling monitor election 2024-05-10T05:26:55.167 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: monmap e2: 2 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: fsmap 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: mgrmap e13: x(active, since 40s) 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: overall HEALTH_OK 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:26:55.168 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:55.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:26:55.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: mon.a calling monitor election 2024-05-10T05:26:55.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: mon.c calling monitor election 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: monmap e2: 2 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: fsmap 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: mgrmap e13: x(active, since 40s) 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: overall HEALTH_OK 2024-05-10T05:26:55.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:55.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:55.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:55.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:26:55.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:56.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:55 smithi131 ceph-mon[37264]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:56.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:55 smithi131 ceph-mon[37264]: Deploying daemon mon.b on smithi142 2024-05-10T05:26:56.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:55 smithi131 ceph-mon[37264]: from='client.? 172.21.15.157:0/1739571312' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T05:26:56.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:26:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:55 smithi157 ceph-mon[78427]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:55 smithi157 ceph-mon[78427]: Deploying daemon mon.b on smithi142 2024-05-10T05:26:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:55 smithi157 ceph-mon[78427]: from='client.? 172.21.15.157:0/1739571312' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T05:26:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:26:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:56.531 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T05:26:56.532 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T05:26:56.762 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:26:57.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 systemd[1]: Started Ceph mon.b for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:26:57.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T05:26:57.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable), process ceph-mon, pid 7 2024-05-10T05:26:57.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: pidfile_write: ignore empty --pid-file 2024-05-10T05:26:57.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: load: jerasure load: lrc 2024-05-10T05:26:57.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: RocksDB version: 7.9.2 2024-05-10T05:26:57.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Git sha 0 2024-05-10T05:26:57.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Compile date 2024-05-10 00:51:33 2024-05-10T05:26:57.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: DB SUMMARY 2024-05-10T05:26:57.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: DB Session ID: L06XC5KJ039HEV7ECM2Q 2024-05-10T05:26:57.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: CURRENT file: CURRENT 2024-05-10T05:26:57.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T05:26:57.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T05:26:57.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-10T05:26:57.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-10T05:26:57.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.error_if_exists: 0 2024-05-10T05:26:57.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.create_if_missing: 0 2024-05-10T05:26:57.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.paranoid_checks: 1 2024-05-10T05:26:57.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T05:26:57.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T05:26:57.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T05:26:57.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.env: 0x56000e330460 2024-05-10T05:26:57.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.info_log: 0x56000f5b7760 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.statistics: (nil) 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.use_fsync: 0 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_log_file_size: 0 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.allow_fallocate: 1 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T05:26:57.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.use_direct_reads: 0 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.db_log_dir: 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.wal_dir: 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T05:26:57.650 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T05:26:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T05:26:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T05:26:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.write_buffer_manager: 0x56000f4fb0e0 2024-05-10T05:26:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T05:26:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T05:26:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T05:26:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T05:26:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T05:26:57.651 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.unordered_write: 0 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.row_cache: None 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.wal_filter: None 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T05:26:57.652 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.two_write_queues: 0 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.wal_compression: 0 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.atomic_flush: 0 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.log_readahead_size: 0 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T05:26:57.653 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T05:26:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T05:26:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T05:26:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_background_jobs: 2 2024-05-10T05:26:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_background_compactions: -1 2024-05-10T05:26:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_subcompactions: 1 2024-05-10T05:26:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T05:26:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T05:26:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T05:26:57.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_open_files: -1 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_background_flushes: -1 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Compression algorithms supported: 2024-05-10T05:26:57.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: kZSTD supported: 0 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: kXpressCompression supported: 0 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: kZlibCompression supported: 1 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: kSnappyCompression supported: 1 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: kLZ4Compression supported: 1 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: kBZip2Compression supported: 0 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T05:26:57.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.merge_operator: 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_filter: None 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56000f5b7880) 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T05:26:57.657 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_type: 0 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_index_type: 0 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_shortening: 1 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: checksum: 4 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: no_block_cache: 0 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache: 0x56000f57d1f0 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_options: 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: capacity : 536870912 2024-05-10T05:26:57.658 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_shard_bits : 4 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: strict_capacity_limit : 0 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_compressed: (nil) 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: persistent_cache: (nil) 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size: 4096 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size_deviation: 10 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_restart_interval: 16 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_block_restart_interval: 1 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: metadata_block_size: 4096 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: partition_filters: 0 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: use_delta_encoding: 1 2024-05-10T05:26:57.659 INFO:journalctl@ceph.mon.b.smithi142.stdout: filter_policy: bloomfilter 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout: whole_key_filtering: 1 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout: verify_compression: 0 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout: format_version: 5 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_index_compression: 1 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_align: 0 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout: max_auto_readahead_size: 262144 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout: prepopulate_block_cache: 0 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout: initial_auto_readahead_size: 8192 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T05:26:57.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compression: NoCompression 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.num_levels: 7 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T05:26:57.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T05:26:57.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T05:26:57.663 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T05:26:57.664 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.table_properties_collectors: 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.inplace_update_support: 0 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.bloom_locality: 0 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.max_successive_merges: 0 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T05:26:57.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.ttl: 2592000 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.enable_blob_files: false 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.min_blob_size: 0 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T05:26:57.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: dcc2b654-5e5b-4c44-a777-50b609dc1709 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715318817307189, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715318817307635, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715318817, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "dcc2b654-5e5b-4c44-a777-50b609dc1709", "db_session_id": "L06XC5KJ039HEV7ECM2Q", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715318817307709, "job": 1, "event": "recovery_finished"} 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56000f50fc00 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: DB pointer 0x5600142fe000 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-10T05:26:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** DB Stats ** 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T05:26:57.668 INFO:journalctl@ceph.mon.b.smithi142.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T05:26:57.669 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache BinnedLRUCache@0x56000f57d1f0#7 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: starting mon.b rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(???) e0 preinit fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).mds e1 new map 2024-05-10T05:26:57.670 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout: e1 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout: legacy client fscid: -1 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout: No filesystems configured 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T05:26:57.671 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: Deploying cephadm binary to smithi157 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: Added host smithi157 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:26:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: Updating smithi157:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: Updating smithi157:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/153329554' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/153329554' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi142:172.21.15.142=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: Saving service mon spec with placement smithi131:172.21.15.131=a;smithi142:172.21.15.142=b;smithi157:172.21.15.157=c;count:3 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:57.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: Deploying daemon mon.c on smithi157 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.a calling monitor election 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.c calling monitor election 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T05:26:57.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: monmap e2: 2 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: fsmap 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mgrmap e13: x(active, since 40s) 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: overall HEALTH_OK 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: Deploying daemon mon.b on smithi142 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='client.? 172.21.15.157:0/1739571312' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:26:57.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:26:57 smithi142 ceph-mon[77667]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T05:27:02.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:27:02.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: mon.c calling monitor election 2024-05-10T05:27:02.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: mon.a calling monitor election 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: fsmap 2024-05-10T05:27:02.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:27:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: mgrmap e13: x(active, since 47s) 2024-05-10T05:27:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-10T05:27:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-10T05:27:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-10T05:27:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2024-05-10T05:27:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:27:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: mon.c calling monitor election 2024-05-10T05:27:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: mon.a calling monitor election 2024-05-10T05:27:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: fsmap 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: mgrmap e13: x(active, since 47s) 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-10T05:27:02.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-10T05:27:02.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-10T05:27:02.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2024-05-10T05:27:02.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:02.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:02.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:02.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:03.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:03.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:03 smithi131 ceph-mon[37264]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:03.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:03.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:03.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:03.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.713+0000 7f7fb568f700 1 -- 172.21.15.157:0/3815840848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb01065b0 msgr2=0x7f7f9c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.713+0000 7f7fb568f700 1 --2- 172.21.15.157:0/3815840848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb01065b0 0x7f7f9c005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00098a0 tx=0x7f7fa00075c0 comp rx=0 tx=0).stop 2024-05-10T05:27:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.713+0000 7f7fb568f700 1 -- 172.21.15.157:0/3815840848 shutdown_connections 2024-05-10T05:27:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.713+0000 7f7fb568f700 1 --2- 172.21.15.157:0/3815840848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb01065b0 0x7f7f9c005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.713+0000 7f7fb568f700 1 -- 172.21.15.157:0/3815840848 >> 172.21.15.157:0/3815840848 conn(0x7f7fb00fbfb0 msgr2=0x7f7fb00fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.713+0000 7f7fb568f700 1 -- 172.21.15.157:0/3815840848 shutdown_connections 2024-05-10T05:27:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.713+0000 7f7fb568f700 1 -- 172.21.15.157:0/3815840848 wait complete. 2024-05-10T05:27:03.716 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.714+0000 7f7fb568f700 1 Processor -- start 2024-05-10T05:27:03.716 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.714+0000 7f7fb568f700 1 -- start start 2024-05-10T05:27:03.716 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.714+0000 7f7fb568f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb01065b0 0x7f7fb0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:03.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fb568f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fb0198520 0x7f7fb0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:03.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fb568f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb01a1050 0x7f7fb01a3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:03.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fb568f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01a3b00 con 0x7f7fb01065b0 2024-05-10T05:27:03.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fb568f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01a3c70 con 0x7f7fb01a1050 2024-05-10T05:27:03.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fb568f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01a3e30 con 0x7f7fb0198520 2024-05-10T05:27:03.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fae7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fb0198520 0x7f7fb0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:03.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fae7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fb0198520 0x7f7fb0198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48744/0 (socket says 172.21.15.157:48744) 2024-05-10T05:27:03.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fae7fc700 1 -- 172.21.15.157:0/2429218151 learned_addr learned my addr 172.21.15.157:0/2429218151 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-10T05:27:03.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fae7fc700 1 -- 172.21.15.157:0/2429218151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fb0198520 msgr2=0x7f7fb0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-10T05:27:03.717 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fae7fc700 1 -- 172.21.15.157:0/2429218151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fb0198520 msgr2=0x7f7fb0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T05:27:03.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fae7fc700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fb0198520 0x7f7fb0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T05:27:03.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faf7fe700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb01a1050 0x7f7fb01a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:03.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faeffd700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb01065b0 0x7f7fb0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:03.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7fae7fc700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fb0198520 0x7f7fb0198930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T05:27:03.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faf7fe700 1 -- 172.21.15.157:0/2429218151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb01a1050 msgr2=0x7f7fb01a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-10T05:27:03.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faf7fe700 1 -- 172.21.15.157:0/2429218151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb01a1050 msgr2=0x7f7fb01a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T05:27:03.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faf7fe700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb01a1050 0x7f7fb01a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T05:27:03.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faf7fe700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb01a1050 0x7f7fb01a3470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T05:27:03.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faeffd700 1 -- 172.21.15.157:0/2429218151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fb0198520 msgr2=0x7f7fb0198930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:27:03.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faeffd700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fb0198520 0x7f7fb0198930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:03.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faeffd700 1 -- 172.21.15.157:0/2429218151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb01a1050 msgr2=0x7f7fb01a3470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:27:03.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faeffd700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb01a1050 0x7f7fb01a3470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:03.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faeffd700 1 -- 172.21.15.157:0/2429218151 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa00099b0 con 0x7f7fb01065b0 2024-05-10T05:27:03.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.715+0000 7f7faeffd700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb01065b0 0x7f7fb0197fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00098a0 tx=0x7f7fa00061f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:03.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.716+0000 7f7fa7fff700 1 -- 172.21.15.157:0/2429218151 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0003c00 con 0x7f7fb01065b0 2024-05-10T05:27:03.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.716+0000 7f7fa7fff700 1 -- 172.21.15.157:0/2429218151 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fa0003d60 con 0x7f7fb01065b0 2024-05-10T05:27:03.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.716+0000 7f7fb568f700 1 -- 172.21.15.157:0/2429218151 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb01a40b0 con 0x7f7fb01065b0 2024-05-10T05:27:03.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.716+0000 7f7fa7fff700 1 -- 172.21.15.157:0/2429218151 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0004350 con 0x7f7fb01065b0 2024-05-10T05:27:03.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.716+0000 7f7fb568f700 1 -- 172.21.15.157:0/2429218151 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb01a4620 con 0x7f7fb01065b0 2024-05-10T05:27:03.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.717+0000 7f7fa7fff700 1 -- 172.21.15.157:0/2429218151 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f7fa00044b0 con 0x7f7fb01065b0 2024-05-10T05:27:03.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.717+0000 7f7fa7fff700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7f90038c70 0x7f7f9003b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:03.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.717+0000 7f7fb568f700 1 -- 172.21.15.157:0/2429218151 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb018fb50 con 0x7f7fb01065b0 2024-05-10T05:27:03.720 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.717+0000 7f7fa7fff700 1 -- 172.21.15.157:0/2429218151 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7fa004edf0 con 0x7f7fb01065b0 2024-05-10T05:27:03.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.717+0000 7f7fae7fc700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7f90038c70 0x7f7f9003b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:03.721 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.718+0000 7f7fae7fc700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7f90038c70 0x7f7f9003b180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0105da0 tx=0x7f7f98008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:03.723 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:03.723+0000 7f7fa7fff700 1 -- 172.21.15.157:0/2429218151 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7fa0004760 con 0x7f7fb01065b0 2024-05-10T05:27:03.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:03.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:03 smithi157 ceph-mon[78427]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:03.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:03.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:03.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:03.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:04.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.014+0000 7f7fb568f700 1 -- 172.21.15.157:0/2429218151 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7fb0069b00 con 0x7f7fb01065b0 2024-05-10T05:27:04.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.014+0000 7f7fa7fff700 1 -- 172.21.15.157:0/2429218151 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1278 (secure 0 0 0) 0x7f7fa0034b40 con 0x7f7fb01065b0 2024-05-10T05:27:04.015 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-10T05:27:04.015 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","modified":"2024-05-10T05:26:57.340444Z","created":"2024-05-10T05:25:15.890090Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-10T05:27:04.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.016+0000 7f7fb568f700 1 -- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7f90038c70 msgr2=0x7f7f9003b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:04.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.016+0000 7f7fb568f700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7f90038c70 0x7f7f9003b180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0105da0 tx=0x7f7f98008040 comp rx=0 tx=0).stop 2024-05-10T05:27:04.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.016+0000 7f7fb568f700 1 -- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb01065b0 msgr2=0x7f7fb0197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:04.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.016+0000 7f7fb568f700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb01065b0 0x7f7fb0197fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00098a0 tx=0x7f7fa00061f0 comp rx=0 tx=0).stop 2024-05-10T05:27:04.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.016+0000 7f7fb568f700 1 -- 172.21.15.157:0/2429218151 shutdown_connections 2024-05-10T05:27:04.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.016+0000 7f7fb568f700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7f90038c70 0x7f7f9003b180 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:04.017 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.016+0000 7f7fb568f700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fb01065b0 0x7f7fb0197fe0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.016+0000 7f7fb568f700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb01a1050 0x7f7fb01a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.017+0000 7f7fb568f700 1 --2- 172.21.15.157:0/2429218151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fb0198520 0x7f7fb0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.017+0000 7f7fb568f700 1 -- 172.21.15.157:0/2429218151 >> 172.21.15.157:0/2429218151 conn(0x7f7fb00fbfb0 msgr2=0x7f7fb00fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.017+0000 7f7fb568f700 1 -- 172.21.15.157:0/2429218151 shutdown_connections 2024-05-10T05:27:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:04.017+0000 7f7fb568f700 1 -- 172.21.15.157:0/2429218151 wait complete. 2024-05-10T05:27:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2024-05-10T05:27:04.399 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-10T05:27:04.399 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-10T05:27:04.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mon.c calling monitor election 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mon.a calling monitor election 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T05:27:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: fsmap 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mgrmap e13: x(active, since 47s) 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:04.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:04.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:04.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:04.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:04.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:04.757 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:27:05.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: mon.b calling monitor election 2024-05-10T05:27:05.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: mon.b calling monitor election 2024-05-10T05:27:05.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: mon.c calling monitor election 2024-05-10T05:27:05.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: mon.a calling monitor election 2024-05-10T05:27:05.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T05:27:05.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: fsmap 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: mgrmap e13: x(active, since 49s) 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: Cluster is now healthy 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: overall HEALTH_OK 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:27:05.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T05:27:05.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:05.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: mon.b calling monitor election 2024-05-10T05:27:05.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: mon.b calling monitor election 2024-05-10T05:27:05.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: mon.c calling monitor election 2024-05-10T05:27:05.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: mon.a calling monitor election 2024-05-10T05:27:05.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T05:27:05.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:27:05.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: fsmap 2024-05-10T05:27:05.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:27:05.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: mgrmap e13: x(active, since 49s) 2024-05-10T05:27:05.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-10T05:27:05.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: Cluster is now healthy 2024-05-10T05:27:05.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: overall HEALTH_OK 2024-05-10T05:27:05.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:27:05.307 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T05:27:05.307 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:05.307 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mon.b calling monitor election 2024-05-10T05:27:05.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mon.b calling monitor election 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mon.c calling monitor election 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mon.a calling monitor election 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: fsmap 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: mgrmap e13: x(active, since 49s) 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: Cluster is now healthy 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: overall HEALTH_OK 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:27:05.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T05:27:05.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:05.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:05.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.948+0000 7f6b56400700 1 -- 172.21.15.131:0/1428575619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b50107580 msgr2=0x7f6b50107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:05.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.948+0000 7f6b56400700 1 --2- 172.21.15.131:0/1428575619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b50107580 0x7f6b50107990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009910 tx=0x7f6b40009c20 comp rx=0 tx=0).stop 2024-05-10T05:27:05.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.948+0000 7f6b56400700 1 -- 172.21.15.131:0/1428575619 shutdown_connections 2024-05-10T05:27:05.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.948+0000 7f6b56400700 1 --2- 172.21.15.131:0/1428575619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b50107580 0x7f6b50107990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:05.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.948+0000 7f6b56400700 1 -- 172.21.15.131:0/1428575619 >> 172.21.15.131:0/1428575619 conn(0x7f6b500fcfc0 msgr2=0x7f6b500ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:05.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.948+0000 7f6b56400700 1 -- 172.21.15.131:0/1428575619 shutdown_connections 2024-05-10T05:27:05.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.948+0000 7f6b56400700 1 -- 172.21.15.131:0/1428575619 wait complete. 2024-05-10T05:27:05.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.950+0000 7f6b56400700 1 Processor -- start 2024-05-10T05:27:05.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.950+0000 7f6b56400700 1 -- start start 2024-05-10T05:27:05.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b56400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b50107580 0x7f6b50073510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:05.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b56400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b50073a50 0x7f6b50073e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:05.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b56400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b501a33f0 0x7f6b501a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:05.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b56400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b501a5d80 con 0x7f6b50073a50 2024-05-10T05:27:05.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b56400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b501a5ef0 con 0x7f6b50107580 2024-05-10T05:27:05.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b56400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b501a60b0 con 0x7f6b501a33f0 2024-05-10T05:27:05.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b47e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b50073a50 0x7f6b50073e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:05.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b4f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b50107580 0x7f6b50073510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:05.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b47e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b50073a50 0x7f6b50073e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37166/0 (socket says 172.21.15.131:37166) 2024-05-10T05:27:05.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b47e00700 1 -- 172.21.15.131:0/2879415020 learned_addr learned my addr 172.21.15.131:0/2879415020 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:27:05.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b4fe00700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b501a33f0 0x7f6b501a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:05.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b47e00700 1 -- 172.21.15.131:0/2879415020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b501a33f0 msgr2=0x7f6b501a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:05.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b47e00700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b501a33f0 0x7f6b501a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:05.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b47e00700 1 -- 172.21.15.131:0/2879415020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b50107580 msgr2=0x7f6b50073510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:05.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b47e00700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b50107580 0x7f6b50073510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:05.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.951+0000 7f6b47e00700 1 -- 172.21.15.131:0/2879415020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400095f0 con 0x7f6b50073a50 2024-05-10T05:27:05.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.952+0000 7f6b47e00700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b50073a50 0x7f6b50073e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800b780 tx=0x7f6b3800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:05.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.952+0000 7f6b4cc00700 1 -- 172.21.15.131:0/2879415020 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b38004230 con 0x7f6b50073a50 2024-05-10T05:27:05.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.952+0000 7f6b4cc00700 1 -- 172.21.15.131:0/2879415020 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b38004390 con 0x7f6b50073a50 2024-05-10T05:27:05.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.952+0000 7f6b56400700 1 -- 172.21.15.131:0/2879415020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b501a6390 con 0x7f6b50073a50 2024-05-10T05:27:05.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.952+0000 7f6b4cc00700 1 -- 172.21.15.131:0/2879415020 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b38011b90 con 0x7f6b50073a50 2024-05-10T05:27:05.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.952+0000 7f6b56400700 1 -- 172.21.15.131:0/2879415020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b501a6910 con 0x7f6b50073a50 2024-05-10T05:27:05.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.953+0000 7f6b56400700 1 -- 172.21.15.131:0/2879415020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b50102bc0 con 0x7f6b50073a50 2024-05-10T05:27:05.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.953+0000 7f6b4cc00700 1 -- 172.21.15.131:0/2879415020 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f6b38011cf0 con 0x7f6b50073a50 2024-05-10T05:27:05.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.954+0000 7f6b4cc00700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6b34038c70 0x7f6b3403b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:05.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.954+0000 7f6b4cc00700 1 -- 172.21.15.131:0/2879415020 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6b38049fe0 con 0x7f6b50073a50 2024-05-10T05:27:05.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.954+0000 7f6b4f400700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6b34038c70 0x7f6b3403b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:05.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.955+0000 7f6b4f400700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6b34038c70 0x7f6b3403b180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009fd0 tx=0x7f6b4001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:05.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:05.960+0000 7f6b4cc00700 1 -- 172.21.15.131:0/2879415020 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b38003bf0 con 0x7f6b50073a50 2024-05-10T05:27:06.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.176+0000 7f6b56400700 1 -- 172.21.15.131:0/2879415020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6b5010b860 con 0x7f6b50073a50 2024-05-10T05:27:06.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.176+0000 7f6b4cc00700 1 -- 172.21.15.131:0/2879415020 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f6b38016030 con 0x7f6b50073a50 2024-05-10T05:27:06.177 INFO:teuthology.orchestra.run.smithi131.stdout:# minimal ceph.conf for 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:27:06.177 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-05-10T05:27:06.178 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:27:06.178 INFO:teuthology.orchestra.run.smithi131.stdout: mon_host = [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-05-10T05:27:06.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 -- 172.21.15.131:0/2879415020 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6b34038c70 msgr2=0x7f6b3403b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:06.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6b34038c70 0x7f6b3403b180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009fd0 tx=0x7f6b4001a040 comp rx=0 tx=0).stop 2024-05-10T05:27:06.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 -- 172.21.15.131:0/2879415020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b50073a50 msgr2=0x7f6b50073e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:06.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b50073a50 0x7f6b50073e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800b780 tx=0x7f6b3800ba90 comp rx=0 tx=0).stop 2024-05-10T05:27:06.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 -- 172.21.15.131:0/2879415020 shutdown_connections 2024-05-10T05:27:06.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6b34038c70 0x7f6b3403b180 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:06.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b501a33f0 0x7f6b501a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:06.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b50107580 0x7f6b50073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:06.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 --2- 172.21.15.131:0/2879415020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b50073a50 0x7f6b50073e60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:06.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 -- 172.21.15.131:0/2879415020 >> 172.21.15.131:0/2879415020 conn(0x7f6b500fcfc0 msgr2=0x7f6b500fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:06.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 -- 172.21.15.131:0/2879415020 shutdown_connections 2024-05-10T05:27:06.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:06.179+0000 7f6b56400700 1 -- 172.21.15.131:0/2879415020 wait complete. 2024-05-10T05:27:06.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:05 smithi157 ceph-mon[78427]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T05:27:06.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:05 smithi157 ceph-mon[78427]: Reconfiguring daemon mon.a on smithi131 2024-05-10T05:27:06.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:05 smithi157 ceph-mon[78427]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:06.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:05 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:06.358 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:05 smithi131 ceph-mon[37264]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T05:27:06.358 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:05 smithi131 ceph-mon[37264]: Reconfiguring daemon mon.a on smithi131 2024-05-10T05:27:06.358 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:05 smithi131 ceph-mon[37264]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:06.359 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:05 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:05 smithi142 ceph-mon[77667]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T05:27:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:05 smithi142 ceph-mon[77667]: Reconfiguring daemon mon.a on smithi131 2024-05-10T05:27:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:05 smithi142 ceph-mon[77667]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:06.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:05 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:27:06.553 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-10T05:27:06.553 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:27:06.554 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T05:27:06.622 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:27:06.623 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T05:27:06.728 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:27:06.728 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T05:27:06.761 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:27:06.761 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T05:27:06.828 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:27:06.828 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T05:27:06.866 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:27:06.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T05:27:06.937 INFO:tasks.cephadm:Adding mgr.x on smithi131 2024-05-10T05:27:06.937 INFO:tasks.cephadm:Adding mgr.y on smithi142 2024-05-10T05:27:06.937 INFO:tasks.cephadm:Adding mgr.z on smithi157 2024-05-10T05:27:06.937 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi131=x;smithi142=y;smithi157=z' 2024-05-10T05:27:07.092 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:06 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2879415020' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:07.093 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:06 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:07.093 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:06 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:07.093 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:06 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:27:07.093 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:06 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T05:27:07.093 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:06 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:07.192 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:27:07.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:06 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2879415020' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:07.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:06 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:07.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:06 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:07.290 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:06 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:27:07.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:06 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T05:27:07.291 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:06 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:07.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:06 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2879415020' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:07.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:06 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:07.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:06 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:07.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:06 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:27:07.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:06 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T05:27:07.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:06 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:08.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:07 smithi157 ceph-mon[78427]: Reconfiguring mon.b (monmap changed)... 2024-05-10T05:27:08.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:07 smithi157 ceph-mon[78427]: Reconfiguring daemon mon.b on smithi142 2024-05-10T05:27:08.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:07 smithi157 ceph-mon[78427]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:08.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:08.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:08.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:27:08.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T05:27:08.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:08.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:07 smithi142 ceph-mon[77667]: Reconfiguring mon.b (monmap changed)... 2024-05-10T05:27:08.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:07 smithi142 ceph-mon[77667]: Reconfiguring daemon mon.b on smithi142 2024-05-10T05:27:08.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:07 smithi142 ceph-mon[77667]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:08.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:08.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:08.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:27:08.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T05:27:08.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.454+0000 7f47130f3700 1 -- 172.21.15.157:0/2868690591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f470c101bd0 msgr2=0x7f470c102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.454+0000 7f47130f3700 1 --2- 172.21.15.157:0/2868690591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f470c101bd0 0x7f470c102060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4700009910 tx=0x7f4700009c20 comp rx=0 tx=0).stop 2024-05-10T05:27:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.454+0000 7f47130f3700 1 -- 172.21.15.157:0/2868690591 shutdown_connections 2024-05-10T05:27:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.454+0000 7f47130f3700 1 --2- 172.21.15.157:0/2868690591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f470c101bd0 0x7f470c102060 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.454+0000 7f47130f3700 1 -- 172.21.15.157:0/2868690591 >> 172.21.15.157:0/2868690591 conn(0x7f470c0fb310 msgr2=0x7f470c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.454+0000 7f47130f3700 1 -- 172.21.15.157:0/2868690591 shutdown_connections 2024-05-10T05:27:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.454+0000 7f47130f3700 1 -- 172.21.15.157:0/2868690591 wait complete. 2024-05-10T05:27:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f47130f3700 1 Processor -- start 2024-05-10T05:27:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f47130f3700 1 -- start start 2024-05-10T05:27:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f47130f3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f470c101bd0 0x7f470c195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f47130f3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f470c196500 0x7f470c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f47130f3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f470c19b9d0 0x7f470c19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f47130f3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c19e330 con 0x7f470c101bd0 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f47130f3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c19e470 con 0x7f470c196500 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f47130f3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c19e5b0 con 0x7f470c19b9d0 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f4711690700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f470c19b9d0 0x7f470c19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f4711690700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f470c19b9d0 0x7f470c19ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36356/0 (socket says 172.21.15.157:36356) 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f4711690700 1 -- 172.21.15.157:0/3820812887 learned_addr learned my addr 172.21.15.157:0/3820812887 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f4710e8f700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f470c101bd0 0x7f470c195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f470bfff700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f470c196500 0x7f470c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f4711690700 1 -- 172.21.15.157:0/3820812887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f470c196500 msgr2=0x7f470c19b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f4711690700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f470c196500 0x7f470c19b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f4711690700 1 -- 172.21.15.157:0/3820812887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f470c101bd0 msgr2=0x7f470c195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.455+0000 7f4711690700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f470c101bd0 0x7f470c195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.456+0000 7f4711690700 1 -- 172.21.15.157:0/3820812887 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47000095f0 con 0x7f470c19b9d0 2024-05-10T05:27:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.456+0000 7f4711690700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f470c19b9d0 0x7f470c19ddf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f470400ba50 tx=0x7f470400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.456+0000 7f4709ffb700 1 -- 172.21.15.157:0/3820812887 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470400cab0 con 0x7f470c19b9d0 2024-05-10T05:27:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.456+0000 7f47130f3700 1 -- 172.21.15.157:0/3820812887 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c19e840 con 0x7f470c19b9d0 2024-05-10T05:27:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.456+0000 7f4709ffb700 1 -- 172.21.15.157:0/3820812887 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f470400cc10 con 0x7f470c19b9d0 2024-05-10T05:27:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.456+0000 7f4709ffb700 1 -- 172.21.15.157:0/3820812887 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040075b0 con 0x7f470c19b9d0 2024-05-10T05:27:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.456+0000 7f47130f3700 1 -- 172.21.15.157:0/3820812887 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c0ff950 con 0x7f470c19b9d0 2024-05-10T05:27:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.457+0000 7f47130f3700 1 -- 172.21.15.157:0/3820812887 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c062840 con 0x7f470c19b9d0 2024-05-10T05:27:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.457+0000 7f4709ffb700 1 -- 172.21.15.157:0/3820812887 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f4704014440 con 0x7f470c19b9d0 2024-05-10T05:27:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.457+0000 7f4709ffb700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f46fc03ae30 0x7f46fc03d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.458+0000 7f4709ffb700 1 -- 172.21.15.157:0/3820812887 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f470404ab20 con 0x7f470c19b9d0 2024-05-10T05:27:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.458+0000 7f4710e8f700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f46fc03ae30 0x7f46fc03d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.459+0000 7f4710e8f700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f46fc03ae30 0x7f46fc03d340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4700005260 tx=0x7f470001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:08.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:07 smithi131 ceph-mon[37264]: Reconfiguring mon.b (monmap changed)... 2024-05-10T05:27:08.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:07 smithi131 ceph-mon[37264]: Reconfiguring daemon mon.b on smithi142 2024-05-10T05:27:08.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:07 smithi131 ceph-mon[37264]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:08.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:08.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:08.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T05:27:08.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T05:27:08.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.463+0000 7f4709ffb700 1 -- 172.21.15.157:0/3820812887 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4704007710 con 0x7f470c19b9d0 2024-05-10T05:27:08.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.676+0000 7f47130f3700 1 -- 172.21.15.157:0/3820812887 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi142=y;smithi157=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f470c0fd0e0 con 0x7f46fc03ae30 2024-05-10T05:27:08.683 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.683+0000 7f4709ffb700 1 -- 172.21.15.157:0/3820812887 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f470c0fd0e0 con 0x7f46fc03ae30 2024-05-10T05:27:08.684 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2024-05-10T05:27:08.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.685+0000 7f47130f3700 1 -- 172.21.15.157:0/3820812887 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f46fc03ae30 msgr2=0x7f46fc03d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:08.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.685+0000 7f47130f3700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f46fc03ae30 0x7f46fc03d340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4700005260 tx=0x7f470001a040 comp rx=0 tx=0).stop 2024-05-10T05:27:08.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.685+0000 7f47130f3700 1 -- 172.21.15.157:0/3820812887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f470c19b9d0 msgr2=0x7f470c19ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:08.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.685+0000 7f47130f3700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f470c19b9d0 0x7f470c19ddf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f470400ba50 tx=0x7f470400be10 comp rx=0 tx=0).stop 2024-05-10T05:27:08.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.686+0000 7f47130f3700 1 -- 172.21.15.157:0/3820812887 shutdown_connections 2024-05-10T05:27:08.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.686+0000 7f47130f3700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f46fc03ae30 0x7f46fc03d340 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:08.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.686+0000 7f47130f3700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f470c101bd0 0x7f470c195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:08.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.686+0000 7f47130f3700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f470c19b9d0 0x7f470c19ddf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:08.687 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.686+0000 7f47130f3700 1 --2- 172.21.15.157:0/3820812887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f470c196500 0x7f470c19b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:08.687 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.686+0000 7f47130f3700 1 -- 172.21.15.157:0/3820812887 >> 172.21.15.157:0/3820812887 conn(0x7f470c0fb310 msgr2=0x7f470c0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:08.687 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.686+0000 7f47130f3700 1 -- 172.21.15.157:0/3820812887 shutdown_connections 2024-05-10T05:27:08.687 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:08.686+0000 7f47130f3700 1 -- 172.21.15.157:0/3820812887 wait complete. 2024-05-10T05:27:09.071 DEBUG:teuthology.orchestra.run.smithi142:mgr.y> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.y.service 2024-05-10T05:27:09.074 DEBUG:teuthology.orchestra.run.smithi157:mgr.z> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.z.service 2024-05-10T05:27:09.076 INFO:tasks.cephadm:Deploying OSDs... 2024-05-10T05:27:09.077 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:27:09.077 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T05:27:09.105 INFO:journalctl@ceph.mgr.z.smithi157.stdout:-- Logs begin at Fri 2024-05-10 05:00:45 UTC. -- 2024-05-10T05:27:09.108 INFO:journalctl@ceph.mgr.y.smithi142.stdout:-- Logs begin at Fri 2024-05-10 04:59:48 UTC. -- 2024-05-10T05:27:09.117 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T05:27:09.118 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-05-10T05:27:09.203 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T05:27:09.203 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:09.203 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 452 Links: 1 2024-05-10T05:27:09.203 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:09.203 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:09.203 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-10 05:26:25.957499868 +0000 2024-05-10T05:27:09.203 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-10 05:18:23.673029313 +0000 2024-05-10T05:27:09.203 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-10 05:18:23.673029313 +0000 2024-05-10T05:27:09.204 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-10 05:18:23.646029848 +0000 2024-05-10T05:27:09.204 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T05:27:09.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: Reconfiguring mon.c (monmap changed)... 2024-05-10T05:27:09.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: Reconfiguring daemon mon.c on smithi157 2024-05-10T05:27:09.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:09.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:09.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:09.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T05:27:09.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T05:27:09.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:09.304 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: Reconfiguring mon.c (monmap changed)... 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: Reconfiguring daemon mon.c on smithi157 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T05:27:09.305 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-10T05:27:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-10T05:27:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000312765 s, 1.6 MB/s 2024-05-10T05:27:09.315 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T05:27:09.359 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-05-10T05:27:09.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: Reconfiguring mon.c (monmap changed)... 2024-05-10T05:27:09.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: Reconfiguring daemon mon.c on smithi157 2024-05-10T05:27:09.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:09.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:09.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:09.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:09.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T05:27:09.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T05:27:09.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:09.445 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T05:27:09.446 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:09.446 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 458 Links: 1 2024-05-10T05:27:09.446 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:09.446 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:09.446 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-10 05:26:26.025498492 +0000 2024-05-10T05:27:09.446 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-10 05:18:24.202018849 +0000 2024-05-10T05:27:09.446 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-10 05:18:24.202018849 +0000 2024-05-10T05:27:09.446 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-10 05:18:24.179019304 +0000 2024-05-10T05:27:09.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T05:27:09.549 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-10T05:27:09.549 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-10T05:27:09.549 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000294173 s, 1.7 MB/s 2024-05-10T05:27:09.552 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T05:27:09.596 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-05-10T05:27:09.682 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T05:27:09.682 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:09.682 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 464 Links: 1 2024-05-10T05:27:09.682 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:09.682 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:09.682 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-10 05:26:26.091497156 +0000 2024-05-10T05:27:09.683 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-10 05:18:24.771007593 +0000 2024-05-10T05:27:09.683 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-10 05:18:24.771007593 +0000 2024-05-10T05:27:09.683 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-10 05:18:24.745008107 +0000 2024-05-10T05:27:09.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T05:27:09.785 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-10T05:27:09.785 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-10T05:27:09.785 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000285531 s, 1.8 MB/s 2024-05-10T05:27:09.788 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T05:27:09.831 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-05-10T05:27:09.914 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T05:27:09.914 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:09.914 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 470 Links: 1 2024-05-10T05:27:09.914 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:09.914 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:09.914 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-10 05:26:26.155495861 +0000 2024-05-10T05:27:09.914 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-10 05:18:25.305997010 +0000 2024-05-10T05:27:09.915 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-10 05:18:25.305997010 +0000 2024-05-10T05:27:09.915 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-10 05:18:25.281997485 +0000 2024-05-10T05:27:09.915 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T05:27:10.020 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-10T05:27:10.020 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-10T05:27:10.020 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000319242 s, 1.6 MB/s 2024-05-10T05:27:10.022 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T05:27:10.069 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T05:27:10.069 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T05:27:10.092 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T05:27:10.092 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-05-10T05:27:10.152 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T05:27:10.152 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:10.152 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 216145 Links: 1 2024-05-10T05:27:10.152 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:10.152 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:10.152 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-10 05:26:56.882408234 +0000 2024-05-10T05:27:10.152 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-10 05:17:28.933755090 +0000 2024-05-10T05:27:10.152 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-10 05:17:28.933755090 +0000 2024-05-10T05:27:10.152 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-05-10T05:27:10.153 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T05:27:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:09 smithi131 ceph-mon[37264]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi142=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:09 smithi131 ceph-mon[37264]: Saving service mgr spec with placement smithi131=x;smithi142=y;smithi157=z;count:3 2024-05-10T05:27:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:09 smithi131 ceph-mon[37264]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:09 smithi131 ceph-mon[37264]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T05:27:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:09 smithi131 ceph-mon[37264]: Reconfiguring daemon mgr.x on smithi131 2024-05-10T05:27:10.221 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-10T05:27:10.221 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-10T05:27:10.221 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000232966 s, 2.2 MB/s 2024-05-10T05:27:10.222 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T05:27:10.282 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-05-10T05:27:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:09 smithi157 ceph-mon[78427]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi142=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:09 smithi157 ceph-mon[78427]: Saving service mgr spec with placement smithi131=x;smithi142=y;smithi157=z;count:3 2024-05-10T05:27:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:09 smithi157 ceph-mon[78427]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:09 smithi157 ceph-mon[78427]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T05:27:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:09 smithi157 ceph-mon[78427]: Reconfiguring daemon mgr.x on smithi131 2024-05-10T05:27:10.341 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T05:27:10.341 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:10.341 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 210571 Links: 1 2024-05-10T05:27:10.341 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:10.341 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:10.341 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-10 05:26:56.882408234 +0000 2024-05-10T05:27:10.341 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-10 05:17:29.279748920 +0000 2024-05-10T05:27:10.341 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-10 05:17:29.279748920 +0000 2024-05-10T05:27:10.341 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-05-10T05:27:10.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T05:27:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:09 smithi142 ceph-mon[77667]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi131=x;smithi142=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:09 smithi142 ceph-mon[77667]: Saving service mgr spec with placement smithi131=x;smithi142=y;smithi157=z;count:3 2024-05-10T05:27:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:09 smithi142 ceph-mon[77667]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:09 smithi142 ceph-mon[77667]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T05:27:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:09 smithi142 ceph-mon[77667]: Reconfiguring daemon mgr.x on smithi131 2024-05-10T05:27:10.408 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-10T05:27:10.409 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-10T05:27:10.409 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00021929 s, 2.3 MB/s 2024-05-10T05:27:10.410 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T05:27:10.469 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-05-10T05:27:10.527 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T05:27:10.528 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:10.528 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 209864 Links: 1 2024-05-10T05:27:10.528 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:10.528 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:10.528 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-10 05:26:56.882408234 +0000 2024-05-10T05:27:10.529 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-10 05:17:29.623742785 +0000 2024-05-10T05:27:10.529 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-10 05:17:29.623742785 +0000 2024-05-10T05:27:10.529 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-05-10T05:27:10.529 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T05:27:10.593 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-10T05:27:10.593 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-10T05:27:10.594 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000237508 s, 2.2 MB/s 2024-05-10T05:27:10.595 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T05:27:10.653 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-05-10T05:27:10.710 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T05:27:10.711 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:10.711 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 213622 Links: 1 2024-05-10T05:27:10.711 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:10.711 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:10.711 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-10 05:26:56.882408234 +0000 2024-05-10T05:27:10.711 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-10 05:17:29.964736704 +0000 2024-05-10T05:27:10.711 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-10 05:17:29.964736704 +0000 2024-05-10T05:27:10.711 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-05-10T05:27:10.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T05:27:10.776 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-10T05:27:10.776 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-10T05:27:10.776 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000216053 s, 2.4 MB/s 2024-05-10T05:27:10.777 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T05:27:10.835 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-10T05:27:10.835 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T05:27:10.852 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T05:27:10.852 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-05-10T05:27:10.910 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T05:27:10.910 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:10.910 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 213068 Links: 1 2024-05-10T05:27:10.910 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:10.910 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:10.910 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-10 05:26:49.282832703 +0000 2024-05-10T05:27:10.910 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-10 05:17:05.615904766 +0000 2024-05-10T05:27:10.911 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-10 05:17:05.615904766 +0000 2024-05-10T05:27:10.911 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-05-10T05:27:10.911 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T05:27:10.978 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-10T05:27:10.978 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-10T05:27:10.978 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000119017 s, 4.3 MB/s 2024-05-10T05:27:10.979 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T05:27:11.038 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-05-10T05:27:11.106 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T05:27:11.107 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:11.107 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 211619 Links: 1 2024-05-10T05:27:11.107 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:11.107 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:11.107 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-10 05:26:49.282832703 +0000 2024-05-10T05:27:11.107 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-10 05:17:05.953898516 +0000 2024-05-10T05:27:11.109 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-10 05:17:05.953898516 +0000 2024-05-10T05:27:11.109 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-05-10T05:27:11.110 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T05:27:11.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-10T05:27:11.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-10T05:27:11.177 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.7828e-05 s, 5.2 MB/s 2024-05-10T05:27:11.178 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T05:27:11.244 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-05-10T05:27:11.302 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T05:27:11.303 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:11.303 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 213193 Links: 1 2024-05-10T05:27:11.303 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:11.303 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:11.303 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-10 05:27:11.258427081 +0000 2024-05-10T05:27:11.303 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-10 05:17:06.295892191 +0000 2024-05-10T05:27:11.304 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-10 05:17:06.295892191 +0000 2024-05-10T05:27:11.304 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-05-10T05:27:11.304 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T05:27:11.370 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:11 smithi157 systemd[1]: Starting Ceph mgr.z for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: Deploying daemon mgr.z on smithi157 2024-05-10T05:27:11.371 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mon[78427]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:11.373 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-10T05:27:11.373 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-10T05:27:11.373 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000104992 s, 4.9 MB/s 2024-05-10T05:27:11.375 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T05:27:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:11.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:11.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:11.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:11.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:11.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T05:27:11.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T05:27:11.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T05:27:11.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:11.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: Deploying daemon mgr.z on smithi157 2024-05-10T05:27:11.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:11 smithi142 ceph-mon[77667]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:11.434 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-05-10T05:27:11.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:11.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:11.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:11.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:11.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:11.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:11.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T05:27:11.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T05:27:11.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T05:27:11.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:11.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: Deploying daemon mgr.z on smithi157 2024-05-10T05:27:11.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:11 smithi131 ceph-mon[37264]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:11.491 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T05:27:11.492 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T05:27:11.492 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 213296 Links: 1 2024-05-10T05:27:11.492 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:27:11.492 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T05:27:11.492 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-10 05:27:11.258427081 +0000 2024-05-10T05:27:11.492 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-10 05:17:06.642885774 +0000 2024-05-10T05:27:11.492 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-10 05:17:06.642885774 +0000 2024-05-10T05:27:11.492 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2024-05-10T05:27:11.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T05:27:11.559 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-10T05:27:11.559 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-10T05:27:11.559 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000104005 s, 4.9 MB/s 2024-05-10T05:27:11.560 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T05:27:11.619 INFO:tasks.cephadm:Deploying osd.0 on smithi131 with /dev/vg_nvme/lv_4... 2024-05-10T05:27:11.620 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T05:27:11.979 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:27:12.053 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:11 smithi157 bash[80450]: a8a1ead4ea426363653deaf7891f7672c86d27807d21c2186b8ef148fb850ede 2024-05-10T05:27:12.053 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:11 smithi157 systemd[1]: Started Ceph mgr.z for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:27:12.053 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:11 smithi157 ceph-mgr[80571]: -- 172.21.15.157:0/1759004108 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e2e00b81a0 con 0x55e2e0115a80 2024-05-10T05:27:12.179 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:12.086+0000 7f35b5d7a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T05:27:12.350 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:12.178+0000 7f35b5d7a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T05:27:12.350 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:12.265+0000 7f35b5d7a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T05:27:12.350 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:12.349+0000 7f35b5d7a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:12 smithi142 ceph-mon[77667]: Deploying daemon mgr.y on smithi142 2024-05-10T05:27:13.053 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:12.984+0000 7f35b5d7a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T05:27:13.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:13.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:13.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:13.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T05:27:13.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T05:27:13.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T05:27:13.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:13.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:12 smithi157 ceph-mon[78427]: Deploying daemon mgr.y on smithi142 2024-05-10T05:27:13.144 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:13 smithi142 systemd[1]: Starting Ceph mgr.y for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:27:13.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:13.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:13.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:13.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T05:27:13.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T05:27:13.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T05:27:13.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:13.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:12 smithi131 ceph-mon[37264]: Deploying daemon mgr.y on smithi142 2024-05-10T05:27:13.553 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:13.236+0000 7f35b5d7a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T05:27:13.553 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:13.322+0000 7f35b5d7a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T05:27:13.553 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:13.431+0000 7f35b5d7a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T05:27:13.644 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:13 smithi142 ceph-mgr[79167]: -- 172.21.15.142:0/222930621 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2451021a0 con 0x55b24515f600 2024-05-10T05:27:13.644 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:13 smithi142 bash[79129]: c657d2af24d8e6060473ecc5cff04ce0e10c67ccd47df3bcb1f6b91e80680200 2024-05-10T05:27:13.644 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:13 smithi142 systemd[1]: Started Ceph mgr.y for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:27:13.749 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T05:27:13.749 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T05:27:13.749 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-10T05:27:13.750 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-10T05:27:13.750 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0293495 s, 357 MB/s 2024-05-10T05:27:13.750 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-10T05:27:13.908 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:13 smithi142 ceph-mon[77667]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:13.908 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:13.908 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:13.908 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:13.908 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:13.909 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:13 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:13.808+0000 7f05ba9d9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T05:27:14.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:13 smithi157 ceph-mon[78427]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:14.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.054 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:13.591+0000 7f35b5d7a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T05:27:14.054 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:13.717+0000 7f35b5d7a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T05:27:14.054 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:13.805+0000 7f35b5d7a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T05:27:14.095 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:13 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:13.908+0000 7f05ba9d9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T05:27:14.095 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:13 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:13.999+0000 7f05ba9d9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T05:27:14.125 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:13 smithi131 ceph-mon[37264]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:14.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.155 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2024-05-10T05:27:14.394 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:14 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:14.095+0000 7f05ba9d9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T05:27:14.505 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:27:14.553 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:14 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:14.217+0000 7f35b5d7a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T05:27:14.928 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:14 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:14.637+0000 7f35b5d7a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T05:27:14.928 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:14 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:14.834+0000 7f35b5d7a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T05:27:14.928 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:14 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:14.928+0000 7f35b5d7a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T05:27:14.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:14.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:14.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:14.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:14.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:14.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:14.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:15.036 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:14 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:14.779+0000 7f05ba9d9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T05:27:15.036 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:15.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:15.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:15.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:15.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:15.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:15.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:15.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:15.391 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:15 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:15.036+0000 7f05ba9d9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T05:27:15.392 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:15 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:15.123+0000 7f05ba9d9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T05:27:15.392 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:15 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:15.233+0000 7f05ba9d9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T05:27:15.392 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:15 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:15.392+0000 7f05ba9d9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T05:27:15.644 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:15 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:15.519+0000 7f05ba9d9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T05:27:15.644 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:15 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:15.604+0000 7f05ba9d9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T05:27:15.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.794+0000 7fbc0aa00700 1 -- 172.21.15.131:0/3178958884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04101ca0 msgr2=0x7fbc041020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:15.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.794+0000 7fbc0aa00700 1 --2- 172.21.15.131:0/3178958884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04101ca0 0x7fbc041020b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009990 tx=0x7fbbf4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:27:15.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.794+0000 7fbc0aa00700 1 -- 172.21.15.131:0/3178958884 shutdown_connections 2024-05-10T05:27:15.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.794+0000 7fbc0aa00700 1 --2- 172.21.15.131:0/3178958884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc04103770 0x7fbc0410a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:15.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.794+0000 7fbc0aa00700 1 --2- 172.21.15.131:0/3178958884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04101ca0 0x7fbc041020b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:15.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.794+0000 7fbc0aa00700 1 --2- 172.21.15.131:0/3178958884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc04102dd0 0x7fbc041031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:15.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.794+0000 7fbc0aa00700 1 -- 172.21.15.131:0/3178958884 >> 172.21.15.131:0/3178958884 conn(0x7fbc040fd2c0 msgr2=0x7fbc040ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:15.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.794+0000 7fbc0aa00700 1 -- 172.21.15.131:0/3178958884 shutdown_connections 2024-05-10T05:27:15.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.795+0000 7fbc0aa00700 1 -- 172.21.15.131:0/3178958884 wait complete. 2024-05-10T05:27:15.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.795+0000 7fbc0aa00700 1 Processor -- start 2024-05-10T05:27:15.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc0aa00700 1 -- start start 2024-05-10T05:27:15.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc0aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc04101ca0 0x7fbc0419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:15.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc0aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102dd0 0x7fbc0419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:15.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc0aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc04103770 0x7fbc0419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:15.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc0aa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0419bd00 con 0x7fbc04103770 2024-05-10T05:27:15.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc0aa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0419be40 con 0x7fbc04101ca0 2024-05-10T05:27:15.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc0aa00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0419c000 con 0x7fbc04102dd0 2024-05-10T05:27:15.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc03e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc04101ca0 0x7fbc0419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:15.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc03400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102dd0 0x7fbc0419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:15.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc03e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc04101ca0 0x7fbc0419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56184/0 (socket says 172.21.15.131:56184) 2024-05-10T05:27:15.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc03e00700 1 -- 172.21.15.131:0/1551169087 learned_addr learned my addr 172.21.15.131:0/1551169087 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:27:15.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.796+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc04103770 0x7fbc0419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:15.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc03400700 1 -- 172.21.15.131:0/1551169087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc04101ca0 msgr2=0x7fbc0419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:15.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc03400700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc04101ca0 0x7fbc0419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:15.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc03400700 1 -- 172.21.15.131:0/1551169087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc04103770 msgr2=0x7fbc0419b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:15.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc03400700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc04103770 0x7fbc0419b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:15.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc03400700 1 -- 172.21.15.131:0/1551169087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf40095f0 con 0x7fbc04102dd0 2024-05-10T05:27:15.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc03e00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc04101ca0 0x7fbc0419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:27:15.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc03400700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102dd0 0x7fbc0419a940 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000e9f0 tx=0x7fbbf000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:15.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc00c00700 1 -- 172.21.15.131:0/1551169087 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000c9c0 con 0x7fbc04102dd0 2024-05-10T05:27:15.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc00c00700 1 -- 172.21.15.131:0/1551169087 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf0010430 con 0x7fbc04102dd0 2024-05-10T05:27:15.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc0aa00700 1 -- 172.21.15.131:0/1551169087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0419c2e0 con 0x7fbc04102dd0 2024-05-10T05:27:15.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc00c00700 1 -- 172.21.15.131:0/1551169087 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0018680 con 0x7fbc04102dd0 2024-05-10T05:27:15.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.797+0000 7fbc0aa00700 1 -- 172.21.15.131:0/1551169087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc041a2ac0 con 0x7fbc04102dd0 2024-05-10T05:27:15.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.798+0000 7fbc0aa00700 1 -- 172.21.15.131:0/1551169087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc04100fa0 con 0x7fbc04102dd0 2024-05-10T05:27:15.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.798+0000 7fbc00c00700 1 -- 172.21.15.131:0/1551169087 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fbbf00187e0 con 0x7fbc04102dd0 2024-05-10T05:27:15.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.799+0000 7fbc00c00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80389d0 0x7fbbe803aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:15.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.799+0000 7fbc00c00700 1 -- 172.21.15.131:0/1551169087 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbbf0014070 con 0x7fbc04102dd0 2024-05-10T05:27:15.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.799+0000 7fbc03e00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80389d0 0x7fbbe803aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:15.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.800+0000 7fbc03e00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80389d0 0x7fbbe803aee0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400b2e0 tx=0x7fbbf401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:15.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:15.804+0000 7fbc00c00700 1 -- 172.21.15.131:0/1551169087 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbbf001fd50 con 0x7fbc04102dd0 2024-05-10T05:27:16.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:16.016+0000 7fbc0aa00700 1 -- 172.21.15.131:0/1551169087 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc04074b60 con 0x7fbbe80389d0 2024-05-10T05:27:16.053 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:15 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:15.693+0000 7f35b5d7a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T05:27:16.053 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:15 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:15.791+0000 7f35b5d7a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T05:27:16.053 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:15 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:15.941+0000 7f35b5d7a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T05:27:16.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:15 smithi157 ceph-mon[78427]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:16.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:16.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:16.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:16.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:16.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:16.144 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:15 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:15.976+0000 7f05ba9d9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T05:27:16.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:15 smithi142 ceph-mon[77667]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:16.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:16.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:16.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:16.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:16.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:16.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:15 smithi131 ceph-mon[37264]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:16.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:16.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:16.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:16.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:16.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:16.644 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:16.366+0000 7f05ba9d9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T05:27:16.644 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:16.561+0000 7f05ba9d9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T05:27:16.944 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:16 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:16.568+0000 7f35b5d7a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T05:27:16.945 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:16 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:16.749+0000 7f35b5d7a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T05:27:16.945 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:27:16.945 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:27:16.945 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:16.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:27:16.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:27:16.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:17.144 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:16.653+0000 7f05ba9d9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T05:27:17.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:27:17.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:27:17.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:17.303 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:16 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:16.944+0000 7f35b5d7a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T05:27:17.766 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:17 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:17.418+0000 7f05ba9d9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T05:27:17.766 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:17 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:17.515+0000 7f05ba9d9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T05:27:17.766 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:17 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:17.663+0000 7f05ba9d9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T05:27:18.053 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:17 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:17.872+0000 7f35b5d7a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T05:27:18.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:17 smithi157 ceph-mon[78427]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:18.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:17 smithi157 ceph-mon[78427]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:18.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:17 smithi142 ceph-mon[77667]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:18.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:17 smithi142 ceph-mon[77667]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:18.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:17 smithi131 ceph-mon[37264]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:18.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:17 smithi131 ceph-mon[37264]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:18.436 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:18 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:18.157+0000 7f35b5d7a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T05:27:18.436 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:18 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:18.245+0000 7f35b5d7a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T05:27:18.644 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:18.294+0000 7f05ba9d9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T05:27:18.644 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:18.476+0000 7f05ba9d9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T05:27:18.765 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:27:18 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z[80505]: 2024-05-10T05:27:18.436+0000 7f35b5d7a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T05:27:18.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:18.773+0000 7fbc00c00700 1 -- 172.21.15.131:0/1551169087 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mgrmap(e 14) v1 ==== 90708+0+0 (secure 0 0 0) 0x7fbbf0010860 con 0x7fbc04102dd0 2024-05-10T05:27:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:18 smithi157 ceph-mon[78427]: Standby manager daemon z started 2024-05-10T05:27:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:18 smithi157 ceph-mon[78427]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T05:27:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:18 smithi157 ceph-mon[78427]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T05:27:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:18 smithi157 ceph-mon[78427]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T05:27:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:18 smithi157 ceph-mon[78427]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T05:27:19.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:18 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2413482737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ad2b260-d7ae-4270-ab31-8855bf4b056a"}]: dispatch 2024-05-10T05:27:19.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:18 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ad2b260-d7ae-4270-ab31-8855bf4b056a"}]: dispatch 2024-05-10T05:27:19.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:18 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ad2b260-d7ae-4270-ab31-8855bf4b056a"}]': finished 2024-05-10T05:27:19.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:18 smithi157 ceph-mon[78427]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T05:27:19.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:18 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:19.144 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:18.670+0000 7f05ba9d9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T05:27:19.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:18 smithi142 ceph-mon[77667]: Standby manager daemon z started 2024-05-10T05:27:19.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:18 smithi142 ceph-mon[77667]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T05:27:19.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:18 smithi142 ceph-mon[77667]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T05:27:19.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:18 smithi142 ceph-mon[77667]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T05:27:19.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:18 smithi142 ceph-mon[77667]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T05:27:19.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:18 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2413482737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ad2b260-d7ae-4270-ab31-8855bf4b056a"}]: dispatch 2024-05-10T05:27:19.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:18 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ad2b260-d7ae-4270-ab31-8855bf4b056a"}]: dispatch 2024-05-10T05:27:19.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:18 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ad2b260-d7ae-4270-ab31-8855bf4b056a"}]': finished 2024-05-10T05:27:19.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:18 smithi142 ceph-mon[77667]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T05:27:19.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:18 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:19.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:18 smithi131 ceph-mon[37264]: Standby manager daemon z started 2024-05-10T05:27:19.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:18 smithi131 ceph-mon[37264]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T05:27:19.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:18 smithi131 ceph-mon[37264]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T05:27:19.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:18 smithi131 ceph-mon[37264]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T05:27:19.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:18 smithi131 ceph-mon[37264]: from='mgr.? 172.21.15.157:0/1019829809' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T05:27:19.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:18 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2413482737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ad2b260-d7ae-4270-ab31-8855bf4b056a"}]: dispatch 2024-05-10T05:27:19.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:18 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ad2b260-d7ae-4270-ab31-8855bf4b056a"}]: dispatch 2024-05-10T05:27:19.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:18 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ad2b260-d7ae-4270-ab31-8855bf4b056a"}]': finished 2024-05-10T05:27:19.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:18 smithi131 ceph-mon[37264]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T05:27:19.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:18 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:19.890 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:19 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:19.605+0000 7f05ba9d9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T05:27:19.890 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:19 smithi142 ceph-mon[77667]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:19.890 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:19 smithi142 ceph-mon[77667]: mgrmap e14: x(active, since 64s), standbys: z 2024-05-10T05:27:19.890 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:19 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T05:27:19.890 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:19 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4059873672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:27:20.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:19 smithi157 ceph-mon[78427]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:20.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:19 smithi157 ceph-mon[78427]: mgrmap e14: x(active, since 64s), standbys: z 2024-05-10T05:27:20.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:19 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T05:27:20.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:19 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4059873672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:27:20.144 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:19 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:19.889+0000 7f05ba9d9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T05:27:20.144 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:19 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:19.978+0000 7f05ba9d9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T05:27:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:19 smithi131 ceph-mon[37264]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:19 smithi131 ceph-mon[37264]: mgrmap e14: x(active, since 64s), standbys: z 2024-05-10T05:27:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:19 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T05:27:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:19 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4059873672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:27:20.644 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:27:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-y[79144]: 2024-05-10T05:27:20.169+0000 7f05ba9d9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T05:27:20.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:20.787+0000 7fbc00c00700 1 -- 172.21.15.131:0/1551169087 <== mon.1 v2:172.21.15.157:3300/0 8 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fbbf001d700 con 0x7fbc04102dd0 2024-05-10T05:27:21.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:20 smithi157 ceph-mon[78427]: Standby manager daemon y started 2024-05-10T05:27:21.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:20 smithi157 ceph-mon[78427]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T05:27:21.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:20 smithi157 ceph-mon[78427]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T05:27:21.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:20 smithi157 ceph-mon[78427]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T05:27:21.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:20 smithi157 ceph-mon[78427]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T05:27:21.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:20 smithi142 ceph-mon[77667]: Standby manager daemon y started 2024-05-10T05:27:21.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:20 smithi142 ceph-mon[77667]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T05:27:21.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:20 smithi142 ceph-mon[77667]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T05:27:21.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:20 smithi142 ceph-mon[77667]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T05:27:21.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:20 smithi142 ceph-mon[77667]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T05:27:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:20 smithi131 ceph-mon[37264]: Standby manager daemon y started 2024-05-10T05:27:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:20 smithi131 ceph-mon[37264]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T05:27:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:20 smithi131 ceph-mon[37264]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T05:27:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:20 smithi131 ceph-mon[37264]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T05:27:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:20 smithi131 ceph-mon[37264]: from='mgr.? 172.21.15.142:0/3970219311' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T05:27:22.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:21 smithi157 ceph-mon[78427]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:22.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:21 smithi157 ceph-mon[78427]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-05-10T05:27:22.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:21 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T05:27:22.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:21 smithi142 ceph-mon[77667]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:22.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:21 smithi142 ceph-mon[77667]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-05-10T05:27:22.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:21 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T05:27:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:21 smithi131 ceph-mon[37264]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:21 smithi131 ceph-mon[37264]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-05-10T05:27:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:21 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T05:27:24.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:23 smithi157 ceph-mon[78427]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:24.101 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:23 smithi131 ceph-mon[37264]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:24.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:23 smithi142 ceph-mon[77667]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:25.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:25 smithi131 ceph-mon[37264]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:25.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:25 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T05:27:25.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:25 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:25 smithi157 ceph-mon[78427]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:25 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T05:27:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:25 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:25 smithi142 ceph-mon[77667]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:25 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T05:27:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:25 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:26.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:26 smithi131 ceph-mon[37264]: Deploying daemon osd.0 on smithi131 2024-05-10T05:27:27.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:26 smithi157 ceph-mon[78427]: Deploying daemon osd.0 on smithi131 2024-05-10T05:27:27.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:26 smithi142 ceph-mon[77667]: Deploying daemon osd.0 on smithi131 2024-05-10T05:27:27.880 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:27 smithi131 ceph-mon[37264]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:28.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:27 smithi157 ceph-mon[78427]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:28.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:27 smithi142 ceph-mon[77667]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:30.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:29 smithi142 ceph-mon[77667]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:30.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:29 smithi131 ceph-mon[37264]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:29 smithi157 ceph-mon[78427]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:32.109 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:31 smithi131 ceph-mon[37264]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:32.109 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:32.109 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:32.109 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:32.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:31 smithi142 ceph-mon[77667]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:32.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:32.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:32.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:31 smithi157 ceph-mon[78427]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:34.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:33 smithi142 ceph-mon[77667]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:34.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:34.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:33 smithi131 ceph-mon[37264]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:33 smithi157 ceph-mon[78427]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:34.898 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:34.898 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:34.898 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:34.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.977+0000 7fbc00c00700 1 -- 172.21.15.131:0/1551169087 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbc04074b60 con 0x7fbbe80389d0 2024-05-10T05:27:34.978 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 0 on host 'smithi131' 2024-05-10T05:27:34.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 -- 172.21.15.131:0/1551169087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80389d0 msgr2=0x7fbbe803aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:34.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80389d0 0x7fbbe803aee0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400b2e0 tx=0x7fbbf401a040 comp rx=0 tx=0).stop 2024-05-10T05:27:34.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 -- 172.21.15.131:0/1551169087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102dd0 msgr2=0x7fbc0419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:34.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102dd0 0x7fbc0419a940 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000e9f0 tx=0x7fbbf000ed00 comp rx=0 tx=0).stop 2024-05-10T05:27:34.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 -- 172.21.15.131:0/1551169087 shutdown_connections 2024-05-10T05:27:34.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80389d0 0x7fbbe803aee0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:34.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc04103770 0x7fbc0419b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:34.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc04101ca0 0x7fbc0419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:34.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 --2- 172.21.15.131:0/1551169087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102dd0 0x7fbc0419a940 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:34.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 -- 172.21.15.131:0/1551169087 >> 172.21.15.131:0/1551169087 conn(0x7fbc040fd2c0 msgr2=0x7fbc04073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:34.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 -- 172.21.15.131:0/1551169087 shutdown_connections 2024-05-10T05:27:34.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:27:34.980+0000 7fbc0aa00700 1 -- 172.21.15.131:0/1551169087 wait complete. 2024-05-10T05:27:35.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:35.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:35.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:35.372 DEBUG:teuthology.orchestra.run.smithi131:osd.0> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.0.service 2024-05-10T05:27:35.376 INFO:tasks.cephadm:Deploying osd.1 on smithi142 with /dev/vg_nvme/lv_4... 2024-05-10T05:27:35.376 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T05:27:35.457 INFO:journalctl@ceph.osd.0.smithi131.stdout:-- Logs begin at Fri 2024-05-10 05:06:53 UTC. -- 2024-05-10T05:27:35.590 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:27:36.045 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:35 smithi142 ceph-mon[77667]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:36.046 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:35 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:36.046 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:35 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:36.046 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:35 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:36.046 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:35 smithi142 ceph-mon[77667]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T05:27:36.087 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:35 smithi131 ceph-mon[37264]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:36.087 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:36.087 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:36.087 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:36.087 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:35 smithi131 ceph-mon[37264]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T05:27:36.088 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:27:35 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-0[55073]: 2024-05-10T05:27:35.721+0000 7f2858af87c0 -1 osd.0 0 log_to_monitors true 2024-05-10T05:27:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:35 smithi157 ceph-mon[78427]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:35 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:35 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:35 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:35 smithi157 ceph-mon[78427]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T05:27:36.896 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T05:27:36.897 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T05:27:36.897 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-10T05:27:36.897 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-10T05:27:36.897 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112525 s, 932 MB/s 2024-05-10T05:27:36.897 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-10T05:27:37.316 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-05-10T05:27:37.533 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:27:37.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:37 smithi157 ceph-mon[78427]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T05:27:37.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:37 smithi157 ceph-mon[78427]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T05:27:37.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:37.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:37 smithi157 ceph-mon[78427]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:27:37.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:37 smithi157 ceph-mon[78427]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:37.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:37 smithi142 ceph-mon[77667]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T05:27:37.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:37 smithi142 ceph-mon[77667]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T05:27:37.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:37.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:37 smithi142 ceph-mon[77667]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:27:37.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:37 smithi142 ceph-mon[77667]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:37 smithi131 ceph-mon[37264]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T05:27:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:37 smithi131 ceph-mon[37264]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T05:27:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:37 smithi131 ceph-mon[37264]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:27:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:37 smithi131 ceph-mon[37264]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:38.210 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:27:37 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-0[55073]: 2024-05-10T05:27:37.933+0000 7f284d000700 -1 osd.0 0 waiting for initial osdmap 2024-05-10T05:27:38.210 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:27:37 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-0[55073]: 2024-05-10T05:27:37.951+0000 7f2842e00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:27:38.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.500+0000 7fac4a725700 1 -- 172.21.15.142:0/1167302889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac44101100 msgr2=0x7fac44101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:38.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.500+0000 7fac4a725700 1 --2- 172.21.15.142:0/1167302889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac44101100 0x7fac44101510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009990 tx=0x7fac2c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:27:38.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.500+0000 7fac4a725700 1 -- 172.21.15.142:0/1167302889 shutdown_connections 2024-05-10T05:27:38.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.500+0000 7fac4a725700 1 --2- 172.21.15.142:0/1167302889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac44102640 0x7fac441090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:38.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.500+0000 7fac4a725700 1 --2- 172.21.15.142:0/1167302889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac44101100 0x7fac44101510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:38.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.500+0000 7fac4a725700 1 --2- 172.21.15.142:0/1167302889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac44101ae0 0x7fac44101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:38.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.500+0000 7fac4a725700 1 -- 172.21.15.142:0/1167302889 >> 172.21.15.142:0/1167302889 conn(0x7fac440fb950 msgr2=0x7fac440fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:38.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.500+0000 7fac4a725700 1 -- 172.21.15.142:0/1167302889 shutdown_connections 2024-05-10T05:27:38.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.500+0000 7fac4a725700 1 -- 172.21.15.142:0/1167302889 wait complete. 2024-05-10T05:27:38.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac4a725700 1 Processor -- start 2024-05-10T05:27:38.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac4a725700 1 -- start start 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac4a725700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac44101100 0x7fac4419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac4a725700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac44101ae0 0x7fac4419d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac4a725700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac44102640 0x7fac4419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac4a725700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac44100cb0 con 0x7fac44102640 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac4a725700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4419fd80 con 0x7fac44101ae0 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac4a725700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4419fef0 con 0x7fac44101100 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac437fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac44101ae0 0x7fac4419d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac437fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac44101ae0 0x7fac4419d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:36964/0 (socket says 172.21.15.142:36964) 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac437fe700 1 -- 172.21.15.142:0/3691887151 learned_addr learned my addr 172.21.15.142:0/3691887151 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.501+0000 7fac43fff700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac44101100 0x7fac4419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:38.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac48cc2700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac44102640 0x7fac4419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:38.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac437fe700 1 -- 172.21.15.142:0/3691887151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac44101100 msgr2=0x7fac4419ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:38.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac437fe700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac44101100 0x7fac4419ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:38.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac437fe700 1 -- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac44102640 msgr2=0x7fac4419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:38.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac437fe700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac44102640 0x7fac4419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:38.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac437fe700 1 -- 172.21.15.142:0/3691887151 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c0095f0 con 0x7fac44101ae0 2024-05-10T05:27:38.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac48cc2700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac44102640 0x7fac4419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:27:38.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac437fe700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac44101ae0 0x7fac4419d2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fac3400c9b0 tx=0x7fac3400ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:38.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac417fa700 1 -- 172.21.15.142:0/3691887151 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34007b00 con 0x7fac44101ae0 2024-05-10T05:27:38.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac4a725700 1 -- 172.21.15.142:0/3691887151 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac441a01d0 con 0x7fac44101ae0 2024-05-10T05:27:38.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac417fa700 1 -- 172.21.15.142:0/3691887151 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac34007c60 con 0x7fac44101ae0 2024-05-10T05:27:38.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac417fa700 1 -- 172.21.15.142:0/3691887151 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34018660 con 0x7fac44101ae0 2024-05-10T05:27:38.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.502+0000 7fac4a725700 1 -- 172.21.15.142:0/3691887151 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac441a06d0 con 0x7fac44101ae0 2024-05-10T05:27:38.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.503+0000 7fac4a725700 1 -- 172.21.15.142:0/3691887151 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac44107f70 con 0x7fac44101ae0 2024-05-10T05:27:38.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.504+0000 7fac417fa700 1 -- 172.21.15.142:0/3691887151 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fac3400f450 con 0x7fac44101ae0 2024-05-10T05:27:38.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.505+0000 7fac417fa700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac300a1090 0x7fac300a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:38.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.505+0000 7fac417fa700 1 -- 172.21.15.142:0/3691887151 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7fac340cacc0 con 0x7fac44101ae0 2024-05-10T05:27:38.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.505+0000 7fac43fff700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac300a1090 0x7fac300a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:38.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.505+0000 7fac43fff700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac300a1090 0x7fac300a35a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac441006e0 tx=0x7fac2c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:38.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.510+0000 7fac417fa700 1 -- 172.21.15.142:0/3691887151 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fac34095fe0 con 0x7fac44101ae0 2024-05-10T05:27:38.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:38 smithi157 ceph-mon[78427]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:27:38.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:38 smithi157 ceph-mon[78427]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T05:27:38.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:38.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:38.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:38 smithi157 ceph-mon[78427]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' 2024-05-10T05:27:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:38 smithi142 ceph-mon[77667]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:27:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:38 smithi142 ceph-mon[77667]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T05:27:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:38 smithi142 ceph-mon[77667]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' 2024-05-10T05:27:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:38 smithi131 ceph-mon[37264]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:27:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:38 smithi131 ceph-mon[37264]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T05:27:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:38 smithi131 ceph-mon[37264]: from='osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550]' entity='osd.0' 2024-05-10T05:27:38.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:38.717+0000 7fac4a725700 1 -- 172.21.15.142:0/3691887151 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fac4410c5d0 con 0x7fac300a1090 2024-05-10T05:27:39.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi131 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi131 to 17269M 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:39.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:27:39.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:27:39.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:39.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:39.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] boot 2024-05-10T05:27:39.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T05:27:39.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:39.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:27:39.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:27:39.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:39.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi131 2024-05-10T05:27:39.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:27:39.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi131 to 17269M 2024-05-10T05:27:39.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.555 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:39.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:39.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:39.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:27:39.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:27:39.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:39.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:39.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] boot 2024-05-10T05:27:39.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T05:27:39.557 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:39.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:27:39.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:27:39.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:39.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi131 2024-05-10T05:27:39.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi131 to 17269M 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T05:27:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: osd.0 [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] boot 2024-05-10T05:27:39.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T05:27:39.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T05:27:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:41 smithi157 ceph-mon[78427]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T05:27:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:41 smithi157 ceph-mon[78427]: from='client.? 172.21.15.142:0/4148658716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "908f4583-6e05-4832-add9-714bf647e807"}]: dispatch 2024-05-10T05:27:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:41 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "908f4583-6e05-4832-add9-714bf647e807"}]: dispatch 2024-05-10T05:27:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:41 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "908f4583-6e05-4832-add9-714bf647e807"}]': finished 2024-05-10T05:27:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:41 smithi157 ceph-mon[78427]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T05:27:41.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:41 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:41.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:41 smithi157 ceph-mon[78427]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:41.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:41 smithi157 ceph-mon[78427]: from='client.? 172.21.15.142:0/2309251536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:27:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:41 smithi142 ceph-mon[77667]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T05:27:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:41 smithi142 ceph-mon[77667]: from='client.? 172.21.15.142:0/4148658716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "908f4583-6e05-4832-add9-714bf647e807"}]: dispatch 2024-05-10T05:27:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:41 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "908f4583-6e05-4832-add9-714bf647e807"}]: dispatch 2024-05-10T05:27:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:41 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "908f4583-6e05-4832-add9-714bf647e807"}]': finished 2024-05-10T05:27:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:41 smithi142 ceph-mon[77667]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T05:27:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:41 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:41 smithi142 ceph-mon[77667]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:41 smithi142 ceph-mon[77667]: from='client.? 172.21.15.142:0/2309251536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:27:41.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:41 smithi131 ceph-mon[37264]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T05:27:41.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:41 smithi131 ceph-mon[37264]: from='client.? 172.21.15.142:0/4148658716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "908f4583-6e05-4832-add9-714bf647e807"}]: dispatch 2024-05-10T05:27:41.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:41 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "908f4583-6e05-4832-add9-714bf647e807"}]: dispatch 2024-05-10T05:27:41.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:41 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "908f4583-6e05-4832-add9-714bf647e807"}]': finished 2024-05-10T05:27:41.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:41 smithi131 ceph-mon[37264]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T05:27:41.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:41 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:41.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:41 smithi131 ceph-mon[37264]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:41.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:41 smithi131 ceph-mon[37264]: from='client.? 172.21.15.142:0/2309251536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:27:44.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:43 smithi142 ceph-mon[77667]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:44.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:43 smithi131 ceph-mon[37264]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:43 smithi157 ceph-mon[78427]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:46.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:45 smithi142 ceph-mon[77667]: pgmap v45: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:46.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:45 smithi131 ceph-mon[37264]: pgmap v45: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:45 smithi157 ceph-mon[78427]: pgmap v45: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:47.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T05:27:47.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:47.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T05:27:47.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T05:27:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:47.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:47 smithi142 ceph-mon[77667]: Deploying daemon osd.1 on smithi142 2024-05-10T05:27:47.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:47 smithi142 ceph-mon[77667]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:47 smithi131 ceph-mon[37264]: Deploying daemon osd.1 on smithi142 2024-05-10T05:27:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:47 smithi131 ceph-mon[37264]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:47 smithi157 ceph-mon[78427]: Deploying daemon osd.1 on smithi142 2024-05-10T05:27:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:47 smithi157 ceph-mon[78427]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:50.134 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:49 smithi142 ceph-mon[77667]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:49 smithi131 ceph-mon[37264]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:49 smithi157 ceph-mon[78427]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:51.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:51.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:51.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:51.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:51 smithi142 ceph-mon[77667]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:51.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:51.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:51.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:51.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:51 smithi131 ceph-mon[37264]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:51.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:51.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:51.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:51.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:51 smithi157 ceph-mon[78427]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:52.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:52 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:52.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:52 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:52.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:52 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:52.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:52 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:52.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:52 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:52.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.526+0000 7fac417fa700 1 -- 172.21.15.142:0/3691887151 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fac4410c5d0 con 0x7fac300a1090 2024-05-10T05:27:52.528 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 1 on host 'smithi142' 2024-05-10T05:27:52.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 -- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac300a1090 msgr2=0x7fac300a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:52.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac300a1090 0x7fac300a35a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac441006e0 tx=0x7fac2c01a040 comp rx=0 tx=0).stop 2024-05-10T05:27:52.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 -- 172.21.15.142:0/3691887151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac44101ae0 msgr2=0x7fac4419d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:52.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac44101ae0 0x7fac4419d2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fac3400c9b0 tx=0x7fac3400ccc0 comp rx=0 tx=0).stop 2024-05-10T05:27:52.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 -- 172.21.15.142:0/3691887151 shutdown_connections 2024-05-10T05:27:52.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fac300a1090 0x7fac300a35a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:52.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac44102640 0x7fac4419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:52.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac44101100 0x7fac4419ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:52.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 --2- 172.21.15.142:0/3691887151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac44101ae0 0x7fac4419d2f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:52.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 -- 172.21.15.142:0/3691887151 >> 172.21.15.142:0/3691887151 conn(0x7fac440fb950 msgr2=0x7fac441050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:52.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 -- 172.21.15.142:0/3691887151 shutdown_connections 2024-05-10T05:27:52.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:27:52.529+0000 7fac4a725700 1 -- 172.21.15.142:0/3691887151 wait complete. 2024-05-10T05:27:52.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:52 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:52.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:52 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:52.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:52 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:52.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:52 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:52.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:52 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:52.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:52 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:52.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:52 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:52.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:52 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:52.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:52 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:52.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:52 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:52.939 DEBUG:teuthology.orchestra.run.smithi142:osd.1> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.1.service 2024-05-10T05:27:52.941 INFO:tasks.cephadm:Deploying osd.2 on smithi157 with /dev/vg_nvme/lv_4... 2024-05-10T05:27:52.941 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T05:27:52.972 INFO:journalctl@ceph.osd.1.smithi142.stdout:-- Logs begin at Fri 2024-05-10 04:59:48 UTC. -- 2024-05-10T05:27:53.168 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:27:53.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:53.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:53.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:53.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:53 smithi157 ceph-mon[78427]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:53.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:53.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:53.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:53.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:53 smithi142 ceph-mon[77667]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:53.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:53.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:53.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:27:53.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:53 smithi131 ceph-mon[37264]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:54.443 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T05:27:54.444 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T05:27:54.444 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-10T05:27:54.444 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-10T05:27:54.444 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00834531 s, 1.3 GB/s 2024-05-10T05:27:54.444 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-10T05:27:54.493 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:27:54 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-1[82760]: 2024-05-10T05:27:54.225+0000 7fec172bd7c0 -1 osd.1 0 log_to_monitors true 2024-05-10T05:27:54.745 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:54 smithi157 ceph-mon[78427]: from='osd.1 [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T05:27:54.746 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:54 smithi157 ceph-mon[78427]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T05:27:54.770 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:54 smithi142 ceph-mon[77667]: from='osd.1 [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T05:27:54.770 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:54 smithi142 ceph-mon[77667]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T05:27:54.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2024-05-10T05:27:54.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:54 smithi131 ceph-mon[37264]: from='osd.1 [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T05:27:54.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:54 smithi131 ceph-mon[37264]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T05:27:55.074 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:27:55.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T05:27:55.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T05:27:55.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:55.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='osd.1 [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:27:55.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:27:55.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:55.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:27:55.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:55.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:55.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='osd.1 [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:55.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T05:27:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T05:27:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='osd.1 [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:27:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:27:55.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:55.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:27:55.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:55.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:55.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:27:55.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.057+0000 7fb3f8b11700 1 -- 172.21.15.157:0/2680109068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4102650 msgr2=0x7fb3f4108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.057+0000 7fb3f8b11700 1 --2- 172.21.15.157:0/2680109068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4102650 0x7fb3f4108f00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00b0a0 tx=0x7fb3ec00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.057+0000 7fb3f8b11700 1 -- 172.21.15.157:0/2680109068 shutdown_connections 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.057+0000 7fb3f8b11700 1 --2- 172.21.15.157:0/2680109068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4102650 0x7fb3f4108f00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.057+0000 7fb3f8b11700 1 --2- 172.21.15.157:0/2680109068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3f4100b80 0x7fb3f4100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.057+0000 7fb3f8b11700 1 --2- 172.21.15.157:0/2680109068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4101cb0 0x7fb3f41020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.057+0000 7fb3f8b11700 1 -- 172.21.15.157:0/2680109068 >> 172.21.15.157:0/2680109068 conn(0x7fb3f40fc1c0 msgr2=0x7fb3f40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.057+0000 7fb3f8b11700 1 -- 172.21.15.157:0/2680109068 shutdown_connections 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.057+0000 7fb3f8b11700 1 -- 172.21.15.157:0/2680109068 wait complete. 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f8b11700 1 Processor -- start 2024-05-10T05:27:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f8b11700 1 -- start start 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f8b11700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100b80 0x7fb3f419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f8b11700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4101cb0 0x7fb3f419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f8b11700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3f4102650 0x7fb3f419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f8b11700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4069c70 con 0x7fb3f4100b80 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f8b11700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f419ff00 con 0x7fb3f4102650 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f8b11700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f41a0070 con 0x7fb3f4101cb0 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f1d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4101cb0 0x7fb3f419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f1d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4101cb0 0x7fb3f419d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52506/0 (socket says 172.21.15.157:52506) 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f1d9b700 1 -- 172.21.15.157:0/3242464920 learned_addr learned my addr 172.21.15.157:0/3242464920 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f2d9d700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3f4102650 0x7fb3f419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.058+0000 7fb3f259c700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100b80 0x7fb3f419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3f1d9b700 1 -- 172.21.15.157:0/3242464920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3f4102650 msgr2=0x7fb3f419f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3f1d9b700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3f4102650 0x7fb3f419f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3f1d9b700 1 -- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100b80 msgr2=0x7fb3f419af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:27:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3f1d9b700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100b80 0x7fb3f419af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:27:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3f1d9b700 1 -- 172.21.15.157:0/3242464920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ec009d00 con 0x7fb3f4101cb0 2024-05-10T05:27:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3f259c700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100b80 0x7fb3f419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:27:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3f1d9b700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4101cb0 0x7fb3f419d470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800b530 tx=0x7fb3e800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3e37fe700 1 -- 172.21.15.157:0/3242464920 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8011830 con 0x7fb3f4101cb0 2024-05-10T05:27:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3f8b11700 1 -- 172.21.15.157:0/3242464920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f41a0350 con 0x7fb3f4101cb0 2024-05-10T05:27:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3e37fe700 1 -- 172.21.15.157:0/3242464920 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e8011e50 con 0x7fb3f4101cb0 2024-05-10T05:27:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3e37fe700 1 -- 172.21.15.157:0/3242464920 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e800f2e0 con 0x7fb3f4101cb0 2024-05-10T05:27:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.059+0000 7fb3f8b11700 1 -- 172.21.15.157:0/3242464920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f41a08d0 con 0x7fb3f4101cb0 2024-05-10T05:27:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.060+0000 7fb3f8b11700 1 -- 172.21.15.157:0/3242464920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f4107da0 con 0x7fb3f4101cb0 2024-05-10T05:27:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.061+0000 7fb3e37fe700 1 -- 172.21.15.157:0/3242464920 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fb3e8010380 con 0x7fb3f4101cb0 2024-05-10T05:27:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.061+0000 7fb3e37fe700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb3dc0a1090 0x7fb3dc0a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:27:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.062+0000 7fb3e37fe700 1 -- 172.21.15.157:0/3242464920 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7fb3e80caac0 con 0x7fb3f4101cb0 2024-05-10T05:27:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.062+0000 7fb3f259c700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb3dc0a1090 0x7fb3dc0a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:27:56.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.062+0000 7fb3f259c700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb3dc0a1090 0x7fb3dc0a35a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40696a0 tx=0x7fb3e400a2b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:27:56.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.066+0000 7fb3e37fe700 1 -- 172.21.15.157:0/3242464920 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb3e8095bc0 con 0x7fb3f4101cb0 2024-05-10T05:27:56.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:27:56.264+0000 7fb3f8b11700 1 -- 172.21.15.157:0/3242464920 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3f410c670 con 0x7fb3dc0a1090 2024-05-10T05:27:56.394 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:27:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-1[82760]: 2024-05-10T05:27:55.973+0000 7fec0c921700 -1 osd.1 0 waiting for initial osdmap 2024-05-10T05:27:56.394 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:27:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-1[82760]: 2024-05-10T05:27:55.977+0000 7fec046d0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:27:56.774 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:56 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi142 2024-05-10T05:27:56.774 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:56 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi142 to 17113M 2024-05-10T05:27:56.774 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:56 smithi157 ceph-mon[78427]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:27:56.774 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:56 smithi157 ceph-mon[78427]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T05:27:56.774 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:56.774 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:56.775 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:56 smithi157 ceph-mon[78427]: from='osd.1 ' entity='osd.1' 2024-05-10T05:27:56.775 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:27:56.775 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:27:56.775 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:56.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:56 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi142 2024-05-10T05:27:56.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:56 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi142 to 17113M 2024-05-10T05:27:56.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:56 smithi142 ceph-mon[77667]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:27:56.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:56 smithi142 ceph-mon[77667]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T05:27:56.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:56.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:56.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:56 smithi142 ceph-mon[77667]: from='osd.1 ' entity='osd.1' 2024-05-10T05:27:56.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:27:56.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:27:56.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:56.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:56 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi142 2024-05-10T05:27:56.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:56 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi142 to 17113M 2024-05-10T05:27:56.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:56 smithi131 ceph-mon[37264]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:27:56.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:56 smithi131 ceph-mon[37264]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T05:27:56.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:56.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:56.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:56 smithi131 ceph-mon[37264]: from='osd.1 ' entity='osd.1' 2024-05-10T05:27:56.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:27:56.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:27:56.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:27:57.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:57 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:27:57.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:57 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:27:57.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:57 smithi157 ceph-mon[78427]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:57.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:57 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:57.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:57 smithi157 ceph-mon[78427]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:57.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:57 smithi157 ceph-mon[78427]: osd.1 [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] boot 2024-05-10T05:27:57.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:57 smithi157 ceph-mon[78427]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T05:27:57.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:57 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:57.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:57 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:27:57.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:57 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:27:57.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:57 smithi142 ceph-mon[77667]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:57.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:57.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:57 smithi142 ceph-mon[77667]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:57.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:57 smithi142 ceph-mon[77667]: osd.1 [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] boot 2024-05-10T05:27:57.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:57 smithi142 ceph-mon[77667]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T05:27:57.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:57 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:27:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:57 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:27:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:57 smithi131 ceph-mon[37264]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:27:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:57 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:57 smithi131 ceph-mon[37264]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-10T05:27:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:57 smithi131 ceph-mon[37264]: osd.1 [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] boot 2024-05-10T05:27:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:57 smithi131 ceph-mon[37264]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T05:27:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:57 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T05:27:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:58 smithi157 ceph-mon[78427]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T05:27:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.157:0/2587821362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2da9e21-2747-4101-8566-feb07d70e809"}]: dispatch 2024-05-10T05:27:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:58 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2da9e21-2747-4101-8566-feb07d70e809"}]: dispatch 2024-05-10T05:27:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:58 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2da9e21-2747-4101-8566-feb07d70e809"}]': finished 2024-05-10T05:27:59.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:58 smithi157 ceph-mon[78427]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T05:27:59.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:27:59.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.157:0/1939181635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:27:59.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:58 smithi142 ceph-mon[77667]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T05:27:59.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.157:0/2587821362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2da9e21-2747-4101-8566-feb07d70e809"}]: dispatch 2024-05-10T05:27:59.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:58 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2da9e21-2747-4101-8566-feb07d70e809"}]: dispatch 2024-05-10T05:27:59.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:58 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2da9e21-2747-4101-8566-feb07d70e809"}]': finished 2024-05-10T05:27:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:58 smithi142 ceph-mon[77667]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T05:27:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:27:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.157:0/1939181635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:27:59.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:58 smithi131 ceph-mon[37264]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T05:27:59.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.157:0/2587821362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2da9e21-2747-4101-8566-feb07d70e809"}]: dispatch 2024-05-10T05:27:59.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:58 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2da9e21-2747-4101-8566-feb07d70e809"}]: dispatch 2024-05-10T05:27:59.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:58 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2da9e21-2747-4101-8566-feb07d70e809"}]': finished 2024-05-10T05:27:59.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:58 smithi131 ceph-mon[37264]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T05:27:59.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:27:59.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.157:0/1939181635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:28:00.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:27:59 smithi157 ceph-mon[78427]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:27:59 smithi142 ceph-mon[77667]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:00.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:27:59 smithi131 ceph-mon[37264]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:02.136 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:01 smithi157 ceph-mon[78427]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:02.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:01 smithi142 ceph-mon[77667]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:02.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:01 smithi131 ceph-mon[37264]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:03 smithi157 ceph-mon[78427]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:04.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:03 smithi142 ceph-mon[77667]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:04.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:03 smithi131 ceph-mon[37264]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:05.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T05:28:05.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:05.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T05:28:05.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:05.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T05:28:05.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:06.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:06 smithi157 ceph-mon[78427]: Deploying daemon osd.2 on smithi157 2024-05-10T05:28:06.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:06 smithi157 ceph-mon[78427]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:06 smithi142 ceph-mon[77667]: Deploying daemon osd.2 on smithi157 2024-05-10T05:28:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:06 smithi142 ceph-mon[77667]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:06.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:06 smithi131 ceph-mon[37264]: Deploying daemon osd.2 on smithi157 2024-05-10T05:28:06.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:06 smithi131 ceph-mon[37264]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:07.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:07 smithi157 ceph-mon[78427]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:07 smithi142 ceph-mon[77667]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:07.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:07 smithi131 ceph-mon[37264]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:08.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:08.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:08.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:08 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:08.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:08.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:08.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:08 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:09.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:09.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:09.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:08 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:09.927 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:09 smithi157 ceph-mon[78427]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:09.927 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:09.927 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:09.927 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:09.927 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:09.927 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:09 smithi131 ceph-mon[37264]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:09.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:09.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:10.073 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 2 on host 'smithi157' 2024-05-10T05:28:10.073 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.072+0000 7fb3e37fe700 1 -- 172.21.15.157:0/3242464920 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb3f410c670 con 0x7fb3dc0a1090 2024-05-10T05:28:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 -- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb3dc0a1090 msgr2=0x7fb3dc0a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb3dc0a1090 0x7fb3dc0a35a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40696a0 tx=0x7fb3e400a2b0 comp rx=0 tx=0).stop 2024-05-10T05:28:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 -- 172.21.15.157:0/3242464920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4101cb0 msgr2=0x7fb3f419d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4101cb0 0x7fb3f419d470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800b530 tx=0x7fb3e800b840 comp rx=0 tx=0).stop 2024-05-10T05:28:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 -- 172.21.15.157:0/3242464920 shutdown_connections 2024-05-10T05:28:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb3dc0a1090 0x7fb3dc0a35a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:10.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f4100b80 0x7fb3f419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:10.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3f4102650 0x7fb3f419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:10.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 --2- 172.21.15.157:0/3242464920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f4101cb0 0x7fb3f419d470 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:10.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 -- 172.21.15.157:0/3242464920 >> 172.21.15.157:0/3242464920 conn(0x7fb3f40fc1c0 msgr2=0x7fb3f4104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:28:10.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 -- 172.21.15.157:0/3242464920 shutdown_connections 2024-05-10T05:28:10.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:10.075+0000 7fb3f8b11700 1 -- 172.21.15.157:0/3242464920 wait complete. 2024-05-10T05:28:10.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:09 smithi142 ceph-mon[77667]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:10.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:10.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:10.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:10.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:10.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:10.457 DEBUG:teuthology.orchestra.run.smithi157:osd.2> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.2.service 2024-05-10T05:28:10.459 INFO:tasks.cephadm:Deploying osd.3 on smithi131 with /dev/vg_nvme/lv_3... 2024-05-10T05:28:10.460 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T05:28:10.491 INFO:journalctl@ceph.osd.2.smithi157.stdout:-- Logs begin at Fri 2024-05-10 05:00:45 UTC. -- 2024-05-10T05:28:10.824 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:28:11.274 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:11.274 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:11.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:11.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:11 smithi157 ceph-mon[78427]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:11.340 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:11.340 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:11.340 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:11.340 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:11 smithi131 ceph-mon[37264]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:11 smithi142 ceph-mon[77667]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:12.023 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-2[84228]: 2024-05-10T05:28:11.715+0000 7fb1aa0737c0 -1 osd.2 0 log_to_monitors true 2024-05-10T05:28:12.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:12 smithi131 ceph-mon[37264]: from='osd.2 [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T05:28:12.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:12 smithi131 ceph-mon[37264]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T05:28:12.298 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:12 smithi157 ceph-mon[78427]: from='osd.2 [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T05:28:12.298 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:12 smithi157 ceph-mon[78427]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T05:28:12.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:12 smithi142 ceph-mon[77667]: from='osd.2 [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T05:28:12.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:12 smithi142 ceph-mon[77667]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T05:28:12.634 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T05:28:12.635 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T05:28:12.635 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-10T05:28:12.635 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-10T05:28:12.635 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0369669 s, 284 MB/s 2024-05-10T05:28:12.635 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-10T05:28:13.151 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2024-05-10T05:28:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T05:28:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T05:28:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='osd.2 [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:28:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:28:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi157 2024-05-10T05:28:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi157 to 17114M 2024-05-10T05:28:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:13 smithi157 ceph-mon[78427]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:13.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T05:28:13.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T05:28:13.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:13.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='osd.2 [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:28:13.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:28:13.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi157 2024-05-10T05:28:13.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:13.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi157 to 17114M 2024-05-10T05:28:13.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:13.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:13.397 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.397 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:13 smithi142 ceph-mon[77667]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:13.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T05:28:13.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T05:28:13.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:13.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='osd.2 [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:28:13.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:28:13.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi157 2024-05-10T05:28:13.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:13.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi157 to 17114M 2024-05-10T05:28:13.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:13.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:13.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:13.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:13 smithi131 ceph-mon[37264]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:13.504 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:28:13.803 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-2[84228]: 2024-05-10T05:28:13.498+0000 7fb19f6d7700 -1 osd.2 0 waiting for initial osdmap 2024-05-10T05:28:13.803 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-2[84228]: 2024-05-10T05:28:13.502+0000 7fb197486700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:28:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:14 smithi157 ceph-mon[78427]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:28:14.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:14 smithi157 ceph-mon[78427]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T05:28:14.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:14.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:14.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:14 smithi157 ceph-mon[78427]: from='osd.2 ' entity='osd.2' 2024-05-10T05:28:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:14 smithi142 ceph-mon[77667]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:28:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:14 smithi142 ceph-mon[77667]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T05:28:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:14.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:14.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:14 smithi142 ceph-mon[77667]: from='osd.2 ' entity='osd.2' 2024-05-10T05:28:14.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:14 smithi131 ceph-mon[37264]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:28:14.395 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:14 smithi131 ceph-mon[37264]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T05:28:14.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:14.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:14.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:14 smithi131 ceph-mon[37264]: from='osd.2 ' entity='osd.2' 2024-05-10T05:28:14.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.701+0000 7f711e400700 1 -- 172.21.15.131:0/74993743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7118104dc0 msgr2=0x7f71181072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:14.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.701+0000 7f711e400700 1 --2- 172.21.15.131:0/74993743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7118104dc0 0x7f71181072a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7100009990 tx=0x7f7100009ca0 comp rx=0 tx=0).stop 2024-05-10T05:28:14.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.702+0000 7f711e400700 1 -- 172.21.15.131:0/74993743 shutdown_connections 2024-05-10T05:28:14.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.702+0000 7f711e400700 1 --2- 172.21.15.131:0/74993743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71180699e0 0x7f71181047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:14.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.702+0000 7f711e400700 1 --2- 172.21.15.131:0/74993743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118107930 0x7f7118109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:14.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.702+0000 7f711e400700 1 --2- 172.21.15.131:0/74993743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7118104dc0 0x7f71181072a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:14.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.702+0000 7f711e400700 1 -- 172.21.15.131:0/74993743 >> 172.21.15.131:0/74993743 conn(0x7f71180fc1c0 msgr2=0x7f71180fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:28:14.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.702+0000 7f711e400700 1 -- 172.21.15.131:0/74993743 shutdown_connections 2024-05-10T05:28:14.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.702+0000 7f711e400700 1 -- 172.21.15.131:0/74993743 wait complete. 2024-05-10T05:28:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f711e400700 1 Processor -- start 2024-05-10T05:28:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f711e400700 1 -- start start 2024-05-10T05:28:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f711e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71180699e0 0x7f7118102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f711e400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104dc0 0x7f71181006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f711e400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7118107930 0x7f7118100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f711e400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7118101480 con 0x7f71180699e0 2024-05-10T05:28:14.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f711e400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71181a2120 con 0x7f7118104dc0 2024-05-10T05:28:14.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f711e400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71181a2260 con 0x7f7118107930 2024-05-10T05:28:14.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f7117400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71180699e0 0x7f7118102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:14.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f7117400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71180699e0 0x7f7118102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49472/0 (socket says 172.21.15.131:49472) 2024-05-10T05:28:14.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f7117400700 1 -- 172.21.15.131:0/1784378256 learned_addr learned my addr 172.21.15.131:0/1784378256 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:28:14.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.703+0000 7f7117e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7118107930 0x7f7118100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:14.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f710fe00700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104dc0 0x7f71181006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:14.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f7117e00700 1 -- 172.21.15.131:0/1784378256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104dc0 msgr2=0x7f71181006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:14.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f7117e00700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104dc0 0x7f71181006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:14.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f7117e00700 1 -- 172.21.15.131:0/1784378256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71180699e0 msgr2=0x7f7118102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:14.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f7117e00700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71180699e0 0x7f7118102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:14.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f7117e00700 1 -- 172.21.15.131:0/1784378256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7108009520 con 0x7f7118107930 2024-05-10T05:28:14.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f7117e00700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7118107930 0x7f7118100d50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f711000efc0 tx=0x7f711000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:28:14.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f7114c00700 1 -- 172.21.15.131:0/1784378256 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110018070 con 0x7f7118107930 2024-05-10T05:28:14.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f7114c00700 1 -- 172.21.15.131:0/1784378256 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f711000f040 con 0x7f7118107930 2024-05-10T05:28:14.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f711e400700 1 -- 172.21.15.131:0/1784378256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71000095f0 con 0x7f7118107930 2024-05-10T05:28:14.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.704+0000 7f7114c00700 1 -- 172.21.15.131:0/1784378256 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711000cd60 con 0x7f7118107930 2024-05-10T05:28:14.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.705+0000 7f711e400700 1 -- 172.21.15.131:0/1784378256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71181a27d0 con 0x7f7118107930 2024-05-10T05:28:14.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.706+0000 7f711e400700 1 -- 172.21.15.131:0/1784378256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71180fdd70 con 0x7f7118107930 2024-05-10T05:28:14.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.708+0000 7f7114c00700 1 -- 172.21.15.131:0/1784378256 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f7110003a00 con 0x7f7118107930 2024-05-10T05:28:14.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.709+0000 7f7114c00700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70fc0a1090 0x7f70fc0a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:14.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.709+0000 7f7114c00700 1 -- 172.21.15.131:0/1784378256 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f71100cadf0 con 0x7f7118107930 2024-05-10T05:28:14.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.709+0000 7f7117400700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70fc0a1090 0x7f70fc0a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:14.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.710+0000 7f7117400700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70fc0a1090 0x7f70fc0a35a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7118101f60 tx=0x7f7108009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:28:14.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.712+0000 7f7114c00700 1 -- 172.21.15.131:0/1784378256 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7110095be0 con 0x7f7118107930 2024-05-10T05:28:14.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:14.932+0000 7f711e400700 1 -- 172.21.15.131:0/1784378256 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f711810d770 con 0x7f70fc0a1090 2024-05-10T05:28:15.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:15 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:28:15.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:15 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:28:15.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:15.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:15 smithi142 ceph-mon[77667]: osd.2 [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] boot 2024-05-10T05:28:15.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:15 smithi142 ceph-mon[77667]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T05:28:15.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:15.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:15 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:28:15.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:15 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:28:15.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:15.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:15 smithi131 ceph-mon[37264]: osd.2 [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] boot 2024-05-10T05:28:15.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:15 smithi131 ceph-mon[37264]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T05:28:15.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:15 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:28:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:15 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:28:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:15 smithi157 ceph-mon[78427]: osd.2 [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] boot 2024-05-10T05:28:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:15 smithi157 ceph-mon[78427]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T05:28:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T05:28:16.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:16 smithi142 ceph-mon[77667]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:16.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T05:28:16.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:16 smithi142 ceph-mon[77667]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:28:16.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:28:16.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:28:16.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:16.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T05:28:16.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:16 smithi142 ceph-mon[77667]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T05:28:16.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T05:28:16.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:16 smithi131 ceph-mon[37264]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:16.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T05:28:16.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:16 smithi131 ceph-mon[37264]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:28:16.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:28:16.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:28:16.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:16.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T05:28:16.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:16 smithi131 ceph-mon[37264]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T05:28:16.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T05:28:16.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:16 smithi157 ceph-mon[78427]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-10T05:28:16.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T05:28:16.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:16 smithi157 ceph-mon[78427]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:28:16.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:28:16.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:28:16.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:16.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T05:28:16.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:16 smithi157 ceph-mon[78427]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T05:28:16.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T05:28:17.460 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:28:17 smithi131 sudo[61395]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T05:28:17.460 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:28:17 smithi131 sudo[61395]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:17.460 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:28:17 smithi131 sudo[61395]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:17.460 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:28:17 smithi131 sudo[61395]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:17.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:17.527+0000 7f7114c00700 1 -- 172.21.15.131:0/1784378256 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7110095fb0 con 0x7f7118107930 2024-05-10T05:28:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:17 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T05:28:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:17 smithi157 ceph-mon[78427]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T05:28:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:17 smithi157 ceph-mon[78427]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:17.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:17 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T05:28:17.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:17 smithi142 ceph-mon[77667]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T05:28:17.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:17 smithi142 ceph-mon[77667]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:17.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:17 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T05:28:17.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:17 smithi131 ceph-mon[37264]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T05:28:17.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:17 smithi131 ceph-mon[37264]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:17.960 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:28:17 smithi131 sudo[61418]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T05:28:17.961 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:28:17 smithi131 sudo[61418]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:17.961 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:28:17 smithi131 sudo[61418]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:17.961 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:28:17 smithi131 sudo[61418]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:18.275 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:28:18 smithi142 sudo[88254]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T05:28:18.275 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:28:18 smithi142 sudo[88254]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:18.275 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:28:18 smithi142 sudo[88254]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:18.275 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:28:18 smithi142 sudo[88254]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:18.275 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:28:18 smithi142 sudo[88257]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T05:28:18.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:18 smithi142 ceph-mon[77667]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T05:28:18.525 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:18 smithi142 ceph-mon[77667]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T05:28:18.525 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:28:18 smithi142 sudo[88257]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:18.525 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:28:18 smithi142 sudo[88257]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:18.525 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:28:18 smithi142 sudo[88257]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:18.786 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:18 smithi157 ceph-mon[78427]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T05:28:18.786 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:18 smithi157 ceph-mon[78427]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T05:28:18.786 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:18 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:18.786 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:18 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2931669675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "665394bb-71dc-422c-bebc-5b0557311dec"}]: dispatch 2024-05-10T05:28:18.786 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:18 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "665394bb-71dc-422c-bebc-5b0557311dec"}]: dispatch 2024-05-10T05:28:18.786 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:18 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "665394bb-71dc-422c-bebc-5b0557311dec"}]': finished 2024-05-10T05:28:18.787 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:18 smithi157 ceph-mon[78427]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T05:28:18.787 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:18 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:18.787 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:18 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:18.787 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:18 smithi157 sudo[88919]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T05:28:18.787 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:18 smithi157 sudo[88919]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:18.787 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:18 smithi157 sudo[88919]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:18.787 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:18 smithi157 sudo[88919]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:18.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:18 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:18.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:18 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2931669675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "665394bb-71dc-422c-bebc-5b0557311dec"}]: dispatch 2024-05-10T05:28:18.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:18 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "665394bb-71dc-422c-bebc-5b0557311dec"}]: dispatch 2024-05-10T05:28:18.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:18 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "665394bb-71dc-422c-bebc-5b0557311dec"}]': finished 2024-05-10T05:28:18.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:18 smithi142 ceph-mon[77667]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T05:28:18.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:18 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:18.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:18 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:18 smithi131 ceph-mon[37264]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T05:28:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:18 smithi131 ceph-mon[37264]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T05:28:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:18 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:18 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2931669675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "665394bb-71dc-422c-bebc-5b0557311dec"}]: dispatch 2024-05-10T05:28:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:18 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "665394bb-71dc-422c-bebc-5b0557311dec"}]: dispatch 2024-05-10T05:28:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:18 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "665394bb-71dc-422c-bebc-5b0557311dec"}]': finished 2024-05-10T05:28:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:18 smithi131 ceph-mon[37264]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T05:28:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:18 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:18.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:18 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:19.053 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:18 smithi157 sudo[88922]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T05:28:19.053 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:18 smithi157 sudo[88922]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:19.053 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:18 smithi157 sudo[88922]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:19.053 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:28:18 smithi157 sudo[88922]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:19.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:19 smithi131 sudo[61477]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T05:28:19.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:19 smithi131 sudo[61477]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:19.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:19 smithi131 sudo[61477]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:19.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:19 smithi131 sudo[61477]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:20.152 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:19 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:20.153 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:19 smithi157 ceph-mon[78427]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T05:28:20.153 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 sudo[88925]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T05:28:20.153 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 sudo[88925]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:20.153 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 sudo[88925]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:20.153 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 sudo[88925]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:19 smithi131 sudo[61480]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T05:28:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:19 smithi131 sudo[61480]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:19 smithi131 sudo[61480]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:19 smithi131 sudo[61480]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:19 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:19 smithi131 ceph-mon[37264]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T05:28:20.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:19 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:20.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:19 smithi142 ceph-mon[77667]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T05:28:20.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 sudo[88928]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T05:28:20.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 sudo[88928]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:20.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 sudo[88928]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:20.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 sudo[88928]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:20.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 sudo[88260]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T05:28:20.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 sudo[88260]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:20.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 sudo[88260]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:20.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 sudo[88260]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:20.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 sudo[88263]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T05:28:20.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 sudo[88263]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T05:28:20.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 sudo[88263]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T05:28:20.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 sudo[88263]: pam_unix(sudo:session): session closed for user root 2024-05-10T05:28:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: pgmap v74: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T05:28:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:28:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1626008814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:28:21.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:28:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:20 smithi131 ceph-mon[37264]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T05:28:21.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: pgmap v74: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:21.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T05:28:21.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:28:21.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1626008814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:28:21.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:28:21.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:28:21.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T05:28:21.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:28:21.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:28:21.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:28:21.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T05:28:21.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T05:28:21.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:28:21.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:28:21.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:28:21.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:20 smithi157 ceph-mon[78427]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T05:28:21.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: pgmap v74: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:21.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1626008814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T05:28:21.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T05:28:21.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:20 smithi142 ceph-mon[77667]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T05:28:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:21 smithi131 ceph-mon[37264]: pgmap v75: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:21 smithi157 ceph-mon[78427]: pgmap v75: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:21 smithi142 ceph-mon[77667]: pgmap v75: 1 pgs: 1 active+clean; 449 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:24.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:23 smithi131 ceph-mon[37264]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 6.8 KiB/s rd, 348 KiB/s wr, 23 op/s 2024-05-10T05:28:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:23 smithi157 ceph-mon[78427]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 6.8 KiB/s rd, 348 KiB/s wr, 23 op/s 2024-05-10T05:28:24.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:23 smithi142 ceph-mon[77667]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 6.8 KiB/s rd, 348 KiB/s wr, 23 op/s 2024-05-10T05:28:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:25 smithi131 ceph-mon[37264]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 252 KiB/s wr, 16 op/s 2024-05-10T05:28:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:25 smithi157 ceph-mon[78427]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 252 KiB/s wr, 16 op/s 2024-05-10T05:28:26.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:25 smithi142 ceph-mon[77667]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 252 KiB/s wr, 16 op/s 2024-05-10T05:28:27.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:26 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T05:28:27.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:26 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:27.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:26 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T05:28:27.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:26 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:27.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:26 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T05:28:27.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:26 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:28.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:27 smithi131 ceph-mon[37264]: Deploying daemon osd.3 on smithi131 2024-05-10T05:28:28.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:27 smithi131 ceph-mon[37264]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 197 KiB/s wr, 13 op/s 2024-05-10T05:28:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:27 smithi157 ceph-mon[78427]: Deploying daemon osd.3 on smithi131 2024-05-10T05:28:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:27 smithi157 ceph-mon[78427]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 197 KiB/s wr, 13 op/s 2024-05-10T05:28:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:27 smithi142 ceph-mon[77667]: Deploying daemon osd.3 on smithi131 2024-05-10T05:28:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:27 smithi142 ceph-mon[77667]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 197 KiB/s wr, 13 op/s 2024-05-10T05:28:30.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:29 smithi131 ceph-mon[37264]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 170 KiB/s wr, 11 op/s 2024-05-10T05:28:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:29 smithi157 ceph-mon[78427]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 170 KiB/s wr, 11 op/s 2024-05-10T05:28:30.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:29 smithi142 ceph-mon[77667]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 170 KiB/s wr, 11 op/s 2024-05-10T05:28:32.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:31 smithi131 ceph-mon[37264]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-10T05:28:32.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:32.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:32.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:31 smithi157 ceph-mon[78427]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-10T05:28:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:31 smithi142 ceph-mon[77667]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-10T05:28:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:33 smithi131 ceph-mon[37264]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-10T05:28:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:33 smithi157 ceph-mon[78427]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-10T05:28:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:33 smithi142 ceph-mon[77667]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-10T05:28:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:34.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:34.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:35.357 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 3 on host 'smithi131' 2024-05-10T05:28:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.355+0000 7f7114c00700 1 -- 172.21.15.131:0/1784378256 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f711810d770 con 0x7f70fc0a1090 2024-05-10T05:28:35.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.358+0000 7f711e400700 1 -- 172.21.15.131:0/1784378256 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70fc0a1090 msgr2=0x7f70fc0a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:35.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.358+0000 7f711e400700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70fc0a1090 0x7f70fc0a35a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7118101f60 tx=0x7f7108009260 comp rx=0 tx=0).stop 2024-05-10T05:28:35.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.358+0000 7f711e400700 1 -- 172.21.15.131:0/1784378256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7118107930 msgr2=0x7f7118100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:35.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.358+0000 7f711e400700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7118107930 0x7f7118100d50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f711000efc0 tx=0x7f711000c3a0 comp rx=0 tx=0).stop 2024-05-10T05:28:35.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.359+0000 7f711e400700 1 -- 172.21.15.131:0/1784378256 shutdown_connections 2024-05-10T05:28:35.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.359+0000 7f711e400700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70fc0a1090 0x7f70fc0a35a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:35.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.359+0000 7f711e400700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71180699e0 0x7f7118102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:35.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.359+0000 7f711e400700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7118107930 0x7f7118100d50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:35.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.359+0000 7f711e400700 1 --2- 172.21.15.131:0/1784378256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104dc0 0x7f71181006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:35.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.359+0000 7f711e400700 1 -- 172.21.15.131:0/1784378256 >> 172.21.15.131:0/1784378256 conn(0x7f71180fc1c0 msgr2=0x7f71181043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:28:35.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.359+0000 7f711e400700 1 -- 172.21.15.131:0/1784378256 shutdown_connections 2024-05-10T05:28:35.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:28:35.359+0000 7f711e400700 1 -- 172.21.15.131:0/1784378256 wait complete. 2024-05-10T05:28:35.766 DEBUG:teuthology.orchestra.run.smithi131:osd.3> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.3.service 2024-05-10T05:28:35.771 INFO:tasks.cephadm:Deploying osd.4 on smithi142 with /dev/vg_nvme/lv_3... 2024-05-10T05:28:35.771 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T05:28:35.878 INFO:journalctl@ceph.osd.3.smithi131.stdout:-- Logs begin at Fri 2024-05-10 05:06:53 UTC. -- 2024-05-10T05:28:36.016 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:28:36.127 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:35 smithi142 ceph-mon[77667]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:36.127 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:35 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:36.128 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:35 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:36.128 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:35 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:36.128 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:35 smithi142 ceph-mon[77667]: from='osd.3 [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T05:28:36.128 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:35 smithi142 ceph-mon[77667]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T05:28:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:35 smithi131 ceph-mon[37264]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:35 smithi131 ceph-mon[37264]: from='osd.3 [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T05:28:36.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:35 smithi131 ceph-mon[37264]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T05:28:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:35 smithi157 ceph-mon[78427]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:35 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:35 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:35 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:35 smithi157 ceph-mon[78427]: from='osd.3 [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T05:28:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:35 smithi157 ceph-mon[78427]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T05:28:37.319 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T05:28:37.319 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T05:28:37.319 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-10T05:28:37.319 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-10T05:28:37.320 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129421 s, 810 MB/s 2024-05-10T05:28:37.320 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-10T05:28:37.605 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:37 smithi142 ceph-mon[77667]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T05:28:37.605 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:37 smithi142 ceph-mon[77667]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T05:28:37.606 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:37.606 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:37 smithi142 ceph-mon[77667]: from='osd.3 [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:28:37.606 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:37 smithi142 ceph-mon[77667]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:28:37.606 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:37 smithi142 ceph-mon[77667]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:37.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-05-10T05:28:37.711 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 10 05:28:37 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-3[63713]: 2024-05-10T05:28:37.346+0000 7f32e8600700 -1 osd.3 0 waiting for initial osdmap 2024-05-10T05:28:37.711 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 10 05:28:37 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-3[63713]: 2024-05-10T05:28:37.354+0000 7f32e3400700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:28:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:37 smithi131 ceph-mon[37264]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T05:28:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:37 smithi131 ceph-mon[37264]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T05:28:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:37 smithi131 ceph-mon[37264]: from='osd.3 [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:28:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:37 smithi131 ceph-mon[37264]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:28:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:37 smithi131 ceph-mon[37264]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:37 smithi157 ceph-mon[78427]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T05:28:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:37 smithi157 ceph-mon[78427]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T05:28:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:37 smithi157 ceph-mon[78427]: from='osd.3 [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:28:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:37 smithi157 ceph-mon[78427]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:28:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:37 smithi157 ceph-mon[78427]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-10T05:28:37.871 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:28:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:38 smithi142 ceph-mon[77667]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:28:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:38 smithi142 ceph-mon[77667]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T05:28:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:38 smithi131 ceph-mon[37264]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:28:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:38 smithi131 ceph-mon[37264]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T05:28:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:38 smithi157 ceph-mon[78427]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:28:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:38 smithi157 ceph-mon[78427]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T05:28:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:38.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.890+0000 7fbec2dde700 1 -- 172.21.15.142:0/3233329631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc103230 msgr2=0x7fbebc103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:38.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.890+0000 7fbec2dde700 1 --2- 172.21.15.142:0/3233329631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc103230 0x7fbebc103640 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009990 tx=0x7fbeb0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:28:38.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.890+0000 7fbec2dde700 1 -- 172.21.15.142:0/3233329631 shutdown_connections 2024-05-10T05:28:38.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.890+0000 7fbec2dde700 1 --2- 172.21.15.142:0/3233329631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc1005d0 0x7fbebc10a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:38.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.890+0000 7fbec2dde700 1 --2- 172.21.15.142:0/3233329631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc103230 0x7fbebc103640 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:38.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.890+0000 7fbec2dde700 1 --2- 172.21.15.142:0/3233329631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc0ffc80 0x7fbebc100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:38.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.890+0000 7fbec2dde700 1 -- 172.21.15.142:0/3233329631 >> 172.21.15.142:0/3233329631 conn(0x7fbebc0fb950 msgr2=0x7fbebc0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:28:38.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.890+0000 7fbec2dde700 1 -- 172.21.15.142:0/3233329631 shutdown_connections 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.890+0000 7fbec2dde700 1 -- 172.21.15.142:0/3233329631 wait complete. 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbec2dde700 1 Processor -- start 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbec2dde700 1 -- start start 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbec2dde700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc0ffc80 0x7fbebc198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbec2dde700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc1005d0 0x7fbebc198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbec2dde700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc103230 0x7fbebc199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbec2dde700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc199a80 con 0x7fbebc0ffc80 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbec2dde700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc199bc0 con 0x7fbebc1005d0 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbec2dde700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc199d80 con 0x7fbebc103230 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc1005d0 0x7fbebc198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc1005d0 0x7fbebc198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35374/0 (socket says 172.21.15.142:35374) 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.891+0000 7fbebbfff700 1 -- 172.21.15.142:0/2851185982 learned_addr learned my addr 172.21.15.142:0/2851185982 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T05:28:38.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbec0b7a700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc0ffc80 0x7fbebc198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbec137b700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc103230 0x7fbebc199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbebbfff700 1 -- 172.21.15.142:0/2851185982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc103230 msgr2=0x7fbebc199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbebbfff700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc103230 0x7fbebc199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbebbfff700 1 -- 172.21.15.142:0/2851185982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc0ffc80 msgr2=0x7fbebc198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbebbfff700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc0ffc80 0x7fbebc198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbebbfff700 1 -- 172.21.15.142:0/2851185982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb00095f0 con 0x7fbebc1005d0 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbebbfff700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc1005d0 0x7fbebc198670 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbea800d950 tx=0x7fbea800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbeb9ffb700 1 -- 172.21.15.142:0/2851185982 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8009940 con 0x7fbebc1005d0 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbeb9ffb700 1 -- 172.21.15.142:0/2851185982 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbea8010460 con 0x7fbebc1005d0 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbec2dde700 1 -- 172.21.15.142:0/2851185982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc19a060 con 0x7fbebc1005d0 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbeb9ffb700 1 -- 172.21.15.142:0/2851185982 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800f620 con 0x7fbebc1005d0 2024-05-10T05:28:38.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.892+0000 7fbec2dde700 1 -- 172.21.15.142:0/2851185982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc1a0840 con 0x7fbebc1005d0 2024-05-10T05:28:38.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.893+0000 7fbec2dde700 1 -- 172.21.15.142:0/2851185982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbebc109090 con 0x7fbebc1005d0 2024-05-10T05:28:38.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.894+0000 7fbeb9ffb700 1 -- 172.21.15.142:0/2851185982 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbea80108d0 con 0x7fbebc1005d0 2024-05-10T05:28:38.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.895+0000 7fbeb9ffb700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbeac0a11a0 0x7fbeac0a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:38.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.895+0000 7fbeb9ffb700 1 -- 172.21.15.142:0/2851185982 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fbea80cb4d0 con 0x7fbebc1005d0 2024-05-10T05:28:38.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.895+0000 7fbec0b7a700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbeac0a11a0 0x7fbeac0a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:38.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.896+0000 7fbec0b7a700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbeac0a11a0 0x7fbeac0a36b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000b5c0 tx=0x7fbeb0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:28:38.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:38.900+0000 7fbeb9ffb700 1 -- 172.21.15.142:0/2851185982 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbea8095e60 con 0x7fbebc1005d0 2024-05-10T05:28:39.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:39.099+0000 7fbec2dde700 1 -- 172.21.15.142:0/2851185982 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbebc10d620 con 0x7fbeac0a11a0 2024-05-10T05:28:39.615 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:28:39.615 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:28:39.615 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: osd.3 [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] boot 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 122 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:39.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:39.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:28:39.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:28:39.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: osd.3 [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] boot 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 122 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:39.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:28:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:28:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: osd.3 [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] boot 2024-05-10T05:28:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T05:28:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T05:28:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T05:28:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 122 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:28:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:28:39.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:39.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:39.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:39.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:39.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:40.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:40 smithi142 ceph-mon[77667]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:28:40.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:40 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi131 2024-05-10T05:28:40.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:40 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi131 to 8634M 2024-05-10T05:28:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:40 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:40.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:40 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:40.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:40 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:40.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:40 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:40.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:40 smithi142 ceph-mon[77667]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T05:28:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:40 smithi131 ceph-mon[37264]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:28:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:40 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi131 2024-05-10T05:28:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:40 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi131 to 8634M 2024-05-10T05:28:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:40 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:40 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:40 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:40.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:40 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:40.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:40 smithi131 ceph-mon[37264]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T05:28:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:40 smithi157 ceph-mon[78427]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:28:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:40 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi131 2024-05-10T05:28:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:40 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi131 to 8634M 2024-05-10T05:28:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:40 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:40 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:40 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:40 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:40 smithi157 ceph-mon[78427]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T05:28:41.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:41 smithi131 ceph-mon[37264]: osdmap e28: 4 total, 4 up, 4 in 2024-05-10T05:28:41.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:41 smithi131 ceph-mon[37264]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:41.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:41 smithi131 ceph-mon[37264]: from='client.? 172.21.15.142:0/3971866673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a346957d-a63e-4865-b454-204ba17320f3"}]: dispatch 2024-05-10T05:28:41.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:41 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a346957d-a63e-4865-b454-204ba17320f3"}]: dispatch 2024-05-10T05:28:41.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:41 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a346957d-a63e-4865-b454-204ba17320f3"}]': finished 2024-05-10T05:28:41.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:41 smithi131 ceph-mon[37264]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T05:28:41.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:41 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:41 smithi157 ceph-mon[78427]: osdmap e28: 4 total, 4 up, 4 in 2024-05-10T05:28:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:41 smithi157 ceph-mon[78427]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:41 smithi157 ceph-mon[78427]: from='client.? 172.21.15.142:0/3971866673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a346957d-a63e-4865-b454-204ba17320f3"}]: dispatch 2024-05-10T05:28:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:41 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a346957d-a63e-4865-b454-204ba17320f3"}]: dispatch 2024-05-10T05:28:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:41 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a346957d-a63e-4865-b454-204ba17320f3"}]': finished 2024-05-10T05:28:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:41 smithi157 ceph-mon[78427]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T05:28:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:41 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:41.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:41 smithi142 ceph-mon[77667]: osdmap e28: 4 total, 4 up, 4 in 2024-05-10T05:28:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:41 smithi142 ceph-mon[77667]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:41 smithi142 ceph-mon[77667]: from='client.? 172.21.15.142:0/3971866673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a346957d-a63e-4865-b454-204ba17320f3"}]: dispatch 2024-05-10T05:28:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:41 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a346957d-a63e-4865-b454-204ba17320f3"}]: dispatch 2024-05-10T05:28:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:41 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a346957d-a63e-4865-b454-204ba17320f3"}]': finished 2024-05-10T05:28:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:41 smithi142 ceph-mon[77667]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T05:28:41.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:41 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:42.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:42 smithi131 ceph-mon[37264]: from='client.? 172.21.15.142:0/2789951613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:28:42.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:42 smithi157 ceph-mon[78427]: from='client.? 172.21.15.142:0/2789951613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:28:42.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:42 smithi142 ceph-mon[77667]: from='client.? 172.21.15.142:0/2789951613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:28:43.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:43 smithi131 ceph-mon[37264]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-05-10T05:28:43.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:43 smithi157 ceph-mon[78427]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-05-10T05:28:43.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:43 smithi142 ceph-mon[77667]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-05-10T05:28:45.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:45 smithi142 ceph-mon[77667]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T05:28:46.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:45 smithi131 ceph-mon[37264]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T05:28:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:45 smithi157 ceph-mon[78427]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T05:28:47.107 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T05:28:47.107 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:47.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T05:28:47.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T05:28:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:48.100 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:47 smithi142 ceph-mon[77667]: Deploying daemon osd.4 on smithi142 2024-05-10T05:28:48.100 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:47 smithi142 ceph-mon[77667]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-10T05:28:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:47 smithi131 ceph-mon[37264]: Deploying daemon osd.4 on smithi142 2024-05-10T05:28:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:47 smithi131 ceph-mon[37264]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-10T05:28:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:47 smithi157 ceph-mon[78427]: Deploying daemon osd.4 on smithi142 2024-05-10T05:28:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:47 smithi157 ceph-mon[78427]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-10T05:28:50.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:49 smithi142 ceph-mon[77667]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-10T05:28:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:49 smithi131 ceph-mon[37264]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-10T05:28:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:49 smithi157 ceph-mon[78427]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-10T05:28:51.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:51.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:51.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:51.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:51 smithi142 ceph-mon[77667]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T05:28:51.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:51.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:51.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:51.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:51 smithi131 ceph-mon[37264]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T05:28:51.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:51.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:51.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:51.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:51 smithi157 ceph-mon[78427]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T05:28:52.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.807+0000 7fbeb9ffb700 1 -- 172.21.15.142:0/2851185982 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbebc10d620 con 0x7fbeac0a11a0 2024-05-10T05:28:52.808 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 4 on host 'smithi142' 2024-05-10T05:28:52.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 -- 172.21.15.142:0/2851185982 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbeac0a11a0 msgr2=0x7fbeac0a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:52.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbeac0a11a0 0x7fbeac0a36b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000b5c0 tx=0x7fbeb0005fb0 comp rx=0 tx=0).stop 2024-05-10T05:28:52.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 -- 172.21.15.142:0/2851185982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc1005d0 msgr2=0x7fbebc198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:52.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc1005d0 0x7fbebc198670 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbea800d950 tx=0x7fbea800dc60 comp rx=0 tx=0).stop 2024-05-10T05:28:52.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 -- 172.21.15.142:0/2851185982 shutdown_connections 2024-05-10T05:28:52.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbeac0a11a0 0x7fbeac0a36b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:52.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc0ffc80 0x7fbebc198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:52.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc103230 0x7fbebc199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:52.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 --2- 172.21.15.142:0/2851185982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc1005d0 0x7fbebc198670 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:52.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 -- 172.21.15.142:0/2851185982 >> 172.21.15.142:0/2851185982 conn(0x7fbebc0fb950 msgr2=0x7fbebc1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:28:52.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 -- 172.21.15.142:0/2851185982 shutdown_connections 2024-05-10T05:28:52.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:28:52.809+0000 7fbec2dde700 1 -- 172.21.15.142:0/2851185982 wait complete. 2024-05-10T05:28:53.175 DEBUG:teuthology.orchestra.run.smithi142:osd.4> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.4.service 2024-05-10T05:28:53.177 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_3... 2024-05-10T05:28:53.177 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T05:28:53.209 INFO:journalctl@ceph.osd.4.smithi142.stdout:-- Logs begin at Fri 2024-05-10 04:59:48 UTC. -- 2024-05-10T05:28:53.282 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:53.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:53.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:53 smithi142 ceph-mon[77667]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-10T05:28:53.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:53 smithi157 ceph-mon[78427]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-10T05:28:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:53.408 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:28:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:53.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:53 smithi131 ceph-mon[37264]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-10T05:28:53.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:28:54.645 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 10 05:28:54 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-4[91367]: 2024-05-10T05:28:54.341+0000 7f74f4b4f7c0 -1 osd.4 0 log_to_monitors true 2024-05-10T05:28:54.689 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T05:28:54.689 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T05:28:54.689 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-10T05:28:54.689 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-10T05:28:54.689 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00842416 s, 1.2 GB/s 2024-05-10T05:28:54.689 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-10T05:28:55.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:54 smithi157 ceph-mon[78427]: from='osd.4 [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T05:28:55.010 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:54 smithi157 ceph-mon[78427]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T05:28:55.120 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2024-05-10T05:28:55.132 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:54 smithi142 ceph-mon[77667]: from='osd.4 [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T05:28:55.133 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:54 smithi142 ceph-mon[77667]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T05:28:55.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:54 smithi131 ceph-mon[37264]: from='osd.4 [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T05:28:55.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:54 smithi131 ceph-mon[37264]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T05:28:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:28:56.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:56.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='osd.4 [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:56.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:56.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='osd.4 [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:56.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T05:28:56.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T05:28:56.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:56.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='osd.4 [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:28:56.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:28:56.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:56.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:28:56.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:56.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:28:56.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:28:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.321+0000 7f0e10f17700 1 -- 172.21.15.157:0/726330882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c100bb0 msgr2=0x7f0e0c100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.321+0000 7f0e10f17700 1 --2- 172.21.15.157:0/726330882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c100bb0 0x7f0e0c100fc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009990 tx=0x7f0e00009ca0 comp rx=0 tx=0).stop 2024-05-10T05:28:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.321+0000 7f0e10f17700 1 -- 172.21.15.157:0/726330882 shutdown_connections 2024-05-10T05:28:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.321+0000 7f0e10f17700 1 --2- 172.21.15.157:0/726330882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e0c102680 0x7f0e0c108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.321+0000 7f0e10f17700 1 --2- 172.21.15.157:0/726330882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c100bb0 0x7f0e0c100fc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.321+0000 7f0e10f17700 1 --2- 172.21.15.157:0/726330882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c101ce0 0x7f0e0c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.321+0000 7f0e10f17700 1 -- 172.21.15.157:0/726330882 >> 172.21.15.157:0/726330882 conn(0x7f0e0c0fc210 msgr2=0x7f0e0c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:28:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.321+0000 7f0e10f17700 1 -- 172.21.15.157:0/726330882 shutdown_connections 2024-05-10T05:28:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.321+0000 7f0e10f17700 1 -- 172.21.15.157:0/726330882 wait complete. 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e10f17700 1 Processor -- start 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e10f17700 1 -- start start 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e10f17700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c100bb0 0x7f0e0c19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e10f17700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c101ce0 0x7f0e0c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e10f17700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e0c102680 0x7f0e0c19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e10f17700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c19fe80 con 0x7f0e0c100bb0 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e10f17700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c19ffc0 con 0x7f0e0c102680 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e10f17700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c1a0100 con 0x7f0e0c101ce0 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e09d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c101ce0 0x7f0e0c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e09d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c101ce0 0x7f0e0c19d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41660/0 (socket says 172.21.15.157:41660) 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e09d9b700 1 -- 172.21.15.157:0/2410790698 learned_addr learned my addr 172.21.15.157:0/2410790698 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-10T05:28:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e0ad9d700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e0c102680 0x7f0e0c19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e09d9b700 1 -- 172.21.15.157:0/2410790698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e0c102680 msgr2=0x7f0e0c19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e09d9b700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e0c102680 0x7f0e0c19f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e0a59c700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c100bb0 0x7f0e0c19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e09d9b700 1 -- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c100bb0 msgr2=0x7f0e0c19aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e09d9b700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c100bb0 0x7f0e0c19aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e09d9b700 1 -- 172.21.15.157:0/2410790698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e000095f0 con 0x7f0e0c101ce0 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.322+0000 7f0e0a59c700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c100bb0 0x7f0e0c19aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.323+0000 7f0e09d9b700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c101ce0 0x7f0e0c19d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00b530 tx=0x7f0dfc00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.323+0000 7f0dfb7fe700 1 -- 172.21.15.157:0/2410790698 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc011830 con 0x7f0e0c101ce0 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.323+0000 7f0dfb7fe700 1 -- 172.21.15.157:0/2410790698 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dfc011e50 con 0x7f0e0c101ce0 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.323+0000 7f0dfb7fe700 1 -- 172.21.15.157:0/2410790698 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc00f2e0 con 0x7f0e0c101ce0 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.323+0000 7f0e10f17700 1 -- 172.21.15.157:0/2410790698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c1a0360 con 0x7f0e0c101ce0 2024-05-10T05:28:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.323+0000 7f0e10f17700 1 -- 172.21.15.157:0/2410790698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c1a08e0 con 0x7f0e0c101ce0 2024-05-10T05:28:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.324+0000 7f0e10f17700 1 -- 172.21.15.157:0/2410790698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0c107dd0 con 0x7f0e0c101ce0 2024-05-10T05:28:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.325+0000 7f0dfb7fe700 1 -- 172.21.15.157:0/2410790698 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0dfc010580 con 0x7f0e0c101ce0 2024-05-10T05:28:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.326+0000 7f0dfb7fe700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0df40a11a0 0x7f0df40a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:28:56.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.326+0000 7f0dfb7fe700 1 -- 172.21.15.157:0/2410790698 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3821+0+0 (secure 0 0 0) 0x7f0dfc0cb1f0 con 0x7f0e0c101ce0 2024-05-10T05:28:56.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.326+0000 7f0e0a59c700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0df40a11a0 0x7f0df40a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:28:56.328 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.327+0000 7f0e0a59c700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0df40a11a0 0x7f0df40a36b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000b2e0 tx=0x7f0e00005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:28:56.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.331+0000 7f0dfb7fe700 1 -- 172.21.15.157:0/2410790698 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0dfc095b10 con 0x7f0e0c101ce0 2024-05-10T05:28:56.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:28:56.530+0000 7f0e10f17700 1 -- 172.21.15.157:0/2410790698 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e0c10c6c0 con 0x7f0df40a11a0 2024-05-10T05:28:56.644 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 10 05:28:56 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-4[91367]: 2024-05-10T05:28:56.220+0000 7f74ea1b3700 -1 osd.4 0 waiting for initial osdmap 2024-05-10T05:28:56.644 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 10 05:28:56 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-4[91367]: 2024-05-10T05:28:56.227+0000 7f74e1f62700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:28:57.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:56 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi142 2024-05-10T05:28:57.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:56 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi142 to 8556M 2024-05-10T05:28:57.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:56 smithi157 ceph-mon[78427]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:28:57.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:56 smithi157 ceph-mon[78427]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T05:28:57.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:57.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:57.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:56 smithi157 ceph-mon[78427]: from='osd.4 ' entity='osd.4' 2024-05-10T05:28:57.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:28:57.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:28:57.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:57.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:56 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi142 2024-05-10T05:28:57.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:56 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi142 to 8556M 2024-05-10T05:28:57.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:56 smithi142 ceph-mon[77667]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:28:57.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:56 smithi142 ceph-mon[77667]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T05:28:57.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:57.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:57.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:56 smithi142 ceph-mon[77667]: from='osd.4 ' entity='osd.4' 2024-05-10T05:28:57.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:28:57.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:28:57.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:57.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:56 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi142 2024-05-10T05:28:57.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:56 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi142 to 8556M 2024-05-10T05:28:57.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:56 smithi131 ceph-mon[37264]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:28:57.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:56 smithi131 ceph-mon[37264]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T05:28:57.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:57.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:57.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:56 smithi131 ceph-mon[37264]: from='osd.4 ' entity='osd.4' 2024-05-10T05:28:57.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:28:57.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:28:57.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:28:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:57 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:28:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:57 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:28:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:57 smithi157 ceph-mon[78427]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:28:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:57 smithi157 ceph-mon[78427]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:57 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:57 smithi157 ceph-mon[78427]: osd.4 [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] boot 2024-05-10T05:28:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:57 smithi157 ceph-mon[78427]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T05:28:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:57 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:58.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:57 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:28:58.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:57 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:28:58.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:57 smithi142 ceph-mon[77667]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:28:58.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:57 smithi142 ceph-mon[77667]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:58.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:58.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:57 smithi142 ceph-mon[77667]: osd.4 [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] boot 2024-05-10T05:28:58.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:57 smithi142 ceph-mon[77667]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T05:28:58.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:57 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:28:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:57 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:28:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:57 smithi131 ceph-mon[37264]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:28:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:57 smithi131 ceph-mon[37264]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-10T05:28:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:57 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:57 smithi131 ceph-mon[37264]: osd.4 [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] boot 2024-05-10T05:28:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:57 smithi131 ceph-mon[37264]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T05:28:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:57 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T05:28:59.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:59 smithi157 ceph-mon[78427]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T05:28:59.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:59 smithi157 ceph-mon[78427]: from='client.? 172.21.15.157:0/2998777043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33496762-ef3a-4ac9-99a6-0b85e46fd5df"}]: dispatch 2024-05-10T05:28:59.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:59 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33496762-ef3a-4ac9-99a6-0b85e46fd5df"}]: dispatch 2024-05-10T05:28:59.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:59 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33496762-ef3a-4ac9-99a6-0b85e46fd5df"}]': finished 2024-05-10T05:28:59.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:59 smithi157 ceph-mon[78427]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T05:28:59.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:59 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:28:59.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:59 smithi157 ceph-mon[78427]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:28:59.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:28:59 smithi157 ceph-mon[78427]: from='client.? 172.21.15.157:0/1073391853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:28:59.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:59 smithi142 ceph-mon[77667]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T05:28:59.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:59 smithi142 ceph-mon[77667]: from='client.? 172.21.15.157:0/2998777043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33496762-ef3a-4ac9-99a6-0b85e46fd5df"}]: dispatch 2024-05-10T05:28:59.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:59 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33496762-ef3a-4ac9-99a6-0b85e46fd5df"}]: dispatch 2024-05-10T05:28:59.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:59 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33496762-ef3a-4ac9-99a6-0b85e46fd5df"}]': finished 2024-05-10T05:28:59.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:59 smithi142 ceph-mon[77667]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T05:28:59.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:59 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:28:59.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:59 smithi142 ceph-mon[77667]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:28:59.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:28:59 smithi142 ceph-mon[77667]: from='client.? 172.21.15.157:0/1073391853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:28:59.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:59 smithi131 ceph-mon[37264]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T05:28:59.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:59 smithi131 ceph-mon[37264]: from='client.? 172.21.15.157:0/2998777043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33496762-ef3a-4ac9-99a6-0b85e46fd5df"}]: dispatch 2024-05-10T05:28:59.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:59 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33496762-ef3a-4ac9-99a6-0b85e46fd5df"}]: dispatch 2024-05-10T05:28:59.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:59 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33496762-ef3a-4ac9-99a6-0b85e46fd5df"}]': finished 2024-05-10T05:28:59.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:59 smithi131 ceph-mon[37264]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T05:28:59.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:59 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:28:59.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:59 smithi131 ceph-mon[37264]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:28:59.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:28:59 smithi131 ceph-mon[37264]: from='client.? 172.21.15.157:0/1073391853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:29:02.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:01 smithi142 ceph-mon[77667]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:02.194 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:01 smithi157 ceph-mon[78427]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:02.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:01 smithi131 ceph-mon[37264]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:04.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:03 smithi157 ceph-mon[78427]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:04.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:03 smithi142 ceph-mon[77667]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:04.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:03 smithi131 ceph-mon[37264]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:05.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T05:29:05.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:05.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T05:29:05.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:05.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T05:29:05.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:06.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:05 smithi142 ceph-mon[77667]: Deploying daemon osd.5 on smithi157 2024-05-10T05:29:06.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:05 smithi142 ceph-mon[77667]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:06.149 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:05 smithi157 ceph-mon[78427]: Deploying daemon osd.5 on smithi157 2024-05-10T05:29:06.150 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:05 smithi157 ceph-mon[78427]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:06.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:05 smithi131 ceph-mon[37264]: Deploying daemon osd.5 on smithi157 2024-05-10T05:29:06.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:05 smithi131 ceph-mon[37264]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:08.115 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:07 smithi157 ceph-mon[78427]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:08.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:07 smithi142 ceph-mon[77667]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:08.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:07 smithi131 ceph-mon[37264]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:09.530 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:09.530 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:09.531 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:09.531 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:09 smithi157 ceph-mon[78427]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:09.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:09.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:09.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:09.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:09 smithi142 ceph-mon[77667]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:09.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:09.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:09.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:09.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:09 smithi131 ceph-mon[37264]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:10.540 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.539+0000 7f0dfb7fe700 1 -- 172.21.15.157:0/2410790698 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0e0c10c6c0 con 0x7f0df40a11a0 2024-05-10T05:29:10.540 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-05-10T05:29:10.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.541+0000 7f0e10f17700 1 -- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0df40a11a0 msgr2=0x7f0df40a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:10.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.541+0000 7f0e10f17700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0df40a11a0 0x7f0df40a36b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000b2e0 tx=0x7f0e00005fd0 comp rx=0 tx=0).stop 2024-05-10T05:29:10.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.541+0000 7f0e10f17700 1 -- 172.21.15.157:0/2410790698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c101ce0 msgr2=0x7f0e0c19d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:10.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.541+0000 7f0e10f17700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c101ce0 0x7f0e0c19d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00b530 tx=0x7f0dfc00b8f0 comp rx=0 tx=0).stop 2024-05-10T05:29:10.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.541+0000 7f0e10f17700 1 -- 172.21.15.157:0/2410790698 shutdown_connections 2024-05-10T05:29:10.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.541+0000 7f0e10f17700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0df40a11a0 0x7f0df40a36b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:10.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.541+0000 7f0e10f17700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c100bb0 0x7f0e0c19aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:10.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.541+0000 7f0e10f17700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e0c102680 0x7f0e0c19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:10.542 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.541+0000 7f0e10f17700 1 --2- 172.21.15.157:0/2410790698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e0c101ce0 0x7f0e0c19d3f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:10.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.541+0000 7f0e10f17700 1 -- 172.21.15.157:0/2410790698 >> 172.21.15.157:0/2410790698 conn(0x7f0e0c0fc210 msgr2=0x7f0e0c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:29:10.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.542+0000 7f0e10f17700 1 -- 172.21.15.157:0/2410790698 shutdown_connections 2024-05-10T05:29:10.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:10.542+0000 7f0e10f17700 1 -- 172.21.15.157:0/2410790698 wait complete. 2024-05-10T05:29:10.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:10 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:10.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:10 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:10.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:10 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:10.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:10 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:10.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:10 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:10.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:10 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:10.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:10 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:10.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:10 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:10.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:10 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:10.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:10 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:10.928 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.5.service 2024-05-10T05:29:10.930 INFO:tasks.cephadm:Deploying osd.6 on smithi131 with /dev/vg_nvme/lv_2... 2024-05-10T05:29:10.931 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T05:29:10.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:10 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:10.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:10 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:10.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:10 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:10.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:10 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:10.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:10 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:10.961 INFO:journalctl@ceph.osd.5.smithi157.stdout:-- Logs begin at Fri 2024-05-10 05:00:45 UTC. -- 2024-05-10T05:29:11.296 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:29:11.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:11.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:11.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:11 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:11.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:11 smithi157 ceph-mon[78427]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:11.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:11.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:11.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:11 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:11.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:11 smithi131 ceph-mon[37264]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:11.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:11.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:11.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:11 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:11.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:11 smithi142 ceph-mon[77667]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:12.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:12 smithi157 ceph-mon[78427]: from='osd.5 [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T05:29:12.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:12 smithi157 ceph-mon[78427]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T05:29:12.554 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:29:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:29:12.279+0000 7f5a766387c0 -1 osd.5 0 log_to_monitors true 2024-05-10T05:29:12.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:12 smithi142 ceph-mon[77667]: from='osd.5 [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T05:29:12.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:12 smithi142 ceph-mon[77667]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T05:29:12.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:12 smithi131 ceph-mon[37264]: from='osd.5 [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T05:29:12.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:12 smithi131 ceph-mon[37264]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T05:29:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T05:29:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T05:29:13.042 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-10T05:29:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-10T05:29:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0434587 s, 241 MB/s 2024-05-10T05:29:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-10T05:29:13.467 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2024-05-10T05:29:13.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T05:29:13.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='osd.5 [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi157 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi157 to 8557M 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T05:29:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T05:29:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='osd.5 [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:29:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:29:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi157 2024-05-10T05:29:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:13.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:13.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi157 to 8557M 2024-05-10T05:29:13.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:13.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:13.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.865 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:29:13.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='osd.5 [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi157 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi157 to 8557M 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:13.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:13.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:14.303 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:29:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:29:13.978+0000 7f5a6bc9c700 -1 osd.5 0 waiting for initial osdmap 2024-05-10T05:29:14.303 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:29:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:29:13.988+0000 7f5a63a4b700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:29:14.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:14 smithi131 ceph-mon[37264]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:29:14.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:14 smithi131 ceph-mon[37264]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T05:29:14.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:14.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:14.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:14 smithi131 ceph-mon[37264]: from='osd.5 ' entity='osd.5' 2024-05-10T05:29:14.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:14 smithi157 ceph-mon[78427]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:29:14.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:14 smithi157 ceph-mon[78427]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T05:29:14.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:14.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:14.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:14 smithi157 ceph-mon[78427]: from='osd.5 ' entity='osd.5' 2024-05-10T05:29:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:14 smithi142 ceph-mon[77667]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:29:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:14 smithi142 ceph-mon[77667]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T05:29:14.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:14.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:14.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:14 smithi142 ceph-mon[77667]: from='osd.5 ' entity='osd.5' 2024-05-10T05:29:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.079+0000 7f331a400700 1 -- 172.21.15.131:0/433946546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3314101ca0 msgr2=0x7f33141020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.079+0000 7f331a400700 1 --2- 172.21.15.131:0/433946546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3314101ca0 0x7f33141020b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3304009990 tx=0x7f3304009ca0 comp rx=0 tx=0).stop 2024-05-10T05:29:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.079+0000 7f331a400700 1 -- 172.21.15.131:0/433946546 shutdown_connections 2024-05-10T05:29:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.079+0000 7f331a400700 1 --2- 172.21.15.131:0/433946546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3314101ca0 0x7f33141020b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.079+0000 7f331a400700 1 --2- 172.21.15.131:0/433946546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3314103770 0x7f331410a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.079+0000 7f331a400700 1 --2- 172.21.15.131:0/433946546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3314102dd0 0x7f33141031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.079+0000 7f331a400700 1 -- 172.21.15.131:0/433946546 >> 172.21.15.131:0/433946546 conn(0x7f33140fd2c0 msgr2=0x7f33140ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:29:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.079+0000 7f331a400700 1 -- 172.21.15.131:0/433946546 shutdown_connections 2024-05-10T05:29:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.080+0000 7f331a400700 1 -- 172.21.15.131:0/433946546 wait complete. 2024-05-10T05:29:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.080+0000 7f331a400700 1 Processor -- start 2024-05-10T05:29:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f331a400700 1 -- start start 2024-05-10T05:29:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f331a400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3314101ca0 0x7f331419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:15.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f331a400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3314102dd0 0x7f331419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:15.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f331a400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3314103770 0x7f331419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:15.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f331a400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331419bce0 con 0x7f3314101ca0 2024-05-10T05:29:15.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f331a400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331419be20 con 0x7f3314103770 2024-05-10T05:29:15.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f331a400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331419bfe0 con 0x7f3314102dd0 2024-05-10T05:29:15.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f3313400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3314101ca0 0x7f331419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:15.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f3313400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3314101ca0 0x7f331419a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43114/0 (socket says 172.21.15.131:43114) 2024-05-10T05:29:15.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f3313400700 1 -- 172.21.15.131:0/926506779 learned_addr learned my addr 172.21.15.131:0/926506779 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.081+0000 7f3312a00700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3314102dd0 0x7f331419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f3313e00700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3314103770 0x7f331419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f3313400700 1 -- 172.21.15.131:0/926506779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3314102dd0 msgr2=0x7f331419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f3313400700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3314102dd0 0x7f331419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f3313400700 1 -- 172.21.15.131:0/926506779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3314103770 msgr2=0x7f331419b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f3313400700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3314103770 0x7f331419b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f3313400700 1 -- 172.21.15.131:0/926506779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33040095f0 con 0x7f3314101ca0 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f3312a00700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3314102dd0 0x7f331419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f3313400700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3314101ca0 0x7f331419a390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3304006010 tx=0x7f3304005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f330be00700 1 -- 172.21.15.131:0/926506779 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330401d070 con 0x7f3314101ca0 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f330be00700 1 -- 172.21.15.131:0/926506779 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3304021410 con 0x7f3314101ca0 2024-05-10T05:29:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f331a400700 1 -- 172.21.15.131:0/926506779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331419c260 con 0x7f3314101ca0 2024-05-10T05:29:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.082+0000 7f330be00700 1 -- 172.21.15.131:0/926506779 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304017650 con 0x7f3314101ca0 2024-05-10T05:29:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.083+0000 7f331a400700 1 -- 172.21.15.131:0/926506779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33141a2aa0 con 0x7f3314101ca0 2024-05-10T05:29:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.084+0000 7f331a400700 1 -- 172.21.15.131:0/926506779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3314100fa0 con 0x7f3314101ca0 2024-05-10T05:29:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.086+0000 7f330be00700 1 -- 172.21.15.131:0/926506779 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f330400fad0 con 0x7f3314101ca0 2024-05-10T05:29:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.087+0000 7f330be00700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32f80a11a0 0x7f32f80a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.087+0000 7f330be00700 1 -- 172.21.15.131:0/926506779 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4566+0+0 (secure 0 0 0) 0x7f33040cd480 con 0x7f3314101ca0 2024-05-10T05:29:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.087+0000 7f3312a00700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32f80a11a0 0x7f32f80a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:15.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.088+0000 7f3312a00700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32f80a11a0 0x7f32f80a36b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f331419b360 tx=0x7f3300009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:29:15.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.090+0000 7f330be00700 1 -- 172.21.15.131:0/926506779 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3304097aa0 con 0x7f3314101ca0 2024-05-10T05:29:15.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:15.303+0000 7f331a400700 1 -- 172.21.15.131:0/926506779 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f33140ff340 con 0x7f32f80a11a0 2024-05-10T05:29:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:15 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:29:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:15 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:29:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:15 smithi157 ceph-mon[78427]: osd.5 [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] boot 2024-05-10T05:29:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:15 smithi157 ceph-mon[78427]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T05:29:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:15 smithi157 ceph-mon[78427]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:29:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:29:15.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:15.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:15 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:29:15.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:15 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:29:15.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:15 smithi142 ceph-mon[77667]: osd.5 [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] boot 2024-05-10T05:29:15.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:15 smithi142 ceph-mon[77667]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T05:29:15.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:15.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:15 smithi142 ceph-mon[77667]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:15.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:29:15.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:29:15.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:15.909 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:15 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:29:15.909 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:15 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:29:15.909 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:15 smithi131 ceph-mon[37264]: osd.5 [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] boot 2024-05-10T05:29:15.909 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:15 smithi131 ceph-mon[37264]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T05:29:15.909 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T05:29:15.909 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:15 smithi131 ceph-mon[37264]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:15.910 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:29:15.910 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:29:15.910 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:16.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:16 smithi157 ceph-mon[78427]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:29:16.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:16 smithi157 ceph-mon[78427]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T05:29:16.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:16 smithi142 ceph-mon[77667]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:29:16.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:16 smithi142 ceph-mon[77667]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T05:29:16.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:16 smithi131 ceph-mon[37264]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:29:16.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:16 smithi131 ceph-mon[37264]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T05:29:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:17 smithi157 ceph-mon[78427]: osdmap e39: 6 total, 6 up, 6 in 2024-05-10T05:29:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:17 smithi157 ceph-mon[78427]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:17.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:17 smithi142 ceph-mon[77667]: osdmap e39: 6 total, 6 up, 6 in 2024-05-10T05:29:17.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:17 smithi142 ceph-mon[77667]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:17.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:17 smithi131 ceph-mon[37264]: osdmap e39: 6 total, 6 up, 6 in 2024-05-10T05:29:17.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:17 smithi131 ceph-mon[37264]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:18.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:18 smithi142 ceph-mon[77667]: osdmap e40: 6 total, 6 up, 6 in 2024-05-10T05:29:18.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:18 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2179055876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f5e4557-11d2-40d7-9cee-aa6e36659940"}]: dispatch 2024-05-10T05:29:18.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:18 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2179055876' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f5e4557-11d2-40d7-9cee-aa6e36659940"}]': finished 2024-05-10T05:29:18.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:18 smithi142 ceph-mon[77667]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T05:29:18.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:18 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:18 smithi131 ceph-mon[37264]: osdmap e40: 6 total, 6 up, 6 in 2024-05-10T05:29:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:18 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2179055876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f5e4557-11d2-40d7-9cee-aa6e36659940"}]: dispatch 2024-05-10T05:29:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:18 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2179055876' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f5e4557-11d2-40d7-9cee-aa6e36659940"}]': finished 2024-05-10T05:29:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:18 smithi131 ceph-mon[37264]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T05:29:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:18 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:18 smithi157 ceph-mon[78427]: osdmap e40: 6 total, 6 up, 6 in 2024-05-10T05:29:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:18 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2179055876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f5e4557-11d2-40d7-9cee-aa6e36659940"}]: dispatch 2024-05-10T05:29:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:18 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2179055876' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f5e4557-11d2-40d7-9cee-aa6e36659940"}]': finished 2024-05-10T05:29:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:18 smithi157 ceph-mon[78427]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T05:29:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:18 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:19.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:19 smithi142 ceph-mon[77667]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 188 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T05:29:19.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:19 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3971446243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:29:19.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:19 smithi131 ceph-mon[37264]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 188 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T05:29:19.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:19 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3971446243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:29:20.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:19 smithi157 ceph-mon[78427]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 188 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T05:29:20.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:19 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3971446243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:29:22.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:21 smithi142 ceph-mon[77667]: pgmap v124: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-10T05:29:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:21 smithi131 ceph-mon[37264]: pgmap v124: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-10T05:29:22.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:21 smithi157 ceph-mon[78427]: pgmap v124: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-10T05:29:24.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:23 smithi142 ceph-mon[77667]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-10T05:29:24.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:23 smithi131 ceph-mon[37264]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-10T05:29:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:23 smithi157 ceph-mon[78427]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-10T05:29:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:25 smithi142 ceph-mon[77667]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T05:29:26.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:25 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T05:29:26.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:25 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:26.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:25 smithi142 ceph-mon[77667]: Deploying daemon osd.6 on smithi131 2024-05-10T05:29:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:25 smithi131 ceph-mon[37264]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T05:29:26.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:25 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T05:29:26.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:25 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:26.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:25 smithi131 ceph-mon[37264]: Deploying daemon osd.6 on smithi131 2024-05-10T05:29:26.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:25 smithi157 ceph-mon[78427]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T05:29:26.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:25 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T05:29:26.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:25 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:26.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:25 smithi157 ceph-mon[78427]: Deploying daemon osd.6 on smithi131 2024-05-10T05:29:27.904 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:27 smithi131 ceph-mon[37264]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:28.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:27 smithi142 ceph-mon[77667]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:28.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:27 smithi157 ceph-mon[78427]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:30.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:29 smithi142 ceph-mon[77667]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:30.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:29 smithi131 ceph-mon[37264]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:29 smithi157 ceph-mon[78427]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:31.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:31.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:31.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:31.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:31 smithi131 ceph-mon[37264]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:32.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:32.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:32.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:32.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:31 smithi157 ceph-mon[78427]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:32.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:32.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:32.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:32.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:31 smithi142 ceph-mon[77667]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:34.135 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:33 smithi131 ceph-mon[37264]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:34.136 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:34.136 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:34.136 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:34.136 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:34.136 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:34.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:33 smithi142 ceph-mon[77667]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:34.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:34.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:34.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:34.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:34.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:33 smithi157 ceph-mon[78427]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:34.360 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 6 on host 'smithi131' 2024-05-10T05:29:34.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.358+0000 7f330be00700 1 -- 172.21.15.131:0/926506779 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f33140ff340 con 0x7f32f80a11a0 2024-05-10T05:29:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.361+0000 7f331a400700 1 -- 172.21.15.131:0/926506779 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32f80a11a0 msgr2=0x7f32f80a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.361+0000 7f331a400700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32f80a11a0 0x7f32f80a36b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f331419b360 tx=0x7f3300009260 comp rx=0 tx=0).stop 2024-05-10T05:29:34.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.362+0000 7f331a400700 1 -- 172.21.15.131:0/926506779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3314101ca0 msgr2=0x7f331419a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:34.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.362+0000 7f331a400700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3314101ca0 0x7f331419a390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3304006010 tx=0x7f3304005740 comp rx=0 tx=0).stop 2024-05-10T05:29:34.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.362+0000 7f331a400700 1 -- 172.21.15.131:0/926506779 shutdown_connections 2024-05-10T05:29:34.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.362+0000 7f331a400700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32f80a11a0 0x7f32f80a36b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:34.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.362+0000 7f331a400700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3314101ca0 0x7f331419a390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:34.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.362+0000 7f331a400700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3314103770 0x7f331419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:34.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.362+0000 7f331a400700 1 --2- 172.21.15.131:0/926506779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3314102dd0 0x7f331419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:34.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.362+0000 7f331a400700 1 -- 172.21.15.131:0/926506779 >> 172.21.15.131:0/926506779 conn(0x7f33140fd2c0 msgr2=0x7f33140fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:29:34.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.362+0000 7f331a400700 1 -- 172.21.15.131:0/926506779 shutdown_connections 2024-05-10T05:29:34.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:29:34.362+0000 7f331a400700 1 -- 172.21.15.131:0/926506779 wait complete. 2024-05-10T05:29:34.776 DEBUG:teuthology.orchestra.run.smithi131:osd.6> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.6.service 2024-05-10T05:29:34.781 INFO:tasks.cephadm:Deploying osd.7 on smithi142 with /dev/vg_nvme/lv_2... 2024-05-10T05:29:34.781 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T05:29:34.860 INFO:journalctl@ceph.osd.6.smithi131.stdout:-- Logs begin at Fri 2024-05-10 05:06:53 UTC. -- 2024-05-10T05:29:35.015 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:29:35.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:35 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:35.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:35 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:35.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:35 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:35.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:35 smithi142 ceph-mon[77667]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:35.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:35 smithi142 ceph-mon[77667]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T05:29:35.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:35.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:35.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:35.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:35 smithi131 ceph-mon[37264]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:35.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:35 smithi131 ceph-mon[37264]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T05:29:35.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:35 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:35.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:35 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:35.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:35 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:35.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:35 smithi157 ceph-mon[78427]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:35.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:35 smithi157 ceph-mon[78427]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T05:29:36.364 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T05:29:36.364 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T05:29:36.364 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-10T05:29:36.364 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-10T05:29:36.364 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114548 s, 915 MB/s 2024-05-10T05:29:36.364 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-10T05:29:36.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:36 smithi142 ceph-mon[77667]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T05:29:36.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:36 smithi142 ceph-mon[77667]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T05:29:36.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:36 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:36.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:36 smithi142 ceph-mon[77667]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:29:36.701 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-05-10T05:29:36.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:36 smithi131 ceph-mon[37264]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T05:29:36.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:36 smithi131 ceph-mon[37264]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T05:29:36.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:36 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:36.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:36 smithi131 ceph-mon[37264]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:29:36.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:36 smithi157 ceph-mon[78427]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T05:29:36.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:36 smithi157 ceph-mon[78427]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T05:29:36.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:36 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:36.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:36 smithi157 ceph-mon[78427]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:29:36.920 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:29:37.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:29:36 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:29:36.942+0000 7f64d8e00700 -1 osd.6 0 waiting for initial osdmap 2024-05-10T05:29:37.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:29:36 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:29:36.970+0000 7f64cec00700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:29:37.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:37 smithi142 ceph-mon[77667]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:29:37.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:37 smithi142 ceph-mon[77667]: osdmap e43: 7 total, 6 up, 7 in 2024-05-10T05:29:37.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:37.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:37.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:37 smithi142 ceph-mon[77667]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:37.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:37 smithi142 ceph-mon[77667]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' 2024-05-10T05:29:37.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:37 smithi131 ceph-mon[37264]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:29:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:37 smithi131 ceph-mon[37264]: osdmap e43: 7 total, 6 up, 7 in 2024-05-10T05:29:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:37 smithi131 ceph-mon[37264]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:37.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:37 smithi131 ceph-mon[37264]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' 2024-05-10T05:29:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:37 smithi157 ceph-mon[78427]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:29:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:37 smithi157 ceph-mon[78427]: osdmap e43: 7 total, 6 up, 7 in 2024-05-10T05:29:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:37 smithi157 ceph-mon[78427]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-05-10T05:29:37.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:37 smithi157 ceph-mon[78427]: from='osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866]' entity='osd.6' 2024-05-10T05:29:37.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.975+0000 7f966e5a6700 1 -- 172.21.15.142:0/2027173059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9668105110 msgr2=0x7f9668107530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:37.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.975+0000 7f966e5a6700 1 --2- 172.21.15.142:0/2027173059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9668105110 0x7f9668107530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f965c009990 tx=0x7f965c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:29:37.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.975+0000 7f966e5a6700 1 -- 172.21.15.142:0/2027173059 shutdown_connections 2024-05-10T05:29:37.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.975+0000 7f966e5a6700 1 --2- 172.21.15.142:0/2027173059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9668107a70 0x7f9668109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:37.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.975+0000 7f966e5a6700 1 --2- 172.21.15.142:0/2027173059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96681027b0 0x7f9668104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:37.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.975+0000 7f966e5a6700 1 --2- 172.21.15.142:0/2027173059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9668105110 0x7f9668107530 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:37.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.975+0000 7f966e5a6700 1 -- 172.21.15.142:0/2027173059 >> 172.21.15.142:0/2027173059 conn(0x7f96680fc210 msgr2=0x7f96680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.975+0000 7f966e5a6700 1 -- 172.21.15.142:0/2027173059 shutdown_connections 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.975+0000 7f966e5a6700 1 -- 172.21.15.142:0/2027173059 wait complete. 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.975+0000 7f966e5a6700 1 Processor -- start 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f966e5a6700 1 -- start start 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f966e5a6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96681027b0 0x7f9668198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f966e5a6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9668105110 0x7f96681986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f966e5a6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9668107a70 0x7f9668199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f966e5a6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9668199ad0 con 0x7f9668105110 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f966e5a6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9668199c10 con 0x7f96681027b0 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f966e5a6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9668199dd0 con 0x7f9668107a70 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f9667fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96681027b0 0x7f9668198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:37.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f9667fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96681027b0 0x7f9668198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45688/0 (socket says 172.21.15.142:45688) 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f9667fff700 1 -- 172.21.15.142:0/1840724575 learned_addr learned my addr 172.21.15.142:0/1840724575 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f966cb43700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9668107a70 0x7f9668199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f96677fe700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9668105110 0x7f96681986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f9667fff700 1 -- 172.21.15.142:0/1840724575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9668107a70 msgr2=0x7f9668199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f9667fff700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9668107a70 0x7f9668199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f9667fff700 1 -- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9668105110 msgr2=0x7f96681986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f9667fff700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9668105110 0x7f96681986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f9667fff700 1 -- 172.21.15.142:0/1840724575 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c0095f0 con 0x7f96681027b0 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f96677fe700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9668105110 0x7f96681986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.976+0000 7f9667fff700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96681027b0 0x7f9668198180 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f965800e970 tx=0x7f965800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.977+0000 7f96657fa700 1 -- 172.21.15.142:0/1840724575 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658009790 con 0x7f96681027b0 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.977+0000 7f96657fa700 1 -- 172.21.15.142:0/1840724575 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f965800eec0 con 0x7f96681027b0 2024-05-10T05:29:37.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.977+0000 7f966e5a6700 1 -- 172.21.15.142:0/1840724575 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966819a0b0 con 0x7f96681027b0 2024-05-10T05:29:37.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.977+0000 7f96657fa700 1 -- 172.21.15.142:0/1840724575 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658005590 con 0x7f96681027b0 2024-05-10T05:29:37.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.977+0000 7f966e5a6700 1 -- 172.21.15.142:0/1840724575 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96681a0910 con 0x7f96681027b0 2024-05-10T05:29:37.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.978+0000 7f966e5a6700 1 -- 172.21.15.142:0/1840724575 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96680fddc0 con 0x7f96681027b0 2024-05-10T05:29:37.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.979+0000 7f96657fa700 1 -- 172.21.15.142:0/1840724575 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f965800cbd0 con 0x7f96681027b0 2024-05-10T05:29:37.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.980+0000 7f96657fa700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96500a11a0 0x7f96500a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:37.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.980+0000 7f96657fa700 1 -- 172.21.15.142:0/1840724575 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f9658014070 con 0x7f96681027b0 2024-05-10T05:29:37.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.980+0000 7f96677fe700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96500a11a0 0x7f96500a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:37.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.980+0000 7f96677fe700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96500a11a0 0x7f96500a36b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b2e0 tx=0x7f965c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:29:37.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:37.983+0000 7f96657fa700 1 -- 172.21.15.142:0/1840724575 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9658096460 con 0x7f96681027b0 2024-05-10T05:29:38.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:38.183+0000 7f966e5a6700 1 -- 172.21.15.142:0/1840724575 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f966810d7e0 con 0x7f96500a11a0 2024-05-10T05:29:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi131 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi131 to 5756M 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:38.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:38.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] boot 2024-05-10T05:29:38.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: osdmap e44: 7 total, 7 up, 7 in 2024-05-10T05:29:38.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:38.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:29:38.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:29:38.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:29:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:29:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi131 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi131 to 5756M 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] boot 2024-05-10T05:29:38.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: osdmap e44: 7 total, 7 up, 7 in 2024-05-10T05:29:38.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:38.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:29:38.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:29:38.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:29:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:29:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi131 2024-05-10T05:29:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi131 to 5756M 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: osd.6 [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] boot 2024-05-10T05:29:38.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: osdmap e44: 7 total, 7 up, 7 in 2024-05-10T05:29:38.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T05:29:38.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:29:38.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:29:38.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:39.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:39 smithi142 ceph-mon[77667]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:29:39.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:39 smithi142 ceph-mon[77667]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:39.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:39 smithi131 ceph-mon[37264]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:29:39.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:39 smithi131 ceph-mon[37264]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:39 smithi157 ceph-mon[78427]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:29:39.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:39 smithi157 ceph-mon[78427]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:40 smithi142 ceph-mon[77667]: osdmap e45: 7 total, 7 up, 7 in 2024-05-10T05:29:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:40 smithi142 ceph-mon[77667]: from='client.? 172.21.15.142:0/3912223985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfcb0014-f452-4449-98aa-2f041be39249"}]: dispatch 2024-05-10T05:29:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:40 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfcb0014-f452-4449-98aa-2f041be39249"}]: dispatch 2024-05-10T05:29:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:40 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfcb0014-f452-4449-98aa-2f041be39249"}]': finished 2024-05-10T05:29:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:40 smithi142 ceph-mon[77667]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T05:29:40.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:40 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:40 smithi131 ceph-mon[37264]: osdmap e45: 7 total, 7 up, 7 in 2024-05-10T05:29:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:40 smithi131 ceph-mon[37264]: from='client.? 172.21.15.142:0/3912223985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfcb0014-f452-4449-98aa-2f041be39249"}]: dispatch 2024-05-10T05:29:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:40 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfcb0014-f452-4449-98aa-2f041be39249"}]: dispatch 2024-05-10T05:29:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:40 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfcb0014-f452-4449-98aa-2f041be39249"}]': finished 2024-05-10T05:29:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:40 smithi131 ceph-mon[37264]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T05:29:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:40 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:40 smithi157 ceph-mon[78427]: osdmap e45: 7 total, 7 up, 7 in 2024-05-10T05:29:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:40 smithi157 ceph-mon[78427]: from='client.? 172.21.15.142:0/3912223985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfcb0014-f452-4449-98aa-2f041be39249"}]: dispatch 2024-05-10T05:29:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:40 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfcb0014-f452-4449-98aa-2f041be39249"}]: dispatch 2024-05-10T05:29:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:40 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfcb0014-f452-4449-98aa-2f041be39249"}]': finished 2024-05-10T05:29:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:40 smithi157 ceph-mon[78427]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T05:29:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:40 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:41.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:41 smithi142 ceph-mon[77667]: from='client.? 172.21.15.142:0/2921848855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:29:41.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:41 smithi142 ceph-mon[77667]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:41.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:41 smithi131 ceph-mon[37264]: from='client.? 172.21.15.142:0/2921848855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:29:41.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:41 smithi131 ceph-mon[37264]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:41 smithi157 ceph-mon[78427]: from='client.? 172.21.15.142:0/2921848855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:29:41.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:41 smithi157 ceph-mon[78427]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:44.104 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:43 smithi142 ceph-mon[77667]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:44.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:43 smithi131 ceph-mon[37264]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:43 smithi157 ceph-mon[78427]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:45.970 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:45 smithi142 ceph-mon[77667]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:46.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:45 smithi131 ceph-mon[37264]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:45 smithi157 ceph-mon[78427]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:47.099 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T05:29:47.100 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:47.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T05:29:47.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T05:29:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:48.030 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:47 smithi142 ceph-mon[77667]: Deploying daemon osd.7 on smithi142 2024-05-10T05:29:48.030 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:47 smithi142 ceph-mon[77667]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:47 smithi131 ceph-mon[37264]: Deploying daemon osd.7 on smithi142 2024-05-10T05:29:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:47 smithi131 ceph-mon[37264]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:47 smithi157 ceph-mon[78427]: Deploying daemon osd.7 on smithi142 2024-05-10T05:29:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:47 smithi157 ceph-mon[78427]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:50.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:49 smithi142 ceph-mon[77667]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:50.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:49 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:50.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:49 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:50.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:49 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:49 smithi131 ceph-mon[37264]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:49 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:49 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:49 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:49 smithi157 ceph-mon[78427]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:51.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:51 smithi142 ceph-mon[77667]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:51.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:51.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:51.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:51.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:51.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:52.039 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 7 on host 'smithi142' 2024-05-10T05:29:52.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.038+0000 7f96657fa700 1 -- 172.21.15.142:0/1840724575 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f966810d7e0 con 0x7f96500a11a0 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 -- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96500a11a0 msgr2=0x7f96500a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96500a11a0 0x7f96500a36b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b2e0 tx=0x7f965c01a040 comp rx=0 tx=0).stop 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 -- 172.21.15.142:0/1840724575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96681027b0 msgr2=0x7f9668198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96681027b0 0x7f9668198180 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f965800e970 tx=0x7f965800ec80 comp rx=0 tx=0).stop 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 -- 172.21.15.142:0/1840724575 shutdown_connections 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96500a11a0 0x7f96500a36b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9668107a70 0x7f9668199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96681027b0 0x7f9668198180 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 --2- 172.21.15.142:0/1840724575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9668105110 0x7f96681986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 -- 172.21.15.142:0/1840724575 >> 172.21.15.142:0/1840724575 conn(0x7f96680fc210 msgr2=0x7f9668103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 -- 172.21.15.142:0/1840724575 shutdown_connections 2024-05-10T05:29:52.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:29:52.040+0000 7f966e5a6700 1 -- 172.21.15.142:0/1840724575 wait complete. 2024-05-10T05:29:52.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:51 smithi131 ceph-mon[37264]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:52.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:52.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:52.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:52.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:52.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:51 smithi157 ceph-mon[78427]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:52.413 DEBUG:teuthology.orchestra.run.smithi142:osd.7> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.7.service 2024-05-10T05:29:52.415 INFO:tasks.cephadm:Deploying osd.8 on smithi157 with /dev/vg_nvme/lv_2... 2024-05-10T05:29:52.416 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T05:29:52.449 INFO:journalctl@ceph.osd.7.smithi142.stdout:-- Logs begin at Fri 2024-05-10 04:59:48 UTC. -- 2024-05-10T05:29:52.649 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:29:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:53.308 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:53.308 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:53.308 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:29:53.953 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T05:29:53.953 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T05:29:53.953 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-10T05:29:53.953 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-10T05:29:53.953 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00820515 s, 1.3 GB/s 2024-05-10T05:29:53.953 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-10T05:29:54.144 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:29:53 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:29:53.854+0000 7f8a5247c7c0 -1 osd.7 0 log_to_monitors true 2024-05-10T05:29:54.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:54 smithi142 ceph-mon[77667]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:54.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:54 smithi142 ceph-mon[77667]: from='osd.7 [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T05:29:54.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:54 smithi142 ceph-mon[77667]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T05:29:54.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:54 smithi157 ceph-mon[78427]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:54.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:54 smithi157 ceph-mon[78427]: from='osd.7 [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T05:29:54.275 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:54 smithi157 ceph-mon[78427]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T05:29:54.344 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2024-05-10T05:29:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:54 smithi131 ceph-mon[37264]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:54 smithi131 ceph-mon[37264]: from='osd.7 [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T05:29:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:54 smithi131 ceph-mon[37264]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T05:29:54.566 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:29:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T05:29:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T05:29:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='osd.7 [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:29:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:29:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:29:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:55 smithi157 ceph-mon[78427]: osdmap e48: 8 total, 7 up, 8 in 2024-05-10T05:29:55.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T05:29:55.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T05:29:55.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='osd.7 [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:29:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:55 smithi142 ceph-mon[77667]: osdmap e48: 8 total, 7 up, 8 in 2024-05-10T05:29:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T05:29:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T05:29:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='osd.7 [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:29:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:29:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:55.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:55.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:29:55.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:55.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:29:55.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:29:55.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:29:55.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:55 smithi131 ceph-mon[37264]: osdmap e48: 8 total, 7 up, 8 in 2024-05-10T05:29:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.577+0000 7fbeee507700 1 -- 172.21.15.157:0/647887913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8102680 msgr2=0x7fbee8108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.577+0000 7fbeee507700 1 --2- 172.21.15.157:0/647887913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8102680 0x7fbee8108f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbee000b0a0 tx=0x7fbee000b3b0 comp rx=0 tx=0).stop 2024-05-10T05:29:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.577+0000 7fbeee507700 1 -- 172.21.15.157:0/647887913 shutdown_connections 2024-05-10T05:29:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.577+0000 7fbeee507700 1 --2- 172.21.15.157:0/647887913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbee8100bb0 0x7fbee8100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.577+0000 7fbeee507700 1 --2- 172.21.15.157:0/647887913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8102680 0x7fbee8108f30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.577+0000 7fbeee507700 1 --2- 172.21.15.157:0/647887913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbee8101ce0 0x7fbee81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.577+0000 7fbeee507700 1 -- 172.21.15.157:0/647887913 >> 172.21.15.157:0/647887913 conn(0x7fbee80fc1d0 msgr2=0x7fbee80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:29:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.577+0000 7fbeee507700 1 -- 172.21.15.157:0/647887913 shutdown_connections 2024-05-10T05:29:55.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.577+0000 7fbeee507700 1 -- 172.21.15.157:0/647887913 wait complete. 2024-05-10T05:29:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbeee507700 1 Processor -- start 2024-05-10T05:29:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbeee507700 1 -- start start 2024-05-10T05:29:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbeee507700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8100bb0 0x7fbee819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbeee507700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbee8101ce0 0x7fbee819d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbeee507700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbee8102680 0x7fbee819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbeee507700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee819fdf0 con 0x7fbee8102680 2024-05-10T05:29:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbeee507700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee819ff30 con 0x7fbee8101ce0 2024-05-10T05:29:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbeee507700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee81a0070 con 0x7fbee8100bb0 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbee7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8100bb0 0x7fbee819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbee7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8100bb0 0x7fbee819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55964/0 (socket says 172.21.15.157:55964) 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.578+0000 7fbee7fff700 1 -- 172.21.15.157:0/3586637321 learned_addr learned my addr 172.21.15.157:0/3586637321 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbee77fe700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbee8101ce0 0x7fbee819d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbee7fff700 1 -- 172.21.15.157:0/3586637321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbee8101ce0 msgr2=0x7fbee819d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbee7fff700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbee8101ce0 0x7fbee819d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbee7fff700 1 -- 172.21.15.157:0/3586637321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbee8102680 msgr2=0x7fbee819f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbee7fff700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbee8102680 0x7fbee819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbee7fff700 1 -- 172.21.15.157:0/3586637321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0009d00 con 0x7fbee8100bb0 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbee7fff700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8100bb0 0x7fbee819ae10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b4d0 tx=0x7fbed800b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbee57fa700 1 -- 172.21.15.157:0/3586637321 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8011830 con 0x7fbee8100bb0 2024-05-10T05:29:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbeee507700 1 -- 172.21.15.157:0/3586637321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee81a02d0 con 0x7fbee8100bb0 2024-05-10T05:29:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbee57fa700 1 -- 172.21.15.157:0/3586637321 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed8011e50 con 0x7fbee8100bb0 2024-05-10T05:29:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbee57fa700 1 -- 172.21.15.157:0/3586637321 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8010e70 con 0x7fbee8100bb0 2024-05-10T05:29:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.579+0000 7fbeee507700 1 -- 172.21.15.157:0/3586637321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee81a0850 con 0x7fbee8100bb0 2024-05-10T05:29:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.580+0000 7fbeee507700 1 -- 172.21.15.157:0/3586637321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee8107dd0 con 0x7fbee8100bb0 2024-05-10T05:29:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.582+0000 7fbee57fa700 1 -- 172.21.15.157:0/3586637321 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbed8010510 con 0x7fbee8100bb0 2024-05-10T05:29:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.583+0000 7fbee57fa700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbed40a1120 0x7fbed40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:29:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.583+0000 7fbee57fa700 1 -- 172.21.15.157:0/3586637321 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5117+0+0 (secure 0 0 0) 0x7fbed80cbce0 con 0x7fbee8100bb0 2024-05-10T05:29:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.583+0000 7fbee77fe700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbed40a1120 0x7fbed40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:29:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.584+0000 7fbee77fe700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbed40a1120 0x7fbed40a3630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbee8069d60 tx=0x7fbed000a300 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:29:55.586 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.586+0000 7fbee57fa700 1 -- 172.21.15.157:0/3586637321 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbed8096050 con 0x7fbee8100bb0 2024-05-10T05:29:55.787 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:29:55.786+0000 7fbeee507700 1 -- 172.21.15.157:0/3586637321 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbee810c700 con 0x7fbed40a1120 2024-05-10T05:29:55.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:29:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:29:55.479+0000 7f8a47ae0700 -1 osd.7 0 waiting for initial osdmap 2024-05-10T05:29:55.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:29:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:29:55.490+0000 7f8a3f88f700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:29:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi142 2024-05-10T05:29:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi142 to 5704M 2024-05-10T05:29:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: from='osd.7 ' entity='osd.7' 2024-05-10T05:29:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:29:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:29:56.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:29:56.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:56.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: osd.7 [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] boot 2024-05-10T05:29:56.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T05:29:56.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi142 2024-05-10T05:29:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi142 to 5704M 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: from='osd.7 ' entity='osd.7' 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: osd.7 [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] boot 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T05:29:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi142 2024-05-10T05:29:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi142 to 5704M 2024-05-10T05:29:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-05-10T05:29:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: from='osd.7 ' entity='osd.7' 2024-05-10T05:29:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:29:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:29:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:29:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:29:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: osd.7 [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] boot 2024-05-10T05:29:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T05:29:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T05:29:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:57 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:29:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:57 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:29:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:57 smithi157 ceph-mon[78427]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:29:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:57 smithi157 ceph-mon[78427]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T05:29:57.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:57 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:29:57.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:57 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:29:57.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:57 smithi142 ceph-mon[77667]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:29:57.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:57 smithi142 ceph-mon[77667]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T05:29:57.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:57 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:29:57.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:57 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:29:57.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:57 smithi131 ceph-mon[37264]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:29:57.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:57 smithi131 ceph-mon[37264]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T05:29:58.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.157:0/2457680519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb1c9d3-d3b2-47db-9e17-16b16513a568"}]: dispatch 2024-05-10T05:29:58.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:58 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb1c9d3-d3b2-47db-9e17-16b16513a568"}]: dispatch 2024-05-10T05:29:58.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:58 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fb1c9d3-d3b2-47db-9e17-16b16513a568"}]': finished 2024-05-10T05:29:58.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:58 smithi157 ceph-mon[78427]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T05:29:58.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:29:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.157:0/2457680519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb1c9d3-d3b2-47db-9e17-16b16513a568"}]: dispatch 2024-05-10T05:29:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:58 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb1c9d3-d3b2-47db-9e17-16b16513a568"}]: dispatch 2024-05-10T05:29:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:58 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fb1c9d3-d3b2-47db-9e17-16b16513a568"}]': finished 2024-05-10T05:29:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:58 smithi142 ceph-mon[77667]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T05:29:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:29:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.157:0/2457680519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb1c9d3-d3b2-47db-9e17-16b16513a568"}]: dispatch 2024-05-10T05:29:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:58 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb1c9d3-d3b2-47db-9e17-16b16513a568"}]: dispatch 2024-05-10T05:29:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:58 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fb1c9d3-d3b2-47db-9e17-16b16513a568"}]': finished 2024-05-10T05:29:58.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:58 smithi131 ceph-mon[37264]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T05:29:58.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:29:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:59 smithi157 ceph-mon[78427]: from='client.? 172.21.15.157:0/493478352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:29:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:59 smithi157 ceph-mon[78427]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T05:29:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:59 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:29:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:29:59 smithi157 ceph-mon[78427]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:29:59.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:59 smithi142 ceph-mon[77667]: from='client.? 172.21.15.157:0/493478352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:29:59.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:59 smithi142 ceph-mon[77667]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T05:29:59.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:59 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:29:59.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:29:59 smithi142 ceph-mon[77667]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:29:59.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:59 smithi131 ceph-mon[37264]: from='client.? 172.21.15.157:0/493478352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:29:59.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:59 smithi131 ceph-mon[37264]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T05:29:59.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:59 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:29:59.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:29:59 smithi131 ceph-mon[37264]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:00 smithi157 ceph-mon[78427]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T05:30:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:00 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:00 smithi157 ceph-mon[78427]: overall HEALTH_OK 2024-05-10T05:30:00.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:00 smithi142 ceph-mon[77667]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T05:30:00.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:00 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:00.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:00 smithi142 ceph-mon[77667]: overall HEALTH_OK 2024-05-10T05:30:00.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:00 smithi131 ceph-mon[37264]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T05:30:00.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:00 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:00.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:00 smithi131 ceph-mon[37264]: overall HEALTH_OK 2024-05-10T05:30:01.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:01 smithi157 ceph-mon[78427]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:01.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:01 smithi142 ceph-mon[77667]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:01.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:01 smithi131 ceph-mon[37264]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:04.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:03 smithi157 ceph-mon[78427]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:04.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T05:30:04.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:04.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:03 smithi142 ceph-mon[77667]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:04.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:03 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T05:30:04.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:03 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:04.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:03 smithi131 ceph-mon[37264]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:04.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T05:30:04.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:05.115 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:04 smithi157 ceph-mon[78427]: Deploying daemon osd.8 on smithi157 2024-05-10T05:30:05.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:04 smithi142 ceph-mon[77667]: Deploying daemon osd.8 on smithi157 2024-05-10T05:30:05.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:04 smithi131 ceph-mon[37264]: Deploying daemon osd.8 on smithi157 2024-05-10T05:30:06.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:05 smithi142 ceph-mon[77667]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:06.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:05 smithi131 ceph-mon[37264]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:06.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:05 smithi157 ceph-mon[78427]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:07.908 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:07 smithi157 ceph-mon[78427]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:07.908 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:07.908 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:07.908 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:08.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:07 smithi142 ceph-mon[77667]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:08.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:08.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:08.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:08.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:07 smithi131 ceph-mon[37264]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:08.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:08.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:08.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:09.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.547+0000 7fbee57fa700 1 -- 172.21.15.157:0/3586637321 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbee810c700 con 0x7fbed40a1120 2024-05-10T05:30:09.548 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 8 on host 'smithi157' 2024-05-10T05:30:09.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.549+0000 7fbeee507700 1 -- 172.21.15.157:0/3586637321 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbed40a1120 msgr2=0x7fbed40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:09.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.549+0000 7fbeee507700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbed40a1120 0x7fbed40a3630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbee8069d60 tx=0x7fbed000a300 comp rx=0 tx=0).stop 2024-05-10T05:30:09.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.549+0000 7fbeee507700 1 -- 172.21.15.157:0/3586637321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8100bb0 msgr2=0x7fbee819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:09.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.549+0000 7fbeee507700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8100bb0 0x7fbee819ae10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b4d0 tx=0x7fbed800b7e0 comp rx=0 tx=0).stop 2024-05-10T05:30:09.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.550+0000 7fbeee507700 1 -- 172.21.15.157:0/3586637321 shutdown_connections 2024-05-10T05:30:09.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.550+0000 7fbeee507700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbed40a1120 0x7fbed40a3630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:09.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.550+0000 7fbeee507700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbee8102680 0x7fbee819f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:09.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.550+0000 7fbeee507700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee8100bb0 0x7fbee819ae10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:09.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.550+0000 7fbeee507700 1 --2- 172.21.15.157:0/3586637321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbee8101ce0 0x7fbee819d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:09.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.550+0000 7fbeee507700 1 -- 172.21.15.157:0/3586637321 >> 172.21.15.157:0/3586637321 conn(0x7fbee80fc1d0 msgr2=0x7fbee8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:30:09.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.550+0000 7fbeee507700 1 -- 172.21.15.157:0/3586637321 shutdown_connections 2024-05-10T05:30:09.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:09.550+0000 7fbeee507700 1 -- 172.21.15.157:0/3586637321 wait complete. 2024-05-10T05:30:09.836 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.836 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.836 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:09 smithi157 ceph-mon[78427]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:09.836 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:09.836 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:09.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:09.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:09 smithi142 ceph-mon[77667]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.929 DEBUG:teuthology.orchestra.run.smithi157:osd.8> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.8.service 2024-05-10T05:30:09.931 INFO:tasks.cephadm:Deploying osd.9 on smithi131 with /dev/vg_nvme/lv_1... 2024-05-10T05:30:09.932 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T05:30:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:09 smithi131 ceph-mon[37264]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:09.961 INFO:journalctl@ceph.osd.8.smithi157.stdout:-- Logs begin at Fri 2024-05-10 05:00:45 UTC. -- 2024-05-10T05:30:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:30:11.553 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:30:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:30:11.173+0000 7fe312c6b7c0 -1 osd.8 0 log_to_monitors true 2024-05-10T05:30:12.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:11 smithi142 ceph-mon[77667]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:12.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:11 smithi142 ceph-mon[77667]: from='osd.8 [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T05:30:12.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:11 smithi142 ceph-mon[77667]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T05:30:12.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:11 smithi131 ceph-mon[37264]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:12.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:11 smithi131 ceph-mon[37264]: from='osd.8 [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T05:30:12.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:11 smithi131 ceph-mon[37264]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T05:30:12.218 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T05:30:12.218 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T05:30:12.218 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-10T05:30:12.218 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-10T05:30:12.218 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0369081 s, 284 MB/s 2024-05-10T05:30:12.219 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-10T05:30:12.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:11 smithi157 ceph-mon[78427]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:12.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:11 smithi157 ceph-mon[78427]: from='osd.8 [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T05:30:12.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:11 smithi157 ceph-mon[78427]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T05:30:12.678 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2024-05-10T05:30:12.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T05:30:12.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='osd.8 [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi157 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi157 to 5704M 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:12.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:13.080 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:30:13.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='osd.8 [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi157 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi157 to 5704M 2024-05-10T05:30:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:13.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:13.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:13.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:13.303 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:30:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:30:12.877+0000 7fe3082cf700 -1 osd.8 0 waiting for initial osdmap 2024-05-10T05:30:13.303 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:30:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:30:12.889+0000 7fe304086700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:30:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T05:30:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T05:30:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='osd.8 [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi157 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi157 to 5704M 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:14.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:13 smithi142 ceph-mon[77667]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:14.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:13 smithi142 ceph-mon[77667]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:30:14.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:13 smithi142 ceph-mon[77667]: osdmap e55: 9 total, 8 up, 9 in 2024-05-10T05:30:14.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:14.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:14.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:13 smithi142 ceph-mon[77667]: osd.8 [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] boot 2024-05-10T05:30:14.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:13 smithi142 ceph-mon[77667]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T05:30:14.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:14.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:13 smithi131 ceph-mon[37264]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:14.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:13 smithi131 ceph-mon[37264]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:30:14.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:13 smithi131 ceph-mon[37264]: osdmap e55: 9 total, 8 up, 9 in 2024-05-10T05:30:14.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:14.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:14.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:13 smithi131 ceph-mon[37264]: osd.8 [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] boot 2024-05-10T05:30:14.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:13 smithi131 ceph-mon[37264]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T05:30:14.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:13 smithi157 ceph-mon[78427]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-05-10T05:30:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:13 smithi157 ceph-mon[78427]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:30:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:13 smithi157 ceph-mon[78427]: osdmap e55: 9 total, 8 up, 9 in 2024-05-10T05:30:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:13 smithi157 ceph-mon[78427]: osd.8 [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] boot 2024-05-10T05:30:14.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:13 smithi157 ceph-mon[78427]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T05:30:14.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T05:30:14.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.342+0000 7f0a0d400700 1 -- 172.21.15.131:0/1228422293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a081035b0 msgr2=0x7f0a08109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.342+0000 7f0a0d400700 1 --2- 172.21.15.131:0/1228422293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a081035b0 0x7f0a08109e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000b0a0 tx=0x7f0a0000b3b0 comp rx=0 tx=0).stop 2024-05-10T05:30:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.342+0000 7f0a0d400700 1 -- 172.21.15.131:0/1228422293 shutdown_connections 2024-05-10T05:30:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.342+0000 7f0a0d400700 1 --2- 172.21.15.131:0/1228422293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a08101ae0 0x7f0a08101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.342+0000 7f0a0d400700 1 --2- 172.21.15.131:0/1228422293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a081035b0 0x7f0a08109e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.342+0000 7f0a0d400700 1 --2- 172.21.15.131:0/1228422293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08102c10 0x7f0a08103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.342+0000 7f0a0d400700 1 -- 172.21.15.131:0/1228422293 >> 172.21.15.131:0/1228422293 conn(0x7f0a080fd120 msgr2=0x7f0a080ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:30:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.342+0000 7f0a0d400700 1 -- 172.21.15.131:0/1228422293 shutdown_connections 2024-05-10T05:30:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.342+0000 7f0a0d400700 1 -- 172.21.15.131:0/1228422293 wait complete. 2024-05-10T05:30:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.343+0000 7f0a0d400700 1 Processor -- start 2024-05-10T05:30:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.343+0000 7f0a0d400700 1 -- start start 2024-05-10T05:30:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a0d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08101ae0 0x7f0a0819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a0d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a08102c10 0x7f0a0819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a0d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a081035b0 0x7f0a081a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a0d400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a081a1c60 con 0x7f0a08102c10 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a0d400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a081a1da0 con 0x7f0a081035b0 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a0d400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a081a1f10 con 0x7f0a08101ae0 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a07e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08101ae0 0x7f0a0819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a07400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a08102c10 0x7f0a0819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a07e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08101ae0 0x7f0a0819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51096/0 (socket says 172.21.15.131:51096) 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a07e00700 1 -- 172.21.15.131:0/4257635161 learned_addr learned my addr 172.21.15.131:0/4257635161 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.344+0000 7f0a0ca00700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a081035b0 0x7f0a081a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a07e00700 1 -- 172.21.15.131:0/4257635161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a081035b0 msgr2=0x7f0a081a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a07e00700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a081035b0 0x7f0a081a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a07e00700 1 -- 172.21.15.131:0/4257635161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a08102c10 msgr2=0x7f0a0819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a07e00700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a08102c10 0x7f0a0819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a07e00700 1 -- 172.21.15.131:0/4257635161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f8009520 con 0x7f0a08101ae0 2024-05-10T05:30:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a07e00700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08101ae0 0x7f0a0819a1b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f09f800ea10 tx=0x7f09f800ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:30:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a04c00700 1 -- 172.21.15.131:0/4257635161 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f800cbd0 con 0x7f0a08101ae0 2024-05-10T05:30:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a0d400700 1 -- 172.21.15.131:0/4257635161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a00009d00 con 0x7f0a08101ae0 2024-05-10T05:30:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a04c00700 1 -- 172.21.15.131:0/4257635161 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f800eeb0 con 0x7f0a08101ae0 2024-05-10T05:30:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a04c00700 1 -- 172.21.15.131:0/4257635161 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8018680 con 0x7f0a08101ae0 2024-05-10T05:30:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.345+0000 7f0a0d400700 1 -- 172.21.15.131:0/4257635161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a081a25b0 con 0x7f0a08101ae0 2024-05-10T05:30:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.346+0000 7f0a0d400700 1 -- 172.21.15.131:0/4257635161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a08068a70 con 0x7f0a08101ae0 2024-05-10T05:30:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.348+0000 7f0a04c00700 1 -- 172.21.15.131:0/4257635161 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f09f8010430 con 0x7f0a08101ae0 2024-05-10T05:30:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.349+0000 7f0a04c00700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09ec0a1120 0x7f09ec0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.350+0000 7f0a04c00700 1 -- 172.21.15.131:0/4257635161 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f09f8014070 con 0x7f0a08101ae0 2024-05-10T05:30:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.350+0000 7f0a07400700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09ec0a1120 0x7f09ec0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.350+0000 7f0a07400700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09ec0a1120 0x7f09ec0a3630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0a0819b260 tx=0x7f09f4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:30:14.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.352+0000 7f0a04c00700 1 -- 172.21.15.131:0/4257635161 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f09f8096720 con 0x7f0a08101ae0 2024-05-10T05:30:14.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:14.566+0000 7f0a0d400700 1 -- 172.21.15.131:0/4257635161 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a08074b60 con 0x7f09ec0a1120 2024-05-10T05:30:15.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:14 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:30:15.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:14 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:30:15.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:14 smithi142 ceph-mon[77667]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T05:30:15.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:30:15.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:30:15.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:15.158 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:14 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:30:15.158 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:14 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:30:15.158 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:14 smithi131 ceph-mon[37264]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T05:30:15.158 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:30:15.159 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:30:15.159 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:14 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:30:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:14 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:30:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:14 smithi157 ceph-mon[78427]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T05:30:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:30:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:30:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:16.137 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:15 smithi131 ceph-mon[37264]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:30:16.137 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:15 smithi131 ceph-mon[37264]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:16.137 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:15 smithi131 ceph-mon[37264]: osdmap e58: 9 total, 9 up, 9 in 2024-05-10T05:30:16.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:15 smithi142 ceph-mon[77667]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:30:16.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:15 smithi142 ceph-mon[77667]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:16.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:15 smithi142 ceph-mon[77667]: osdmap e58: 9 total, 9 up, 9 in 2024-05-10T05:30:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:15 smithi157 ceph-mon[78427]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:30:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:15 smithi157 ceph-mon[78427]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:15 smithi157 ceph-mon[78427]: osdmap e58: 9 total, 9 up, 9 in 2024-05-10T05:30:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:17 smithi157 ceph-mon[78427]: osdmap e59: 9 total, 9 up, 9 in 2024-05-10T05:30:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:17 smithi157 ceph-mon[78427]: pgmap v170: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-05-10T05:30:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:17 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/535200655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e98635-37d4-43eb-a609-a43df9f89afd"}]: dispatch 2024-05-10T05:30:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:17 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/535200655' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31e98635-37d4-43eb-a609-a43df9f89afd"}]': finished 2024-05-10T05:30:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:17 smithi157 ceph-mon[78427]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T05:30:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:17 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:17.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:17 smithi142 ceph-mon[77667]: osdmap e59: 9 total, 9 up, 9 in 2024-05-10T05:30:17.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:17 smithi142 ceph-mon[77667]: pgmap v170: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-05-10T05:30:17.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:17 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/535200655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e98635-37d4-43eb-a609-a43df9f89afd"}]: dispatch 2024-05-10T05:30:17.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:17 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/535200655' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31e98635-37d4-43eb-a609-a43df9f89afd"}]': finished 2024-05-10T05:30:17.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:17 smithi142 ceph-mon[77667]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T05:30:17.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:17 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:17.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:17 smithi131 ceph-mon[37264]: osdmap e59: 9 total, 9 up, 9 in 2024-05-10T05:30:17.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:17 smithi131 ceph-mon[37264]: pgmap v170: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-05-10T05:30:17.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:17 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/535200655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e98635-37d4-43eb-a609-a43df9f89afd"}]: dispatch 2024-05-10T05:30:17.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:17 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/535200655' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31e98635-37d4-43eb-a609-a43df9f89afd"}]': finished 2024-05-10T05:30:17.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:17 smithi131 ceph-mon[37264]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T05:30:17.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:17 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:18.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:18 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/48910750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:30:18.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:18 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/48910750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:30:18.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:18 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/48910750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:30:19.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:19 smithi157 ceph-mon[78427]: pgmap v172: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-05-10T05:30:19.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:19 smithi142 ceph-mon[77667]: pgmap v172: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-05-10T05:30:19.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:19 smithi131 ceph-mon[37264]: pgmap v172: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-05-10T05:30:22.128 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:21 smithi131 ceph-mon[37264]: pgmap v173: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T05:30:22.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:21 smithi142 ceph-mon[77667]: pgmap v173: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T05:30:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:21 smithi157 ceph-mon[78427]: pgmap v173: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T05:30:24.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:23 smithi142 ceph-mon[77667]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-10T05:30:24.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:23 smithi131 ceph-mon[37264]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-10T05:30:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:23 smithi157 ceph-mon[78427]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-10T05:30:25.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:24 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T05:30:25.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:24 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:25.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:24 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T05:30:25.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:24 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:24 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T05:30:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:24 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:25 smithi142 ceph-mon[77667]: Deploying daemon osd.9 on smithi131 2024-05-10T05:30:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:25 smithi142 ceph-mon[77667]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:25 smithi131 ceph-mon[37264]: Deploying daemon osd.9 on smithi131 2024-05-10T05:30:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:25 smithi131 ceph-mon[37264]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:25 smithi157 ceph-mon[78427]: Deploying daemon osd.9 on smithi131 2024-05-10T05:30:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:25 smithi157 ceph-mon[78427]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:28.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:27 smithi142 ceph-mon[77667]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:28.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:27 smithi131 ceph-mon[37264]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:27 smithi157 ceph-mon[78427]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:30.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:29 smithi142 ceph-mon[77667]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:30.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:29 smithi131 ceph-mon[37264]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:29 smithi157 ceph-mon[78427]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:31.593 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:31.593 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:31.593 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:31.593 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:31 smithi131 ceph-mon[37264]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:31.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:31.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:31.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:31.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:31 smithi157 ceph-mon[78427]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:31.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:31.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:31.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:31.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:31 smithi142 ceph-mon[77667]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:33.759 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:33.759 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:33.759 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:33.759 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:33.759 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:33.759 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:33 smithi131 ceph-mon[37264]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:33.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:33.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:33.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:33.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:33.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:33.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:33 smithi157 ceph-mon[78427]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:33.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:33.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:33.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:33.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:33.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:33.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:33 smithi142 ceph-mon[77667]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.018+0000 7f0a04c00700 1 -- 172.21.15.131:0/4257635161 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0a08074b60 con 0x7f09ec0a1120 2024-05-10T05:30:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 9 on host 'smithi131' 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.021+0000 7f0a0d400700 1 -- 172.21.15.131:0/4257635161 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09ec0a1120 msgr2=0x7f09ec0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.021+0000 7f0a0d400700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09ec0a1120 0x7f09ec0a3630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0a0819b260 tx=0x7f09f4009260 comp rx=0 tx=0).stop 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.021+0000 7f0a0d400700 1 -- 172.21.15.131:0/4257635161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08101ae0 msgr2=0x7f0a0819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.021+0000 7f0a0d400700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08101ae0 0x7f0a0819a1b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f09f800ea10 tx=0x7f09f800ed20 comp rx=0 tx=0).stop 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.022+0000 7f0a0d400700 1 -- 172.21.15.131:0/4257635161 shutdown_connections 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.022+0000 7f0a0d400700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09ec0a1120 0x7f09ec0a3630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.022+0000 7f0a0d400700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a081035b0 0x7f0a081a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.022+0000 7f0a0d400700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a08101ae0 0x7f0a0819a1b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.022+0000 7f0a0d400700 1 --2- 172.21.15.131:0/4257635161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a08102c10 0x7f0a0819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.022+0000 7f0a0d400700 1 -- 172.21.15.131:0/4257635161 >> 172.21.15.131:0/4257635161 conn(0x7f0a080fd120 msgr2=0x7f0a08073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.022+0000 7f0a0d400700 1 -- 172.21.15.131:0/4257635161 shutdown_connections 2024-05-10T05:30:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:30:34.022+0000 7f0a0d400700 1 -- 172.21.15.131:0/4257635161 wait complete. 2024-05-10T05:30:34.441 DEBUG:teuthology.orchestra.run.smithi131:osd.9> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.9.service 2024-05-10T05:30:34.446 INFO:tasks.cephadm:Deploying osd.10 on smithi142 with /dev/vg_nvme/lv_1... 2024-05-10T05:30:34.446 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T05:30:34.517 INFO:journalctl@ceph.osd.9.smithi131.stdout:-- Logs begin at Fri 2024-05-10 05:06:53 UTC. -- 2024-05-10T05:30:34.677 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:30:34.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:30:34 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:30:34.605+0000 7fe9c04ea7c0 -1 osd.9 0 log_to_monitors true 2024-05-10T05:30:35.108 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:35.108 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:35.108 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:35.108 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:34 smithi142 ceph-mon[77667]: from='osd.9 [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T05:30:35.108 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:34 smithi142 ceph-mon[77667]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T05:30:35.242 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:35.243 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:35.243 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:35.243 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:34 smithi131 ceph-mon[37264]: from='osd.9 [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T05:30:35.243 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:34 smithi131 ceph-mon[37264]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T05:30:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:34 smithi157 ceph-mon[78427]: from='osd.9 [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T05:30:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:34 smithi157 ceph-mon[78427]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T05:30:35.977 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T05:30:35.977 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T05:30:35.978 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-10T05:30:35.978 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-10T05:30:35.978 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0153403 s, 684 MB/s 2024-05-10T05:30:35.978 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-10T05:30:36.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:36 smithi142 ceph-mon[77667]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:36.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:36 smithi142 ceph-mon[77667]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T05:30:36.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:36 smithi142 ceph-mon[77667]: osdmap e61: 10 total, 9 up, 10 in 2024-05-10T05:30:36.297 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:36 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:36.298 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:36 smithi142 ceph-mon[77667]: from='osd.9 [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:30:36.298 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:36 smithi142 ceph-mon[77667]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:30:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:36 smithi157 ceph-mon[78427]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:36 smithi157 ceph-mon[78427]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T05:30:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:36 smithi157 ceph-mon[78427]: osdmap e61: 10 total, 9 up, 10 in 2024-05-10T05:30:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:36 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:36 smithi157 ceph-mon[78427]: from='osd.9 [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:30:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:36 smithi157 ceph-mon[78427]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:30:36.354 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2024-05-10T05:30:36.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:36 smithi131 ceph-mon[37264]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:36.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:36 smithi131 ceph-mon[37264]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T05:30:36.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:36 smithi131 ceph-mon[37264]: osdmap e61: 10 total, 9 up, 10 in 2024-05-10T05:30:36.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:36 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:36.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:36 smithi131 ceph-mon[37264]: from='osd.9 [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:30:36.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:36 smithi131 ceph-mon[37264]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-10T05:30:36.573 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:30:36.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:30:36 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:30:36.608+0000 7fe9b4a00700 -1 osd.9 0 waiting for initial osdmap 2024-05-10T05:30:36.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:30:36 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:30:36.642+0000 7fe9aa800700 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:30:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:37 smithi157 ceph-mon[78427]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:30:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:37 smithi157 ceph-mon[78427]: osdmap e62: 10 total, 9 up, 10 in 2024-05-10T05:30:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:37 smithi157 ceph-mon[78427]: from='osd.9 ' entity='osd.9' 2024-05-10T05:30:37.350 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:37 smithi131 ceph-mon[37264]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:30:37.350 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:37 smithi131 ceph-mon[37264]: osdmap e62: 10 total, 9 up, 10 in 2024-05-10T05:30:37.350 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:37.350 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:37.350 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:37 smithi131 ceph-mon[37264]: from='osd.9 ' entity='osd.9' 2024-05-10T05:30:37.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:37 smithi142 ceph-mon[77667]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-10T05:30:37.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:37 smithi142 ceph-mon[77667]: osdmap e62: 10 total, 9 up, 10 in 2024-05-10T05:30:37.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:37.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:37.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:37 smithi142 ceph-mon[77667]: from='osd.9 ' entity='osd.9' 2024-05-10T05:30:37.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.598+0000 7f7713e93700 1 -- 172.21.15.142:0/3729435686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f770c102e80 msgr2=0x7f770c103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:37.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.598+0000 7f7713e93700 1 --2- 172.21.15.142:0/3729435686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f770c102e80 0x7f770c103290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7708009990 tx=0x7f7708009ca0 comp rx=0 tx=0).stop 2024-05-10T05:30:37.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.598+0000 7f7713e93700 1 -- 172.21.15.142:0/3729435686 shutdown_connections 2024-05-10T05:30:37.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.598+0000 7f7713e93700 1 --2- 172.21.15.142:0/3729435686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c103820 0x7f770c10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:37.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.598+0000 7f7713e93700 1 --2- 172.21.15.142:0/3729435686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c101d50 0x7f770c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:37.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.598+0000 7f7713e93700 1 --2- 172.21.15.142:0/3729435686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f770c102e80 0x7f770c103290 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:37.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.598+0000 7f7713e93700 1 -- 172.21.15.142:0/3729435686 >> 172.21.15.142:0/3729435686 conn(0x7f770c0fd390 msgr2=0x7f770c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:30:37.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.599+0000 7f7713e93700 1 -- 172.21.15.142:0/3729435686 shutdown_connections 2024-05-10T05:30:37.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.599+0000 7f7713e93700 1 -- 172.21.15.142:0/3729435686 wait complete. 2024-05-10T05:30:37.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.599+0000 7f7713e93700 1 Processor -- start 2024-05-10T05:30:37.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7713e93700 1 -- start start 2024-05-10T05:30:37.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7713e93700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c101d50 0x7f770c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:37.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7713e93700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c102e80 0x7f770c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:37.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7713e93700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f770c103820 0x7f770c1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:37.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7713e93700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c199b20 con 0x7f770c102e80 2024-05-10T05:30:37.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7713e93700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c199c60 con 0x7f770c103820 2024-05-10T05:30:37.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7713e93700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c199e20 con 0x7f770c101d50 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7712430700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f770c103820 0x7f770c1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7712430700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f770c103820 0x7f770c1994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43492/0 (socket says 172.21.15.142:43492) 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7712430700 1 -- 172.21.15.142:0/2272941587 learned_addr learned my addr 172.21.15.142:0/2272941587 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7711c2f700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c101d50 0x7f770c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f771142e700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c102e80 0x7f770c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7712430700 1 -- 172.21.15.142:0/2272941587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c101d50 msgr2=0x7f770c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7712430700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c101d50 0x7f770c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7712430700 1 -- 172.21.15.142:0/2272941587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c102e80 msgr2=0x7f770c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7712430700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c102e80 0x7f770c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.600+0000 7f7712430700 1 -- 172.21.15.142:0/2272941587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77080095f0 con 0x7f770c103820 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.601+0000 7f7712430700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f770c103820 0x7f770c1994c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7704007ee0 tx=0x7f770400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:30:37.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.601+0000 7f7702ffd700 1 -- 172.21.15.142:0/2272941587 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704017070 con 0x7f770c103820 2024-05-10T05:30:37.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.601+0000 7f7702ffd700 1 -- 172.21.15.142:0/2272941587 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f770400f040 con 0x7f770c103820 2024-05-10T05:30:37.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.601+0000 7f7713e93700 1 -- 172.21.15.142:0/2272941587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c19a100 con 0x7f770c103820 2024-05-10T05:30:37.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.601+0000 7f7702ffd700 1 -- 172.21.15.142:0/2272941587 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704005740 con 0x7f770c103820 2024-05-10T05:30:37.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.601+0000 7f7713e93700 1 -- 172.21.15.142:0/2272941587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c1a08e0 con 0x7f770c103820 2024-05-10T05:30:37.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.602+0000 7f7713e93700 1 -- 172.21.15.142:0/2272941587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c108f70 con 0x7f770c103820 2024-05-10T05:30:37.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.603+0000 7f7702ffd700 1 -- 172.21.15.142:0/2272941587 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f770400dc60 con 0x7f770c103820 2024-05-10T05:30:37.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.603+0000 7f7702ffd700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f76f80a1220 0x7f76f80a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:37.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.604+0000 7f7702ffd700 1 -- 172.21.15.142:0/2272941587 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5981+0+0 (secure 0 0 0) 0x7f77040cb7e0 con 0x7f770c103820 2024-05-10T05:30:37.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.604+0000 7f7711c2f700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f76f80a1220 0x7f76f80a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:37.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.604+0000 7f7711c2f700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f76f80a1220 0x7f76f80a3730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f770c102d80 tx=0x7f76fc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:30:37.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.608+0000 7f7702ffd700 1 -- 172.21.15.142:0/2272941587 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f77040957f0 con 0x7f770c103820 2024-05-10T05:30:37.812 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:37.811+0000 7f7713e93700 1 -- 172.21.15.142:0/2272941587 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f770c10d6c0 con 0x7f76f80a1220 2024-05-10T05:30:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:30:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:30:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:38.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:38.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: osd.9 [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] boot 2024-05-10T05:30:38.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: osdmap e63: 10 total, 10 up, 10 in 2024-05-10T05:30:38.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:38.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:30:38.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:30:38.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:38.382 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:30:38.382 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:30:38.382 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: osd.9 [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] boot 2024-05-10T05:30:38.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: osdmap e63: 10 total, 10 up, 10 in 2024-05-10T05:30:38.384 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:38.384 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:30:38.384 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:30:38.384 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:38.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:30:38.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:30:38.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-05-10T05:30:38.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:38.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: osd.9 [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] boot 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: osdmap e63: 10 total, 10 up, 10 in 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:30:38.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:30:38.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:39.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:39 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi131 2024-05-10T05:30:39.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:39 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi131 to 4317M 2024-05-10T05:30:39.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:39 smithi142 ceph-mon[77667]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:30:39.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:39 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi131 2024-05-10T05:30:39.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:39 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi131 to 4317M 2024-05-10T05:30:39.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:39 smithi157 ceph-mon[78427]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:30:39.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:39 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi131 2024-05-10T05:30:39.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:39 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi131 to 4317M 2024-05-10T05:30:39.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:39 smithi131 ceph-mon[37264]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:30:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:40 smithi157 ceph-mon[78427]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:40 smithi157 ceph-mon[78427]: osdmap e64: 10 total, 10 up, 10 in 2024-05-10T05:30:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:40 smithi157 ceph-mon[78427]: from='client.? 172.21.15.142:0/1908284610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a073aaf2-1cff-4d5e-8cf7-20f41deeab54"}]: dispatch 2024-05-10T05:30:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:40 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a073aaf2-1cff-4d5e-8cf7-20f41deeab54"}]: dispatch 2024-05-10T05:30:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:40 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a073aaf2-1cff-4d5e-8cf7-20f41deeab54"}]': finished 2024-05-10T05:30:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:40 smithi157 ceph-mon[78427]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T05:30:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:40 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:40 smithi142 ceph-mon[77667]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:40 smithi142 ceph-mon[77667]: osdmap e64: 10 total, 10 up, 10 in 2024-05-10T05:30:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:40 smithi142 ceph-mon[77667]: from='client.? 172.21.15.142:0/1908284610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a073aaf2-1cff-4d5e-8cf7-20f41deeab54"}]: dispatch 2024-05-10T05:30:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:40 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a073aaf2-1cff-4d5e-8cf7-20f41deeab54"}]: dispatch 2024-05-10T05:30:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:40 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a073aaf2-1cff-4d5e-8cf7-20f41deeab54"}]': finished 2024-05-10T05:30:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:40 smithi142 ceph-mon[77667]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T05:30:40.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:40 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:40 smithi131 ceph-mon[37264]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:40 smithi131 ceph-mon[37264]: osdmap e64: 10 total, 10 up, 10 in 2024-05-10T05:30:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:40 smithi131 ceph-mon[37264]: from='client.? 172.21.15.142:0/1908284610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a073aaf2-1cff-4d5e-8cf7-20f41deeab54"}]: dispatch 2024-05-10T05:30:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:40 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a073aaf2-1cff-4d5e-8cf7-20f41deeab54"}]: dispatch 2024-05-10T05:30:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:40 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a073aaf2-1cff-4d5e-8cf7-20f41deeab54"}]': finished 2024-05-10T05:30:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:40 smithi131 ceph-mon[37264]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T05:30:40.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:40 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:41.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:41 smithi157 ceph-mon[78427]: from='client.? 172.21.15.142:0/4120320637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:30:41.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:41 smithi142 ceph-mon[77667]: from='client.? 172.21.15.142:0/4120320637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:30:41.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:41 smithi131 ceph-mon[37264]: from='client.? 172.21.15.142:0/4120320637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:30:42.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:42 smithi157 ceph-mon[78427]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:42.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:42 smithi142 ceph-mon[77667]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:42.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:42 smithi131 ceph-mon[37264]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:44 smithi157 ceph-mon[78427]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:44.304 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:44 smithi142 ceph-mon[77667]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:44.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:44 smithi131 ceph-mon[37264]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:46.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:46 smithi142 ceph-mon[77667]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:46.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T05:30:46.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:46 smithi157 ceph-mon[78427]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:46.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T05:30:46.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:46.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:46 smithi131 ceph-mon[37264]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:46.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T05:30:46.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:47.285 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:47 smithi142 ceph-mon[77667]: Deploying daemon osd.10 on smithi142 2024-05-10T05:30:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:47 smithi157 ceph-mon[78427]: Deploying daemon osd.10 on smithi142 2024-05-10T05:30:47.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:47 smithi131 ceph-mon[37264]: Deploying daemon osd.10 on smithi142 2024-05-10T05:30:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:48 smithi157 ceph-mon[78427]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:48.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:48 smithi142 ceph-mon[77667]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:48.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:48 smithi131 ceph-mon[37264]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:49.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:49 smithi157 ceph-mon[78427]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:49.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:49 smithi142 ceph-mon[77667]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:49.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:49 smithi131 ceph-mon[37264]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:50.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:50 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:50.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:50 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:50.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:50 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:50.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:50 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:50.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:50 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:50.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:50 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:50.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:50 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:50.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:50 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:50.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:50 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:51.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.717+0000 7f7702ffd700 1 -- 172.21.15.142:0/2272941587 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f770c10d6c0 con 0x7f76f80a1220 2024-05-10T05:30:51.717 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 10 on host 'smithi142' 2024-05-10T05:30:51.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 -- 172.21.15.142:0/2272941587 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f76f80a1220 msgr2=0x7f76f80a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:51.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f76f80a1220 0x7f76f80a3730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f770c102d80 tx=0x7f76fc009260 comp rx=0 tx=0).stop 2024-05-10T05:30:51.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 -- 172.21.15.142:0/2272941587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f770c103820 msgr2=0x7f770c1994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:51.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f770c103820 0x7f770c1994c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7704007ee0 tx=0x7f770400d3b0 comp rx=0 tx=0).stop 2024-05-10T05:30:51.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 -- 172.21.15.142:0/2272941587 shutdown_connections 2024-05-10T05:30:51.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f76f80a1220 0x7f76f80a3730 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:51.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f770c103820 0x7f770c1994c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:51.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f770c101d50 0x7f770c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:51.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 --2- 172.21.15.142:0/2272941587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f770c102e80 0x7f770c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:51.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 -- 172.21.15.142:0/2272941587 >> 172.21.15.142:0/2272941587 conn(0x7f770c0fd390 msgr2=0x7f770c1060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:30:51.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 -- 172.21.15.142:0/2272941587 shutdown_connections 2024-05-10T05:30:51.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T05:30:51.719+0000 7f7713e93700 1 -- 172.21.15.142:0/2272941587 wait complete. 2024-05-10T05:30:52.101 DEBUG:teuthology.orchestra.run.smithi142:osd.10> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.10.service 2024-05-10T05:30:52.104 INFO:tasks.cephadm:Deploying osd.11 on smithi157 with /dev/vg_nvme/lv_1... 2024-05-10T05:30:52.104 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T05:30:52.137 INFO:journalctl@ceph.osd.10.smithi142.stdout:-- Logs begin at Fri 2024-05-10 04:59:48 UTC. -- 2024-05-10T05:30:52.221 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:51 smithi142 ceph-mon[77667]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:52.222 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.222 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.222 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:52.222 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:52.222 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.222 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:51 smithi157 ceph-mon[78427]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:52.340 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:30:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:51 smithi131 ceph-mon[37264]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:52.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:30:53.668 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T05:30:53.668 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T05:30:53.668 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-10T05:30:53.668 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-10T05:30:53.668 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00865798 s, 1.2 GB/s 2024-05-10T05:30:53.668 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-10T05:30:53.879 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:30:53 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:30:53.498+0000 7f38e7c917c0 -1 osd.10 0 log_to_monitors true 2024-05-10T05:30:54.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2024-05-10T05:30:54.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:53 smithi142 ceph-mon[77667]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:54.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:53 smithi142 ceph-mon[77667]: from='osd.10 [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T05:30:54.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:53 smithi142 ceph-mon[77667]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T05:30:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:30:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:53 smithi157 ceph-mon[78427]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:53 smithi157 ceph-mon[78427]: from='osd.10 [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T05:30:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:53 smithi157 ceph-mon[78427]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T05:30:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:53 smithi131 ceph-mon[37264]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:53 smithi131 ceph-mon[37264]: from='osd.10 [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T05:30:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:53 smithi131 ceph-mon[37264]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T05:30:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.243+0000 7ff6fe11b700 1 -- 172.21.15.157:0/2185668315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8100bd0 msgr2=0x7ff6f8100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.243+0000 7ff6fe11b700 1 --2- 172.21.15.157:0/2185668315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8100bd0 0x7ff6f8100fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009990 tx=0x7ff6e0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:30:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.243+0000 7ff6fe11b700 1 -- 172.21.15.157:0/2185668315 shutdown_connections 2024-05-10T05:30:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.243+0000 7ff6fe11b700 1 --2- 172.21.15.157:0/2185668315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6f81026a0 0x7ff6f8108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.243+0000 7ff6fe11b700 1 --2- 172.21.15.157:0/2185668315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8100bd0 0x7ff6f8100fe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.243+0000 7ff6fe11b700 1 --2- 172.21.15.157:0/2185668315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6f8101d00 0x7ff6f8102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.243+0000 7ff6fe11b700 1 -- 172.21.15.157:0/2185668315 >> 172.21.15.157:0/2185668315 conn(0x7ff6f80fc230 msgr2=0x7ff6f80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:30:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.243+0000 7ff6fe11b700 1 -- 172.21.15.157:0/2185668315 shutdown_connections 2024-05-10T05:30:55.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.243+0000 7ff6fe11b700 1 -- 172.21.15.157:0/2185668315 wait complete. 2024-05-10T05:30:55.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6fe11b700 1 Processor -- start 2024-05-10T05:30:55.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6fe11b700 1 -- start start 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6fe11b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6f8100bd0 0x7ff6f819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6fe11b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8101d00 0x7ff6f819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6fe11b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6f81026a0 0x7ff6f81a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6fe11b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f806a6d0 con 0x7ff6f81026a0 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6fe11b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f806a840 con 0x7ff6f8100bd0 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6fe11b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f81a21d0 con 0x7ff6f8101d00 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6f6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8101d00 0x7ff6f819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6f77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6f8100bd0 0x7ff6f819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6f6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8101d00 0x7ff6f819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38642/0 (socket says 172.21.15.157:38642) 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.244+0000 7ff6f6ffd700 1 -- 172.21.15.157:0/1693453787 learned_addr learned my addr 172.21.15.157:0/1693453787 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-10T05:30:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f7fff700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6f81026a0 0x7ff6f81a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f6ffd700 1 -- 172.21.15.157:0/1693453787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6f8100bd0 msgr2=0x7ff6f819d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f6ffd700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6f8100bd0 0x7ff6f819d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f6ffd700 1 -- 172.21.15.157:0/1693453787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6f81026a0 msgr2=0x7ff6f81a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f6ffd700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6f81026a0 0x7ff6f81a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f6ffd700 1 -- 172.21.15.157:0/1693453787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e00095f0 con 0x7ff6f8101d00 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f77fe700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6f8100bd0 0x7ff6f819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f6ffd700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8101d00 0x7ff6f819f740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800d950 tx=0x7ff6e800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f4ff9700 1 -- 172.21.15.157:0/1693453787 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8009940 con 0x7ff6f8101d00 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f4ff9700 1 -- 172.21.15.157:0/1693453787 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6e8010460 con 0x7ff6f8101d00 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6fe11b700 1 -- 172.21.15.157:0/1693453787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f81a24b0 con 0x7ff6f8101d00 2024-05-10T05:30:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6f4ff9700 1 -- 172.21.15.157:0/1693453787 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800f620 con 0x7ff6f8101d00 2024-05-10T05:30:55.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.245+0000 7ff6fe11b700 1 -- 172.21.15.157:0/1693453787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f81a29b0 con 0x7ff6f8101d00 2024-05-10T05:30:55.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.246+0000 7ff6fe11b700 1 -- 172.21.15.157:0/1693453787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f8107df0 con 0x7ff6f8101d00 2024-05-10T05:30:55.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.247+0000 7ff6f4ff9700 1 -- 172.21.15.157:0/1693453787 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff6e80108d0 con 0x7ff6f8101d00 2024-05-10T05:30:55.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.248+0000 7ff6f4ff9700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6e40a1120 0x7ff6e40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:30:55.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.248+0000 7ff6f4ff9700 1 -- 172.21.15.157:0/1693453787 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff6e80cc0b0 con 0x7ff6f8101d00 2024-05-10T05:30:55.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.248+0000 7ff6f77fe700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6e40a1120 0x7ff6e40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:30:55.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.249+0000 7ff6f77fe700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6e40a1120 0x7ff6e40a3630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff6f806a270 tx=0x7ff6e0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:30:55.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.252+0000 7ff6f4ff9700 1 -- 172.21.15.157:0/1693453787 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff6e8095f10 con 0x7ff6f8101d00 2024-05-10T05:30:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T05:30:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: osdmap e66: 11 total, 10 up, 11 in 2024-05-10T05:30:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='osd.10 [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:30:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:30:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:55.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:30:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:30:55.004+0000 7f38dd2f5700 -1 osd.10 0 waiting for initial osdmap 2024-05-10T05:30:55.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:30:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:30:55.019+0000 7f38d90ac700 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:30:55.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: osdmap e66: 11 total, 10 up, 11 in 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='osd.10 [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:55.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:55.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:30:55.453+0000 7ff6fe11b700 1 -- 172.21.15.157:0/1693453787 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6f80008d0 con 0x7ff6e40a1120 2024-05-10T05:30:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:54 smithi131 ceph-mon[37264]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T05:30:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:54 smithi131 ceph-mon[37264]: osdmap e66: 11 total, 10 up, 11 in 2024-05-10T05:30:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:54 smithi131 ceph-mon[37264]: from='osd.10 [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:30:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:54 smithi131 ceph-mon[37264]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T05:30:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:30:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:30:55.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi142 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi142 to 4278M 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: osdmap e67: 11 total, 10 up, 11 in 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: osd.10 [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] boot 2024-05-10T05:30:56.195 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T05:30:56.196 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi142 2024-05-10T05:30:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi142 to 4278M 2024-05-10T05:30:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:30:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: osdmap e67: 11 total, 10 up, 11 in 2024-05-10T05:30:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:30:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:30:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: osd.10 [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] boot 2024-05-10T05:30:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T05:30:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi142 2024-05-10T05:30:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi142 to 4278M 2024-05-10T05:30:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 894 GiB / 894 GiB avail 2024-05-10T05:30:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T05:30:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: osdmap e67: 11 total, 10 up, 11 in 2024-05-10T05:30:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T05:30:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T05:30:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:30:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: osd.10 [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] boot 2024-05-10T05:30:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T05:30:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T05:30:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:57 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:30:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:57 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:30:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:57 smithi157 ceph-mon[78427]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:30:57.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:57 smithi157 ceph-mon[78427]: osdmap e69: 11 total, 11 up, 11 in 2024-05-10T05:30:57.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:57 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:30:57.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:57 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:30:57.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:57 smithi142 ceph-mon[77667]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:30:57.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:57 smithi142 ceph-mon[77667]: osdmap e69: 11 total, 11 up, 11 in 2024-05-10T05:30:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:57 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:30:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:57 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:30:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:57 smithi131 ceph-mon[37264]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:30:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:57 smithi131 ceph-mon[37264]: osdmap e69: 11 total, 11 up, 11 in 2024-05-10T05:30:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:58 smithi157 ceph-mon[78427]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:30:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.157:0/3615394370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d1207f7-17c7-4e22-85e7-a589fd87f906"}]: dispatch 2024-05-10T05:30:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:58 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d1207f7-17c7-4e22-85e7-a589fd87f906"}]: dispatch 2024-05-10T05:30:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:58 smithi157 ceph-mon[78427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d1207f7-17c7-4e22-85e7-a589fd87f906"}]': finished 2024-05-10T05:30:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:58 smithi157 ceph-mon[78427]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T05:30:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:30:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.157:0/923295857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:30:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:58 smithi142 ceph-mon[77667]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:30:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.157:0/3615394370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d1207f7-17c7-4e22-85e7-a589fd87f906"}]: dispatch 2024-05-10T05:30:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:58 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d1207f7-17c7-4e22-85e7-a589fd87f906"}]: dispatch 2024-05-10T05:30:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:58 smithi142 ceph-mon[77667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d1207f7-17c7-4e22-85e7-a589fd87f906"}]': finished 2024-05-10T05:30:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:58 smithi142 ceph-mon[77667]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T05:30:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:30:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.157:0/923295857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:30:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:58 smithi131 ceph-mon[37264]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:30:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.157:0/3615394370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d1207f7-17c7-4e22-85e7-a589fd87f906"}]: dispatch 2024-05-10T05:30:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:58 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d1207f7-17c7-4e22-85e7-a589fd87f906"}]: dispatch 2024-05-10T05:30:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:58 smithi131 ceph-mon[37264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d1207f7-17c7-4e22-85e7-a589fd87f906"}]': finished 2024-05-10T05:30:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:58 smithi131 ceph-mon[37264]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T05:30:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:30:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.157:0/923295857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T05:30:59.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:59 smithi142 ceph-mon[77667]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T05:30:59.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:59 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:30:59.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:30:59 smithi142 ceph-mon[77667]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:30:59.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:59 smithi131 ceph-mon[37264]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T05:30:59.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:59 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:30:59.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:30:59 smithi131 ceph-mon[37264]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:30:59.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:59 smithi157 ceph-mon[78427]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T05:30:59.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:59 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:30:59.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:30:59 smithi157 ceph-mon[78427]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:31:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:00 smithi142 ceph-mon[77667]: osdmap e72: 12 total, 11 up, 12 in 2024-05-10T05:31:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:00 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:00.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:00 smithi131 ceph-mon[37264]: osdmap e72: 12 total, 11 up, 12 in 2024-05-10T05:31:00.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:00 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:00 smithi157 ceph-mon[78427]: osdmap e72: 12 total, 11 up, 12 in 2024-05-10T05:31:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:00 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:01.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:01 smithi142 ceph-mon[77667]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:31:01.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:01 smithi131 ceph-mon[37264]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:31:01.784 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:01 smithi157 ceph-mon[78427]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:31:04.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:03 smithi142 ceph-mon[77667]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-10T05:31:04.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:03 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T05:31:04.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:03 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:04.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:03 smithi131 ceph-mon[37264]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-10T05:31:04.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T05:31:04.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:03 smithi157 ceph-mon[78427]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-10T05:31:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T05:31:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:05.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:04 smithi157 ceph-mon[78427]: Deploying daemon osd.11 on smithi157 2024-05-10T05:31:05.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:04 smithi142 ceph-mon[77667]: Deploying daemon osd.11 on smithi157 2024-05-10T05:31:05.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:04 smithi131 ceph-mon[37264]: Deploying daemon osd.11 on smithi157 2024-05-10T05:31:06.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:05 smithi157 ceph-mon[78427]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-10T05:31:06.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:05 smithi142 ceph-mon[77667]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-10T05:31:06.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:05 smithi131 ceph-mon[37264]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-10T05:31:08.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:07 smithi157 ceph-mon[78427]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-10T05:31:08.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:08.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:08.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:08.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:07 smithi142 ceph-mon[77667]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-10T05:31:08.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:08.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:08.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:08.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:07 smithi131 ceph-mon[37264]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-10T05:31:08.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:08.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:08.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:09.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.059+0000 7ff6f4ff9700 1 -- 172.21.15.157:0/1693453787 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff6f80008d0 con 0x7ff6e40a1120 2024-05-10T05:31:09.060 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 11 on host 'smithi157' 2024-05-10T05:31:09.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 -- 172.21.15.157:0/1693453787 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6e40a1120 msgr2=0x7ff6e40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:09.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6e40a1120 0x7ff6e40a3630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff6f806a270 tx=0x7ff6e0005fb0 comp rx=0 tx=0).stop 2024-05-10T05:31:09.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 -- 172.21.15.157:0/1693453787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8101d00 msgr2=0x7ff6f819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:09.062 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8101d00 0x7ff6f819f740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800d950 tx=0x7ff6e800dc60 comp rx=0 tx=0).stop 2024-05-10T05:31:09.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 -- 172.21.15.157:0/1693453787 shutdown_connections 2024-05-10T05:31:09.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6e40a1120 0x7ff6e40a3630 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:09.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6f81026a0 0x7ff6f81a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:09.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6f8100bd0 0x7ff6f819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:09.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 --2- 172.21.15.157:0/1693453787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8101d00 0x7ff6f819f740 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:09.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 -- 172.21.15.157:0/1693453787 >> 172.21.15.157:0/1693453787 conn(0x7ff6f80fc230 msgr2=0x7ff6f8104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:09.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 -- 172.21.15.157:0/1693453787 shutdown_connections 2024-05-10T05:31:09.063 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:09.062+0000 7ff6fe11b700 1 -- 172.21.15.157:0/1693453787 wait complete. 2024-05-10T05:31:09.432 DEBUG:teuthology.orchestra.run.smithi157:osd.11> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.11.service 2024-05-10T05:31:09.434 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-10T05:31:09.434 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T05:31:09.464 INFO:journalctl@ceph.osd.11.smithi157.stdout:-- Logs begin at Fri 2024-05-10 05:00:45 UTC. -- 2024-05-10T05:31:09.798 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:31:09.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:09.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:09.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:09 smithi157 ceph-mon[78427]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T05:31:09.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:09.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:09 smithi142 ceph-mon[77667]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T05:31:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:09.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:09 smithi131 ceph-mon[37264]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T05:31:09.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:09.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:09.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:11.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.040+0000 7ffaad000700 1 -- 172.21.15.131:0/4058431204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa8101ca0 msgr2=0x7ffaa81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:11.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.040+0000 7ffaad000700 1 --2- 172.21.15.131:0/4058431204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa8101ca0 0x7ffaa81020b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa980098f0 tx=0x7ffa98009c00 comp rx=0 tx=0).stop 2024-05-10T05:31:11.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.040+0000 7ffaad000700 1 -- 172.21.15.131:0/4058431204 shutdown_connections 2024-05-10T05:31:11.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.040+0000 7ffaad000700 1 --2- 172.21.15.131:0/4058431204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa8103770 0x7ffaa810a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:11.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.040+0000 7ffaad000700 1 --2- 172.21.15.131:0/4058431204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa8101ca0 0x7ffaa81020b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:11.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.040+0000 7ffaad000700 1 --2- 172.21.15.131:0/4058431204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa8102dd0 0x7ffaa81031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:11.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.040+0000 7ffaad000700 1 -- 172.21.15.131:0/4058431204 >> 172.21.15.131:0/4058431204 conn(0x7ffaa80fd300 msgr2=0x7ffaa80ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:11.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.040+0000 7ffaad000700 1 -- 172.21.15.131:0/4058431204 shutdown_connections 2024-05-10T05:31:11.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.041+0000 7ffaad000700 1 -- 172.21.15.131:0/4058431204 wait complete. 2024-05-10T05:31:11.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaad000700 1 Processor -- start 2024-05-10T05:31:11.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaad000700 1 -- start start 2024-05-10T05:31:11.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaad000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa8101ca0 0x7ffaa819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaad000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa8102dd0 0x7ffaa819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaad000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa8103770 0x7ffaa819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaad000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa819bd30 con 0x7ffaa8102dd0 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaad000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa819be70 con 0x7ffaa8101ca0 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaad000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa819c030 con 0x7ffaa8103770 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaa5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa8102dd0 0x7ffaa819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaa5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa8102dd0 0x7ffaa819a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60630/0 (socket says 172.21.15.131:60630) 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaa5400700 1 -- 172.21.15.131:0/2658543219 learned_addr learned my addr 172.21.15.131:0/2658543219 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaa6800700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa8103770 0x7ffaa819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.042+0000 7ffaa5e00700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa8101ca0 0x7ffaa819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.043+0000 7ffaa5400700 1 -- 172.21.15.131:0/2658543219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa8103770 msgr2=0x7ffaa819b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.043+0000 7ffaa5400700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa8103770 0x7ffaa819b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.043+0000 7ffaa5400700 1 -- 172.21.15.131:0/2658543219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa8101ca0 msgr2=0x7ffaa819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:11.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.043+0000 7ffaa5400700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa8101ca0 0x7ffaa819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:11.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.043+0000 7ffaa5400700 1 -- 172.21.15.131:0/2658543219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa0009d00 con 0x7ffaa8102dd0 2024-05-10T05:31:11.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.043+0000 7ffaa5400700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa8102dd0 0x7ffaa819a920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400e9a0 tx=0x7ffa9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:11.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.043+0000 7ffa9ea00700 1 -- 172.21.15.131:0/2658543219 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94009790 con 0x7ffaa8102dd0 2024-05-10T05:31:11.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.043+0000 7ffa9ea00700 1 -- 172.21.15.131:0/2658543219 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa94010430 con 0x7ffaa8102dd0 2024-05-10T05:31:11.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.043+0000 7ffaad000700 1 -- 172.21.15.131:0/2658543219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa98009520 con 0x7ffaa8102dd0 2024-05-10T05:31:11.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.043+0000 7ffa9ea00700 1 -- 172.21.15.131:0/2658543219 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94018680 con 0x7ffaa8102dd0 2024-05-10T05:31:11.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.044+0000 7ffaad000700 1 -- 172.21.15.131:0/2658543219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa81a2920 con 0x7ffaa8102dd0 2024-05-10T05:31:11.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.045+0000 7ffaad000700 1 -- 172.21.15.131:0/2658543219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa8100fa0 con 0x7ffaa8102dd0 2024-05-10T05:31:11.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.047+0000 7ffa9ea00700 1 -- 172.21.15.131:0/2658543219 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ffa940105a0 con 0x7ffaa8102dd0 2024-05-10T05:31:11.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.048+0000 7ffa9ea00700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffa8c0a1120 0x7ffa8c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:11.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.048+0000 7ffa9ea00700 1 -- 172.21.15.131:0/2658543219 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ffa94014070 con 0x7ffaa8102dd0 2024-05-10T05:31:11.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.048+0000 7ffaa5e00700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffa8c0a1120 0x7ffa8c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:11.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.049+0000 7ffaa5e00700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffa8c0a1120 0x7ffa8c0a3630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa98000c00 tx=0x7ffa98019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:11.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.053+0000 7ffa9ea00700 1 -- 172.21.15.131:0/2658543219 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffa94052d60 con 0x7ffaa8102dd0 2024-05-10T05:31:11.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.256+0000 7ffaad000700 1 -- 172.21.15.131:0/2658543219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ffaa81a2b10 con 0x7ffaa8102dd0 2024-05-10T05:31:11.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.257+0000 7ffa9ea00700 1 -- 172.21.15.131:0/2658543219 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) v1 ==== 74+0+133 (secure 0 0 0) 0x7ffa94096230 con 0x7ffaa8102dd0 2024-05-10T05:31:11.258 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:11.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 -- 172.21.15.131:0/2658543219 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffa8c0a1120 msgr2=0x7ffa8c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:11.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffa8c0a1120 0x7ffa8c0a3630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa98000c00 tx=0x7ffa98019040 comp rx=0 tx=0).stop 2024-05-10T05:31:11.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 -- 172.21.15.131:0/2658543219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa8102dd0 msgr2=0x7ffaa819a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:11.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa8102dd0 0x7ffaa819a920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400e9a0 tx=0x7ffa9400ecb0 comp rx=0 tx=0).stop 2024-05-10T05:31:11.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 -- 172.21.15.131:0/2658543219 shutdown_connections 2024-05-10T05:31:11.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffa8c0a1120 0x7ffa8c0a3630 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaa8103770 0x7ffaa819b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa8101ca0 0x7ffaa819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 --2- 172.21.15.131:0/2658543219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa8102dd0 0x7ffaa819a920 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 -- 172.21.15.131:0/2658543219 >> 172.21.15.131:0/2658543219 conn(0x7ffaa80fd300 msgr2=0x7ffaa8073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.259+0000 7ffaad000700 1 -- 172.21.15.131:0/2658543219 shutdown_connections 2024-05-10T05:31:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:11.260+0000 7ffaad000700 1 -- 172.21.15.131:0/2658543219 wait complete. 2024-05-10T05:31:11.303 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:31:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:31:11.027+0000 7fa9b6a527c0 -1 osd.11 0 log_to_monitors true 2024-05-10T05:31:11.669 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1715319055,"num_in_osds":12,"osd_in_since":1715319057,"num_remapped_pgs":0} 2024-05-10T05:31:11.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:11 smithi131 ceph-mon[37264]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-10T05:31:11.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:11 smithi131 ceph-mon[37264]: from='osd.11 [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T05:31:11.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:11 smithi131 ceph-mon[37264]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T05:31:11.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:11 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2658543219' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T05:31:12.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:11 smithi157 ceph-mon[78427]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-10T05:31:12.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:11 smithi157 ceph-mon[78427]: from='osd.11 [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T05:31:12.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:11 smithi157 ceph-mon[78427]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T05:31:12.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:11 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2658543219' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T05:31:12.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:11 smithi142 ceph-mon[77667]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-10T05:31:12.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:11 smithi142 ceph-mon[77667]: from='osd.11 [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T05:31:12.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:11 smithi142 ceph-mon[77667]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T05:31:12.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:11 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2658543219' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T05:31:12.670 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T05:31:13.032 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:31:13.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T05:31:13.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: osdmap e73: 12 total, 11 up, 12 in 2024-05-10T05:31:13.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:13.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='osd.11 [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:31:13.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: Detected new or changed devices on smithi157 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi157 to 4278M 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:13.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T05:31:13.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: osdmap e73: 12 total, 11 up, 12 in 2024-05-10T05:31:13.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:13.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='osd.11 [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: Detected new or changed devices on smithi157 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi157 to 4278M 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:13.193 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.303 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:31:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:31:12.895+0000 7fa9ac0b6700 -1 osd.11 0 waiting for initial osdmap 2024-05-10T05:31:13.303 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:31:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:31:12.915+0000 7fa9a7e6d700 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T05:31:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T05:31:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: osdmap e73: 12 total, 11 up, 12 in 2024-05-10T05:31:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:13.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='osd.11 [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: Detected new or changed devices on smithi157 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi157 to 4278M 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:13.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:13.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:14.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:13 smithi131 ceph-mon[37264]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:31:14.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:13 smithi131 ceph-mon[37264]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:31:14.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:13 smithi131 ceph-mon[37264]: osdmap e74: 12 total, 11 up, 12 in 2024-05-10T05:31:14.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:14.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:14.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.279+0000 7f30d6a00700 1 -- 172.21.15.131:0/1043068153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30d01050e0 msgr2=0x7f30d0107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:14.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.279+0000 7f30d6a00700 1 --2- 172.21.15.131:0/1043068153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30d01050e0 0x7f30d0107500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009990 tx=0x7f30bc009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.280+0000 7f30d6a00700 1 -- 172.21.15.131:0/1043068153 shutdown_connections 2024-05-10T05:31:14.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.280+0000 7f30d6a00700 1 --2- 172.21.15.131:0/1043068153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d0107a40 0x7f30d0109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.280+0000 7f30d6a00700 1 --2- 172.21.15.131:0/1043068153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d0102780 0x7f30d0104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.280+0000 7f30d6a00700 1 --2- 172.21.15.131:0/1043068153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30d01050e0 0x7f30d0107500 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.280+0000 7f30d6a00700 1 -- 172.21.15.131:0/1043068153 >> 172.21.15.131:0/1043068153 conn(0x7f30d00fc1c0 msgr2=0x7f30d00fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:14.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.280+0000 7f30d6a00700 1 -- 172.21.15.131:0/1043068153 shutdown_connections 2024-05-10T05:31:14.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.280+0000 7f30d6a00700 1 -- 172.21.15.131:0/1043068153 wait complete. 2024-05-10T05:31:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.281+0000 7f30d6a00700 1 Processor -- start 2024-05-10T05:31:14.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.281+0000 7f30d6a00700 1 -- start start 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.281+0000 7f30d6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d0102780 0x7f30d01021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.281+0000 7f30d6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30d01050e0 0x7f30d0100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.281+0000 7f30d6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d0107a40 0x7f30d0100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.281+0000 7f30d6a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d0101420 con 0x7f30d01050e0 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.281+0000 7f30d6a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01a2160 con 0x7f30d0107a40 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.281+0000 7f30d6a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01a22a0 con 0x7f30d0102780 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30cf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30d01050e0 0x7f30d0100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30cfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d0102780 0x7f30d01021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30cf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30d01050e0 0x7f30d0100690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41116/0 (socket says 172.21.15.131:41116) 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30cf400700 1 -- 172.21.15.131:0/3236869188 learned_addr learned my addr 172.21.15.131:0/3236869188 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30d4a00700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d0107a40 0x7f30d0100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30cf400700 1 -- 172.21.15.131:0/3236869188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d0102780 msgr2=0x7f30d01021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30cf400700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d0102780 0x7f30d01021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30cf400700 1 -- 172.21.15.131:0/3236869188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d0107a40 msgr2=0x7f30d0100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:14.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30cf400700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d0107a40 0x7f30d0100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30cf400700 1 -- 172.21.15.131:0/3236869188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc0095f0 con 0x7f30d01050e0 2024-05-10T05:31:14.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30cf400700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30d01050e0 0x7f30d0100690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f30bc006010 tx=0x7f30bc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:14.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.282+0000 7f30ccc00700 1 -- 172.21.15.131:0/3236869188 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc01d070 con 0x7f30d01050e0 2024-05-10T05:31:14.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.283+0000 7f30ccc00700 1 -- 172.21.15.131:0/3236869188 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30bc005510 con 0x7f30d01050e0 2024-05-10T05:31:14.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.283+0000 7f30d6a00700 1 -- 172.21.15.131:0/3236869188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d01a24a0 con 0x7f30d01050e0 2024-05-10T05:31:14.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.283+0000 7f30ccc00700 1 -- 172.21.15.131:0/3236869188 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc004b30 con 0x7f30d01050e0 2024-05-10T05:31:14.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.283+0000 7f30d6a00700 1 -- 172.21.15.131:0/3236869188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d01a28b0 con 0x7f30d01050e0 2024-05-10T05:31:14.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.284+0000 7f30d6a00700 1 -- 172.21.15.131:0/3236869188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d00fdd70 con 0x7f30d01050e0 2024-05-10T05:31:14.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.286+0000 7f30ccc00700 1 -- 172.21.15.131:0/3236869188 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f30bc00fb50 con 0x7f30d01050e0 2024-05-10T05:31:14.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.287+0000 7f30ccc00700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f30b40a1120 0x7f30b40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:14.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.287+0000 7f30ccc00700 1 -- 172.21.15.131:0/3236869188 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f30bc0cd8d0 con 0x7f30d01050e0 2024-05-10T05:31:14.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.287+0000 7f30cfe00700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f30b40a1120 0x7f30b40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:14.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.288+0000 7f30cfe00700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f30b40a1120 0x7f30b40a3630 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f30d0101f00 tx=0x7f30c00073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:14.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.290+0000 7f30ccc00700 1 -- 172.21.15.131:0/3236869188 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f30bc09b0a0 con 0x7f30d01050e0 2024-05-10T05:31:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:13 smithi157 ceph-mon[78427]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:31:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:13 smithi157 ceph-mon[78427]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:31:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:13 smithi157 ceph-mon[78427]: osdmap e74: 12 total, 11 up, 12 in 2024-05-10T05:31:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:13 smithi142 ceph-mon[77667]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:31:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:13 smithi142 ceph-mon[77667]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-10T05:31:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:13 smithi142 ceph-mon[77667]: osdmap e74: 12 total, 11 up, 12 in 2024-05-10T05:31:14.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:14.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:14.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.497+0000 7f30d6a00700 1 -- 172.21.15.131:0/3236869188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f30d00ffe80 con 0x7f30d01050e0 2024-05-10T05:31:14.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.497+0000 7f30ccc00700 1 -- 172.21.15.131:0/3236869188 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f30bc09ac30 con 0x7f30d01050e0 2024-05-10T05:31:14.499 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:14.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 -- 172.21.15.131:0/3236869188 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f30b40a1120 msgr2=0x7f30b40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:14.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f30b40a1120 0x7f30b40a3630 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f30d0101f00 tx=0x7f30c00073d0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 -- 172.21.15.131:0/3236869188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30d01050e0 msgr2=0x7f30d0100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:14.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30d01050e0 0x7f30d0100690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f30bc006010 tx=0x7f30bc00f690 comp rx=0 tx=0).stop 2024-05-10T05:31:14.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 -- 172.21.15.131:0/3236869188 shutdown_connections 2024-05-10T05:31:14.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f30b40a1120 0x7f30b40a3630 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d0107a40 0x7f30d0100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30d0102780 0x7f30d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 --2- 172.21.15.131:0/3236869188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30d01050e0 0x7f30d0100690 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:14.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 -- 172.21.15.131:0/3236869188 >> 172.21.15.131:0/3236869188 conn(0x7f30d00fc1c0 msgr2=0x7f30d01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:14.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.500+0000 7f30d6a00700 1 -- 172.21.15.131:0/3236869188 shutdown_connections 2024-05-10T05:31:14.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:14.501+0000 7f30d6a00700 1 -- 172.21.15.131:0/3236869188 wait complete. 2024-05-10T05:31:14.912 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1715319073,"num_in_osds":12,"osd_in_since":1715319057,"num_remapped_pgs":0} 2024-05-10T05:31:14.913 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T05:31:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:14 smithi131 ceph-mon[37264]: purged_snaps scrub starts 2024-05-10T05:31:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:14 smithi131 ceph-mon[37264]: purged_snaps scrub ok 2024-05-10T05:31:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:14 smithi131 ceph-mon[37264]: osd.11 [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] boot 2024-05-10T05:31:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:14 smithi131 ceph-mon[37264]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T05:31:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:14 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3236869188' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T05:31:15.272 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:31:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:14 smithi157 ceph-mon[78427]: purged_snaps scrub starts 2024-05-10T05:31:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:14 smithi157 ceph-mon[78427]: purged_snaps scrub ok 2024-05-10T05:31:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:14 smithi157 ceph-mon[78427]: osd.11 [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] boot 2024-05-10T05:31:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:14 smithi157 ceph-mon[78427]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T05:31:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:14 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3236869188' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T05:31:15.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:14 smithi142 ceph-mon[77667]: purged_snaps scrub starts 2024-05-10T05:31:15.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:14 smithi142 ceph-mon[77667]: purged_snaps scrub ok 2024-05-10T05:31:15.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:14 smithi142 ceph-mon[77667]: osd.11 [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] boot 2024-05-10T05:31:15.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:14 smithi142 ceph-mon[77667]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T05:31:15.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T05:31:15.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:14 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3236869188' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T05:31:16.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:15 smithi131 ceph-mon[37264]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:31:16.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:15 smithi131 ceph-mon[37264]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T05:31:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:15 smithi157 ceph-mon[78427]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:31:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:15 smithi157 ceph-mon[78427]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T05:31:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:15 smithi142 ceph-mon[77667]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-05-10T05:31:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:15 smithi142 ceph-mon[77667]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T05:31:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.577+0000 7fdd97600700 1 -- 172.21.15.131:0/4132249022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd90101f30 msgr2=0x7fdd90102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.577+0000 7fdd97600700 1 --2- 172.21.15.131:0/4132249022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd90101f30 0x7fdd90102340 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009990 tx=0x7fdd84009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.578+0000 7fdd97600700 1 -- 172.21.15.131:0/4132249022 shutdown_connections 2024-05-10T05:31:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.578+0000 7fdd97600700 1 --2- 172.21.15.131:0/4132249022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd90101f30 0x7fdd90102340 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.578+0000 7fdd97600700 1 --2- 172.21.15.131:0/4132249022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd901071b0 0x7fdd9010eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.578+0000 7fdd97600700 1 --2- 172.21.15.131:0/4132249022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd90106860 0x7fdd90106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.578+0000 7fdd97600700 1 -- 172.21.15.131:0/4132249022 >> 172.21.15.131:0/4132249022 conn(0x7fdd900fd9c0 msgr2=0x7fdd900ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:16.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.578+0000 7fdd97600700 1 -- 172.21.15.131:0/4132249022 shutdown_connections 2024-05-10T05:31:16.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.578+0000 7fdd97600700 1 -- 172.21.15.131:0/4132249022 wait complete. 2024-05-10T05:31:16.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.579+0000 7fdd97600700 1 Processor -- start 2024-05-10T05:31:16.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.579+0000 7fdd97600700 1 -- start start 2024-05-10T05:31:16.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd97600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd90101f30 0x7fdd901a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:16.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd97600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd90106860 0x7fdd901a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:16.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd97600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd901071b0 0x7fdd9019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:16.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd97600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9019ab00 con 0x7fdd90106860 2024-05-10T05:31:16.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd97600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9019ac40 con 0x7fdd901071b0 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd97600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9019ae00 con 0x7fdd90101f30 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd8fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd90106860 0x7fdd901a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd8fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd90106860 0x7fdd901a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41146/0 (socket says 172.21.15.131:41146) 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd8fe00700 1 -- 172.21.15.131:0/2892150247 learned_addr learned my addr 172.21.15.131:0/2892150247 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd94c00700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd90101f30 0x7fdd901a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd95600700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd901071b0 0x7fdd9019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd8fe00700 1 -- 172.21.15.131:0/2892150247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd90101f30 msgr2=0x7fdd901a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd8fe00700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd90101f30 0x7fdd901a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd8fe00700 1 -- 172.21.15.131:0/2892150247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd901071b0 msgr2=0x7fdd9019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd8fe00700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd901071b0 0x7fdd9019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.580+0000 7fdd8fe00700 1 -- 172.21.15.131:0/2892150247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840095f0 con 0x7fdd90106860 2024-05-10T05:31:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.581+0000 7fdd8fe00700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd90106860 0x7fdd901a0b10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00e970 tx=0x7fdd7c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.581+0000 7fdd8d600700 1 -- 172.21.15.131:0/2892150247 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c009790 con 0x7fdd90106860 2024-05-10T05:31:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.581+0000 7fdd8d600700 1 -- 172.21.15.131:0/2892150247 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd7c00c7e0 con 0x7fdd90106860 2024-05-10T05:31:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.581+0000 7fdd97600700 1 -- 172.21.15.131:0/2892150247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9019b0e0 con 0x7fdd90106860 2024-05-10T05:31:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.581+0000 7fdd8d600700 1 -- 172.21.15.131:0/2892150247 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0054e0 con 0x7fdd90106860 2024-05-10T05:31:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.581+0000 7fdd97600700 1 -- 172.21.15.131:0/2892150247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd901a75a0 con 0x7fdd90106860 2024-05-10T05:31:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.582+0000 7fdd97600700 1 -- 172.21.15.131:0/2892150247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd901033e0 con 0x7fdd90106860 2024-05-10T05:31:16.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.584+0000 7fdd8d600700 1 -- 172.21.15.131:0/2892150247 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fdd7c00c950 con 0x7fdd90106860 2024-05-10T05:31:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.586+0000 7fdd8d600700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd780a1120 0x7fdd780a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:16.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.586+0000 7fdd8d600700 1 -- 172.21.15.131:0/2892150247 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdd7c014070 con 0x7fdd90106860 2024-05-10T05:31:16.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.586+0000 7fdd94c00700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd780a1120 0x7fdd780a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:16.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.587+0000 7fdd94c00700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd780a1120 0x7fdd780a3630 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400b2e0 tx=0x7fdd8401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.588+0000 7fdd8d600700 1 -- 172.21.15.131:0/2892150247 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdd7c0964f0 con 0x7fdd90106860 2024-05-10T05:31:16.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.792+0000 7fdd97600700 1 -- 172.21.15.131:0/2892150247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd90101500 con 0x7fdd90106860 2024-05-10T05:31:16.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.793+0000 7fdd8d600700 1 -- 172.21.15.131:0/2892150247 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18494 (secure 0 0 0) 0x7fdd7c099b10 con 0x7fdd90106860 2024-05-10T05:31:16.795 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:16.795 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":76,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:31:14.901400+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:31:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 -- 172.21.15.131:0/2892150247 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd780a1120 msgr2=0x7fdd780a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd780a1120 0x7fdd780a3630 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400b2e0 tx=0x7fdd8401a040 comp rx=0 tx=0).stop 2024-05-10T05:31:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 -- 172.21.15.131:0/2892150247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd90106860 msgr2=0x7fdd901a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd90106860 0x7fdd901a0b10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00e970 tx=0x7fdd7c00ec80 comp rx=0 tx=0).stop 2024-05-10T05:31:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 -- 172.21.15.131:0/2892150247 shutdown_connections 2024-05-10T05:31:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd780a1120 0x7fdd780a3630 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd901071b0 0x7fdd9019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd90101f30 0x7fdd901a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 --2- 172.21.15.131:0/2892150247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd90106860 0x7fdd901a0b10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 -- 172.21.15.131:0/2892150247 >> 172.21.15.131:0/2892150247 conn(0x7fdd900fd9c0 msgr2=0x7fdd9010dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 -- 172.21.15.131:0/2892150247 shutdown_connections 2024-05-10T05:31:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:16.796+0000 7fdd97600700 1 -- 172.21.15.131:0/2892150247 wait complete. 2024-05-10T05:31:17.160 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:16 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2892150247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:17.189 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:31:17.190 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-10T05:31:17.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:16 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2892150247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:17.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:16 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2892150247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:17.545 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:31:18.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:17 smithi131 ceph-mon[37264]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:18.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:17 smithi157 ceph-mon[78427]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:18.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:17 smithi142 ceph-mon[77667]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.968+0000 7f4bf6800700 1 -- 172.21.15.131:0/316052966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf0101f40 msgr2=0x7f4bf0102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.968+0000 7f4bf6800700 1 --2- 172.21.15.131:0/316052966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf0101f40 0x7f4bf0102350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009990 tx=0x7f4be0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.969+0000 7f4bf6800700 1 -- 172.21.15.131:0/316052966 shutdown_connections 2024-05-10T05:31:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.969+0000 7f4bf6800700 1 --2- 172.21.15.131:0/316052966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf0101f40 0x7f4bf0102350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.969+0000 7f4bf6800700 1 --2- 172.21.15.131:0/316052966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf01071c0 0x7f4bf010eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.969+0000 7f4bf6800700 1 --2- 172.21.15.131:0/316052966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0106870 0x7f4bf0106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.969+0000 7f4bf6800700 1 -- 172.21.15.131:0/316052966 >> 172.21.15.131:0/316052966 conn(0x7f4bf00fd7f0 msgr2=0x7f4bf00ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.969+0000 7f4bf6800700 1 -- 172.21.15.131:0/316052966 shutdown_connections 2024-05-10T05:31:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.969+0000 7f4bf6800700 1 -- 172.21.15.131:0/316052966 wait complete. 2024-05-10T05:31:18.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.970+0000 7f4bf6800700 1 Processor -- start 2024-05-10T05:31:18.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.970+0000 7f4bf6800700 1 -- start start 2024-05-10T05:31:18.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.970+0000 7f4bf6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0101f40 0x7f4bf0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:18.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.970+0000 7f4bf6800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf0106870 0x7f4bf0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:18.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.970+0000 7f4bf6800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf01071c0 0x7f4bf019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:18.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.970+0000 7f4bf6800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf019d530 con 0x7f4bf0106870 2024-05-10T05:31:18.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.970+0000 7f4bf6800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf019d6a0 con 0x7f4bf0101f40 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.970+0000 7f4bf6800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf019d860 con 0x7f4bf01071c0 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.970+0000 7f4be7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf0106870 0x7f4bf0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4be7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf0106870 0x7f4bf0198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41160/0 (socket says 172.21.15.131:41160) 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4be7e00700 1 -- 172.21.15.131:0/3234556562 learned_addr learned my addr 172.21.15.131:0/3234556562 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4bef400700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0101f40 0x7f4bf0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4befe00700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf01071c0 0x7f4bf019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4be7e00700 1 -- 172.21.15.131:0/3234556562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf01071c0 msgr2=0x7f4bf019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4be7e00700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf01071c0 0x7f4bf019cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4be7e00700 1 -- 172.21.15.131:0/3234556562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0101f40 msgr2=0x7f4bf0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4be7e00700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0101f40 0x7f4bf0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4be7e00700 1 -- 172.21.15.131:0/3234556562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00095f0 con 0x7f4bf0106870 2024-05-10T05:31:18.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4be7e00700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf0106870 0x7f4bf0198ab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800d980 tx=0x7f4bd800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:18.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.971+0000 7f4becc00700 1 -- 172.21.15.131:0/3234556562 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800f890 con 0x7f4bf0106870 2024-05-10T05:31:18.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.972+0000 7f4becc00700 1 -- 172.21.15.131:0/3234556562 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bd8010460 con 0x7f4bf0106870 2024-05-10T05:31:18.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.972+0000 7f4bf6800700 1 -- 172.21.15.131:0/3234556562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf019db40 con 0x7f4bf0106870 2024-05-10T05:31:18.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.972+0000 7f4becc00700 1 -- 172.21.15.131:0/3234556562 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800e610 con 0x7f4bf0106870 2024-05-10T05:31:18.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.972+0000 7f4bf6800700 1 -- 172.21.15.131:0/3234556562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf01a5440 con 0x7f4bf0106870 2024-05-10T05:31:18.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.973+0000 7f4bf6800700 1 -- 172.21.15.131:0/3234556562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf010a210 con 0x7f4bf0106870 2024-05-10T05:31:18.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.975+0000 7f4becc00700 1 -- 172.21.15.131:0/3234556562 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4bd80105d0 con 0x7f4bf0106870 2024-05-10T05:31:18.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.976+0000 7f4becc00700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4bd40a11a0 0x7f4bd40a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:18.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.976+0000 7f4becc00700 1 -- 172.21.15.131:0/3234556562 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4bd80cbe60 con 0x7f4bf0106870 2024-05-10T05:31:18.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.976+0000 7f4bef400700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4bd40a11a0 0x7f4bd40a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:18.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.977+0000 7f4bef400700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4bd40a11a0 0x7f4bd40a36b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4be00098e0 tx=0x7f4be0005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:18.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:18.981+0000 7f4becc00700 1 -- 172.21.15.131:0/3234556562 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4bd80959f0 con 0x7f4bf0106870 2024-05-10T05:31:19.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.182+0000 7f4bf6800700 1 -- 172.21.15.131:0/3234556562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bf004edc0 con 0x7f4bf0106870 2024-05-10T05:31:19.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.183+0000 7f4becc00700 1 -- 172.21.15.131:0/3234556562 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bd8099010 con 0x7f4bf0106870 2024-05-10T05:31:19.184 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:31:19.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.185+0000 7f4bf6800700 1 -- 172.21.15.131:0/3234556562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4bd40a11a0 msgr2=0x7f4bd40a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:19.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.185+0000 7f4bf6800700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4bd40a11a0 0x7f4bd40a36b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4be00098e0 tx=0x7f4be0005960 comp rx=0 tx=0).stop 2024-05-10T05:31:19.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.185+0000 7f4bf6800700 1 -- 172.21.15.131:0/3234556562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf0106870 msgr2=0x7f4bf0198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.185+0000 7f4bf6800700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf0106870 0x7f4bf0198ab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800d980 tx=0x7f4bd800dc90 comp rx=0 tx=0).stop 2024-05-10T05:31:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.185+0000 7f4bf6800700 1 -- 172.21.15.131:0/3234556562 shutdown_connections 2024-05-10T05:31:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.185+0000 7f4bf6800700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4bd40a11a0 0x7f4bd40a36b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.185+0000 7f4bf6800700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bf01071c0 0x7f4bf019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.185+0000 7f4bf6800700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0101f40 0x7f4bf0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.185+0000 7f4bf6800700 1 --2- 172.21.15.131:0/3234556562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf0106870 0x7f4bf0198ab0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.185+0000 7f4bf6800700 1 -- 172.21.15.131:0/3234556562 >> 172.21.15.131:0/3234556562 conn(0x7f4bf00fd7f0 msgr2=0x7f4bf00ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.186+0000 7f4bf6800700 1 -- 172.21.15.131:0/3234556562 shutdown_connections 2024-05-10T05:31:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:19.186+0000 7f4bf6800700 1 -- 172.21.15.131:0/3234556562 wait complete. 2024-05-10T05:31:19.620 INFO:tasks.cephadm:Adding mds.a on smithi131 2024-05-10T05:31:19.620 INFO:tasks.cephadm:Adding mds.d on smithi131 2024-05-10T05:31:19.621 INFO:tasks.cephadm:Adding mds.g on smithi131 2024-05-10T05:31:19.621 INFO:tasks.cephadm:Adding mds.j on smithi131 2024-05-10T05:31:19.621 INFO:tasks.cephadm:Adding mds.b on smithi142 2024-05-10T05:31:19.621 INFO:tasks.cephadm:Adding mds.e on smithi142 2024-05-10T05:31:19.621 INFO:tasks.cephadm:Adding mds.h on smithi142 2024-05-10T05:31:19.621 INFO:tasks.cephadm:Adding mds.k on smithi142 2024-05-10T05:31:19.621 INFO:tasks.cephadm:Adding mds.c on smithi157 2024-05-10T05:31:19.621 INFO:tasks.cephadm:Adding mds.f on smithi157 2024-05-10T05:31:19.621 INFO:tasks.cephadm:Adding mds.i on smithi157 2024-05-10T05:31:19.621 INFO:tasks.cephadm:Adding mds.l on smithi157 2024-05-10T05:31:19.621 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l' 2024-05-10T05:31:19.860 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:31:20.181 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:19 smithi157 ceph-mon[78427]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:20.181 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:19 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3234556562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:31:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:19 smithi131 ceph-mon[37264]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:19 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3234556562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:31:20.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:19 smithi142 ceph-mon[77667]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:20.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:19 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3234556562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:31:20.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.855+0000 7f62a97d7700 1 -- 172.21.15.157:0/580239913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a41037b0 msgr2=0x7f62a410a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:20.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.855+0000 7f62a97d7700 1 --2- 172.21.15.157:0/580239913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a41037b0 0x7f62a410a060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f629c00b0a0 tx=0x7f629c00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:31:20.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.855+0000 7f62a97d7700 1 -- 172.21.15.157:0/580239913 shutdown_connections 2024-05-10T05:31:20.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.855+0000 7f62a97d7700 1 --2- 172.21.15.157:0/580239913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a41037b0 0x7f62a410a060 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:20.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.855+0000 7f62a97d7700 1 --2- 172.21.15.157:0/580239913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a4101ce0 0x7f62a41020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:20.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.855+0000 7f62a97d7700 1 --2- 172.21.15.157:0/580239913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4102e10 0x7f62a4103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:20.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.855+0000 7f62a97d7700 1 -- 172.21.15.157:0/580239913 >> 172.21.15.157:0/580239913 conn(0x7f62a40fd320 msgr2=0x7f62a40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:20.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.855+0000 7f62a97d7700 1 -- 172.21.15.157:0/580239913 shutdown_connections 2024-05-10T05:31:20.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.855+0000 7f62a97d7700 1 -- 172.21.15.157:0/580239913 wait complete. 2024-05-10T05:31:20.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a97d7700 1 Processor -- start 2024-05-10T05:31:20.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a97d7700 1 -- start start 2024-05-10T05:31:20.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a97d7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4101ce0 0x7f62a41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:20.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a97d7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4102e10 0x7f62a4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:20.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a97d7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a41037b0 0x7f62a41994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:20.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a97d7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a4199aa0 con 0x7f62a4101ce0 2024-05-10T05:31:20.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a97d7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a4199be0 con 0x7f62a41037b0 2024-05-10T05:31:20.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a97d7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a4199da0 con 0x7f62a4102e10 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a27fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4102e10 0x7f62a4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a27fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4102e10 0x7f62a4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51944/0 (socket says 172.21.15.157:51944) 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a27fc700 1 -- 172.21.15.157:0/4083595658 learned_addr learned my addr 172.21.15.157:0/4083595658 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a37fe700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a41037b0 0x7f62a41994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a27fc700 1 -- 172.21.15.157:0/4083595658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a41037b0 msgr2=0x7f62a41994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a27fc700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a41037b0 0x7f62a41994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a27fc700 1 -- 172.21.15.157:0/4083595658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4101ce0 msgr2=0x7f62a41981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a27fc700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4101ce0 0x7f62a41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.856+0000 7f62a27fc700 1 -- 172.21.15.157:0/4083595658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c009d00 con 0x7f62a4102e10 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.857+0000 7f62a27fc700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4102e10 0x7f62a4198720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f629800c9b0 tx=0x7f629800cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.857+0000 7f628bfff700 1 -- 172.21.15.157:0/4083595658 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298007b00 con 0x7f62a4102e10 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.857+0000 7f628bfff700 1 -- 172.21.15.157:0/4083595658 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6298007c60 con 0x7f62a4102e10 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.857+0000 7f62a97d7700 1 -- 172.21.15.157:0/4083595658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a419a080 con 0x7f62a4102e10 2024-05-10T05:31:20.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.857+0000 7f628bfff700 1 -- 172.21.15.157:0/4083595658 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298018660 con 0x7f62a4102e10 2024-05-10T05:31:20.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.857+0000 7f62a97d7700 1 -- 172.21.15.157:0/4083595658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a41a08f0 con 0x7f62a4102e10 2024-05-10T05:31:20.861 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.858+0000 7f62a97d7700 1 -- 172.21.15.157:0/4083595658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a4108f00 con 0x7f62a4102e10 2024-05-10T05:31:20.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.859+0000 7f628bfff700 1 -- 172.21.15.157:0/4083595658 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6298018880 con 0x7f62a4102e10 2024-05-10T05:31:20.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.860+0000 7f628bfff700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f628c0a1120 0x7f628c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:20.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.860+0000 7f628bfff700 1 -- 172.21.15.157:0/4083595658 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f62980cc7b0 con 0x7f62a4102e10 2024-05-10T05:31:20.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.860+0000 7f62a2ffd700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f628c0a1120 0x7f628c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:20.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.861+0000 7f62a2ffd700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f628c0a1120 0x7f628c0a3630 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f62a4102d10 tx=0x7f6294005ca0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:20.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:20.864+0000 7f628bfff700 1 -- 172.21.15.157:0/4083595658 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6298096470 con 0x7f62a4102e10 2024-05-10T05:31:21.068 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.067+0000 7f62a97d7700 1 -- 172.21.15.157:0/4083595658 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f62a4061650 con 0x7f628c0a1120 2024-05-10T05:31:21.077 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.076+0000 7f628bfff700 1 -- 172.21.15.157:0/4083595658 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f62a4061650 con 0x7f628c0a1120 2024-05-10T05:31:21.078 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mds.all update... 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 -- 172.21.15.157:0/4083595658 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f628c0a1120 msgr2=0x7f628c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f628c0a1120 0x7f628c0a3630 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f62a4102d10 tx=0x7f6294005ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 -- 172.21.15.157:0/4083595658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4102e10 msgr2=0x7f62a4198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4102e10 0x7f62a4198720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f629800c9b0 tx=0x7f629800cd70 comp rx=0 tx=0).stop 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 -- 172.21.15.157:0/4083595658 shutdown_connections 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f628c0a1120 0x7f628c0a3630 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4101ce0 0x7f62a41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a41037b0 0x7f62a41994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 --2- 172.21.15.157:0/4083595658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4102e10 0x7f62a4198720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 -- 172.21.15.157:0/4083595658 >> 172.21.15.157:0/4083595658 conn(0x7f62a40fd320 msgr2=0x7f62a4106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:21.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 -- 172.21.15.157:0/4083595658 shutdown_connections 2024-05-10T05:31:21.081 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-10T05:31:21.079+0000 7f62a97d7700 1 -- 172.21.15.157:0/4083595658 wait complete. 2024-05-10T05:31:21.455 DEBUG:teuthology.orchestra.run.smithi131:mds.a> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.a.service 2024-05-10T05:31:21.460 DEBUG:teuthology.orchestra.run.smithi131:mds.d> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.d.service 2024-05-10T05:31:21.505 DEBUG:teuthology.orchestra.run.smithi131:mds.g> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.g.service 2024-05-10T05:31:21.536 INFO:journalctl@ceph.mds.a.smithi131.stdout:-- Logs begin at Fri 2024-05-10 05:06:53 UTC. -- 2024-05-10T05:31:21.539 DEBUG:teuthology.orchestra.run.smithi131:mds.j> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.j.service 2024-05-10T05:31:21.578 INFO:journalctl@ceph.mds.d.smithi131.stdout:-- Logs begin at Fri 2024-05-10 05:06:53 UTC. -- 2024-05-10T05:31:21.581 DEBUG:teuthology.orchestra.run.smithi142:mds.b> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.b.service 2024-05-10T05:31:21.583 DEBUG:teuthology.orchestra.run.smithi142:mds.e> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.e.service 2024-05-10T05:31:21.609 INFO:journalctl@ceph.mds.g.smithi131.stdout:-- Logs begin at Fri 2024-05-10 05:06:53 UTC. -- 2024-05-10T05:31:21.622 INFO:journalctl@ceph.mds.b.smithi142.stdout:-- Logs begin at Fri 2024-05-10 04:59:48 UTC. -- 2024-05-10T05:31:21.623 DEBUG:teuthology.orchestra.run.smithi142:mds.h> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.h.service 2024-05-10T05:31:21.646 INFO:journalctl@ceph.mds.j.smithi131.stdout:-- Logs begin at Fri 2024-05-10 05:06:53 UTC. -- 2024-05-10T05:31:21.652 INFO:journalctl@ceph.mds.e.smithi142.stdout:-- Logs begin at Fri 2024-05-10 04:59:48 UTC. -- 2024-05-10T05:31:21.653 DEBUG:teuthology.orchestra.run.smithi142:mds.k> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.k.service 2024-05-10T05:31:21.681 INFO:journalctl@ceph.mds.h.smithi142.stdout:-- Logs begin at Fri 2024-05-10 04:59:48 UTC. -- 2024-05-10T05:31:21.682 DEBUG:teuthology.orchestra.run.smithi157:mds.c> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.c.service 2024-05-10T05:31:21.685 DEBUG:teuthology.orchestra.run.smithi157:mds.f> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.f.service 2024-05-10T05:31:21.709 INFO:journalctl@ceph.mds.k.smithi142.stdout:-- Logs begin at Fri 2024-05-10 04:59:48 UTC. -- 2024-05-10T05:31:21.715 INFO:journalctl@ceph.mds.c.smithi157.stdout:-- Logs begin at Fri 2024-05-10 05:00:45 UTC. -- 2024-05-10T05:31:21.716 DEBUG:teuthology.orchestra.run.smithi157:mds.i> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.i.service 2024-05-10T05:31:21.743 INFO:journalctl@ceph.mds.f.smithi157.stdout:-- Logs begin at Fri 2024-05-10 05:00:45 UTC. -- 2024-05-10T05:31:21.744 DEBUG:teuthology.orchestra.run.smithi157:mds.l> sudo journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.l.service 2024-05-10T05:31:21.770 INFO:journalctl@ceph.mds.i.smithi157.stdout:-- Logs begin at Fri 2024-05-10 05:00:45 UTC. -- 2024-05-10T05:31:21.770 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-10T05:31:21.771 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:31:21.796 INFO:journalctl@ceph.mds.l.smithi157.stdout:-- Logs begin at Fri 2024-05-10 05:00:45 UTC. -- 2024-05-10T05:31:22.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.054+0000 7ff0b4000700 1 -- 172.21.15.131:0/1649266879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0ac101be0 msgr2=0x7ff0ac101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:22.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.054+0000 7ff0b4000700 1 --2- 172.21.15.131:0/1649266879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0ac101be0 0x7ff0ac101ff0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009990 tx=0x7ff0a0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.054+0000 7ff0b4000700 1 -- 172.21.15.131:0/1649266879 shutdown_connections 2024-05-10T05:31:22.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.054+0000 7ff0b4000700 1 --2- 172.21.15.131:0/1649266879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0ac101be0 0x7ff0ac101ff0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.054+0000 7ff0b4000700 1 --2- 172.21.15.131:0/1649266879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0ac1036a0 0x7ff0ac109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.054+0000 7ff0b4000700 1 --2- 172.21.15.131:0/1649266879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0ac102d00 0x7ff0ac103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.054+0000 7ff0b4000700 1 -- 172.21.15.131:0/1649266879 >> 172.21.15.131:0/1649266879 conn(0x7ff0ac0fd210 msgr2=0x7ff0ac0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:22.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.055+0000 7ff0b4000700 1 -- 172.21.15.131:0/1649266879 shutdown_connections 2024-05-10T05:31:22.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.055+0000 7ff0b4000700 1 -- 172.21.15.131:0/1649266879 wait complete. 2024-05-10T05:31:22.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b4000700 1 Processor -- start 2024-05-10T05:31:22.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b4000700 1 -- start start 2024-05-10T05:31:22.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b4000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0ac101be0 0x7ff0ac19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b4000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0ac102d00 0x7ff0ac19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b4000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0ac1036a0 0x7ff0ac1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b4000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac1a1c80 con 0x7ff0ac101be0 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b4000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac1a1df0 con 0x7ff0ac1036a0 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b4000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac1a1fb0 con 0x7ff0ac102d00 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b1600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0ac101be0 0x7ff0ac19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b1600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0ac101be0 0x7ff0ac19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41188/0 (socket says 172.21.15.131:41188) 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b1600700 1 -- 172.21.15.131:0/2058921310 learned_addr learned my addr 172.21.15.131:0/2058921310 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b0c00700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0ac102d00 0x7ff0ac19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.056+0000 7ff0b2000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0ac1036a0 0x7ff0ac1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0b2000700 1 -- 172.21.15.131:0/2058921310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0ac102d00 msgr2=0x7ff0ac19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0b2000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0ac102d00 0x7ff0ac19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0b2000700 1 -- 172.21.15.131:0/2058921310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0ac101be0 msgr2=0x7ff0ac19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:22.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0b2000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0ac101be0 0x7ff0ac19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0b2000700 1 -- 172.21.15.131:0/2058921310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00095f0 con 0x7ff0ac1036a0 2024-05-10T05:31:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0b1600700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0ac101be0 0x7ff0ac19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0b2000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0ac1036a0 0x7ff0ac1a1670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400efc0 tx=0x7ff0a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0aa000700 1 -- 172.21.15.131:0/2058921310 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4018070 con 0x7ff0ac1036a0 2024-05-10T05:31:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0b4000700 1 -- 172.21.15.131:0/2058921310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ac1a2230 con 0x7ff0ac1036a0 2024-05-10T05:31:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0aa000700 1 -- 172.21.15.131:0/2058921310 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a400f040 con 0x7ff0ac1036a0 2024-05-10T05:31:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.057+0000 7ff0aa000700 1 -- 172.21.15.131:0/2058921310 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a400cd40 con 0x7ff0ac1036a0 2024-05-10T05:31:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.058+0000 7ff0b4000700 1 -- 172.21.15.131:0/2058921310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ac1a2740 con 0x7ff0ac1036a0 2024-05-10T05:31:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.058+0000 7ff0b4000700 1 -- 172.21.15.131:0/2058921310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ac0694d0 con 0x7ff0ac1036a0 2024-05-10T05:31:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.060+0000 7ff0aa000700 1 -- 172.21.15.131:0/2058921310 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff0a4003a00 con 0x7ff0ac1036a0 2024-05-10T05:31:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.062+0000 7ff0aa000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0940a1120 0x7ff0940a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.062+0000 7ff0aa000700 1 -- 172.21.15.131:0/2058921310 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff0a40cc050 con 0x7ff0ac1036a0 2024-05-10T05:31:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.062+0000 7ff0b1600700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0940a1120 0x7ff0940a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.062+0000 7ff0b1600700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0940a1120 0x7ff0940a3630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b2e0 tx=0x7ff0a001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:22.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.066+0000 7ff0aa000700 1 -- 172.21.15.131:0/2058921310 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff0a40ffd60 con 0x7ff0ac1036a0 2024-05-10T05:31:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:21 smithi131 ceph-mon[37264]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:21 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:21 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:21 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:21 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:21 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:21 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:22.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:21 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:22.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:21 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:22.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.275+0000 7ff0b4000700 1 -- 172.21.15.131:0/2058921310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0ac1a2a60 con 0x7ff0ac1036a0 2024-05-10T05:31:22.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.277+0000 7ff0aa000700 1 -- 172.21.15.131:0/2058921310 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18494 (secure 0 0 0) 0x7ff0a4095c70 con 0x7ff0ac1036a0 2024-05-10T05:31:22.278 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:22.278 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":76,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:31:14.901400+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.279+0000 7ff0b4000700 1 -- 172.21.15.131:0/2058921310 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0940a1120 msgr2=0x7ff0940a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0940a1120 0x7ff0940a3630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b2e0 tx=0x7ff0a001a040 comp rx=0 tx=0).stop 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 -- 172.21.15.131:0/2058921310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0ac1036a0 msgr2=0x7ff0ac1a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0ac1036a0 0x7ff0ac1a1670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400efc0 tx=0x7ff0a400c3a0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 -- 172.21.15.131:0/2058921310 shutdown_connections 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0940a1120 0x7ff0940a3630 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0ac101be0 0x7ff0ac19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0ac1036a0 0x7ff0ac1a1670 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 --2- 172.21.15.131:0/2058921310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0ac102d00 0x7ff0ac19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 -- 172.21.15.131:0/2058921310 >> 172.21.15.131:0/2058921310 conn(0x7ff0ac0fd210 msgr2=0x7ff0ac073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 -- 172.21.15.131:0/2058921310 shutdown_connections 2024-05-10T05:31:22.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.280+0000 7ff0b4000700 1 -- 172.21.15.131:0/2058921310 wait complete. 2024-05-10T05:31:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:21 smithi157 ceph-mon[78427]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:21 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:21 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:21 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:21 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:21 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:21 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:21 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:21 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:22.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:31:22.311 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:31:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:21 smithi142 ceph-mon[77667]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:21 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:21 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:21 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:21 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:21 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:21 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:22.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:21 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:22.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:21 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:22.562 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 10 05:31:22 smithi157 systemd[1]: Starting Ceph mds.l for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.577+0000 7f3fd1e00700 1 -- 172.21.15.131:0/3343762379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc102d00 msgr2=0x7f3fcc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.577+0000 7f3fd1e00700 1 --2- 172.21.15.131:0/3343762379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc102d00 0x7f3fcc103110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009990 tx=0x7f3fb8009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.577+0000 7f3fd1e00700 1 -- 172.21.15.131:0/3343762379 shutdown_connections 2024-05-10T05:31:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.577+0000 7f3fd1e00700 1 --2- 172.21.15.131:0/3343762379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fcc1036a0 0x7f3fcc109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.577+0000 7f3fd1e00700 1 --2- 172.21.15.131:0/3343762379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fcc101be0 0x7f3fcc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.577+0000 7f3fd1e00700 1 --2- 172.21.15.131:0/3343762379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc102d00 0x7f3fcc103110 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.577+0000 7f3fd1e00700 1 -- 172.21.15.131:0/3343762379 >> 172.21.15.131:0/3343762379 conn(0x7f3fcc0fd210 msgr2=0x7f3fcc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.577+0000 7f3fd1e00700 1 -- 172.21.15.131:0/3343762379 shutdown_connections 2024-05-10T05:31:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.577+0000 7f3fd1e00700 1 -- 172.21.15.131:0/3343762379 wait complete. 2024-05-10T05:31:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.578+0000 7f3fd1e00700 1 Processor -- start 2024-05-10T05:31:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.578+0000 7f3fd1e00700 1 -- start start 2024-05-10T05:31:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fd1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fcc101be0 0x7f3fcc19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fd1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fcc102d00 0x7f3fcc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fd1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc1036a0 0x7f3fcc1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fd1e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1a1bc0 con 0x7f3fcc1036a0 2024-05-10T05:31:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fd1e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1a1d30 con 0x7f3fcc101be0 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fd1e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1a1ef0 con 0x7f3fcc102d00 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fcb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc1036a0 0x7f3fcc1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fca000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fcc102d00 0x7f3fcc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fca000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fcc102d00 0x7f3fcc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43096/0 (socket says 172.21.15.131:43096) 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fca000700 1 -- 172.21.15.131:0/942076503 learned_addr learned my addr 172.21.15.131:0/942076503 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fcaa00700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fcc101be0 0x7f3fcc19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fca000700 1 -- 172.21.15.131:0/942076503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fcc101be0 msgr2=0x7f3fcc19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fca000700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fcc101be0 0x7f3fcc19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fca000700 1 -- 172.21.15.131:0/942076503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc1036a0 msgr2=0x7f3fcc1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fca000700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc1036a0 0x7f3fcc1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fca000700 1 -- 172.21.15.131:0/942076503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb80095f0 con 0x7f3fcc102d00 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.579+0000 7f3fcb400700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc1036a0 0x7f3fcc1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.580+0000 7f3fca000700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fcc102d00 0x7f3fcc19a620 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8000c00 tx=0x7f3fb8005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.580+0000 7f3fd0a00700 1 -- 172.21.15.131:0/942076503 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb801d070 con 0x7f3fcc102d00 2024-05-10T05:31:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.580+0000 7f3fd1e00700 1 -- 172.21.15.131:0/942076503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc1a2170 con 0x7f3fcc102d00 2024-05-10T05:31:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.580+0000 7f3fd0a00700 1 -- 172.21.15.131:0/942076503 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb8021410 con 0x7f3fcc102d00 2024-05-10T05:31:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.580+0000 7f3fd0a00700 1 -- 172.21.15.131:0/942076503 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8017650 con 0x7f3fcc102d00 2024-05-10T05:31:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.580+0000 7f3fd1e00700 1 -- 172.21.15.131:0/942076503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc1a2590 con 0x7f3fcc102d00 2024-05-10T05:31:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.581+0000 7f3fd1e00700 1 -- 172.21.15.131:0/942076503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fcc0694b0 con 0x7f3fcc102d00 2024-05-10T05:31:22.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.583+0000 7f3fd0a00700 1 -- 172.21.15.131:0/942076503 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3fb8021580 con 0x7f3fcc102d00 2024-05-10T05:31:22.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.584+0000 7f3fd0a00700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3fb00a11c0 0x7f3fb00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.584+0000 7f3fd0a00700 1 -- 172.21.15.131:0/942076503 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3fb80cd820 con 0x7f3fcc102d00 2024-05-10T05:31:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.584+0000 7f3fcaa00700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3fb00a11c0 0x7f3fb00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.585+0000 7f3fcaa00700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3fb00a11c0 0x7f3fb00a36d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc102c00 tx=0x7f3fbc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:22.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.588+0000 7f3fd0a00700 1 -- 172.21.15.131:0/942076503 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3fb8097440 con 0x7f3fcc102d00 2024-05-10T05:31:22.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.794+0000 7f3fd1e00700 1 -- 172.21.15.131:0/942076503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fcc1a28b0 con 0x7f3fcc102d00 2024-05-10T05:31:22.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.795+0000 7f3fd0a00700 1 -- 172.21.15.131:0/942076503 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fb809aa60 con 0x7f3fcc102d00 2024-05-10T05:31:22.796 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:31:22.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 -- 172.21.15.131:0/942076503 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3fb00a11c0 msgr2=0x7f3fb00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:22.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3fb00a11c0 0x7f3fb00a36d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc102c00 tx=0x7f3fbc009260 comp rx=0 tx=0).stop 2024-05-10T05:31:22.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 -- 172.21.15.131:0/942076503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fcc102d00 msgr2=0x7f3fcc19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:22.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fcc102d00 0x7f3fcc19a620 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8000c00 tx=0x7f3fb8005720 comp rx=0 tx=0).stop 2024-05-10T05:31:22.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 -- 172.21.15.131:0/942076503 shutdown_connections 2024-05-10T05:31:22.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3fb00a11c0 0x7f3fb00a36d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fcc1036a0 0x7f3fcc1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fcc101be0 0x7f3fcc19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 --2- 172.21.15.131:0/942076503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fcc102d00 0x7f3fcc19a620 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:22.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 -- 172.21.15.131:0/942076503 >> 172.21.15.131:0/942076503 conn(0x7f3fcc0fd210 msgr2=0x7f3fcc073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:22.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 -- 172.21.15.131:0/942076503 shutdown_connections 2024-05-10T05:31:22.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:22.797+0000 7f3fd1e00700 1 -- 172.21.15.131:0/942076503 wait complete. 2024-05-10T05:31:22.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:31:23.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.096+0000 7fc5dbe00700 1 -- 172.21.15.131:0/3186346705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc075140 msgr2=0x7fc5dc073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:23.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.096+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/3186346705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc075140 0x7fc5dc073600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009990 tx=0x7fc5cc009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:23.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.096+0000 7fc5dbe00700 1 -- 172.21.15.131:0/3186346705 shutdown_connections 2024-05-10T05:31:23.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.096+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/3186346705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc10e040 0x7fc5dc110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:23.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.096+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/3186346705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc075140 0x7fc5dc073600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:23.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.096+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/3186346705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc073b40 0x7fc5dc074010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:23.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.096+0000 7fc5dbe00700 1 -- 172.21.15.131:0/3186346705 >> 172.21.15.131:0/3186346705 conn(0x7fc5dc0fd010 msgr2=0x7fc5dc0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:23.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.097+0000 7fc5dbe00700 1 -- 172.21.15.131:0/3186346705 shutdown_connections 2024-05-10T05:31:23.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.097+0000 7fc5dbe00700 1 -- 172.21.15.131:0/3186346705 wait complete. 2024-05-10T05:31:23.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5dbe00700 1 Processor -- start 2024-05-10T05:31:23.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5dbe00700 1 -- start start 2024-05-10T05:31:23.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5dbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc073b40 0x7fc5dc19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:23.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5dbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc075140 0x7fc5dc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:23.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5dbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc10e040 0x7fc5dc1a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5dbe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc1a5db0 con 0x7fc5dc075140 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5dbe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc1a5ef0 con 0x7fc5dc10e040 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5dbe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc1a6080 con 0x7fc5dc073b40 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5da000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc075140 0x7fc5dc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5daa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc073b40 0x7fc5dc19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5da000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc075140 0x7fc5dc19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48744/0 (socket says 172.21.15.131:48744) 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5daa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc073b40 0x7fc5dc19e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51516/0 (socket says 172.21.15.131:51516) 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5da000700 1 -- 172.21.15.131:0/2995060330 learned_addr learned my addr 172.21.15.131:0/2995060330 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.098+0000 7fc5db400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc10e040 0x7fc5dc1a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5daa00700 1 -- 172.21.15.131:0/2995060330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc10e040 msgr2=0x7fc5dc1a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5daa00700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc10e040 0x7fc5dc1a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5daa00700 1 -- 172.21.15.131:0/2995060330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc075140 msgr2=0x7fc5dc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5daa00700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc075140 0x7fc5dc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:23.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5daa00700 1 -- 172.21.15.131:0/2995060330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c8009520 con 0x7fc5dc073b40 2024-05-10T05:31:23.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5daa00700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc073b40 0x7fc5dc19e310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0098e0 tx=0x7fc5cc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:23.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5d3400700 1 -- 172.21.15.131:0/2995060330 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc01d070 con 0x7fc5dc073b40 2024-05-10T05:31:23.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5d3400700 1 -- 172.21.15.131:0/2995060330 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5cc021410 con 0x7fc5dc073b40 2024-05-10T05:31:23.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5dbe00700 1 -- 172.21.15.131:0/2995060330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc0095f0 con 0x7fc5dc073b40 2024-05-10T05:31:23.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5d3400700 1 -- 172.21.15.131:0/2995060330 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc017650 con 0x7fc5dc073b40 2024-05-10T05:31:23.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.099+0000 7fc5dbe00700 1 -- 172.21.15.131:0/2995060330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5dc1a6590 con 0x7fc5dc073b40 2024-05-10T05:31:23.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.101+0000 7fc5dbe00700 1 -- 172.21.15.131:0/2995060330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5dc1042a0 con 0x7fc5dc073b40 2024-05-10T05:31:23.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.102+0000 7fc5d3400700 1 -- 172.21.15.131:0/2995060330 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc5cc0177b0 con 0x7fc5dc073b40 2024-05-10T05:31:23.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.103+0000 7fc5d3400700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11c0 0x7fc5c00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:23.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.103+0000 7fc5d3400700 1 -- 172.21.15.131:0/2995060330 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc5cc0cd860 con 0x7fc5dc073b40 2024-05-10T05:31:23.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.103+0000 7fc5da000700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11c0 0x7fc5c00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:23.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.104+0000 7fc5da000700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11c0 0x7fc5c00a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc19f3b0 tx=0x7fc5c8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.107+0000 7fc5d3400700 1 -- 172.21.15.131:0/2995060330 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc5cc097500 con 0x7fc5dc073b40 2024-05-10T05:31:23.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:22 smithi131 ceph-mon[37264]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:31:23.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:22 smithi131 ceph-mon[37264]: Saving service mds.all spec with placement smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-10T05:31:23.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:22 smithi131 ceph-mon[37264]: Deploying daemon mds.l on smithi157 2024-05-10T05:31:23.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:22 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2058921310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:23.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:22 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/942076503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:31:23.303 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 10 05:31:22 smithi157 ceph-mds[114277]: -- 172.21.15.157:0/1016687601 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c131c761a0 con 0x55c131cd6880 2024-05-10T05:31:23.303 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 10 05:31:22 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-l[114254]: starting mds.l at 2024-05-10T05:31:23.303 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 10 05:31:22 smithi157 bash[114239]: 2b8a05138a286a7c2445f4780c73c71a7c9325b87736e7b92ecd15202f345090 2024-05-10T05:31:23.303 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 10 05:31:22 smithi157 systemd[1]: Started Ceph mds.l for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:23.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:22 smithi157 ceph-mon[78427]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:31:23.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:22 smithi157 ceph-mon[78427]: Saving service mds.all spec with placement smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-10T05:31:23.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:22 smithi157 ceph-mon[78427]: Deploying daemon mds.l on smithi157 2024-05-10T05:31:23.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:22 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2058921310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:23.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:22 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/942076503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:31:23.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.359+0000 7fc5dbe00700 1 -- 172.21.15.131:0/2995060330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5dc103c80 con 0x7fc5dc073b40 2024-05-10T05:31:23.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.360+0000 7fc5d3400700 1 -- 172.21.15.131:0/2995060330 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7fc5cc09ab20 con 0x7fc5dc073b40 2024-05-10T05:31:23.361 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:23.361 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-10T05:31:23.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.362+0000 7fc5dbe00700 1 -- 172.21.15.131:0/2995060330 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11c0 msgr2=0x7fc5c00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.362+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11c0 0x7fc5c00a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc19f3b0 tx=0x7fc5c8009260 comp rx=0 tx=0).stop 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.362+0000 7fc5dbe00700 1 -- 172.21.15.131:0/2995060330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc073b40 msgr2=0x7fc5dc19e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.362+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc073b40 0x7fc5dc19e310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0098e0 tx=0x7fc5cc005740 comp rx=0 tx=0).stop 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.363+0000 7fc5dbe00700 1 -- 172.21.15.131:0/2995060330 shutdown_connections 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.363+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11c0 0x7fc5c00a36d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.363+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc10e040 0x7fc5dc1a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.363+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc073b40 0x7fc5dc19e310 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.363+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/2995060330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc075140 0x7fc5dc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.363+0000 7fc5dbe00700 1 -- 172.21.15.131:0/2995060330 >> 172.21.15.131:0/2995060330 conn(0x7fc5dc0fd010 msgr2=0x7fc5dc1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.363+0000 7fc5dbe00700 1 -- 172.21.15.131:0/2995060330 shutdown_connections 2024-05-10T05:31:23.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:23.363+0000 7fc5dbe00700 1 -- 172.21.15.131:0/2995060330 wait complete. 2024-05-10T05:31:23.365 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 2 2024-05-10T05:31:23.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:22 smithi142 ceph-mon[77667]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:31:23.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:22 smithi142 ceph-mon[77667]: Saving service mds.all spec with placement smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-10T05:31:23.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:22 smithi142 ceph-mon[77667]: Deploying daemon mds.l on smithi157 2024-05-10T05:31:23.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:22 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2058921310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:23.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:22 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/942076503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:31:24.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:24.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.157:6832/3274595788,v1:172.21.15.157:6833/3274595788] up:boot 2024-05-10T05:31:24.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: fsmap 1 up:standby 2024-05-10T05:31:24.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T05:31:24.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:24.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:24.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:24.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:24.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:24.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:24.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:23 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2995060330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.157:6832/3274595788,v1:172.21.15.157:6833/3274595788] up:boot 2024-05-10T05:31:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: fsmap 1 up:standby 2024-05-10T05:31:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T05:31:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:24.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:23 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2995060330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:24.394 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:31:24 smithi142 systemd[1]: Starting Ceph mds.k for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.157:6832/3274595788,v1:172.21.15.157:6833/3274595788] up:boot 2024-05-10T05:31:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: fsmap 1 up:standby 2024-05-10T05:31:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T05:31:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:24.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:24.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:24.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:24.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:24.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:24.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:23 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2995060330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:24.982 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:31:24 smithi142 bash[114155]: f874d82e7854a6e0eacf5465a73907bd7083d7be69e72c603e54c3b88164c22a 2024-05-10T05:31:24.982 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:31:24 smithi142 systemd[1]: Started Ceph mds.k for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:24.983 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:31:24 smithi142 ceph-mds[114193]: -- 172.21.15.142:0/4250087447 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564e75a2a1a0 con 0x564e75a8a400 2024-05-10T05:31:24.983 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:31:24 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-k[114170]: starting mds.k at 2024-05-10T05:31:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:24 smithi157 ceph-mon[78427]: Deploying daemon mds.k on smithi142 2024-05-10T05:31:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:24 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:24 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:24 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:24 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:24 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:24 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:25.395 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:31:25.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:24 smithi142 ceph-mon[77667]: Deploying daemon mds.k on smithi142 2024-05-10T05:31:25.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:24 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:25.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:24 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:25.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:24 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:25.396 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:24 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:25.397 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:24 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:25.397 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:24 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:25.434 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:24 smithi131 ceph-mon[37264]: Deploying daemon mds.k on smithi142 2024-05-10T05:31:25.434 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:24 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:25.434 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:24 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:25.435 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:24 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:25.435 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:24 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:25.435 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:24 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:25.435 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:24 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.681+0000 7f8970a00700 1 -- 172.21.15.131:0/1011736232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f896c101a00 msgr2=0x7f896c101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.681+0000 7f8970a00700 1 --2- 172.21.15.131:0/1011736232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f896c101a00 0x7f896c101e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f895c009990 tx=0x7f895c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.681+0000 7f8970a00700 1 -- 172.21.15.131:0/1011736232 shutdown_connections 2024-05-10T05:31:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.681+0000 7f8970a00700 1 --2- 172.21.15.131:0/1011736232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f896c101a00 0x7f896c101e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.681+0000 7f8970a00700 1 --2- 172.21.15.131:0/1011736232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c1034c0 0x7f896c109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.681+0000 7f8970a00700 1 --2- 172.21.15.131:0/1011736232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c102b20 0x7f896c102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.681+0000 7f8970a00700 1 -- 172.21.15.131:0/1011736232 >> 172.21.15.131:0/1011736232 conn(0x7f896c0fd010 msgr2=0x7f896c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.681+0000 7f8970a00700 1 -- 172.21.15.131:0/1011736232 shutdown_connections 2024-05-10T05:31:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.681+0000 7f8970a00700 1 -- 172.21.15.131:0/1011736232 wait complete. 2024-05-10T05:31:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.682+0000 7f8970a00700 1 Processor -- start 2024-05-10T05:31:25.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.682+0000 7f8970a00700 1 -- start start 2024-05-10T05:31:25.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f8970a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c101a00 0x7f896c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:25.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f8970a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c102b20 0x7f896c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f8970a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f896c1034c0 0x7f896c1a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f8970a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c1a1a00 con 0x7f896c1034c0 2024-05-10T05:31:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f8970a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c1a1b40 con 0x7f896c102b20 2024-05-10T05:31:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f8970a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c1a1cd0 con 0x7f896c101a00 2024-05-10T05:31:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f896be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f896c1034c0 0x7f896c1a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f896aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c102b20 0x7f896c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f896be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f896c1034c0 0x7f896c1a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48760/0 (socket says 172.21.15.131:48760) 2024-05-10T05:31:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f896be00700 1 -- 172.21.15.131:0/4248311882 learned_addr learned my addr 172.21.15.131:0/4248311882 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f896b400700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c101a00 0x7f896c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f896be00700 1 -- 172.21.15.131:0/4248311882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c101a00 msgr2=0x7f896c199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f896be00700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c101a00 0x7f896c199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f896be00700 1 -- 172.21.15.131:0/4248311882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c102b20 msgr2=0x7f896c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f896be00700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c102b20 0x7f896c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.683+0000 7f896be00700 1 -- 172.21.15.131:0/4248311882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0095f0 con 0x7f896c1034c0 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.684+0000 7f896be00700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f896c1034c0 0x7f896c1a1430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f896400ef60 tx=0x7f896400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.684+0000 7f8963e00700 1 -- 172.21.15.131:0/4248311882 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964018070 con 0x7f896c1034c0 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.684+0000 7f8963e00700 1 -- 172.21.15.131:0/4248311882 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896400f040 con 0x7f896c1034c0 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.684+0000 7f8963e00700 1 -- 172.21.15.131:0/4248311882 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896400ccc0 con 0x7f896c1034c0 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.684+0000 7f8970a00700 1 -- 172.21.15.131:0/4248311882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896c1a1f50 con 0x7f896c1034c0 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.684+0000 7f8970a00700 1 -- 172.21.15.131:0/4248311882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c1a2320 con 0x7f896c1034c0 2024-05-10T05:31:25.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.685+0000 7f8970a00700 1 -- 172.21.15.131:0/4248311882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896c068110 con 0x7f896c1034c0 2024-05-10T05:31:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.687+0000 7f8963e00700 1 -- 172.21.15.131:0/4248311882 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8964003a00 con 0x7f896c1034c0 2024-05-10T05:31:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.688+0000 7f8963e00700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89500a1240 0x7f89500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.689+0000 7f8963e00700 1 -- 172.21.15.131:0/4248311882 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f89640cbe50 con 0x7f896c1034c0 2024-05-10T05:31:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.689+0000 7f896b400700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89500a1240 0x7f89500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.689+0000 7f896b400700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89500a1240 0x7f89500a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f895c00b7b0 tx=0x7f895c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:25.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.693+0000 7f8963e00700 1 -- 172.21.15.131:0/4248311882 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8964095b10 con 0x7f896c1034c0 2024-05-10T05:31:25.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.960+0000 7f8970a00700 1 -- 172.21.15.131:0/4248311882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f896c10d230 con 0x7f896c1034c0 2024-05-10T05:31:25.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.960+0000 7f8963e00700 1 -- 172.21.15.131:0/4248311882 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f8964099130 con 0x7f896c1034c0 2024-05-10T05:31:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:25.961 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 -- 172.21.15.131:0/4248311882 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89500a1240 msgr2=0x7f89500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89500a1240 0x7f89500a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f895c00b7b0 tx=0x7f895c01a040 comp rx=0 tx=0).stop 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 -- 172.21.15.131:0/4248311882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f896c1034c0 msgr2=0x7f896c1a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f896c1034c0 0x7f896c1a1430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f896400ef60 tx=0x7f896400ef90 comp rx=0 tx=0).stop 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 -- 172.21.15.131:0/4248311882 shutdown_connections 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89500a1240 0x7f89500a3750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f896c1034c0 0x7f896c1a1430 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f896c101a00 0x7f896c199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 --2- 172.21.15.131:0/4248311882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f896c102b20 0x7f896c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 -- 172.21.15.131:0/4248311882 >> 172.21.15.131:0/4248311882 conn(0x7f896c0fd010 msgr2=0x7f896c073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 -- 172.21.15.131:0/4248311882 shutdown_connections 2024-05-10T05:31:25.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:25.963+0000 7f8970a00700 1 -- 172.21.15.131:0/4248311882 wait complete. 2024-05-10T05:31:25.965 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 3 2024-05-10T05:31:26.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:25 smithi131 ceph-mon[37264]: Deploying daemon mds.j on smithi131 2024-05-10T05:31:26.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:25 smithi131 ceph-mon[37264]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:26.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:25 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.142:6832/3035299290,v1:172.21.15.142:6833/3035299290] up:boot 2024-05-10T05:31:26.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:25 smithi131 ceph-mon[37264]: fsmap 2 up:standby 2024-05-10T05:31:26.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:25 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T05:31:26.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:25 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4248311882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:25 smithi157 ceph-mon[78427]: Deploying daemon mds.j on smithi131 2024-05-10T05:31:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:25 smithi157 ceph-mon[78427]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:25 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.142:6832/3035299290,v1:172.21.15.142:6833/3035299290] up:boot 2024-05-10T05:31:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:25 smithi157 ceph-mon[78427]: fsmap 2 up:standby 2024-05-10T05:31:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:25 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T05:31:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:25 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4248311882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:26.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:25 smithi142 ceph-mon[77667]: Deploying daemon mds.j on smithi131 2024-05-10T05:31:26.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:25 smithi142 ceph-mon[77667]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:26.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:25 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.142:6832/3035299290,v1:172.21.15.142:6833/3035299290] up:boot 2024-05-10T05:31:26.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:25 smithi142 ceph-mon[77667]: fsmap 2 up:standby 2024-05-10T05:31:26.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:25 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T05:31:26.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:25 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4248311882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:26.711 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 10 05:31:26 smithi131 systemd[1]: Starting Ceph mds.j for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:27.460 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 10 05:31:27 smithi131 bash[87275]: b59369d9641962915144f993d842bcc6bd597cb53605787bb170d40b83077063 2024-05-10T05:31:27.460 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 10 05:31:27 smithi131 systemd[1]: Started Ceph mds.j for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:27.460 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 10 05:31:27 smithi131 ceph-mds[87316]: -- 172.21.15.131:0/2752312532 <== mon.0 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5631255001a0 con 0x563125562880 2024-05-10T05:31:27.460 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 10 05:31:27 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-j[87290]: starting mds.j at 2024-05-10T05:31:27.991 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:31:28.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:28 smithi131 ceph-mon[37264]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:28.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:28.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:28.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:28.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:28.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:28.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:28.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.272+0000 7f00fe600700 1 -- 172.21.15.131:0/3480019833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8102d00 msgr2=0x7f00f8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:28.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.272+0000 7f00fe600700 1 --2- 172.21.15.131:0/3480019833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8102d00 0x7f00f8103110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f00e40098f0 tx=0x7f00e4009c00 comp rx=0 tx=0).stop 2024-05-10T05:31:28.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.272+0000 7f00fe600700 1 -- 172.21.15.131:0/3480019833 shutdown_connections 2024-05-10T05:31:28.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.272+0000 7f00fe600700 1 --2- 172.21.15.131:0/3480019833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f81036a0 0x7f00f8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:28.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.272+0000 7f00fe600700 1 --2- 172.21.15.131:0/3480019833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f8101be0 0x7f00f8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:28.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.272+0000 7f00fe600700 1 --2- 172.21.15.131:0/3480019833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8102d00 0x7f00f8103110 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:28.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.272+0000 7f00fe600700 1 -- 172.21.15.131:0/3480019833 >> 172.21.15.131:0/3480019833 conn(0x7f00f80fd210 msgr2=0x7f00f80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:28.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.272+0000 7f00fe600700 1 -- 172.21.15.131:0/3480019833 shutdown_connections 2024-05-10T05:31:28.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.272+0000 7f00fe600700 1 -- 172.21.15.131:0/3480019833 wait complete. 2024-05-10T05:31:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.273+0000 7f00fe600700 1 Processor -- start 2024-05-10T05:31:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.273+0000 7f00fe600700 1 -- start start 2024-05-10T05:31:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00fe600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f8101be0 0x7f00f819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00fe600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f8102d00 0x7f00f819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00fe600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f81036a0 0x7f00f81a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00fe600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81a1d30 con 0x7f00f81036a0 2024-05-10T05:31:28.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00fe600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81a1ea0 con 0x7f00f8101be0 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00fe600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81a2060 con 0x7f00f8102d00 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00f7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f8101be0 0x7f00f819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00f7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f81036a0 0x7f00f81a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00f7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f8101be0 0x7f00f819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:52482/0 (socket says 172.21.15.131:52482) 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00f7400700 1 -- 172.21.15.131:0/197153728 learned_addr learned my addr 172.21.15.131:0/197153728 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00f6a00700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f8102d00 0x7f00f819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00f7e00700 1 -- 172.21.15.131:0/197153728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f8102d00 msgr2=0x7f00f819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00f7e00700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f8102d00 0x7f00f819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00f7e00700 1 -- 172.21.15.131:0/197153728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f8101be0 msgr2=0x7f00f819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00f7e00700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f8101be0 0x7f00f819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.274+0000 7f00f7e00700 1 -- 172.21.15.131:0/197153728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e4009520 con 0x7f00f81036a0 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.275+0000 7f00f7e00700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f81036a0 0x7f00f81a1720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f00f000efc0 tx=0x7f00f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:28.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.275+0000 7f00efe00700 1 -- 172.21.15.131:0/197153728 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0018070 con 0x7f00f81036a0 2024-05-10T05:31:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.275+0000 7f00fe600700 1 -- 172.21.15.131:0/197153728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f81a22e0 con 0x7f00f81036a0 2024-05-10T05:31:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.275+0000 7f00efe00700 1 -- 172.21.15.131:0/197153728 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f000f040 con 0x7f00f81036a0 2024-05-10T05:31:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.275+0000 7f00efe00700 1 -- 172.21.15.131:0/197153728 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000ccb0 con 0x7f00f81036a0 2024-05-10T05:31:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.275+0000 7f00fe600700 1 -- 172.21.15.131:0/197153728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f81a27a0 con 0x7f00f81036a0 2024-05-10T05:31:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.276+0000 7f00fe600700 1 -- 172.21.15.131:0/197153728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f80694d0 con 0x7f00f81036a0 2024-05-10T05:31:28.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.278+0000 7f00efe00700 1 -- 172.21.15.131:0/197153728 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f00f0003a00 con 0x7f00f81036a0 2024-05-10T05:31:28.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.279+0000 7f00efe00700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f00dc0a1120 0x7f00dc0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:28.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.279+0000 7f00efe00700 1 -- 172.21.15.131:0/197153728 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f00f00cbf20 con 0x7f00f81036a0 2024-05-10T05:31:28.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.280+0000 7f00f7400700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f00dc0a1120 0x7f00dc0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:28.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.280+0000 7f00f7400700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f00dc0a1120 0x7f00dc0a3630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009cc0 tx=0x7f00e8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:28.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.284+0000 7f00efe00700 1 -- 172.21.15.131:0/197153728 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f00f0095b40 con 0x7f00f81036a0 2024-05-10T05:31:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:28 smithi157 ceph-mon[78427]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:28 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:28 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:28 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:28 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:28 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:28 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:28 smithi142 ceph-mon[77667]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:28 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:28.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:28 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:28.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:28 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:28.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:28 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:28.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:28 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:28.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:28 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.549+0000 7f00fe600700 1 -- 172.21.15.131:0/197153728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00f810d430 con 0x7f00f81036a0 2024-05-10T05:31:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.549+0000 7f00efe00700 1 -- 172.21.15.131:0/197153728 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2780 (secure 0 0 0) 0x7f00f0099160 con 0x7f00f81036a0 2024-05-10T05:31:28.550 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:28.551 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-10T05:31:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 -- 172.21.15.131:0/197153728 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f00dc0a1120 msgr2=0x7f00dc0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f00dc0a1120 0x7f00dc0a3630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009cc0 tx=0x7f00e8009260 comp rx=0 tx=0).stop 2024-05-10T05:31:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 -- 172.21.15.131:0/197153728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f81036a0 msgr2=0x7f00f81a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f81036a0 0x7f00f81a1720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f00f000efc0 tx=0x7f00f000c3a0 comp rx=0 tx=0).stop 2024-05-10T05:31:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 -- 172.21.15.131:0/197153728 shutdown_connections 2024-05-10T05:31:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f00dc0a1120 0x7f00dc0a3630 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f81036a0 0x7f00f81a1720 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f8101be0 0x7f00f819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 --2- 172.21.15.131:0/197153728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00f8102d00 0x7f00f819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 -- 172.21.15.131:0/197153728 >> 172.21.15.131:0/197153728 conn(0x7f00f80fd210 msgr2=0x7f00f8073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 -- 172.21.15.131:0/197153728 shutdown_connections 2024-05-10T05:31:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:28.552+0000 7f00fe600700 1 -- 172.21.15.131:0/197153728 wait complete. 2024-05-10T05:31:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 4 2024-05-10T05:31:29.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:29 smithi131 ceph-mon[37264]: Deploying daemon mds.d on smithi131 2024-05-10T05:31:29.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:29 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.131:6834/2077120233,v1:172.21.15.131:6835/2077120233] up:boot 2024-05-10T05:31:29.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:29 smithi131 ceph-mon[37264]: fsmap 3 up:standby 2024-05-10T05:31:29.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:29 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T05:31:29.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:29 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/197153728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:29.211 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:31:29 smithi131 systemd[1]: Starting Ceph mds.d for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:29.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:29 smithi157 ceph-mon[78427]: Deploying daemon mds.d on smithi131 2024-05-10T05:31:29.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:29 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.131:6834/2077120233,v1:172.21.15.131:6835/2077120233] up:boot 2024-05-10T05:31:29.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:29 smithi157 ceph-mon[78427]: fsmap 3 up:standby 2024-05-10T05:31:29.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:29 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T05:31:29.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:29 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/197153728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:29.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:29 smithi142 ceph-mon[77667]: Deploying daemon mds.d on smithi131 2024-05-10T05:31:29.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:29 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.131:6834/2077120233,v1:172.21.15.131:6835/2077120233] up:boot 2024-05-10T05:31:29.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:29 smithi142 ceph-mon[77667]: fsmap 3 up:standby 2024-05-10T05:31:29.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:29 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T05:31:29.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:29 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/197153728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:29.960 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:31:29 smithi131 bash[87663]: 3800de5cf8b23d88bd454705f06af7731f418e6c7decafae9ebda732d11121be 2024-05-10T05:31:29.960 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:31:29 smithi131 systemd[1]: Started Ceph mds.d for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:29.960 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:31:29 smithi131 ceph-mds[87701]: -- 172.21.15.131:0/3660461191 <== mon.0 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563f771f01a0 con 0x563f77252880 2024-05-10T05:31:29.960 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:31:29 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-d[87678]: starting mds.d at 2024-05-10T05:31:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:30 smithi157 ceph-mon[78427]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:30 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:30 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:30 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:30 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:30 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:30 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:30.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:30 smithi142 ceph-mon[77667]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:30.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:30 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:30.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:30 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:30.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:30 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:30.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:30 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:30.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:30 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:30.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:30 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:30.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:30 smithi131 ceph-mon[37264]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:30.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:30 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:30.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:30 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:30.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:30 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:30.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:30 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:30.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:30 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:30.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:30 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:30.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:31:30.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.864+0000 7f357b200700 1 -- 172.21.15.131:0/2333802195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f357c1019f0 msgr2=0x7f357c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.864+0000 7f357b200700 1 --2- 172.21.15.131:0/2333802195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f357c1019f0 0x7f357c101e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f356c0099c0 tx=0x7f356c009cd0 comp rx=0 tx=0).stop 2024-05-10T05:31:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.864+0000 7f357b200700 1 -- 172.21.15.131:0/2333802195 shutdown_connections 2024-05-10T05:31:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.864+0000 7f357b200700 1 --2- 172.21.15.131:0/2333802195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f357c1034b0 0x7f357c109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.864+0000 7f357b200700 1 --2- 172.21.15.131:0/2333802195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f357c1019f0 0x7f357c101e00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.864+0000 7f357b200700 1 --2- 172.21.15.131:0/2333802195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f357c102b10 0x7f357c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.864+0000 7f357b200700 1 -- 172.21.15.131:0/2333802195 >> 172.21.15.131:0/2333802195 conn(0x7f357c0fcfe0 msgr2=0x7f357c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.864+0000 7f357b200700 1 -- 172.21.15.131:0/2333802195 shutdown_connections 2024-05-10T05:31:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.864+0000 7f357b200700 1 -- 172.21.15.131:0/2333802195 wait complete. 2024-05-10T05:31:30.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.865+0000 7f357b200700 1 Processor -- start 2024-05-10T05:31:30.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.865+0000 7f357b200700 1 -- start start 2024-05-10T05:31:30.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f357b200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f357c1019f0 0x7f357c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:30.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f357b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f357c102b10 0x7f357c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:30.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f357b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f357c1034b0 0x7f357c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:30.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f357b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c19f7b0 con 0x7f357c1034b0 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f357b200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c19f8f0 con 0x7f357c1019f0 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f357b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c19fab0 con 0x7f357c102b10 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f3579e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f357c1019f0 0x7f357c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f3579e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f357c1019f0 0x7f357c197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:52516/0 (socket says 172.21.15.131:52516) 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f357a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f357c1034b0 0x7f357c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f3579e00700 1 -- 172.21.15.131:0/1932742208 learned_addr learned my addr 172.21.15.131:0/1932742208 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f3579400700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f357c102b10 0x7f357c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f3579e00700 1 -- 172.21.15.131:0/1932742208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f357c102b10 msgr2=0x7f357c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f3579e00700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f357c102b10 0x7f357c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f3579e00700 1 -- 172.21.15.131:0/1932742208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f357c1034b0 msgr2=0x7f357c19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f3579e00700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f357c1034b0 0x7f357c19f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.866+0000 7f3579e00700 1 -- 172.21.15.131:0/1932742208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c0095f0 con 0x7f357c1019f0 2024-05-10T05:31:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.867+0000 7f3579e00700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f357c1019f0 0x7f357c197d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f356c0098e0 tx=0x7f356c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:30.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.867+0000 7f3572a00700 1 -- 172.21.15.131:0/1932742208 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c01d070 con 0x7f357c1019f0 2024-05-10T05:31:30.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.867+0000 7f3572a00700 1 -- 172.21.15.131:0/1932742208 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f356c005030 con 0x7f357c1019f0 2024-05-10T05:31:30.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.867+0000 7f3572a00700 1 -- 172.21.15.131:0/1932742208 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c004a90 con 0x7f357c1019f0 2024-05-10T05:31:30.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.867+0000 7f357b200700 1 -- 172.21.15.131:0/1932742208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357c19fd30 con 0x7f357c1019f0 2024-05-10T05:31:30.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.867+0000 7f357b200700 1 -- 172.21.15.131:0/1932742208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c1a0150 con 0x7f357c1019f0 2024-05-10T05:31:30.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.868+0000 7f357b200700 1 -- 172.21.15.131:0/1932742208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357c108c00 con 0x7f357c1019f0 2024-05-10T05:31:30.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.870+0000 7f3572a00700 1 -- 172.21.15.131:0/1932742208 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f356c0051a0 con 0x7f357c1019f0 2024-05-10T05:31:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.871+0000 7f3572a00700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35600a1240 0x7f35600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.871+0000 7f3572a00700 1 -- 172.21.15.131:0/1932742208 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f356c0cd1a0 con 0x7f357c1019f0 2024-05-10T05:31:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.871+0000 7f3579400700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35600a1240 0x7f35600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.872+0000 7f3579400700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35600a1240 0x7f35600a3750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3568009bf0 tx=0x7f3568009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:30.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:30.876+0000 7f3572a00700 1 -- 172.21.15.131:0/1932742208 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f356c096dc0 con 0x7f357c1019f0 2024-05-10T05:31:31.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.130+0000 7f357b200700 1 -- 172.21.15.131:0/1932742208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f357c068110 con 0x7f357c1019f0 2024-05-10T05:31:31.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.130+0000 7f3572a00700 1 -- 172.21.15.131:0/1932742208 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3540 (secure 0 0 0) 0x7f356c09a3e0 con 0x7f357c1019f0 2024-05-10T05:31:31.131 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:31.131 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2024-05-10T05:31:31.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 -- 172.21.15.131:0/1932742208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35600a1240 msgr2=0x7f35600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:31.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35600a1240 0x7f35600a3750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3568009bf0 tx=0x7f3568009260 comp rx=0 tx=0).stop 2024-05-10T05:31:31.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 -- 172.21.15.131:0/1932742208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f357c1019f0 msgr2=0x7f357c197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:31.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f357c1019f0 0x7f357c197d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f356c0098e0 tx=0x7f356c00f690 comp rx=0 tx=0).stop 2024-05-10T05:31:31.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 -- 172.21.15.131:0/1932742208 shutdown_connections 2024-05-10T05:31:31.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35600a1240 0x7f35600a3750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:31.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f357c1034b0 0x7f357c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:31.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f357c1019f0 0x7f357c197d50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:31.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 --2- 172.21.15.131:0/1932742208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f357c102b10 0x7f357c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:31.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 -- 172.21.15.131:0/1932742208 >> 172.21.15.131:0/1932742208 conn(0x7f357c0fcfe0 msgr2=0x7f357c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:31.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 -- 172.21.15.131:0/1932742208 shutdown_connections 2024-05-10T05:31:31.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:31.133+0000 7f357b200700 1 -- 172.21.15.131:0/1932742208 wait complete. 2024-05-10T05:31:31.135 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 5 2024-05-10T05:31:31.258 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:31 smithi142 ceph-mon[77667]: Deploying daemon mds.e on smithi142 2024-05-10T05:31:31.258 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:31 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.131:6836/3356790716,v1:172.21.15.131:6837/3356790716] up:boot 2024-05-10T05:31:31.259 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:31 smithi142 ceph-mon[77667]: fsmap 4 up:standby 2024-05-10T05:31:31.259 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:31 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T05:31:31.259 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:31:31 smithi142 systemd[1]: Starting Ceph mds.e for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:31.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:31 smithi157 ceph-mon[78427]: Deploying daemon mds.e on smithi142 2024-05-10T05:31:31.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:31 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.131:6836/3356790716,v1:172.21.15.131:6837/3356790716] up:boot 2024-05-10T05:31:31.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:31 smithi157 ceph-mon[78427]: fsmap 4 up:standby 2024-05-10T05:31:31.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:31 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T05:31:31.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:31 smithi131 ceph-mon[37264]: Deploying daemon mds.e on smithi142 2024-05-10T05:31:31.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:31 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.131:6836/3356790716,v1:172.21.15.131:6837/3356790716] up:boot 2024-05-10T05:31:31.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:31 smithi131 ceph-mon[37264]: fsmap 4 up:standby 2024-05-10T05:31:31.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:31 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T05:31:31.644 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:31:31 smithi142 bash[114499]: e53cf021c4c04482e1e8a28983c62286263df8560c5009ce45f498e4ddda53b3 2024-05-10T05:31:31.644 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:31:31 smithi142 systemd[1]: Started Ceph mds.e for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:31.644 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:31:31 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-e[114513]: starting mds.e at 2024-05-10T05:31:32.271 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:32 smithi157 ceph-mon[78427]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:32.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:32 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1932742208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:32.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:32 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:32.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:32 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:32.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:32 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:32.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:32 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:32.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:32 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:32.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:32 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:32 smithi142 ceph-mon[77667]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:32 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1932742208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:32 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:32.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:32 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:32.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:32 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:32.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:32 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:32.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:32 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:32.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:32 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:32.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:32 smithi131 ceph-mon[37264]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:32.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:32 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1932742208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:32.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:32 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:32.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:32 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:32.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:32 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:32.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:32 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:32.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:32 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:32.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:32 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:33.017 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:31:32 smithi157 systemd[1]: Starting Ceph mds.f for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:33.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:31:33.303 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:31:33 smithi157 ceph-mds[114619]: -- 172.21.15.157:0/3917829130 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55634e6641a0 con 0x55634e6c4880 2024-05-10T05:31:33.303 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:31:33 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-f[114596]: starting mds.f at 2024-05-10T05:31:33.303 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:31:33 smithi157 bash[114581]: 632df6c0ff83968003ceeacf0d021068d5142304cc1055b000435278129ad616 2024-05-10T05:31:33.303 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:31:33 smithi157 systemd[1]: Started Ceph mds.f for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:33.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:33 smithi157 ceph-mon[78427]: Deploying daemon mds.f on smithi157 2024-05-10T05:31:33.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:33 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.142:6834/3168061272,v1:172.21.15.142:6835/3168061272] up:boot 2024-05-10T05:31:33.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:33 smithi157 ceph-mon[78427]: fsmap 5 up:standby 2024-05-10T05:31:33.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:33 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T05:31:33.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:33 smithi142 ceph-mon[77667]: Deploying daemon mds.f on smithi157 2024-05-10T05:31:33.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:33 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.142:6834/3168061272,v1:172.21.15.142:6835/3168061272] up:boot 2024-05-10T05:31:33.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:33 smithi142 ceph-mon[77667]: fsmap 5 up:standby 2024-05-10T05:31:33.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:33 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T05:31:33.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.442+0000 7fd939600700 1 -- 172.21.15.131:0/3060566623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9340750e0 msgr2=0x7fd9340735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:33.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.442+0000 7fd939600700 1 --2- 172.21.15.131:0/3060566623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9340750e0 0x7fd9340735a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd924009990 tx=0x7fd924009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.442+0000 7fd939600700 1 -- 172.21.15.131:0/3060566623 shutdown_connections 2024-05-10T05:31:33.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.442+0000 7fd939600700 1 --2- 172.21.15.131:0/3060566623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd934109ae0 0x7fd93410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.442+0000 7fd939600700 1 --2- 172.21.15.131:0/3060566623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9340750e0 0x7fd9340735a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.442+0000 7fd939600700 1 --2- 172.21.15.131:0/3060566623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd934073ae0 0x7fd934073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.442+0000 7fd939600700 1 -- 172.21.15.131:0/3060566623 >> 172.21.15.131:0/3060566623 conn(0x7fd9340fd210 msgr2=0x7fd9340ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:33.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.442+0000 7fd939600700 1 -- 172.21.15.131:0/3060566623 shutdown_connections 2024-05-10T05:31:33.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.442+0000 7fd939600700 1 -- 172.21.15.131:0/3060566623 wait complete. 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.443+0000 7fd939600700 1 Processor -- start 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.443+0000 7fd939600700 1 -- start start 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.443+0000 7fd939600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd934073ae0 0x7fd93419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.443+0000 7fd939600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9340750e0 0x7fd93419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.443+0000 7fd939600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd934109ae0 0x7fd9341a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.443+0000 7fd939600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9341a1cc0 con 0x7fd934109ae0 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.443+0000 7fd939600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9341a1e30 con 0x7fd9340750e0 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.443+0000 7fd939600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9341a1ff0 con 0x7fd934073ae0 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd932a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd934073ae0 0x7fd93419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd932a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd934073ae0 0x7fd93419a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59916/0 (socket says 172.21.15.131:59916) 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd932a00700 1 -- 172.21.15.131:0/3731570368 learned_addr learned my addr 172.21.15.131:0/3731570368 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:33.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd932000700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9340750e0 0x7fd93419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd933400700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd934109ae0 0x7fd9341a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd932a00700 1 -- 172.21.15.131:0/3731570368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9340750e0 msgr2=0x7fd93419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd932a00700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9340750e0 0x7fd93419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd932a00700 1 -- 172.21.15.131:0/3731570368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd934109ae0 msgr2=0x7fd9341a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd932a00700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd934109ae0 0x7fd9341a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd932a00700 1 -- 172.21.15.131:0/3731570368 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9240095f0 con 0x7fd934073ae0 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.444+0000 7fd932a00700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd934073ae0 0x7fd93419a1e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9240098e0 tx=0x7fd924005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.445+0000 7fd92b400700 1 -- 172.21.15.131:0/3731570368 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92401d070 con 0x7fd934073ae0 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.445+0000 7fd92b400700 1 -- 172.21.15.131:0/3731570368 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd924021410 con 0x7fd934073ae0 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.445+0000 7fd939600700 1 -- 172.21.15.131:0/3731570368 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9341a2270 con 0x7fd934073ae0 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.445+0000 7fd92b400700 1 -- 172.21.15.131:0/3731570368 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924017650 con 0x7fd934073ae0 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.445+0000 7fd939600700 1 -- 172.21.15.131:0/3731570368 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9341a26f0 con 0x7fd934073ae0 2024-05-10T05:31:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.446+0000 7fd939600700 1 -- 172.21.15.131:0/3731570368 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd934108930 con 0x7fd934073ae0 2024-05-10T05:31:33.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.448+0000 7fd92b400700 1 -- 172.21.15.131:0/3731570368 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd9240217f0 con 0x7fd934073ae0 2024-05-10T05:31:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.449+0000 7fd92b400700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd9180a11a0 0x7fd9180a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.449+0000 7fd92b400700 1 -- 172.21.15.131:0/3731570368 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd9240cd930 con 0x7fd934073ae0 2024-05-10T05:31:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.449+0000 7fd932000700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd9180a11a0 0x7fd9180a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:33.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.450+0000 7fd932000700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd9180a11a0 0x7fd9180a36b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd93419b280 tx=0x7fd9200073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.453+0000 7fd92b400700 1 -- 172.21.15.131:0/3731570368 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd924097550 con 0x7fd934073ae0 2024-05-10T05:31:33.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:33 smithi131 ceph-mon[37264]: Deploying daemon mds.f on smithi157 2024-05-10T05:31:33.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:33 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.142:6834/3168061272,v1:172.21.15.142:6835/3168061272] up:boot 2024-05-10T05:31:33.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:33 smithi131 ceph-mon[37264]: fsmap 5 up:standby 2024-05-10T05:31:33.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:33 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T05:31:33.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.707+0000 7fd939600700 1 -- 172.21.15.131:0/3731570368 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9340694b0 con 0x7fd934073ae0 2024-05-10T05:31:33.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.709+0000 7fd92b400700 1 -- 172.21.15.131:0/3731570368 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4300 (secure 0 0 0) 0x7fd92409ab70 con 0x7fd934073ae0 2024-05-10T05:31:33.710 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:33.710 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 -- 172.21.15.131:0/3731570368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd9180a11a0 msgr2=0x7fd9180a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd9180a11a0 0x7fd9180a36b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd93419b280 tx=0x7fd9200073d0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 -- 172.21.15.131:0/3731570368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd934073ae0 msgr2=0x7fd93419a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd934073ae0 0x7fd93419a1e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9240098e0 tx=0x7fd924005740 comp rx=0 tx=0).stop 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 -- 172.21.15.131:0/3731570368 shutdown_connections 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd9180a11a0 0x7fd9180a36b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd934109ae0 0x7fd9341a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd934073ae0 0x7fd93419a1e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 --2- 172.21.15.131:0/3731570368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9340750e0 0x7fd93419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 -- 172.21.15.131:0/3731570368 >> 172.21.15.131:0/3731570368 conn(0x7fd9340fd210 msgr2=0x7fd934105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:33.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 -- 172.21.15.131:0/3731570368 shutdown_connections 2024-05-10T05:31:33.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:33.712+0000 7fd939600700 1 -- 172.21.15.131:0/3731570368 wait complete. 2024-05-10T05:31:33.714 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 6 2024-05-10T05:31:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:34 smithi157 ceph-mon[78427]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:34.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:34 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:34.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:34 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3731570368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:34 smithi142 ceph-mon[77667]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:34.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:34.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:34.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:34 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:34.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:34 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3731570368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:34.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:34 smithi131 ceph-mon[37264]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:34.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:34.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:34.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:34.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:34.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:34.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:34 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:34.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:34 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3731570368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:34.915 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:31:34 smithi157 systemd[1]: Starting Ceph mds.c for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:35 smithi157 ceph-mon[78427]: Deploying daemon mds.c on smithi157 2024-05-10T05:31:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:35 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.157:6834/72787128,v1:172.21.15.157:6835/72787128] up:boot 2024-05-10T05:31:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:35 smithi157 ceph-mon[78427]: fsmap 6 up:standby 2024-05-10T05:31:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:35 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T05:31:35.303 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:31:35 smithi157 ceph-mds[114962]: -- 172.21.15.157:0/539260328 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561f8e55e1a0 con 0x561f8e5be880 2024-05-10T05:31:35.303 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:31:35 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-c[114939]: starting mds.c at 2024-05-10T05:31:35.303 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:31:35 smithi157 bash[114923]: 861c5f373185c7e033976da097268048a09a84176650e2882dd8fd618c1ad430 2024-05-10T05:31:35.303 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:31:35 smithi157 systemd[1]: Started Ceph mds.c for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:35.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:35 smithi142 ceph-mon[77667]: Deploying daemon mds.c on smithi157 2024-05-10T05:31:35.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:35 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.157:6834/72787128,v1:172.21.15.157:6835/72787128] up:boot 2024-05-10T05:31:35.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:35 smithi142 ceph-mon[77667]: fsmap 6 up:standby 2024-05-10T05:31:35.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:35 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T05:31:35.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:35 smithi131 ceph-mon[37264]: Deploying daemon mds.c on smithi157 2024-05-10T05:31:35.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:35 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.157:6834/72787128,v1:172.21.15.157:6835/72787128] up:boot 2024-05-10T05:31:35.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:35 smithi131 ceph-mon[37264]: fsmap 6 up:standby 2024-05-10T05:31:35.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:35 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T05:31:35.739 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:31:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.013+0000 7f3e09600700 1 -- 172.21.15.131:0/2989984735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e04102b20 msgr2=0x7f3e04102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.013+0000 7f3e09600700 1 --2- 172.21.15.131:0/2989984735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e04102b20 0x7f3e04102f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3df00099c0 tx=0x7f3df0009cd0 comp rx=0 tx=0).stop 2024-05-10T05:31:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.014+0000 7f3e09600700 1 -- 172.21.15.131:0/2989984735 shutdown_connections 2024-05-10T05:31:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.014+0000 7f3e09600700 1 --2- 172.21.15.131:0/2989984735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e041034c0 0x7f3e04109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.014+0000 7f3e09600700 1 --2- 172.21.15.131:0/2989984735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04101a00 0x7f3e04101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.014+0000 7f3e09600700 1 --2- 172.21.15.131:0/2989984735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e04102b20 0x7f3e04102f30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.014+0000 7f3e09600700 1 -- 172.21.15.131:0/2989984735 >> 172.21.15.131:0/2989984735 conn(0x7f3e040fd010 msgr2=0x7f3e040ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:36.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.014+0000 7f3e09600700 1 -- 172.21.15.131:0/2989984735 shutdown_connections 2024-05-10T05:31:36.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.014+0000 7f3e09600700 1 -- 172.21.15.131:0/2989984735 wait complete. 2024-05-10T05:31:36.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.015+0000 7f3e09600700 1 Processor -- start 2024-05-10T05:31:36.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.015+0000 7f3e09600700 1 -- start start 2024-05-10T05:31:36.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.015+0000 7f3e09600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04101a00 0x7f3e04197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:36.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.015+0000 7f3e09600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e04102b20 0x7f3e041982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:36.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.015+0000 7f3e09600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e041034c0 0x7f3e0419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:36.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.015+0000 7f3e09600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0419f790 con 0x7f3e041034c0 2024-05-10T05:31:36.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.015+0000 7f3e09600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0419f8d0 con 0x7f3e04102b20 2024-05-10T05:31:36.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.015+0000 7f3e09600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0419fa90 con 0x7f3e04101a00 2024-05-10T05:31:36.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e04102b20 0x7f3e041982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04101a00 0x7f3e04197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e04102b20 0x7f3e041982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:58102/0 (socket says 172.21.15.131:58102) 2024-05-10T05:31:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04101a00 0x7f3e04197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59932/0 (socket says 172.21.15.131:59932) 2024-05-10T05:31:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03400700 1 -- 172.21.15.131:0/4098191175 learned_addr learned my addr 172.21.15.131:0/4098191175 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e08c00700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e041034c0 0x7f3e0419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03400700 1 -- 172.21.15.131:0/4098191175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04101a00 msgr2=0x7f3e04197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03400700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04101a00 0x7f3e04197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03400700 1 -- 172.21.15.131:0/4098191175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e041034c0 msgr2=0x7f3e0419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03400700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e041034c0 0x7f3e0419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03400700 1 -- 172.21.15.131:0/4098191175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc009c40 con 0x7f3e04102b20 2024-05-10T05:31:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e03400700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e04102b20 0x7f3e041982c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3df00098e0 tx=0x7f3df000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e00c00700 1 -- 172.21.15.131:0/4098191175 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df001d070 con 0x7f3e04102b20 2024-05-10T05:31:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e00c00700 1 -- 172.21.15.131:0/4098191175 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3df00054f0 con 0x7f3e04102b20 2024-05-10T05:31:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e09600700 1 -- 172.21.15.131:0/4098191175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df00095f0 con 0x7f3e04102b20 2024-05-10T05:31:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.016+0000 7f3e00c00700 1 -- 172.21.15.131:0/4098191175 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0004970 con 0x7f3e04102b20 2024-05-10T05:31:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.017+0000 7f3e09600700 1 -- 172.21.15.131:0/4098191175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e040736d0 con 0x7f3e04102b20 2024-05-10T05:31:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.017+0000 7f3e09600700 1 -- 172.21.15.131:0/4098191175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e04108c10 con 0x7f3e04102b20 2024-05-10T05:31:36.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.020+0000 7f3e00c00700 1 -- 172.21.15.131:0/4098191175 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3df0004b90 con 0x7f3e04102b20 2024-05-10T05:31:36.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.021+0000 7f3e00c00700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3de80a11c0 0x7f3de80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:36.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.021+0000 7f3e00c00700 1 -- 172.21.15.131:0/4098191175 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3df00cd300 con 0x7f3e04102b20 2024-05-10T05:31:36.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.021+0000 7f3e03e00700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3de80a11c0 0x7f3de80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:36.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.022+0000 7f3e03e00700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3de80a11c0 0x7f3de80a36d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3e04199120 tx=0x7f3df4012040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:36.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.024+0000 7f3e00c00700 1 -- 172.21.15.131:0/4098191175 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3df0096fc0 con 0x7f3e04102b20 2024-05-10T05:31:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:36 smithi131 ceph-mon[37264]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:36 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:36 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:36 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:36 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:36 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:36 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.277+0000 7f3e09600700 1 -- 172.21.15.131:0/4098191175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e0410d250 con 0x7f3e04102b20 2024-05-10T05:31:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.278+0000 7f3e00c00700 1 -- 172.21.15.131:0/4098191175 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5811 (secure 0 0 0) 0x7f3df009a5e0 con 0x7f3e04102b20 2024-05-10T05:31:36.279 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:36.279 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-10T05:31:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 -- 172.21.15.131:0/4098191175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3de80a11c0 msgr2=0x7f3de80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3de80a11c0 0x7f3de80a36d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3e04199120 tx=0x7f3df4012040 comp rx=0 tx=0).stop 2024-05-10T05:31:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 -- 172.21.15.131:0/4098191175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e04102b20 msgr2=0x7f3e041982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e04102b20 0x7f3e041982c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3df00098e0 tx=0x7f3df000f690 comp rx=0 tx=0).stop 2024-05-10T05:31:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 -- 172.21.15.131:0/4098191175 shutdown_connections 2024-05-10T05:31:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3de80a11c0 0x7f3de80a36d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e041034c0 0x7f3e0419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04101a00 0x7f3e04197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 --2- 172.21.15.131:0/4098191175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e04102b20 0x7f3e041982c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 -- 172.21.15.131:0/4098191175 >> 172.21.15.131:0/4098191175 conn(0x7f3e040fd010 msgr2=0x7f3e04105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 -- 172.21.15.131:0/4098191175 shutdown_connections 2024-05-10T05:31:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:36.280+0000 7f3e09600700 1 -- 172.21.15.131:0/4098191175 wait complete. 2024-05-10T05:31:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 8 2024-05-10T05:31:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:36 smithi157 ceph-mon[78427]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:36 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:36 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:36 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:36.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:36 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:36.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:36 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:36.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:36 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:36.329 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:36 smithi142 ceph-mon[77667]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:36.329 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:36 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:36.329 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:36 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:36.329 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:36 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:36.329 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:36 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:36.329 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:36 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:36.329 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:36 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:36.644 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:31:36 smithi142 systemd[1]: Starting Ceph mds.h for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:37.144 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:31:36 smithi142 ceph-mds[114875]: -- 172.21.15.142:0/287466751 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5646db9281a0 con 0x5646db988400 2024-05-10T05:31:37.144 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:31:36 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-h[114852]: starting mds.h at 2024-05-10T05:31:37.145 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:31:36 smithi142 bash[114837]: fe535d2872834aa2235621da275a99e1de62ad43ef70fec81c65ac316f4cbab0 2024-05-10T05:31:37.145 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:31:36 smithi142 systemd[1]: Started Ceph mds.h for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:37.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: Deploying daemon mds.h on smithi142 2024-05-10T05:31:37.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] up:boot 2024-05-10T05:31:37.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: fsmap 7 up:standby 2024-05-10T05:31:37.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T05:31:37.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4098191175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:37.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:37.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:37.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:37.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:37.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:37.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:37 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: Deploying daemon mds.h on smithi142 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] up:boot 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: fsmap 7 up:standby 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4098191175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:37 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:37.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: Deploying daemon mds.h on smithi142 2024-05-10T05:31:37.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] up:boot 2024-05-10T05:31:37.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: fsmap 7 up:standby 2024-05-10T05:31:37.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T05:31:37.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4098191175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:37.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:37.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:37.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:37.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:37.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:37.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:37 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:38.267 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:38 smithi157 ceph-mon[78427]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:38.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:38 smithi157 ceph-mon[78427]: Deploying daemon mds.i on smithi157 2024-05-10T05:31:38.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:38 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.142:6836/2645029376,v1:172.21.15.142:6837/2645029376] up:boot 2024-05-10T05:31:38.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:38 smithi157 ceph-mon[78427]: fsmap 8 up:standby 2024-05-10T05:31:38.268 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:38 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T05:31:38.268 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:31:38 smithi157 systemd[1]: Starting Ceph mds.i for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:38.308 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:31:38.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:38 smithi131 ceph-mon[37264]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:38.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:38 smithi131 ceph-mon[37264]: Deploying daemon mds.i on smithi157 2024-05-10T05:31:38.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:38 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.142:6836/2645029376,v1:172.21.15.142:6837/2645029376] up:boot 2024-05-10T05:31:38.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:38 smithi131 ceph-mon[37264]: fsmap 8 up:standby 2024-05-10T05:31:38.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:38 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T05:31:38.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:38 smithi142 ceph-mon[77667]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:38.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:38 smithi142 ceph-mon[77667]: Deploying daemon mds.i on smithi157 2024-05-10T05:31:38.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:38 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.142:6836/2645029376,v1:172.21.15.142:6837/2645029376] up:boot 2024-05-10T05:31:38.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:38 smithi142 ceph-mon[77667]: fsmap 8 up:standby 2024-05-10T05:31:38.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:38 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T05:31:38.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.582+0000 7ff137200700 1 -- 172.21.15.131:0/3490262522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff130101be0 msgr2=0x7ff130101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.582+0000 7ff137200700 1 --2- 172.21.15.131:0/3490262522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff130101be0 0x7ff130101ff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff120009920 tx=0x7ff120009c30 comp rx=0 tx=0).stop 2024-05-10T05:31:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.582+0000 7ff137200700 1 -- 172.21.15.131:0/3490262522 shutdown_connections 2024-05-10T05:31:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.582+0000 7ff137200700 1 --2- 172.21.15.131:0/3490262522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1301036a0 0x7ff130109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.582+0000 7ff137200700 1 --2- 172.21.15.131:0/3490262522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff130101be0 0x7ff130101ff0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.582+0000 7ff137200700 1 --2- 172.21.15.131:0/3490262522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff130102d00 0x7ff130103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.582+0000 7ff137200700 1 -- 172.21.15.131:0/3490262522 >> 172.21.15.131:0/3490262522 conn(0x7ff1300fd210 msgr2=0x7ff1300ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.582+0000 7ff137200700 1 -- 172.21.15.131:0/3490262522 shutdown_connections 2024-05-10T05:31:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.582+0000 7ff137200700 1 -- 172.21.15.131:0/3490262522 wait complete. 2024-05-10T05:31:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.584+0000 7ff137200700 1 Processor -- start 2024-05-10T05:31:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.584+0000 7ff137200700 1 -- start start 2024-05-10T05:31:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.584+0000 7ff137200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff130101be0 0x7ff1301980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.584+0000 7ff137200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff130102d00 0x7ff130198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.584+0000 7ff137200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1301036a0 0x7ff13019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.584+0000 7ff137200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13019fb80 con 0x7ff1301036a0 2024-05-10T05:31:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.584+0000 7ff137200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13019fcf0 con 0x7ff130102d00 2024-05-10T05:31:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.584+0000 7ff137200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13019feb0 con 0x7ff130101be0 2024-05-10T05:31:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff135200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1301036a0 0x7ff13019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff12fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff130101be0 0x7ff1301980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff12f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff130102d00 0x7ff130198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff135200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1301036a0 0x7ff13019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43298/0 (socket says 172.21.15.131:43298) 2024-05-10T05:31:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff12f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff130102d00 0x7ff130198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:58126/0 (socket says 172.21.15.131:58126) 2024-05-10T05:31:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff135200700 1 -- 172.21.15.131:0/984917522 learned_addr learned my addr 172.21.15.131:0/984917522 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff135200700 1 -- 172.21.15.131:0/984917522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff130101be0 msgr2=0x7ff1301980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff135200700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff130101be0 0x7ff1301980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff135200700 1 -- 172.21.15.131:0/984917522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff130102d00 msgr2=0x7ff130198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff135200700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff130102d00 0x7ff130198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff135200700 1 -- 172.21.15.131:0/984917522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128009d00 con 0x7ff1301036a0 2024-05-10T05:31:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff12f400700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff130102d00 0x7ff130198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff135200700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1301036a0 0x7ff13019f570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff12800e3c0 tx=0x7ff12800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.585+0000 7ff12cc00700 1 -- 172.21.15.131:0/984917522 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1280090d0 con 0x7ff1301036a0 2024-05-10T05:31:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.586+0000 7ff137200700 1 -- 172.21.15.131:0/984917522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff120009520 con 0x7ff1301036a0 2024-05-10T05:31:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.586+0000 7ff12cc00700 1 -- 172.21.15.131:0/984917522 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff12800f040 con 0x7ff1301036a0 2024-05-10T05:31:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.586+0000 7ff12cc00700 1 -- 172.21.15.131:0/984917522 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128004dd0 con 0x7ff1301036a0 2024-05-10T05:31:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.586+0000 7ff137200700 1 -- 172.21.15.131:0/984917522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1301a04c0 con 0x7ff1301036a0 2024-05-10T05:31:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.587+0000 7ff137200700 1 -- 172.21.15.131:0/984917522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff130108e60 con 0x7ff1301036a0 2024-05-10T05:31:38.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.589+0000 7ff12cc00700 1 -- 172.21.15.131:0/984917522 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff128003a00 con 0x7ff1301036a0 2024-05-10T05:31:38.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.590+0000 7ff12cc00700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff1140a1120 0x7ff1140a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:38.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.590+0000 7ff12cc00700 1 -- 172.21.15.131:0/984917522 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff1280cc920 con 0x7ff1301036a0 2024-05-10T05:31:38.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.590+0000 7ff12fe00700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff1140a1120 0x7ff1140a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:38.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.591+0000 7ff12fe00700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff1140a1120 0x7ff1140a3630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff12000fab0 tx=0x7ff120019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:38.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.593+0000 7ff12cc00700 1 -- 172.21.15.131:0/984917522 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff128096540 con 0x7ff1301036a0 2024-05-10T05:31:38.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.859+0000 7ff137200700 1 -- 172.21.15.131:0/984917522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1300694d0 con 0x7ff1301036a0 2024-05-10T05:31:38.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.860+0000 7ff12cc00700 1 -- 172.21.15.131:0/984917522 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6571 (secure 0 0 0) 0x7ff128099b60 con 0x7ff1301036a0 2024-05-10T05:31:38.861 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:38.862 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-10T05:31:38.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.862+0000 7ff137200700 1 -- 172.21.15.131:0/984917522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff1140a1120 msgr2=0x7ff1140a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:38.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.862+0000 7ff137200700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff1140a1120 0x7ff1140a3630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff12000fab0 tx=0x7ff120019040 comp rx=0 tx=0).stop 2024-05-10T05:31:38.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.862+0000 7ff137200700 1 -- 172.21.15.131:0/984917522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1301036a0 msgr2=0x7ff13019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:38.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.862+0000 7ff137200700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1301036a0 0x7ff13019f570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff12800e3c0 tx=0x7ff12800e6d0 comp rx=0 tx=0).stop 2024-05-10T05:31:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.863+0000 7ff137200700 1 -- 172.21.15.131:0/984917522 shutdown_connections 2024-05-10T05:31:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.863+0000 7ff137200700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff1140a1120 0x7ff1140a3630 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.863+0000 7ff137200700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1301036a0 0x7ff13019f570 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.863+0000 7ff137200700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff130101be0 0x7ff1301980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.863+0000 7ff137200700 1 --2- 172.21.15.131:0/984917522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff130102d00 0x7ff130198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.863+0000 7ff137200700 1 -- 172.21.15.131:0/984917522 >> 172.21.15.131:0/984917522 conn(0x7ff1300fd210 msgr2=0x7ff130105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.863+0000 7ff137200700 1 -- 172.21.15.131:0/984917522 shutdown_connections 2024-05-10T05:31:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:38.863+0000 7ff137200700 1 -- 172.21.15.131:0/984917522 wait complete. 2024-05-10T05:31:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-05-10T05:31:39.053 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:31:38 smithi157 bash[115264]: 58c40dc5938e49aaef1f975752e9ecf5938963875e044293b88e904ebee6b50a 2024-05-10T05:31:39.053 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:31:38 smithi157 systemd[1]: Started Ceph mds.i for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:39.053 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:31:38 smithi157 ceph-mds[115302]: -- 172.21.15.157:0/146207094 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557704ca61a0 con 0x557704d06400 2024-05-10T05:31:39.053 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:31:38 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-i[115279]: starting mds.i at 2024-05-10T05:31:39.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:39.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:39.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:39.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:39.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:39.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:39.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: Deploying daemon mds.g on smithi131 2024-05-10T05:31:39.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:39.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/984917522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:39.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] up:boot 2024-05-10T05:31:39.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: fsmap 9 up:standby 2024-05-10T05:31:39.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:39 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T05:31:40.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:40.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:40.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:40.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:40.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:40.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:40.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: Deploying daemon mds.g on smithi131 2024-05-10T05:31:40.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:40.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/984917522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:40.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] up:boot 2024-05-10T05:31:40.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: fsmap 9 up:standby 2024-05-10T05:31:40.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:39 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T05:31:40.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: Deploying daemon mds.g on smithi131 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/984917522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] up:boot 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: fsmap 9 up:standby 2024-05-10T05:31:40.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:39 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T05:31:40.710 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:31:40 smithi131 systemd[1]: Starting Ceph mds.g for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:40.894 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:31:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.186+0000 7f1f0f600700 1 -- 172.21.15.131:0/1681644024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f081058c0 msgr2=0x7f1f0810c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.186+0000 7f1f0f600700 1 --2- 172.21.15.131:0/1681644024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f081058c0 0x7f1f0810c170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000b0a0 tx=0x7f1f0000b3b0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.186+0000 7f1f0f600700 1 -- 172.21.15.131:0/1681644024 shutdown_connections 2024-05-10T05:31:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.186+0000 7f1f0f600700 1 --2- 172.21.15.131:0/1681644024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f081058c0 0x7f1f0810c170 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.186+0000 7f1f0f600700 1 --2- 172.21.15.131:0/1681644024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f08103e00 0x7f1f08104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.186+0000 7f1f0f600700 1 --2- 172.21.15.131:0/1681644024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f08104f20 0x7f1f08105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.186+0000 7f1f0f600700 1 -- 172.21.15.131:0/1681644024 >> 172.21.15.131:0/1681644024 conn(0x7f1f080ff410 msgr2=0x7f1f081018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.186+0000 7f1f0f600700 1 -- 172.21.15.131:0/1681644024 shutdown_connections 2024-05-10T05:31:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.186+0000 7f1f0f600700 1 -- 172.21.15.131:0/1681644024 wait complete. 2024-05-10T05:31:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.187+0000 7f1f0f600700 1 Processor -- start 2024-05-10T05:31:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.187+0000 7f1f0f600700 1 -- start start 2024-05-10T05:31:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.187+0000 7f1f0f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f08103e00 0x7f1f0819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f0f600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f08104f20 0x7f1f0819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f0f600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f081058c0 0x7f1f081a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f0f600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081a1d80 con 0x7f1f08104f20 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f0f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081a1ef0 con 0x7f1f08103e00 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f0f600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081a20b0 con 0x7f1f081058c0 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f07e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f08104f20 0x7f1f0819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f07e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f08104f20 0x7f1f0819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43348/0 (socket says 172.21.15.131:43348) 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f07e00700 1 -- 172.21.15.131:0/2133305279 learned_addr learned my addr 172.21.15.131:0/2133305279 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f0cc00700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f08103e00 0x7f1f0819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f0d600700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f081058c0 0x7f1f081a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f07e00700 1 -- 172.21.15.131:0/2133305279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f081058c0 msgr2=0x7f1f081a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f07e00700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f081058c0 0x7f1f081a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f07e00700 1 -- 172.21.15.131:0/2133305279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f08103e00 msgr2=0x7f1f0819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f07e00700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f08103e00 0x7f1f0819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f07e00700 1 -- 172.21.15.131:0/2133305279 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f00009d00 con 0x7f1f08104f20 2024-05-10T05:31:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.188+0000 7f1f0cc00700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f08103e00 0x7f1f0819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.189+0000 7f1f07e00700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f08104f20 0x7f1f0819a7e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400e9a0 tx=0x7f1ef400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.189+0000 7f1f05600700 1 -- 172.21.15.131:0/2133305279 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4009790 con 0x7f1f08104f20 2024-05-10T05:31:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.189+0000 7f1f05600700 1 -- 172.21.15.131:0/2133305279 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef4010430 con 0x7f1f08104f20 2024-05-10T05:31:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.189+0000 7f1f0f600700 1 -- 172.21.15.131:0/2133305279 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f081a2390 con 0x7f1f08104f20 2024-05-10T05:31:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.189+0000 7f1f05600700 1 -- 172.21.15.131:0/2133305279 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4018680 con 0x7f1f08104f20 2024-05-10T05:31:41.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.189+0000 7f1f0f600700 1 -- 172.21.15.131:0/2133305279 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f081a28a0 con 0x7f1f08104f20 2024-05-10T05:31:41.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.190+0000 7f1f0f600700 1 -- 172.21.15.131:0/2133305279 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0810b010 con 0x7f1f08104f20 2024-05-10T05:31:41.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.193+0000 7f1f05600700 1 -- 172.21.15.131:0/2133305279 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1ef40105a0 con 0x7f1f08104f20 2024-05-10T05:31:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.195+0000 7f1f05600700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ef00a1120 0x7f1ef00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.195+0000 7f1f05600700 1 -- 172.21.15.131:0/2133305279 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1ef4014070 con 0x7f1f08104f20 2024-05-10T05:31:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.195+0000 7f1f0cc00700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ef00a1120 0x7f1ef00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.196+0000 7f1f0cc00700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ef00a1120 0x7f1ef00a3630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1f08104e20 tx=0x7f1efc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:41.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.197+0000 7f1f05600700 1 -- 172.21.15.131:0/2133305279 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ef4096400 con 0x7f1f08104f20 2024-05-10T05:31:41.210 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:31:40 smithi131 bash[88194]: 23194a56e2d4bf9e5301a22f832c9ca802ba409b44d5cc64b4574bd5456de122 2024-05-10T05:31:41.210 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:31:40 smithi131 systemd[1]: Started Ceph mds.g for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:41.210 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:31:41 smithi131 ceph-mds[88251]: -- 172.21.15.131:0/3344012310 <== mon.0 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56170485c1a0 con 0x5617048be400 2024-05-10T05:31:41.210 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:31:41 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-g[88209]: starting mds.g at 2024-05-10T05:31:41.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.464+0000 7f1f0f600700 1 -- 172.21.15.131:0/2133305279 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f08068730 con 0x7f1f08104f20 2024-05-10T05:31:41.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.465+0000 7f1f05600700 1 -- 172.21.15.131:0/2133305279 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7333 (secure 0 0 0) 0x7f1ef4099a20 con 0x7f1f08104f20 2024-05-10T05:31:41.464 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:41.465 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-10T05:31:41.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 -- 172.21.15.131:0/2133305279 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ef00a1120 msgr2=0x7f1ef00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ef00a1120 0x7f1ef00a3630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1f08104e20 tx=0x7f1efc009260 comp rx=0 tx=0).stop 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 -- 172.21.15.131:0/2133305279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f08104f20 msgr2=0x7f1f0819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f08104f20 0x7f1f0819a7e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400e9a0 tx=0x7f1ef400ecb0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 -- 172.21.15.131:0/2133305279 shutdown_connections 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ef00a1120 0x7f1ef00a3630 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f081058c0 0x7f1f081a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f08103e00 0x7f1f0819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 --2- 172.21.15.131:0/2133305279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f08104f20 0x7f1f0819a7e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 -- 172.21.15.131:0/2133305279 >> 172.21.15.131:0/2133305279 conn(0x7f1f080ff410 msgr2=0x7f1f08108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 -- 172.21.15.131:0/2133305279 shutdown_connections 2024-05-10T05:31:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:41.468+0000 7f1f0f600700 1 -- 172.21.15.131:0/2133305279 wait complete. 2024-05-10T05:31:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 10 2024-05-10T05:31:42.132 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:41 smithi131 ceph-mon[37264]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:42.132 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:41 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:42.132 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:41 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:42.133 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:41 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:42.133 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:41 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:42.133 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:41 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:42.133 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:41 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:42.133 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:41 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2133305279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:42.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:41 smithi142 ceph-mon[77667]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:42.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:41 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:42.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:41 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:42.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:41 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:42.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:41 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:42.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:41 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:42.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:41 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:42.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:41 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2133305279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:42.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:41 smithi157 ceph-mon[78427]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:42.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:41 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:42.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:41 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:42.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:41 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:42.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:41 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:42.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:41 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:42.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:41 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:42.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:41 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2133305279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:42 smithi131 ceph-mon[37264]: Deploying daemon mds.a on smithi131 2024-05-10T05:31:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:42 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] up:boot 2024-05-10T05:31:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:42 smithi131 ceph-mon[37264]: fsmap 10 up:standby 2024-05-10T05:31:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:42 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T05:31:43.019 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:31:43 smithi131 systemd[1]: Starting Ceph mds.a for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:43.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:42 smithi142 ceph-mon[77667]: Deploying daemon mds.a on smithi131 2024-05-10T05:31:43.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:42 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] up:boot 2024-05-10T05:31:43.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:42 smithi142 ceph-mon[77667]: fsmap 10 up:standby 2024-05-10T05:31:43.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:42 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T05:31:43.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:42 smithi157 ceph-mon[78427]: Deploying daemon mds.a on smithi131 2024-05-10T05:31:43.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:42 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] up:boot 2024-05-10T05:31:43.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:42 smithi157 ceph-mon[78427]: fsmap 10 up:standby 2024-05-10T05:31:43.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:42 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T05:31:43.501 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:31:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.785+0000 7f2266600700 1 -- 172.21.15.131:0/417722304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2260101be0 msgr2=0x7f2260101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:43.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.785+0000 7f2266600700 1 --2- 172.21.15.131:0/417722304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2260101be0 0x7f2260101ff0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f22500099c0 tx=0x7f2250009cd0 comp rx=0 tx=0).stop 2024-05-10T05:31:43.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.785+0000 7f2266600700 1 -- 172.21.15.131:0/417722304 shutdown_connections 2024-05-10T05:31:43.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.785+0000 7f2266600700 1 --2- 172.21.15.131:0/417722304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22601036a0 0x7f2260109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:43.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.785+0000 7f2266600700 1 --2- 172.21.15.131:0/417722304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2260101be0 0x7f2260101ff0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:43.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.785+0000 7f2266600700 1 --2- 172.21.15.131:0/417722304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2260102d00 0x7f2260103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:43.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.785+0000 7f2266600700 1 -- 172.21.15.131:0/417722304 >> 172.21.15.131:0/417722304 conn(0x7f22600fd1d0 msgr2=0x7f22600ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:43.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.785+0000 7f2266600700 1 -- 172.21.15.131:0/417722304 shutdown_connections 2024-05-10T05:31:43.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.785+0000 7f2266600700 1 -- 172.21.15.131:0/417722304 wait complete. 2024-05-10T05:31:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.786+0000 7f2266600700 1 Processor -- start 2024-05-10T05:31:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.786+0000 7f2266600700 1 -- start start 2024-05-10T05:31:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f2266600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2260101be0 0x7f226019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:43.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f2266600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2260102d00 0x7f226019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:43.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f2266600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22601036a0 0x7f22601a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:43.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f2266600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22601a1d20 con 0x7f22601036a0 2024-05-10T05:31:43.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f2266600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22601a1e90 con 0x7f2260101be0 2024-05-10T05:31:43.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f2266600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22601a2050 con 0x7f2260102d00 2024-05-10T05:31:43.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f225f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2260101be0 0x7f226019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:43.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f225f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2260101be0 0x7f226019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:36328/0 (socket says 172.21.15.131:36328) 2024-05-10T05:31:43.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f225f400700 1 -- 172.21.15.131:0/1220500184 learned_addr learned my addr 172.21.15.131:0/1220500184 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:43.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f225fe00700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22601036a0 0x7f22601a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:43.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.787+0000 7f225ea00700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2260102d00 0x7f226019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:43.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f225f400700 1 -- 172.21.15.131:0/1220500184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2260102d00 msgr2=0x7f226019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:43.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f225f400700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2260102d00 0x7f226019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:43.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f225f400700 1 -- 172.21.15.131:0/1220500184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22601036a0 msgr2=0x7f22601a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:43.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f225f400700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22601036a0 0x7f22601a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:43.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f225f400700 1 -- 172.21.15.131:0/1220500184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22500095f0 con 0x7f2260101be0 2024-05-10T05:31:43.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f225f400700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2260101be0 0x7f226019a240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22500058e0 tx=0x7f225000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:43.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f2257e00700 1 -- 172.21.15.131:0/1220500184 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225001d070 con 0x7f2260101be0 2024-05-10T05:31:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f2266600700 1 -- 172.21.15.131:0/1220500184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22601a22d0 con 0x7f2260101be0 2024-05-10T05:31:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f2257e00700 1 -- 172.21.15.131:0/1220500184 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2250005050 con 0x7f2260101be0 2024-05-10T05:31:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f2257e00700 1 -- 172.21.15.131:0/1220500184 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2250004950 con 0x7f2260101be0 2024-05-10T05:31:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.788+0000 7f2266600700 1 -- 172.21.15.131:0/1220500184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22601a2700 con 0x7f2260101be0 2024-05-10T05:31:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.789+0000 7f2266600700 1 -- 172.21.15.131:0/1220500184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22600694b0 con 0x7f2260101be0 2024-05-10T05:31:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.791+0000 7f2257e00700 1 -- 172.21.15.131:0/1220500184 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2250005430 con 0x7f2260101be0 2024-05-10T05:31:43.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.792+0000 7f2257e00700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f22440a1120 0x7f22440a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:43.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.793+0000 7f2257e00700 1 -- 172.21.15.131:0/1220500184 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f22500cd200 con 0x7f2260101be0 2024-05-10T05:31:43.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.793+0000 7f225ea00700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f22440a1120 0x7f22440a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:43.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.793+0000 7f225ea00700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f22440a1120 0x7f22440a3630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f226019b2e0 tx=0x7f224c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:43.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:43.797+0000 7f2257e00700 1 -- 172.21.15.131:0/1220500184 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2250096e20 con 0x7f2260101be0 2024-05-10T05:31:43.891 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:31:43 smithi131 bash[88584]: 48c990417b3cf2a43fef5390c05ba15160f5f1454f27313c0b251b41b96b4007 2024-05-10T05:31:43.892 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:31:43 smithi131 systemd[1]: Started Ceph mds.a for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:43.892 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:31:43 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-a[88598]: starting mds.a at 2024-05-10T05:31:44.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.051+0000 7f2266600700 1 -- 172.21.15.131:0/1220500184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22601a29c0 con 0x7f2260101be0 2024-05-10T05:31:44.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.052+0000 7f2257e00700 1 -- 172.21.15.131:0/1220500184 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8094 (secure 0 0 0) 0x7f225009a440 con 0x7f2260101be0 2024-05-10T05:31:44.052 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:44.052 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[]} 2024-05-10T05:31:44.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 -- 172.21.15.131:0/1220500184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f22440a1120 msgr2=0x7f22440a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:44.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f22440a1120 0x7f22440a3630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f226019b2e0 tx=0x7f224c009260 comp rx=0 tx=0).stop 2024-05-10T05:31:44.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 -- 172.21.15.131:0/1220500184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2260101be0 msgr2=0x7f226019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:44.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2260101be0 0x7f226019a240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22500058e0 tx=0x7f225000f690 comp rx=0 tx=0).stop 2024-05-10T05:31:44.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 -- 172.21.15.131:0/1220500184 shutdown_connections 2024-05-10T05:31:44.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f22440a1120 0x7f22440a3630 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:44.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22601036a0 0x7f22601a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:44.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2260101be0 0x7f226019a240 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:44.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 --2- 172.21.15.131:0/1220500184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2260102d00 0x7f226019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:44.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 -- 172.21.15.131:0/1220500184 >> 172.21.15.131:0/1220500184 conn(0x7f22600fd1d0 msgr2=0x7f2260073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:44.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 -- 172.21.15.131:0/1220500184 shutdown_connections 2024-05-10T05:31:44.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:44.055+0000 7f2266600700 1 -- 172.21.15.131:0/1220500184 wait complete. 2024-05-10T05:31:44.056 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 11 2024-05-10T05:31:44.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:43 smithi142 ceph-mon[77667]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:44.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:43 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:44.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:43 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:44.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:43 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:44.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:43 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:44.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:43 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:44.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:43 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:44.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:43 smithi131 ceph-mon[37264]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:44.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:44.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:44.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:44.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:44.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:44.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:43 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:43 smithi157 ceph-mon[78427]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:43 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:43 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:43 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:43 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T05:31:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:43 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T05:31:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:43 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:45.204 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:44 smithi142 ceph-mon[77667]: Deploying daemon mds.b on smithi142 2024-05-10T05:31:45.204 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:44 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1220500184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:45.204 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:44 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] up:boot 2024-05-10T05:31:45.204 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:44 smithi142 ceph-mon[77667]: fsmap 11 up:standby 2024-05-10T05:31:45.204 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:44 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T05:31:45.205 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:31:44 smithi142 systemd[1]: Starting Ceph mds.b for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:31:45.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:44 smithi131 ceph-mon[37264]: Deploying daemon mds.b on smithi142 2024-05-10T05:31:45.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:44 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1220500184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:45.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:44 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] up:boot 2024-05-10T05:31:45.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:44 smithi131 ceph-mon[37264]: fsmap 11 up:standby 2024-05-10T05:31:45.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:44 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T05:31:45.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:44 smithi157 ceph-mon[78427]: Deploying daemon mds.b on smithi142 2024-05-10T05:31:45.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:44 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1220500184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:45.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:44 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] up:boot 2024-05-10T05:31:45.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:44 smithi157 ceph-mon[78427]: fsmap 11 up:standby 2024-05-10T05:31:45.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:44 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T05:31:45.644 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:31:45 smithi142 bash[115188]: b7f8069e25b10ca6b2dda7982ed0f15d16bb87afb47c8b8d96a3aa9e47537922 2024-05-10T05:31:45.644 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:31:45 smithi142 systemd[1]: Started Ceph mds.b for 6e8de474-0e8d-11ef-bc98-c7b262605968. 2024-05-10T05:31:45.644 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:31:45 smithi142 ceph-mds[115226]: -- 172.21.15.142:0/1159580998 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ae165a61a0 con 0x55ae16606400 2024-05-10T05:31:45.644 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:31:45 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-b[115203]: starting mds.b at 2024-05-10T05:31:46.083 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:31:46.152 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:45 smithi131 ceph-mon[37264]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:46.152 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:45 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.152 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:45 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.152 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:45 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.152 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:45 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.152 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:45 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.153 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:45 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:45 smithi157 ceph-mon[78427]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:45 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:45 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:45 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:45 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:45 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:45 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:46.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.373+0000 7fa0b9200700 1 -- 172.21.15.131:0/3504016992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b41036a0 msgr2=0x7fa0b4109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:46.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.373+0000 7fa0b9200700 1 --2- 172.21.15.131:0/3504016992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b41036a0 0x7fa0b4109f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00b0d0 tx=0x7fa0ac00b3e0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.373+0000 7fa0b9200700 1 -- 172.21.15.131:0/3504016992 shutdown_connections 2024-05-10T05:31:46.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.373+0000 7fa0b9200700 1 --2- 172.21.15.131:0/3504016992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b41036a0 0x7fa0b4109f50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.373+0000 7fa0b9200700 1 --2- 172.21.15.131:0/3504016992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4101be0 0x7fa0b4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.373+0000 7fa0b9200700 1 --2- 172.21.15.131:0/3504016992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b4102d00 0x7fa0b4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.373+0000 7fa0b9200700 1 -- 172.21.15.131:0/3504016992 >> 172.21.15.131:0/3504016992 conn(0x7fa0b40fd1d0 msgr2=0x7fa0b40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:46.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.374+0000 7fa0b9200700 1 -- 172.21.15.131:0/3504016992 shutdown_connections 2024-05-10T05:31:46.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.374+0000 7fa0b9200700 1 -- 172.21.15.131:0/3504016992 wait complete. 2024-05-10T05:31:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b9200700 1 Processor -- start 2024-05-10T05:31:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b9200700 1 -- start start 2024-05-10T05:31:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b4101be0 0x7fa0b419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b9200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102d00 0x7fa0b419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b9200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b41036a0 0x7fa0b41a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b9200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b41a1dd0 con 0x7fa0b41036a0 2024-05-10T05:31:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b9200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b41a1f40 con 0x7fa0b4102d00 2024-05-10T05:31:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b9200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b41a2100 con 0x7fa0b4101be0 2024-05-10T05:31:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b4101be0 0x7fa0b419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b41036a0 0x7fa0b41a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102d00 0x7fa0b419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b4101be0 0x7fa0b419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34116/0 (socket says 172.21.15.131:34116) 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102d00 0x7fa0b419a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:36350/0 (socket says 172.21.15.131:36350) 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b1e00700 1 -- 172.21.15.131:0/601149847 learned_addr learned my addr 172.21.15.131:0/601149847 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.375+0000 7fa0b2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b41036a0 0x7fa0b41a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40836/0 (socket says 172.21.15.131:40836) 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0b2800700 1 -- 172.21.15.131:0/601149847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b4101be0 msgr2=0x7fa0b419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0b2800700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b4101be0 0x7fa0b419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0b2800700 1 -- 172.21.15.131:0/601149847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102d00 msgr2=0x7fa0b419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0b2800700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102d00 0x7fa0b419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0b2800700 1 -- 172.21.15.131:0/601149847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a0009520 con 0x7fa0b41036a0 2024-05-10T05:31:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0b1400700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102d00 0x7fa0b419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:46.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0b2800700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b41036a0 0x7fa0b41a17c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009fd0 tx=0x7fa0ac008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:46.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0aaa00700 1 -- 172.21.15.131:0/601149847 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00e040 con 0x7fa0b41036a0 2024-05-10T05:31:46.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0b9200700 1 -- 172.21.15.131:0/601149847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac009d00 con 0x7fa0b41036a0 2024-05-10T05:31:46.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0aaa00700 1 -- 172.21.15.131:0/601149847 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ac005070 con 0x7fa0b41036a0 2024-05-10T05:31:46.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0aaa00700 1 -- 172.21.15.131:0/601149847 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac01d6d0 con 0x7fa0b41036a0 2024-05-10T05:31:46.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.376+0000 7fa0b9200700 1 -- 172.21.15.131:0/601149847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b41a2670 con 0x7fa0b41036a0 2024-05-10T05:31:46.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.377+0000 7fa0b9200700 1 -- 172.21.15.131:0/601149847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b40694d0 con 0x7fa0b41036a0 2024-05-10T05:31:46.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.382+0000 7fa0aaa00700 1 -- 172.21.15.131:0/601149847 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa0ac0051e0 con 0x7fa0b41036a0 2024-05-10T05:31:46.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.383+0000 7fa0aaa00700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0980a1120 0x7fa0980a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:46.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.383+0000 7fa0aaa00700 1 -- 172.21.15.131:0/601149847 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa0ac0cd4a0 con 0x7fa0b41036a0 2024-05-10T05:31:46.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.383+0000 7fa0b1e00700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0980a1120 0x7fa0980a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:46.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.384+0000 7fa0b1e00700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0980a1120 0x7fa0980a3630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009cc0 tx=0x7fa0a4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:46.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.385+0000 7fa0aaa00700 1 -- 172.21.15.131:0/601149847 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa0ac097150 con 0x7fa0b41036a0 2024-05-10T05:31:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:45 smithi142 ceph-mon[77667]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:45 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:45 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:45 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:45 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:45 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:45 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:46.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.650+0000 7fa0b9200700 1 -- 172.21.15.131:0/601149847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0b410d440 con 0x7fa0b41036a0 2024-05-10T05:31:46.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.651+0000 7fa0aaa00700 1 -- 172.21.15.131:0/601149847 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9616 (secure 0 0 0) 0x7fa0ac09a770 con 0x7fa0b41036a0 2024-05-10T05:31:46.651 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:46.651 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24500,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-10T05:31:46.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.653+0000 7fa0b9200700 1 -- 172.21.15.131:0/601149847 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0980a1120 msgr2=0x7fa0980a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.653+0000 7fa0b9200700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0980a1120 0x7fa0980a3630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009cc0 tx=0x7fa0a4009260 comp rx=0 tx=0).stop 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.653+0000 7fa0b9200700 1 -- 172.21.15.131:0/601149847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b41036a0 msgr2=0x7fa0b41a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.653+0000 7fa0b9200700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b41036a0 0x7fa0b41a17c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009fd0 tx=0x7fa0ac008e40 comp rx=0 tx=0).stop 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.654+0000 7fa0b9200700 1 -- 172.21.15.131:0/601149847 shutdown_connections 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.654+0000 7fa0b9200700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0980a1120 0x7fa0980a3630 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.654+0000 7fa0b9200700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b41036a0 0x7fa0b41a17c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.654+0000 7fa0b9200700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b4101be0 0x7fa0b419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.654+0000 7fa0b9200700 1 --2- 172.21.15.131:0/601149847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102d00 0x7fa0b419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.654+0000 7fa0b9200700 1 -- 172.21.15.131:0/601149847 >> 172.21.15.131:0/601149847 conn(0x7fa0b40fd1d0 msgr2=0x7fa0b4073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.654+0000 7fa0b9200700 1 -- 172.21.15.131:0/601149847 shutdown_connections 2024-05-10T05:31:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.654+0000 7fa0b9200700 1 -- 172.21.15.131:0/601149847 wait complete. 2024-05-10T05:31:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 13 2024-05-10T05:31:46.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:31:46.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:46 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] up:boot 2024-05-10T05:31:46.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:46 smithi131 ceph-mon[37264]: fsmap 12 up:standby 2024-05-10T05:31:46.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T05:31:46.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:46 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/601149847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:46.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:46 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:46.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.973+0000 7f28e5000700 1 -- 172.21.15.131:0/2682492375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e0101be0 msgr2=0x7f28e0101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:46.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.973+0000 7f28e5000700 1 --2- 172.21.15.131:0/2682492375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e0101be0 0x7f28e0101ff0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009990 tx=0x7f28d0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.973+0000 7f28e5000700 1 -- 172.21.15.131:0/2682492375 shutdown_connections 2024-05-10T05:31:46.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.973+0000 7f28e5000700 1 --2- 172.21.15.131:0/2682492375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e0101be0 0x7f28e0101ff0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.973+0000 7f28e5000700 1 --2- 172.21.15.131:0/2682492375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e01036a0 0x7f28e0109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.973+0000 7f28e5000700 1 --2- 172.21.15.131:0/2682492375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e0102d00 0x7f28e0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.973+0000 7f28e5000700 1 -- 172.21.15.131:0/2682492375 >> 172.21.15.131:0/2682492375 conn(0x7f28e00fd210 msgr2=0x7f28e00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:46.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.973+0000 7f28e5000700 1 -- 172.21.15.131:0/2682492375 shutdown_connections 2024-05-10T05:31:46.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.974+0000 7f28e5000700 1 -- 172.21.15.131:0/2682492375 wait complete. 2024-05-10T05:31:46.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.975+0000 7f28e5000700 1 Processor -- start 2024-05-10T05:31:46.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.975+0000 7f28e5000700 1 -- start start 2024-05-10T05:31:46.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28e5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e0101be0 0x7f28e019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:46.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28e5000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e0102d00 0x7f28e019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28e5000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e01036a0 0x7f28e01a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28e5000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01a1ce0 con 0x7f28e0102d00 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28e5000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01a1e50 con 0x7f28e0101be0 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28e5000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01a2010 con 0x7f28e01036a0 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28dde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e0101be0 0x7f28e019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28dd400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e0102d00 0x7f28e019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28dde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e0101be0 0x7f28e019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:36372/0 (socket says 172.21.15.131:36372) 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28dde00700 1 -- 172.21.15.131:0/3788081616 learned_addr learned my addr 172.21.15.131:0/3788081616 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28de800700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e01036a0 0x7f28e01a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28dde00700 1 -- 172.21.15.131:0/3788081616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e01036a0 msgr2=0x7f28e01a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28dde00700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e01036a0 0x7f28e01a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28dde00700 1 -- 172.21.15.131:0/3788081616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e0102d00 msgr2=0x7f28e019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:46.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28dde00700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e0102d00 0x7f28e019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:46.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.976+0000 7f28dde00700 1 -- 172.21.15.131:0/3788081616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d8009c40 con 0x7f28e0101be0 2024-05-10T05:31:46.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.977+0000 7f28dde00700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e0101be0 0x7f28e019a250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f28d0000c00 tx=0x7f28d0005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:46.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.977+0000 7f28d6a00700 1 -- 172.21.15.131:0/3788081616 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d001d070 con 0x7f28e0101be0 2024-05-10T05:31:46.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.977+0000 7f28d6a00700 1 -- 172.21.15.131:0/3788081616 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d0021410 con 0x7f28e0101be0 2024-05-10T05:31:46.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.977+0000 7f28e5000700 1 -- 172.21.15.131:0/3788081616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d00095f0 con 0x7f28e0101be0 2024-05-10T05:31:46.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.977+0000 7f28d6a00700 1 -- 172.21.15.131:0/3788081616 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0017650 con 0x7f28e0101be0 2024-05-10T05:31:46.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.977+0000 7f28e5000700 1 -- 172.21.15.131:0/3788081616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e01a2580 con 0x7f28e0101be0 2024-05-10T05:31:46.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.978+0000 7f28e5000700 1 -- 172.21.15.131:0/3788081616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e00694b0 con 0x7f28e0101be0 2024-05-10T05:31:46.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.980+0000 7f28d6a00700 1 -- 172.21.15.131:0/3788081616 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f28d000fb10 con 0x7f28e0101be0 2024-05-10T05:31:46.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.981+0000 7f28d6a00700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f28c40a11a0 0x7f28c40a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:46.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.982+0000 7f28d6a00700 1 -- 172.21.15.131:0/3788081616 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28d00cd6b0 con 0x7f28e0101be0 2024-05-10T05:31:46.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.982+0000 7f28dd400700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f28c40a11a0 0x7f28c40a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:46.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.982+0000 7f28dd400700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f28c40a11a0 0x7f28c40a36b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28e019b2a0 tx=0x7f28cc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:46.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:46.985+0000 7f28d6a00700 1 -- 172.21.15.131:0/3788081616 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f28d00972d0 con 0x7f28e0101be0 2024-05-10T05:31:47.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.216+0000 7f28e5000700 1 -- 172.21.15.131:0/3788081616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28e01a27a0 con 0x7f28e0101be0 2024-05-10T05:31:47.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.217+0000 7f28d6a00700 1 -- 172.21.15.131:0/3788081616 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18494 (secure 0 0 0) 0x7f28d009a8f0 con 0x7f28e0101be0 2024-05-10T05:31:47.217 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:47.217 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":76,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:31:14.901400+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:31:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 -- 172.21.15.131:0/3788081616 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f28c40a11a0 msgr2=0x7f28c40a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f28c40a11a0 0x7f28c40a36b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28e019b2a0 tx=0x7f28cc009260 comp rx=0 tx=0).stop 2024-05-10T05:31:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 -- 172.21.15.131:0/3788081616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e0101be0 msgr2=0x7f28e019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e0101be0 0x7f28e019a250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f28d0000c00 tx=0x7f28d0005b00 comp rx=0 tx=0).stop 2024-05-10T05:31:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 -- 172.21.15.131:0/3788081616 shutdown_connections 2024-05-10T05:31:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f28c40a11a0 0x7f28c40a36b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28e01036a0 0x7f28e01a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e0101be0 0x7f28e019a250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 --2- 172.21.15.131:0/3788081616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e0102d00 0x7f28e019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 -- 172.21.15.131:0/3788081616 >> 172.21.15.131:0/3788081616 conn(0x7f28e00fd210 msgr2=0x7f28e0073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:47.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 -- 172.21.15.131:0/3788081616 shutdown_connections 2024-05-10T05:31:47.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.220+0000 7f28e5000700 1 -- 172.21.15.131:0/3788081616 wait complete. 2024-05-10T05:31:47.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:31:47.247 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:31:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:46 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] up:boot 2024-05-10T05:31:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:46 smithi157 ceph-mon[78427]: fsmap 12 up:standby 2024-05-10T05:31:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T05:31:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:46 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/601149847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:46 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:47.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:46 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] up:boot 2024-05-10T05:31:47.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:46 smithi142 ceph-mon[77667]: fsmap 12 up:standby 2024-05-10T05:31:47.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T05:31:47.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:46 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/601149847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:31:47.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:47.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:47.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:47.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:46 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.536+0000 7f62aaa00700 1 -- 172.21.15.131:0/1494659089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4073b40 msgr2=0x7f62a4074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.536+0000 7f62aaa00700 1 --2- 172.21.15.131:0/1494659089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4073b40 0x7f62a4074010 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6290009990 tx=0x7f6290009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.536+0000 7f62aaa00700 1 -- 172.21.15.131:0/1494659089 shutdown_connections 2024-05-10T05:31:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.536+0000 7f62aaa00700 1 --2- 172.21.15.131:0/1494659089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4075140 0x7f62a4073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.536+0000 7f62aaa00700 1 --2- 172.21.15.131:0/1494659089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a4109ac0 0x7f62a410bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.536+0000 7f62aaa00700 1 --2- 172.21.15.131:0/1494659089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4073b40 0x7f62a4074010 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.536+0000 7f62aaa00700 1 -- 172.21.15.131:0/1494659089 >> 172.21.15.131:0/1494659089 conn(0x7f62a40fd1d0 msgr2=0x7f62a40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:47.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.536+0000 7f62aaa00700 1 -- 172.21.15.131:0/1494659089 shutdown_connections 2024-05-10T05:31:47.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.536+0000 7f62aaa00700 1 -- 172.21.15.131:0/1494659089 wait complete. 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.537+0000 7f62aaa00700 1 Processor -- start 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.537+0000 7f62aaa00700 1 -- start start 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62aaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4073b40 0x7f62a419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62aaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a4075140 0x7f62a419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62aaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4109ac0 0x7f62a41a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62aaa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41a1c30 con 0x7f62a4109ac0 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62aaa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41a1da0 con 0x7f62a4075140 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62aaa00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41a1f60 con 0x7f62a4073b40 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a8a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4109ac0 0x7f62a41a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a8a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4109ac0 0x7f62a41a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40874/0 (socket says 172.21.15.131:40874) 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4073b40 0x7f62a419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:47.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a8a00700 1 -- 172.21.15.131:0/534216360 learned_addr learned my addr 172.21.15.131:0/534216360 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:47.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a3400700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a4075140 0x7f62a419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:47.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a8a00700 1 -- 172.21.15.131:0/534216360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4073b40 msgr2=0x7f62a419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:47.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a8a00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4073b40 0x7f62a419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a8a00700 1 -- 172.21.15.131:0/534216360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a4075140 msgr2=0x7f62a419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:47.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a8a00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a4075140 0x7f62a419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a8a00700 1 -- 172.21.15.131:0/534216360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900095f0 con 0x7f62a4109ac0 2024-05-10T05:31:47.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.538+0000 7f62a3e00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4073b40 0x7f62a419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:47.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.539+0000 7f62a8a00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4109ac0 0x7f62a41a1620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f629c00ef30 tx=0x7f629c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:47.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.542+0000 7f62a0c00700 1 -- 172.21.15.131:0/534216360 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c018070 con 0x7f62a4109ac0 2024-05-10T05:31:47.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.542+0000 7f62aaa00700 1 -- 172.21.15.131:0/534216360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a41a21e0 con 0x7f62a4109ac0 2024-05-10T05:31:47.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.542+0000 7f62a0c00700 1 -- 172.21.15.131:0/534216360 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f629c00f040 con 0x7f62a4109ac0 2024-05-10T05:31:47.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.542+0000 7f62a0c00700 1 -- 172.21.15.131:0/534216360 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00cd00 con 0x7f62a4109ac0 2024-05-10T05:31:47.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.543+0000 7f62aaa00700 1 -- 172.21.15.131:0/534216360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a41a26f0 con 0x7f62a4109ac0 2024-05-10T05:31:47.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.544+0000 7f62aaa00700 1 -- 172.21.15.131:0/534216360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a41088d0 con 0x7f62a4109ac0 2024-05-10T05:31:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.546+0000 7f62a0c00700 1 -- 172.21.15.131:0/534216360 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f629c003a00 con 0x7f62a4109ac0 2024-05-10T05:31:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.547+0000 7f62a0c00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f62880a1240 0x7f62880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.547+0000 7f62a0c00700 1 -- 172.21.15.131:0/534216360 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f629c0cc910 con 0x7f62a4109ac0 2024-05-10T05:31:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.547+0000 7f62a3e00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f62880a1240 0x7f62880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.548+0000 7f62a3e00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f62880a1240 0x7f62880a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f62a4074e30 tx=0x7f6294009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:47.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.550+0000 7f62a0c00700 1 -- 172.21.15.131:0/534216360 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f629c096530 con 0x7f62a4109ac0 2024-05-10T05:31:47.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.777+0000 7f62aaa00700 1 -- 172.21.15.131:0/534216360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62a410f850 con 0x7f62a4109ac0 2024-05-10T05:31:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.778+0000 7f62a0c00700 1 -- 172.21.15.131:0/534216360 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f629c099b50 con 0x7f62a4109ac0 2024-05-10T05:31:47.778 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:31:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 -- 172.21.15.131:0/534216360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f62880a1240 msgr2=0x7f62880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f62880a1240 0x7f62880a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f62a4074e30 tx=0x7f6294009260 comp rx=0 tx=0).stop 2024-05-10T05:31:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 -- 172.21.15.131:0/534216360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4109ac0 msgr2=0x7f62a41a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4109ac0 0x7f62a41a1620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f629c00ef30 tx=0x7f629c00c3a0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 -- 172.21.15.131:0/534216360 shutdown_connections 2024-05-10T05:31:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f62880a1240 0x7f62880a3750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62a4109ac0 0x7f62a41a1620 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62a4073b40 0x7f62a419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 --2- 172.21.15.131:0/534216360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62a4075140 0x7f62a419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:47.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 -- 172.21.15.131:0/534216360 >> 172.21.15.131:0/534216360 conn(0x7f62a40fd1d0 msgr2=0x7f62a4105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:47.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 -- 172.21.15.131:0/534216360 shutdown_connections 2024-05-10T05:31:47.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:47.781+0000 7f62aaa00700 1 -- 172.21.15.131:0/534216360 wait complete. 2024-05-10T05:31:47.809 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T05:31:48.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.090+0000 7fbfda200700 1 -- 172.21.15.131:0/3579437490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd4102d00 msgr2=0x7fbfd4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.090+0000 7fbfda200700 1 --2- 172.21.15.131:0/3579437490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd4102d00 0x7fbfd4103110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00099c0 tx=0x7fbfc0009cd0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.090+0000 7fbfda200700 1 -- 172.21.15.131:0/3579437490 shutdown_connections 2024-05-10T05:31:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.090+0000 7fbfda200700 1 --2- 172.21.15.131:0/3579437490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfd41036a0 0x7fbfd4109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.090+0000 7fbfda200700 1 --2- 172.21.15.131:0/3579437490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd4101be0 0x7fbfd4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.090+0000 7fbfda200700 1 --2- 172.21.15.131:0/3579437490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd4102d00 0x7fbfd4103110 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.090+0000 7fbfda200700 1 -- 172.21.15.131:0/3579437490 >> 172.21.15.131:0/3579437490 conn(0x7fbfd40fd210 msgr2=0x7fbfd40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.091+0000 7fbfda200700 1 -- 172.21.15.131:0/3579437490 shutdown_connections 2024-05-10T05:31:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.091+0000 7fbfda200700 1 -- 172.21.15.131:0/3579437490 wait complete. 2024-05-10T05:31:48.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.092+0000 7fbfda200700 1 Processor -- start 2024-05-10T05:31:48.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.092+0000 7fbfda200700 1 -- start start 2024-05-10T05:31:48.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfda200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd4101be0 0x7fbfd419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:48.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfda200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfd4102d00 0x7fbfd419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:48.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfda200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd41036a0 0x7fbfd41a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:48.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfda200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41a1c30 con 0x7fbfd4102d00 2024-05-10T05:31:48.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfda200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41a1da0 con 0x7fbfd41036a0 2024-05-10T05:31:48.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfda200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41a1f60 con 0x7fbfd4101be0 2024-05-10T05:31:48.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfd3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd4101be0 0x7fbfd419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:48.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfd3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd4101be0 0x7fbfd419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34160/0 (socket says 172.21.15.131:34160) 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfd3400700 1 -- 172.21.15.131:0/3380893443 learned_addr learned my addr 172.21.15.131:0/3380893443 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfd2a00700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfd4102d00 0x7fbfd419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfd3e00700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd41036a0 0x7fbfd41a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfd3400700 1 -- 172.21.15.131:0/3380893443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd41036a0 msgr2=0x7fbfd41a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfd3400700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd41036a0 0x7fbfd41a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfd3400700 1 -- 172.21.15.131:0/3380893443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfd4102d00 msgr2=0x7fbfd419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfd3400700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfd4102d00 0x7fbfd419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.093+0000 7fbfd3400700 1 -- 172.21.15.131:0/3380893443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00095f0 con 0x7fbfd4101be0 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.094+0000 7fbfd3400700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd4101be0 0x7fbfd419a150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400e940 tx=0x7fbfc400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.094+0000 7fbfcbe00700 1 -- 172.21.15.131:0/3380893443 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400cbd0 con 0x7fbfd4101be0 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.094+0000 7fbfcbe00700 1 -- 172.21.15.131:0/3380893443 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfc400ee90 con 0x7fbfd4101be0 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.094+0000 7fbfda200700 1 -- 172.21.15.131:0/3380893443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd41a2240 con 0x7fbfd4101be0 2024-05-10T05:31:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.094+0000 7fbfcbe00700 1 -- 172.21.15.131:0/3380893443 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4003cb0 con 0x7fbfd4101be0 2024-05-10T05:31:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.094+0000 7fbfda200700 1 -- 172.21.15.131:0/3380893443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd41a2750 con 0x7fbfd4101be0 2024-05-10T05:31:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.095+0000 7fbfda200700 1 -- 172.21.15.131:0/3380893443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd40694b0 con 0x7fbfd4101be0 2024-05-10T05:31:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.097+0000 7fbfcbe00700 1 -- 172.21.15.131:0/3380893443 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbfc4004020 con 0x7fbfd4101be0 2024-05-10T05:31:48.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.098+0000 7fbfcbe00700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbfb80a1120 0x7fbfb80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:48.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.098+0000 7fbfcbe00700 1 -- 172.21.15.131:0/3380893443 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbfc4014070 con 0x7fbfd4101be0 2024-05-10T05:31:48.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.099+0000 7fbfd2a00700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbfb80a1120 0x7fbfb80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:48.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.099+0000 7fbfd2a00700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbfb80a1120 0x7fbfb80a3630 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009fd0 tx=0x7fbfc0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:48.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.103+0000 7fbfcbe00700 1 -- 172.21.15.131:0/3380893443 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbfc4096300 con 0x7fbfd4101be0 2024-05-10T05:31:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:47 smithi131 ceph-mon[37264]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:47 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3788081616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:47 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/534216360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:31:48.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:47 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:48.218 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:47 smithi142 ceph-mon[77667]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:48.218 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:47 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3788081616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:48.218 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:47 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:48.218 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:47 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:48.218 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:47 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:48.219 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:47 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:48.219 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:47 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:48.219 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:47 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/534216360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:31:48.219 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:47 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:47 smithi157 ceph-mon[78427]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:47 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3788081616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:47 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:47 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:47 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:47 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:47 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:47 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/534216360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:31:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:47 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:31:48.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.343+0000 7fbfda200700 1 -- 172.21.15.131:0/3380893443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbfd410d3e0 con 0x7fbfd4101be0 2024-05-10T05:31:48.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.343+0000 7fbfcbe00700 1 -- 172.21.15.131:0/3380893443 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbfc4099920 con 0x7fbfd4101be0 2024-05-10T05:31:48.346 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:48.347 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-05-10T05:31:48.347 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:31:48.347 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-05-10T05:31:48.347 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-05-10T05:31:48.347 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:31:48.347 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-05-10T05:31:48.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.347+0000 7fbfda200700 1 -- 172.21.15.131:0/3380893443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbfb80a1120 msgr2=0x7fbfb80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:48.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.347+0000 7fbfda200700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbfb80a1120 0x7fbfb80a3630 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009fd0 tx=0x7fbfc0005fb0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.347+0000 7fbfda200700 1 -- 172.21.15.131:0/3380893443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd4101be0 msgr2=0x7fbfd419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:48.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.347+0000 7fbfda200700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd4101be0 0x7fbfd419a150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400e940 tx=0x7fbfc400ec50 comp rx=0 tx=0).stop 2024-05-10T05:31:48.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.348+0000 7fbfda200700 1 -- 172.21.15.131:0/3380893443 shutdown_connections 2024-05-10T05:31:48.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.348+0000 7fbfda200700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbfb80a1120 0x7fbfb80a3630 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.348+0000 7fbfda200700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd41036a0 0x7fbfd41a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.348+0000 7fbfda200700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfd4101be0 0x7fbfd419a150 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.348+0000 7fbfda200700 1 --2- 172.21.15.131:0/3380893443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfd4102d00 0x7fbfd419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.348+0000 7fbfda200700 1 -- 172.21.15.131:0/3380893443 >> 172.21.15.131:0/3380893443 conn(0x7fbfd40fd210 msgr2=0x7fbfd4073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:48.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.348+0000 7fbfda200700 1 -- 172.21.15.131:0/3380893443 shutdown_connections 2024-05-10T05:31:48.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.348+0000 7fbfda200700 1 -- 172.21.15.131:0/3380893443 wait complete. 2024-05-10T05:31:48.381 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T05:31:48.381 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T05:31:48.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.661+0000 7fe8eec00700 1 -- 172.21.15.131:0/3641761445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8e8073b40 msgr2=0x7fe8e8074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:48.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.661+0000 7fe8eec00700 1 --2- 172.21.15.131:0/3641761445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8e8073b40 0x7fe8e8074010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009990 tx=0x7fe8d4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.662+0000 7fe8eec00700 1 -- 172.21.15.131:0/3641761445 shutdown_connections 2024-05-10T05:31:48.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.662+0000 7fe8eec00700 1 --2- 172.21.15.131:0/3641761445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e8075140 0x7fe8e8073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.662+0000 7fe8eec00700 1 --2- 172.21.15.131:0/3641761445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e810e220 0x7fe8e8110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.662+0000 7fe8eec00700 1 --2- 172.21.15.131:0/3641761445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8e8073b40 0x7fe8e8074010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.662+0000 7fe8eec00700 1 -- 172.21.15.131:0/3641761445 >> 172.21.15.131:0/3641761445 conn(0x7fe8e80fd210 msgr2=0x7fe8e80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:48.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.662+0000 7fe8eec00700 1 -- 172.21.15.131:0/3641761445 shutdown_connections 2024-05-10T05:31:48.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.662+0000 7fe8eec00700 1 -- 172.21.15.131:0/3641761445 wait complete. 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.663+0000 7fe8eec00700 1 Processor -- start 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.663+0000 7fe8eec00700 1 -- start start 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8eec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e8073b40 0x7fe8e819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8eec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e8075140 0x7fe8e819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8eec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8e810e220 0x7fe8e81a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8eec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81a6000 con 0x7fe8e8075140 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8eec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81a6170 con 0x7fe8e810e220 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8eec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81a6330 con 0x7fe8e8073b40 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e8075140 0x7fe8e819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8ecc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8e810e220 0x7fe8e81a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8ecc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8e810e220 0x7fe8e81a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:36452/0 (socket says 172.21.15.131:36452) 2024-05-10T05:31:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8ecc00700 1 -- 172.21.15.131:0/4261151933 learned_addr learned my addr 172.21.15.131:0/4261151933 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:48.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.664+0000 7fe8e7e00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e8073b40 0x7fe8e819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8ecc00700 1 -- 172.21.15.131:0/4261151933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e8073b40 msgr2=0x7fe8e819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8ecc00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e8073b40 0x7fe8e819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8ecc00700 1 -- 172.21.15.131:0/4261151933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e8075140 msgr2=0x7fe8e819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8ecc00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e8075140 0x7fe8e819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8ecc00700 1 -- 172.21.15.131:0/4261151933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40095f0 con 0x7fe8e810e220 2024-05-10T05:31:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8e7e00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e8073b40 0x7fe8e819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8ecc00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8e810e220 0x7fe8e81a59f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000ef30 tx=0x7fe8e000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8e4c00700 1 -- 172.21.15.131:0/4261151933 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0018070 con 0x7fe8e810e220 2024-05-10T05:31:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8eec00700 1 -- 172.21.15.131:0/4261151933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e81a65b0 con 0x7fe8e810e220 2024-05-10T05:31:48.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8e4c00700 1 -- 172.21.15.131:0/4261151933 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8e000f040 con 0x7fe8e810e220 2024-05-10T05:31:48.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8e4c00700 1 -- 172.21.15.131:0/4261151933 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000cd00 con 0x7fe8e810e220 2024-05-10T05:31:48.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.665+0000 7fe8eec00700 1 -- 172.21.15.131:0/4261151933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e81a6a70 con 0x7fe8e810e220 2024-05-10T05:31:48.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.666+0000 7fe8eec00700 1 -- 172.21.15.131:0/4261151933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e8104480 con 0x7fe8e810e220 2024-05-10T05:31:48.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.668+0000 7fe8e4c00700 1 -- 172.21.15.131:0/4261151933 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe8e0003a00 con 0x7fe8e810e220 2024-05-10T05:31:48.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.670+0000 7fe8e4c00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe8cc0a11c0 0x7fe8cc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:48.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.670+0000 7fe8e4c00700 1 -- 172.21.15.131:0/4261151933 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe8e00cbf90 con 0x7fe8e810e220 2024-05-10T05:31:48.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.670+0000 7fe8e7e00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe8cc0a11c0 0x7fe8cc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:48.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.670+0000 7fe8e7e00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe8cc0a11c0 0x7fe8cc0a36d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8074e30 tx=0x7fe8d8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:48.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.676+0000 7fe8e4c00700 1 -- 172.21.15.131:0/4261151933 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe8e0095bb0 con 0x7fe8e810e220 2024-05-10T05:31:48.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:48.910+0000 7fe8eec00700 1 -- 172.21.15.131:0/4261151933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe8e8113c50 con 0x7fe8e810e220 2024-05-10T05:31:49.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:48 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3380893443' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T05:31:49.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:48 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:48 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:48 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:48 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:48 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3380893443' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T05:31:49.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:48 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:48 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:48 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:48 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:48 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3380893443' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T05:31:49.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:48 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:48 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:48 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:48 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:49.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.923+0000 7fe8e4c00700 1 -- 172.21.15.131:0/4261151933 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe8e00991d0 con 0x7fe8e810e220 2024-05-10T05:31:49.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.929+0000 7fe8eec00700 1 -- 172.21.15.131:0/4261151933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe8cc0a11c0 msgr2=0x7fe8cc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.929+0000 7fe8eec00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe8cc0a11c0 0x7fe8cc0a36d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8074e30 tx=0x7fe8d8009260 comp rx=0 tx=0).stop 2024-05-10T05:31:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.930+0000 7fe8eec00700 1 -- 172.21.15.131:0/4261151933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8e810e220 msgr2=0x7fe8e81a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.930+0000 7fe8eec00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8e810e220 0x7fe8e81a59f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000ef30 tx=0x7fe8e000c3a0 comp rx=0 tx=0).stop 2024-05-10T05:31:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.930+0000 7fe8eec00700 1 -- 172.21.15.131:0/4261151933 shutdown_connections 2024-05-10T05:31:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.930+0000 7fe8eec00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe8cc0a11c0 0x7fe8cc0a36d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.930+0000 7fe8eec00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8e810e220 0x7fe8e81a59f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.930+0000 7fe8eec00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e8073b40 0x7fe8e819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.930+0000 7fe8eec00700 1 --2- 172.21.15.131:0/4261151933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e8075140 0x7fe8e819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.930+0000 7fe8eec00700 1 -- 172.21.15.131:0/4261151933 >> 172.21.15.131:0/4261151933 conn(0x7fe8e80fd210 msgr2=0x7fe8e81015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.930+0000 7fe8eec00700 1 -- 172.21.15.131:0/4261151933 shutdown_connections 2024-05-10T05:31:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:49.930+0000 7fe8eec00700 1 -- 172.21.15.131:0/4261151933 wait complete. 2024-05-10T05:31:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-05-10T05:31:49.956 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T05:31:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:49 smithi131 ceph-mon[37264]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:49 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4261151933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T05:31:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:49 smithi131 ceph-mon[37264]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T05:31:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:49 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:49 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:49 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:49 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:49 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:49 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.227+0000 7f918ae00700 1 -- 172.21.15.131:0/3877209090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184103620 msgr2=0x7f9184109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.227+0000 7f918ae00700 1 --2- 172.21.15.131:0/3877209090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184103620 0x7f9184109ed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f917c00b0a0 tx=0x7f917c00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:31:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.227+0000 7f918ae00700 1 -- 172.21.15.131:0/3877209090 shutdown_connections 2024-05-10T05:31:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.227+0000 7f918ae00700 1 --2- 172.21.15.131:0/3877209090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184103620 0x7f9184109ed0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.227+0000 7f918ae00700 1 --2- 172.21.15.131:0/3877209090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184101b60 0x7f9184101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.227+0000 7f918ae00700 1 --2- 172.21.15.131:0/3877209090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184102c80 0x7f9184103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.227+0000 7f918ae00700 1 -- 172.21.15.131:0/3877209090 >> 172.21.15.131:0/3877209090 conn(0x7f91840fd150 msgr2=0x7f91840ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.228+0000 7f918ae00700 1 -- 172.21.15.131:0/3877209090 shutdown_connections 2024-05-10T05:31:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.228+0000 7f918ae00700 1 -- 172.21.15.131:0/3877209090 wait complete. 2024-05-10T05:31:50.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f918ae00700 1 Processor -- start 2024-05-10T05:31:50.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f918ae00700 1 -- start start 2024-05-10T05:31:50.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f918ae00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184101b60 0x7f9184197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:50.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f918ae00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102c80 0x7f9184198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f918ae00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184103620 0x7f918419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f918ae00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918419fa80 con 0x7f9184101b60 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f918ae00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918419fbc0 con 0x7f9184103620 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f918ae00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918419fd50 con 0x7f9184102c80 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f9183e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184101b60 0x7f9184197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f9183400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102c80 0x7f9184198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f9183e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184101b60 0x7f9184197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40940/0 (socket says 172.21.15.131:40940) 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.229+0000 7f9183e00700 1 -- 172.21.15.131:0/1335990398 learned_addr learned my addr 172.21.15.131:0/1335990398 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9188e00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184103620 0x7f918419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9183e00700 1 -- 172.21.15.131:0/1335990398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102c80 msgr2=0x7f9184198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9183e00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102c80 0x7f9184198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9183e00700 1 -- 172.21.15.131:0/1335990398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184103620 msgr2=0x7f918419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:50.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9183e00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184103620 0x7f918419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:50.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9183e00700 1 -- 172.21.15.131:0/1335990398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009d00 con 0x7f9184101b60 2024-05-10T05:31:50.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9183400700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102c80 0x7f9184198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:50.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9183e00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184101b60 0x7f9184197fe0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f917400e970 tx=0x7f917400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:50.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9180c00700 1 -- 172.21.15.131:0/1335990398 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174004220 con 0x7f9184101b60 2024-05-10T05:31:50.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9180c00700 1 -- 172.21.15.131:0/1335990398 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9174004380 con 0x7f9184101b60 2024-05-10T05:31:50.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f918ae00700 1 -- 172.21.15.131:0/1335990398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918419ffd0 con 0x7f9184101b60 2024-05-10T05:31:50.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f9180c00700 1 -- 172.21.15.131:0/1335990398 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174003c40 con 0x7f9184101b60 2024-05-10T05:31:50.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.230+0000 7f918ae00700 1 -- 172.21.15.131:0/1335990398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91841a0480 con 0x7f9184101b60 2024-05-10T05:31:50.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.231+0000 7f918ae00700 1 -- 172.21.15.131:0/1335990398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9184108d70 con 0x7f9184101b60 2024-05-10T05:31:50.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.234+0000 7f9180c00700 1 -- 172.21.15.131:0/1335990398 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f917400c7e0 con 0x7f9184101b60 2024-05-10T05:31:50.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.236+0000 7f9180c00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a11c0 0x7f91680a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:50.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.236+0000 7f9180c00700 1 -- 172.21.15.131:0/1335990398 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f9174014070 con 0x7f9184101b60 2024-05-10T05:31:50.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.236+0000 7f9183400700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a11c0 0x7f91680a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:50.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.237+0000 7f9183400700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a11c0 0x7f91680a36d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9184199080 tx=0x7f9170009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:50.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.239+0000 7f9180c00700 1 -- 172.21.15.131:0/1335990398 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9174096370 con 0x7f9184101b60 2024-05-10T05:31:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:49 smithi157 ceph-mon[78427]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:49 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4261151933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T05:31:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:49 smithi157 ceph-mon[78427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T05:31:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:50.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:49 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:49 smithi142 ceph-mon[77667]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:50.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:49 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4261151933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T05:31:50.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:49 smithi142 ceph-mon[77667]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T05:31:50.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:49 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:49 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:49 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:31:50.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:49 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:31:50.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:49 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:49 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:31:50.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.448+0000 7f918ae00700 1 -- 172.21.15.131:0/1335990398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f91840694d0 con 0x7f9184101b60 2024-05-10T05:31:50.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.968+0000 7f9180c00700 1 -- 172.21.15.131:0/1335990398 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9174099990 con 0x7f9184101b60 2024-05-10T05:31:50.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.974+0000 7f918ae00700 1 -- 172.21.15.131:0/1335990398 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a11c0 msgr2=0x7f91680a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:50.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.974+0000 7f918ae00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a11c0 0x7f91680a36d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9184199080 tx=0x7f9170009260 comp rx=0 tx=0).stop 2024-05-10T05:31:50.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.974+0000 7f918ae00700 1 -- 172.21.15.131:0/1335990398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184101b60 msgr2=0x7f9184197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:50.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.974+0000 7f918ae00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184101b60 0x7f9184197fe0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f917400e970 tx=0x7f917400ec80 comp rx=0 tx=0).stop 2024-05-10T05:31:50.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.974+0000 7f918ae00700 1 -- 172.21.15.131:0/1335990398 shutdown_connections 2024-05-10T05:31:50.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.974+0000 7f918ae00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a11c0 0x7f91680a36d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:50.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.974+0000 7f918ae00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184101b60 0x7f9184197fe0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:50.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.974+0000 7f918ae00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184103620 0x7f918419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:50.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.974+0000 7f918ae00700 1 --2- 172.21.15.131:0/1335990398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102c80 0x7f9184198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:50.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.974+0000 7f918ae00700 1 -- 172.21.15.131:0/1335990398 >> 172.21.15.131:0/1335990398 conn(0x7f91840fd150 msgr2=0x7f9184105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:50.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.975+0000 7f918ae00700 1 -- 172.21.15.131:0/1335990398 shutdown_connections 2024-05-10T05:31:50.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:50.975+0000 7f918ae00700 1 -- 172.21.15.131:0/1335990398 wait complete. 2024-05-10T05:31:50.982 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-05-10T05:31:51.015 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T05:31:51.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.280+0000 7f93d5600700 1 -- 172.21.15.131:0/3255294667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93d0102d00 msgr2=0x7f93d0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:51.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.280+0000 7f93d5600700 1 --2- 172.21.15.131:0/3255294667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93d0102d00 0x7f93d0103110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009990 tx=0x7f93bc009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.280+0000 7f93d5600700 1 -- 172.21.15.131:0/3255294667 shutdown_connections 2024-05-10T05:31:51.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.280+0000 7f93d5600700 1 --2- 172.21.15.131:0/3255294667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d01036a0 0x7f93d0109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.280+0000 7f93d5600700 1 --2- 172.21.15.131:0/3255294667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d0101be0 0x7f93d0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.280+0000 7f93d5600700 1 --2- 172.21.15.131:0/3255294667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93d0102d00 0x7f93d0103110 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.280+0000 7f93d5600700 1 -- 172.21.15.131:0/3255294667 >> 172.21.15.131:0/3255294667 conn(0x7f93d00fd1d0 msgr2=0x7f93d00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:51.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.281+0000 7f93d5600700 1 -- 172.21.15.131:0/3255294667 shutdown_connections 2024-05-10T05:31:51.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.281+0000 7f93d5600700 1 -- 172.21.15.131:0/3255294667 wait complete. 2024-05-10T05:31:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93d5600700 1 Processor -- start 2024-05-10T05:31:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93d5600700 1 -- start start 2024-05-10T05:31:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93d5600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d0101be0 0x7f93d0197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93d5600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93d0102d00 0x7f93d0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93d5600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d01036a0 0x7f93d019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93d5600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d019fa00 con 0x7f93d0102d00 2024-05-10T05:31:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93d5600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d019fb70 con 0x7f93d0101be0 2024-05-10T05:31:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93d5600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d019fd30 con 0x7f93d01036a0 2024-05-10T05:31:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93ce000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93d0102d00 0x7f93d0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93ce000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93d0102d00 0x7f93d0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40958/0 (socket says 172.21.15.131:40958) 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93ce000700 1 -- 172.21.15.131:0/305490674 learned_addr learned my addr 172.21.15.131:0/305490674 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93cf400700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d01036a0 0x7f93d019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.282+0000 7f93cea00700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d0101be0 0x7f93d0197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93ce000700 1 -- 172.21.15.131:0/305490674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d01036a0 msgr2=0x7f93d019f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93ce000700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d01036a0 0x7f93d019f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93ce000700 1 -- 172.21.15.131:0/305490674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d0101be0 msgr2=0x7f93d0197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93ce000700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d0101be0 0x7f93d0197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93ce000700 1 -- 172.21.15.131:0/305490674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc0095f0 con 0x7f93d0102d00 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93cea00700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d0101be0 0x7f93d0197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:31:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93ce000700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93d0102d00 0x7f93d0198460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93bc000c00 tx=0x7f93bc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93c7400700 1 -- 172.21.15.131:0/305490674 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc01d070 con 0x7f93d0102d00 2024-05-10T05:31:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93c7400700 1 -- 172.21.15.131:0/305490674 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93bc021410 con 0x7f93d0102d00 2024-05-10T05:31:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93d5600700 1 -- 172.21.15.131:0/305490674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d019ffb0 con 0x7f93d0102d00 2024-05-10T05:31:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93c7400700 1 -- 172.21.15.131:0/305490674 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc017650 con 0x7f93d0102d00 2024-05-10T05:31:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.283+0000 7f93d5600700 1 -- 172.21.15.131:0/305490674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d01a0320 con 0x7f93d0102d00 2024-05-10T05:31:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.284+0000 7f93d5600700 1 -- 172.21.15.131:0/305490674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d0108df0 con 0x7f93d0102d00 2024-05-10T05:31:51.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.287+0000 7f93c7400700 1 -- 172.21.15.131:0/305490674 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f93bc021580 con 0x7f93d0102d00 2024-05-10T05:31:51.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.289+0000 7f93c7400700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f93b40a11c0 0x7f93b40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:51.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.289+0000 7f93c7400700 1 -- 172.21.15.131:0/305490674 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f93bc0ce220 con 0x7f93d0102d00 2024-05-10T05:31:51.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.289+0000 7f93cea00700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f93b40a11c0 0x7f93b40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.290+0000 7f93cea00700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f93b40a11c0 0x7f93b40a36d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f93d0102c00 tx=0x7f93c000b670 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:51.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.292+0000 7f93c7400700 1 -- 172.21.15.131:0/305490674 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f93bc097bf0 con 0x7f93d0102d00 2024-05-10T05:31:51.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:50 smithi157 ceph-mon[78427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T05:31:51.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:50 smithi157 ceph-mon[78427]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T05:31:51.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:50 smithi142 ceph-mon[77667]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T05:31:51.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:50 smithi142 ceph-mon[77667]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T05:31:51.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:50 smithi131 ceph-mon[37264]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T05:31:51.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:50 smithi131 ceph-mon[37264]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T05:31:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.545+0000 7f93d5600700 1 -- 172.21.15.131:0/305490674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f93d010d430 con 0x7f93d0102d00 2024-05-10T05:31:51.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.553+0000 7f93c7400700 1 -- 172.21.15.131:0/305490674 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f93bc09b210 con 0x7f93d0102d00 2024-05-10T05:31:51.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.557+0000 7f93d5600700 1 -- 172.21.15.131:0/305490674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f93b40a11c0 msgr2=0x7f93b40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:51.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.557+0000 7f93d5600700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f93b40a11c0 0x7f93b40a36d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f93d0102c00 tx=0x7f93c000b670 comp rx=0 tx=0).stop 2024-05-10T05:31:51.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.557+0000 7f93d5600700 1 -- 172.21.15.131:0/305490674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93d0102d00 msgr2=0x7f93d0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:51.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.557+0000 7f93d5600700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93d0102d00 0x7f93d0198460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93bc000c00 tx=0x7f93bc009580 comp rx=0 tx=0).stop 2024-05-10T05:31:51.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.558+0000 7f93d5600700 1 -- 172.21.15.131:0/305490674 shutdown_connections 2024-05-10T05:31:51.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.558+0000 7f93d5600700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f93b40a11c0 0x7f93b40a36d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.558+0000 7f93d5600700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d01036a0 0x7f93d019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.558+0000 7f93d5600700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d0101be0 0x7f93d0197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.558+0000 7f93d5600700 1 --2- 172.21.15.131:0/305490674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93d0102d00 0x7f93d0198460 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.558+0000 7f93d5600700 1 -- 172.21.15.131:0/305490674 >> 172.21.15.131:0/305490674 conn(0x7f93d00fd1d0 msgr2=0x7f93d0105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:51.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.560+0000 7f93d5600700 1 -- 172.21.15.131:0/305490674 shutdown_connections 2024-05-10T05:31:51.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.560+0000 7f93d5600700 1 -- 172.21.15.131:0/305490674 wait complete. 2024-05-10T05:31:51.564 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 2 and data pool 3 2024-05-10T05:31:51.602 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:31:51.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.865+0000 7f0acf800700 1 -- 172.21.15.131:0/2329791160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac8101c00 msgr2=0x7f0ac8102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:51.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.865+0000 7f0acf800700 1 --2- 172.21.15.131:0/2329791160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac8101c00 0x7f0ac8102010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009990 tx=0x7f0abc009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.865+0000 7f0acf800700 1 -- 172.21.15.131:0/2329791160 shutdown_connections 2024-05-10T05:31:51.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.865+0000 7f0acf800700 1 --2- 172.21.15.131:0/2329791160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac8101c00 0x7f0ac8102010 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.865+0000 7f0acf800700 1 --2- 172.21.15.131:0/2329791160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac81036c0 0x7f0ac8109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.865+0000 7f0acf800700 1 --2- 172.21.15.131:0/2329791160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac8102d20 0x7f0ac8103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.865+0000 7f0acf800700 1 -- 172.21.15.131:0/2329791160 >> 172.21.15.131:0/2329791160 conn(0x7f0ac80fd210 msgr2=0x7f0ac80ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:51.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.865+0000 7f0acf800700 1 -- 172.21.15.131:0/2329791160 shutdown_connections 2024-05-10T05:31:51.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.865+0000 7f0acf800700 1 -- 172.21.15.131:0/2329791160 wait complete. 2024-05-10T05:31:51.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.866+0000 7f0acf800700 1 Processor -- start 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.866+0000 7f0acf800700 1 -- start start 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acf800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac8101c00 0x7f0ac8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acf800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac8102d20 0x7f0ac8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acf800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac81036c0 0x7f0ac819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acf800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac819fac0 con 0x7f0ac81036c0 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acf800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac819fc30 con 0x7f0ac8101c00 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acf800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac819fdf0 con 0x7f0ac8102d20 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acd800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac81036c0 0x7f0ac819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acd800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac81036c0 0x7f0ac819f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40980/0 (socket says 172.21.15.131:40980) 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acd800700 1 -- 172.21.15.131:0/3934456856 learned_addr learned my addr 172.21.15.131:0/3934456856 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acce00700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac8101c00 0x7f0ac8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:51.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0ac7e00700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac8102d20 0x7f0ac8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acd800700 1 -- 172.21.15.131:0/3934456856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac8102d20 msgr2=0x7f0ac8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acd800700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac8102d20 0x7f0ac8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acd800700 1 -- 172.21.15.131:0/3934456856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac8101c00 msgr2=0x7f0ac8197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acd800700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac8101c00 0x7f0ac8197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acd800700 1 -- 172.21.15.131:0/3934456856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc0095f0 con 0x7f0ac81036c0 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0ac7e00700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac8102d20 0x7f0ac8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.867+0000 7f0acd800700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac81036c0 0x7f0ac819f4b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000ef30 tx=0x7f0ac000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.868+0000 7f0ac5600700 1 -- 172.21.15.131:0/3934456856 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0018070 con 0x7f0ac81036c0 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.868+0000 7f0ac5600700 1 -- 172.21.15.131:0/3934456856 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ac000f040 con 0x7f0ac81036c0 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.868+0000 7f0ac5600700 1 -- 172.21.15.131:0/3934456856 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac000cdb0 con 0x7f0ac81036c0 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.868+0000 7f0acf800700 1 -- 172.21.15.131:0/3934456856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac81a0070 con 0x7f0ac81036c0 2024-05-10T05:31:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.868+0000 7f0acf800700 1 -- 172.21.15.131:0/3934456856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac81a0530 con 0x7f0ac81036c0 2024-05-10T05:31:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.869+0000 7f0acf800700 1 -- 172.21.15.131:0/3934456856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac8108e10 con 0x7f0ac81036c0 2024-05-10T05:31:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.871+0000 7f0ac5600700 1 -- 172.21.15.131:0/3934456856 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0ac0003a00 con 0x7f0ac81036c0 2024-05-10T05:31:51.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.872+0000 7f0ac5600700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ab00a11a0 0x7f0ab00a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:51.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.872+0000 7f0ac5600700 1 -- 172.21.15.131:0/3934456856 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ac00cccf0 con 0x7f0ac81036c0 2024-05-10T05:31:51.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.873+0000 7f0acce00700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ab00a11a0 0x7f0ab00a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:51.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.873+0000 7f0acce00700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ab00a11a0 0x7f0ab00a36b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0abc006010 tx=0x7f0abc01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:51.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:51.875+0000 7f0ac5600700 1 -- 172.21.15.131:0/3934456856 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0ac00965c0 con 0x7f0ac81036c0 2024-05-10T05:31:52.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.085+0000 7f0acf800700 1 -- 172.21.15.131:0/3934456856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ac8068730 con 0x7f0ac81036c0 2024-05-10T05:31:52.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.087+0000 7f0ac5600700 1 -- 172.21.15.131:0/3934456856 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22204 (secure 0 0 0) 0x7f0ac0099be0 con 0x7f0ac81036c0 2024-05-10T05:31:52.088 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:52.088 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":79,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:31:51.546945+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:31:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 -- 172.21.15.131:0/3934456856 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ab00a11a0 msgr2=0x7f0ab00a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ab00a11a0 0x7f0ab00a36b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0abc006010 tx=0x7f0abc01a040 comp rx=0 tx=0).stop 2024-05-10T05:31:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 -- 172.21.15.131:0/3934456856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac81036c0 msgr2=0x7f0ac819f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac81036c0 0x7f0ac819f4b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000ef30 tx=0x7f0ac000c3a0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 -- 172.21.15.131:0/3934456856 shutdown_connections 2024-05-10T05:31:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ab00a11a0 0x7f0ab00a36b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac81036c0 0x7f0ac819f4b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac8101c00 0x7f0ac8197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 --2- 172.21.15.131:0/3934456856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac8102d20 0x7f0ac8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 -- 172.21.15.131:0/3934456856 >> 172.21.15.131:0/3934456856 conn(0x7f0ac80fd210 msgr2=0x7f0ac8105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:52.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 -- 172.21.15.131:0/3934456856 shutdown_connections 2024-05-10T05:31:52.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.090+0000 7f0acf800700 1 -- 172.21.15.131:0/3934456856 wait complete. 2024-05-10T05:31:52.116 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:31:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1335990398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T05:31:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: pgmap v235: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1335990398' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T05:31:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T05:31:52.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T05:31:52.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/305490674' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T05:31:52.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T05:31:52.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T05:31:52.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/305490674' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T05:31:52.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T05:31:52.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:51 smithi157 ceph-mon[78427]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T05:31:52.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.376+0000 7fd809200700 1 -- 172.21.15.131:0/927484642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd804101be0 msgr2=0x7fd804101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:52.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.376+0000 7fd809200700 1 --2- 172.21.15.131:0/927484642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd804101be0 0x7fd804101ff0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009990 tx=0x7fd7f4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.376+0000 7fd809200700 1 -- 172.21.15.131:0/927484642 shutdown_connections 2024-05-10T05:31:52.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.376+0000 7fd809200700 1 --2- 172.21.15.131:0/927484642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd804101be0 0x7fd804101ff0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.376+0000 7fd809200700 1 --2- 172.21.15.131:0/927484642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8041036a0 0x7fd804109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.376+0000 7fd809200700 1 --2- 172.21.15.131:0/927484642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd804102d00 0x7fd804103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.376+0000 7fd809200700 1 -- 172.21.15.131:0/927484642 >> 172.21.15.131:0/927484642 conn(0x7fd8040fd210 msgr2=0x7fd8040ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:52.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.376+0000 7fd809200700 1 -- 172.21.15.131:0/927484642 shutdown_connections 2024-05-10T05:31:52.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.376+0000 7fd809200700 1 -- 172.21.15.131:0/927484642 wait complete. 2024-05-10T05:31:52.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.378+0000 7fd809200700 1 Processor -- start 2024-05-10T05:31:52.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.378+0000 7fd809200700 1 -- start start 2024-05-10T05:31:52.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.378+0000 7fd809200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd804101be0 0x7fd80419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:52.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.378+0000 7fd809200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd804102d00 0x7fd80419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:52.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.378+0000 7fd809200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8041036a0 0x7fd8041a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:52.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.378+0000 7fd809200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8041a1da0 con 0x7fd804101be0 2024-05-10T05:31:52.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.378+0000 7fd809200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8041a1f10 con 0x7fd804102d00 2024-05-10T05:31:52.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.378+0000 7fd809200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8041a20d0 con 0x7fd8041036a0 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd801e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd804101be0 0x7fd80419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd801e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd804101be0 0x7fd80419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41006/0 (socket says 172.21.15.131:41006) 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd801e00700 1 -- 172.21.15.131:0/2658894872 learned_addr learned my addr 172.21.15.131:0/2658894872 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd801400700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd804102d00 0x7fd80419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd802800700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8041036a0 0x7fd8041a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd801e00700 1 -- 172.21.15.131:0/2658894872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8041036a0 msgr2=0x7fd8041a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd801e00700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8041036a0 0x7fd8041a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd801e00700 1 -- 172.21.15.131:0/2658894872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd804102d00 msgr2=0x7fd80419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd801e00700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd804102d00 0x7fd80419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd801e00700 1 -- 172.21.15.131:0/2658894872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f40095f0 con 0x7fd804101be0 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.379+0000 7fd801e00700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd804101be0 0x7fd80419a2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006010 tx=0x7fd7f4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.380+0000 7fd7faa00700 1 -- 172.21.15.131:0/2658894872 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f401d070 con 0x7fd804101be0 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.380+0000 7fd7faa00700 1 -- 172.21.15.131:0/2658894872 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7f4021410 con 0x7fd804101be0 2024-05-10T05:31:52.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.380+0000 7fd809200700 1 -- 172.21.15.131:0/2658894872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8041a2350 con 0x7fd804101be0 2024-05-10T05:31:52.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.380+0000 7fd7faa00700 1 -- 172.21.15.131:0/2658894872 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4017650 con 0x7fd804101be0 2024-05-10T05:31:52.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.380+0000 7fd809200700 1 -- 172.21.15.131:0/2658894872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8041a27d0 con 0x7fd804101be0 2024-05-10T05:31:52.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.381+0000 7fd809200700 1 -- 172.21.15.131:0/2658894872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8040694b0 con 0x7fd804101be0 2024-05-10T05:31:52.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.384+0000 7fd7faa00700 1 -- 172.21.15.131:0/2658894872 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd7f400fad0 con 0x7fd804101be0 2024-05-10T05:31:52.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.385+0000 7fd7faa00700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7e80a11a0 0x7fd7e80a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:52.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.386+0000 7fd7faa00700 1 -- 172.21.15.131:0/2658894872 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd7f40ce180 con 0x7fd804101be0 2024-05-10T05:31:52.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.386+0000 7fd801400700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7e80a11a0 0x7fd7e80a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:52.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.386+0000 7fd801400700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7e80a11a0 0x7fd7e80a36b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd80419b360 tx=0x7fd7f0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:52.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.388+0000 7fd7faa00700 1 -- 172.21.15.131:0/2658894872 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd7f4097a50 con 0x7fd804101be0 2024-05-10T05:31:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1335990398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T05:31:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: pgmap v235: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1335990398' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T05:31:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T05:31:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T05:31:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/305490674' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T05:31:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T05:31:52.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T05:31:52.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/305490674' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T05:31:52.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T05:31:52.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:51 smithi142 ceph-mon[77667]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T05:31:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1335990398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T05:31:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: pgmap v235: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:31:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1335990398' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T05:31:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T05:31:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T05:31:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/305490674' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T05:31:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T05:31:52.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T05:31:52.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/305490674' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T05:31:52.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T05:31:52.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:51 smithi131 ceph-mon[37264]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T05:31:52.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.599+0000 7fd809200700 1 -- 172.21.15.131:0/2658894872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd80410d410 con 0x7fd804101be0 2024-05-10T05:31:52.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.600+0000 7fd7faa00700 1 -- 172.21.15.131:0/2658894872 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22191 (secure 0 0 0) 0x7fd7f409b070 con 0x7fd804101be0 2024-05-10T05:31:52.601 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:31:52.601 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":80,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:31:52.550067+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:31:52.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 -- 172.21.15.131:0/2658894872 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7e80a11a0 msgr2=0x7fd7e80a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:52.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7e80a11a0 0x7fd7e80a36b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd80419b360 tx=0x7fd7f0009260 comp rx=0 tx=0).stop 2024-05-10T05:31:52.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 -- 172.21.15.131:0/2658894872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd804101be0 msgr2=0x7fd80419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd804101be0 0x7fd80419a2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006010 tx=0x7fd7f4005740 comp rx=0 tx=0).stop 2024-05-10T05:31:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 -- 172.21.15.131:0/2658894872 shutdown_connections 2024-05-10T05:31:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7e80a11a0 0x7fd7e80a36b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd804101be0 0x7fd80419a2c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8041036a0 0x7fd8041a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 --2- 172.21.15.131:0/2658894872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd804102d00 0x7fd80419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 -- 172.21.15.131:0/2658894872 >> 172.21.15.131:0/2658894872 conn(0x7fd8040fd210 msgr2=0x7fd804073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 -- 172.21.15.131:0/2658894872 shutdown_connections 2024-05-10T05:31:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.603+0000 7fd809200700 1 -- 172.21.15.131:0/2658894872 wait complete. 2024-05-10T05:31:52.635 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T05:31:52.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.901+0000 7f50b2a00700 1 -- 172.21.15.131:0/2363095903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac101be0 msgr2=0x7f50ac101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:52.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.901+0000 7f50b2a00700 1 --2- 172.21.15.131:0/2363095903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac101be0 0x7f50ac101ff0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f509c009990 tx=0x7f509c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.901+0000 7f50b2a00700 1 -- 172.21.15.131:0/2363095903 shutdown_connections 2024-05-10T05:31:52.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.901+0000 7f50b2a00700 1 --2- 172.21.15.131:0/2363095903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac101be0 0x7f50ac101ff0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.901+0000 7f50b2a00700 1 --2- 172.21.15.131:0/2363095903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1036a0 0x7f50ac109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.901+0000 7f50b2a00700 1 --2- 172.21.15.131:0/2363095903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac102d00 0x7f50ac103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.901+0000 7f50b2a00700 1 -- 172.21.15.131:0/2363095903 >> 172.21.15.131:0/2363095903 conn(0x7f50ac0fd210 msgr2=0x7f50ac0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:52.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.902+0000 7f50b2a00700 1 -- 172.21.15.131:0/2363095903 shutdown_connections 2024-05-10T05:31:52.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.902+0000 7f50b2a00700 1 -- 172.21.15.131:0/2363095903 wait complete. 2024-05-10T05:31:52.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.903+0000 7f50b2a00700 1 Processor -- start 2024-05-10T05:31:52.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.903+0000 7f50b2a00700 1 -- start start 2024-05-10T05:31:52.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.903+0000 7f50b2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac101be0 0x7f50ac19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:52.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.903+0000 7f50b2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac102d00 0x7f50ac19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:52.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.903+0000 7f50b2a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1036a0 0x7f50ac1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:52.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.903+0000 7f50b2a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1a1c20 con 0x7f50ac102d00 2024-05-10T05:31:52.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.903+0000 7f50b2a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1a1d90 con 0x7f50ac101be0 2024-05-10T05:31:52.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.903+0000 7f50b2a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1a1f50 con 0x7f50ac1036a0 2024-05-10T05:31:52.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.903+0000 7f50ab400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac102d00 0x7f50ac19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:52.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50ab400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac102d00 0x7f50ac19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49458/0 (socket says 172.21.15.131:49458) 2024-05-10T05:31:52.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50ab400700 1 -- 172.21.15.131:0/1926855520 learned_addr learned my addr 172.21.15.131:0/1926855520 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50b0a00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1036a0 0x7f50ac1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50abe00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac101be0 0x7f50ac19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50ab400700 1 -- 172.21.15.131:0/1926855520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1036a0 msgr2=0x7f50ac1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50ab400700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1036a0 0x7f50ac1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50ab400700 1 -- 172.21.15.131:0/1926855520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac101be0 msgr2=0x7f50ac19a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50ab400700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac101be0 0x7f50ac19a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50ab400700 1 -- 172.21.15.131:0/1926855520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098009520 con 0x7f50ac102d00 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50ab400700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac102d00 0x7f50ac19a680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f509800eb30 tx=0x7f509800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50a8c00700 1 -- 172.21.15.131:0/1926855520 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098009880 con 0x7f50ac102d00 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50a8c00700 1 -- 172.21.15.131:0/1926855520 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50980044e0 con 0x7f50ac102d00 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50b2a00700 1 -- 172.21.15.131:0/1926855520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c0095f0 con 0x7f50ac102d00 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.904+0000 7f50a8c00700 1 -- 172.21.15.131:0/1926855520 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098003d80 con 0x7f50ac102d00 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.905+0000 7f50b2a00700 1 -- 172.21.15.131:0/1926855520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac1a25b0 con 0x7f50ac102d00 2024-05-10T05:31:52.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.906+0000 7f50b2a00700 1 -- 172.21.15.131:0/1926855520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac0694b0 con 0x7f50ac102d00 2024-05-10T05:31:52.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.908+0000 7f50a8c00700 1 -- 172.21.15.131:0/1926855520 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5098003f20 con 0x7f50ac102d00 2024-05-10T05:31:52.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.910+0000 7f50a8c00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f50900a11c0 0x7f50900a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:52.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.910+0000 7f50a8c00700 1 -- 172.21.15.131:0/1926855520 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5098014070 con 0x7f50ac102d00 2024-05-10T05:31:52.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.910+0000 7f50abe00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f50900a11c0 0x7f50900a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:52.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.910+0000 7f50abe00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f50900a11c0 0x7f50900a36d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f509c00b7b0 tx=0x7f509c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:52.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:52.914+0000 7f50a8c00700 1 -- 172.21.15.131:0/1926855520 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f50980965f0 con 0x7f50ac102d00 2024-05-10T05:31:53.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.166+0000 7f50b2a00700 1 -- 172.21.15.131:0/1926855520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f50ac10d410 con 0x7f50ac102d00 2024-05-10T05:31:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:52 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3934456856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:52 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] up:active 2024-05-10T05:31:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:52 smithi157 ceph-mon[78427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:52 smithi157 ceph-mon[78427]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T05:31:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:52 smithi157 ceph-mon[78427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:52 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2658894872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:52 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3934456856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:52 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] up:active 2024-05-10T05:31:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:52 smithi142 ceph-mon[77667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:52 smithi142 ceph-mon[77667]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T05:31:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:52 smithi142 ceph-mon[77667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:52 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2658894872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:52 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3934456856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:52 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] up:active 2024-05-10T05:31:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:52 smithi131 ceph-mon[37264]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:52 smithi131 ceph-mon[37264]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T05:31:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:52 smithi131 ceph-mon[37264]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:52 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2658894872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:31:53.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.996+0000 7f50a8c00700 1 -- 172.21.15.131:0/1926855520 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5098099c10 con 0x7f50ac102d00 2024-05-10T05:31:54.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 -- 172.21.15.131:0/1926855520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f50900a11c0 msgr2=0x7f50900a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:54.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f50900a11c0 0x7f50900a36d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f509c00b7b0 tx=0x7f509c01a040 comp rx=0 tx=0).stop 2024-05-10T05:31:54.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 -- 172.21.15.131:0/1926855520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac102d00 msgr2=0x7f50ac19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:54.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac102d00 0x7f50ac19a680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f509800eb30 tx=0x7f509800ee40 comp rx=0 tx=0).stop 2024-05-10T05:31:54.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 -- 172.21.15.131:0/1926855520 shutdown_connections 2024-05-10T05:31:54.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f50900a11c0 0x7f50900a36d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:54.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50ac1036a0 0x7f50ac1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:54.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac101be0 0x7f50ac19a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:54.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 --2- 172.21.15.131:0/1926855520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ac102d00 0x7f50ac19a680 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:54.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 -- 172.21.15.131:0/1926855520 >> 172.21.15.131:0/1926855520 conn(0x7f50ac0fd210 msgr2=0x7f50ac073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:54.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 -- 172.21.15.131:0/1926855520 shutdown_connections 2024-05-10T05:31:54.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:53.999+0000 7f50b2a00700 1 -- 172.21.15.131:0/1926855520 wait complete. 2024-05-10T05:31:54.027 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-10T05:31:54.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-10T05:31:54.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.292+0000 7fe309a00700 1 -- 172.21.15.131:0/1914238447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe304102d00 msgr2=0x7fe304103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:54.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.292+0000 7fe309a00700 1 --2- 172.21.15.131:0/1914238447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe304102d00 0x7fe304103110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00098f0 tx=0x7fe2f0009c00 comp rx=0 tx=0).stop 2024-05-10T05:31:54.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.292+0000 7fe309a00700 1 -- 172.21.15.131:0/1914238447 shutdown_connections 2024-05-10T05:31:54.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.292+0000 7fe309a00700 1 --2- 172.21.15.131:0/1914238447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3041036a0 0x7fe304109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:54.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.292+0000 7fe309a00700 1 --2- 172.21.15.131:0/1914238447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304101be0 0x7fe304101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:54.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.292+0000 7fe309a00700 1 --2- 172.21.15.131:0/1914238447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe304102d00 0x7fe304103110 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:54.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.292+0000 7fe309a00700 1 -- 172.21.15.131:0/1914238447 >> 172.21.15.131:0/1914238447 conn(0x7fe3040fd210 msgr2=0x7fe3040ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:54.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.292+0000 7fe309a00700 1 -- 172.21.15.131:0/1914238447 shutdown_connections 2024-05-10T05:31:54.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.292+0000 7fe309a00700 1 -- 172.21.15.131:0/1914238447 wait complete. 2024-05-10T05:31:54.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.293+0000 7fe309a00700 1 Processor -- start 2024-05-10T05:31:54.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.293+0000 7fe309a00700 1 -- start start 2024-05-10T05:31:54.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe309a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304101be0 0x7fe30419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:54.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe309a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe304102d00 0x7fe30419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe309a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3041036a0 0x7fe3041a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe309a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3041a1ca0 con 0x7fe3041036a0 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe309a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3041a1e10 con 0x7fe304102d00 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe309a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3041a1fd0 con 0x7fe304101be0 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe303400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3041036a0 0x7fe3041a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe302a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304101be0 0x7fe30419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe302000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe304102d00 0x7fe30419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe302a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304101be0 0x7fe30419a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32794/0 (socket says 172.21.15.131:32794) 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe302a00700 1 -- 172.21.15.131:0/2034744662 learned_addr learned my addr 172.21.15.131:0/2034744662 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe302000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe304102d00 0x7fe30419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46712/0 (socket says 172.21.15.131:46712) 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe303400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3041036a0 0x7fe3041a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49468/0 (socket says 172.21.15.131:49468) 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe302a00700 1 -- 172.21.15.131:0/2034744662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe304102d00 msgr2=0x7fe30419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe302a00700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe304102d00 0x7fe30419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:54.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe302a00700 1 -- 172.21.15.131:0/2034744662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3041036a0 msgr2=0x7fe3041a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:54.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe302a00700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3041036a0 0x7fe3041a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:54.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe302a00700 1 -- 172.21.15.131:0/2034744662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc009c40 con 0x7fe304101be0 2024-05-10T05:31:54.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.294+0000 7fe303400700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3041036a0 0x7fe3041a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:54.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.295+0000 7fe302a00700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304101be0 0x7fe30419a1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400e9c0 tx=0x7fe2f400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:54.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.295+0000 7fe2fb400700 1 -- 172.21.15.131:0/2034744662 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4004220 con 0x7fe304101be0 2024-05-10T05:31:54.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.295+0000 7fe309a00700 1 -- 172.21.15.131:0/2034744662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f0009520 con 0x7fe304101be0 2024-05-10T05:31:54.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.295+0000 7fe2fb400700 1 -- 172.21.15.131:0/2034744662 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2f4004380 con 0x7fe304101be0 2024-05-10T05:31:54.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.295+0000 7fe2fb400700 1 -- 172.21.15.131:0/2034744662 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4003b40 con 0x7fe304101be0 2024-05-10T05:31:54.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.295+0000 7fe309a00700 1 -- 172.21.15.131:0/2034744662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3041a2680 con 0x7fe304101be0 2024-05-10T05:31:54.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.296+0000 7fe309a00700 1 -- 172.21.15.131:0/2034744662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3040694b0 con 0x7fe304101be0 2024-05-10T05:31:54.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.298+0000 7fe2fb400700 1 -- 172.21.15.131:0/2034744662 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe2f4003ca0 con 0x7fe304101be0 2024-05-10T05:31:54.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.299+0000 7fe2fb400700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2e80a11c0 0x7fe2e80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:54.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.300+0000 7fe2fb400700 1 -- 172.21.15.131:0/2034744662 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe2f4014070 con 0x7fe304101be0 2024-05-10T05:31:54.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.300+0000 7fe302000700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2e80a11c0 0x7fe2e80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:54.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.300+0000 7fe302000700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2e80a11c0 0x7fe2e80a36d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe30419b260 tx=0x7fe2f0019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:54.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:53 smithi157 ceph-mon[78427]: pgmap v239: 97 pgs: 19 active+clean, 15 creating+peering, 63 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-10T05:31:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:53 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1926855520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T05:31:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:53 smithi157 ceph-mon[78427]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T05:31:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:53 smithi157 ceph-mon[78427]: Cluster is now healthy 2024-05-10T05:31:54.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.302+0000 7fe2fb400700 1 -- 172.21.15.131:0/2034744662 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe2f4096830 con 0x7fe304101be0 2024-05-10T05:31:54.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:53 smithi142 ceph-mon[77667]: pgmap v239: 97 pgs: 19 active+clean, 15 creating+peering, 63 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-10T05:31:54.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:53 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1926855520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T05:31:54.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:53 smithi142 ceph-mon[77667]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T05:31:54.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:53 smithi142 ceph-mon[77667]: Cluster is now healthy 2024-05-10T05:31:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:53 smithi131 ceph-mon[37264]: pgmap v239: 97 pgs: 19 active+clean, 15 creating+peering, 63 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-10T05:31:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:53 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1926855520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T05:31:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:53 smithi131 ceph-mon[37264]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T05:31:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:53 smithi131 ceph-mon[37264]: Cluster is now healthy 2024-05-10T05:31:54.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:54.556+0000 7fe309a00700 1 -- 172.21.15.131:0/2034744662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fe3041a2a00 con 0x7fe304101be0 2024-05-10T05:31:55.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.003+0000 7fe2fb400700 1 -- 172.21.15.131:0/2034744662 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe2f4099e50 con 0x7fe304101be0 2024-05-10T05:31:55.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 -- 172.21.15.131:0/2034744662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2e80a11c0 msgr2=0x7fe2e80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:55.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2e80a11c0 0x7fe2e80a36d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe30419b260 tx=0x7fe2f0019040 comp rx=0 tx=0).stop 2024-05-10T05:31:55.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 -- 172.21.15.131:0/2034744662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304101be0 msgr2=0x7fe30419a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:55.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304101be0 0x7fe30419a1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400e9c0 tx=0x7fe2f400ecd0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 -- 172.21.15.131:0/2034744662 shutdown_connections 2024-05-10T05:31:55.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2e80a11c0 0x7fe2e80a36d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3041036a0 0x7fe3041a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304101be0 0x7fe30419a1c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 --2- 172.21.15.131:0/2034744662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe304102d00 0x7fe30419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 -- 172.21.15.131:0/2034744662 >> 172.21.15.131:0/2034744662 conn(0x7fe3040fd210 msgr2=0x7fe304073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:55.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 -- 172.21.15.131:0/2034744662 shutdown_connections 2024-05-10T05:31:55.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.006+0000 7fe309a00700 1 -- 172.21.15.131:0/2034744662 wait complete. 2024-05-10T05:31:55.040 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-10T05:31:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:55 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1926855520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T05:31:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:55 smithi157 ceph-mon[78427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:55 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2034744662' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T05:31:55.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:55 smithi157 ceph-mon[78427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T05:31:55.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.308+0000 7f3c32000700 1 -- 172.21.15.131:0/1210120299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c2c1019f0 msgr2=0x7f3c2c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:55.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.308+0000 7f3c32000700 1 --2- 172.21.15.131:0/1210120299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c2c1019f0 0x7f3c2c101e00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009990 tx=0x7f3c20009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.308+0000 7f3c32000700 1 -- 172.21.15.131:0/1210120299 shutdown_connections 2024-05-10T05:31:55.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.308+0000 7f3c32000700 1 --2- 172.21.15.131:0/1210120299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c2c1019f0 0x7f3c2c101e00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.308+0000 7f3c32000700 1 --2- 172.21.15.131:0/1210120299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c1034b0 0x7f3c2c109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.308+0000 7f3c32000700 1 --2- 172.21.15.131:0/1210120299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c2c102b10 0x7f3c2c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.308+0000 7f3c32000700 1 -- 172.21.15.131:0/1210120299 >> 172.21.15.131:0/1210120299 conn(0x7f3c2c0fd000 msgr2=0x7f3c2c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:55.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.308+0000 7f3c32000700 1 -- 172.21.15.131:0/1210120299 shutdown_connections 2024-05-10T05:31:55.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.308+0000 7f3c32000700 1 -- 172.21.15.131:0/1210120299 wait complete. 2024-05-10T05:31:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.309+0000 7f3c32000700 1 Processor -- start 2024-05-10T05:31:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.309+0000 7f3c32000700 1 -- start start 2024-05-10T05:31:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c32000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c1019f0 0x7f3c2c074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c32000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c2c102b10 0x7f3c2c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c32000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c2c1034b0 0x7f3c2c073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c32000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c074100 con 0x7f3c2c102b10 2024-05-10T05:31:55.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c32000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c074270 con 0x7f3c2c1034b0 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c32000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c074430 con 0x7f3c2c1019f0 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c2be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c2c102b10 0x7f3c2c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c2be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c2c102b10 0x7f3c2c073560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49486/0 (socket says 172.21.15.131:49486) 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c2be00700 1 -- 172.21.15.131:0/1904845944 learned_addr learned my addr 172.21.15.131:0/1904845944 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c30c00700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c1019f0 0x7f3c2c074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c31600700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c2c1034b0 0x7f3c2c073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c2be00700 1 -- 172.21.15.131:0/1904845944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c1019f0 msgr2=0x7f3c2c074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c2be00700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c1019f0 0x7f3c2c074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c2be00700 1 -- 172.21.15.131:0/1904845944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c2c1034b0 msgr2=0x7f3c2c073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c2be00700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c2c1034b0 0x7f3c2c073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.310+0000 7f3c2be00700 1 -- 172.21.15.131:0/1904845944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200095f0 con 0x7f3c2c102b10 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.311+0000 7f3c2be00700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c2c102b10 0x7f3c2c073560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800e9a0 tx=0x7f3c1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.311+0000 7f3c29600700 1 -- 172.21.15.131:0/1904845944 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18009790 con 0x7f3c2c102b10 2024-05-10T05:31:55.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.311+0000 7f3c29600700 1 -- 172.21.15.131:0/1904845944 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c1800cbd0 con 0x7f3c2c102b10 2024-05-10T05:31:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.311+0000 7f3c32000700 1 -- 172.21.15.131:0/1904845944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c1a2130 con 0x7f3c2c102b10 2024-05-10T05:31:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.311+0000 7f3c29600700 1 -- 172.21.15.131:0/1904845944 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18018680 con 0x7f3c2c102b10 2024-05-10T05:31:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.311+0000 7f3c32000700 1 -- 172.21.15.131:0/1904845944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2c1a2690 con 0x7f3c2c102b10 2024-05-10T05:31:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.312+0000 7f3c32000700 1 -- 172.21.15.131:0/1904845944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2c108c00 con 0x7f3c2c102b10 2024-05-10T05:31:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.314+0000 7f3c29600700 1 -- 172.21.15.131:0/1904845944 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3c1800cd40 con 0x7f3c2c102b10 2024-05-10T05:31:55.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.315+0000 7f3c29600700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c140a11a0 0x7f3c140a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:55.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.316+0000 7f3c29600700 1 -- 172.21.15.131:0/1904845944 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3c18014070 con 0x7f3c2c102b10 2024-05-10T05:31:55.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.316+0000 7f3c30c00700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c140a11a0 0x7f3c140a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:55.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.316+0000 7f3c30c00700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c140a11a0 0x7f3c140a36b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000b7b0 tx=0x7f3c2001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:55.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.318+0000 7f3c29600700 1 -- 172.21.15.131:0/1904845944 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3c18096580 con 0x7f3c2c102b10 2024-05-10T05:31:55.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:55 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1926855520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T05:31:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:55 smithi142 ceph-mon[77667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:55 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2034744662' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T05:31:55.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:55 smithi142 ceph-mon[77667]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T05:31:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:55 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1926855520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T05:31:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:55 smithi131 ceph-mon[37264]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:55 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2034744662' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T05:31:55.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:55 smithi131 ceph-mon[37264]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T05:31:55.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:55.573+0000 7f3c32000700 1 -- 172.21.15.131:0/1904845944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f3c2c068110 con 0x7f3c2c102b10 2024-05-10T05:31:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.009+0000 7f3c29600700 1 -- 172.21.15.131:0/1904845944 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3c180260c0 con 0x7f3c2c102b10 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 -- 172.21.15.131:0/1904845944 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c140a11a0 msgr2=0x7f3c140a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c140a11a0 0x7f3c140a36b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000b7b0 tx=0x7f3c2001a040 comp rx=0 tx=0).stop 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 -- 172.21.15.131:0/1904845944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c2c102b10 msgr2=0x7f3c2c073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c2c102b10 0x7f3c2c073560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800e9a0 tx=0x7f3c1800ecb0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 -- 172.21.15.131:0/1904845944 shutdown_connections 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c140a11a0 0x7f3c140a36b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c2c1034b0 0x7f3c2c073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c2c1019f0 0x7f3c2c074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 --2- 172.21.15.131:0/1904845944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c2c102b10 0x7f3c2c073560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 -- 172.21.15.131:0/1904845944 >> 172.21.15.131:0/1904845944 conn(0x7f3c2c0fd000 msgr2=0x7f3c2c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 -- 172.21.15.131:0/1904845944 shutdown_connections 2024-05-10T05:31:56.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.012+0000 7f3c32000700 1 -- 172.21.15.131:0/1904845944 wait complete. 2024-05-10T05:31:56.044 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-10T05:31:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:56 smithi157 ceph-mon[78427]: pgmap v240: 97 pgs: 71 active+clean, 15 creating+peering, 11 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 834 B/s wr, 7 op/s 2024-05-10T05:31:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:56 smithi157 ceph-mon[78427]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T05:31:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:56 smithi157 ceph-mon[78427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-10T05:31:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:56 smithi157 ceph-mon[78427]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T05:31:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:56 smithi157 ceph-mon[78427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:56 smithi157 ceph-mon[78427]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-05-10T05:31:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:56 smithi157 ceph-mon[78427]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-05-10T05:31:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:56 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1904845944' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T05:31:56.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.311+0000 7fe397000700 1 -- 172.21.15.131:0/1433303053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3901019e0 msgr2=0x7fe390101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:56.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.311+0000 7fe397000700 1 --2- 172.21.15.131:0/1433303053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3901019e0 0x7fe390101df0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe384009990 tx=0x7fe384009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.311+0000 7fe397000700 1 -- 172.21.15.131:0/1433303053 shutdown_connections 2024-05-10T05:31:56.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.311+0000 7fe397000700 1 --2- 172.21.15.131:0/1433303053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3901019e0 0x7fe390101df0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.311+0000 7fe397000700 1 --2- 172.21.15.131:0/1433303053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3901034a0 0x7fe390109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.311+0000 7fe397000700 1 --2- 172.21.15.131:0/1433303053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe390102b00 0x7fe390102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.311+0000 7fe397000700 1 -- 172.21.15.131:0/1433303053 >> 172.21.15.131:0/1433303053 conn(0x7fe3900fcfd0 msgr2=0x7fe3900ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:56.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.311+0000 7fe397000700 1 -- 172.21.15.131:0/1433303053 shutdown_connections 2024-05-10T05:31:56.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.311+0000 7fe397000700 1 -- 172.21.15.131:0/1433303053 wait complete. 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.312+0000 7fe397000700 1 Processor -- start 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.312+0000 7fe397000700 1 -- start start 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe397000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3901019e0 0x7fe390074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe397000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe390102b00 0x7fe390073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe397000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3901034a0 0x7fe390073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe397000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe390074100 con 0x7fe3901034a0 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe397000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe390074270 con 0x7fe3901019e0 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe397000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe390074430 con 0x7fe390102b00 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe396600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3901034a0 0x7fe390073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe395c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3901019e0 0x7fe390074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe396600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3901034a0 0x7fe390073aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49512/0 (socket says 172.21.15.131:49512) 2024-05-10T05:31:56.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe396600700 1 -- 172.21.15.131:0/3932278696 learned_addr learned my addr 172.21.15.131:0/3932278696 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe395200700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe390102b00 0x7fe390073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe396600700 1 -- 172.21.15.131:0/3932278696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe390102b00 msgr2=0x7fe390073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe396600700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe390102b00 0x7fe390073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe396600700 1 -- 172.21.15.131:0/3932278696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3901019e0 msgr2=0x7fe390074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe396600700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3901019e0 0x7fe390074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe396600700 1 -- 172.21.15.131:0/3932278696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3840095f0 con 0x7fe3901034a0 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.313+0000 7fe395c00700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3901019e0 0x7fe390074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.314+0000 7fe396600700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3901034a0 0x7fe390073aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe38800ef30 tx=0x7fe38800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.314+0000 7fe38e000700 1 -- 172.21.15.131:0/3932278696 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388018070 con 0x7fe3901034a0 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.314+0000 7fe38e000700 1 -- 172.21.15.131:0/3932278696 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe38800f040 con 0x7fe3901034a0 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.314+0000 7fe38e000700 1 -- 172.21.15.131:0/3932278696 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38800cd00 con 0x7fe3901034a0 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.314+0000 7fe397000700 1 -- 172.21.15.131:0/3932278696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3901a20b0 con 0x7fe3901034a0 2024-05-10T05:31:56.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.314+0000 7fe397000700 1 -- 172.21.15.131:0/3932278696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3901a2570 con 0x7fe3901034a0 2024-05-10T05:31:56.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.315+0000 7fe397000700 1 -- 172.21.15.131:0/3932278696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe390108c60 con 0x7fe3901034a0 2024-05-10T05:31:56.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.317+0000 7fe38e000700 1 -- 172.21.15.131:0/3932278696 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe388003a00 con 0x7fe3901034a0 2024-05-10T05:31:56.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.318+0000 7fe38e000700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3780a11a0 0x7fe3780a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:56.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.318+0000 7fe38e000700 1 -- 172.21.15.131:0/3932278696 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe3880ccc80 con 0x7fe3901034a0 2024-05-10T05:31:56.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.318+0000 7fe395c00700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3780a11a0 0x7fe3780a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:56.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.319+0000 7fe395c00700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3780a11a0 0x7fe3780a36b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe38400b7b0 tx=0x7fe38401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:56.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.321+0000 7fe38e000700 1 -- 172.21.15.131:0/3932278696 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe388096550 con 0x7fe3901034a0 2024-05-10T05:31:56.394 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:31:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-b[115203]: 2024-05-10T05:31:55.905+0000 7f5a1e8cf700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T05:31:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:56 smithi142 ceph-mon[77667]: pgmap v240: 97 pgs: 71 active+clean, 15 creating+peering, 11 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 834 B/s wr, 7 op/s 2024-05-10T05:31:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:56 smithi142 ceph-mon[77667]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T05:31:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:56 smithi142 ceph-mon[77667]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-10T05:31:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:56 smithi142 ceph-mon[77667]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T05:31:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:56 smithi142 ceph-mon[77667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:56 smithi142 ceph-mon[77667]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-05-10T05:31:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:56 smithi142 ceph-mon[77667]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-05-10T05:31:56.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:56 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1904845944' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T05:31:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:56 smithi131 ceph-mon[37264]: pgmap v240: 97 pgs: 71 active+clean, 15 creating+peering, 11 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 834 B/s wr, 7 op/s 2024-05-10T05:31:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:56 smithi131 ceph-mon[37264]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T05:31:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:56 smithi131 ceph-mon[37264]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-10T05:31:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:56 smithi131 ceph-mon[37264]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T05:31:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:56 smithi131 ceph-mon[37264]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T05:31:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:56 smithi131 ceph-mon[37264]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-05-10T05:31:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:56 smithi131 ceph-mon[37264]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-05-10T05:31:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:56 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1904845944' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T05:31:56.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:56.576+0000 7fe397000700 1 -- 172.21.15.131:0/3932278696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fe390068110 con 0x7fe3901034a0 2024-05-10T05:31:57.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.025+0000 7fe38e000700 1 -- 172.21.15.131:0/3932278696 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe388099b70 con 0x7fe3901034a0 2024-05-10T05:31:57.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 -- 172.21.15.131:0/3932278696 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3780a11a0 msgr2=0x7fe3780a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:57.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3780a11a0 0x7fe3780a36b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe38400b7b0 tx=0x7fe38401a040 comp rx=0 tx=0).stop 2024-05-10T05:31:57.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 -- 172.21.15.131:0/3932278696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3901034a0 msgr2=0x7fe390073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:57.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3901034a0 0x7fe390073aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe38800ef30 tx=0x7fe38800c3a0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 -- 172.21.15.131:0/3932278696 shutdown_connections 2024-05-10T05:31:57.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3780a11a0 0x7fe3780a36b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3901034a0 0x7fe390073aa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3901019e0 0x7fe390074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 --2- 172.21.15.131:0/3932278696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe390102b00 0x7fe390073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 -- 172.21.15.131:0/3932278696 >> 172.21.15.131:0/3932278696 conn(0x7fe3900fcfd0 msgr2=0x7fe390105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:57.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 -- 172.21.15.131:0/3932278696 shutdown_connections 2024-05-10T05:31:57.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.028+0000 7fe397000700 1 -- 172.21.15.131:0/3932278696 wait complete. 2024-05-10T05:31:57.060 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-10T05:31:57.060 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-10T05:31:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:57 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] up:active 2024-05-10T05:31:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:57 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1904845944' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T05:31:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:57 smithi157 ceph-mon[78427]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T05:31:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:57 smithi157 ceph-mon[78427]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-05-10T05:31:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:57 smithi157 ceph-mon[78427]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:creating} 9 up:standby 2024-05-10T05:31:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:57 smithi157 ceph-mon[78427]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-05-10T05:31:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:57 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3932278696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T05:31:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.325+0000 7fbba7200700 1 -- 172.21.15.131:0/362916701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba81019f0 msgr2=0x7fbba8101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.325+0000 7fbba7200700 1 --2- 172.21.15.131:0/362916701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba81019f0 0x7fbba8101e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb980099c0 tx=0x7fbb98009cd0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.326+0000 7fbba7200700 1 -- 172.21.15.131:0/362916701 shutdown_connections 2024-05-10T05:31:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.326+0000 7fbba7200700 1 --2- 172.21.15.131:0/362916701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbba81034b0 0x7fbba8109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.326+0000 7fbba7200700 1 --2- 172.21.15.131:0/362916701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba81019f0 0x7fbba8101e00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.326+0000 7fbba7200700 1 --2- 172.21.15.131:0/362916701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba8102b10 0x7fbba8102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.326+0000 7fbba7200700 1 -- 172.21.15.131:0/362916701 >> 172.21.15.131:0/362916701 conn(0x7fbba80fd000 msgr2=0x7fbba80ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.326+0000 7fbba7200700 1 -- 172.21.15.131:0/362916701 shutdown_connections 2024-05-10T05:31:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.326+0000 7fbba7200700 1 -- 172.21.15.131:0/362916701 wait complete. 2024-05-10T05:31:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba7200700 1 Processor -- start 2024-05-10T05:31:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba7200700 1 -- start start 2024-05-10T05:31:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba7200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba81019f0 0x7fbba8074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba7200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbba8102b10 0x7fbba8073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba7200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba81034b0 0x7fbba8073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba7200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8074100 con 0x7fbba8102b10 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba7200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8074270 con 0x7fbba81034b0 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba7200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8074430 con 0x7fbba81019f0 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba81019f0 0x7fbba8074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbba8102b10 0x7fbba8073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba81019f0 0x7fbba8074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32860/0 (socket says 172.21.15.131:32860) 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba5e00700 1 -- 172.21.15.131:0/2744168202 learned_addr learned my addr 172.21.15.131:0/2744168202 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba5400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbba8102b10 0x7fbba8073560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49530/0 (socket says 172.21.15.131:49530) 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.327+0000 7fbba6800700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba81034b0 0x7fbba8073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbba5e00700 1 -- 172.21.15.131:0/2744168202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba81034b0 msgr2=0x7fbba8073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbba5e00700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba81034b0 0x7fbba8073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbba5e00700 1 -- 172.21.15.131:0/2744168202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbba8102b10 msgr2=0x7fbba8073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbba5e00700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbba8102b10 0x7fbba8073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbba5e00700 1 -- 172.21.15.131:0/2744168202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba0009c40 con 0x7fbba81019f0 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbba5400700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbba8102b10 0x7fbba8073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbba5e00700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba81019f0 0x7fbba8074ec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb980098e0 tx=0x7fbb9800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbb9ea00700 1 -- 172.21.15.131:0/2744168202 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9801d070 con 0x7fbba81019f0 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbb9ea00700 1 -- 172.21.15.131:0/2744168202 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb98022470 con 0x7fbba81019f0 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbba7200700 1 -- 172.21.15.131:0/2744168202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb980095f0 con 0x7fbba81019f0 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.328+0000 7fbb9ea00700 1 -- 172.21.15.131:0/2744168202 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98017650 con 0x7fbba81019f0 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.329+0000 7fbba7200700 1 -- 172.21.15.131:0/2744168202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba81a21d0 con 0x7fbba81019f0 2024-05-10T05:31:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.329+0000 7fbba7200700 1 -- 172.21.15.131:0/2744168202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba8108c00 con 0x7fbba81019f0 2024-05-10T05:31:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.332+0000 7fbb9ea00700 1 -- 172.21.15.131:0/2744168202 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbb980225e0 con 0x7fbba81019f0 2024-05-10T05:31:57.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.333+0000 7fbb9ea00700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb8c0a1120 0x7fbb8c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:57.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.333+0000 7fbb9ea00700 1 -- 172.21.15.131:0/2744168202 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbb980cdab0 con 0x7fbba81019f0 2024-05-10T05:31:57.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.334+0000 7fbba5400700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb8c0a1120 0x7fbb8c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:57.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.334+0000 7fbba5400700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb8c0a1120 0x7fbb8c0a3630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbba8074860 tx=0x7fbb94009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:57.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.335+0000 7fbb9ea00700 1 -- 172.21.15.131:0/2744168202 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbb98097380 con 0x7fbba81019f0 2024-05-10T05:31:57.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:57 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] up:active 2024-05-10T05:31:57.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:57 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1904845944' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T05:31:57.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:57 smithi142 ceph-mon[77667]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T05:31:57.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:57 smithi142 ceph-mon[77667]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-05-10T05:31:57.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:57 smithi142 ceph-mon[77667]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:creating} 9 up:standby 2024-05-10T05:31:57.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:57 smithi142 ceph-mon[77667]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-05-10T05:31:57.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:57 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3932278696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T05:31:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:57 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] up:active 2024-05-10T05:31:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:57 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1904845944' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T05:31:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:57 smithi131 ceph-mon[37264]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T05:31:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:57 smithi131 ceph-mon[37264]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-05-10T05:31:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:57 smithi131 ceph-mon[37264]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:creating} 9 up:standby 2024-05-10T05:31:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:57 smithi131 ceph-mon[37264]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-05-10T05:31:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:57 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3932278696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T05:31:57.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.576+0000 7fbba7200700 1 -- 172.21.15.131:0/2744168202 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fbba802cf30 con 0x7fbb8c0a1120 2024-05-10T05:31:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.605+0000 7fbb9ea00700 1 -- 172.21.15.131:0/2744168202 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbba802cf30 con 0x7fbb8c0a1120 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 -- 172.21.15.131:0/2744168202 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb8c0a1120 msgr2=0x7fbb8c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb8c0a1120 0x7fbb8c0a3630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbba8074860 tx=0x7fbb94009260 comp rx=0 tx=0).stop 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 -- 172.21.15.131:0/2744168202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba81019f0 msgr2=0x7fbba8074ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba81019f0 0x7fbba8074ec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb980098e0 tx=0x7fbb9800f690 comp rx=0 tx=0).stop 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 -- 172.21.15.131:0/2744168202 shutdown_connections 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb8c0a1120 0x7fbb8c0a3630 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbba81034b0 0x7fbba8073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba81019f0 0x7fbba8074ec0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 --2- 172.21.15.131:0/2744168202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbba8102b10 0x7fbba8073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 -- 172.21.15.131:0/2744168202 >> 172.21.15.131:0/2744168202 conn(0x7fbba80fd000 msgr2=0x7fbba8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 -- 172.21.15.131:0/2744168202 shutdown_connections 2024-05-10T05:31:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.608+0000 7fbba7200700 1 -- 172.21.15.131:0/2744168202 wait complete. 2024-05-10T05:31:57.638 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-10T05:31:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.906+0000 7ff4dec00700 1 -- 172.21.15.131:0/1568460657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d8069090 msgr2=0x7ff4d8104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.906+0000 7ff4dec00700 1 --2- 172.21.15.131:0/1568460657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d8069090 0x7ff4d8104720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80098f0 tx=0x7ff4c8009c00 comp rx=0 tx=0).stop 2024-05-10T05:31:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.907+0000 7ff4dec00700 1 -- 172.21.15.131:0/1568460657 shutdown_connections 2024-05-10T05:31:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.907+0000 7ff4dec00700 1 --2- 172.21.15.131:0/1568460657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8107880 0x7ff4d8109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.907+0000 7ff4dec00700 1 --2- 172.21.15.131:0/1568460657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d8069090 0x7ff4d8104720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.907+0000 7ff4dec00700 1 --2- 172.21.15.131:0/1568460657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8104d10 0x7ff4d81071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.907+0000 7ff4dec00700 1 -- 172.21.15.131:0/1568460657 >> 172.21.15.131:0/1568460657 conn(0x7ff4d80fc0d0 msgr2=0x7ff4d80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.907+0000 7ff4dec00700 1 -- 172.21.15.131:0/1568460657 shutdown_connections 2024-05-10T05:31:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.907+0000 7ff4dec00700 1 -- 172.21.15.131:0/1568460657 wait complete. 2024-05-10T05:31:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4dec00700 1 Processor -- start 2024-05-10T05:31:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4dec00700 1 -- start start 2024-05-10T05:31:57.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4dec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069090 0x7ff4d8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:57.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4dec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d8104d10 0x7ff4d8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:57.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4dec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8107880 0x7ff4d819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:57.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4dec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d819fa30 con 0x7ff4d8107880 2024-05-10T05:31:57.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4dec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d819fba0 con 0x7ff4d8069090 2024-05-10T05:31:57.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4dec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d819fd60 con 0x7ff4d8104d10 2024-05-10T05:31:57.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4d7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d8104d10 0x7ff4d8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:57.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4dcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8107880 0x7ff4d819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:57.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4d7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d8104d10 0x7ff4d8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32888/0 (socket says 172.21.15.131:32888) 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.908+0000 7ff4d7400700 1 -- 172.21.15.131:0/1633858383 learned_addr learned my addr 172.21.15.131:0/1633858383 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.909+0000 7ff4d7e00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069090 0x7ff4d8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.909+0000 7ff4d7400700 1 -- 172.21.15.131:0/1633858383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069090 msgr2=0x7ff4d8197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.909+0000 7ff4d7400700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069090 0x7ff4d8197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.909+0000 7ff4d7400700 1 -- 172.21.15.131:0/1633858383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8107880 msgr2=0x7ff4d819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.909+0000 7ff4d7400700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8107880 0x7ff4d819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.909+0000 7ff4d7400700 1 -- 172.21.15.131:0/1633858383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c40095f0 con 0x7ff4d8104d10 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.909+0000 7ff4dcc00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8107880 0x7ff4d819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.909+0000 7ff4d7400700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d8104d10 0x7ff4d8198490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400ec10 tx=0x7ff4c400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.909+0000 7ff4d4c00700 1 -- 172.21.15.131:0/1633858383 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400cc90 con 0x7ff4d8104d10 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.910+0000 7ff4dec00700 1 -- 172.21.15.131:0/1633858383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c8009520 con 0x7ff4d8104d10 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.910+0000 7ff4d4c00700 1 -- 172.21.15.131:0/1633858383 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c40044e0 con 0x7ff4d8104d10 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.910+0000 7ff4d4c00700 1 -- 172.21.15.131:0/1633858383 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4010430 con 0x7ff4d8104d10 2024-05-10T05:31:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.910+0000 7ff4dec00700 1 -- 172.21.15.131:0/1633858383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d81a02d0 con 0x7ff4d8104d10 2024-05-10T05:31:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.911+0000 7ff4dec00700 1 -- 172.21.15.131:0/1633858383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d80fdc80 con 0x7ff4d8104d10 2024-05-10T05:31:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.913+0000 7ff4d4c00700 1 -- 172.21.15.131:0/1633858383 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff4c4003f90 con 0x7ff4d8104d10 2024-05-10T05:31:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.914+0000 7ff4d4c00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff4bc0a11c0 0x7ff4bc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.914+0000 7ff4d4c00700 1 -- 172.21.15.131:0/1633858383 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff4c4014070 con 0x7ff4d8104d10 2024-05-10T05:31:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.914+0000 7ff4d7e00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff4bc0a11c0 0x7ff4bc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.915+0000 7ff4d7e00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff4bc0a11c0 0x7ff4bc0a36d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8000c00 tx=0x7ff4c8019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:57.916+0000 7ff4d4c00700 1 -- 172.21.15.131:0/1633858383 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff4c4096710 con 0x7ff4d8104d10 2024-05-10T05:31:57.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:31:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:31:57.583+0000 7fd059800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T05:31:57.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:31:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:31:57.583+0000 7fd059800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T05:31:57.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:31:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:31:57.583+0000 7fd059800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T05:31:57.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:31:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:31:57.583+0000 7fd059800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T05:31:57.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:31:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:31:57.583+0000 7fd059800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T05:31:58.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.162+0000 7ff4dec00700 1 -- 172.21.15.131:0/1633858383 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4d802ce90 con 0x7ff4bc0a11c0 2024-05-10T05:31:58.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.187+0000 7ff4d4c00700 1 -- 172.21.15.131:0/1633858383 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff4d802ce90 con 0x7ff4bc0a11c0 2024-05-10T05:31:58.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.189+0000 7ff4dec00700 1 -- 172.21.15.131:0/1633858383 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff4bc0a11c0 msgr2=0x7ff4bc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.189+0000 7ff4dec00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff4bc0a11c0 0x7ff4bc0a36d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8000c00 tx=0x7ff4c8019040 comp rx=0 tx=0).stop 2024-05-10T05:31:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.189+0000 7ff4dec00700 1 -- 172.21.15.131:0/1633858383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d8104d10 msgr2=0x7ff4d8198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.189+0000 7ff4dec00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d8104d10 0x7ff4d8198490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400ec10 tx=0x7ff4c400ef20 comp rx=0 tx=0).stop 2024-05-10T05:31:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.189+0000 7ff4dec00700 1 -- 172.21.15.131:0/1633858383 shutdown_connections 2024-05-10T05:31:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.189+0000 7ff4dec00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff4bc0a11c0 0x7ff4bc0a36d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.189+0000 7ff4dec00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d8107880 0x7ff4d819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.189+0000 7ff4dec00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069090 0x7ff4d8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.190+0000 7ff4dec00700 1 --2- 172.21.15.131:0/1633858383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d8104d10 0x7ff4d8198490 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.190+0000 7ff4dec00700 1 -- 172.21.15.131:0/1633858383 >> 172.21.15.131:0/1633858383 conn(0x7ff4d80fc0d0 msgr2=0x7ff4d8102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.190+0000 7ff4dec00700 1 -- 172.21.15.131:0/1633858383 shutdown_connections 2024-05-10T05:31:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.190+0000 7ff4dec00700 1 -- 172.21.15.131:0/1633858383 wait complete. 2024-05-10T05:31:58.218 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-10T05:31:58.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:58 smithi131 ceph-mon[37264]: pgmap v241: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 9 op/s 2024-05-10T05:31:58.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] up:active 2024-05-10T05:31:58.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3932278696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T05:31:58.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:58 smithi131 ceph-mon[37264]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-10T05:31:58.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:58 smithi131 ceph-mon[37264]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 9 up:standby 2024-05-10T05:31:58.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:58 smithi131 ceph-mon[37264]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:creating} 8 up:standby 2024-05-10T05:31:58.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:58 smithi131 ceph-mon[37264]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-05-10T05:31:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:58 smithi157 ceph-mon[78427]: pgmap v241: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 9 op/s 2024-05-10T05:31:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] up:active 2024-05-10T05:31:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3932278696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T05:31:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:58 smithi157 ceph-mon[78427]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-10T05:31:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:58 smithi157 ceph-mon[78427]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 9 up:standby 2024-05-10T05:31:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:58 smithi157 ceph-mon[78427]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:creating} 8 up:standby 2024-05-10T05:31:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:58 smithi157 ceph-mon[78427]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-05-10T05:31:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:58 smithi142 ceph-mon[77667]: pgmap v241: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 9 op/s 2024-05-10T05:31:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] up:active 2024-05-10T05:31:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3932278696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T05:31:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:58 smithi142 ceph-mon[77667]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-10T05:31:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:58 smithi142 ceph-mon[77667]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 9 up:standby 2024-05-10T05:31:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:58 smithi142 ceph-mon[77667]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:creating} 8 up:standby 2024-05-10T05:31:58.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:58 smithi142 ceph-mon[77667]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-05-10T05:31:58.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.489+0000 7f7225c00700 1 -- 172.21.15.131:0/4203336454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72201004a0 msgr2=0x7f72201008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:58.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.489+0000 7f7225c00700 1 --2- 172.21.15.131:0/4203336454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72201004a0 0x7f72201008b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f72100098f0 tx=0x7f7210009c00 comp rx=0 tx=0).stop 2024-05-10T05:31:58.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.489+0000 7f7225c00700 1 -- 172.21.15.131:0/4203336454 shutdown_connections 2024-05-10T05:31:58.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.489+0000 7f7225c00700 1 --2- 172.21.15.131:0/4203336454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72201004a0 0x7f72201008b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.489+0000 7f7225c00700 1 --2- 172.21.15.131:0/4203336454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7220101740 0x7f7220101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.489+0000 7f7225c00700 1 --2- 172.21.15.131:0/4203336454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7220100df0 0x7f7220101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.489+0000 7f7225c00700 1 -- 172.21.15.131:0/4203336454 >> 172.21.15.131:0/4203336454 conn(0x7f72200fc0d0 msgr2=0x7f72200fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:58.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.490+0000 7f7225c00700 1 -- 172.21.15.131:0/4203336454 shutdown_connections 2024-05-10T05:31:58.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.490+0000 7f7225c00700 1 -- 172.21.15.131:0/4203336454 wait complete. 2024-05-10T05:31:58.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.491+0000 7f7225c00700 1 Processor -- start 2024-05-10T05:31:58.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.491+0000 7f7225c00700 1 -- start start 2024-05-10T05:31:58.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.491+0000 7f7225c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72201004a0 0x7f722019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:58.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.491+0000 7f7225c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7220100df0 0x7f722019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:58.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.491+0000 7f7225c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7220101740 0x7f72201a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:58.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.491+0000 7f7225c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72201a1c80 con 0x7f7220101740 2024-05-10T05:31:58.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.491+0000 7f7225c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72201a1df0 con 0x7f72201004a0 2024-05-10T05:31:58.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.491+0000 7f7225c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72201a1fb0 con 0x7f7220100df0 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7220100df0 0x7f722019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72201004a0 0x7f722019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72201004a0 0x7f722019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46802/0 (socket says 172.21.15.131:46802) 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721ea00700 1 -- 172.21.15.131:0/1136262175 learned_addr learned my addr 172.21.15.131:0/1136262175 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7220101740 0x7f72201a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721ea00700 1 -- 172.21.15.131:0/1136262175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7220100df0 msgr2=0x7f722019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721ea00700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7220100df0 0x7f722019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721ea00700 1 -- 172.21.15.131:0/1136262175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7220101740 msgr2=0x7f72201a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721ea00700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7220101740 0x7f72201a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721ea00700 1 -- 172.21.15.131:0/1136262175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7218009c40 con 0x7f72201004a0 2024-05-10T05:31:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721e000700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7220100df0 0x7f722019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:58.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.492+0000 7f721f400700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7220101740 0x7f72201a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:31:58.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.493+0000 7f721ea00700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72201004a0 0x7f722019a1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7210009230 tx=0x7f721000bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:58.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.493+0000 7f7217400700 1 -- 172.21.15.131:0/1136262175 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721001c070 con 0x7f72201004a0 2024-05-10T05:31:58.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.493+0000 7f7217400700 1 -- 172.21.15.131:0/1136262175 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7210005040 con 0x7f72201004a0 2024-05-10T05:31:58.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.493+0000 7f7225c00700 1 -- 172.21.15.131:0/1136262175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7210009520 con 0x7f72201004a0 2024-05-10T05:31:58.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.493+0000 7f7217400700 1 -- 172.21.15.131:0/1136262175 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72100038e0 con 0x7f72201004a0 2024-05-10T05:31:58.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.493+0000 7f7225c00700 1 -- 172.21.15.131:0/1136262175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72201a2520 con 0x7f72201004a0 2024-05-10T05:31:58.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.494+0000 7f7225c00700 1 -- 172.21.15.131:0/1136262175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722010b990 con 0x7f72201004a0 2024-05-10T05:31:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.497+0000 7f7217400700 1 -- 172.21.15.131:0/1136262175 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7210020030 con 0x7f72201004a0 2024-05-10T05:31:58.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.498+0000 7f7217400700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72040a1240 0x7f72040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:58.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.498+0000 7f7217400700 1 -- 172.21.15.131:0/1136262175 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72100cd820 con 0x7f72201004a0 2024-05-10T05:31:58.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.498+0000 7f721e000700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72040a1240 0x7f72040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:58.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.499+0000 7f721e000700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72040a1240 0x7f72040a3750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f722019b240 tx=0x7f720c009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:58.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.503+0000 7f7217400700 1 -- 172.21.15.131:0/1136262175 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f72100970f0 con 0x7f72201004a0 2024-05-10T05:31:58.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.745+0000 7f7225c00700 1 -- 172.21.15.131:0/1136262175 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f7220061650 con 0x7f72040a1240 2024-05-10T05:31:58.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.749+0000 7f7217400700 1 -- 172.21.15.131:0/1136262175 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f7220061650 con 0x7f72040a1240 2024-05-10T05:31:58.750 INFO:teuthology.orchestra.run.smithi131.stdout:/volumes/qa/sv_0/4932b25a-6720-4264-ae11-864fcaf5e101 2024-05-10T05:31:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 -- 172.21.15.131:0/1136262175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72040a1240 msgr2=0x7f72040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72040a1240 0x7f72040a3750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f722019b240 tx=0x7f720c009310 comp rx=0 tx=0).stop 2024-05-10T05:31:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 -- 172.21.15.131:0/1136262175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72201004a0 msgr2=0x7f722019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72201004a0 0x7f722019a1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7210009230 tx=0x7f721000bfa0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 -- 172.21.15.131:0/1136262175 shutdown_connections 2024-05-10T05:31:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72040a1240 0x7f72040a3750 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7220101740 0x7f72201a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72201004a0 0x7f722019a1a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 --2- 172.21.15.131:0/1136262175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7220100df0 0x7f722019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 -- 172.21.15.131:0/1136262175 >> 172.21.15.131:0/1136262175 conn(0x7f72200fc0d0 msgr2=0x7f7220108ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:58.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 -- 172.21.15.131:0/1136262175 shutdown_connections 2024-05-10T05:31:58.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:58.752+0000 7f7225c00700 1 -- 172.21.15.131:0/1136262175 wait complete. 2024-05-10T05:31:58.779 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-10T05:31:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.054+0000 7f8f92a00700 1 -- 172.21.15.131:0/3501949308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8c101be0 msgr2=0x7f8f8c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.054+0000 7f8f92a00700 1 --2- 172.21.15.131:0/3501949308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8c101be0 0x7f8f8c101ff0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009990 tx=0x7f8f7c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.054+0000 7f8f92a00700 1 -- 172.21.15.131:0/3501949308 shutdown_connections 2024-05-10T05:31:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.054+0000 7f8f92a00700 1 --2- 172.21.15.131:0/3501949308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8c101be0 0x7f8f8c101ff0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.054+0000 7f8f92a00700 1 --2- 172.21.15.131:0/3501949308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f8c1036a0 0x7f8f8c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.054+0000 7f8f92a00700 1 --2- 172.21.15.131:0/3501949308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c102d00 0x7f8f8c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.054+0000 7f8f92a00700 1 -- 172.21.15.131:0/3501949308 >> 172.21.15.131:0/3501949308 conn(0x7f8f8c0fd210 msgr2=0x7f8f8c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:59.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.054+0000 7f8f92a00700 1 -- 172.21.15.131:0/3501949308 shutdown_connections 2024-05-10T05:31:59.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.054+0000 7f8f92a00700 1 -- 172.21.15.131:0/3501949308 wait complete. 2024-05-10T05:31:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.055+0000 7f8f92a00700 1 Processor -- start 2024-05-10T05:31:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.055+0000 7f8f92a00700 1 -- start start 2024-05-10T05:31:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.055+0000 7f8f92a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f8c101be0 0x7f8f8c19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.055+0000 7f8f92a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c102d00 0x7f8f8c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.055+0000 7f8f92a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8c1036a0 0x7f8f8c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.055+0000 7f8f92a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c1a1c20 con 0x7f8f8c1036a0 2024-05-10T05:31:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.055+0000 7f8f92a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c1a1d90 con 0x7f8f8c102d00 2024-05-10T05:31:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f92a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c1a1f50 con 0x7f8f8c101be0 2024-05-10T05:31:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f8c101be0 0x7f8f8c19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f8c101be0 0x7f8f8c19a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32914/0 (socket says 172.21.15.131:32914) 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f8be00700 1 -- 172.21.15.131:0/3539872786 learned_addr learned my addr 172.21.15.131:0/3539872786 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f8b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c102d00 0x7f8f8c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f90a00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8c1036a0 0x7f8f8c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f8be00700 1 -- 172.21.15.131:0/3539872786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c102d00 msgr2=0x7f8f8c19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f8be00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c102d00 0x7f8f8c19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f8be00700 1 -- 172.21.15.131:0/3539872786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8c1036a0 msgr2=0x7f8f8c1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f8be00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8c1036a0 0x7f8f8c1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.056+0000 7f8f8be00700 1 -- 172.21.15.131:0/3539872786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c0095f0 con 0x7f8f8c101be0 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.057+0000 7f8f8be00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f8c101be0 0x7f8f8c19a140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c000c00 tx=0x7f8f7c005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.057+0000 7f8f88c00700 1 -- 172.21.15.131:0/3539872786 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c01d070 con 0x7f8f8c101be0 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.057+0000 7f8f88c00700 1 -- 172.21.15.131:0/3539872786 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f7c005030 con 0x7f8f8c101be0 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.057+0000 7f8f92a00700 1 -- 172.21.15.131:0/3539872786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c1a21d0 con 0x7f8f8c101be0 2024-05-10T05:31:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.057+0000 7f8f88c00700 1 -- 172.21.15.131:0/3539872786 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c004ab0 con 0x7f8f8c101be0 2024-05-10T05:31:59.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.057+0000 7f8f92a00700 1 -- 172.21.15.131:0/3539872786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c1a25f0 con 0x7f8f8c101be0 2024-05-10T05:31:59.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.058+0000 7f8f92a00700 1 -- 172.21.15.131:0/3539872786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c0694b0 con 0x7f8f8c101be0 2024-05-10T05:31:59.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.060+0000 7f8f88c00700 1 -- 172.21.15.131:0/3539872786 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8f7c0051a0 con 0x7f8f8c101be0 2024-05-10T05:31:59.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.061+0000 7f8f88c00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8f700a1240 0x7f8f700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:59.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.061+0000 7f8f88c00700 1 -- 172.21.15.131:0/3539872786 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8f7c0cd860 con 0x7f8f8c101be0 2024-05-10T05:31:59.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.061+0000 7f8f8b400700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8f700a1240 0x7f8f700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:59.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.062+0000 7f8f8b400700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8f700a1240 0x7f8f700a3750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c19b1e0 tx=0x7f8f78009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:59.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.066+0000 7f8f88c00700 1 -- 172.21.15.131:0/3539872786 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8f7c05bd60 con 0x7f8f8c101be0 2024-05-10T05:31:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:59 smithi157 ceph-mon[78427]: from='client.24679 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:31:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:59 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] up:active 2024-05-10T05:31:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:59 smithi157 ceph-mon[78427]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-10T05:31:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:59 smithi157 ceph-mon[78427]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T05:31:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:59 smithi157 ceph-mon[78427]: Cluster is now healthy 2024-05-10T05:31:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:59 smithi157 ceph-mon[78427]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:active} 8 up:standby 2024-05-10T05:31:59.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:59 smithi157 ceph-mon[78427]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-10T05:31:59.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:59 smithi157 ceph-mon[78427]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-10T05:31:59.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:59 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] up:active 2024-05-10T05:31:59.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:31:59 smithi157 ceph-mon[78427]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:31:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.308+0000 7f8f92a00700 1 -- 172.21.15.131:0/3539872786 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f8c074b60 con 0x7f8f700a1240 2024-05-10T05:31:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.332+0000 7f8f88c00700 1 -- 172.21.15.131:0/3539872786 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8f8c074b60 con 0x7f8f700a1240 2024-05-10T05:31:59.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.334+0000 7f8f92a00700 1 -- 172.21.15.131:0/3539872786 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8f700a1240 msgr2=0x7f8f700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:59.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.334+0000 7f8f92a00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8f700a1240 0x7f8f700a3750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c19b1e0 tx=0x7f8f78009260 comp rx=0 tx=0).stop 2024-05-10T05:31:59.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.334+0000 7f8f92a00700 1 -- 172.21.15.131:0/3539872786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f8c101be0 msgr2=0x7f8f8c19a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:59.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.334+0000 7f8f92a00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f8c101be0 0x7f8f8c19a140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c000c00 tx=0x7f8f7c005b00 comp rx=0 tx=0).stop 2024-05-10T05:31:59.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.334+0000 7f8f92a00700 1 -- 172.21.15.131:0/3539872786 shutdown_connections 2024-05-10T05:31:59.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.334+0000 7f8f92a00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8f700a1240 0x7f8f700a3750 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.334+0000 7f8f92a00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8c1036a0 0x7f8f8c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.334+0000 7f8f92a00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f8c101be0 0x7f8f8c19a140 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.334+0000 7f8f92a00700 1 --2- 172.21.15.131:0/3539872786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f8c102d00 0x7f8f8c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.334+0000 7f8f92a00700 1 -- 172.21.15.131:0/3539872786 >> 172.21.15.131:0/3539872786 conn(0x7f8f8c0fd210 msgr2=0x7f8f8c073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:59.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.335+0000 7f8f92a00700 1 -- 172.21.15.131:0/3539872786 shutdown_connections 2024-05-10T05:31:59.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.335+0000 7f8f92a00700 1 -- 172.21.15.131:0/3539872786 wait complete. 2024-05-10T05:31:59.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:59 smithi131 ceph-mon[37264]: from='client.24679 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:31:59.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:59 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] up:active 2024-05-10T05:31:59.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:59 smithi131 ceph-mon[37264]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-10T05:31:59.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:59 smithi131 ceph-mon[37264]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T05:31:59.362 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:59 smithi131 ceph-mon[37264]: Cluster is now healthy 2024-05-10T05:31:59.362 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:59 smithi131 ceph-mon[37264]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:active} 8 up:standby 2024-05-10T05:31:59.362 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:59 smithi131 ceph-mon[37264]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-10T05:31:59.362 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:59 smithi131 ceph-mon[37264]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-10T05:31:59.362 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:59 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] up:active 2024-05-10T05:31:59.362 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:31:59 smithi131 ceph-mon[37264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:31:59.364 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-10T05:31:59.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:59 smithi142 ceph-mon[77667]: from='client.24679 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:31:59.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:59 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] up:active 2024-05-10T05:31:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:59 smithi142 ceph-mon[77667]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-10T05:31:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:59 smithi142 ceph-mon[77667]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T05:31:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:59 smithi142 ceph-mon[77667]: Cluster is now healthy 2024-05-10T05:31:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:59 smithi142 ceph-mon[77667]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:active} 8 up:standby 2024-05-10T05:31:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:59 smithi142 ceph-mon[77667]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-10T05:31:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:59 smithi142 ceph-mon[77667]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-10T05:31:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:59 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] up:active 2024-05-10T05:31:59.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:31:59 smithi142 ceph-mon[77667]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:31:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.642+0000 7f72cb000700 1 -- 172.21.15.131:0/2021512190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c41036a0 msgr2=0x7f72c4109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.642+0000 7f72cb000700 1 --2- 172.21.15.131:0/2021512190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c41036a0 0x7f72c4109f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00b0a0 tx=0x7f72bc00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.642+0000 7f72cb000700 1 -- 172.21.15.131:0/2021512190 shutdown_connections 2024-05-10T05:31:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.642+0000 7f72cb000700 1 --2- 172.21.15.131:0/2021512190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c41036a0 0x7f72c4109f50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.642+0000 7f72cb000700 1 --2- 172.21.15.131:0/2021512190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c4101be0 0x7f72c4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.642+0000 7f72cb000700 1 --2- 172.21.15.131:0/2021512190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c4102d00 0x7f72c4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.642+0000 7f72cb000700 1 -- 172.21.15.131:0/2021512190 >> 172.21.15.131:0/2021512190 conn(0x7f72c40fd1d0 msgr2=0x7f72c40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:59.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.642+0000 7f72cb000700 1 -- 172.21.15.131:0/2021512190 shutdown_connections 2024-05-10T05:31:59.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.642+0000 7f72cb000700 1 -- 172.21.15.131:0/2021512190 wait complete. 2024-05-10T05:31:59.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.643+0000 7f72cb000700 1 Processor -- start 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.643+0000 7f72cb000700 1 -- start start 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72cb000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c4101be0 0x7f72c4074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72cb000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c4102d00 0x7f72c4073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72cb000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c41036a0 0x7f72c4073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72cb000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4074180 con 0x7f72c4101be0 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72cb000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40742f0 con 0x7f72c4102d00 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72cb000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c41a20d0 con 0x7f72c41036a0 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72c3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c4101be0 0x7f72c4074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72c3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c4101be0 0x7f72c4074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49610/0 (socket says 172.21.15.131:49610) 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72c3e00700 1 -- 172.21.15.131:0/766436857 learned_addr learned my addr 172.21.15.131:0/766436857 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72bbe00700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c4102d00 0x7f72c4073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72c9000700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c41036a0 0x7f72c4073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72c3e00700 1 -- 172.21.15.131:0/766436857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c41036a0 msgr2=0x7f72c4073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72c3e00700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c41036a0 0x7f72c4073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72c3e00700 1 -- 172.21.15.131:0/766436857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c4102d00 msgr2=0x7f72c4073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72c3e00700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c4102d00 0x7f72c4073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.644+0000 7f72c3e00700 1 -- 172.21.15.131:0/766436857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b4009520 con 0x7f72c4101be0 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.645+0000 7f72c3e00700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c4101be0 0x7f72c4074ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f72b400ea10 tx=0x7f72b400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.645+0000 7f72c1600700 1 -- 172.21.15.131:0/766436857 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b400cbd0 con 0x7f72c4101be0 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.645+0000 7f72cb000700 1 -- 172.21.15.131:0/766436857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc009d00 con 0x7f72c4101be0 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.645+0000 7f72c1600700 1 -- 172.21.15.131:0/766436857 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72b400eeb0 con 0x7f72c4101be0 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.645+0000 7f72c1600700 1 -- 172.21.15.131:0/766436857 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4018680 con 0x7f72c4101be0 2024-05-10T05:31:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.645+0000 7f72cb000700 1 -- 172.21.15.131:0/766436857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c41a26d0 con 0x7f72c4101be0 2024-05-10T05:31:59.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.646+0000 7f72cb000700 1 -- 172.21.15.131:0/766436857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c4108df0 con 0x7f72c4101be0 2024-05-10T05:31:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.648+0000 7f72c1600700 1 -- 172.21.15.131:0/766436857 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f72b4010430 con 0x7f72c4101be0 2024-05-10T05:31:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.649+0000 7f72c1600700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72a80a1260 0x7f72a80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:31:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.650+0000 7f72c1600700 1 -- 172.21.15.131:0/766436857 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72b4014070 con 0x7f72c4101be0 2024-05-10T05:31:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.650+0000 7f72bbe00700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72a80a1260 0x7f72a80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:31:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.650+0000 7f72bbe00700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72a80a1260 0x7f72a80a3770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f72c4074860 tx=0x7f72ac009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:31:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.654+0000 7f72c1600700 1 -- 172.21.15.131:0/766436857 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f72b4096470 con 0x7f72c4101be0 2024-05-10T05:31:59.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.901+0000 7f72cb000700 1 -- 172.21.15.131:0/766436857 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f72c402ce70 con 0x7f72a80a1260 2024-05-10T05:31:59.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.905+0000 7f72c1600700 1 -- 172.21.15.131:0/766436857 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f72c402ce70 con 0x7f72a80a1260 2024-05-10T05:31:59.906 INFO:teuthology.orchestra.run.smithi131.stdout:/volumes/qa/sv_1/dce0b8c1-4e28-48fa-a70b-816ec396fce0 2024-05-10T05:31:59.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.907+0000 7f72cb000700 1 -- 172.21.15.131:0/766436857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72a80a1260 msgr2=0x7f72a80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.907+0000 7f72cb000700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72a80a1260 0x7f72a80a3770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f72c4074860 tx=0x7f72ac009260 comp rx=0 tx=0).stop 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.908+0000 7f72cb000700 1 -- 172.21.15.131:0/766436857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c4101be0 msgr2=0x7f72c4074ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.908+0000 7f72cb000700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c4101be0 0x7f72c4074ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f72b400ea10 tx=0x7f72b400ed20 comp rx=0 tx=0).stop 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.908+0000 7f72cb000700 1 -- 172.21.15.131:0/766436857 shutdown_connections 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.908+0000 7f72cb000700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f72a80a1260 0x7f72a80a3770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.908+0000 7f72cb000700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72c4101be0 0x7f72c4074ec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.908+0000 7f72cb000700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c41036a0 0x7f72c4073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.908+0000 7f72cb000700 1 --2- 172.21.15.131:0/766436857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c4102d00 0x7f72c4073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.908+0000 7f72cb000700 1 -- 172.21.15.131:0/766436857 >> 172.21.15.131:0/766436857 conn(0x7f72c40fd1d0 msgr2=0x7f72c4105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.908+0000 7f72cb000700 1 -- 172.21.15.131:0/766436857 shutdown_connections 2024-05-10T05:31:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:31:59.908+0000 7f72cb000700 1 -- 172.21.15.131:0/766436857 wait complete. 2024-05-10T05:31:59.935 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:32:00.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.200+0000 7f48e9e00700 1 -- 172.21.15.131:0/4277599900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e41036c0 msgr2=0x7f48e4109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:00.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.200+0000 7f48e9e00700 1 --2- 172.21.15.131:0/4277599900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e41036c0 0x7f48e4109f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00b0a0 tx=0x7f48dc00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.200+0000 7f48e9e00700 1 -- 172.21.15.131:0/4277599900 shutdown_connections 2024-05-10T05:32:00.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.200+0000 7f48e9e00700 1 --2- 172.21.15.131:0/4277599900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e41036c0 0x7f48e4109f70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.200+0000 7f48e9e00700 1 --2- 172.21.15.131:0/4277599900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48e4101c00 0x7f48e4102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.200+0000 7f48e9e00700 1 --2- 172.21.15.131:0/4277599900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4102d20 0x7f48e4103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.200+0000 7f48e9e00700 1 -- 172.21.15.131:0/4277599900 >> 172.21.15.131:0/4277599900 conn(0x7f48e40fd210 msgr2=0x7f48e40ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:00.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.200+0000 7f48e9e00700 1 -- 172.21.15.131:0/4277599900 shutdown_connections 2024-05-10T05:32:00.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.200+0000 7f48e9e00700 1 -- 172.21.15.131:0/4277599900 wait complete. 2024-05-10T05:32:00.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.201+0000 7f48e9e00700 1 Processor -- start 2024-05-10T05:32:00.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.201+0000 7f48e9e00700 1 -- start start 2024-05-10T05:32:00.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4101c00 0x7f48e419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:00.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4102d20 0x7f48e419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:00.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48e41036c0 0x7f48e41a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:00.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e9e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41a1df0 con 0x7f48e4102d20 2024-05-10T05:32:00.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e9e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41a1f60 con 0x7f48e41036c0 2024-05-10T05:32:00.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e9e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41a2120 con 0x7f48e4101c00 2024-05-10T05:32:00.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e2000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4102d20 0x7f48e419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:00.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e2000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4102d20 0x7f48e419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49638/0 (socket says 172.21.15.131:49638) 2024-05-10T05:32:00.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e2000700 1 -- 172.21.15.131:0/2891591818 learned_addr learned my addr 172.21.15.131:0/2891591818 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e2a00700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4101c00 0x7f48e419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e3400700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48e41036c0 0x7f48e41a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e2000700 1 -- 172.21.15.131:0/2891591818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4101c00 msgr2=0x7f48e419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e2000700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4101c00 0x7f48e419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e2000700 1 -- 172.21.15.131:0/2891591818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48e41036c0 msgr2=0x7f48e41a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e2000700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48e41036c0 0x7f48e41a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.202+0000 7f48e2000700 1 -- 172.21.15.131:0/2891591818 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d0009520 con 0x7f48e4102d20 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.203+0000 7f48e2000700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4102d20 0x7f48e419a850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48d000eb30 tx=0x7f48d000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.203+0000 7f48e8a00700 1 -- 172.21.15.131:0/2891591818 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0009880 con 0x7f48e4102d20 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.203+0000 7f48e8a00700 1 -- 172.21.15.131:0/2891591818 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d00044e0 con 0x7f48e4102d20 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.203+0000 7f48e9e00700 1 -- 172.21.15.131:0/2891591818 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc009d00 con 0x7f48e4102d20 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.203+0000 7f48e8a00700 1 -- 172.21.15.131:0/2891591818 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0003d80 con 0x7f48e4102d20 2024-05-10T05:32:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.203+0000 7f48e9e00700 1 -- 172.21.15.131:0/2891591818 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e41a27d0 con 0x7f48e4102d20 2024-05-10T05:32:00.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.204+0000 7f48e9e00700 1 -- 172.21.15.131:0/2891591818 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e4068730 con 0x7f48e4102d20 2024-05-10T05:32:00.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.206+0000 7f48e8a00700 1 -- 172.21.15.131:0/2891591818 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f48d0003f20 con 0x7f48e4102d20 2024-05-10T05:32:00.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.207+0000 7f48e8a00700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48c80a1230 0x7f48c80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:00.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.207+0000 7f48e8a00700 1 -- 172.21.15.131:0/2891591818 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f48d0014070 con 0x7f48e4102d20 2024-05-10T05:32:00.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.207+0000 7f48e2a00700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48c80a1230 0x7f48c80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:00.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.208+0000 7f48e2a00700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48c80a1230 0x7f48c80a3740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48e4102c20 tx=0x7f48d4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:00.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:00 smithi131 ceph-mon[37264]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:00.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:00 smithi131 ceph-mon[37264]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:00.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:00 smithi131 ceph-mon[37264]: pgmap v242: 97 pgs: 97 active+clean; 580 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s wr, 15 op/s 2024-05-10T05:32:00.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:00 smithi131 ceph-mon[37264]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-10T05:32:00.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.212+0000 7f48e8a00700 1 -- 172.21.15.131:0/2891591818 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48d00d3080 con 0x7f48e4102d20 2024-05-10T05:32:00.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:00 smithi157 ceph-mon[78427]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:00.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:00 smithi157 ceph-mon[78427]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:00.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:00 smithi157 ceph-mon[78427]: pgmap v242: 97 pgs: 97 active+clean; 580 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s wr, 15 op/s 2024-05-10T05:32:00.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:00 smithi157 ceph-mon[78427]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-10T05:32:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:00 smithi142 ceph-mon[77667]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:00 smithi142 ceph-mon[77667]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:00 smithi142 ceph-mon[77667]: pgmap v242: 97 pgs: 97 active+clean; 580 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s wr, 15 op/s 2024-05-10T05:32:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:00 smithi142 ceph-mon[77667]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-10T05:32:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.465+0000 7f48e9e00700 1 -- 172.21.15.131:0/2891591818 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48e410d460 con 0x7f48e4102d20 2024-05-10T05:32:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.465+0000 7f48e8a00700 1 -- 172.21.15.131:0/2891591818 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f48d0096560 con 0x7f48e4102d20 2024-05-10T05:32:00.466 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:00.466 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:31:58.482427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:32:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 -- 172.21.15.131:0/2891591818 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48c80a1230 msgr2=0x7f48c80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48c80a1230 0x7f48c80a3740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48e4102c20 tx=0x7f48d4009260 comp rx=0 tx=0).stop 2024-05-10T05:32:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 -- 172.21.15.131:0/2891591818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4102d20 msgr2=0x7f48e419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4102d20 0x7f48e419a850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48d000eb30 tx=0x7f48d000ee40 comp rx=0 tx=0).stop 2024-05-10T05:32:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 -- 172.21.15.131:0/2891591818 shutdown_connections 2024-05-10T05:32:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48c80a1230 0x7f48c80a3740 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48e41036c0 0x7f48e41a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4101c00 0x7f48e419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 --2- 172.21.15.131:0/2891591818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4102d20 0x7f48e419a850 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 -- 172.21.15.131:0/2891591818 >> 172.21.15.131:0/2891591818 conn(0x7f48e40fd210 msgr2=0x7f48e4074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 -- 172.21.15.131:0/2891591818 shutdown_connections 2024-05-10T05:32:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.468+0000 7f48e9e00700 1 -- 172.21.15.131:0/2891591818 wait complete. 2024-05-10T05:32:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-05-10T05:32:00.495 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:32:00.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.757+0000 7f0ec2e00700 1 -- 172.21.15.131:0/3165107963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ebc109ae0 msgr2=0x7f0ebc10bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:00.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.757+0000 7f0ec2e00700 1 --2- 172.21.15.131:0/3165107963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ebc109ae0 0x7f0ebc10bfb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400b0a0 tx=0x7f0eb400b3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.757+0000 7f0ec2e00700 1 -- 172.21.15.131:0/3165107963 shutdown_connections 2024-05-10T05:32:00.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.757+0000 7f0ec2e00700 1 --2- 172.21.15.131:0/3165107963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ebc109ae0 0x7f0ebc10bfb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.757+0000 7f0ec2e00700 1 --2- 172.21.15.131:0/3165107963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ebc0750e0 0x7f0ebc0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.757+0000 7f0ec2e00700 1 --2- 172.21.15.131:0/3165107963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ebc073ae0 0x7f0ebc073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.757+0000 7f0ec2e00700 1 -- 172.21.15.131:0/3165107963 >> 172.21.15.131:0/3165107963 conn(0x7f0ebc0fd210 msgr2=0x7f0ebc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.757+0000 7f0ec2e00700 1 -- 172.21.15.131:0/3165107963 shutdown_connections 2024-05-10T05:32:00.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.757+0000 7f0ec2e00700 1 -- 172.21.15.131:0/3165107963 wait complete. 2024-05-10T05:32:00.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.758+0000 7f0ec2e00700 1 Processor -- start 2024-05-10T05:32:00.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.758+0000 7f0ec2e00700 1 -- start start 2024-05-10T05:32:00.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ec2e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ebc073ae0 0x7f0ebc19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:00.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ec2e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ebc0750e0 0x7f0ebc19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:00.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ec2e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ebc109ae0 0x7f0ebc1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:00.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ec2e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc1a1d60 con 0x7f0ebc073ae0 2024-05-10T05:32:00.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ec2e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc1a1ed0 con 0x7f0ebc109ae0 2024-05-10T05:32:00.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ec2e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc1a2090 con 0x7f0ebc0750e0 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ebbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ebc073ae0 0x7f0ebc19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ebbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ebc073ae0 0x7f0ebc19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49658/0 (socket says 172.21.15.131:49658) 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ebbe00700 1 -- 172.21.15.131:0/1499649470 learned_addr learned my addr 172.21.15.131:0/1499649470 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ebb400700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ebc0750e0 0x7f0ebc19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ec0e00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ebc109ae0 0x7f0ebc1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ebbe00700 1 -- 172.21.15.131:0/1499649470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ebc0750e0 msgr2=0x7f0ebc19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ebbe00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ebc0750e0 0x7f0ebc19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ebbe00700 1 -- 172.21.15.131:0/1499649470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ebc109ae0 msgr2=0x7f0ebc1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ebbe00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ebc109ae0 0x7f0ebc1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.759+0000 7f0ebbe00700 1 -- 172.21.15.131:0/1499649470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea8009520 con 0x7f0ebc073ae0 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.760+0000 7f0ebbe00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ebc073ae0 0x7f0ebc19a2d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00e970 tx=0x7f0eac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.760+0000 7f0eb8c00700 1 -- 172.21.15.131:0/1499649470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac009790 con 0x7f0ebc073ae0 2024-05-10T05:32:00.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.760+0000 7f0eb8c00700 1 -- 172.21.15.131:0/1499649470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eac00c7e0 con 0x7f0ebc073ae0 2024-05-10T05:32:00.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.760+0000 7f0ec2e00700 1 -- 172.21.15.131:0/1499649470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb4009d00 con 0x7f0ebc073ae0 2024-05-10T05:32:00.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.760+0000 7f0eb8c00700 1 -- 172.21.15.131:0/1499649470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0054e0 con 0x7f0ebc073ae0 2024-05-10T05:32:00.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.760+0000 7f0ec2e00700 1 -- 172.21.15.131:0/1499649470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ebc1a2740 con 0x7f0ebc073ae0 2024-05-10T05:32:00.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.761+0000 7f0ec2e00700 1 -- 172.21.15.131:0/1499649470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ebc108930 con 0x7f0ebc073ae0 2024-05-10T05:32:00.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.763+0000 7f0eb8c00700 1 -- 172.21.15.131:0/1499649470 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0eac00c950 con 0x7f0ebc073ae0 2024-05-10T05:32:00.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.765+0000 7f0eb8c00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ea00a1230 0x7f0ea00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:00.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.765+0000 7f0eb8c00700 1 -- 172.21.15.131:0/1499649470 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0eac014070 con 0x7f0ebc073ae0 2024-05-10T05:32:00.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.765+0000 7f0ebb400700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ea00a1230 0x7f0ea00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:00.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.766+0000 7f0ebb400700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ea00a1230 0x7f0ea00a3740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc19b320 tx=0x7f0ea8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:00.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.769+0000 7f0eb8c00700 1 -- 172.21.15.131:0/1499649470 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0eac0962b0 con 0x7f0ebc073ae0 2024-05-10T05:32:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.977+0000 7f0ec2e00700 1 -- 172.21.15.131:0/1499649470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ebc0694b0 con 0x7f0ebc073ae0 2024-05-10T05:32:00.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.978+0000 7f0eb8c00700 1 -- 172.21.15.131:0/1499649470 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22191 (secure 0 0 0) 0x7f0eac0998d0 con 0x7f0ebc073ae0 2024-05-10T05:32:00.980 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:00.980 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":80,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:31:52.550067+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 -- 172.21.15.131:0/1499649470 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ea00a1230 msgr2=0x7f0ea00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ea00a1230 0x7f0ea00a3740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc19b320 tx=0x7f0ea8009260 comp rx=0 tx=0).stop 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 -- 172.21.15.131:0/1499649470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ebc073ae0 msgr2=0x7f0ebc19a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ebc073ae0 0x7f0ebc19a2d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00e970 tx=0x7f0eac00ec80 comp rx=0 tx=0).stop 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 -- 172.21.15.131:0/1499649470 shutdown_connections 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ea00a1230 0x7f0ea00a3740 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ebc073ae0 0x7f0ebc19a2d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ebc109ae0 0x7f0ebc1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 --2- 172.21.15.131:0/1499649470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ebc0750e0 0x7f0ebc19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 -- 172.21.15.131:0/1499649470 >> 172.21.15.131:0/1499649470 conn(0x7f0ebc0fd210 msgr2=0x7f0ebc105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 -- 172.21.15.131:0/1499649470 shutdown_connections 2024-05-10T05:32:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:00.981+0000 7f0ec2e00700 1 -- 172.21.15.131:0/1499649470 wait complete. 2024-05-10T05:32:01.007 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-10T05:32:01.007 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T05:32:01.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.267+0000 7fa755800700 1 -- 172.21.15.131:0/2246726878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa750102d00 msgr2=0x7fa750103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:01.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.267+0000 7fa755800700 1 --2- 172.21.15.131:0/2246726878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa750102d00 0x7fa750103110 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009990 tx=0x7fa73c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.267+0000 7fa755800700 1 -- 172.21.15.131:0/2246726878 shutdown_connections 2024-05-10T05:32:01.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.267+0000 7fa755800700 1 --2- 172.21.15.131:0/2246726878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7501036a0 0x7fa750109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.267+0000 7fa755800700 1 --2- 172.21.15.131:0/2246726878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa750101be0 0x7fa750101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.267+0000 7fa755800700 1 --2- 172.21.15.131:0/2246726878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa750102d00 0x7fa750103110 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.267+0000 7fa755800700 1 -- 172.21.15.131:0/2246726878 >> 172.21.15.131:0/2246726878 conn(0x7fa7500fd210 msgr2=0x7fa7500ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:01.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.267+0000 7fa755800700 1 -- 172.21.15.131:0/2246726878 shutdown_connections 2024-05-10T05:32:01.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.267+0000 7fa755800700 1 -- 172.21.15.131:0/2246726878 wait complete. 2024-05-10T05:32:01.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.268+0000 7fa755800700 1 Processor -- start 2024-05-10T05:32:01.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.268+0000 7fa755800700 1 -- start start 2024-05-10T05:32:01.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa755800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa750101be0 0x7fa75019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:01.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa755800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750102d00 0x7fa75019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa755800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501036a0 0x7fa7501a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa755800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7501a1cd0 con 0x7fa7501036a0 2024-05-10T05:32:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa755800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7501a1e40 con 0x7fa750101be0 2024-05-10T05:32:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa755800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7501a2000 con 0x7fa750102d00 2024-05-10T05:32:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa74f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501036a0 0x7fa7501a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa74e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750102d00 0x7fa75019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa74ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa750101be0 0x7fa75019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa74f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501036a0 0x7fa7501a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49684/0 (socket says 172.21.15.131:49684) 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa74f400700 1 -- 172.21.15.131:0/4250681652 learned_addr learned my addr 172.21.15.131:0/4250681652 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa74e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750102d00 0x7fa75019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32990/0 (socket says 172.21.15.131:32990) 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa74f400700 1 -- 172.21.15.131:0/4250681652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750102d00 msgr2=0x7fa75019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa74f400700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750102d00 0x7fa75019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa74f400700 1 -- 172.21.15.131:0/4250681652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa750101be0 msgr2=0x7fa75019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.269+0000 7fa74f400700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa750101be0 0x7fa75019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.270+0000 7fa74f400700 1 -- 172.21.15.131:0/4250681652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c0095f0 con 0x7fa7501036a0 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.270+0000 7fa74ea00700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa750101be0 0x7fa75019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.270+0000 7fa74f400700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501036a0 0x7fa7501a16c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa74800ef30 tx=0x7fa74800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.270+0000 7fa747400700 1 -- 172.21.15.131:0/4250681652 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748018070 con 0x7fa7501036a0 2024-05-10T05:32:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.270+0000 7fa747400700 1 -- 172.21.15.131:0/4250681652 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa74800f040 con 0x7fa7501036a0 2024-05-10T05:32:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.270+0000 7fa755800700 1 -- 172.21.15.131:0/4250681652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7501a2280 con 0x7fa7501036a0 2024-05-10T05:32:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.270+0000 7fa747400700 1 -- 172.21.15.131:0/4250681652 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74800cd00 con 0x7fa7501036a0 2024-05-10T05:32:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.270+0000 7fa755800700 1 -- 172.21.15.131:0/4250681652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7501a2790 con 0x7fa7501036a0 2024-05-10T05:32:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.271+0000 7fa755800700 1 -- 172.21.15.131:0/4250681652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7500694d0 con 0x7fa7501036a0 2024-05-10T05:32:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.273+0000 7fa747400700 1 -- 172.21.15.131:0/4250681652 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa748003a00 con 0x7fa7501036a0 2024-05-10T05:32:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.274+0000 7fa747400700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa7340a12b0 0x7fa7340a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.275+0000 7fa747400700 1 -- 172.21.15.131:0/4250681652 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7480cc3f0 con 0x7fa7501036a0 2024-05-10T05:32:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.275+0000 7fa74ea00700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa7340a12b0 0x7fa7340a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.275+0000 7fa74ea00700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa7340a12b0 0x7fa7340a37c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa750102c00 tx=0x7fa740009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:01.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.279+0000 7fa747400700 1 -- 172.21.15.131:0/4250681652 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa748095b60 con 0x7fa7501036a0 2024-05-10T05:32:01.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:01 smithi157 ceph-mon[78427]: from='client.24703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:01.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:01 smithi157 ceph-mon[78427]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:01.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:01 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2891591818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:01.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:01 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1499649470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:01.394 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:32:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-b[115203]: 2024-05-10T05:32:00.905+0000 7f5a1e8cf700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-10T05:32:01.394 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:32:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-b[115203]: 2024-05-10T05:32:00.905+0000 7f5a1e8cf700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-10T05:32:01.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:01 smithi142 ceph-mon[77667]: from='client.24703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:01.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:01 smithi142 ceph-mon[77667]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:01.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:01 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2891591818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:01.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:01 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1499649470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:01.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:01 smithi131 ceph-mon[37264]: from='client.24703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:01.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:01 smithi131 ceph-mon[37264]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:01.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:01 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2891591818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:01.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:01 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1499649470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.483+0000 7fa755800700 1 -- 172.21.15.131:0/4250681652 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa750074b60 con 0x7fa7340a12b0 2024-05-10T05:32:01.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.496+0000 7fa747400700 1 -- 172.21.15.131:0/4250681652 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+328777 (secure 0 0 0) 0x7fa750074b60 con 0x7fa7340a12b0 2024-05-10T05:32:01.503 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:01.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 -- 172.21.15.131:0/4250681652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa7340a12b0 msgr2=0x7fa7340a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:01.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa7340a12b0 0x7fa7340a37c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa750102c00 tx=0x7fa740009260 comp rx=0 tx=0).stop 2024-05-10T05:32:01.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 -- 172.21.15.131:0/4250681652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501036a0 msgr2=0x7fa7501a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:01.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501036a0 0x7fa7501a16c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa74800ef30 tx=0x7fa74800c3a0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 -- 172.21.15.131:0/4250681652 shutdown_connections 2024-05-10T05:32:01.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa7340a12b0 0x7fa7340a37c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501036a0 0x7fa7501a16c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa750101be0 0x7fa75019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 --2- 172.21.15.131:0/4250681652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa750102d00 0x7fa75019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 -- 172.21.15.131:0/4250681652 >> 172.21.15.131:0/4250681652 conn(0x7fa7500fd210 msgr2=0x7fa750073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:01.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 -- 172.21.15.131:0/4250681652 shutdown_connections 2024-05-10T05:32:01.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.501+0000 7fa755800700 1 -- 172.21.15.131:0/4250681652 wait complete. 2024-05-10T05:32:01.507 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-10T05:32:01.527 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-05-10T05:32:00.829646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598194,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":149,"num_read_kb":128,"num_write":362,"num_write_kb":2998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375664,"kb_used_data":10508,"kb_used_omap":18,"kb_used_meta":365037,"kb_avail":1124566160,"statfs":{"total":1151940427776,"available":1151555747840,"internally_reserved":0,"allocated":10760192,"data_stored":6767922,"data_compressed":110038,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19128,"internal_metadata":373798216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7826,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":181,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.279082"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237828+0000","last_change":"2024-05-10T05:31:51.576054+0000","last_active":"2024-05-10T05:31:53.237828+0000","last_peered":"2024-05-10T05:31:53.237828+0000","last_clean":"2024-05-10T05:31:53.237828+0000","last_became_active":"2024-05-10T05:31:51.575186+0000","last_became_peered":"2024-05-10T05:31:51.575186+0000","last_unstale":"2024-05-10T05:31:53.237828+0000","last_undegraded":"2024-05-10T05:31:53.237828+0000","last_fullsized":"2024-05-10T05:31:53.237828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:16:27.654822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258780+0000","last_change":"2024-05-10T05:31:51.576483+0000","last_active":"2024-05-10T05:31:53.258780+0000","last_peered":"2024-05-10T05:31:53.258780+0000","last_clean":"2024-05-10T05:31:53.258780+0000","last_became_active":"2024-05-10T05:31:51.574936+0000","last_became_peered":"2024-05-10T05:31:51.574936+0000","last_unstale":"2024-05-10T05:31:53.258780+0000","last_undegraded":"2024-05-10T05:31:53.258780+0000","last_fullsized":"2024-05-10T05:31:53.258780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:56:51.436880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258626+0000","last_change":"2024-05-10T05:31:51.567114+0000","last_active":"2024-05-10T05:31:53.258626+0000","last_peered":"2024-05-10T05:31:53.258626+0000","last_clean":"2024-05-10T05:31:53.258626+0000","last_became_active":"2024-05-10T05:31:51.564552+0000","last_became_peered":"2024-05-10T05:31:51.564552+0000","last_unstale":"2024-05-10T05:31:53.258626+0000","last_undegraded":"2024-05-10T05:31:53.258626+0000","last_fullsized":"2024-05-10T05:31:53.258626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:40.268764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.814602+0000","last_change":"2024-05-10T05:31:51.584471+0000","last_active":"2024-05-10T05:31:52.814602+0000","last_peered":"2024-05-10T05:31:52.814602+0000","last_clean":"2024-05-10T05:31:52.814602+0000","last_became_active":"2024-05-10T05:31:51.583749+0000","last_became_peered":"2024-05-10T05:31:51.583749+0000","last_unstale":"2024-05-10T05:31:52.814602+0000","last_undegraded":"2024-05-10T05:31:52.814602+0000","last_fullsized":"2024-05-10T05:31:52.814602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:30.775286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258572+0000","last_change":"2024-05-10T05:31:51.577698+0000","last_active":"2024-05-10T05:31:53.258572+0000","last_peered":"2024-05-10T05:31:53.258572+0000","last_clean":"2024-05-10T05:31:53.258572+0000","last_became_active":"2024-05-10T05:31:51.571469+0000","last_became_peered":"2024-05-10T05:31:51.571469+0000","last_unstale":"2024-05-10T05:31:53.258572+0000","last_undegraded":"2024-05-10T05:31:53.258572+0000","last_fullsized":"2024-05-10T05:31:53.258572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:35.161267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.574263+0000","last_change":"2024-05-10T05:31:51.573560+0000","last_active":"2024-05-10T05:31:51.574263+0000","last_peered":"2024-05-10T05:31:51.574263+0000","last_clean":"2024-05-10T05:31:51.574263+0000","last_became_active":"2024-05-10T05:31:51.573227+0000","last_became_peered":"2024-05-10T05:31:51.573227+0000","last_unstale":"2024-05-10T05:31:51.574263+0000","last_undegraded":"2024-05-10T05:31:51.574263+0000","last_fullsized":"2024-05-10T05:31:51.574263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:23.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.579534+0000","last_change":"2024-05-10T05:31:51.573667+0000","last_active":"2024-05-10T05:31:51.579534+0000","last_peered":"2024-05-10T05:31:51.579534+0000","last_clean":"2024-05-10T05:31:51.579534+0000","last_became_active":"2024-05-10T05:31:51.573362+0000","last_became_peered":"2024-05-10T05:31:51.573362+0000","last_unstale":"2024-05-10T05:31:51.579534+0000","last_undegraded":"2024-05-10T05:31:51.579534+0000","last_fullsized":"2024-05-10T05:31:51.579534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:00.300134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258155+0000","last_change":"2024-05-10T05:31:51.583596+0000","last_active":"2024-05-10T05:31:53.258155+0000","last_peered":"2024-05-10T05:31:53.258155+0000","last_clean":"2024-05-10T05:31:53.258155+0000","last_became_active":"2024-05-10T05:31:51.583235+0000","last_became_peered":"2024-05-10T05:31:51.583235+0000","last_unstale":"2024-05-10T05:31:53.258155+0000","last_undegraded":"2024-05-10T05:31:53.258155+0000","last_fullsized":"2024-05-10T05:31:53.258155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:05.575975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534295+0000","last_change":"2024-05-10T05:31:51.567627+0000","last_active":"2024-05-10T05:31:52.534295+0000","last_peered":"2024-05-10T05:31:52.534295+0000","last_clean":"2024-05-10T05:31:52.534295+0000","last_became_active":"2024-05-10T05:31:51.567276+0000","last_became_peered":"2024-05-10T05:31:51.567276+0000","last_unstale":"2024-05-10T05:31:52.534295+0000","last_undegraded":"2024-05-10T05:31:52.534295+0000","last_fullsized":"2024-05-10T05:31:52.534295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:16:27.914802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.574732+0000","last_change":"2024-05-10T05:31:51.573859+0000","last_active":"2024-05-10T05:31:51.574732+0000","last_peered":"2024-05-10T05:31:51.574732+0000","last_clean":"2024-05-10T05:31:51.574732+0000","last_became_active":"2024-05-10T05:31:51.573493+0000","last_became_peered":"2024-05-10T05:31:51.573493+0000","last_unstale":"2024-05-10T05:31:51.574732+0000","last_undegraded":"2024-05-10T05:31:51.574732+0000","last_fullsized":"2024-05-10T05:31:51.574732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:21.024888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.578984+0000","last_change":"2024-05-10T05:31:51.576056+0000","last_active":"2024-05-10T05:31:51.578984+0000","last_peered":"2024-05-10T05:31:51.578984+0000","last_clean":"2024-05-10T05:31:51.578984+0000","last_became_active":"2024-05-10T05:31:51.575146+0000","last_became_peered":"2024-05-10T05:31:51.575146+0000","last_unstale":"2024-05-10T05:31:51.578984+0000","last_undegraded":"2024-05-10T05:31:51.578984+0000","last_fullsized":"2024-05-10T05:31:51.578984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:41.201052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.814589+0000","last_change":"2024-05-10T05:31:51.584710+0000","last_active":"2024-05-10T05:31:52.814589+0000","last_peered":"2024-05-10T05:31:52.814589+0000","last_clean":"2024-05-10T05:31:52.814589+0000","last_became_active":"2024-05-10T05:31:51.584122+0000","last_became_peered":"2024-05-10T05:31:51.584122+0000","last_unstale":"2024-05-10T05:31:52.814589+0000","last_undegraded":"2024-05-10T05:31:52.814589+0000","last_fullsized":"2024-05-10T05:31:52.814589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:43.622981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560032+0000","last_change":"2024-05-10T05:31:51.576104+0000","last_active":"2024-05-10T05:31:52.560032+0000","last_peered":"2024-05-10T05:31:52.560032+0000","last_clean":"2024-05-10T05:31:52.560032+0000","last_became_active":"2024-05-10T05:31:51.575650+0000","last_became_peered":"2024-05-10T05:31:51.575650+0000","last_unstale":"2024-05-10T05:31:52.560032+0000","last_undegraded":"2024-05-10T05:31:52.560032+0000","last_fullsized":"2024-05-10T05:31:52.560032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:34:28.798729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258953+0000","last_change":"2024-05-10T05:31:51.574628+0000","last_active":"2024-05-10T05:31:53.258953+0000","last_peered":"2024-05-10T05:31:53.258953+0000","last_clean":"2024-05-10T05:31:53.258953+0000","last_became_active":"2024-05-10T05:31:51.574256+0000","last_became_peered":"2024-05-10T05:31:51.574256+0000","last_unstale":"2024-05-10T05:31:53.258953+0000","last_undegraded":"2024-05-10T05:31:53.258953+0000","last_fullsized":"2024-05-10T05:31:53.258953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:59.174727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258435+0000","last_change":"2024-05-10T05:31:51.582336+0000","last_active":"2024-05-10T05:31:53.258435+0000","last_peered":"2024-05-10T05:31:53.258435+0000","last_clean":"2024-05-10T05:31:53.258435+0000","last_became_active":"2024-05-10T05:31:51.581985+0000","last_became_peered":"2024-05-10T05:31:51.581985+0000","last_unstale":"2024-05-10T05:31:53.258435+0000","last_undegraded":"2024-05-10T05:31:53.258435+0000","last_fullsized":"2024-05-10T05:31:53.258435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:51.601645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534190+0000","last_change":"2024-05-10T05:31:51.568919+0000","last_active":"2024-05-10T05:31:52.534190+0000","last_peered":"2024-05-10T05:31:52.534190+0000","last_clean":"2024-05-10T05:31:52.534190+0000","last_became_active":"2024-05-10T05:31:51.568582+0000","last_became_peered":"2024-05-10T05:31:51.568582+0000","last_unstale":"2024-05-10T05:31:52.534190+0000","last_undegraded":"2024-05-10T05:31:52.534190+0000","last_fullsized":"2024-05-10T05:31:52.534190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:08.377261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237990+0000","last_change":"2024-05-10T05:31:51.579025+0000","last_active":"2024-05-10T05:31:53.237990+0000","last_peered":"2024-05-10T05:31:53.237990+0000","last_clean":"2024-05-10T05:31:53.237990+0000","last_became_active":"2024-05-10T05:31:51.578704+0000","last_became_peered":"2024-05-10T05:31:51.578704+0000","last_unstale":"2024-05-10T05:31:53.237990+0000","last_undegraded":"2024-05-10T05:31:53.237990+0000","last_fullsized":"2024-05-10T05:31:53.237990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:16.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.012716+0000","last_change":"2024-05-10T05:31:51.573193+0000","last_active":"2024-05-10T05:31:53.012716+0000","last_peered":"2024-05-10T05:31:53.012716+0000","last_clean":"2024-05-10T05:31:53.012716+0000","last_became_active":"2024-05-10T05:31:51.572874+0000","last_became_peered":"2024-05-10T05:31:51.572874+0000","last_unstale":"2024-05-10T05:31:53.012716+0000","last_undegraded":"2024-05-10T05:31:53.012716+0000","last_fullsized":"2024-05-10T05:31:53.012716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:09.867661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.651422+0000","last_change":"2024-05-10T05:31:51.581107+0000","last_active":"2024-05-10T05:31:52.651422+0000","last_peered":"2024-05-10T05:31:52.651422+0000","last_clean":"2024-05-10T05:31:52.651422+0000","last_became_active":"2024-05-10T05:31:51.580765+0000","last_became_peered":"2024-05-10T05:31:51.580765+0000","last_unstale":"2024-05-10T05:31:52.651422+0000","last_undegraded":"2024-05-10T05:31:52.651422+0000","last_fullsized":"2024-05-10T05:31:52.651422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:04.164486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'11","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.042119+0000","last_change":"2024-05-10T05:31:50.986534+0000","last_active":"2024-05-10T05:31:57.042119+0000","last_peered":"2024-05-10T05:31:57.042119+0000","last_clean":"2024-05-10T05:31:57.042119+0000","last_became_active":"2024-05-10T05:31:50.983493+0000","last_became_peered":"2024-05-10T05:31:50.983493+0000","last_unstale":"2024-05-10T05:31:57.042119+0000","last_undegraded":"2024-05-10T05:31:57.042119+0000","last_fullsized":"2024-05-10T05:31:57.042119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:18:56.926317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259563+0000","last_change":"2024-05-10T05:31:51.587121+0000","last_active":"2024-05-10T05:31:53.259563+0000","last_peered":"2024-05-10T05:31:53.259563+0000","last_clean":"2024-05-10T05:31:53.259563+0000","last_became_active":"2024-05-10T05:31:51.586641+0000","last_became_peered":"2024-05-10T05:31:51.586641+0000","last_unstale":"2024-05-10T05:31:53.259563+0000","last_undegraded":"2024-05-10T05:31:53.259563+0000","last_fullsized":"2024-05-10T05:31:53.259563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:52.654918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.348579+0000","last_change":"2024-05-10T05:31:50.999755+0000","last_active":"2024-05-10T05:31:53.348579+0000","last_peered":"2024-05-10T05:31:53.348579+0000","last_clean":"2024-05-10T05:31:53.348579+0000","last_became_active":"2024-05-10T05:31:50.999336+0000","last_became_peered":"2024-05-10T05:31:50.999336+0000","last_unstale":"2024-05-10T05:31:53.348579+0000","last_undegraded":"2024-05-10T05:31:53.348579+0000","last_fullsized":"2024-05-10T05:31:53.348579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:43.755130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259252+0000","last_change":"2024-05-10T05:31:51.576938+0000","last_active":"2024-05-10T05:31:53.259252+0000","last_peered":"2024-05-10T05:31:53.259252+0000","last_clean":"2024-05-10T05:31:53.259252+0000","last_became_active":"2024-05-10T05:31:51.576517+0000","last_became_peered":"2024-05-10T05:31:51.576517+0000","last_unstale":"2024-05-10T05:31:53.259252+0000","last_undegraded":"2024-05-10T05:31:53.259252+0000","last_fullsized":"2024-05-10T05:31:53.259252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:52:38.709063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'11","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.050082+0000","last_change":"2024-05-10T05:31:50.992925+0000","last_active":"2024-05-10T05:31:58.050082+0000","last_peered":"2024-05-10T05:31:58.050082+0000","last_clean":"2024-05-10T05:31:58.050082+0000","last_became_active":"2024-05-10T05:31:50.992574+0000","last_became_peered":"2024-05-10T05:31:50.992574+0000","last_unstale":"2024-05-10T05:31:58.050082+0000","last_undegraded":"2024-05-10T05:31:58.050082+0000","last_fullsized":"2024-05-10T05:31:58.050082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:14.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534257+0000","last_change":"2024-05-10T05:31:51.580145+0000","last_active":"2024-05-10T05:31:52.534257+0000","last_peered":"2024-05-10T05:31:52.534257+0000","last_clean":"2024-05-10T05:31:52.534257+0000","last_became_active":"2024-05-10T05:31:51.576701+0000","last_became_peered":"2024-05-10T05:31:51.576701+0000","last_unstale":"2024-05-10T05:31:52.534257+0000","last_undegraded":"2024-05-10T05:31:52.534257+0000","last_fullsized":"2024-05-10T05:31:52.534257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:36:21.075571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.049013+0000","last_change":"2024-05-10T05:31:50.987863+0000","last_active":"2024-05-10T05:31:58.049013+0000","last_peered":"2024-05-10T05:31:58.049013+0000","last_clean":"2024-05-10T05:31:58.049013+0000","last_became_active":"2024-05-10T05:31:50.987463+0000","last_became_peered":"2024-05-10T05:31:50.987463+0000","last_unstale":"2024-05-10T05:31:58.049013+0000","last_undegraded":"2024-05-10T05:31:58.049013+0000","last_fullsized":"2024-05-10T05:31:58.049013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:12:49.542288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560817+0000","last_change":"2024-05-10T05:31:51.585289+0000","last_active":"2024-05-10T05:31:52.560817+0000","last_peered":"2024-05-10T05:31:52.560817+0000","last_clean":"2024-05-10T05:31:52.560817+0000","last_became_active":"2024-05-10T05:31:51.584059+0000","last_became_peered":"2024-05-10T05:31:51.584059+0000","last_unstale":"2024-05-10T05:31:52.560817+0000","last_undegraded":"2024-05-10T05:31:52.560817+0000","last_fullsized":"2024-05-10T05:31:52.560817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:05:02.271354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.496249+0000","last_change":"2024-05-10T05:31:50.985879+0000","last_active":"2024-05-10T05:31:58.496249+0000","last_peered":"2024-05-10T05:31:58.496249+0000","last_clean":"2024-05-10T05:31:58.496249+0000","last_became_active":"2024-05-10T05:31:50.985476+0000","last_became_peered":"2024-05-10T05:31:50.985476+0000","last_unstale":"2024-05-10T05:31:58.496249+0000","last_undegraded":"2024-05-10T05:31:58.496249+0000","last_fullsized":"2024-05-10T05:31:58.496249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:11:27.737972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258660+0000","last_change":"2024-05-10T05:31:51.570985+0000","last_active":"2024-05-10T05:31:53.258660+0000","last_peered":"2024-05-10T05:31:53.258660+0000","last_clean":"2024-05-10T05:31:53.258660+0000","last_became_active":"2024-05-10T05:31:51.570324+0000","last_became_peered":"2024-05-10T05:31:51.570324+0000","last_unstale":"2024-05-10T05:31:53.258660+0000","last_undegraded":"2024-05-10T05:31:53.258660+0000","last_fullsized":"2024-05-10T05:31:53.258660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:26.947574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.583384+0000","last_change":"2024-05-10T05:31:50.988483+0000","last_active":"2024-05-10T05:31:51.583384+0000","last_peered":"2024-05-10T05:31:51.583384+0000","last_clean":"2024-05-10T05:31:51.583384+0000","last_became_active":"2024-05-10T05:31:50.987897+0000","last_became_peered":"2024-05-10T05:31:50.987897+0000","last_unstale":"2024-05-10T05:31:51.583384+0000","last_undegraded":"2024-05-10T05:31:51.583384+0000","last_fullsized":"2024-05-10T05:31:51.583384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:30:23.474637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560935+0000","last_change":"2024-05-10T05:31:51.585717+0000","last_active":"2024-05-10T05:31:52.560935+0000","last_peered":"2024-05-10T05:31:52.560935+0000","last_clean":"2024-05-10T05:31:52.560935+0000","last_became_active":"2024-05-10T05:31:51.585250+0000","last_became_peered":"2024-05-10T05:31:51.585250+0000","last_unstale":"2024-05-10T05:31:52.560935+0000","last_undegraded":"2024-05-10T05:31:52.560935+0000","last_fullsized":"2024-05-10T05:31:52.560935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:05:10.692140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:56.027722+0000","last_change":"2024-05-10T05:31:50.981877+0000","last_active":"2024-05-10T05:31:56.027722+0000","last_peered":"2024-05-10T05:31:56.027722+0000","last_clean":"2024-05-10T05:31:56.027722+0000","last_became_active":"2024-05-10T05:31:50.981527+0000","last_became_peered":"2024-05-10T05:31:50.981527+0000","last_unstale":"2024-05-10T05:31:56.027722+0000","last_undegraded":"2024-05-10T05:31:56.027722+0000","last_fullsized":"2024-05-10T05:31:56.027722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:40.735304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258317+0000","last_change":"2024-05-10T05:31:51.581027+0000","last_active":"2024-05-10T05:31:53.258317+0000","last_peered":"2024-05-10T05:31:53.258317+0000","last_clean":"2024-05-10T05:31:53.258317+0000","last_became_active":"2024-05-10T05:31:51.580620+0000","last_became_peered":"2024-05-10T05:31:51.580620+0000","last_unstale":"2024-05-10T05:31:53.258317+0000","last_undegraded":"2024-05-10T05:31:53.258317+0000","last_fullsized":"2024-05-10T05:31:53.258317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:30:15.378179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.048077+0000","last_change":"2024-05-10T05:31:50.987633+0000","last_active":"2024-05-10T05:31:58.048077+0000","last_peered":"2024-05-10T05:31:58.048077+0000","last_clean":"2024-05-10T05:31:58.048077+0000","last_became_active":"2024-05-10T05:31:50.986750+0000","last_became_peered":"2024-05-10T05:31:50.986750+0000","last_unstale":"2024-05-10T05:31:58.048077+0000","last_undegraded":"2024-05-10T05:31:58.048077+0000","last_fullsized":"2024-05-10T05:31:58.048077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:17:47.705415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.012674+0000","last_change":"2024-05-10T05:31:51.589484+0000","last_active":"2024-05-10T05:31:53.012674+0000","last_peered":"2024-05-10T05:31:53.012674+0000","last_clean":"2024-05-10T05:31:53.012674+0000","last_became_active":"2024-05-10T05:31:51.588992+0000","last_became_peered":"2024-05-10T05:31:51.588992+0000","last_unstale":"2024-05-10T05:31:53.012674+0000","last_undegraded":"2024-05-10T05:31:53.012674+0000","last_fullsized":"2024-05-10T05:31:53.012674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:32.360333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.018747+0000","last_change":"2024-05-10T05:31:50.988945+0000","last_active":"2024-05-10T05:31:55.018747+0000","last_peered":"2024-05-10T05:31:55.018747+0000","last_clean":"2024-05-10T05:31:55.018747+0000","last_became_active":"2024-05-10T05:31:50.988566+0000","last_became_peered":"2024-05-10T05:31:50.988566+0000","last_unstale":"2024-05-10T05:31:55.018747+0000","last_undegraded":"2024-05-10T05:31:55.018747+0000","last_fullsized":"2024-05-10T05:31:55.018747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:06:43.225346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534284+0000","last_change":"2024-05-10T05:31:51.577949+0000","last_active":"2024-05-10T05:31:52.534284+0000","last_peered":"2024-05-10T05:31:52.534284+0000","last_clean":"2024-05-10T05:31:52.534284+0000","last_became_active":"2024-05-10T05:31:51.577011+0000","last_became_peered":"2024-05-10T05:31:51.577011+0000","last_unstale":"2024-05-10T05:31:52.534284+0000","last_undegraded":"2024-05-10T05:31:52.534284+0000","last_fullsized":"2024-05-10T05:31:52.534284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:09.988700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.016535+0000","last_change":"2024-05-10T05:31:50.984888+0000","last_active":"2024-05-10T05:31:55.016535+0000","last_peered":"2024-05-10T05:31:55.016535+0000","last_clean":"2024-05-10T05:31:55.016535+0000","last_became_active":"2024-05-10T05:31:50.984267+0000","last_became_peered":"2024-05-10T05:31:50.984267+0000","last_unstale":"2024-05-10T05:31:55.016535+0000","last_undegraded":"2024-05-10T05:31:55.016535+0000","last_fullsized":"2024-05-10T05:31:55.016535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:44.797469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259203+0000","last_change":"2024-05-10T05:31:51.576863+0000","last_active":"2024-05-10T05:31:53.259203+0000","last_peered":"2024-05-10T05:31:53.259203+0000","last_clean":"2024-05-10T05:31:53.259203+0000","last_became_active":"2024-05-10T05:31:51.576468+0000","last_became_peered":"2024-05-10T05:31:51.576468+0000","last_unstale":"2024-05-10T05:31:53.259203+0000","last_undegraded":"2024-05-10T05:31:53.259203+0000","last_fullsized":"2024-05-10T05:31:53.259203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:20.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.042227+0000","last_change":"2024-05-10T05:31:51.001566+0000","last_active":"2024-05-10T05:31:57.042227+0000","last_peered":"2024-05-10T05:31:57.042227+0000","last_clean":"2024-05-10T05:31:57.042227+0000","last_became_active":"2024-05-10T05:31:51.001203+0000","last_became_peered":"2024-05-10T05:31:51.001203+0000","last_unstale":"2024-05-10T05:31:57.042227+0000","last_undegraded":"2024-05-10T05:31:57.042227+0000","last_fullsized":"2024-05-10T05:31:57.042227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:17:45.465039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534270+0000","last_change":"2024-05-10T05:31:51.579114+0000","last_active":"2024-05-10T05:31:52.534270+0000","last_peered":"2024-05-10T05:31:52.534270+0000","last_clean":"2024-05-10T05:31:52.534270+0000","last_became_active":"2024-05-10T05:31:51.577507+0000","last_became_peered":"2024-05-10T05:31:51.577507+0000","last_unstale":"2024-05-10T05:31:52.534270+0000","last_undegraded":"2024-05-10T05:31:52.534270+0000","last_fullsized":"2024-05-10T05:31:52.534270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:43.777983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.017780+0000","last_change":"2024-05-10T05:31:50.983558+0000","last_active":"2024-05-10T05:31:55.017780+0000","last_peered":"2024-05-10T05:31:55.017780+0000","last_clean":"2024-05-10T05:31:55.017780+0000","last_became_active":"2024-05-10T05:31:50.983087+0000","last_became_peered":"2024-05-10T05:31:50.983087+0000","last_unstale":"2024-05-10T05:31:55.017780+0000","last_undegraded":"2024-05-10T05:31:55.017780+0000","last_fullsized":"2024-05-10T05:31:55.017780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:08.753898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258772+0000","last_change":"2024-05-10T05:31:51.574024+0000","last_active":"2024-05-10T05:31:53.258772+0000","last_peered":"2024-05-10T05:31:53.258772+0000","last_clean":"2024-05-10T05:31:53.258772+0000","last_became_active":"2024-05-10T05:31:51.573703+0000","last_became_peered":"2024-05-10T05:31:51.573703+0000","last_unstale":"2024-05-10T05:31:53.258772+0000","last_undegraded":"2024-05-10T05:31:53.258772+0000","last_fullsized":"2024-05-10T05:31:53.258772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:55:23.435991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.492258+0000","last_change":"2024-05-10T05:31:50.986795+0000","last_active":"2024-05-10T05:31:58.492258+0000","last_peered":"2024-05-10T05:31:58.492258+0000","last_clean":"2024-05-10T05:31:58.492258+0000","last_became_active":"2024-05-10T05:31:50.986428+0000","last_became_peered":"2024-05-10T05:31:50.986428+0000","last_unstale":"2024-05-10T05:31:58.492258+0000","last_undegraded":"2024-05-10T05:31:58.492258+0000","last_fullsized":"2024-05-10T05:31:58.492258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:11.039632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560044+0000","last_change":"2024-05-10T05:31:51.585595+0000","last_active":"2024-05-10T05:31:52.560044+0000","last_peered":"2024-05-10T05:31:52.560044+0000","last_clean":"2024-05-10T05:31:52.560044+0000","last_became_active":"2024-05-10T05:31:51.585091+0000","last_became_peered":"2024-05-10T05:31:51.585091+0000","last_unstale":"2024-05-10T05:31:52.560044+0000","last_undegraded":"2024-05-10T05:31:52.560044+0000","last_fullsized":"2024-05-10T05:31:52.560044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:10:10.489430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.049211+0000","last_change":"2024-05-10T05:31:50.984408+0000","last_active":"2024-05-10T05:31:58.049211+0000","last_peered":"2024-05-10T05:31:58.049211+0000","last_clean":"2024-05-10T05:31:58.049211+0000","last_became_active":"2024-05-10T05:31:50.983098+0000","last_became_peered":"2024-05-10T05:31:50.983098+0000","last_unstale":"2024-05-10T05:31:58.049211+0000","last_undegraded":"2024-05-10T05:31:58.049211+0000","last_fullsized":"2024-05-10T05:31:58.049211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:31.399649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.579782+0000","last_change":"2024-05-10T05:31:51.576160+0000","last_active":"2024-05-10T05:31:51.579782+0000","last_peered":"2024-05-10T05:31:51.579782+0000","last_clean":"2024-05-10T05:31:51.579782+0000","last_became_active":"2024-05-10T05:31:51.575806+0000","last_became_peered":"2024-05-10T05:31:51.575806+0000","last_unstale":"2024-05-10T05:31:51.579782+0000","last_undegraded":"2024-05-10T05:31:51.579782+0000","last_fullsized":"2024-05-10T05:31:51.579782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:53:25.660988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.040605+0000","last_change":"2024-05-10T05:31:50.982752+0000","last_active":"2024-05-10T05:31:57.040605+0000","last_peered":"2024-05-10T05:31:57.040605+0000","last_clean":"2024-05-10T05:31:57.040605+0000","last_became_active":"2024-05-10T05:31:50.982299+0000","last_became_peered":"2024-05-10T05:31:50.982299+0000","last_unstale":"2024-05-10T05:31:57.040605+0000","last_undegraded":"2024-05-10T05:31:57.040605+0000","last_fullsized":"2024-05-10T05:31:57.040605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:52:40.206967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.578968+0000","last_change":"2024-05-10T05:31:51.577081+0000","last_active":"2024-05-10T05:31:51.578968+0000","last_peered":"2024-05-10T05:31:51.578968+0000","last_clean":"2024-05-10T05:31:51.578968+0000","last_became_active":"2024-05-10T05:31:51.576729+0000","last_became_peered":"2024-05-10T05:31:51.576729+0000","last_unstale":"2024-05-10T05:31:51.578968+0000","last_undegraded":"2024-05-10T05:31:51.578968+0000","last_fullsized":"2024-05-10T05:31:51.578968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:14.072427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.048975+0000","last_change":"2024-05-10T05:31:50.991868+0000","last_active":"2024-05-10T05:31:58.048975+0000","last_peered":"2024-05-10T05:31:58.048975+0000","last_clean":"2024-05-10T05:31:58.048975+0000","last_became_active":"2024-05-10T05:31:50.991471+0000","last_became_peered":"2024-05-10T05:31:50.991471+0000","last_unstale":"2024-05-10T05:31:58.048975+0000","last_undegraded":"2024-05-10T05:31:58.048975+0000","last_fullsized":"2024-05-10T05:31:58.048975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:22.346376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560537+0000","last_change":"2024-05-10T05:31:51.584726+0000","last_active":"2024-05-10T05:31:52.560537+0000","last_peered":"2024-05-10T05:31:52.560537+0000","last_clean":"2024-05-10T05:31:52.560537+0000","last_became_active":"2024-05-10T05:31:51.584327+0000","last_became_peered":"2024-05-10T05:31:51.584327+0000","last_unstale":"2024-05-10T05:31:52.560537+0000","last_undegraded":"2024-05-10T05:31:52.560537+0000","last_fullsized":"2024-05-10T05:31:52.560537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:56:43.561717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.492270+0000","last_change":"2024-05-10T05:31:50.989071+0000","last_active":"2024-05-10T05:31:58.492270+0000","last_peered":"2024-05-10T05:31:58.492270+0000","last_clean":"2024-05-10T05:31:58.492270+0000","last_became_active":"2024-05-10T05:31:50.988044+0000","last_became_peered":"2024-05-10T05:31:50.988044+0000","last_unstale":"2024-05-10T05:31:58.492270+0000","last_undegraded":"2024-05-10T05:31:58.492270+0000","last_fullsized":"2024-05-10T05:31:58.492270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:24.122564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237745+0000","last_change":"2024-05-10T05:31:51.576793+0000","last_active":"2024-05-10T05:31:53.237745+0000","last_peered":"2024-05-10T05:31:53.237745+0000","last_clean":"2024-05-10T05:31:53.237745+0000","last_became_active":"2024-05-10T05:31:51.576333+0000","last_became_peered":"2024-05-10T05:31:51.576333+0000","last_unstale":"2024-05-10T05:31:53.237745+0000","last_undegraded":"2024-05-10T05:31:53.237745+0000","last_fullsized":"2024-05-10T05:31:53.237745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:42.418736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258788+0000","last_change":"2024-05-10T05:31:50.997233+0000","last_active":"2024-05-10T05:31:53.258788+0000","last_peered":"2024-05-10T05:31:53.258788+0000","last_clean":"2024-05-10T05:31:53.258788+0000","last_became_active":"2024-05-10T05:31:50.996857+0000","last_became_peered":"2024-05-10T05:31:50.996857+0000","last_unstale":"2024-05-10T05:31:53.258788+0000","last_undegraded":"2024-05-10T05:31:53.258788+0000","last_fullsized":"2024-05-10T05:31:53.258788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:35.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.586288+0000","last_change":"2024-05-10T05:31:51.584371+0000","last_active":"2024-05-10T05:31:51.586288+0000","last_peered":"2024-05-10T05:31:51.586288+0000","last_clean":"2024-05-10T05:31:51.586288+0000","last_became_active":"2024-05-10T05:31:51.583966+0000","last_became_peered":"2024-05-10T05:31:51.583966+0000","last_unstale":"2024-05-10T05:31:51.586288+0000","last_undegraded":"2024-05-10T05:31:51.586288+0000","last_fullsized":"2024-05-10T05:31:51.586288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:22:13.036852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259956+0000","last_change":"2024-05-10T05:31:50.991113+0000","last_active":"2024-05-10T05:31:53.259956+0000","last_peered":"2024-05-10T05:31:53.259956+0000","last_clean":"2024-05-10T05:31:53.259956+0000","last_became_active":"2024-05-10T05:31:50.990745+0000","last_became_peered":"2024-05-10T05:31:50.990745+0000","last_unstale":"2024-05-10T05:31:53.259956+0000","last_undegraded":"2024-05-10T05:31:53.259956+0000","last_fullsized":"2024-05-10T05:31:53.259956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:49:56.586231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.584042+0000","last_change":"2024-05-10T05:31:51.582057+0000","last_active":"2024-05-10T05:31:51.584042+0000","last_peered":"2024-05-10T05:31:51.584042+0000","last_clean":"2024-05-10T05:31:51.584042+0000","last_became_active":"2024-05-10T05:31:51.581672+0000","last_became_peered":"2024-05-10T05:31:51.581672+0000","last_unstale":"2024-05-10T05:31:51.584042+0000","last_undegraded":"2024-05-10T05:31:51.584042+0000","last_fullsized":"2024-05-10T05:31:51.584042+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:45:09.122256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560845+0000","last_change":"2024-05-10T05:31:51.585659+0000","last_active":"2024-05-10T05:31:52.560845+0000","last_peered":"2024-05-10T05:31:52.560845+0000","last_clean":"2024-05-10T05:31:52.560845+0000","last_became_active":"2024-05-10T05:31:51.584540+0000","last_became_peered":"2024-05-10T05:31:51.584540+0000","last_unstale":"2024-05-10T05:31:52.560845+0000","last_undegraded":"2024-05-10T05:31:52.560845+0000","last_fullsized":"2024-05-10T05:31:52.560845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:20.195369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:56.027631+0000","last_change":"2024-05-10T05:31:50.993547+0000","last_active":"2024-05-10T05:31:56.027631+0000","last_peered":"2024-05-10T05:31:56.027631+0000","last_clean":"2024-05-10T05:31:56.027631+0000","last_became_active":"2024-05-10T05:31:50.993190+0000","last_became_peered":"2024-05-10T05:31:50.993190+0000","last_unstale":"2024-05-10T05:31:56.027631+0000","last_undegraded":"2024-05-10T05:31:56.027631+0000","last_fullsized":"2024-05-10T05:31:56.027631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:06.159572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259316+0000","last_change":"2024-05-10T05:31:51.587678+0000","last_active":"2024-05-10T05:31:53.259316+0000","last_peered":"2024-05-10T05:31:53.259316+0000","last_clean":"2024-05-10T05:31:53.259316+0000","last_became_active":"2024-05-10T05:31:51.586999+0000","last_became_peered":"2024-05-10T05:31:51.586999+0000","last_unstale":"2024-05-10T05:31:53.259316+0000","last_undegraded":"2024-05-10T05:31:53.259316+0000","last_fullsized":"2024-05-10T05:31:53.259316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:18:00.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.042889+0000","last_change":"2024-05-10T05:31:50.979131+0000","last_active":"2024-05-10T05:31:57.042889+0000","last_peered":"2024-05-10T05:31:57.042889+0000","last_clean":"2024-05-10T05:31:57.042889+0000","last_became_active":"2024-05-10T05:31:50.978715+0000","last_became_peered":"2024-05-10T05:31:50.978715+0000","last_unstale":"2024-05-10T05:31:57.042889+0000","last_undegraded":"2024-05-10T05:31:57.042889+0000","last_fullsized":"2024-05-10T05:31:57.042889+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:29:11.953963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.562030+0000","last_change":"2024-05-10T05:30:59.356688+0000","last_active":"2024-05-10T05:31:51.562030+0000","last_peered":"2024-05-10T05:31:51.562030+0000","last_clean":"2024-05-10T05:31:51.562030+0000","last_became_active":"2024-05-10T05:30:59.356313+0000","last_became_peered":"2024-05-10T05:30:59.356313+0000","last_unstale":"2024-05-10T05:31:51.562030+0000","last_undegraded":"2024-05-10T05:31:51.562030+0000","last_fullsized":"2024-05-10T05:31:51.562030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_clean_scrub_stamp":"2024-05-10T05:28:15.503591+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:43.011495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.559640+0000","last_change":"2024-05-10T05:31:51.580259+0000","last_active":"2024-05-10T05:31:52.559640+0000","last_peered":"2024-05-10T05:31:52.559640+0000","last_clean":"2024-05-10T05:31:52.559640+0000","last_became_active":"2024-05-10T05:31:51.578650+0000","last_became_peered":"2024-05-10T05:31:51.578650+0000","last_unstale":"2024-05-10T05:31:52.559640+0000","last_undegraded":"2024-05-10T05:31:52.559640+0000","last_fullsized":"2024-05-10T05:31:52.559640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:48.630183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:56.022783+0000","last_change":"2024-05-10T05:31:50.987708+0000","last_active":"2024-05-10T05:31:56.022783+0000","last_peered":"2024-05-10T05:31:56.022783+0000","last_clean":"2024-05-10T05:31:56.022783+0000","last_became_active":"2024-05-10T05:31:50.987347+0000","last_became_peered":"2024-05-10T05:31:50.987347+0000","last_unstale":"2024-05-10T05:31:56.022783+0000","last_undegraded":"2024-05-10T05:31:56.022783+0000","last_fullsized":"2024-05-10T05:31:56.022783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:16:29.388690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.049419+0000","last_change":"2024-05-10T05:31:50.999338+0000","last_active":"2024-05-10T05:31:58.049419+0000","last_peered":"2024-05-10T05:31:58.049419+0000","last_clean":"2024-05-10T05:31:58.049419+0000","last_became_active":"2024-05-10T05:31:50.999026+0000","last_became_peered":"2024-05-10T05:31:50.999026+0000","last_unstale":"2024-05-10T05:31:58.049419+0000","last_undegraded":"2024-05-10T05:31:58.049419+0000","last_fullsized":"2024-05-10T05:31:58.049419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:31.443409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560043+0000","last_change":"2024-05-10T05:31:51.574201+0000","last_active":"2024-05-10T05:31:52.560043+0000","last_peered":"2024-05-10T05:31:52.560043+0000","last_clean":"2024-05-10T05:31:52.560043+0000","last_became_active":"2024-05-10T05:31:51.571180+0000","last_became_peered":"2024-05-10T05:31:51.571180+0000","last_unstale":"2024-05-10T05:31:52.560043+0000","last_undegraded":"2024-05-10T05:31:52.560043+0000","last_fullsized":"2024-05-10T05:31:52.560043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:33:16.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.043591+0000","last_change":"2024-05-10T05:31:50.985164+0000","last_active":"2024-05-10T05:31:57.043591+0000","last_peered":"2024-05-10T05:31:57.043591+0000","last_clean":"2024-05-10T05:31:57.043591+0000","last_became_active":"2024-05-10T05:31:50.984760+0000","last_became_peered":"2024-05-10T05:31:50.984760+0000","last_unstale":"2024-05-10T05:31:57.043591+0000","last_undegraded":"2024-05-10T05:31:57.043591+0000","last_fullsized":"2024-05-10T05:31:57.043591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:19.827407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259609+0000","last_change":"2024-05-10T05:31:51.577445+0000","last_active":"2024-05-10T05:31:53.259609+0000","last_peered":"2024-05-10T05:31:53.259609+0000","last_clean":"2024-05-10T05:31:53.259609+0000","last_became_active":"2024-05-10T05:31:51.576671+0000","last_became_peered":"2024-05-10T05:31:51.576671+0000","last_unstale":"2024-05-10T05:31:53.259609+0000","last_undegraded":"2024-05-10T05:31:53.259609+0000","last_fullsized":"2024-05-10T05:31:53.259609+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:38.774538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.019858+0000","last_change":"2024-05-10T05:31:50.981495+0000","last_active":"2024-05-10T05:31:55.019858+0000","last_peered":"2024-05-10T05:31:55.019858+0000","last_clean":"2024-05-10T05:31:55.019858+0000","last_became_active":"2024-05-10T05:31:50.981127+0000","last_became_peered":"2024-05-10T05:31:50.981127+0000","last_unstale":"2024-05-10T05:31:55.019858+0000","last_undegraded":"2024-05-10T05:31:55.019858+0000","last_fullsized":"2024-05-10T05:31:55.019858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:22:32.257014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258881+0000","last_change":"2024-05-10T05:31:51.585926+0000","last_active":"2024-05-10T05:31:53.258881+0000","last_peered":"2024-05-10T05:31:53.258881+0000","last_clean":"2024-05-10T05:31:53.258881+0000","last_became_active":"2024-05-10T05:31:51.585576+0000","last_became_peered":"2024-05-10T05:31:51.585576+0000","last_unstale":"2024-05-10T05:31:53.258881+0000","last_undegraded":"2024-05-10T05:31:53.258881+0000","last_fullsized":"2024-05-10T05:31:53.258881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:22.344180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.016517+0000","last_change":"2024-05-10T05:31:50.989261+0000","last_active":"2024-05-10T05:31:55.016517+0000","last_peered":"2024-05-10T05:31:55.016517+0000","last_clean":"2024-05-10T05:31:55.016517+0000","last_became_active":"2024-05-10T05:31:50.988885+0000","last_became_peered":"2024-05-10T05:31:50.988885+0000","last_unstale":"2024-05-10T05:31:55.016517+0000","last_undegraded":"2024-05-10T05:31:55.016517+0000","last_fullsized":"2024-05-10T05:31:55.016517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:48:59.554692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258573+0000","last_change":"2024-05-10T05:31:51.573279+0000","last_active":"2024-05-10T05:31:53.258573+0000","last_peered":"2024-05-10T05:31:53.258573+0000","last_clean":"2024-05-10T05:31:53.258573+0000","last_became_active":"2024-05-10T05:31:51.572916+0000","last_became_peered":"2024-05-10T05:31:51.572916+0000","last_unstale":"2024-05-10T05:31:53.258573+0000","last_undegraded":"2024-05-10T05:31:53.258573+0000","last_fullsized":"2024-05-10T05:31:53.258573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:32:39.802463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.048490+0000","last_change":"2024-05-10T05:31:50.983771+0000","last_active":"2024-05-10T05:31:58.048490+0000","last_peered":"2024-05-10T05:31:58.048490+0000","last_clean":"2024-05-10T05:31:58.048490+0000","last_became_active":"2024-05-10T05:31:50.981849+0000","last_became_peered":"2024-05-10T05:31:50.981849+0000","last_unstale":"2024-05-10T05:31:58.048490+0000","last_undegraded":"2024-05-10T05:31:58.048490+0000","last_fullsized":"2024-05-10T05:31:58.048490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:15.990869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237478+0000","last_change":"2024-05-10T05:31:51.568005+0000","last_active":"2024-05-10T05:31:53.237478+0000","last_peered":"2024-05-10T05:31:53.237478+0000","last_clean":"2024-05-10T05:31:53.237478+0000","last_became_active":"2024-05-10T05:31:51.567347+0000","last_became_peered":"2024-05-10T05:31:51.567347+0000","last_unstale":"2024-05-10T05:31:53.237478+0000","last_undegraded":"2024-05-10T05:31:53.237478+0000","last_fullsized":"2024-05-10T05:31:53.237478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:29.075816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.043607+0000","last_change":"2024-05-10T05:31:50.990237+0000","last_active":"2024-05-10T05:31:57.043607+0000","last_peered":"2024-05-10T05:31:57.043607+0000","last_clean":"2024-05-10T05:31:57.043607+0000","last_became_active":"2024-05-10T05:31:50.989897+0000","last_became_peered":"2024-05-10T05:31:50.989897+0000","last_unstale":"2024-05-10T05:31:57.043607+0000","last_undegraded":"2024-05-10T05:31:57.043607+0000","last_fullsized":"2024-05-10T05:31:57.043607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:30.593071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259318+0000","last_change":"2024-05-10T05:31:51.588255+0000","last_active":"2024-05-10T05:31:53.259318+0000","last_peered":"2024-05-10T05:31:53.259318+0000","last_clean":"2024-05-10T05:31:53.259318+0000","last_became_active":"2024-05-10T05:31:51.587912+0000","last_became_peered":"2024-05-10T05:31:51.587912+0000","last_unstale":"2024-05-10T05:31:53.259318+0000","last_undegraded":"2024-05-10T05:31:53.259318+0000","last_fullsized":"2024-05-10T05:31:53.259318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:18.515739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.017748+0000","last_change":"2024-05-10T05:31:50.992392+0000","last_active":"2024-05-10T05:31:55.017748+0000","last_peered":"2024-05-10T05:31:55.017748+0000","last_clean":"2024-05-10T05:31:55.017748+0000","last_became_active":"2024-05-10T05:31:50.992119+0000","last_became_peered":"2024-05-10T05:31:50.992119+0000","last_unstale":"2024-05-10T05:31:55.017748+0000","last_undegraded":"2024-05-10T05:31:55.017748+0000","last_fullsized":"2024-05-10T05:31:55.017748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:11.515013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.568789+0000","last_change":"2024-05-10T05:31:51.567780+0000","last_active":"2024-05-10T05:31:51.568789+0000","last_peered":"2024-05-10T05:31:51.568789+0000","last_clean":"2024-05-10T05:31:51.568789+0000","last_became_active":"2024-05-10T05:31:51.567455+0000","last_became_peered":"2024-05-10T05:31:51.567455+0000","last_unstale":"2024-05-10T05:31:51.568789+0000","last_undegraded":"2024-05-10T05:31:51.568789+0000","last_fullsized":"2024-05-10T05:31:51.568789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:47.108131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.496788+0000","last_change":"2024-05-10T05:31:50.997374+0000","last_active":"2024-05-10T05:31:58.496788+0000","last_peered":"2024-05-10T05:31:58.496788+0000","last_clean":"2024-05-10T05:31:58.496788+0000","last_became_active":"2024-05-10T05:31:50.997069+0000","last_became_peered":"2024-05-10T05:31:50.997069+0000","last_unstale":"2024-05-10T05:31:58.496788+0000","last_undegraded":"2024-05-10T05:31:58.496788+0000","last_fullsized":"2024-05-10T05:31:58.496788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:25:49.140006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.579011+0000","last_change":"2024-05-10T05:31:51.576345+0000","last_active":"2024-05-10T05:31:51.579011+0000","last_peered":"2024-05-10T05:31:51.579011+0000","last_clean":"2024-05-10T05:31:51.579011+0000","last_became_active":"2024-05-10T05:31:51.575704+0000","last_became_peered":"2024-05-10T05:31:51.575704+0000","last_unstale":"2024-05-10T05:31:51.579011+0000","last_undegraded":"2024-05-10T05:31:51.579011+0000","last_fullsized":"2024-05-10T05:31:51.579011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:15.146591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.050034+0000","last_change":"2024-05-10T05:31:50.984108+0000","last_active":"2024-05-10T05:31:58.050034+0000","last_peered":"2024-05-10T05:31:58.050034+0000","last_clean":"2024-05-10T05:31:58.050034+0000","last_became_active":"2024-05-10T05:31:50.983728+0000","last_became_peered":"2024-05-10T05:31:50.983728+0000","last_unstale":"2024-05-10T05:31:58.050034+0000","last_undegraded":"2024-05-10T05:31:58.050034+0000","last_fullsized":"2024-05-10T05:31:58.050034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:23:20.096286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.575183+0000","last_change":"2024-05-10T05:31:51.572075+0000","last_active":"2024-05-10T05:31:51.575183+0000","last_peered":"2024-05-10T05:31:51.575183+0000","last_clean":"2024-05-10T05:31:51.575183+0000","last_became_active":"2024-05-10T05:31:51.571711+0000","last_became_peered":"2024-05-10T05:31:51.571711+0000","last_unstale":"2024-05-10T05:31:51.575183+0000","last_undegraded":"2024-05-10T05:31:51.575183+0000","last_fullsized":"2024-05-10T05:31:51.575183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:38:27.813001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.049753+0000","last_change":"2024-05-10T05:31:50.983985+0000","last_active":"2024-05-10T05:31:58.049753+0000","last_peered":"2024-05-10T05:31:58.049753+0000","last_clean":"2024-05-10T05:31:58.049753+0000","last_became_active":"2024-05-10T05:31:50.982931+0000","last_became_peered":"2024-05-10T05:31:50.982931+0000","last_unstale":"2024-05-10T05:31:58.049753+0000","last_undegraded":"2024-05-10T05:31:58.049753+0000","last_fullsized":"2024-05-10T05:31:58.049753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:35:17.158696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.588509+0000","last_change":"2024-05-10T05:31:51.587250+0000","last_active":"2024-05-10T05:31:51.588509+0000","last_peered":"2024-05-10T05:31:51.588509+0000","last_clean":"2024-05-10T05:31:51.588509+0000","last_became_active":"2024-05-10T05:31:51.586925+0000","last_became_peered":"2024-05-10T05:31:51.586925+0000","last_unstale":"2024-05-10T05:31:51.588509+0000","last_undegraded":"2024-05-10T05:31:51.588509+0000","last_fullsized":"2024-05-10T05:31:51.588509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:40.585803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237575+0000","last_change":"2024-05-10T05:31:51.567710+0000","last_active":"2024-05-10T05:31:53.237575+0000","last_peered":"2024-05-10T05:31:53.237575+0000","last_clean":"2024-05-10T05:31:53.237575+0000","last_became_active":"2024-05-10T05:31:51.566860+0000","last_became_peered":"2024-05-10T05:31:51.566860+0000","last_unstale":"2024-05-10T05:31:53.237575+0000","last_undegraded":"2024-05-10T05:31:53.237575+0000","last_fullsized":"2024-05-10T05:31:53.237575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:02:09.253345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.651322+0000","last_change":"2024-05-10T05:31:51.577458+0000","last_active":"2024-05-10T05:31:52.651322+0000","last_peered":"2024-05-10T05:31:52.651322+0000","last_clean":"2024-05-10T05:31:52.651322+0000","last_became_active":"2024-05-10T05:31:51.577123+0000","last_became_peered":"2024-05-10T05:31:51.577123+0000","last_unstale":"2024-05-10T05:31:52.651322+0000","last_undegraded":"2024-05-10T05:31:52.651322+0000","last_fullsized":"2024-05-10T05:31:52.651322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:51.791090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.651232+0000","last_change":"2024-05-10T05:31:51.589361+0000","last_active":"2024-05-10T05:31:52.651232+0000","last_peered":"2024-05-10T05:31:52.651232+0000","last_clean":"2024-05-10T05:31:52.651232+0000","last_became_active":"2024-05-10T05:31:51.589049+0000","last_became_peered":"2024-05-10T05:31:51.589049+0000","last_unstale":"2024-05-10T05:31:52.651232+0000","last_undegraded":"2024-05-10T05:31:52.651232+0000","last_fullsized":"2024-05-10T05:31:52.651232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:13.380505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258833+0000","last_change":"2024-05-10T05:31:51.581494+0000","last_active":"2024-05-10T05:31:53.258833+0000","last_peered":"2024-05-10T05:31:53.258833+0000","last_clean":"2024-05-10T05:31:53.258833+0000","last_became_active":"2024-05-10T05:31:51.579748+0000","last_became_peered":"2024-05-10T05:31:51.579748+0000","last_unstale":"2024-05-10T05:31:53.258833+0000","last_undegraded":"2024-05-10T05:31:53.258833+0000","last_fullsized":"2024-05-10T05:31:53.258833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:34:31.363110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.559808+0000","last_change":"2024-05-10T05:31:51.579791+0000","last_active":"2024-05-10T05:31:52.559808+0000","last_peered":"2024-05-10T05:31:52.559808+0000","last_clean":"2024-05-10T05:31:52.559808+0000","last_became_active":"2024-05-10T05:31:51.577635+0000","last_became_peered":"2024-05-10T05:31:51.577635+0000","last_unstale":"2024-05-10T05:31:52.559808+0000","last_undegraded":"2024-05-10T05:31:52.559808+0000","last_fullsized":"2024-05-10T05:31:52.559808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:54.077805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237707+0000","last_change":"2024-05-10T05:31:51.584601+0000","last_active":"2024-05-10T05:31:53.237707+0000","last_peered":"2024-05-10T05:31:53.237707+0000","last_clean":"2024-05-10T05:31:53.237707+0000","last_became_active":"2024-05-10T05:31:51.584172+0000","last_became_peered":"2024-05-10T05:31:51.584172+0000","last_unstale":"2024-05-10T05:31:53.237707+0000","last_undegraded":"2024-05-10T05:31:53.237707+0000","last_fullsized":"2024-05-10T05:31:53.237707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:49:44.987379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.651584+0000","last_change":"2024-05-10T05:31:51.573784+0000","last_active":"2024-05-10T05:31:52.651584+0000","last_peered":"2024-05-10T05:31:52.651584+0000","last_clean":"2024-05-10T05:31:52.651584+0000","last_became_active":"2024-05-10T05:31:51.573279+0000","last_became_peered":"2024-05-10T05:31:51.573279+0000","last_unstale":"2024-05-10T05:31:52.651584+0000","last_undegraded":"2024-05-10T05:31:52.651584+0000","last_fullsized":"2024-05-10T05:31:52.651584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:08.375964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259590+0000","last_change":"2024-05-10T05:31:51.585577+0000","last_active":"2024-05-10T05:31:53.259590+0000","last_peered":"2024-05-10T05:31:53.259590+0000","last_clean":"2024-05-10T05:31:53.259590+0000","last_became_active":"2024-05-10T05:31:51.585222+0000","last_became_peered":"2024-05-10T05:31:51.585222+0000","last_unstale":"2024-05-10T05:31:53.259590+0000","last_undegraded":"2024-05-10T05:31:53.259590+0000","last_fullsized":"2024-05-10T05:31:53.259590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:47:27.069513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560668+0000","last_change":"2024-05-10T05:31:51.571405+0000","last_active":"2024-05-10T05:31:52.560668+0000","last_peered":"2024-05-10T05:31:52.560668+0000","last_clean":"2024-05-10T05:31:52.560668+0000","last_became_active":"2024-05-10T05:31:51.571125+0000","last_became_peered":"2024-05-10T05:31:51.571125+0000","last_unstale":"2024-05-10T05:31:52.560668+0000","last_undegraded":"2024-05-10T05:31:52.560668+0000","last_fullsized":"2024-05-10T05:31:52.560668+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:49:29.069121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258681+0000","last_change":"2024-05-10T05:31:51.578883+0000","last_active":"2024-05-10T05:31:53.258681+0000","last_peered":"2024-05-10T05:31:53.258681+0000","last_clean":"2024-05-10T05:31:53.258681+0000","last_became_active":"2024-05-10T05:31:51.578582+0000","last_became_peered":"2024-05-10T05:31:51.578582+0000","last_unstale":"2024-05-10T05:31:53.258681+0000","last_undegraded":"2024-05-10T05:31:53.258681+0000","last_fullsized":"2024-05-10T05:31:53.258681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:16.402282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534229+0000","last_change":"2024-05-10T05:31:51.589269+0000","last_active":"2024-05-10T05:31:52.534229+0000","last_peered":"2024-05-10T05:31:52.534229+0000","last_clean":"2024-05-10T05:31:52.534229+0000","last_became_active":"2024-05-10T05:31:51.588875+0000","last_became_peered":"2024-05-10T05:31:51.588875+0000","last_unstale":"2024-05-10T05:31:52.534229+0000","last_undegraded":"2024-05-10T05:31:52.534229+0000","last_fullsized":"2024-05-10T05:31:52.534229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:12.536555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.559944+0000","last_change":"2024-05-10T05:31:51.567789+0000","last_active":"2024-05-10T05:31:52.559944+0000","last_peered":"2024-05-10T05:31:52.559944+0000","last_clean":"2024-05-10T05:31:52.559944+0000","last_became_active":"2024-05-10T05:31:51.566625+0000","last_became_peered":"2024-05-10T05:31:51.566625+0000","last_unstale":"2024-05-10T05:31:52.559944+0000","last_undegraded":"2024-05-10T05:31:52.559944+0000","last_fullsized":"2024-05-10T05:31:52.559944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:43.042557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237554+0000","last_change":"2024-05-10T05:31:51.585838+0000","last_active":"2024-05-10T05:31:53.237554+0000","last_peered":"2024-05-10T05:31:53.237554+0000","last_clean":"2024-05-10T05:31:53.237554+0000","last_became_active":"2024-05-10T05:31:51.585473+0000","last_became_peered":"2024-05-10T05:31:51.585473+0000","last_unstale":"2024-05-10T05:31:53.237554+0000","last_undegraded":"2024-05-10T05:31:53.237554+0000","last_fullsized":"2024-05-10T05:31:53.237554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:11:11.282076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247530,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7826,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":181,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":59209,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":412791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":10,"up_from":68,"seq":292057776142,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":1004535,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":9,"up_from":63,"seq":270582939666,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":880640,"data_stored":412555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.77700000000000002}]},{"osd":2,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.81599999999999995}]},{"osd":7,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.75}]},{"osd":8,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.71599999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":413147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55200000000000005}]},{"osd":1,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.621},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.877}]},{"osd":4,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.82499999999999996}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":1.0169999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397530,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":917504,"data_stored":1003171,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45000000000000001}]},{"osd":5,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.621}]},{"osd":6,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.55900000000000005}]},{"osd":8,"last update":"Fri May 10 05:31:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.69799999999999995}]},{"osd":9,"last update":"Fri May 10 05:31:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.755}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":6,"up_from":44,"seq":188978561055,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":864256,"data_stored":411759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.76700000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.54200000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.65500000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.59499999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.84499999999999997,"15min":0.83399999999999996},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.56799999999999995}]},{"osd":7,"last update":"Fri May 10 05:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.77700000000000002}]},{"osd":8,"last update":"Fri May 10 05:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.64200000000000002}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.85299999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":868352,"data_stored":413211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.73999999999999999,"15min":0.82699999999999996},"min":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73899999999999999,"5min":6.3810000000000002,"15min":6.3810000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81000000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri May 10 05:31:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.629},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri May 10 05:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.42999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.307},"max":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.439,"15min":0.433},"min":{"1min":0.34000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.57399999999999995}]},{"osd":5,"last update":"Fri May 10 05:31:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59499999999999997,"15min":0.59399999999999997},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri May 10 05:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.52400000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86499999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.77100000000000002}]},{"osd":8,"last update":"Fri May 10 05:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.76000000000000001}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.86299999999999999}]},{"osd":10,"last update":"Fri May 10 05:31:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":897024,"data_stored":413777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 05:31:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96299999999999997,"5min":1.256,"15min":1.256},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri May 10 05:31:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.95099999999999996,"5min":1.177,"15min":1.177},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.629,"15min":0.60999999999999999},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92700000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.625}]},{"osd":3,"last update":"Fri May 10 05:30:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri May 10 05:31:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.66500000000000004,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri May 10 05:31:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.69999999999999996,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.84899999999999998}]},{"osd":6,"last update":"Fri May 10 05:31:39 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.75600000000000001,"15min":0.74199999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.347,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.746,"15min":0.73299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.077,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90700000000000003}]},{"osd":8,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.82999999999999996}]},{"osd":9,"last update":"Fri May 10 05:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.94599999999999995}]},{"osd":10,"last update":"Fri May 10 05:31:58 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.877},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.92700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36120,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709032,"statfs":{"total":95995035648,"available":95958048768,"internally_reserved":0,"allocated":933888,"data_stored":453498,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.89400000000000002}]},{"osd":1,"last update":"Fri May 10 05:31:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.51600000000000001,"15min":0.49299999999999999},"min":{"1min":0.40400000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.51500000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.41999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.81100000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.878}]},{"osd":5,"last update":"Fri May 10 05:31:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42199999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.745}]},{"osd":8,"last update":"Fri May 10 05:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63400000000000001}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.755}]},{"osd":10,"last update":"Fri May 10 05:31:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.94499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":412055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.42499999999999999,"15min":0.40300000000000002},"min":{"1min":0.38300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.626},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.436,"15min":0.40799999999999997},"min":{"1min":0.36499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60599999999999998,"15min":0.59399999999999997},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.53300000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65700000000000003,"15min":0.63100000000000001},"min":{"1min":0.47399999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0589999999999999,"5min":1.206,"15min":1.206},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.91600000000000004}]},{"osd":5,"last update":"Fri May 10 05:31:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.53500000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.74399999999999999,"15min":0.72999999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.59099999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.095,"5min":1.119,"15min":1.119},"last":0.92300000000000004}]},{"osd":7,"last update":"Fri May 10 05:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.999}]},{"osd":8,"last update":"Fri May 10 05:31:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.753}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.86499999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":4,"up_from":32,"seq":137438953510,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":868352,"data_stored":412243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.36899999999999999,"15min":0.35699999999999998},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.38800000000000001},"min":{"1min":0.308,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.496,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.754}]},{"osd":3,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.47899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri May 10 05:31:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61599999999999999,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82599999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63400000000000001}]},{"osd":6,"last update":"Fri May 10 05:31:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.497,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55100000000000005}]},{"osd":7,"last update":"Fri May 10 05:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.81999999999999995}]},{"osd":8,"last update":"Fri May 10 05:31:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73399999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789986,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36136,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709016,"statfs":{"total":95995035648,"available":95958032384,"internally_reserved":0,"allocated":950272,"data_stored":1005180,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.622},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.61199999999999999,"15min":0.57099999999999995},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":4.524,"5min":4.524,"15min":4.524},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.47099999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.748},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.52400000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65100000000000002}]},{"osd":2,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.432},"min":{"1min":0.32100000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.438,"15min":0.436},"min":{"1min":0.316,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.83199999999999996}]},{"osd":4,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.495,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77800000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66000000000000003,"15min":0.65900000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri May 10 05:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":1}]},{"osd":8,"last update":"Fri May 10 05:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.97699999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":41851,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T05:32:01.535 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T05:32:01.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.831+0000 7f3caba00700 1 -- 172.21.15.131:0/651035223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca41036c0 msgr2=0x7f3ca4109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:01.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.831+0000 7f3caba00700 1 --2- 172.21.15.131:0/651035223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca41036c0 0x7f3ca4109f70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00b0d0 tx=0x7f3c9c00b3e0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.831+0000 7f3caba00700 1 -- 172.21.15.131:0/651035223 shutdown_connections 2024-05-10T05:32:01.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.831+0000 7f3caba00700 1 --2- 172.21.15.131:0/651035223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca41036c0 0x7f3ca4109f70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.831+0000 7f3caba00700 1 --2- 172.21.15.131:0/651035223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca4101c00 0x7f3ca4102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.831+0000 7f3caba00700 1 --2- 172.21.15.131:0/651035223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ca4102d20 0x7f3ca4103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.831+0000 7f3caba00700 1 -- 172.21.15.131:0/651035223 >> 172.21.15.131:0/651035223 conn(0x7f3ca40fd210 msgr2=0x7f3ca40ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:01.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.831+0000 7f3caba00700 1 -- 172.21.15.131:0/651035223 shutdown_connections 2024-05-10T05:32:01.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.831+0000 7f3caba00700 1 -- 172.21.15.131:0/651035223 wait complete. 2024-05-10T05:32:01.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.832+0000 7f3caba00700 1 Processor -- start 2024-05-10T05:32:01.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.832+0000 7f3caba00700 1 -- start start 2024-05-10T05:32:01.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.832+0000 7f3caba00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ca4101c00 0x7f3ca419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:01.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3caba00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca4102d20 0x7f3ca419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:01.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3caba00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca41036c0 0x7f3ca41a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:01.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3caba00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca41a1dc0 con 0x7f3ca4101c00 2024-05-10T05:32:01.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3caba00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca41a1f30 con 0x7f3ca4102d20 2024-05-10T05:32:01.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3caba00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca41a20f0 con 0x7f3ca41036c0 2024-05-10T05:32:01.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ca4101c00 0x7f3ca419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:01.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ca4101c00 0x7f3ca419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49710/0 (socket says 172.21.15.131:49710) 2024-05-10T05:32:01.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9000700 1 -- 172.21.15.131:0/1888018426 learned_addr learned my addr 172.21.15.131:0/1888018426 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:01.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca3e00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca4102d20 0x7f3ca419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:01.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9a00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca41036c0 0x7f3ca41a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:01.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9000700 1 -- 172.21.15.131:0/1888018426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca41036c0 msgr2=0x7f3ca41a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:01.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9000700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca41036c0 0x7f3ca41a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9000700 1 -- 172.21.15.131:0/1888018426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca4102d20 msgr2=0x7f3ca419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:01.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9000700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca4102d20 0x7f3ca419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:01.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9000700 1 -- 172.21.15.131:0/1888018426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c009d00 con 0x7f3ca4101c00 2024-05-10T05:32:01.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9a00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca41036c0 0x7f3ca41a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:01.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.833+0000 7f3ca9000700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ca4101c00 0x7f3ca419a2e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800e940 tx=0x7f3c9800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:01.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.834+0000 7f3ca1600700 1 -- 172.21.15.131:0/1888018426 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800cbd0 con 0x7f3ca4101c00 2024-05-10T05:32:01.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.834+0000 7f3ca1600700 1 -- 172.21.15.131:0/1888018426 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c9800ee90 con 0x7f3ca4101c00 2024-05-10T05:32:01.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.834+0000 7f3caba00700 1 -- 172.21.15.131:0/1888018426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca41a23d0 con 0x7f3ca4101c00 2024-05-10T05:32:01.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.834+0000 7f3ca1600700 1 -- 172.21.15.131:0/1888018426 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98003cb0 con 0x7f3ca4101c00 2024-05-10T05:32:01.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.834+0000 7f3caba00700 1 -- 172.21.15.131:0/1888018426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca41a28e0 con 0x7f3ca4101c00 2024-05-10T05:32:01.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.835+0000 7f3caba00700 1 -- 172.21.15.131:0/1888018426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca4068730 con 0x7f3ca4101c00 2024-05-10T05:32:01.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.839+0000 7f3ca1600700 1 -- 172.21.15.131:0/1888018426 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3c9800cd30 con 0x7f3ca4101c00 2024-05-10T05:32:01.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.840+0000 7f3ca1600700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c8c0a11b0 0x7f3c8c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:01.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.840+0000 7f3ca1600700 1 -- 172.21.15.131:0/1888018426 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3c98014070 con 0x7f3ca4101c00 2024-05-10T05:32:01.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.840+0000 7f3ca3e00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c8c0a11b0 0x7f3c8c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:01.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.841+0000 7f3ca3e00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c8c0a11b0 0x7f3c8c0a36c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c90005950 tx=0x7f3c9000a300 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:01.842+0000 7f3ca1600700 1 -- 172.21.15.131:0/1888018426 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3c98096420 con 0x7f3ca4101c00 2024-05-10T05:32:02.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.051+0000 7f3caba00700 1 -- 172.21.15.131:0/1888018426 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ca4075c50 con 0x7f3c8c0a11b0 2024-05-10T05:32:02.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.061+0000 7f3ca1600700 1 -- 172.21.15.131:0/1888018426 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+328777 (secure 0 0 0) 0x7f3ca4075c50 con 0x7f3c8c0a11b0 2024-05-10T05:32:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 -- 172.21.15.131:0/1888018426 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c8c0a11b0 msgr2=0x7f3c8c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c8c0a11b0 0x7f3c8c0a36c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c90005950 tx=0x7f3c9000a300 comp rx=0 tx=0).stop 2024-05-10T05:32:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 -- 172.21.15.131:0/1888018426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ca4101c00 msgr2=0x7f3ca419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ca4101c00 0x7f3ca419a2e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800e940 tx=0x7f3c9800ec50 comp rx=0 tx=0).stop 2024-05-10T05:32:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 -- 172.21.15.131:0/1888018426 shutdown_connections 2024-05-10T05:32:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3c8c0a11b0 0x7f3c8c0a36c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ca4101c00 0x7f3ca419a2e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca41036c0 0x7f3ca41a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 --2- 172.21.15.131:0/1888018426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca4102d20 0x7f3ca419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 -- 172.21.15.131:0/1888018426 >> 172.21.15.131:0/1888018426 conn(0x7f3ca40fd210 msgr2=0x7f3ca4074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 -- 172.21.15.131:0/1888018426 shutdown_connections 2024-05-10T05:32:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:02.066+0000 7f3caba00700 1 -- 172.21.15.131:0/1888018426 wait complete. 2024-05-10T05:32:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-10T05:32:02.095 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-05-10T05:32:00.829646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598194,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":149,"num_read_kb":128,"num_write":362,"num_write_kb":2998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375664,"kb_used_data":10508,"kb_used_omap":18,"kb_used_meta":365037,"kb_avail":1124566160,"statfs":{"total":1151940427776,"available":1151555747840,"internally_reserved":0,"allocated":10760192,"data_stored":6767922,"data_compressed":110038,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19128,"internal_metadata":373798216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7826,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":181,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.279082"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237828+0000","last_change":"2024-05-10T05:31:51.576054+0000","last_active":"2024-05-10T05:31:53.237828+0000","last_peered":"2024-05-10T05:31:53.237828+0000","last_clean":"2024-05-10T05:31:53.237828+0000","last_became_active":"2024-05-10T05:31:51.575186+0000","last_became_peered":"2024-05-10T05:31:51.575186+0000","last_unstale":"2024-05-10T05:31:53.237828+0000","last_undegraded":"2024-05-10T05:31:53.237828+0000","last_fullsized":"2024-05-10T05:31:53.237828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:16:27.654822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258780+0000","last_change":"2024-05-10T05:31:51.576483+0000","last_active":"2024-05-10T05:31:53.258780+0000","last_peered":"2024-05-10T05:31:53.258780+0000","last_clean":"2024-05-10T05:31:53.258780+0000","last_became_active":"2024-05-10T05:31:51.574936+0000","last_became_peered":"2024-05-10T05:31:51.574936+0000","last_unstale":"2024-05-10T05:31:53.258780+0000","last_undegraded":"2024-05-10T05:31:53.258780+0000","last_fullsized":"2024-05-10T05:31:53.258780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:56:51.436880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258626+0000","last_change":"2024-05-10T05:31:51.567114+0000","last_active":"2024-05-10T05:31:53.258626+0000","last_peered":"2024-05-10T05:31:53.258626+0000","last_clean":"2024-05-10T05:31:53.258626+0000","last_became_active":"2024-05-10T05:31:51.564552+0000","last_became_peered":"2024-05-10T05:31:51.564552+0000","last_unstale":"2024-05-10T05:31:53.258626+0000","last_undegraded":"2024-05-10T05:31:53.258626+0000","last_fullsized":"2024-05-10T05:31:53.258626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:40.268764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.814602+0000","last_change":"2024-05-10T05:31:51.584471+0000","last_active":"2024-05-10T05:31:52.814602+0000","last_peered":"2024-05-10T05:31:52.814602+0000","last_clean":"2024-05-10T05:31:52.814602+0000","last_became_active":"2024-05-10T05:31:51.583749+0000","last_became_peered":"2024-05-10T05:31:51.583749+0000","last_unstale":"2024-05-10T05:31:52.814602+0000","last_undegraded":"2024-05-10T05:31:52.814602+0000","last_fullsized":"2024-05-10T05:31:52.814602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:30.775286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258572+0000","last_change":"2024-05-10T05:31:51.577698+0000","last_active":"2024-05-10T05:31:53.258572+0000","last_peered":"2024-05-10T05:31:53.258572+0000","last_clean":"2024-05-10T05:31:53.258572+0000","last_became_active":"2024-05-10T05:31:51.571469+0000","last_became_peered":"2024-05-10T05:31:51.571469+0000","last_unstale":"2024-05-10T05:31:53.258572+0000","last_undegraded":"2024-05-10T05:31:53.258572+0000","last_fullsized":"2024-05-10T05:31:53.258572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:35.161267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.574263+0000","last_change":"2024-05-10T05:31:51.573560+0000","last_active":"2024-05-10T05:31:51.574263+0000","last_peered":"2024-05-10T05:31:51.574263+0000","last_clean":"2024-05-10T05:31:51.574263+0000","last_became_active":"2024-05-10T05:31:51.573227+0000","last_became_peered":"2024-05-10T05:31:51.573227+0000","last_unstale":"2024-05-10T05:31:51.574263+0000","last_undegraded":"2024-05-10T05:31:51.574263+0000","last_fullsized":"2024-05-10T05:31:51.574263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:23.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.579534+0000","last_change":"2024-05-10T05:31:51.573667+0000","last_active":"2024-05-10T05:31:51.579534+0000","last_peered":"2024-05-10T05:31:51.579534+0000","last_clean":"2024-05-10T05:31:51.579534+0000","last_became_active":"2024-05-10T05:31:51.573362+0000","last_became_peered":"2024-05-10T05:31:51.573362+0000","last_unstale":"2024-05-10T05:31:51.579534+0000","last_undegraded":"2024-05-10T05:31:51.579534+0000","last_fullsized":"2024-05-10T05:31:51.579534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:00.300134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258155+0000","last_change":"2024-05-10T05:31:51.583596+0000","last_active":"2024-05-10T05:31:53.258155+0000","last_peered":"2024-05-10T05:31:53.258155+0000","last_clean":"2024-05-10T05:31:53.258155+0000","last_became_active":"2024-05-10T05:31:51.583235+0000","last_became_peered":"2024-05-10T05:31:51.583235+0000","last_unstale":"2024-05-10T05:31:53.258155+0000","last_undegraded":"2024-05-10T05:31:53.258155+0000","last_fullsized":"2024-05-10T05:31:53.258155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:05.575975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534295+0000","last_change":"2024-05-10T05:31:51.567627+0000","last_active":"2024-05-10T05:31:52.534295+0000","last_peered":"2024-05-10T05:31:52.534295+0000","last_clean":"2024-05-10T05:31:52.534295+0000","last_became_active":"2024-05-10T05:31:51.567276+0000","last_became_peered":"2024-05-10T05:31:51.567276+0000","last_unstale":"2024-05-10T05:31:52.534295+0000","last_undegraded":"2024-05-10T05:31:52.534295+0000","last_fullsized":"2024-05-10T05:31:52.534295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:16:27.914802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.574732+0000","last_change":"2024-05-10T05:31:51.573859+0000","last_active":"2024-05-10T05:31:51.574732+0000","last_peered":"2024-05-10T05:31:51.574732+0000","last_clean":"2024-05-10T05:31:51.574732+0000","last_became_active":"2024-05-10T05:31:51.573493+0000","last_became_peered":"2024-05-10T05:31:51.573493+0000","last_unstale":"2024-05-10T05:31:51.574732+0000","last_undegraded":"2024-05-10T05:31:51.574732+0000","last_fullsized":"2024-05-10T05:31:51.574732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:21.024888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.578984+0000","last_change":"2024-05-10T05:31:51.576056+0000","last_active":"2024-05-10T05:31:51.578984+0000","last_peered":"2024-05-10T05:31:51.578984+0000","last_clean":"2024-05-10T05:31:51.578984+0000","last_became_active":"2024-05-10T05:31:51.575146+0000","last_became_peered":"2024-05-10T05:31:51.575146+0000","last_unstale":"2024-05-10T05:31:51.578984+0000","last_undegraded":"2024-05-10T05:31:51.578984+0000","last_fullsized":"2024-05-10T05:31:51.578984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:41.201052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.814589+0000","last_change":"2024-05-10T05:31:51.584710+0000","last_active":"2024-05-10T05:31:52.814589+0000","last_peered":"2024-05-10T05:31:52.814589+0000","last_clean":"2024-05-10T05:31:52.814589+0000","last_became_active":"2024-05-10T05:31:51.584122+0000","last_became_peered":"2024-05-10T05:31:51.584122+0000","last_unstale":"2024-05-10T05:31:52.814589+0000","last_undegraded":"2024-05-10T05:31:52.814589+0000","last_fullsized":"2024-05-10T05:31:52.814589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:43.622981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560032+0000","last_change":"2024-05-10T05:31:51.576104+0000","last_active":"2024-05-10T05:31:52.560032+0000","last_peered":"2024-05-10T05:31:52.560032+0000","last_clean":"2024-05-10T05:31:52.560032+0000","last_became_active":"2024-05-10T05:31:51.575650+0000","last_became_peered":"2024-05-10T05:31:51.575650+0000","last_unstale":"2024-05-10T05:31:52.560032+0000","last_undegraded":"2024-05-10T05:31:52.560032+0000","last_fullsized":"2024-05-10T05:31:52.560032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:34:28.798729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258953+0000","last_change":"2024-05-10T05:31:51.574628+0000","last_active":"2024-05-10T05:31:53.258953+0000","last_peered":"2024-05-10T05:31:53.258953+0000","last_clean":"2024-05-10T05:31:53.258953+0000","last_became_active":"2024-05-10T05:31:51.574256+0000","last_became_peered":"2024-05-10T05:31:51.574256+0000","last_unstale":"2024-05-10T05:31:53.258953+0000","last_undegraded":"2024-05-10T05:31:53.258953+0000","last_fullsized":"2024-05-10T05:31:53.258953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:59.174727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258435+0000","last_change":"2024-05-10T05:31:51.582336+0000","last_active":"2024-05-10T05:31:53.258435+0000","last_peered":"2024-05-10T05:31:53.258435+0000","last_clean":"2024-05-10T05:31:53.258435+0000","last_became_active":"2024-05-10T05:31:51.581985+0000","last_became_peered":"2024-05-10T05:31:51.581985+0000","last_unstale":"2024-05-10T05:31:53.258435+0000","last_undegraded":"2024-05-10T05:31:53.258435+0000","last_fullsized":"2024-05-10T05:31:53.258435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:51.601645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534190+0000","last_change":"2024-05-10T05:31:51.568919+0000","last_active":"2024-05-10T05:31:52.534190+0000","last_peered":"2024-05-10T05:31:52.534190+0000","last_clean":"2024-05-10T05:31:52.534190+0000","last_became_active":"2024-05-10T05:31:51.568582+0000","last_became_peered":"2024-05-10T05:31:51.568582+0000","last_unstale":"2024-05-10T05:31:52.534190+0000","last_undegraded":"2024-05-10T05:31:52.534190+0000","last_fullsized":"2024-05-10T05:31:52.534190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:08.377261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237990+0000","last_change":"2024-05-10T05:31:51.579025+0000","last_active":"2024-05-10T05:31:53.237990+0000","last_peered":"2024-05-10T05:31:53.237990+0000","last_clean":"2024-05-10T05:31:53.237990+0000","last_became_active":"2024-05-10T05:31:51.578704+0000","last_became_peered":"2024-05-10T05:31:51.578704+0000","last_unstale":"2024-05-10T05:31:53.237990+0000","last_undegraded":"2024-05-10T05:31:53.237990+0000","last_fullsized":"2024-05-10T05:31:53.237990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:16.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.012716+0000","last_change":"2024-05-10T05:31:51.573193+0000","last_active":"2024-05-10T05:31:53.012716+0000","last_peered":"2024-05-10T05:31:53.012716+0000","last_clean":"2024-05-10T05:31:53.012716+0000","last_became_active":"2024-05-10T05:31:51.572874+0000","last_became_peered":"2024-05-10T05:31:51.572874+0000","last_unstale":"2024-05-10T05:31:53.012716+0000","last_undegraded":"2024-05-10T05:31:53.012716+0000","last_fullsized":"2024-05-10T05:31:53.012716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:09.867661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.651422+0000","last_change":"2024-05-10T05:31:51.581107+0000","last_active":"2024-05-10T05:31:52.651422+0000","last_peered":"2024-05-10T05:31:52.651422+0000","last_clean":"2024-05-10T05:31:52.651422+0000","last_became_active":"2024-05-10T05:31:51.580765+0000","last_became_peered":"2024-05-10T05:31:51.580765+0000","last_unstale":"2024-05-10T05:31:52.651422+0000","last_undegraded":"2024-05-10T05:31:52.651422+0000","last_fullsized":"2024-05-10T05:31:52.651422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:04.164486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'11","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.042119+0000","last_change":"2024-05-10T05:31:50.986534+0000","last_active":"2024-05-10T05:31:57.042119+0000","last_peered":"2024-05-10T05:31:57.042119+0000","last_clean":"2024-05-10T05:31:57.042119+0000","last_became_active":"2024-05-10T05:31:50.983493+0000","last_became_peered":"2024-05-10T05:31:50.983493+0000","last_unstale":"2024-05-10T05:31:57.042119+0000","last_undegraded":"2024-05-10T05:31:57.042119+0000","last_fullsized":"2024-05-10T05:31:57.042119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:18:56.926317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259563+0000","last_change":"2024-05-10T05:31:51.587121+0000","last_active":"2024-05-10T05:31:53.259563+0000","last_peered":"2024-05-10T05:31:53.259563+0000","last_clean":"2024-05-10T05:31:53.259563+0000","last_became_active":"2024-05-10T05:31:51.586641+0000","last_became_peered":"2024-05-10T05:31:51.586641+0000","last_unstale":"2024-05-10T05:31:53.259563+0000","last_undegraded":"2024-05-10T05:31:53.259563+0000","last_fullsized":"2024-05-10T05:31:53.259563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:52.654918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.348579+0000","last_change":"2024-05-10T05:31:50.999755+0000","last_active":"2024-05-10T05:31:53.348579+0000","last_peered":"2024-05-10T05:31:53.348579+0000","last_clean":"2024-05-10T05:31:53.348579+0000","last_became_active":"2024-05-10T05:31:50.999336+0000","last_became_peered":"2024-05-10T05:31:50.999336+0000","last_unstale":"2024-05-10T05:31:53.348579+0000","last_undegraded":"2024-05-10T05:31:53.348579+0000","last_fullsized":"2024-05-10T05:31:53.348579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:43.755130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259252+0000","last_change":"2024-05-10T05:31:51.576938+0000","last_active":"2024-05-10T05:31:53.259252+0000","last_peered":"2024-05-10T05:31:53.259252+0000","last_clean":"2024-05-10T05:31:53.259252+0000","last_became_active":"2024-05-10T05:31:51.576517+0000","last_became_peered":"2024-05-10T05:31:51.576517+0000","last_unstale":"2024-05-10T05:31:53.259252+0000","last_undegraded":"2024-05-10T05:31:53.259252+0000","last_fullsized":"2024-05-10T05:31:53.259252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:52:38.709063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'11","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.050082+0000","last_change":"2024-05-10T05:31:50.992925+0000","last_active":"2024-05-10T05:31:58.050082+0000","last_peered":"2024-05-10T05:31:58.050082+0000","last_clean":"2024-05-10T05:31:58.050082+0000","last_became_active":"2024-05-10T05:31:50.992574+0000","last_became_peered":"2024-05-10T05:31:50.992574+0000","last_unstale":"2024-05-10T05:31:58.050082+0000","last_undegraded":"2024-05-10T05:31:58.050082+0000","last_fullsized":"2024-05-10T05:31:58.050082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:14.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534257+0000","last_change":"2024-05-10T05:31:51.580145+0000","last_active":"2024-05-10T05:31:52.534257+0000","last_peered":"2024-05-10T05:31:52.534257+0000","last_clean":"2024-05-10T05:31:52.534257+0000","last_became_active":"2024-05-10T05:31:51.576701+0000","last_became_peered":"2024-05-10T05:31:51.576701+0000","last_unstale":"2024-05-10T05:31:52.534257+0000","last_undegraded":"2024-05-10T05:31:52.534257+0000","last_fullsized":"2024-05-10T05:31:52.534257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:36:21.075571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.049013+0000","last_change":"2024-05-10T05:31:50.987863+0000","last_active":"2024-05-10T05:31:58.049013+0000","last_peered":"2024-05-10T05:31:58.049013+0000","last_clean":"2024-05-10T05:31:58.049013+0000","last_became_active":"2024-05-10T05:31:50.987463+0000","last_became_peered":"2024-05-10T05:31:50.987463+0000","last_unstale":"2024-05-10T05:31:58.049013+0000","last_undegraded":"2024-05-10T05:31:58.049013+0000","last_fullsized":"2024-05-10T05:31:58.049013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:12:49.542288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560817+0000","last_change":"2024-05-10T05:31:51.585289+0000","last_active":"2024-05-10T05:31:52.560817+0000","last_peered":"2024-05-10T05:31:52.560817+0000","last_clean":"2024-05-10T05:31:52.560817+0000","last_became_active":"2024-05-10T05:31:51.584059+0000","last_became_peered":"2024-05-10T05:31:51.584059+0000","last_unstale":"2024-05-10T05:31:52.560817+0000","last_undegraded":"2024-05-10T05:31:52.560817+0000","last_fullsized":"2024-05-10T05:31:52.560817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:05:02.271354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.496249+0000","last_change":"2024-05-10T05:31:50.985879+0000","last_active":"2024-05-10T05:31:58.496249+0000","last_peered":"2024-05-10T05:31:58.496249+0000","last_clean":"2024-05-10T05:31:58.496249+0000","last_became_active":"2024-05-10T05:31:50.985476+0000","last_became_peered":"2024-05-10T05:31:50.985476+0000","last_unstale":"2024-05-10T05:31:58.496249+0000","last_undegraded":"2024-05-10T05:31:58.496249+0000","last_fullsized":"2024-05-10T05:31:58.496249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:11:27.737972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258660+0000","last_change":"2024-05-10T05:31:51.570985+0000","last_active":"2024-05-10T05:31:53.258660+0000","last_peered":"2024-05-10T05:31:53.258660+0000","last_clean":"2024-05-10T05:31:53.258660+0000","last_became_active":"2024-05-10T05:31:51.570324+0000","last_became_peered":"2024-05-10T05:31:51.570324+0000","last_unstale":"2024-05-10T05:31:53.258660+0000","last_undegraded":"2024-05-10T05:31:53.258660+0000","last_fullsized":"2024-05-10T05:31:53.258660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:26.947574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.583384+0000","last_change":"2024-05-10T05:31:50.988483+0000","last_active":"2024-05-10T05:31:51.583384+0000","last_peered":"2024-05-10T05:31:51.583384+0000","last_clean":"2024-05-10T05:31:51.583384+0000","last_became_active":"2024-05-10T05:31:50.987897+0000","last_became_peered":"2024-05-10T05:31:50.987897+0000","last_unstale":"2024-05-10T05:31:51.583384+0000","last_undegraded":"2024-05-10T05:31:51.583384+0000","last_fullsized":"2024-05-10T05:31:51.583384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:30:23.474637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560935+0000","last_change":"2024-05-10T05:31:51.585717+0000","last_active":"2024-05-10T05:31:52.560935+0000","last_peered":"2024-05-10T05:31:52.560935+0000","last_clean":"2024-05-10T05:31:52.560935+0000","last_became_active":"2024-05-10T05:31:51.585250+0000","last_became_peered":"2024-05-10T05:31:51.585250+0000","last_unstale":"2024-05-10T05:31:52.560935+0000","last_undegraded":"2024-05-10T05:31:52.560935+0000","last_fullsized":"2024-05-10T05:31:52.560935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:05:10.692140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:56.027722+0000","last_change":"2024-05-10T05:31:50.981877+0000","last_active":"2024-05-10T05:31:56.027722+0000","last_peered":"2024-05-10T05:31:56.027722+0000","last_clean":"2024-05-10T05:31:56.027722+0000","last_became_active":"2024-05-10T05:31:50.981527+0000","last_became_peered":"2024-05-10T05:31:50.981527+0000","last_unstale":"2024-05-10T05:31:56.027722+0000","last_undegraded":"2024-05-10T05:31:56.027722+0000","last_fullsized":"2024-05-10T05:31:56.027722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:40.735304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258317+0000","last_change":"2024-05-10T05:31:51.581027+0000","last_active":"2024-05-10T05:31:53.258317+0000","last_peered":"2024-05-10T05:31:53.258317+0000","last_clean":"2024-05-10T05:31:53.258317+0000","last_became_active":"2024-05-10T05:31:51.580620+0000","last_became_peered":"2024-05-10T05:31:51.580620+0000","last_unstale":"2024-05-10T05:31:53.258317+0000","last_undegraded":"2024-05-10T05:31:53.258317+0000","last_fullsized":"2024-05-10T05:31:53.258317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:30:15.378179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.048077+0000","last_change":"2024-05-10T05:31:50.987633+0000","last_active":"2024-05-10T05:31:58.048077+0000","last_peered":"2024-05-10T05:31:58.048077+0000","last_clean":"2024-05-10T05:31:58.048077+0000","last_became_active":"2024-05-10T05:31:50.986750+0000","last_became_peered":"2024-05-10T05:31:50.986750+0000","last_unstale":"2024-05-10T05:31:58.048077+0000","last_undegraded":"2024-05-10T05:31:58.048077+0000","last_fullsized":"2024-05-10T05:31:58.048077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:17:47.705415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.012674+0000","last_change":"2024-05-10T05:31:51.589484+0000","last_active":"2024-05-10T05:31:53.012674+0000","last_peered":"2024-05-10T05:31:53.012674+0000","last_clean":"2024-05-10T05:31:53.012674+0000","last_became_active":"2024-05-10T05:31:51.588992+0000","last_became_peered":"2024-05-10T05:31:51.588992+0000","last_unstale":"2024-05-10T05:31:53.012674+0000","last_undegraded":"2024-05-10T05:31:53.012674+0000","last_fullsized":"2024-05-10T05:31:53.012674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:32.360333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.018747+0000","last_change":"2024-05-10T05:31:50.988945+0000","last_active":"2024-05-10T05:31:55.018747+0000","last_peered":"2024-05-10T05:31:55.018747+0000","last_clean":"2024-05-10T05:31:55.018747+0000","last_became_active":"2024-05-10T05:31:50.988566+0000","last_became_peered":"2024-05-10T05:31:50.988566+0000","last_unstale":"2024-05-10T05:31:55.018747+0000","last_undegraded":"2024-05-10T05:31:55.018747+0000","last_fullsized":"2024-05-10T05:31:55.018747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:06:43.225346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534284+0000","last_change":"2024-05-10T05:31:51.577949+0000","last_active":"2024-05-10T05:31:52.534284+0000","last_peered":"2024-05-10T05:31:52.534284+0000","last_clean":"2024-05-10T05:31:52.534284+0000","last_became_active":"2024-05-10T05:31:51.577011+0000","last_became_peered":"2024-05-10T05:31:51.577011+0000","last_unstale":"2024-05-10T05:31:52.534284+0000","last_undegraded":"2024-05-10T05:31:52.534284+0000","last_fullsized":"2024-05-10T05:31:52.534284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:09.988700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.016535+0000","last_change":"2024-05-10T05:31:50.984888+0000","last_active":"2024-05-10T05:31:55.016535+0000","last_peered":"2024-05-10T05:31:55.016535+0000","last_clean":"2024-05-10T05:31:55.016535+0000","last_became_active":"2024-05-10T05:31:50.984267+0000","last_became_peered":"2024-05-10T05:31:50.984267+0000","last_unstale":"2024-05-10T05:31:55.016535+0000","last_undegraded":"2024-05-10T05:31:55.016535+0000","last_fullsized":"2024-05-10T05:31:55.016535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:44.797469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259203+0000","last_change":"2024-05-10T05:31:51.576863+0000","last_active":"2024-05-10T05:31:53.259203+0000","last_peered":"2024-05-10T05:31:53.259203+0000","last_clean":"2024-05-10T05:31:53.259203+0000","last_became_active":"2024-05-10T05:31:51.576468+0000","last_became_peered":"2024-05-10T05:31:51.576468+0000","last_unstale":"2024-05-10T05:31:53.259203+0000","last_undegraded":"2024-05-10T05:31:53.259203+0000","last_fullsized":"2024-05-10T05:31:53.259203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:20.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.042227+0000","last_change":"2024-05-10T05:31:51.001566+0000","last_active":"2024-05-10T05:31:57.042227+0000","last_peered":"2024-05-10T05:31:57.042227+0000","last_clean":"2024-05-10T05:31:57.042227+0000","last_became_active":"2024-05-10T05:31:51.001203+0000","last_became_peered":"2024-05-10T05:31:51.001203+0000","last_unstale":"2024-05-10T05:31:57.042227+0000","last_undegraded":"2024-05-10T05:31:57.042227+0000","last_fullsized":"2024-05-10T05:31:57.042227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:17:45.465039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534270+0000","last_change":"2024-05-10T05:31:51.579114+0000","last_active":"2024-05-10T05:31:52.534270+0000","last_peered":"2024-05-10T05:31:52.534270+0000","last_clean":"2024-05-10T05:31:52.534270+0000","last_became_active":"2024-05-10T05:31:51.577507+0000","last_became_peered":"2024-05-10T05:31:51.577507+0000","last_unstale":"2024-05-10T05:31:52.534270+0000","last_undegraded":"2024-05-10T05:31:52.534270+0000","last_fullsized":"2024-05-10T05:31:52.534270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:43.777983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.017780+0000","last_change":"2024-05-10T05:31:50.983558+0000","last_active":"2024-05-10T05:31:55.017780+0000","last_peered":"2024-05-10T05:31:55.017780+0000","last_clean":"2024-05-10T05:31:55.017780+0000","last_became_active":"2024-05-10T05:31:50.983087+0000","last_became_peered":"2024-05-10T05:31:50.983087+0000","last_unstale":"2024-05-10T05:31:55.017780+0000","last_undegraded":"2024-05-10T05:31:55.017780+0000","last_fullsized":"2024-05-10T05:31:55.017780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:08.753898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258772+0000","last_change":"2024-05-10T05:31:51.574024+0000","last_active":"2024-05-10T05:31:53.258772+0000","last_peered":"2024-05-10T05:31:53.258772+0000","last_clean":"2024-05-10T05:31:53.258772+0000","last_became_active":"2024-05-10T05:31:51.573703+0000","last_became_peered":"2024-05-10T05:31:51.573703+0000","last_unstale":"2024-05-10T05:31:53.258772+0000","last_undegraded":"2024-05-10T05:31:53.258772+0000","last_fullsized":"2024-05-10T05:31:53.258772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:55:23.435991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.492258+0000","last_change":"2024-05-10T05:31:50.986795+0000","last_active":"2024-05-10T05:31:58.492258+0000","last_peered":"2024-05-10T05:31:58.492258+0000","last_clean":"2024-05-10T05:31:58.492258+0000","last_became_active":"2024-05-10T05:31:50.986428+0000","last_became_peered":"2024-05-10T05:31:50.986428+0000","last_unstale":"2024-05-10T05:31:58.492258+0000","last_undegraded":"2024-05-10T05:31:58.492258+0000","last_fullsized":"2024-05-10T05:31:58.492258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:11.039632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560044+0000","last_change":"2024-05-10T05:31:51.585595+0000","last_active":"2024-05-10T05:31:52.560044+0000","last_peered":"2024-05-10T05:31:52.560044+0000","last_clean":"2024-05-10T05:31:52.560044+0000","last_became_active":"2024-05-10T05:31:51.585091+0000","last_became_peered":"2024-05-10T05:31:51.585091+0000","last_unstale":"2024-05-10T05:31:52.560044+0000","last_undegraded":"2024-05-10T05:31:52.560044+0000","last_fullsized":"2024-05-10T05:31:52.560044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:10:10.489430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.049211+0000","last_change":"2024-05-10T05:31:50.984408+0000","last_active":"2024-05-10T05:31:58.049211+0000","last_peered":"2024-05-10T05:31:58.049211+0000","last_clean":"2024-05-10T05:31:58.049211+0000","last_became_active":"2024-05-10T05:31:50.983098+0000","last_became_peered":"2024-05-10T05:31:50.983098+0000","last_unstale":"2024-05-10T05:31:58.049211+0000","last_undegraded":"2024-05-10T05:31:58.049211+0000","last_fullsized":"2024-05-10T05:31:58.049211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:31.399649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.579782+0000","last_change":"2024-05-10T05:31:51.576160+0000","last_active":"2024-05-10T05:31:51.579782+0000","last_peered":"2024-05-10T05:31:51.579782+0000","last_clean":"2024-05-10T05:31:51.579782+0000","last_became_active":"2024-05-10T05:31:51.575806+0000","last_became_peered":"2024-05-10T05:31:51.575806+0000","last_unstale":"2024-05-10T05:31:51.579782+0000","last_undegraded":"2024-05-10T05:31:51.579782+0000","last_fullsized":"2024-05-10T05:31:51.579782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:53:25.660988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.040605+0000","last_change":"2024-05-10T05:31:50.982752+0000","last_active":"2024-05-10T05:31:57.040605+0000","last_peered":"2024-05-10T05:31:57.040605+0000","last_clean":"2024-05-10T05:31:57.040605+0000","last_became_active":"2024-05-10T05:31:50.982299+0000","last_became_peered":"2024-05-10T05:31:50.982299+0000","last_unstale":"2024-05-10T05:31:57.040605+0000","last_undegraded":"2024-05-10T05:31:57.040605+0000","last_fullsized":"2024-05-10T05:31:57.040605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:52:40.206967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.578968+0000","last_change":"2024-05-10T05:31:51.577081+0000","last_active":"2024-05-10T05:31:51.578968+0000","last_peered":"2024-05-10T05:31:51.578968+0000","last_clean":"2024-05-10T05:31:51.578968+0000","last_became_active":"2024-05-10T05:31:51.576729+0000","last_became_peered":"2024-05-10T05:31:51.576729+0000","last_unstale":"2024-05-10T05:31:51.578968+0000","last_undegraded":"2024-05-10T05:31:51.578968+0000","last_fullsized":"2024-05-10T05:31:51.578968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:14.072427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.048975+0000","last_change":"2024-05-10T05:31:50.991868+0000","last_active":"2024-05-10T05:31:58.048975+0000","last_peered":"2024-05-10T05:31:58.048975+0000","last_clean":"2024-05-10T05:31:58.048975+0000","last_became_active":"2024-05-10T05:31:50.991471+0000","last_became_peered":"2024-05-10T05:31:50.991471+0000","last_unstale":"2024-05-10T05:31:58.048975+0000","last_undegraded":"2024-05-10T05:31:58.048975+0000","last_fullsized":"2024-05-10T05:31:58.048975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:22.346376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560537+0000","last_change":"2024-05-10T05:31:51.584726+0000","last_active":"2024-05-10T05:31:52.560537+0000","last_peered":"2024-05-10T05:31:52.560537+0000","last_clean":"2024-05-10T05:31:52.560537+0000","last_became_active":"2024-05-10T05:31:51.584327+0000","last_became_peered":"2024-05-10T05:31:51.584327+0000","last_unstale":"2024-05-10T05:31:52.560537+0000","last_undegraded":"2024-05-10T05:31:52.560537+0000","last_fullsized":"2024-05-10T05:31:52.560537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:56:43.561717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.492270+0000","last_change":"2024-05-10T05:31:50.989071+0000","last_active":"2024-05-10T05:31:58.492270+0000","last_peered":"2024-05-10T05:31:58.492270+0000","last_clean":"2024-05-10T05:31:58.492270+0000","last_became_active":"2024-05-10T05:31:50.988044+0000","last_became_peered":"2024-05-10T05:31:50.988044+0000","last_unstale":"2024-05-10T05:31:58.492270+0000","last_undegraded":"2024-05-10T05:31:58.492270+0000","last_fullsized":"2024-05-10T05:31:58.492270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:24.122564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237745+0000","last_change":"2024-05-10T05:31:51.576793+0000","last_active":"2024-05-10T05:31:53.237745+0000","last_peered":"2024-05-10T05:31:53.237745+0000","last_clean":"2024-05-10T05:31:53.237745+0000","last_became_active":"2024-05-10T05:31:51.576333+0000","last_became_peered":"2024-05-10T05:31:51.576333+0000","last_unstale":"2024-05-10T05:31:53.237745+0000","last_undegraded":"2024-05-10T05:31:53.237745+0000","last_fullsized":"2024-05-10T05:31:53.237745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:42.418736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258788+0000","last_change":"2024-05-10T05:31:50.997233+0000","last_active":"2024-05-10T05:31:53.258788+0000","last_peered":"2024-05-10T05:31:53.258788+0000","last_clean":"2024-05-10T05:31:53.258788+0000","last_became_active":"2024-05-10T05:31:50.996857+0000","last_became_peered":"2024-05-10T05:31:50.996857+0000","last_unstale":"2024-05-10T05:31:53.258788+0000","last_undegraded":"2024-05-10T05:31:53.258788+0000","last_fullsized":"2024-05-10T05:31:53.258788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:35.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.586288+0000","last_change":"2024-05-10T05:31:51.584371+0000","last_active":"2024-05-10T05:31:51.586288+0000","last_peered":"2024-05-10T05:31:51.586288+0000","last_clean":"2024-05-10T05:31:51.586288+0000","last_became_active":"2024-05-10T05:31:51.583966+0000","last_became_peered":"2024-05-10T05:31:51.583966+0000","last_unstale":"2024-05-10T05:31:51.586288+0000","last_undegraded":"2024-05-10T05:31:51.586288+0000","last_fullsized":"2024-05-10T05:31:51.586288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:22:13.036852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259956+0000","last_change":"2024-05-10T05:31:50.991113+0000","last_active":"2024-05-10T05:31:53.259956+0000","last_peered":"2024-05-10T05:31:53.259956+0000","last_clean":"2024-05-10T05:31:53.259956+0000","last_became_active":"2024-05-10T05:31:50.990745+0000","last_became_peered":"2024-05-10T05:31:50.990745+0000","last_unstale":"2024-05-10T05:31:53.259956+0000","last_undegraded":"2024-05-10T05:31:53.259956+0000","last_fullsized":"2024-05-10T05:31:53.259956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:49:56.586231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.584042+0000","last_change":"2024-05-10T05:31:51.582057+0000","last_active":"2024-05-10T05:31:51.584042+0000","last_peered":"2024-05-10T05:31:51.584042+0000","last_clean":"2024-05-10T05:31:51.584042+0000","last_became_active":"2024-05-10T05:31:51.581672+0000","last_became_peered":"2024-05-10T05:31:51.581672+0000","last_unstale":"2024-05-10T05:31:51.584042+0000","last_undegraded":"2024-05-10T05:31:51.584042+0000","last_fullsized":"2024-05-10T05:31:51.584042+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:45:09.122256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560845+0000","last_change":"2024-05-10T05:31:51.585659+0000","last_active":"2024-05-10T05:31:52.560845+0000","last_peered":"2024-05-10T05:31:52.560845+0000","last_clean":"2024-05-10T05:31:52.560845+0000","last_became_active":"2024-05-10T05:31:51.584540+0000","last_became_peered":"2024-05-10T05:31:51.584540+0000","last_unstale":"2024-05-10T05:31:52.560845+0000","last_undegraded":"2024-05-10T05:31:52.560845+0000","last_fullsized":"2024-05-10T05:31:52.560845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:20.195369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:56.027631+0000","last_change":"2024-05-10T05:31:50.993547+0000","last_active":"2024-05-10T05:31:56.027631+0000","last_peered":"2024-05-10T05:31:56.027631+0000","last_clean":"2024-05-10T05:31:56.027631+0000","last_became_active":"2024-05-10T05:31:50.993190+0000","last_became_peered":"2024-05-10T05:31:50.993190+0000","last_unstale":"2024-05-10T05:31:56.027631+0000","last_undegraded":"2024-05-10T05:31:56.027631+0000","last_fullsized":"2024-05-10T05:31:56.027631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:06.159572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259316+0000","last_change":"2024-05-10T05:31:51.587678+0000","last_active":"2024-05-10T05:31:53.259316+0000","last_peered":"2024-05-10T05:31:53.259316+0000","last_clean":"2024-05-10T05:31:53.259316+0000","last_became_active":"2024-05-10T05:31:51.586999+0000","last_became_peered":"2024-05-10T05:31:51.586999+0000","last_unstale":"2024-05-10T05:31:53.259316+0000","last_undegraded":"2024-05-10T05:31:53.259316+0000","last_fullsized":"2024-05-10T05:31:53.259316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:18:00.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.042889+0000","last_change":"2024-05-10T05:31:50.979131+0000","last_active":"2024-05-10T05:31:57.042889+0000","last_peered":"2024-05-10T05:31:57.042889+0000","last_clean":"2024-05-10T05:31:57.042889+0000","last_became_active":"2024-05-10T05:31:50.978715+0000","last_became_peered":"2024-05-10T05:31:50.978715+0000","last_unstale":"2024-05-10T05:31:57.042889+0000","last_undegraded":"2024-05-10T05:31:57.042889+0000","last_fullsized":"2024-05-10T05:31:57.042889+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:29:11.953963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.562030+0000","last_change":"2024-05-10T05:30:59.356688+0000","last_active":"2024-05-10T05:31:51.562030+0000","last_peered":"2024-05-10T05:31:51.562030+0000","last_clean":"2024-05-10T05:31:51.562030+0000","last_became_active":"2024-05-10T05:30:59.356313+0000","last_became_peered":"2024-05-10T05:30:59.356313+0000","last_unstale":"2024-05-10T05:31:51.562030+0000","last_undegraded":"2024-05-10T05:31:51.562030+0000","last_fullsized":"2024-05-10T05:31:51.562030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_clean_scrub_stamp":"2024-05-10T05:28:15.503591+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:43.011495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.559640+0000","last_change":"2024-05-10T05:31:51.580259+0000","last_active":"2024-05-10T05:31:52.559640+0000","last_peered":"2024-05-10T05:31:52.559640+0000","last_clean":"2024-05-10T05:31:52.559640+0000","last_became_active":"2024-05-10T05:31:51.578650+0000","last_became_peered":"2024-05-10T05:31:51.578650+0000","last_unstale":"2024-05-10T05:31:52.559640+0000","last_undegraded":"2024-05-10T05:31:52.559640+0000","last_fullsized":"2024-05-10T05:31:52.559640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:48.630183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:56.022783+0000","last_change":"2024-05-10T05:31:50.987708+0000","last_active":"2024-05-10T05:31:56.022783+0000","last_peered":"2024-05-10T05:31:56.022783+0000","last_clean":"2024-05-10T05:31:56.022783+0000","last_became_active":"2024-05-10T05:31:50.987347+0000","last_became_peered":"2024-05-10T05:31:50.987347+0000","last_unstale":"2024-05-10T05:31:56.022783+0000","last_undegraded":"2024-05-10T05:31:56.022783+0000","last_fullsized":"2024-05-10T05:31:56.022783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:16:29.388690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.049419+0000","last_change":"2024-05-10T05:31:50.999338+0000","last_active":"2024-05-10T05:31:58.049419+0000","last_peered":"2024-05-10T05:31:58.049419+0000","last_clean":"2024-05-10T05:31:58.049419+0000","last_became_active":"2024-05-10T05:31:50.999026+0000","last_became_peered":"2024-05-10T05:31:50.999026+0000","last_unstale":"2024-05-10T05:31:58.049419+0000","last_undegraded":"2024-05-10T05:31:58.049419+0000","last_fullsized":"2024-05-10T05:31:58.049419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:31.443409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560043+0000","last_change":"2024-05-10T05:31:51.574201+0000","last_active":"2024-05-10T05:31:52.560043+0000","last_peered":"2024-05-10T05:31:52.560043+0000","last_clean":"2024-05-10T05:31:52.560043+0000","last_became_active":"2024-05-10T05:31:51.571180+0000","last_became_peered":"2024-05-10T05:31:51.571180+0000","last_unstale":"2024-05-10T05:31:52.560043+0000","last_undegraded":"2024-05-10T05:31:52.560043+0000","last_fullsized":"2024-05-10T05:31:52.560043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:33:16.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.043591+0000","last_change":"2024-05-10T05:31:50.985164+0000","last_active":"2024-05-10T05:31:57.043591+0000","last_peered":"2024-05-10T05:31:57.043591+0000","last_clean":"2024-05-10T05:31:57.043591+0000","last_became_active":"2024-05-10T05:31:50.984760+0000","last_became_peered":"2024-05-10T05:31:50.984760+0000","last_unstale":"2024-05-10T05:31:57.043591+0000","last_undegraded":"2024-05-10T05:31:57.043591+0000","last_fullsized":"2024-05-10T05:31:57.043591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:19.827407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259609+0000","last_change":"2024-05-10T05:31:51.577445+0000","last_active":"2024-05-10T05:31:53.259609+0000","last_peered":"2024-05-10T05:31:53.259609+0000","last_clean":"2024-05-10T05:31:53.259609+0000","last_became_active":"2024-05-10T05:31:51.576671+0000","last_became_peered":"2024-05-10T05:31:51.576671+0000","last_unstale":"2024-05-10T05:31:53.259609+0000","last_undegraded":"2024-05-10T05:31:53.259609+0000","last_fullsized":"2024-05-10T05:31:53.259609+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:38.774538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.019858+0000","last_change":"2024-05-10T05:31:50.981495+0000","last_active":"2024-05-10T05:31:55.019858+0000","last_peered":"2024-05-10T05:31:55.019858+0000","last_clean":"2024-05-10T05:31:55.019858+0000","last_became_active":"2024-05-10T05:31:50.981127+0000","last_became_peered":"2024-05-10T05:31:50.981127+0000","last_unstale":"2024-05-10T05:31:55.019858+0000","last_undegraded":"2024-05-10T05:31:55.019858+0000","last_fullsized":"2024-05-10T05:31:55.019858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:22:32.257014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258881+0000","last_change":"2024-05-10T05:31:51.585926+0000","last_active":"2024-05-10T05:31:53.258881+0000","last_peered":"2024-05-10T05:31:53.258881+0000","last_clean":"2024-05-10T05:31:53.258881+0000","last_became_active":"2024-05-10T05:31:51.585576+0000","last_became_peered":"2024-05-10T05:31:51.585576+0000","last_unstale":"2024-05-10T05:31:53.258881+0000","last_undegraded":"2024-05-10T05:31:53.258881+0000","last_fullsized":"2024-05-10T05:31:53.258881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:22.344180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.016517+0000","last_change":"2024-05-10T05:31:50.989261+0000","last_active":"2024-05-10T05:31:55.016517+0000","last_peered":"2024-05-10T05:31:55.016517+0000","last_clean":"2024-05-10T05:31:55.016517+0000","last_became_active":"2024-05-10T05:31:50.988885+0000","last_became_peered":"2024-05-10T05:31:50.988885+0000","last_unstale":"2024-05-10T05:31:55.016517+0000","last_undegraded":"2024-05-10T05:31:55.016517+0000","last_fullsized":"2024-05-10T05:31:55.016517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:48:59.554692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258573+0000","last_change":"2024-05-10T05:31:51.573279+0000","last_active":"2024-05-10T05:31:53.258573+0000","last_peered":"2024-05-10T05:31:53.258573+0000","last_clean":"2024-05-10T05:31:53.258573+0000","last_became_active":"2024-05-10T05:31:51.572916+0000","last_became_peered":"2024-05-10T05:31:51.572916+0000","last_unstale":"2024-05-10T05:31:53.258573+0000","last_undegraded":"2024-05-10T05:31:53.258573+0000","last_fullsized":"2024-05-10T05:31:53.258573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:32:39.802463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.048490+0000","last_change":"2024-05-10T05:31:50.983771+0000","last_active":"2024-05-10T05:31:58.048490+0000","last_peered":"2024-05-10T05:31:58.048490+0000","last_clean":"2024-05-10T05:31:58.048490+0000","last_became_active":"2024-05-10T05:31:50.981849+0000","last_became_peered":"2024-05-10T05:31:50.981849+0000","last_unstale":"2024-05-10T05:31:58.048490+0000","last_undegraded":"2024-05-10T05:31:58.048490+0000","last_fullsized":"2024-05-10T05:31:58.048490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:15.990869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237478+0000","last_change":"2024-05-10T05:31:51.568005+0000","last_active":"2024-05-10T05:31:53.237478+0000","last_peered":"2024-05-10T05:31:53.237478+0000","last_clean":"2024-05-10T05:31:53.237478+0000","last_became_active":"2024-05-10T05:31:51.567347+0000","last_became_peered":"2024-05-10T05:31:51.567347+0000","last_unstale":"2024-05-10T05:31:53.237478+0000","last_undegraded":"2024-05-10T05:31:53.237478+0000","last_fullsized":"2024-05-10T05:31:53.237478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:29.075816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:57.043607+0000","last_change":"2024-05-10T05:31:50.990237+0000","last_active":"2024-05-10T05:31:57.043607+0000","last_peered":"2024-05-10T05:31:57.043607+0000","last_clean":"2024-05-10T05:31:57.043607+0000","last_became_active":"2024-05-10T05:31:50.989897+0000","last_became_peered":"2024-05-10T05:31:50.989897+0000","last_unstale":"2024-05-10T05:31:57.043607+0000","last_undegraded":"2024-05-10T05:31:57.043607+0000","last_fullsized":"2024-05-10T05:31:57.043607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:30.593071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259318+0000","last_change":"2024-05-10T05:31:51.588255+0000","last_active":"2024-05-10T05:31:53.259318+0000","last_peered":"2024-05-10T05:31:53.259318+0000","last_clean":"2024-05-10T05:31:53.259318+0000","last_became_active":"2024-05-10T05:31:51.587912+0000","last_became_peered":"2024-05-10T05:31:51.587912+0000","last_unstale":"2024-05-10T05:31:53.259318+0000","last_undegraded":"2024-05-10T05:31:53.259318+0000","last_fullsized":"2024-05-10T05:31:53.259318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:18.515739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:55.017748+0000","last_change":"2024-05-10T05:31:50.992392+0000","last_active":"2024-05-10T05:31:55.017748+0000","last_peered":"2024-05-10T05:31:55.017748+0000","last_clean":"2024-05-10T05:31:55.017748+0000","last_became_active":"2024-05-10T05:31:50.992119+0000","last_became_peered":"2024-05-10T05:31:50.992119+0000","last_unstale":"2024-05-10T05:31:55.017748+0000","last_undegraded":"2024-05-10T05:31:55.017748+0000","last_fullsized":"2024-05-10T05:31:55.017748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:11.515013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.568789+0000","last_change":"2024-05-10T05:31:51.567780+0000","last_active":"2024-05-10T05:31:51.568789+0000","last_peered":"2024-05-10T05:31:51.568789+0000","last_clean":"2024-05-10T05:31:51.568789+0000","last_became_active":"2024-05-10T05:31:51.567455+0000","last_became_peered":"2024-05-10T05:31:51.567455+0000","last_unstale":"2024-05-10T05:31:51.568789+0000","last_undegraded":"2024-05-10T05:31:51.568789+0000","last_fullsized":"2024-05-10T05:31:51.568789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:47.108131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.496788+0000","last_change":"2024-05-10T05:31:50.997374+0000","last_active":"2024-05-10T05:31:58.496788+0000","last_peered":"2024-05-10T05:31:58.496788+0000","last_clean":"2024-05-10T05:31:58.496788+0000","last_became_active":"2024-05-10T05:31:50.997069+0000","last_became_peered":"2024-05-10T05:31:50.997069+0000","last_unstale":"2024-05-10T05:31:58.496788+0000","last_undegraded":"2024-05-10T05:31:58.496788+0000","last_fullsized":"2024-05-10T05:31:58.496788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:25:49.140006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.579011+0000","last_change":"2024-05-10T05:31:51.576345+0000","last_active":"2024-05-10T05:31:51.579011+0000","last_peered":"2024-05-10T05:31:51.579011+0000","last_clean":"2024-05-10T05:31:51.579011+0000","last_became_active":"2024-05-10T05:31:51.575704+0000","last_became_peered":"2024-05-10T05:31:51.575704+0000","last_unstale":"2024-05-10T05:31:51.579011+0000","last_undegraded":"2024-05-10T05:31:51.579011+0000","last_fullsized":"2024-05-10T05:31:51.579011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:15.146591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.050034+0000","last_change":"2024-05-10T05:31:50.984108+0000","last_active":"2024-05-10T05:31:58.050034+0000","last_peered":"2024-05-10T05:31:58.050034+0000","last_clean":"2024-05-10T05:31:58.050034+0000","last_became_active":"2024-05-10T05:31:50.983728+0000","last_became_peered":"2024-05-10T05:31:50.983728+0000","last_unstale":"2024-05-10T05:31:58.050034+0000","last_undegraded":"2024-05-10T05:31:58.050034+0000","last_fullsized":"2024-05-10T05:31:58.050034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:23:20.096286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.575183+0000","last_change":"2024-05-10T05:31:51.572075+0000","last_active":"2024-05-10T05:31:51.575183+0000","last_peered":"2024-05-10T05:31:51.575183+0000","last_clean":"2024-05-10T05:31:51.575183+0000","last_became_active":"2024-05-10T05:31:51.571711+0000","last_became_peered":"2024-05-10T05:31:51.571711+0000","last_unstale":"2024-05-10T05:31:51.575183+0000","last_undegraded":"2024-05-10T05:31:51.575183+0000","last_fullsized":"2024-05-10T05:31:51.575183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:38:27.813001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:58.049753+0000","last_change":"2024-05-10T05:31:50.983985+0000","last_active":"2024-05-10T05:31:58.049753+0000","last_peered":"2024-05-10T05:31:58.049753+0000","last_clean":"2024-05-10T05:31:58.049753+0000","last_became_active":"2024-05-10T05:31:50.982931+0000","last_became_peered":"2024-05-10T05:31:50.982931+0000","last_unstale":"2024-05-10T05:31:58.049753+0000","last_undegraded":"2024-05-10T05:31:58.049753+0000","last_fullsized":"2024-05-10T05:31:58.049753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:35:17.158696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:51.588509+0000","last_change":"2024-05-10T05:31:51.587250+0000","last_active":"2024-05-10T05:31:51.588509+0000","last_peered":"2024-05-10T05:31:51.588509+0000","last_clean":"2024-05-10T05:31:51.588509+0000","last_became_active":"2024-05-10T05:31:51.586925+0000","last_became_peered":"2024-05-10T05:31:51.586925+0000","last_unstale":"2024-05-10T05:31:51.588509+0000","last_undegraded":"2024-05-10T05:31:51.588509+0000","last_fullsized":"2024-05-10T05:31:51.588509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:40.585803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237575+0000","last_change":"2024-05-10T05:31:51.567710+0000","last_active":"2024-05-10T05:31:53.237575+0000","last_peered":"2024-05-10T05:31:53.237575+0000","last_clean":"2024-05-10T05:31:53.237575+0000","last_became_active":"2024-05-10T05:31:51.566860+0000","last_became_peered":"2024-05-10T05:31:51.566860+0000","last_unstale":"2024-05-10T05:31:53.237575+0000","last_undegraded":"2024-05-10T05:31:53.237575+0000","last_fullsized":"2024-05-10T05:31:53.237575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:02:09.253345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.651322+0000","last_change":"2024-05-10T05:31:51.577458+0000","last_active":"2024-05-10T05:31:52.651322+0000","last_peered":"2024-05-10T05:31:52.651322+0000","last_clean":"2024-05-10T05:31:52.651322+0000","last_became_active":"2024-05-10T05:31:51.577123+0000","last_became_peered":"2024-05-10T05:31:51.577123+0000","last_unstale":"2024-05-10T05:31:52.651322+0000","last_undegraded":"2024-05-10T05:31:52.651322+0000","last_fullsized":"2024-05-10T05:31:52.651322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:51.791090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.651232+0000","last_change":"2024-05-10T05:31:51.589361+0000","last_active":"2024-05-10T05:31:52.651232+0000","last_peered":"2024-05-10T05:31:52.651232+0000","last_clean":"2024-05-10T05:31:52.651232+0000","last_became_active":"2024-05-10T05:31:51.589049+0000","last_became_peered":"2024-05-10T05:31:51.589049+0000","last_unstale":"2024-05-10T05:31:52.651232+0000","last_undegraded":"2024-05-10T05:31:52.651232+0000","last_fullsized":"2024-05-10T05:31:52.651232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:13.380505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258833+0000","last_change":"2024-05-10T05:31:51.581494+0000","last_active":"2024-05-10T05:31:53.258833+0000","last_peered":"2024-05-10T05:31:53.258833+0000","last_clean":"2024-05-10T05:31:53.258833+0000","last_became_active":"2024-05-10T05:31:51.579748+0000","last_became_peered":"2024-05-10T05:31:51.579748+0000","last_unstale":"2024-05-10T05:31:53.258833+0000","last_undegraded":"2024-05-10T05:31:53.258833+0000","last_fullsized":"2024-05-10T05:31:53.258833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:34:31.363110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.559808+0000","last_change":"2024-05-10T05:31:51.579791+0000","last_active":"2024-05-10T05:31:52.559808+0000","last_peered":"2024-05-10T05:31:52.559808+0000","last_clean":"2024-05-10T05:31:52.559808+0000","last_became_active":"2024-05-10T05:31:51.577635+0000","last_became_peered":"2024-05-10T05:31:51.577635+0000","last_unstale":"2024-05-10T05:31:52.559808+0000","last_undegraded":"2024-05-10T05:31:52.559808+0000","last_fullsized":"2024-05-10T05:31:52.559808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:54.077805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237707+0000","last_change":"2024-05-10T05:31:51.584601+0000","last_active":"2024-05-10T05:31:53.237707+0000","last_peered":"2024-05-10T05:31:53.237707+0000","last_clean":"2024-05-10T05:31:53.237707+0000","last_became_active":"2024-05-10T05:31:51.584172+0000","last_became_peered":"2024-05-10T05:31:51.584172+0000","last_unstale":"2024-05-10T05:31:53.237707+0000","last_undegraded":"2024-05-10T05:31:53.237707+0000","last_fullsized":"2024-05-10T05:31:53.237707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:49:44.987379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.651584+0000","last_change":"2024-05-10T05:31:51.573784+0000","last_active":"2024-05-10T05:31:52.651584+0000","last_peered":"2024-05-10T05:31:52.651584+0000","last_clean":"2024-05-10T05:31:52.651584+0000","last_became_active":"2024-05-10T05:31:51.573279+0000","last_became_peered":"2024-05-10T05:31:51.573279+0000","last_unstale":"2024-05-10T05:31:52.651584+0000","last_undegraded":"2024-05-10T05:31:52.651584+0000","last_fullsized":"2024-05-10T05:31:52.651584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:08.375964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.259590+0000","last_change":"2024-05-10T05:31:51.585577+0000","last_active":"2024-05-10T05:31:53.259590+0000","last_peered":"2024-05-10T05:31:53.259590+0000","last_clean":"2024-05-10T05:31:53.259590+0000","last_became_active":"2024-05-10T05:31:51.585222+0000","last_became_peered":"2024-05-10T05:31:51.585222+0000","last_unstale":"2024-05-10T05:31:53.259590+0000","last_undegraded":"2024-05-10T05:31:53.259590+0000","last_fullsized":"2024-05-10T05:31:53.259590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:47:27.069513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.560668+0000","last_change":"2024-05-10T05:31:51.571405+0000","last_active":"2024-05-10T05:31:52.560668+0000","last_peered":"2024-05-10T05:31:52.560668+0000","last_clean":"2024-05-10T05:31:52.560668+0000","last_became_active":"2024-05-10T05:31:51.571125+0000","last_became_peered":"2024-05-10T05:31:51.571125+0000","last_unstale":"2024-05-10T05:31:52.560668+0000","last_undegraded":"2024-05-10T05:31:52.560668+0000","last_fullsized":"2024-05-10T05:31:52.560668+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:49:29.069121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.258681+0000","last_change":"2024-05-10T05:31:51.578883+0000","last_active":"2024-05-10T05:31:53.258681+0000","last_peered":"2024-05-10T05:31:53.258681+0000","last_clean":"2024-05-10T05:31:53.258681+0000","last_became_active":"2024-05-10T05:31:51.578582+0000","last_became_peered":"2024-05-10T05:31:51.578582+0000","last_unstale":"2024-05-10T05:31:53.258681+0000","last_undegraded":"2024-05-10T05:31:53.258681+0000","last_fullsized":"2024-05-10T05:31:53.258681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:16.402282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.534229+0000","last_change":"2024-05-10T05:31:51.589269+0000","last_active":"2024-05-10T05:31:52.534229+0000","last_peered":"2024-05-10T05:31:52.534229+0000","last_clean":"2024-05-10T05:31:52.534229+0000","last_became_active":"2024-05-10T05:31:51.588875+0000","last_became_peered":"2024-05-10T05:31:51.588875+0000","last_unstale":"2024-05-10T05:31:52.534229+0000","last_undegraded":"2024-05-10T05:31:52.534229+0000","last_fullsized":"2024-05-10T05:31:52.534229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:12.536555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:52.559944+0000","last_change":"2024-05-10T05:31:51.567789+0000","last_active":"2024-05-10T05:31:52.559944+0000","last_peered":"2024-05-10T05:31:52.559944+0000","last_clean":"2024-05-10T05:31:52.559944+0000","last_became_active":"2024-05-10T05:31:51.566625+0000","last_became_peered":"2024-05-10T05:31:51.566625+0000","last_unstale":"2024-05-10T05:31:52.559944+0000","last_undegraded":"2024-05-10T05:31:52.559944+0000","last_fullsized":"2024-05-10T05:31:52.559944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:43.042557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-10T05:31:53.237554+0000","last_change":"2024-05-10T05:31:51.585838+0000","last_active":"2024-05-10T05:31:53.237554+0000","last_peered":"2024-05-10T05:31:53.237554+0000","last_clean":"2024-05-10T05:31:53.237554+0000","last_became_active":"2024-05-10T05:31:51.585473+0000","last_became_peered":"2024-05-10T05:31:51.585473+0000","last_unstale":"2024-05-10T05:31:53.237554+0000","last_undegraded":"2024-05-10T05:31:53.237554+0000","last_fullsized":"2024-05-10T05:31:53.237554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:11:11.282076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247530,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7826,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":181,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":59209,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":412791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":10,"up_from":68,"seq":292057776142,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":1004535,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":9,"up_from":63,"seq":270582939666,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":880640,"data_stored":412555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.77700000000000002}]},{"osd":2,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.81599999999999995}]},{"osd":7,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.75}]},{"osd":8,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.71599999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":413147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55200000000000005}]},{"osd":1,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.621},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.877}]},{"osd":4,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Fri May 10 05:31:16 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.82499999999999996}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":1.0169999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397530,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":917504,"data_stored":1003171,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45000000000000001}]},{"osd":5,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.621}]},{"osd":6,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.55900000000000005}]},{"osd":8,"last update":"Fri May 10 05:31:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.69799999999999995}]},{"osd":9,"last update":"Fri May 10 05:31:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.755}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":6,"up_from":44,"seq":188978561055,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":864256,"data_stored":411759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.76700000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.54200000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.65500000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.59499999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.84499999999999997,"15min":0.83399999999999996},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.56799999999999995}]},{"osd":7,"last update":"Fri May 10 05:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.77700000000000002}]},{"osd":8,"last update":"Fri May 10 05:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.64200000000000002}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.85299999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":868352,"data_stored":413211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.73999999999999999,"15min":0.82699999999999996},"min":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73899999999999999,"5min":6.3810000000000002,"15min":6.3810000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81000000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri May 10 05:31:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.629},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri May 10 05:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.42999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.307},"max":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.439,"15min":0.433},"min":{"1min":0.34000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.57399999999999995}]},{"osd":5,"last update":"Fri May 10 05:31:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59499999999999997,"15min":0.59399999999999997},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri May 10 05:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.52400000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86499999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.77100000000000002}]},{"osd":8,"last update":"Fri May 10 05:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.76000000000000001}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.86299999999999999}]},{"osd":10,"last update":"Fri May 10 05:31:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":897024,"data_stored":413777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 05:31:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96299999999999997,"5min":1.256,"15min":1.256},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri May 10 05:31:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.44800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.95099999999999996,"5min":1.177,"15min":1.177},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.629,"15min":0.60999999999999999},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92700000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.625}]},{"osd":3,"last update":"Fri May 10 05:30:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri May 10 05:31:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.66500000000000004,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri May 10 05:31:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.69999999999999996,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.84899999999999998}]},{"osd":6,"last update":"Fri May 10 05:31:39 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.75600000000000001,"15min":0.74199999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.347,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.746,"15min":0.73299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.077,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90700000000000003}]},{"osd":8,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.82999999999999996}]},{"osd":9,"last update":"Fri May 10 05:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.94599999999999995}]},{"osd":10,"last update":"Fri May 10 05:31:58 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.877},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.92700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36120,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709032,"statfs":{"total":95995035648,"available":95958048768,"internally_reserved":0,"allocated":933888,"data_stored":453498,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.89400000000000002}]},{"osd":1,"last update":"Fri May 10 05:31:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.51600000000000001,"15min":0.49299999999999999},"min":{"1min":0.40400000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.51500000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.41999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.81100000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.878}]},{"osd":5,"last update":"Fri May 10 05:31:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42199999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.745}]},{"osd":8,"last update":"Fri May 10 05:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63400000000000001}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.755}]},{"osd":10,"last update":"Fri May 10 05:31:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.94499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":412055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.42499999999999999,"15min":0.40300000000000002},"min":{"1min":0.38300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.626},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.436,"15min":0.40799999999999997},"min":{"1min":0.36499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60599999999999998,"15min":0.59399999999999997},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.53300000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65700000000000003,"15min":0.63100000000000001},"min":{"1min":0.47399999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0589999999999999,"5min":1.206,"15min":1.206},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.91600000000000004}]},{"osd":5,"last update":"Fri May 10 05:31:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.53500000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.74399999999999999,"15min":0.72999999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.59099999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.095,"5min":1.119,"15min":1.119},"last":0.92300000000000004}]},{"osd":7,"last update":"Fri May 10 05:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.999}]},{"osd":8,"last update":"Fri May 10 05:31:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.753}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.86499999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":4,"up_from":32,"seq":137438953510,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":868352,"data_stored":412243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.36899999999999999,"15min":0.35699999999999998},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.38800000000000001},"min":{"1min":0.308,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.496,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.754}]},{"osd":3,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.47899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri May 10 05:31:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61599999999999999,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82599999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63400000000000001}]},{"osd":6,"last update":"Fri May 10 05:31:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.497,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55100000000000005}]},{"osd":7,"last update":"Fri May 10 05:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.81999999999999995}]},{"osd":8,"last update":"Fri May 10 05:31:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73399999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789986,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36136,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709016,"statfs":{"total":95995035648,"available":95958032384,"internally_reserved":0,"allocated":950272,"data_stored":1005180,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.622},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.61199999999999999,"15min":0.57099999999999995},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":4.524,"5min":4.524,"15min":4.524},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.47099999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.748},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.52400000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65100000000000002}]},{"osd":2,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.432},"min":{"1min":0.32100000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.438,"15min":0.436},"min":{"1min":0.316,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.83199999999999996}]},{"osd":4,"last update":"Fri May 10 05:31:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.495,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77800000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66000000000000003,"15min":0.65900000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri May 10 05:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":1}]},{"osd":8,"last update":"Fri May 10 05:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.97699999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":41851,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T05:32:02.105 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-10T05:32:02.106 INFO:tasks.cephadm:Setting up client nodes... 2024-05-10T05:32:02.106 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-10T05:32:02.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:02 smithi157 ceph-mon[78427]: pgmap v243: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-05-10T05:32:02.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:02 smithi157 ceph-mon[78427]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:02.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:02 smithi142 ceph-mon[77667]: pgmap v243: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-05-10T05:32:02.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:02 smithi142 ceph-mon[77667]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:02.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:02 smithi131 ceph-mon[37264]: pgmap v243: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-05-10T05:32:02.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:02 smithi131 ceph-mon[37264]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:03.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:03 smithi157 ceph-mon[78427]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:03.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:03 smithi157 ceph-mon[78427]: pgmap v244: 97 pgs: 97 active+clean; 627 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-05-10T05:32:03.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:03 smithi142 ceph-mon[77667]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:03.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:03 smithi142 ceph-mon[77667]: pgmap v244: 97 pgs: 97 active+clean; 627 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-05-10T05:32:03.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:03 smithi131 ceph-mon[37264]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:03.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:03 smithi131 ceph-mon[37264]: pgmap v244: 97 pgs: 97 active+clean; 627 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-05-10T05:32:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.820+0000 7fcf43200700 1 -- 172.21.15.131:0/1782979075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c104fa0 msgr2=0x7fcf3c1053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.820+0000 7fcf43200700 1 --2- 172.21.15.131:0/1782979075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c104fa0 0x7fcf3c1053b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009990 tx=0x7fcf28009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.821+0000 7fcf43200700 1 -- 172.21.15.131:0/1782979075 shutdown_connections 2024-05-10T05:32:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.821+0000 7fcf43200700 1 --2- 172.21.15.131:0/1782979075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c1058f0 0x7fcf3c10c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.821+0000 7fcf43200700 1 --2- 172.21.15.131:0/1782979075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c103dc0 0x7fcf3c1041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.821+0000 7fcf43200700 1 --2- 172.21.15.131:0/1782979075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c104fa0 0x7fcf3c1053b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.821+0000 7fcf43200700 1 -- 172.21.15.131:0/1782979075 >> 172.21.15.131:0/1782979075 conn(0x7fcf3c0ff530 msgr2=0x7fcf3c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.821+0000 7fcf43200700 1 -- 172.21.15.131:0/1782979075 shutdown_connections 2024-05-10T05:32:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.821+0000 7fcf43200700 1 -- 172.21.15.131:0/1782979075 wait complete. 2024-05-10T05:32:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.822+0000 7fcf43200700 1 Processor -- start 2024-05-10T05:32:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.822+0000 7fcf43200700 1 -- start start 2024-05-10T05:32:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.822+0000 7fcf43200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c103dc0 0x7fcf3c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.822+0000 7fcf43200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c104fa0 0x7fcf3c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.822+0000 7fcf43200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c1058f0 0x7fcf3c19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf43200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c19bce0 con 0x7fcf3c1058f0 2024-05-10T05:32:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf43200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c19be20 con 0x7fcf3c103dc0 2024-05-10T05:32:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf43200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c19bfe0 con 0x7fcf3c104fa0 2024-05-10T05:32:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf3b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c104fa0 0x7fcf3c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf3b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c104fa0 0x7fcf3c19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46380/0 (socket says 172.21.15.131:46380) 2024-05-10T05:32:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf41200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c1058f0 0x7fcf3c19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf3be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c103dc0 0x7fcf3c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf3b400700 1 -- 172.21.15.131:0/1881891661 learned_addr learned my addr 172.21.15.131:0/1881891661 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf3b400700 1 -- 172.21.15.131:0/1881891661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c103dc0 msgr2=0x7fcf3c19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf3b400700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c103dc0 0x7fcf3c19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf3b400700 1 -- 172.21.15.131:0/1881891661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c1058f0 msgr2=0x7fcf3c19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:03.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf3b400700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c1058f0 0x7fcf3c19b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf3b400700 1 -- 172.21.15.131:0/1881891661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf280095f0 con 0x7fcf3c104fa0 2024-05-10T05:32:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.823+0000 7fcf3be00700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c103dc0 0x7fcf3c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.824+0000 7fcf3b400700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c104fa0 0x7fcf3c19a8d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf28000c00 tx=0x7fcf28005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.824+0000 7fcf38c00700 1 -- 172.21.15.131:0/1881891661 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2801d070 con 0x7fcf3c104fa0 2024-05-10T05:32:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.824+0000 7fcf43200700 1 -- 172.21.15.131:0/1881891661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c19c260 con 0x7fcf3c104fa0 2024-05-10T05:32:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.824+0000 7fcf38c00700 1 -- 172.21.15.131:0/1881891661 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf28021410 con 0x7fcf3c104fa0 2024-05-10T05:32:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.824+0000 7fcf38c00700 1 -- 172.21.15.131:0/1881891661 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28017650 con 0x7fcf3c104fa0 2024-05-10T05:32:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.824+0000 7fcf43200700 1 -- 172.21.15.131:0/1881891661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c1a2aa0 con 0x7fcf3c104fa0 2024-05-10T05:32:03.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.825+0000 7fcf43200700 1 -- 172.21.15.131:0/1881891661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c10b120 con 0x7fcf3c104fa0 2024-05-10T05:32:03.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.827+0000 7fcf38c00700 1 -- 172.21.15.131:0/1881891661 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fcf28021580 con 0x7fcf3c104fa0 2024-05-10T05:32:03.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.828+0000 7fcf38c00700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcf200a11b0 0x7fcf200a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:03.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.829+0000 7fcf38c00700 1 -- 172.21.15.131:0/1881891661 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcf280cdd00 con 0x7fcf3c104fa0 2024-05-10T05:32:03.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.829+0000 7fcf3be00700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcf200a11b0 0x7fcf200a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:03.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.829+0000 7fcf3be00700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcf200a11b0 0x7fcf200a36c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c103b80 tx=0x7fcf2c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:03.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:03.833+0000 7fcf38c00700 1 -- 172.21.15.131:0/1881891661 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcf28097470 con 0x7fcf3c104fa0 2024-05-10T05:32:04.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.120+0000 7fcf43200700 1 -- 172.21.15.131:0/1881891661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fcf3c1031d0 con 0x7fcf3c104fa0 2024-05-10T05:32:04.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.128+0000 7fcf38c00700 1 -- 172.21.15.131:0/1881891661 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fcf2809aa90 con 0x7fcf3c104fa0 2024-05-10T05:32:04.129 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-05-10T05:32:04.129 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQBUsT1mbBVcBxAAje4DMQK8QyPPDxkyL59iTg== 2024-05-10T05:32:04.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.130+0000 7fcf43200700 1 -- 172.21.15.131:0/1881891661 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcf200a11b0 msgr2=0x7fcf200a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:04.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.130+0000 7fcf43200700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcf200a11b0 0x7fcf200a36c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c103b80 tx=0x7fcf2c009260 comp rx=0 tx=0).stop 2024-05-10T05:32:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.130+0000 7fcf43200700 1 -- 172.21.15.131:0/1881891661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c104fa0 msgr2=0x7fcf3c19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.130+0000 7fcf43200700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c104fa0 0x7fcf3c19a8d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf28000c00 tx=0x7fcf28005720 comp rx=0 tx=0).stop 2024-05-10T05:32:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.130+0000 7fcf43200700 1 -- 172.21.15.131:0/1881891661 shutdown_connections 2024-05-10T05:32:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.130+0000 7fcf43200700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcf200a11b0 0x7fcf200a36c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.130+0000 7fcf43200700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c1058f0 0x7fcf3c19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.130+0000 7fcf43200700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf3c103dc0 0x7fcf3c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.130+0000 7fcf43200700 1 --2- 172.21.15.131:0/1881891661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3c104fa0 0x7fcf3c19a8d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.130+0000 7fcf43200700 1 -- 172.21.15.131:0/1881891661 >> 172.21.15.131:0/1881891661 conn(0x7fcf3c0ff530 msgr2=0x7fcf3c108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.131+0000 7fcf43200700 1 -- 172.21.15.131:0/1881891661 shutdown_connections 2024-05-10T05:32:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:04.131+0000 7fcf43200700 1 -- 172.21.15.131:0/1881891661 wait complete. 2024-05-10T05:32:04.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:04 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1881891661' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T05:32:04.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:04 smithi131 ceph-mon[37264]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T05:32:04.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:04 smithi131 ceph-mon[37264]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T05:32:04.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:04 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1881891661' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T05:32:04.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:04 smithi157 ceph-mon[78427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T05:32:04.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:04 smithi157 ceph-mon[78427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T05:32:04.570 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:32:04.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-10T05:32:04.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-10T05:32:04.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:04 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1881891661' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T05:32:04.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:04 smithi142 ceph-mon[77667]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T05:32:04.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:04 smithi142 ceph-mon[77667]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T05:32:04.665 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-10T05:32:04.665 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-10T05:32:04.666 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-10T05:32:05.067 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:05.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:05 smithi131 ceph-mon[37264]: pgmap v245: 97 pgs: 97 active+clean; 627 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-05-10T05:32:05.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:05 smithi157 ceph-mon[78427]: pgmap v245: 97 pgs: 97 active+clean; 627 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-05-10T05:32:05.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:05 smithi142 ceph-mon[77667]: pgmap v245: 97 pgs: 97 active+clean; 627 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-05-10T05:32:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.359+0000 7fd281400700 1 -- 172.21.15.131:0/212982079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1071b0 msgr2=0x7fd27c10eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.359+0000 7fd281400700 1 --2- 172.21.15.131:0/212982079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1071b0 0x7fd27c10eb10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd274009f90 tx=0x7fd27400b330 comp rx=0 tx=0).stop 2024-05-10T05:32:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.360+0000 7fd281400700 1 -- 172.21.15.131:0/212982079 shutdown_connections 2024-05-10T05:32:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.360+0000 7fd281400700 1 --2- 172.21.15.131:0/212982079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1071b0 0x7fd27c10eb10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.360+0000 7fd281400700 1 --2- 172.21.15.131:0/212982079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd27c101f30 0x7fd27c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.360+0000 7fd281400700 1 --2- 172.21.15.131:0/212982079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd27c106860 0x7fd27c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.360+0000 7fd281400700 1 -- 172.21.15.131:0/212982079 >> 172.21.15.131:0/212982079 conn(0x7fd27c0fd9c0 msgr2=0x7fd27c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.360+0000 7fd281400700 1 -- 172.21.15.131:0/212982079 shutdown_connections 2024-05-10T05:32:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.360+0000 7fd281400700 1 -- 172.21.15.131:0/212982079 wait complete. 2024-05-10T05:32:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.361+0000 7fd281400700 1 Processor -- start 2024-05-10T05:32:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.361+0000 7fd281400700 1 -- start start 2024-05-10T05:32:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd281400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c101f30 0x7fd27c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd281400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd27c106860 0x7fd27c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd281400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd27c1071b0 0x7fd27c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd281400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19f800 con 0x7fd27c1071b0 2024-05-10T05:32:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd281400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19f940 con 0x7fd27c101f30 2024-05-10T05:32:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd281400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19fb00 con 0x7fd27c106860 2024-05-10T05:32:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd27b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd27c1071b0 0x7fd27c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd27a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c101f30 0x7fd27c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd27b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd27c1071b0 0x7fd27c19f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57454/0 (socket says 172.21.15.131:57454) 2024-05-10T05:32:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd27b200700 1 -- 172.21.15.131:0/1816961249 learned_addr learned my addr 172.21.15.131:0/1816961249 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:06.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.362+0000 7fd279e00700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd27c106860 0x7fd27c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:06.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd27b200700 1 -- 172.21.15.131:0/1816961249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd27c106860 msgr2=0x7fd27c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:06.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd27b200700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd27c106860 0x7fd27c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd27b200700 1 -- 172.21.15.131:0/1816961249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c101f30 msgr2=0x7fd27c19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd27b200700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c101f30 0x7fd27c19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd27b200700 1 -- 172.21.15.131:0/1816961249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274009c40 con 0x7fd27c1071b0 2024-05-10T05:32:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd27a800700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c101f30 0x7fd27c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd27b200700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd27c1071b0 0x7fd27c19f2c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2740094d0 tx=0x7fd274009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd273400700 1 -- 172.21.15.131:0/1816961249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27400e040 con 0x7fd27c1071b0 2024-05-10T05:32:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd273400700 1 -- 172.21.15.131:0/1816961249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd274007e40 con 0x7fd27c1071b0 2024-05-10T05:32:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd281400700 1 -- 172.21.15.131:0/1816961249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c19fd80 con 0x7fd27c1071b0 2024-05-10T05:32:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd273400700 1 -- 172.21.15.131:0/1816961249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274005040 con 0x7fd27c1071b0 2024-05-10T05:32:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.363+0000 7fd281400700 1 -- 172.21.15.131:0/1816961249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c1a7570 con 0x7fd27c1071b0 2024-05-10T05:32:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.364+0000 7fd281400700 1 -- 172.21.15.131:0/1816961249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27c1033e0 con 0x7fd27c1071b0 2024-05-10T05:32:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.366+0000 7fd273400700 1 -- 172.21.15.131:0/1816961249 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd27401b450 con 0x7fd27c1071b0 2024-05-10T05:32:06.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.368+0000 7fd273400700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd2600a1230 0x7fd2600a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:06.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.368+0000 7fd273400700 1 -- 172.21.15.131:0/1816961249 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd2740cdb60 con 0x7fd27c1071b0 2024-05-10T05:32:06.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.368+0000 7fd27a800700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd2600a1230 0x7fd2600a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:06.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.368+0000 7fd27a800700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd2600a1230 0x7fd2600a3740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd27c19c040 tx=0x7fd26c009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:06.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.372+0000 7fd273400700 1 -- 172.21.15.131:0/1816961249 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd2740972d0 con 0x7fd27c1071b0 2024-05-10T05:32:06.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.637+0000 7fd281400700 1 -- 172.21.15.131:0/1816961249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd27c04edc0 con 0x7fd27c1071b0 2024-05-10T05:32:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.642+0000 7fd273400700 1 -- 172.21.15.131:0/1816961249 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260289 (secure 0 0 0) 0x7fd27409a8f0 con 0x7fd27c1071b0 2024-05-10T05:32:06.648 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 -- 172.21.15.131:0/1816961249 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd2600a1230 msgr2=0x7fd2600a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd2600a1230 0x7fd2600a3740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd27c19c040 tx=0x7fd26c009310 comp rx=0 tx=0).stop 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 -- 172.21.15.131:0/1816961249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd27c1071b0 msgr2=0x7fd27c19f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd27c1071b0 0x7fd27c19f2c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2740094d0 tx=0x7fd274009500 comp rx=0 tx=0).stop 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 -- 172.21.15.131:0/1816961249 shutdown_connections 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd2600a1230 0x7fd2600a3740 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd27c1071b0 0x7fd27c19f2c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c101f30 0x7fd27c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 --2- 172.21.15.131:0/1816961249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd27c106860 0x7fd27c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 -- 172.21.15.131:0/1816961249 >> 172.21.15.131:0/1816961249 conn(0x7fd27c0fd9c0 msgr2=0x7fd27c10e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 -- 172.21.15.131:0/1816961249 shutdown_connections 2024-05-10T05:32:06.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:06.647+0000 7fd281400700 1 -- 172.21.15.131:0/1816961249 wait complete. 2024-05-10T05:32:06.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:06 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1816961249' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T05:32:07.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:06 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1816961249' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T05:32:07.102 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"flags":0,"active_gid":14154,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2109111580},{"type":"v1","addr":"172.21.15.131:6801","nonce":2109111580}]},"active_addr":"172.21.15.131:6801/2109111580","active_change":"2024-05-10T05:26:14.753928+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.131:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":19029692}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":50255390}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":980393774}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1960720190}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":374858816}]}]} 2024-05-10T05:32:07.107 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-10T05:32:07.107 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-10T05:32:07.108 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T05:32:07.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:06 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1816961249' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T05:32:07.471 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:07.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:07 smithi131 ceph-mon[37264]: pgmap v246: 97 pgs: 97 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-05-10T05:32:08.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:07 smithi157 ceph-mon[78427]: pgmap v246: 97 pgs: 97 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-05-10T05:32:08.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:07 smithi142 ceph-mon[77667]: pgmap v246: 97 pgs: 97 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-05-10T05:32:08.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.821+0000 7f35ffe00700 1 -- 172.21.15.131:0/1534956353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3600073780 msgr2=0x7f3600073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:08.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.821+0000 7f35ffe00700 1 --2- 172.21.15.131:0/1534956353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3600073780 0x7f3600073b90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009990 tx=0x7f35f0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:08.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.821+0000 7f35ffe00700 1 -- 172.21.15.131:0/1534956353 shutdown_connections 2024-05-10T05:32:08.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.821+0000 7f35ffe00700 1 --2- 172.21.15.131:0/1534956353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3600073780 0x7f3600073b90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:08.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.821+0000 7f35ffe00700 1 --2- 172.21.15.131:0/1534956353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360010e340 0x7f36001107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:08.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.821+0000 7f35ffe00700 1 --2- 172.21.15.131:0/1534956353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36000740d0 0x7f360010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:08.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.821+0000 7f35ffe00700 1 -- 172.21.15.131:0/1534956353 >> 172.21.15.131:0/1534956353 conn(0x7f3600077650 msgr2=0x7f3600077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:08.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.821+0000 7f35ffe00700 1 -- 172.21.15.131:0/1534956353 shutdown_connections 2024-05-10T05:32:08.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.822+0000 7f35ffe00700 1 -- 172.21.15.131:0/1534956353 wait complete. 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.823+0000 7f35ffe00700 1 Processor -- start 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.823+0000 7f35ffe00700 1 -- start start 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.823+0000 7f35ffe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3600073780 0x7f360019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.823+0000 7f35ffe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36000740d0 0x7f360019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.823+0000 7f35ffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360010e340 0x7f360019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.823+0000 7f35ffe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360019f3f0 con 0x7f36000740d0 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.823+0000 7f35ffe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360019f560 con 0x7f360010e340 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.823+0000 7f35ffe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360019f720 con 0x7f3600073780 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.823+0000 7f35fe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36000740d0 0x7f360019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f35fe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36000740d0 0x7f360019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57476/0 (socket says 172.21.15.131:57476) 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f35fe000700 1 -- 172.21.15.131:0/2161866994 learned_addr learned my addr 172.21.15.131:0/2161866994 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f35fea00700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3600073780 0x7f360019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f35ff400700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360010e340 0x7f360019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f35fe000700 1 -- 172.21.15.131:0/2161866994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3600073780 msgr2=0x7f360019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f35fe000700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3600073780 0x7f360019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f35fe000700 1 -- 172.21.15.131:0/2161866994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360010e340 msgr2=0x7f360019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f35fe000700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360010e340 0x7f360019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f35fe000700 1 -- 172.21.15.131:0/2161866994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00095f0 con 0x7f36000740d0 2024-05-10T05:32:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f35fe000700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36000740d0 0x7f360019ab80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00e9a0 tx=0x7f35ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.824+0000 7f3604a00700 1 -- 172.21.15.131:0/2161866994 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec009790 con 0x7f36000740d0 2024-05-10T05:32:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.825+0000 7f3604a00700 1 -- 172.21.15.131:0/2161866994 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35ec010430 con 0x7f36000740d0 2024-05-10T05:32:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.825+0000 7f35ffe00700 1 -- 172.21.15.131:0/2161866994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360019fa00 con 0x7f36000740d0 2024-05-10T05:32:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.825+0000 7f3604a00700 1 -- 172.21.15.131:0/2161866994 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec018680 con 0x7f36000740d0 2024-05-10T05:32:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.825+0000 7f35ffe00700 1 -- 172.21.15.131:0/2161866994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36001a7300 con 0x7f36000740d0 2024-05-10T05:32:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.826+0000 7f35ffe00700 1 -- 172.21.15.131:0/2161866994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3600109550 con 0x7f36000740d0 2024-05-10T05:32:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.829+0000 7f3604a00700 1 -- 172.21.15.131:0/2161866994 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f35ec0105a0 con 0x7f36000740d0 2024-05-10T05:32:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.831+0000 7f3604a00700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35e40a1230 0x7f35e40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.831+0000 7f3604a00700 1 -- 172.21.15.131:0/2161866994 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f35ec014070 con 0x7f36000740d0 2024-05-10T05:32:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.831+0000 7f35fea00700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35e40a1230 0x7f35e40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:08.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.832+0000 7f35fea00700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35e40a1230 0x7f35e40a3740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f35f000b2e0 tx=0x7f35f001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:08.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:08.833+0000 7f3604a00700 1 -- 172.21.15.131:0/2161866994 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f35ec096460 con 0x7f36000740d0 2024-05-10T05:32:09.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.038+0000 7f35ffe00700 1 -- 172.21.15.131:0/2161866994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f360019b940 con 0x7f36000740d0 2024-05-10T05:32:09.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.040+0000 7f3604a00700 1 -- 172.21.15.131:0/2161866994 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22191 (secure 0 0 0) 0x7f35ec026020 con 0x7f36000740d0 2024-05-10T05:32:09.041 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:09.041 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":80,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:31:52.550067+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 -- 172.21.15.131:0/2161866994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35e40a1230 msgr2=0x7f35e40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35e40a1230 0x7f35e40a3740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f35f000b2e0 tx=0x7f35f001a040 comp rx=0 tx=0).stop 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 -- 172.21.15.131:0/2161866994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36000740d0 msgr2=0x7f360019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36000740d0 0x7f360019ab80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00e9a0 tx=0x7f35ec00ecb0 comp rx=0 tx=0).stop 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 -- 172.21.15.131:0/2161866994 shutdown_connections 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f35e40a1230 0x7f35e40a3740 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360010e340 0x7f360019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3600073780 0x7f360019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 --2- 172.21.15.131:0/2161866994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36000740d0 0x7f360019ab80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 -- 172.21.15.131:0/2161866994 >> 172.21.15.131:0/2161866994 conn(0x7f3600077650 msgr2=0x7f36000fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 -- 172.21.15.131:0/2161866994 shutdown_connections 2024-05-10T05:32:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:09.043+0000 7f35ffe00700 1 -- 172.21.15.131:0/2161866994 wait complete. 2024-05-10T05:32:09.440 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-10T05:32:09.440 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T05:32:09.804 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:10.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:09 smithi142 ceph-mon[77667]: pgmap v247: 97 pgs: 97 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-05-10T05:32:10.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:09 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2161866994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:09 smithi131 ceph-mon[37264]: pgmap v247: 97 pgs: 97 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-05-10T05:32:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:09 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2161866994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:09 smithi157 ceph-mon[78427]: pgmap v247: 97 pgs: 97 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-05-10T05:32:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:09 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2161866994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:11.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.113+0000 7f6cb3800700 1 -- 172.21.15.131:0/709631943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cac106860 msgr2=0x7f6cac106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:11.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.114+0000 7f6cb3800700 1 --2- 172.21.15.131:0/709631943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cac106860 0x7f6cac106c70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009990 tx=0x7f6c98009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.114+0000 7f6cb3800700 1 -- 172.21.15.131:0/709631943 shutdown_connections 2024-05-10T05:32:11.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.114+0000 7f6cb3800700 1 --2- 172.21.15.131:0/709631943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cac1071b0 0x7f6cac10eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.114+0000 7f6cb3800700 1 --2- 172.21.15.131:0/709631943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac101f30 0x7f6cac102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.114+0000 7f6cb3800700 1 --2- 172.21.15.131:0/709631943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cac106860 0x7f6cac106c70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.114+0000 7f6cb3800700 1 -- 172.21.15.131:0/709631943 >> 172.21.15.131:0/709631943 conn(0x7f6cac0fd9c0 msgr2=0x7f6cac0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:11.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.114+0000 7f6cb3800700 1 -- 172.21.15.131:0/709631943 shutdown_connections 2024-05-10T05:32:11.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.114+0000 7f6cb3800700 1 -- 172.21.15.131:0/709631943 wait complete. 2024-05-10T05:32:11.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.115+0000 7f6cb3800700 1 Processor -- start 2024-05-10T05:32:11.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.115+0000 7f6cb3800700 1 -- start start 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.115+0000 7f6cb3800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cac101f30 0x7f6cac1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.115+0000 7f6cb3800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac106860 0x7f6cac1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.115+0000 7f6cb3800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cac1071b0 0x7f6cac19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb3800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac19abe0 con 0x7f6cac1071b0 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb3800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac19ad50 con 0x7f6cac101f30 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb3800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac19af10 con 0x7f6cac106860 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cac101f30 0x7f6cac1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cac101f30 0x7f6cac1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41652/0 (socket says 172.21.15.131:41652) 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb0e00700 1 -- 172.21.15.131:0/843528602 learned_addr learned my addr 172.21.15.131:0/843528602 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cabe00700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac106860 0x7f6cac1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb1800700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cac1071b0 0x7f6cac19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:11.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb0e00700 1 -- 172.21.15.131:0/843528602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac106860 msgr2=0x7f6cac1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb0e00700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac106860 0x7f6cac1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb0e00700 1 -- 172.21.15.131:0/843528602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cac1071b0 msgr2=0x7f6cac19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb0e00700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cac1071b0 0x7f6cac19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb0e00700 1 -- 172.21.15.131:0/843528602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c980095f0 con 0x7f6cac101f30 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.116+0000 7f6cb0e00700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cac101f30 0x7f6cac1a0620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000e9a0 tx=0x7f6ca000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.117+0000 7f6ca9600700 1 -- 172.21.15.131:0/843528602 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0009790 con 0x7f6cac101f30 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.117+0000 7f6cb3800700 1 -- 172.21.15.131:0/843528602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac19b1f0 con 0x7f6cac101f30 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.117+0000 7f6ca9600700 1 -- 172.21.15.131:0/843528602 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ca0010430 con 0x7f6cac101f30 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.117+0000 7f6ca9600700 1 -- 172.21.15.131:0/843528602 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0018680 con 0x7f6cac101f30 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.117+0000 7f6cb3800700 1 -- 172.21.15.131:0/843528602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac1a7650 con 0x7f6cac101f30 2024-05-10T05:32:11.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.118+0000 7f6cb3800700 1 -- 172.21.15.131:0/843528602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac1033e0 con 0x7f6cac101f30 2024-05-10T05:32:11.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.120+0000 7f6ca9600700 1 -- 172.21.15.131:0/843528602 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6ca00105a0 con 0x7f6cac101f30 2024-05-10T05:32:11.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.121+0000 7f6ca9600700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6c940a11b0 0x7f6c940a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:11.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.121+0000 7f6ca9600700 1 -- 172.21.15.131:0/843528602 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6ca0014070 con 0x7f6cac101f30 2024-05-10T05:32:11.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.122+0000 7f6cabe00700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6c940a11b0 0x7f6c940a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:11.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.122+0000 7f6cabe00700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6c940a11b0 0x7f6c940a36c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6cac10e8e0 tx=0x7f6c9801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.126+0000 7f6ca9600700 1 -- 172.21.15.131:0/843528602 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6ca0052d60 con 0x7f6cac101f30 2024-05-10T05:32:11.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.330+0000 7f6cb3800700 1 -- 172.21.15.131:0/843528602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cac04edc0 con 0x7f6cac101f30 2024-05-10T05:32:11.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.332+0000 7f6ca9600700 1 -- 172.21.15.131:0/843528602 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22191 (secure 0 0 0) 0x7f6ca0096260 con 0x7f6cac101f30 2024-05-10T05:32:11.333 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:11.333 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":80,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:31:52.550067+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:32:11.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.334+0000 7f6cb3800700 1 -- 172.21.15.131:0/843528602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6c940a11b0 msgr2=0x7f6c940a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:11.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.334+0000 7f6cb3800700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6c940a11b0 0x7f6c940a36c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6cac10e8e0 tx=0x7f6c9801a040 comp rx=0 tx=0).stop 2024-05-10T05:32:11.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.334+0000 7f6cb3800700 1 -- 172.21.15.131:0/843528602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cac101f30 msgr2=0x7f6cac1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:11.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.334+0000 7f6cb3800700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cac101f30 0x7f6cac1a0620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000e9a0 tx=0x7f6ca000ecb0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.334+0000 7f6cb3800700 1 -- 172.21.15.131:0/843528602 shutdown_connections 2024-05-10T05:32:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.334+0000 7f6cb3800700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6c940a11b0 0x7f6c940a36c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.334+0000 7f6cb3800700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cac1071b0 0x7f6cac19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.334+0000 7f6cb3800700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cac101f30 0x7f6cac1a0620 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.334+0000 7f6cb3800700 1 --2- 172.21.15.131:0/843528602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cac106860 0x7f6cac1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.334+0000 7f6cb3800700 1 -- 172.21.15.131:0/843528602 >> 172.21.15.131:0/843528602 conn(0x7f6cac0fd9c0 msgr2=0x7f6cac108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.335+0000 7f6cb3800700 1 -- 172.21.15.131:0/843528602 shutdown_connections 2024-05-10T05:32:11.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:11.335+0000 7f6cb3800700 1 -- 172.21.15.131:0/843528602 wait complete. 2024-05-10T05:32:11.723 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-10T05:32:11.723 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-10T05:32:11.724 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-10T05:32:11.724 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-10T05:32:11.725 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-10T05:32:11.725 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-10T05:32:11.725 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-10T05:32:11.726 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-10T05:32:11.726 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-10T05:32:11.727 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-10T05:32:11.727 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-10T05:32:11.728 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-10T05:32:11.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:11 smithi131 ceph-mon[37264]: pgmap v248: 97 pgs: 97 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 19 op/s 2024-05-10T05:32:11.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:11 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/843528602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:12.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:11 smithi157 ceph-mon[78427]: pgmap v248: 97 pgs: 97 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 19 op/s 2024-05-10T05:32:12.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:11 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/843528602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:12.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:11 smithi142 ceph-mon[77667]: pgmap v248: 97 pgs: 97 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 19 op/s 2024-05-10T05:32:12.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:11 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/843528602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:13.503 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:13.554 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:13.605 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:13.911 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:13.912 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:13.964 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:14.105 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:14.113 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:13 smithi131 ceph-mon[37264]: pgmap v249: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2024-05-10T05:32:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:13 smithi157 ceph-mon[78427]: pgmap v249: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2024-05-10T05:32:14.358 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:13 smithi142 ceph-mon[77667]: pgmap v249: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2024-05-10T05:32:14.560 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-10T05:32:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-10T05:32:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-05-10T05:32:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 3, 6]}]: dispatch 2024-05-10T05:32:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7, 5, 9]}]: dispatch 2024-05-10T05:32:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-10T05:32:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-10T05:32:14.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-05-10T05:32:15.019 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:15.125 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-10T05:32:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-10T05:32:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-05-10T05:32:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 3, 6]}]: dispatch 2024-05-10T05:32:15.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7, 5, 9]}]: dispatch 2024-05-10T05:32:15.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-10T05:32:15.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-10T05:32:15.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-05-10T05:32:15.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-10T05:32:15.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-10T05:32:15.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-05-10T05:32:15.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 3, 6]}]: dispatch 2024-05-10T05:32:15.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7, 5, 9]}]: dispatch 2024-05-10T05:32:15.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-10T05:32:15.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-10T05:32:15.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-05-10T05:32:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:16.163 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:15 smithi131 ceph-mon[37264]: pgmap v250: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-10T05:32:16.163 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-10T05:32:16.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-10T05:32:16.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-05-10T05:32:16.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 3, 6]}]': finished 2024-05-10T05:32:16.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7, 5, 9]}]': finished 2024-05-10T05:32:16.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-10T05:32:16.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-10T05:32:16.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:15 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-05-10T05:32:16.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:15 smithi131 ceph-mon[37264]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T05:32:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:15 smithi157 ceph-mon[78427]: pgmap v250: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-10T05:32:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-10T05:32:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-10T05:32:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-05-10T05:32:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 3, 6]}]': finished 2024-05-10T05:32:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7, 5, 9]}]': finished 2024-05-10T05:32:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-10T05:32:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-10T05:32:16.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:15 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-05-10T05:32:16.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:15 smithi157 ceph-mon[78427]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T05:32:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:15 smithi142 ceph-mon[77667]: pgmap v250: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-10T05:32:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-10T05:32:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-10T05:32:16.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-05-10T05:32:16.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 3, 6]}]': finished 2024-05-10T05:32:16.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7, 5, 9]}]': finished 2024-05-10T05:32:16.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-10T05:32:16.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-10T05:32:16.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:15 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-05-10T05:32:16.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:15 smithi142 ceph-mon[77667]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T05:32:17.002 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:16 smithi131 ceph-mon[37264]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T05:32:17.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:16 smithi157 ceph-mon[78427]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T05:32:17.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:16 smithi142 ceph-mon[77667]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T05:32:18.209 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:17 smithi131 ceph-mon[37264]: pgmap v253: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 895 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:18.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:17 smithi157 ceph-mon[78427]: pgmap v253: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 895 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:18.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:17 smithi142 ceph-mon[77667]: pgmap v253: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 895 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:19 smithi131 ceph-mon[37264]: pgmap v254: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 895 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:20.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:19 smithi157 ceph-mon[78427]: pgmap v254: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 895 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.392+0000 7fa94d800700 1 -- 172.21.15.131:0/99880802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9480740d0 msgr2=0x7fa94810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.392+0000 7fa94d800700 1 --2- 172.21.15.131:0/99880802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9480740d0 0x7fa94810dfb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa9340098f0 tx=0x7fa934009c00 comp rx=0 tx=0).stop 2024-05-10T05:32:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.392+0000 7fa94d800700 1 -- 172.21.15.131:0/99880802 shutdown_connections 2024-05-10T05:32:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.392+0000 7fa94d800700 1 --2- 172.21.15.131:0/99880802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa948073780 0x7fa948073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.392+0000 7fa94d800700 1 --2- 172.21.15.131:0/99880802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94810e4f0 0x7fa948110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.392+0000 7fa94d800700 1 --2- 172.21.15.131:0/99880802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9480740d0 0x7fa94810dfb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.392+0000 7fa94d800700 1 -- 172.21.15.131:0/99880802 >> 172.21.15.131:0/99880802 conn(0x7fa9480fd7f0 msgr2=0x7fa9480ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.392+0000 7fa94d800700 1 -- 172.21.15.131:0/99880802 shutdown_connections 2024-05-10T05:32:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.392+0000 7fa94d800700 1 -- 172.21.15.131:0/99880802 wait complete. 2024-05-10T05:32:20.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:19 smithi142 ceph-mon[77667]: pgmap v254: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 895 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.394+0000 7fa94d800700 1 Processor -- start 2024-05-10T05:32:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.394+0000 7fa94d800700 1 -- start start 2024-05-10T05:32:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa94d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa948073780 0x7fa9481a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa94d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9480740d0 0x7fa9481a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa94d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94810e4f0 0x7fa94819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa94d800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94819ab80 con 0x7fa9480740d0 2024-05-10T05:32:20.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa94d800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94819acc0 con 0x7fa948073780 2024-05-10T05:32:20.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa94d800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94819ae80 con 0x7fa94810e4f0 2024-05-10T05:32:20.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa947400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94810e4f0 0x7fa94819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa946000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9480740d0 0x7fa9481a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa947400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94810e4f0 0x7fa94819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37126/0 (socket says 172.21.15.131:37126) 2024-05-10T05:32:20.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa947400700 1 -- 172.21.15.131:0/1232405512 learned_addr learned my addr 172.21.15.131:0/1232405512 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:20.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fa946a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa948073780 0x7fa9481a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa947400700 1 -- 172.21.15.131:0/1232405512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa948073780 msgr2=0x7fa9481a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa947400700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa948073780 0x7fa9481a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa947400700 1 -- 172.21.15.131:0/1232405512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9480740d0 msgr2=0x7fa9481a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa947400700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9480740d0 0x7fa9481a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa947400700 1 -- 172.21.15.131:0/1232405512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa934009520 con 0x7fa94810e4f0 2024-05-10T05:32:20.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa947400700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94810e4f0 0x7fa94819a5b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa94000ca20 tx=0x7fa94000cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:20.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa93f400700 1 -- 172.21.15.131:0/1232405512 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940009e70 con 0x7fa94810e4f0 2024-05-10T05:32:20.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94819b160 con 0x7fa94810e4f0 2024-05-10T05:32:20.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa93f400700 1 -- 172.21.15.131:0/1232405512 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa940004bb0 con 0x7fa94810e4f0 2024-05-10T05:32:20.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa93f400700 1 -- 172.21.15.131:0/1232405512 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940004810 con 0x7fa94810e4f0 2024-05-10T05:32:20.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa948101d10 con 0x7fa94810e4f0 2024-05-10T05:32:20.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.398+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa948074cf0 con 0x7fa94810e4f0 2024-05-10T05:32:20.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.401+0000 7fa93f400700 1 -- 172.21.15.131:0/1232405512 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa940004d20 con 0x7fa94810e4f0 2024-05-10T05:32:20.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.403+0000 7fa93f400700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa92c0a1230 0x7fa92c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.403+0000 7fa93f400700 1 -- 172.21.15.131:0/1232405512 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa940016e40 con 0x7fa94810e4f0 2024-05-10T05:32:20.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.403+0000 7fa946a00700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa92c0a1230 0x7fa92c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.403+0000 7fa93f400700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] conn(0x7fa92c0aadc0 0x7fa92c0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.403+0000 7fa93f400700 1 -- 172.21.15.131:0/1232405512 --> [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa92c0ad980 con 0x7fa92c0aadc0 2024-05-10T05:32:20.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.403+0000 7fa93f400700 1 -- 172.21.15.131:0/1232405512 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9400d6bc0 con 0x7fa94810e4f0 2024-05-10T05:32:20.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.404+0000 7fa946000700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] conn(0x7fa92c0aadc0 0x7fa92c0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.404+0000 7fa946a00700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa92c0a1230 0x7fa92c0a3740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa948100e10 tx=0x7fa938009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:20.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.404+0000 7fa946000700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] conn(0x7fa92c0aadc0 0x7fa92c0ad2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:20.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.405+0000 7fa93f400700 1 -- 172.21.15.131:0/1232405512 <== osd.1 v2:172.21.15.142:6800/451743952 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa92c0ad980 con 0x7fa92c0aadc0 2024-05-10T05:32:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.437+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 --> [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa9481140d0 con 0x7fa92c0aadc0 2024-05-10T05:32:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.438+0000 7fa93f400700 1 -- 172.21.15.131:0/1232405512 <== osd.1 v2:172.21.15.142:6800/451743952 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa93401d040 con 0x7fa92c0aadc0 2024-05-10T05:32:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.438+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 >> [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] conn(0x7fa92c0aadc0 msgr2=0x7fa92c0ad2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.439+0000 7fa94d800700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] conn(0x7fa92c0aadc0 0x7fa92c0ad2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.439+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa92c0a1230 msgr2=0x7fa92c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.439+0000 7fa94d800700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa92c0a1230 0x7fa92c0a3740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa948100e10 tx=0x7fa938009310 comp rx=0 tx=0).stop 2024-05-10T05:32:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.439+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94810e4f0 msgr2=0x7fa94819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.439+0000 7fa94d800700 1 --2- 172.21.15.131:0/1232405512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa94810e4f0 0x7fa94819a5b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa94000ca20 tx=0x7fa94000cd30 comp rx=0 tx=0).stop 2024-05-10T05:32:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.439+0000 7fa947400700 1 -- 172.21.15.131:0/1232405512 reap_dead start 2024-05-10T05:32:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.439+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 shutdown_connections 2024-05-10T05:32:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.439+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 >> 172.21.15.131:0/1232405512 conn(0x7fa9480fd7f0 msgr2=0x7fa9480fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.439+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 shutdown_connections 2024-05-10T05:32:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.439+0000 7fa94d800700 1 -- 172.21.15.131:0/1232405512 wait complete. 2024-05-10T05:32:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.534+0000 7f1775400700 1 -- 172.21.15.131:0/2485852395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1770102da0 msgr2=0x7f17701031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.534+0000 7f1775400700 1 --2- 172.21.15.131:0/2485852395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1770102da0 0x7f17701031b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1760009990 tx=0x7f1760009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.534+0000 7f1775400700 1 -- 172.21.15.131:0/2485852395 shutdown_connections 2024-05-10T05:32:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.534+0000 7f1775400700 1 --2- 172.21.15.131:0/2485852395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f177010c210 0x7f177010e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.534+0000 7f1775400700 1 --2- 172.21.15.131:0/2485852395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1770102da0 0x7f17701031b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.534+0000 7f1775400700 1 --2- 172.21.15.131:0/2485852395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17701036f0 0x7f177010bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.534+0000 7f1775400700 1 -- 172.21.15.131:0/2485852395 >> 172.21.15.131:0/2485852395 conn(0x7f17700fc7a0 msgr2=0x7f17700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.535+0000 7f1775400700 1 -- 172.21.15.131:0/2485852395 shutdown_connections 2024-05-10T05:32:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.535+0000 7f1775400700 1 -- 172.21.15.131:0/2485852395 wait complete. 2024-05-10T05:32:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.536+0000 7f1775400700 1 Processor -- start 2024-05-10T05:32:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.536+0000 7f1775400700 1 -- start start 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.537+0000 7f1775400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1770102da0 0x7f17701a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.537+0000 7f1775400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17701036f0 0x7f17701a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.537+0000 7f1775400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f177010c210 0x7f177019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.537+0000 7f1775400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177019ab40 con 0x7f177010c210 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.537+0000 7f1775400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177019ac80 con 0x7f17701036f0 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.537+0000 7f1775400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177019ae40 con 0x7f1770102da0 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.538+0000 7f176e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17701036f0 0x7f17701a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.538+0000 7f176f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f177010c210 0x7f177019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.538+0000 7f176e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17701036f0 0x7f17701a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43982/0 (socket says 172.21.15.131:43982) 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.538+0000 7f176e800700 1 -- 172.21.15.131:0/1114952417 learned_addr learned my addr 172.21.15.131:0/1114952417 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.539+0000 7f1767e00700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1770102da0 0x7f17701a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.539+0000 7f176f200700 1 -- 172.21.15.131:0/1114952417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1770102da0 msgr2=0x7f17701a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.539+0000 7f176f200700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1770102da0 0x7f17701a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.539+0000 7f176f200700 1 -- 172.21.15.131:0/1114952417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17701036f0 msgr2=0x7f17701a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.539+0000 7f176f200700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17701036f0 0x7f17701a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.539+0000 7f176f200700 1 -- 172.21.15.131:0/1114952417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17600095f0 con 0x7f177010c210 2024-05-10T05:32:20.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.539+0000 7f176e800700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17701036f0 0x7f17701a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:20.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.540+0000 7f176f200700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f177010c210 0x7f177019a570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f176800efc0 tx=0x7f176800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:20.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.540+0000 7f1767400700 1 -- 172.21.15.131:0/1114952417 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1768018070 con 0x7f177010c210 2024-05-10T05:32:20.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.540+0000 7f1767400700 1 -- 172.21.15.131:0/1114952417 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f176800f040 con 0x7f177010c210 2024-05-10T05:32:20.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.540+0000 7f1767400700 1 -- 172.21.15.131:0/1114952417 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176800cdf0 con 0x7f177010c210 2024-05-10T05:32:20.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.540+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177019b0c0 con 0x7f177010c210 2024-05-10T05:32:20.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.542+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17701a7410 con 0x7f177010c210 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.543+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1770069600 con 0x7f177010c210 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.546+0000 7f1767400700 1 -- 172.21.15.131:0/1114952417 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1768007500 con 0x7f177010c210 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.547+0000 7f1767400700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17500a1230 0x7f17500a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.548+0000 7f1767400700 1 -- 172.21.15.131:0/1114952417 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f17680cc230 con 0x7f177010c210 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.548+0000 7f1767400700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] conn(0x7f17500aadc0 0x7f17500ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.548+0000 7f1767400700 1 -- 172.21.15.131:0/1114952417 --> [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17500ad980 con 0x7f17500aadc0 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.548+0000 7f1767400700 1 -- 172.21.15.131:0/1114952417 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17680ce8b0 con 0x7f177010c210 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.548+0000 7f1767e00700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17500a1230 0x7f17500a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.549+0000 7f176e800700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] conn(0x7f17500aadc0 0x7f17500ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.549+0000 7f1767e00700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17500a1230 0x7f17500a3740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f176000b2e0 tx=0x7f1760005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.550+0000 7f176e800700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] conn(0x7f17500aadc0 0x7f17500ad2b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.551+0000 7f1767400700 1 -- 172.21.15.131:0/1114952417 <== osd.5 v2:172.21.15.157:6808/1157754257 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f17500ad980 con 0x7f17500aadc0 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.575+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 --> [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1770111e90 con 0x7f17500aadc0 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.575+0000 7f1767400700 1 -- 172.21.15.131:0/1114952417 <== osd.5 v2:172.21.15.157:6808/1157754257 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1758009e40 con 0x7f17500aadc0 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.576+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 >> [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] conn(0x7f17500aadc0 msgr2=0x7f17500ad2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.576+0000 7f1775400700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] conn(0x7f17500aadc0 0x7f17500ad2b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.576+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17500a1230 msgr2=0x7f17500a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.576+0000 7f1775400700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17500a1230 0x7f17500a3740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f176000b2e0 tx=0x7f1760005fb0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.576+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f177010c210 msgr2=0x7f177019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.576+0000 7f1775400700 1 --2- 172.21.15.131:0/1114952417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f177010c210 0x7f177019a570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f176800efc0 tx=0x7f176800c3a0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.576+0000 7f176f200700 1 -- 172.21.15.131:0/1114952417 reap_dead start 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.577+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 shutdown_connections 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.577+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 >> 172.21.15.131:0/1114952417 conn(0x7f17700fc7a0 msgr2=0x7f17700fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.577+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 shutdown_connections 2024-05-10T05:32:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.577+0000 7f1775400700 1 -- 172.21.15.131:0/1114952417 wait complete. 2024-05-10T05:32:20.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.545+0000 7f596b800700 1 -- 172.21.15.131:0/1702024173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59641071c0 msgr2=0x7f596410eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.545+0000 7f596b800700 1 --2- 172.21.15.131:0/1702024173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59641071c0 0x7f596410eb20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f595c00b0a0 tx=0x7f595c00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.546+0000 7f596b800700 1 -- 172.21.15.131:0/1702024173 shutdown_connections 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.546+0000 7f596b800700 1 --2- 172.21.15.131:0/1702024173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59641071c0 0x7f596410eb20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.546+0000 7f596b800700 1 --2- 172.21.15.131:0/1702024173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5964101f40 0x7f5964102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.546+0000 7f596b800700 1 --2- 172.21.15.131:0/1702024173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5964106870 0x7f5964106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.546+0000 7f596b800700 1 -- 172.21.15.131:0/1702024173 >> 172.21.15.131:0/1702024173 conn(0x7f59640fd7f0 msgr2=0x7f59640ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.546+0000 7f596b800700 1 -- 172.21.15.131:0/1702024173 shutdown_connections 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.546+0000 7f596b800700 1 -- 172.21.15.131:0/1702024173 wait complete. 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.547+0000 7f596b800700 1 Processor -- start 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.547+0000 7f596b800700 1 -- start start 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f596b800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5964101f40 0x7f596419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f596b800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5964106870 0x7f596419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f596b800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59641071c0 0x7f5964198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f596b800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59641989e0 con 0x7f5964101f40 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f596b800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5964198b50 con 0x7f59641071c0 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f596b800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5964198d10 con 0x7f5964106870 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f5968e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5964101f40 0x7f596419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f5968e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5964101f40 0x7f596419e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49662/0 (socket says 172.21.15.131:49662) 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f5968e00700 1 -- 172.21.15.131:0/2768793924 learned_addr learned my addr 172.21.15.131:0/2768793924 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f5963e00700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5964106870 0x7f596419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.548+0000 7f5969800700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59641071c0 0x7f5964198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.549+0000 7f5963e00700 1 -- 172.21.15.131:0/2768793924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59641071c0 msgr2=0x7f5964198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.549+0000 7f5963e00700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59641071c0 0x7f5964198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.549+0000 7f5963e00700 1 -- 172.21.15.131:0/2768793924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5964101f40 msgr2=0x7f596419e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.549+0000 7f5963e00700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5964101f40 0x7f596419e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.549+0000 7f5963e00700 1 -- 172.21.15.131:0/2768793924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c009d00 con 0x7f5964106870 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.549+0000 7f5968e00700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5964101f40 0x7f596419e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.549+0000 7f5963e00700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5964106870 0x7f596419eaa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f595000c940 tx=0x7f595000cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.549+0000 7f5961600700 1 -- 172.21.15.131:0/2768793924 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950004440 con 0x7f5964106870 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.550+0000 7f5961600700 1 -- 172.21.15.131:0/2768793924 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5950003b40 con 0x7f5964106870 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.550+0000 7f5961600700 1 -- 172.21.15.131:0/2768793924 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950007b90 con 0x7f5964106870 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.550+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5964198ff0 con 0x7f5964106870 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.550+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5964199320 con 0x7f5964106870 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.551+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5964111e70 con 0x7f5964106870 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.554+0000 7f5961600700 1 -- 172.21.15.131:0/2768793924 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5950003cb0 con 0x7f5964106870 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.555+0000 7f5961600700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f594c0a1230 0x7f594c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.555+0000 7f5961600700 1 -- 172.21.15.131:0/2768793924 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f59500ccca0 con 0x7f5964106870 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.556+0000 7f5961600700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] conn(0x7f594c0aadc0 0x7f594c0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.556+0000 7f5961600700 1 -- 172.21.15.131:0/2768793924 --> [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f594c0ad980 con 0x7f594c0aadc0 2024-05-10T05:32:20.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.556+0000 7f5961600700 1 -- 172.21.15.131:0/2768793924 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59500cd010 con 0x7f5964106870 2024-05-10T05:32:20.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.556+0000 7f5968e00700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f594c0a1230 0x7f594c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.556+0000 7f5969800700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] conn(0x7f594c0aadc0 0x7f594c0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:20.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.557+0000 7f5968e00700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f594c0a1230 0x7f594c0a3740 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f595800f4d0 tx=0x7f5958005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:20.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.557+0000 7f5969800700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] conn(0x7f594c0aadc0 0x7f594c0ad2b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:20.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.558+0000 7f5961600700 1 -- 172.21.15.131:0/2768793924 <== osd.8 v2:172.21.15.157:6816/2904635811 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f594c0ad980 con 0x7f594c0aadc0 2024-05-10T05:32:20.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.577+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 --> [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5964061910 con 0x7f594c0aadc0 2024-05-10T05:32:20.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.578+0000 7f5961600700 1 -- 172.21.15.131:0/2768793924 <== osd.8 v2:172.21.15.157:6816/2904635811 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5964061910 con 0x7f594c0aadc0 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.578+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 >> [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] conn(0x7f594c0aadc0 msgr2=0x7f594c0ad2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.578+0000 7f596b800700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] conn(0x7f594c0aadc0 0x7f594c0ad2b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.578+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f594c0a1230 msgr2=0x7f594c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.578+0000 7f596b800700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f594c0a1230 0x7f594c0a3740 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f595800f4d0 tx=0x7f5958005f90 comp rx=0 tx=0).stop 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.578+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5964106870 msgr2=0x7f596419eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.578+0000 7f596b800700 1 --2- 172.21.15.131:0/2768793924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5964106870 0x7f596419eaa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f595000c940 tx=0x7f595000cc50 comp rx=0 tx=0).stop 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.578+0000 7f5969800700 1 -- 172.21.15.131:0/2768793924 reap_dead start 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.578+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 shutdown_connections 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.578+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 >> 172.21.15.131:0/2768793924 conn(0x7f59640fd7f0 msgr2=0x7f5964106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.579+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 shutdown_connections 2024-05-10T05:32:20.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:18.579+0000 7f596b800700 1 -- 172.21.15.131:0/2768793924 wait complete. 2024-05-10T05:32:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.258+0000 7f9728a00700 1 -- 172.21.15.131:0/3148581765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9724101f30 msgr2=0x7f9724102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.258+0000 7f9728a00700 1 --2- 172.21.15.131:0/3148581765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9724101f30 0x7f9724102340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9714009990 tx=0x7f9714009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.258+0000 7f9728a00700 1 -- 172.21.15.131:0/3148581765 shutdown_connections 2024-05-10T05:32:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.258+0000 7f9728a00700 1 --2- 172.21.15.131:0/3148581765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97241071b0 0x7f972410eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.258+0000 7f9728a00700 1 --2- 172.21.15.131:0/3148581765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9724101f30 0x7f9724102340 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.258+0000 7f9728a00700 1 --2- 172.21.15.131:0/3148581765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9724106860 0x7f9724106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.258+0000 7f9728a00700 1 -- 172.21.15.131:0/3148581765 >> 172.21.15.131:0/3148581765 conn(0x7f97240fd9c0 msgr2=0x7f97240ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.258+0000 7f9728a00700 1 -- 172.21.15.131:0/3148581765 shutdown_connections 2024-05-10T05:32:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.258+0000 7f9728a00700 1 -- 172.21.15.131:0/3148581765 wait complete. 2024-05-10T05:32:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.259+0000 7f9728a00700 1 Processor -- start 2024-05-10T05:32:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9728a00700 1 -- start start 2024-05-10T05:32:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9728a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9724101f30 0x7f9724198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9728a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9724106860 0x7f9724198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9728a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97241071b0 0x7f972419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9728a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972419d520 con 0x7f9724106860 2024-05-10T05:32:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9728a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972419d690 con 0x7f97241071b0 2024-05-10T05:32:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9728a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972419d850 con 0x7f9724101f30 2024-05-10T05:32:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9721e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9724101f30 0x7f9724198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9721400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9724106860 0x7f9724198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9721e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9724101f30 0x7f9724198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37160/0 (socket says 172.21.15.131:37160) 2024-05-10T05:32:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.260+0000 7f9721e00700 1 -- 172.21.15.131:0/1526848285 learned_addr learned my addr 172.21.15.131:0/1526848285 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f9722800700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97241071b0 0x7f972419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f9721400700 1 -- 172.21.15.131:0/1526848285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9724101f30 msgr2=0x7f9724198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f9721400700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9724101f30 0x7f9724198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f9721400700 1 -- 172.21.15.131:0/1526848285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97241071b0 msgr2=0x7f972419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f9721400700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97241071b0 0x7f972419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f9721400700 1 -- 172.21.15.131:0/1526848285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97140095f0 con 0x7f9724106860 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f9721e00700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9724101f30 0x7f9724198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f9721400700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9724106860 0x7f9724198aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f971000e9c0 tx=0x7f971000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f971aa00700 1 -- 172.21.15.131:0/1526848285 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971000c9c0 con 0x7f9724106860 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f971aa00700 1 -- 172.21.15.131:0/1526848285 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9710010430 con 0x7f9724106860 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972419db30 con 0x7f9724106860 2024-05-10T05:32:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.261+0000 7f971aa00700 1 -- 172.21.15.131:0/1526848285 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710018680 con 0x7f9724106860 2024-05-10T05:32:21.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.262+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97241a5430 con 0x7f9724106860 2024-05-10T05:32:21.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.263+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97241033e0 con 0x7f9724106860 2024-05-10T05:32:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.266+0000 7f971aa00700 1 -- 172.21.15.131:0/1526848285 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f971000cb20 con 0x7f9724106860 2024-05-10T05:32:21.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.267+0000 7f971aa00700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97080a11b0 0x7f97080a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.267+0000 7f971aa00700 1 -- 172.21.15.131:0/1526848285 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9710014070 con 0x7f9724106860 2024-05-10T05:32:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.268+0000 7f9721e00700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97080a11b0 0x7f97080a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.268+0000 7f971aa00700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] conn(0x7f97080aad40 0x7f97080ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.268+0000 7f971aa00700 1 -- 172.21.15.131:0/1526848285 --> [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97080ad900 con 0x7f97080aad40 2024-05-10T05:32:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.268+0000 7f971aa00700 1 -- 172.21.15.131:0/1526848285 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97100cd230 con 0x7f9724106860 2024-05-10T05:32:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.268+0000 7f9722800700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] conn(0x7f97080aad40 0x7f97080ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.268+0000 7f9721e00700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97080a11b0 0x7f97080a36c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9714006010 tx=0x7f971401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:21.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.269+0000 7f9722800700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] conn(0x7f97080aad40 0x7f97080ad230 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:21.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.270+0000 7f971aa00700 1 -- 172.21.15.131:0/1526848285 <== osd.6 v2:172.21.15.131:6818/1462313866 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f97080ad900 con 0x7f97080aad40 2024-05-10T05:32:21.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.290+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 --> [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9724061890 con 0x7f97080aad40 2024-05-10T05:32:21.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.290+0000 7f971aa00700 1 -- 172.21.15.131:0/1526848285 <== osd.6 v2:172.21.15.131:6818/1462313866 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9724061890 con 0x7f97080aad40 2024-05-10T05:32:21.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.291+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] conn(0x7f97080aad40 msgr2=0x7f97080ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:21.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.291+0000 7f9728a00700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] conn(0x7f97080aad40 0x7f97080ad230 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:21.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.291+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97080a11b0 msgr2=0x7f97080a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:21.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.291+0000 7f9728a00700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97080a11b0 0x7f97080a36c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9714006010 tx=0x7f971401a040 comp rx=0 tx=0).stop 2024-05-10T05:32:21.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.291+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9724106860 msgr2=0x7f9724198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:21.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.291+0000 7f9728a00700 1 --2- 172.21.15.131:0/1526848285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9724106860 0x7f9724198aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f971000e9c0 tx=0x7f971000ecd0 comp rx=0 tx=0).stop 2024-05-10T05:32:21.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.291+0000 7f9722800700 1 -- 172.21.15.131:0/1526848285 reap_dead start 2024-05-10T05:32:21.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.291+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 shutdown_connections 2024-05-10T05:32:21.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.291+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 >> 172.21.15.131:0/1526848285 conn(0x7f97240fd9c0 msgr2=0x7f9724108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:21.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.291+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 shutdown_connections 2024-05-10T05:32:21.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:21.292+0000 7f9728a00700 1 -- 172.21.15.131:0/1526848285 wait complete. 2024-05-10T05:32:21.747 INFO:teuthology.orchestra.run.smithi131.stdout:55834574904 2024-05-10T05:32:21.748 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-10T05:32:22.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:21 smithi131 ceph-mon[37264]: pgmap v255: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:22.011 INFO:teuthology.orchestra.run.smithi131.stdout:240518168602 2024-05-10T05:32:22.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-10T05:32:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.364+0000 7fe1c7600700 1 -- 172.21.15.131:0/1891205970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1c0102da0 msgr2=0x7fe1c01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.364+0000 7fe1c7600700 1 --2- 172.21.15.131:0/1891205970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1c0102da0 0x7fe1c01031b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40098f0 tx=0x7fe1b4009c00 comp rx=0 tx=0).stop 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.364+0000 7fe1c7600700 1 -- 172.21.15.131:0/1891205970 shutdown_connections 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.364+0000 7fe1c7600700 1 --2- 172.21.15.131:0/1891205970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1c0102da0 0x7fe1c01031b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.364+0000 7fe1c7600700 1 --2- 172.21.15.131:0/1891205970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1c010c210 0x7fe1c010e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.364+0000 7fe1c7600700 1 --2- 172.21.15.131:0/1891205970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c01036f0 0x7fe1c010bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.364+0000 7fe1c7600700 1 -- 172.21.15.131:0/1891205970 >> 172.21.15.131:0/1891205970 conn(0x7fe1c00fc7a0 msgr2=0x7fe1c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.365+0000 7fe1c7600700 1 -- 172.21.15.131:0/1891205970 shutdown_connections 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.365+0000 7fe1c7600700 1 -- 172.21.15.131:0/1891205970 wait complete. 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.366+0000 7fe1c7600700 1 Processor -- start 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.367+0000 7fe1c7600700 1 -- start start 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.367+0000 7fe1c7600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c0102da0 0x7fe1c0100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.367+0000 7fe1c7600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1c01036f0 0x7fe1c0100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.367+0000 7fe1c7600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1c010c210 0x7fe1c01a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.367+0000 7fe1c7600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c0101840 con 0x7fe1c01036f0 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.367+0000 7fe1c7600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c01a6ca0 con 0x7fe1c0102da0 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.367+0000 7fe1c7600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c01a6de0 con 0x7fe1c010c210 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.367+0000 7fe1bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1c01036f0 0x7fe1c0100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.368+0000 7fe1c4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c0102da0 0x7fe1c0100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.368+0000 7fe1c5600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1c010c210 0x7fe1c01a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.368+0000 7fe1bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1c01036f0 0x7fe1c0100fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49678/0 (socket says 172.21.15.131:49678) 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.368+0000 7fe1bfe00700 1 -- 172.21.15.131:0/1691808455 learned_addr learned my addr 172.21.15.131:0/1691808455 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.368+0000 7fe1c4c00700 1 -- 172.21.15.131:0/1691808455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1c010c210 msgr2=0x7fe1c01a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.368+0000 7fe1c4c00700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1c010c210 0x7fe1c01a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.368+0000 7fe1c4c00700 1 -- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1c01036f0 msgr2=0x7fe1c0100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.368+0000 7fe1c4c00700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1c01036f0 0x7fe1c0100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.368+0000 7fe1c4c00700 1 -- 172.21.15.131:0/1691808455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4009520 con 0x7fe1c0102da0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.368+0000 7fe1c4c00700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c0102da0 0x7fe1c0100a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4000c00 tx=0x7fe1b400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.369+0000 7fe1bd600700 1 -- 172.21.15.131:0/1691808455 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b401c070 con 0x7fe1c0102da0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.369+0000 7fe1bd600700 1 -- 172.21.15.131:0/1691808455 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1b400bb20 con 0x7fe1c0102da0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.369+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c01a7010 con 0x7fe1c0102da0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.369+0000 7fe1bd600700 1 -- 172.21.15.131:0/1691808455 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4005720 con 0x7fe1c0102da0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.369+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c01a7480 con 0x7fe1c0102da0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.370+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1c0069600 con 0x7fe1c0102da0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.372+0000 7fe1bd600700 1 -- 172.21.15.131:0/1691808455 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe1b4020030 con 0x7fe1c0102da0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.373+0000 7fe1bd600700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe1a80a1230 0x7fe1a80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.373+0000 7fe1bd600700 1 -- 172.21.15.131:0/1691808455 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe1b40cda40 con 0x7fe1c0102da0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.374+0000 7fe1bd600700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] conn(0x7fe1a80aadc0 0x7fe1a80ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.374+0000 7fe1bd600700 1 -- 172.21.15.131:0/1691808455 --> [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a80ad980 con 0x7fe1a80aadc0 2024-05-10T05:32:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.374+0000 7fe1bd600700 1 -- 172.21.15.131:0/1691808455 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1b40cddb0 con 0x7fe1c0102da0 2024-05-10T05:32:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.374+0000 7fe1bfe00700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe1a80a1230 0x7fe1a80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.374+0000 7fe1c5600700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] conn(0x7fe1a80aadc0 0x7fe1a80ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.375+0000 7fe1bfe00700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe1a80a1230 0x7fe1a80a3740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00ffc40 tx=0x7fe1ac009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.375+0000 7fe1c5600700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] conn(0x7fe1a80aadc0 0x7fe1a80ad2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.376+0000 7fe1bd600700 1 -- 172.21.15.131:0/1691808455 <== osd.3 v2:172.21.15.131:6810/2743036600 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe1a80ad980 con 0x7fe1a80aadc0 2024-05-10T05:32:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.394+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 --> [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1c00618d0 con 0x7fe1a80aadc0 2024-05-10T05:32:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.394+0000 7fe1bd600700 1 -- 172.21.15.131:0/1691808455 <== osd.3 v2:172.21.15.131:6810/2743036600 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe1c00618d0 con 0x7fe1a80aadc0 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] conn(0x7fe1a80aadc0 msgr2=0x7fe1a80ad2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fe1c7600700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] conn(0x7fe1a80aadc0 0x7fe1a80ad2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe1a80a1230 msgr2=0x7fe1a80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fe1c7600700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe1a80a1230 0x7fe1a80a3740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00ffc40 tx=0x7fe1ac009310 comp rx=0 tx=0).stop 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c0102da0 msgr2=0x7fe1c0100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fe1c7600700 1 --2- 172.21.15.131:0/1691808455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c0102da0 0x7fe1c0100a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4000c00 tx=0x7fe1b400fa00 comp rx=0 tx=0).stop 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fe1c5600700 1 -- 172.21.15.131:0/1691808455 reap_dead start 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 shutdown_connections 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.395+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 >> 172.21.15.131:0/1691808455 conn(0x7fe1c00fc7a0 msgr2=0x7fe1c00fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 shutdown_connections 2024-05-10T05:32:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:20.396+0000 7fe1c7600700 1 -- 172.21.15.131:0/1691808455 wait complete. 2024-05-10T05:32:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:21 smithi157 ceph-mon[78427]: pgmap v255: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:21 smithi142 ceph-mon[77667]: pgmap v255: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:22.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.411+0000 7ff8ae800700 1 -- 172.21.15.131:0/1872618225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8a8101370 msgr2=0x7ff8a810e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.411+0000 7ff8ae800700 1 --2- 172.21.15.131:0/1872618225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8a8101370 0x7ff8a810e090 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff894009990 tx=0x7ff894009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.411+0000 7ff8ae800700 1 -- 172.21.15.131:0/1872618225 shutdown_connections 2024-05-10T05:32:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.411+0000 7ff8ae800700 1 --2- 172.21.15.131:0/1872618225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a810e5d0 0x7ff8a81109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.411+0000 7ff8ae800700 1 --2- 172.21.15.131:0/1872618225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a8100a20 0x7ff8a8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.411+0000 7ff8ae800700 1 --2- 172.21.15.131:0/1872618225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8a8101370 0x7ff8a810e090 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.411+0000 7ff8ae800700 1 -- 172.21.15.131:0/1872618225 >> 172.21.15.131:0/1872618225 conn(0x7ff8a80fc7a0 msgr2=0x7ff8a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.411+0000 7ff8ae800700 1 -- 172.21.15.131:0/1872618225 shutdown_connections 2024-05-10T05:32:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.411+0000 7ff8ae800700 1 -- 172.21.15.131:0/1872618225 wait complete. 2024-05-10T05:32:22.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.413+0000 7ff8ae800700 1 Processor -- start 2024-05-10T05:32:22.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.413+0000 7ff8ae800700 1 -- start start 2024-05-10T05:32:22.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8ae800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a8100a20 0x7ff8a81a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8ae800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a8101370 0x7ff8a81a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8ae800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8a810e5d0 0x7ff8a819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8ae800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a819aab0 con 0x7ff8a810e5d0 2024-05-10T05:32:22.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8ae800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a819ac20 con 0x7ff8a8100a20 2024-05-10T05:32:22.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8ae800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a819ade0 con 0x7ff8a8101370 2024-05-10T05:32:22.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8a7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8a810e5d0 0x7ff8a819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8a6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a8101370 0x7ff8a81a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8a7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a8100a20 0x7ff8a81a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8a7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8a810e5d0 0x7ff8a819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49754/0 (socket says 172.21.15.131:49754) 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8a6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a8101370 0x7ff8a81a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37164/0 (socket says 172.21.15.131:37164) 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.414+0000 7ff8a7e00700 1 -- 172.21.15.131:0/2061968651 learned_addr learned my addr 172.21.15.131:0/2061968651 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.415+0000 7ff8a7400700 1 -- 172.21.15.131:0/2061968651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a8101370 msgr2=0x7ff8a81a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.415+0000 7ff8a7400700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8a8101370 0x7ff8a81a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.415+0000 7ff8a7400700 1 -- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8a810e5d0 msgr2=0x7ff8a819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.415+0000 7ff8a7400700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8a810e5d0 0x7ff8a819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.415+0000 7ff8a7400700 1 -- 172.21.15.131:0/2061968651 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8940095f0 con 0x7ff8a8100a20 2024-05-10T05:32:22.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.415+0000 7ff8a7400700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a8100a20 0x7ff8a81a0580 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff89800e9f0 tx=0x7ff89800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.415+0000 7ff89fe00700 1 -- 172.21.15.131:0/2061968651 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800c9c0 con 0x7ff8a8100a20 2024-05-10T05:32:22.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.415+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a819b0c0 con 0x7ff8a8100a20 2024-05-10T05:32:22.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.415+0000 7ff89fe00700 1 -- 172.21.15.131:0/2061968651 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff898010430 con 0x7ff8a8100a20 2024-05-10T05:32:22.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.416+0000 7ff89fe00700 1 -- 172.21.15.131:0/2061968651 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898018680 con 0x7ff8a8100a20 2024-05-10T05:32:22.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.416+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a81a7550 con 0x7ff8a8100a20 2024-05-10T05:32:22.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.416+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8a80695a0 con 0x7ff8a8100a20 2024-05-10T05:32:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.420+0000 7ff89fe00700 1 -- 172.21.15.131:0/2061968651 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff898010860 con 0x7ff8a8100a20 2024-05-10T05:32:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.422+0000 7ff89fe00700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff88c0a11b0 0x7ff88c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.422+0000 7ff89fe00700 1 -- 172.21.15.131:0/2061968651 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff898014070 con 0x7ff8a8100a20 2024-05-10T05:32:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.422+0000 7ff8a6a00700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff88c0a11b0 0x7ff88c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.422+0000 7ff89fe00700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] conn(0x7ff88c0aad40 0x7ff88c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.422+0000 7ff89fe00700 1 -- 172.21.15.131:0/2061968651 --> [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff88c0ad900 con 0x7ff88c0aad40 2024-05-10T05:32:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.422+0000 7ff89fe00700 1 -- 172.21.15.131:0/2061968651 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8980cd8f0 con 0x7ff8a8100a20 2024-05-10T05:32:22.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.422+0000 7ff8a7e00700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] conn(0x7ff88c0aad40 0x7ff88c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.423+0000 7ff8a6a00700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff88c0a11b0 0x7ff88c0a36c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff894006010 tx=0x7ff89401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.423+0000 7ff8a7e00700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] conn(0x7ff88c0aad40 0x7ff88c0ad230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.424+0000 7ff89fe00700 1 -- 172.21.15.131:0/2061968651 <== osd.9 v2:172.21.15.131:6826/558554742 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff88c0ad900 con 0x7ff88c0aad40 2024-05-10T05:32:22.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.441+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 --> [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8a8061910 con 0x7ff88c0aad40 2024-05-10T05:32:22.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.442+0000 7ff89fe00700 1 -- 172.21.15.131:0/2061968651 <== osd.9 v2:172.21.15.131:6826/558554742 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff8a8061910 con 0x7ff88c0aad40 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.442+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] conn(0x7ff88c0aad40 msgr2=0x7ff88c0ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.442+0000 7ff8ae800700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] conn(0x7ff88c0aad40 0x7ff88c0ad230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.442+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff88c0a11b0 msgr2=0x7ff88c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.443+0000 7ff8ae800700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff88c0a11b0 0x7ff88c0a36c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff894006010 tx=0x7ff89401a040 comp rx=0 tx=0).stop 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.443+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a8100a20 msgr2=0x7ff8a81a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.443+0000 7ff8ae800700 1 --2- 172.21.15.131:0/2061968651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a8100a20 0x7ff8a81a0580 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff89800e9f0 tx=0x7ff89800ed00 comp rx=0 tx=0).stop 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.443+0000 7ff8a7e00700 1 -- 172.21.15.131:0/2061968651 reap_dead start 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.443+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 shutdown_connections 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.443+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 >> 172.21.15.131:0/2061968651 conn(0x7ff8a80fc7a0 msgr2=0x7ff8a8110150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.443+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 shutdown_connections 2024-05-10T05:32:22.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.443+0000 7ff8ae800700 1 -- 172.21.15.131:0/2061968651 wait complete. 2024-05-10T05:32:22.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.491+0000 7fb2d8400700 1 -- 172.21.15.131:0/2310695672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d0073530 msgr2=0x7fb2d0073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.491+0000 7fb2d8400700 1 --2- 172.21.15.131:0/2310695672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d0073530 0x7fb2d0073940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009990 tx=0x7fb2c4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.491+0000 7fb2d8400700 1 -- 172.21.15.131:0/2310695672 shutdown_connections 2024-05-10T05:32:22.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.491+0000 7fb2d8400700 1 --2- 172.21.15.131:0/2310695672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d0073530 0x7fb2d0073940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.491+0000 7fb2d8400700 1 --2- 172.21.15.131:0/2310695672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d010e560 0x7fb2d0110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.491+0000 7fb2d8400700 1 --2- 172.21.15.131:0/2310695672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2d0073e80 0x7fb2d010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.491+0000 7fb2d8400700 1 -- 172.21.15.131:0/2310695672 >> 172.21.15.131:0/2310695672 conn(0x7fb2d00fd9c0 msgr2=0x7fb2d00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.491+0000 7fb2d8400700 1 -- 172.21.15.131:0/2310695672 shutdown_connections 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.491+0000 7fb2d8400700 1 -- 172.21.15.131:0/2310695672 wait complete. 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.493+0000 7fb2d8400700 1 Processor -- start 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.493+0000 7fb2d8400700 1 -- start start 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.493+0000 7fb2d8400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d0073530 0x7fb2d019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.493+0000 7fb2d8400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2d0073e80 0x7fb2d019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.493+0000 7fb2d8400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d010e560 0x7fb2d019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.493+0000 7fb2d8400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d019f700 con 0x7fb2d010e560 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.493+0000 7fb2d8400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d019f870 con 0x7fb2d0073e80 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.493+0000 7fb2d8400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d019fa30 con 0x7fb2d0073530 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d6400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d010e560 0x7fb2d019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d5a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d0073530 0x7fb2d019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d6400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d010e560 0x7fb2d019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49766/0 (socket says 172.21.15.131:49766) 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d6400700 1 -- 172.21.15.131:0/2604800124 learned_addr learned my addr 172.21.15.131:0/2604800124 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:22.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d5000700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2d0073e80 0x7fb2d019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d6400700 1 -- 172.21.15.131:0/2604800124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d0073530 msgr2=0x7fb2d019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d6400700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d0073530 0x7fb2d019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d6400700 1 -- 172.21.15.131:0/2604800124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2d0073e80 msgr2=0x7fb2d019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d6400700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2d0073e80 0x7fb2d019ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d6400700 1 -- 172.21.15.131:0/2604800124 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c40095f0 con 0x7fb2d010e560 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.494+0000 7fb2d5a00700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d0073530 0x7fb2d019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.495+0000 7fb2d6400700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d010e560 0x7fb2d019f190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800ef30 tx=0x7fb2c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.495+0000 7fb2ce000700 1 -- 172.21.15.131:0/2604800124 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8018070 con 0x7fb2d010e560 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.495+0000 7fb2ce000700 1 -- 172.21.15.131:0/2604800124 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2c800f040 con 0x7fb2d010e560 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.495+0000 7fb2ce000700 1 -- 172.21.15.131:0/2604800124 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800cd00 con 0x7fb2d010e560 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.495+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d019fcb0 con 0x7fb2d010e560 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.495+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d01a7460 con 0x7fb2d010e560 2024-05-10T05:32:22.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.496+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2d0074c70 con 0x7fb2d010e560 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.500+0000 7fb2ce000700 1 -- 172.21.15.131:0/2604800124 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb2c8003a00 con 0x7fb2d010e560 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.501+0000 7fb2ce000700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2b80a11b0 0x7fb2b80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.501+0000 7fb2ce000700 1 -- 172.21.15.131:0/2604800124 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb2c80ccf70 con 0x7fb2d010e560 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.502+0000 7fb2ce000700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] conn(0x7fb2b80aad40 0x7fb2b80ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.502+0000 7fb2ce000700 1 -- 172.21.15.131:0/2604800124 --> [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2b80ad900 con 0x7fb2b80aad40 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.502+0000 7fb2d5a00700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2b80a11b0 0x7fb2b80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.502+0000 7fb2ce000700 1 -- 172.21.15.131:0/2604800124 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2c80cd2e0 con 0x7fb2d010e560 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.502+0000 7fb2d5000700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] conn(0x7fb2b80aad40 0x7fb2b80ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.502+0000 7fb2d5a00700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2b80a11b0 0x7fb2b80a36c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00749f0 tx=0x7fb2c401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.503+0000 7fb2d5000700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] conn(0x7fb2b80aad40 0x7fb2b80ad230 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.504+0000 7fb2ce000700 1 -- 172.21.15.131:0/2604800124 <== osd.4 v2:172.21.15.142:6808/1165225030 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb2b80ad900 con 0x7fb2b80aad40 2024-05-10T05:32:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.522+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 --> [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb2d0061890 con 0x7fb2b80aad40 2024-05-10T05:32:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.523+0000 7fb2ce000700 1 -- 172.21.15.131:0/2604800124 <== osd.4 v2:172.21.15.142:6808/1165225030 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb2d0061890 con 0x7fb2b80aad40 2024-05-10T05:32:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.523+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 >> [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] conn(0x7fb2b80aad40 msgr2=0x7fb2b80ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.523+0000 7fb2d8400700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] conn(0x7fb2b80aad40 0x7fb2b80ad230 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.523+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2b80a11b0 msgr2=0x7fb2b80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.523+0000 7fb2d8400700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2b80a11b0 0x7fb2b80a36c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00749f0 tx=0x7fb2c401a040 comp rx=0 tx=0).stop 2024-05-10T05:32:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.523+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d010e560 msgr2=0x7fb2d019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.523+0000 7fb2d8400700 1 --2- 172.21.15.131:0/2604800124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d010e560 0x7fb2d019f190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800ef30 tx=0x7fb2c800c3a0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.524+0000 7fb2d6400700 1 -- 172.21.15.131:0/2604800124 reap_dead start 2024-05-10T05:32:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.524+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 shutdown_connections 2024-05-10T05:32:22.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.524+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 >> 172.21.15.131:0/2604800124 conn(0x7fb2d00fd9c0 msgr2=0x7fb2d00ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:22.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.524+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 shutdown_connections 2024-05-10T05:32:22.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.524+0000 7fb2d8400700 1 -- 172.21.15.131:0/2604800124 wait complete. 2024-05-10T05:32:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.619+0000 7f02cac00700 1 -- 172.21.15.131:0/2711208130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c4104160 msgr2=0x7f02c4104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.619+0000 7f02cac00700 1 --2- 172.21.15.131:0/2711208130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c4104160 0x7f02c4104570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009990 tx=0x7f02b4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.619+0000 7f02cac00700 1 -- 172.21.15.131:0/2711208130 shutdown_connections 2024-05-10T05:32:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.619+0000 7f02cac00700 1 --2- 172.21.15.131:0/2711208130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c4104160 0x7f02c4104570 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.619+0000 7f02cac00700 1 --2- 172.21.15.131:0/2711208130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02c41093e0 0x7f02c4110d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.619+0000 7f02cac00700 1 --2- 172.21.15.131:0/2711208130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c4108a90 0x7f02c4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.619+0000 7f02cac00700 1 -- 172.21.15.131:0/2711208130 >> 172.21.15.131:0/2711208130 conn(0x7f02c40ffa30 msgr2=0x7f02c4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.619+0000 7f02cac00700 1 -- 172.21.15.131:0/2711208130 shutdown_connections 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.619+0000 7f02cac00700 1 -- 172.21.15.131:0/2711208130 wait complete. 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.621+0000 7f02cac00700 1 Processor -- start 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.621+0000 7f02cac00700 1 -- start start 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.621+0000 7f02cac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c4104160 0x7f02c41a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02cac00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c4108a90 0x7f02c41a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02cac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02c41093e0 0x7f02c419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02cac00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c419aba0 con 0x7f02c4108a90 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02cac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c419ace0 con 0x7f02c41093e0 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02cac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c419aea0 con 0x7f02c4104160 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02bbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c4108a90 0x7f02c41a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02c3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c4104160 0x7f02c41a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02c3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c4104160 0x7f02c41a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37188/0 (socket says 172.21.15.131:37188) 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02c3e00700 1 -- 172.21.15.131:0/2170616403 learned_addr learned my addr 172.21.15.131:0/2170616403 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02c8c00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02c41093e0 0x7f02c419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02c3e00700 1 -- 172.21.15.131:0/2170616403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02c41093e0 msgr2=0x7f02c419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02c3e00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02c41093e0 0x7f02c419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02c3e00700 1 -- 172.21.15.131:0/2170616403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c4108a90 msgr2=0x7f02c41a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02c3e00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c4108a90 0x7f02c41a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.622+0000 7f02c3e00700 1 -- 172.21.15.131:0/2170616403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc009c40 con 0x7f02c4104160 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.623+0000 7f02c3e00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c4104160 0x7f02c41a0620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f02b4005770 tx=0x7f02b40057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.623+0000 7f02c1600700 1 -- 172.21.15.131:0/2170616403 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b401d070 con 0x7f02c4104160 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.623+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b40095f0 con 0x7f02c4104160 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.623+0000 7f02c1600700 1 -- 172.21.15.131:0/2170616403 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b4005a00 con 0x7f02c4104160 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.623+0000 7f02c1600700 1 -- 172.21.15.131:0/2170616403 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4021da0 con 0x7f02c4104160 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.623+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c41a7420 con 0x7f02c4104160 2024-05-10T05:32:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.624+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02c4105610 con 0x7f02c4104160 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.626+0000 7f02c1600700 1 -- 172.21.15.131:0/2170616403 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f02b40214a0 con 0x7f02c4104160 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.627+0000 7f02c1600700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f02a80a1230 0x7f02a80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.628+0000 7f02c1600700 1 -- 172.21.15.131:0/2170616403 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f02b40ce570 con 0x7f02c4104160 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.628+0000 7f02bbe00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f02a80a1230 0x7f02a80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.628+0000 7f02c1600700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] conn(0x7f02a80aadc0 0x7f02a80ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.628+0000 7f02c1600700 1 -- 172.21.15.131:0/2170616403 --> [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02a80ad980 con 0x7f02a80aadc0 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.628+0000 7f02c1600700 1 -- 172.21.15.131:0/2170616403 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b40ce8e0 con 0x7f02c4104160 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.628+0000 7f02c8c00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] conn(0x7f02a80aadc0 0x7f02a80ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.628+0000 7f02bbe00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f02a80a1230 0x7f02a80a3740 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f02c419bef0 tx=0x7f02ac009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.629+0000 7f02c8c00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] conn(0x7f02a80aadc0 0x7f02a80ad2b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.630+0000 7f02c1600700 1 -- 172.21.15.131:0/2170616403 <== osd.2 v2:172.21.15.157:6800/3652465947 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f02a80ad980 con 0x7f02a80aadc0 2024-05-10T05:32:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.647+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 --> [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f02c41140b0 con 0x7f02a80aadc0 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.648+0000 7f02c1600700 1 -- 172.21.15.131:0/2170616403 <== osd.2 v2:172.21.15.157:6800/3652465947 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f02bc00f040 con 0x7f02a80aadc0 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 >> [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] conn(0x7f02a80aadc0 msgr2=0x7f02a80ad2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02cac00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] conn(0x7f02a80aadc0 0x7f02a80ad2b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f02a80a1230 msgr2=0x7f02a80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02cac00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f02a80a1230 0x7f02a80a3740 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f02c419bef0 tx=0x7f02ac009260 comp rx=0 tx=0).stop 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c4104160 msgr2=0x7f02c41a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02cac00700 1 --2- 172.21.15.131:0/2170616403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c4104160 0x7f02c41a0620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f02b4005770 tx=0x7f02b40057a0 comp rx=0 tx=0).stop 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02c8c00700 1 -- 172.21.15.131:0/2170616403 reap_dead start 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 shutdown_connections 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 >> 172.21.15.131:0/2170616403 conn(0x7f02c40ffa30 msgr2=0x7f02c4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 shutdown_connections 2024-05-10T05:32:22.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.649+0000 7f02cac00700 1 -- 172.21.15.131:0/2170616403 wait complete. 2024-05-10T05:32:23.133 INFO:teuthology.orchestra.run.smithi131.stdout:188978561060 2024-05-10T05:32:23.134 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-10T05:32:23.295 INFO:teuthology.orchestra.run.smithi131.stdout:158913789991 2024-05-10T05:32:23.295 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-10T05:32:23.439 INFO:teuthology.orchestra.run.smithi131.stdout:137438953515 2024-05-10T05:32:23.440 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-10T05:32:23.701 INFO:teuthology.orchestra.run.smithi131.stdout:107374182447 2024-05-10T05:32:23.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-10T05:32:23.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.886+0000 7f9f12800700 1 -- 172.21.15.131:0/2580181809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f0c073780 msgr2=0x7f9f0c073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:23.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.886+0000 7f9f12800700 1 --2- 172.21.15.131:0/2580181809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f0c073780 0x7f9f0c073b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009990 tx=0x7f9efc009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:23.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.886+0000 7f9f12800700 1 -- 172.21.15.131:0/2580181809 shutdown_connections 2024-05-10T05:32:23.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.886+0000 7f9f12800700 1 --2- 172.21.15.131:0/2580181809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f0c073780 0x7f9f0c073b90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:23.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.886+0000 7f9f12800700 1 --2- 172.21.15.131:0/2580181809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c10e4f0 0x7f9f0c110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:23.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.886+0000 7f9f12800700 1 --2- 172.21.15.131:0/2580181809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f0c0740d0 0x7f9f0c10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:23.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.886+0000 7f9f12800700 1 -- 172.21.15.131:0/2580181809 >> 172.21.15.131:0/2580181809 conn(0x7f9f0c0fd7f0 msgr2=0x7f9f0c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:23.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.887+0000 7f9f12800700 1 -- 172.21.15.131:0/2580181809 shutdown_connections 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.887+0000 7f9f12800700 1 -- 172.21.15.131:0/2580181809 wait complete. 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.888+0000 7f9f12800700 1 Processor -- start 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.888+0000 7f9f12800700 1 -- start start 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f12800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f0c073780 0x7f9f0c1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f12800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0740d0 0x7f9f0c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f12800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f0c10e4f0 0x7f9f0c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f12800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c19ab00 con 0x7f9f0c073780 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f12800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c19ac40 con 0x7f9f0c10e4f0 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f12800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c19ae00 con 0x7f9f0c0740d0 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f0b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f0c073780 0x7f9f0c1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0740d0 0x7f9f0c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f0b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f0c073780 0x7f9f0c1a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35858/0 (socket says 172.21.15.131:35858) 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f0b400700 1 -- 172.21.15.131:0/1813319553 learned_addr learned my addr 172.21.15.131:0/1813319553 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.889+0000 7f9f0be00700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f0c10e4f0 0x7f9f0c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:23.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f03e00700 1 -- 172.21.15.131:0/1813319553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f0c10e4f0 msgr2=0x7f9f0c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f03e00700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f0c10e4f0 0x7f9f0c19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f03e00700 1 -- 172.21.15.131:0/1813319553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f0c073780 msgr2=0x7f9f0c1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f03e00700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f0c073780 0x7f9f0c1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f03e00700 1 -- 172.21.15.131:0/1813319553 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc0095f0 con 0x7f9f0c0740d0 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f0b400700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f0c073780 0x7f9f0c1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f03e00700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0740d0 0x7f9f0c1a0b10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400e9f0 tx=0x7f9ef400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f08c00700 1 -- 172.21.15.131:0/1813319553 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef400c9c0 con 0x7f9f0c0740d0 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0c19b0e0 con 0x7f9f0c0740d0 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f08c00700 1 -- 172.21.15.131:0/1813319553 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ef4010430 con 0x7f9f0c0740d0 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f08c00700 1 -- 172.21.15.131:0/1813319553 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4018680 con 0x7f9f0c0740d0 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.890+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c1a75a0 con 0x7f9f0c0740d0 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.891+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9f0c074cf0 con 0x7f9f0c0740d0 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.893+0000 7f9f08c00700 1 -- 172.21.15.131:0/1813319553 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9ef4010860 con 0x7f9f0c0740d0 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.895+0000 7f9f08c00700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9ef00a11b0 0x7f9ef00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.895+0000 7f9f08c00700 1 -- 172.21.15.131:0/1813319553 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9ef4014070 con 0x7f9f0c0740d0 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.895+0000 7f9f0b400700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9ef00a11b0 0x7f9ef00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.895+0000 7f9f08c00700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] conn(0x7f9ef00aad40 0x7f9ef00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.895+0000 7f9f08c00700 1 -- 172.21.15.131:0/1813319553 --> [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ef00ad900 con 0x7f9ef00aad40 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.895+0000 7f9f08c00700 1 -- 172.21.15.131:0/1813319553 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ef40cd230 con 0x7f9f0c0740d0 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.896+0000 7f9f0be00700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] conn(0x7f9ef00aad40 0x7f9ef00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.896+0000 7f9f0b400700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9ef00a11b0 0x7f9ef00a36c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00b2e0 tx=0x7f9efc01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.896+0000 7f9f0be00700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] conn(0x7f9ef00aad40 0x7f9ef00ad230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.897+0000 7f9f08c00700 1 -- 172.21.15.131:0/1813319553 <== osd.7 v2:172.21.15.142:6816/1027002285 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9ef00ad900 con 0x7f9ef00aad40 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.915+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 --> [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9f0c0618d0 con 0x7f9ef00aad40 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.915+0000 7f9f08c00700 1 -- 172.21.15.131:0/1813319553 <== osd.7 v2:172.21.15.142:6816/1027002285 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9f0c0618d0 con 0x7f9ef00aad40 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 >> [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] conn(0x7f9ef00aad40 msgr2=0x7f9ef00ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f12800700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] conn(0x7f9ef00aad40 0x7f9ef00ad230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9ef00a11b0 msgr2=0x7f9ef00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f12800700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9ef00a11b0 0x7f9ef00a36c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00b2e0 tx=0x7f9efc01a040 comp rx=0 tx=0).stop 2024-05-10T05:32:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0740d0 msgr2=0x7f9f0c1a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:23.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f12800700 1 --2- 172.21.15.131:0/1813319553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f0c0740d0 0x7f9f0c1a0b10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400e9f0 tx=0x7f9ef400ed00 comp rx=0 tx=0).stop 2024-05-10T05:32:23.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f0be00700 1 -- 172.21.15.131:0/1813319553 reap_dead start 2024-05-10T05:32:23.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 shutdown_connections 2024-05-10T05:32:23.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 >> 172.21.15.131:0/1813319553 conn(0x7f9f0c0fd7f0 msgr2=0x7f9f0c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:23.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 shutdown_connections 2024-05-10T05:32:23.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:22.916+0000 7f9f12800700 1 -- 172.21.15.131:0/1813319553 wait complete. 2024-05-10T05:32:24.183 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:23 smithi131 ceph-mon[37264]: pgmap v256: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:24.205 INFO:teuthology.orchestra.run.smithi131.stdout:77309411380 2024-05-10T05:32:24.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-10T05:32:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:23 smithi157 ceph-mon[78427]: pgmap v256: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:24.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:23 smithi142 ceph-mon[77667]: pgmap v256: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-05-10T05:32:24.481 INFO:teuthology.orchestra.run.smithi131.stdout:270582939671 2024-05-10T05:32:24.481 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-10T05:32:24.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.653+0000 7fe1e8000700 1 -- 172.21.15.131:0/2128685228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e0073f00 msgr2=0x7fe1e0112770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.653+0000 7fe1e8000700 1 --2- 172.21.15.131:0/2128685228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e0073f00 0x7fe1e0112770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009990 tx=0x7fe1cc009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.653+0000 7fe1e8000700 1 -- 172.21.15.131:0/2128685228 shutdown_connections 2024-05-10T05:32:24.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.653+0000 7fe1e8000700 1 --2- 172.21.15.131:0/2128685228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e0112cb0 0x7fe1e01150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.653+0000 7fe1e8000700 1 --2- 172.21.15.131:0/2128685228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e00735b0 0x7fe1e00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.653+0000 7fe1e8000700 1 --2- 172.21.15.131:0/2128685228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e0073f00 0x7fe1e0112770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.653+0000 7fe1e8000700 1 -- 172.21.15.131:0/2128685228 >> 172.21.15.131:0/2128685228 conn(0x7fe1e00fd9c0 msgr2=0x7fe1e00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:24.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.653+0000 7fe1e8000700 1 -- 172.21.15.131:0/2128685228 shutdown_connections 2024-05-10T05:32:24.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.653+0000 7fe1e8000700 1 -- 172.21.15.131:0/2128685228 wait complete. 2024-05-10T05:32:24.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.654+0000 7fe1e8000700 1 Processor -- start 2024-05-10T05:32:24.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.655+0000 7fe1e8000700 1 -- start start 2024-05-10T05:32:24.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.655+0000 7fe1e8000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e00735b0 0x7fe1e01a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:24.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.655+0000 7fe1e8000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e0073f00 0x7fe1e01a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.655+0000 7fe1e8000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e0112cb0 0x7fe1e019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.655+0000 7fe1e8000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e019ef70 con 0x7fe1e0112cb0 2024-05-10T05:32:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.655+0000 7fe1e8000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e019f0e0 con 0x7fe1e00735b0 2024-05-10T05:32:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.655+0000 7fe1e8000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e019f2a0 con 0x7fe1e0073f00 2024-05-10T05:32:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e0112cb0 0x7fe1e019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e4c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e0073f00 0x7fe1e01a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e0112cb0 0x7fe1e019e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35878/0 (socket says 172.21.15.131:35878) 2024-05-10T05:32:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e6000700 1 -- 172.21.15.131:0/2144680208 learned_addr learned my addr 172.21.15.131:0/2144680208 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e4c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e0073f00 0x7fe1e01a4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45206/0 (socket says 172.21.15.131:45206) 2024-05-10T05:32:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e5600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e00735b0 0x7fe1e01a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:24.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e4c00700 1 -- 172.21.15.131:0/2144680208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e00735b0 msgr2=0x7fe1e01a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e4c00700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e00735b0 0x7fe1e01a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e4c00700 1 -- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e0112cb0 msgr2=0x7fe1e019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e4c00700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e0112cb0 0x7fe1e019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e4c00700 1 -- 172.21.15.131:0/2144680208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc0095f0 con 0x7fe1e0073f00 2024-05-10T05:32:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.656+0000 7fe1e6000700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e0112cb0 0x7fe1e019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.657+0000 7fe1e4c00700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e0073f00 0x7fe1e01a4f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0098e0 tx=0x7fe1cc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.658+0000 7fe1de000700 1 -- 172.21.15.131:0/2144680208 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc01d070 con 0x7fe1e0073f00 2024-05-10T05:32:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.658+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e019f520 con 0x7fe1e0073f00 2024-05-10T05:32:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.658+0000 7fe1de000700 1 -- 172.21.15.131:0/2144680208 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1cc022470 con 0x7fe1e0073f00 2024-05-10T05:32:24.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.658+0000 7fe1de000700 1 -- 172.21.15.131:0/2144680208 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc017650 con 0x7fe1e0073f00 2024-05-10T05:32:24.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.658+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e019f7f0 con 0x7fe1e0073f00 2024-05-10T05:32:24.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.659+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1e0074cf0 con 0x7fe1e0073f00 2024-05-10T05:32:24.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.662+0000 7fe1de000700 1 -- 172.21.15.131:0/2144680208 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe1cc0225e0 con 0x7fe1e0073f00 2024-05-10T05:32:24.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.664+0000 7fe1de000700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe1c80a11b0 0x7fe1c80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:24.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.664+0000 7fe1de000700 1 -- 172.21.15.131:0/2144680208 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe1cc0ce6d0 con 0x7fe1e0073f00 2024-05-10T05:32:24.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.664+0000 7fe1de000700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] conn(0x7fe1c80aad40 0x7fe1c80ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:24.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.664+0000 7fe1de000700 1 -- 172.21.15.131:0/2144680208 --> [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1c80ad900 con 0x7fe1c80aad40 2024-05-10T05:32:24.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.664+0000 7fe1de000700 1 -- 172.21.15.131:0/2144680208 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1cc0ceae0 con 0x7fe1e0073f00 2024-05-10T05:32:24.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.664+0000 7fe1e5600700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe1c80a11b0 0x7fe1c80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:24.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.664+0000 7fe1e6000700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] conn(0x7fe1c80aad40 0x7fe1c80ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:24.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.665+0000 7fe1e5600700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe1c80a11b0 0x7fe1c80a36c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1e019fe10 tx=0x7fe1d4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:24.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.665+0000 7fe1e6000700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] conn(0x7fe1c80aad40 0x7fe1c80ad230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:24.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.666+0000 7fe1de000700 1 -- 172.21.15.131:0/2144680208 <== osd.0 v2:172.21.15.131:6802/3101739550 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe1c80ad900 con 0x7fe1c80aad40 2024-05-10T05:32:24.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.685+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 --> [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1e0061890 con 0x7fe1c80aad40 2024-05-10T05:32:24.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.685+0000 7fe1de000700 1 -- 172.21.15.131:0/2144680208 <== osd.0 v2:172.21.15.131:6802/3101739550 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe1e0061890 con 0x7fe1c80aad40 2024-05-10T05:32:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] conn(0x7fe1c80aad40 msgr2=0x7fe1c80ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e8000700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] conn(0x7fe1c80aad40 0x7fe1c80ad230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe1c80a11b0 msgr2=0x7fe1c80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e8000700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe1c80a11b0 0x7fe1c80a36c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1e019fe10 tx=0x7fe1d4009260 comp rx=0 tx=0).stop 2024-05-10T05:32:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e0073f00 msgr2=0x7fe1e01a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e8000700 1 --2- 172.21.15.131:0/2144680208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e0073f00 0x7fe1e01a4f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0098e0 tx=0x7fe1cc005740 comp rx=0 tx=0).stop 2024-05-10T05:32:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e6000700 1 -- 172.21.15.131:0/2144680208 reap_dead start 2024-05-10T05:32:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 shutdown_connections 2024-05-10T05:32:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 >> 172.21.15.131:0/2144680208 conn(0x7fe1e00fd9c0 msgr2=0x7fe1e00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:24.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 shutdown_connections 2024-05-10T05:32:24.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.686+0000 7fe1e8000700 1 -- 172.21.15.131:0/2144680208 wait complete. 2024-05-10T05:32:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.937+0000 7f420b000700 1 -- 172.21.15.131:0/4124349085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42041071c0 msgr2=0x7f420410eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.937+0000 7f420b000700 1 --2- 172.21.15.131:0/4124349085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42041071c0 0x7f420410eb20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00b0a0 tx=0x7f41fc00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.937+0000 7f420b000700 1 -- 172.21.15.131:0/4124349085 shutdown_connections 2024-05-10T05:32:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.937+0000 7f420b000700 1 --2- 172.21.15.131:0/4124349085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42041071c0 0x7f420410eb20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.937+0000 7f420b000700 1 --2- 172.21.15.131:0/4124349085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4204101f40 0x7f4204102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.937+0000 7f420b000700 1 --2- 172.21.15.131:0/4124349085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4204106870 0x7f4204106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.937+0000 7f420b000700 1 -- 172.21.15.131:0/4124349085 >> 172.21.15.131:0/4124349085 conn(0x7f42040fd7f0 msgr2=0x7f42040ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:24.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.937+0000 7f420b000700 1 -- 172.21.15.131:0/4124349085 shutdown_connections 2024-05-10T05:32:24.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.937+0000 7f420b000700 1 -- 172.21.15.131:0/4124349085 wait complete. 2024-05-10T05:32:24.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.938+0000 7f420b000700 1 Processor -- start 2024-05-10T05:32:24.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.939+0000 7f420b000700 1 -- start start 2024-05-10T05:32:24.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.939+0000 7f420b000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4204101f40 0x7f4204198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:24.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.939+0000 7f420b000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4204106870 0x7f4204198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:24.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.939+0000 7f420b000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42041071c0 0x7f420419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:24.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.939+0000 7f420b000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420419d590 con 0x7f4204106870 2024-05-10T05:32:24.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.939+0000 7f420b000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420419d700 con 0x7f4204101f40 2024-05-10T05:32:24.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.939+0000 7f420b000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420419d8c0 con 0x7f42041071c0 2024-05-10T05:32:24.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.939+0000 7f4203400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4204106870 0x7f4204198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.939+0000 7f4203400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4204106870 0x7f4204198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35902/0 (socket says 172.21.15.131:35902) 2024-05-10T05:32:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.939+0000 7f4203400700 1 -- 172.21.15.131:0/2515504169 learned_addr learned my addr 172.21.15.131:0/2515504169 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.940+0000 7f4209000700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42041071c0 0x7f420419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.940+0000 7f4203400700 1 -- 172.21.15.131:0/2515504169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42041071c0 msgr2=0x7f420419d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.940+0000 7f4203400700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42041071c0 0x7f420419d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.940+0000 7f4203400700 1 -- 172.21.15.131:0/2515504169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4204101f40 msgr2=0x7f4204198570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:32:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.940+0000 7f4203400700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4204101f40 0x7f4204198570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.940+0000 7f4203400700 1 -- 172.21.15.131:0/2515504169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0009520 con 0x7f4204106870 2024-05-10T05:32:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.940+0000 7f4203400700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4204106870 0x7f4204198b10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f41f000ea30 tx=0x7f41f000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.940+0000 7f4200c00700 1 -- 172.21.15.131:0/2515504169 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0009820 con 0x7f4204106870 2024-05-10T05:32:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.941+0000 7f4200c00700 1 -- 172.21.15.131:0/2515504169 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41f00044e0 con 0x7f4204106870 2024-05-10T05:32:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.941+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc009d00 con 0x7f4204106870 2024-05-10T05:32:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.941+0000 7f4200c00700 1 -- 172.21.15.131:0/2515504169 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0003dc0 con 0x7f4204106870 2024-05-10T05:32:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.941+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4204073440 con 0x7f4204106870 2024-05-10T05:32:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.942+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42041033f0 con 0x7f4204106870 2024-05-10T05:32:24.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.944+0000 7f4200c00700 1 -- 172.21.15.131:0/2515504169 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f41f0003f20 con 0x7f4204106870 2024-05-10T05:32:24.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.945+0000 7f4200c00700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f41e80a1230 0x7f41e80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:24.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.945+0000 7f4200c00700 1 -- 172.21.15.131:0/2515504169 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f41f0014070 con 0x7f4204106870 2024-05-10T05:32:24.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.945+0000 7f4203e00700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f41e80a1230 0x7f41e80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:24.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.946+0000 7f4200c00700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] conn(0x7f41e80aadc0 0x7f41e80ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:24.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.946+0000 7f4200c00700 1 -- 172.21.15.131:0/2515504169 --> [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41e80ad980 con 0x7f41e80aadc0 2024-05-10T05:32:24.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.946+0000 7f4200c00700 1 -- 172.21.15.131:0/2515504169 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41f00cd2a0 con 0x7f4204106870 2024-05-10T05:32:24.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.946+0000 7f4203e00700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f41e80a1230 0x7f41e80a3740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009710 tx=0x7f41f4006c60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:24.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.950+0000 7f4209000700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] conn(0x7f41e80aadc0 0x7f41e80ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:24.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.951+0000 7f4209000700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] conn(0x7f41e80aadc0 0x7f41e80ad2b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:24.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.952+0000 7f4200c00700 1 -- 172.21.15.131:0/2515504169 <== osd.11 v2:172.21.15.157:6824/1016688620 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f41e80ad980 con 0x7f41e80aadc0 2024-05-10T05:32:24.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.969+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 --> [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4204061890 con 0x7f41e80aadc0 2024-05-10T05:32:24.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.970+0000 7f4200c00700 1 -- 172.21.15.131:0/2515504169 <== osd.11 v2:172.21.15.157:6824/1016688620 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4204061890 con 0x7f41e80aadc0 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.970+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 >> [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] conn(0x7f41e80aadc0 msgr2=0x7f41e80ad2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.971+0000 7f420b000700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] conn(0x7f41e80aadc0 0x7f41e80ad2b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.971+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f41e80a1230 msgr2=0x7f41e80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.971+0000 7f420b000700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f41e80a1230 0x7f41e80a3740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009710 tx=0x7f41f4006c60 comp rx=0 tx=0).stop 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.971+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4204106870 msgr2=0x7f4204198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.971+0000 7f420b000700 1 --2- 172.21.15.131:0/2515504169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4204106870 0x7f4204198b10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f41f000ea30 tx=0x7f41f000ed40 comp rx=0 tx=0).stop 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.971+0000 7f4209000700 1 -- 172.21.15.131:0/2515504169 reap_dead start 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.971+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 shutdown_connections 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.971+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 >> 172.21.15.131:0/2515504169 conn(0x7f42040fd7f0 msgr2=0x7f4204108120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.971+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 shutdown_connections 2024-05-10T05:32:24.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:24.971+0000 7f420b000700 1 -- 172.21.15.131:0/2515504169 wait complete. 2024-05-10T05:32:25.157 INFO:teuthology.orchestra.run.smithi131.stdout:210453397536 2024-05-10T05:32:25.158 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-10T05:32:25.319 INFO:teuthology.orchestra.run.smithi131.stdout:34359738429 2024-05-10T05:32:25.319 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-10T05:32:25.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.562+0000 7f5daba00700 1 -- 172.21.15.131:0/1934489615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4106080 msgr2=0x7f5da410d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:25.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.562+0000 7f5daba00700 1 --2- 172.21.15.131:0/1934489615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4106080 0x7f5da410d9e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00b0a0 tx=0x7f5d9c00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:25.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.563+0000 7f5daba00700 1 -- 172.21.15.131:0/1934489615 shutdown_connections 2024-05-10T05:32:25.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.563+0000 7f5daba00700 1 --2- 172.21.15.131:0/1934489615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4106080 0x7f5da410d9e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:25.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.563+0000 7f5daba00700 1 --2- 172.21.15.131:0/1934489615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4100e00 0x7f5da4101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:25.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.563+0000 7f5daba00700 1 --2- 172.21.15.131:0/1934489615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da4105730 0x7f5da4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:25.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.563+0000 7f5daba00700 1 -- 172.21.15.131:0/1934489615 >> 172.21.15.131:0/1934489615 conn(0x7f5da40fc7a0 msgr2=0x7f5da40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:25.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.563+0000 7f5daba00700 1 -- 172.21.15.131:0/1934489615 shutdown_connections 2024-05-10T05:32:25.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.563+0000 7f5daba00700 1 -- 172.21.15.131:0/1934489615 wait complete. 2024-05-10T05:32:25.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.564+0000 7f5daba00700 1 Processor -- start 2024-05-10T05:32:25.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.564+0000 7f5daba00700 1 -- start start 2024-05-10T05:32:25.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.564+0000 7f5daba00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da4100e00 0x7f5da419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:25.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.564+0000 7f5daba00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4105730 0x7f5da419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:25.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.564+0000 7f5daba00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4106080 0x7f5da4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:25.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.564+0000 7f5daba00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4198a00 con 0x7f5da4106080 2024-05-10T05:32:25.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.564+0000 7f5daba00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4198b70 con 0x7f5da4100e00 2024-05-10T05:32:25.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.564+0000 7f5daba00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4198d30 con 0x7f5da4105730 2024-05-10T05:32:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4105730 0x7f5da419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4105730 0x7f5da419ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45228/0 (socket says 172.21.15.131:45228) 2024-05-10T05:32:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da3e00700 1 -- 172.21.15.131:0/1102437022 learned_addr learned my addr 172.21.15.131:0/1102437022 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da9a00700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4106080 0x7f5da4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da9000700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da4100e00 0x7f5da419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da3e00700 1 -- 172.21.15.131:0/1102437022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da4100e00 msgr2=0x7f5da419e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da3e00700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da4100e00 0x7f5da419e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:25.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da3e00700 1 -- 172.21.15.131:0/1102437022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4106080 msgr2=0x7f5da4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:25.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da3e00700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4106080 0x7f5da4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:25.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da3e00700 1 -- 172.21.15.131:0/1102437022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d90009520 con 0x7f5da4105730 2024-05-10T05:32:25.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.565+0000 7f5da3e00700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4105730 0x7f5da419ea10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000ea30 tx=0x7f5d9000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:25.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.566+0000 7f5da1600700 1 -- 172.21.15.131:0/1102437022 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90009820 con 0x7f5da4105730 2024-05-10T05:32:25.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.566+0000 7f5da1600700 1 -- 172.21.15.131:0/1102437022 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d900044e0 con 0x7f5da4105730 2024-05-10T05:32:25.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.566+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c009d00 con 0x7f5da4105730 2024-05-10T05:32:25.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.566+0000 7f5da1600700 1 -- 172.21.15.131:0/1102437022 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90003dc0 con 0x7f5da4105730 2024-05-10T05:32:25.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.566+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da41a52d0 con 0x7f5da4105730 2024-05-10T05:32:25.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.567+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5da4110cc0 con 0x7f5da4105730 2024-05-10T05:32:25.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.569+0000 7f5da1600700 1 -- 172.21.15.131:0/1102437022 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5d90003f60 con 0x7f5da4105730 2024-05-10T05:32:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.571+0000 7f5da1600700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d8c0a11b0 0x7f5d8c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:25.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.571+0000 7f5da1600700 1 -- 172.21.15.131:0/1102437022 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5d90014070 con 0x7f5da4105730 2024-05-10T05:32:25.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.572+0000 7f5da9000700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d8c0a11b0 0x7f5d8c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:25.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.572+0000 7f5da1600700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] conn(0x7f5d8c0aad40 0x7f5d8c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:25.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.572+0000 7f5da1600700 1 -- 172.21.15.131:0/1102437022 --> [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d8c0ad900 con 0x7f5d8c0aad40 2024-05-10T05:32:25.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.572+0000 7f5da1600700 1 -- 172.21.15.131:0/1102437022 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d900cd410 con 0x7f5da4105730 2024-05-10T05:32:25.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.572+0000 7f5da9a00700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] conn(0x7f5d8c0aad40 0x7f5d8c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:25.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.572+0000 7f5da9000700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d8c0a11b0 0x7f5d8c0a36c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5da41998a0 tx=0x7f5d98009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:25.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.573+0000 7f5da9a00700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] conn(0x7f5d8c0aad40 0x7f5d8c0ad230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:25.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.574+0000 7f5da1600700 1 -- 172.21.15.131:0/1102437022 <== osd.10 v2:172.21.15.142:6824/2763887998 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f5d8c0ad900 con 0x7f5d8c0aad40 2024-05-10T05:32:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.592+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 --> [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5da4061910 con 0x7f5d8c0aad40 2024-05-10T05:32:25.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.592+0000 7f5da1600700 1 -- 172.21.15.131:0/1102437022 <== osd.10 v2:172.21.15.142:6824/2763887998 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5da4061910 con 0x7f5d8c0aad40 2024-05-10T05:32:25.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 >> [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] conn(0x7f5d8c0aad40 msgr2=0x7f5d8c0ad230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:25.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5daba00700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] conn(0x7f5d8c0aad40 0x7f5d8c0ad230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:25.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d8c0a11b0 msgr2=0x7f5d8c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:25.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5daba00700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d8c0a11b0 0x7f5d8c0a36c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5da41998a0 tx=0x7f5d98009260 comp rx=0 tx=0).stop 2024-05-10T05:32:25.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4105730 msgr2=0x7f5da419ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5daba00700 1 --2- 172.21.15.131:0/1102437022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4105730 0x7f5da419ea10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000ea30 tx=0x7f5d9000ed40 comp rx=0 tx=0).stop 2024-05-10T05:32:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5da9a00700 1 -- 172.21.15.131:0/1102437022 reap_dead start 2024-05-10T05:32:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 shutdown_connections 2024-05-10T05:32:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 >> 172.21.15.131:0/1102437022 conn(0x7f5da40fc7a0 msgr2=0x7f5da40690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 shutdown_connections 2024-05-10T05:32:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:25.593+0000 7f5daba00700 1 -- 172.21.15.131:0/1102437022 wait complete. 2024-05-10T05:32:25.882 INFO:teuthology.orchestra.run.smithi131.stdout:322122547218 2024-05-10T05:32:25.882 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-10T05:32:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:25 smithi131 ceph-mon[37264]: pgmap v257: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-05-10T05:32:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:25 smithi157 ceph-mon[78427]: pgmap v257: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-05-10T05:32:26.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:25 smithi142 ceph-mon[77667]: pgmap v257: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-05-10T05:32:26.908 INFO:teuthology.orchestra.run.smithi131.stdout:292057776148 2024-05-10T05:32:26.908 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-10T05:32:26.954 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:27.209 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:27.715 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:27.716 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:28.200 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:27 smithi131 ceph-mon[37264]: pgmap v258: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 187 B/s wr, 11 op/s; 10 B/s, 0 objects/s recovering 2024-05-10T05:32:28.272 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:27 smithi157 ceph-mon[78427]: pgmap v258: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 187 B/s wr, 11 op/s; 10 B/s, 0 objects/s recovering 2024-05-10T05:32:28.311 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:28.323 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:27 smithi142 ceph-mon[77667]: pgmap v258: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 187 B/s wr, 11 op/s; 10 B/s, 0 objects/s recovering 2024-05-10T05:32:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.682+0000 7fdd5ac00700 1 -- 172.21.15.131:0/2532655308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd54102e20 msgr2=0x7fdd54103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.682+0000 7fdd5ac00700 1 --2- 172.21.15.131:0/2532655308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd54102e20 0x7fdd54103230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009990 tx=0x7fdd44009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.682+0000 7fdd5ac00700 1 -- 172.21.15.131:0/2532655308 shutdown_connections 2024-05-10T05:32:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.682+0000 7fdd5ac00700 1 --2- 172.21.15.131:0/2532655308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd5410c250 0x7fdd5410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.682+0000 7fdd5ac00700 1 --2- 172.21.15.131:0/2532655308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd54102e20 0x7fdd54103230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.682+0000 7fdd5ac00700 1 --2- 172.21.15.131:0/2532655308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd54103770 0x7fdd5410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.682+0000 7fdd5ac00700 1 -- 172.21.15.131:0/2532655308 >> 172.21.15.131:0/2532655308 conn(0x7fdd540fc7a0 msgr2=0x7fdd540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.682+0000 7fdd5ac00700 1 -- 172.21.15.131:0/2532655308 shutdown_connections 2024-05-10T05:32:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.682+0000 7fdd5ac00700 1 -- 172.21.15.131:0/2532655308 wait complete. 2024-05-10T05:32:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.683+0000 7fdd5ac00700 1 Processor -- start 2024-05-10T05:32:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.684+0000 7fdd5ac00700 1 -- start start 2024-05-10T05:32:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.684+0000 7fdd5ac00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd54102e20 0x7fdd541a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.684+0000 7fdd5ac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd54103770 0x7fdd541a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.684+0000 7fdd5ac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd5410c250 0x7fdd5419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.684+0000 7fdd5ac00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5419ab60 con 0x7fdd54102e20 2024-05-10T05:32:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.684+0000 7fdd5ac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5419aca0 con 0x7fdd5410c250 2024-05-10T05:32:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.684+0000 7fdd5ac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5419ae60 con 0x7fdd54103770 2024-05-10T05:32:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd53e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd54102e20 0x7fdd541a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd58c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd5410c250 0x7fdd5419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd58c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd5410c250 0x7fdd5419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:52106/0 (socket says 172.21.15.131:52106) 2024-05-10T05:32:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd58c00700 1 -- 172.21.15.131:0/3219272272 learned_addr learned my addr 172.21.15.131:0/3219272272 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd53400700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd54103770 0x7fdd541a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd58c00700 1 -- 172.21.15.131:0/3219272272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd54103770 msgr2=0x7fdd541a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd58c00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd54103770 0x7fdd541a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd58c00700 1 -- 172.21.15.131:0/3219272272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd54102e20 msgr2=0x7fdd541a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd58c00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd54102e20 0x7fdd541a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd58c00700 1 -- 172.21.15.131:0/3219272272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd440095f0 con 0x7fdd5410c250 2024-05-10T05:32:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.685+0000 7fdd53400700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd54103770 0x7fdd541a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.686+0000 7fdd58c00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd5410c250 0x7fdd5419a590 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00ef30 tx=0x7fdd4c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.686+0000 7fdd50c00700 1 -- 172.21.15.131:0/3219272272 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c018070 con 0x7fdd5410c250 2024-05-10T05:32:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.686+0000 7fdd5ac00700 1 -- 172.21.15.131:0/3219272272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5419b0e0 con 0x7fdd5410c250 2024-05-10T05:32:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.686+0000 7fdd50c00700 1 -- 172.21.15.131:0/3219272272 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd4c00f040 con 0x7fdd5410c250 2024-05-10T05:32:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.686+0000 7fdd50c00700 1 -- 172.21.15.131:0/3219272272 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00cdb0 con 0x7fdd5410c250 2024-05-10T05:32:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.686+0000 7fdd5ac00700 1 -- 172.21.15.131:0/3219272272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd541a7430 con 0x7fdd5410c250 2024-05-10T05:32:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.687+0000 7fdd5ac00700 1 -- 172.21.15.131:0/3219272272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd54107410 con 0x7fdd5410c250 2024-05-10T05:32:28.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.689+0000 7fdd50c00700 1 -- 172.21.15.131:0/3219272272 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fdd4c003a00 con 0x7fdd5410c250 2024-05-10T05:32:28.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.691+0000 7fdd50c00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd380a11b0 0x7fdd380a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:28.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.691+0000 7fdd50c00700 1 -- 172.21.15.131:0/3219272272 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fdd4c0cc840 con 0x7fdd5410c250 2024-05-10T05:32:28.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.691+0000 7fdd53e00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd380a11b0 0x7fdd380a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:28.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.692+0000 7fdd53e00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd380a11b0 0x7fdd380a36c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd540ffce0 tx=0x7fdd4401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.694+0000 7fdd50c00700 1 -- 172.21.15.131:0/3219272272 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdd4c095ea0 con 0x7fdd5410c250 2024-05-10T05:32:28.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.928+0000 7fdd5ac00700 1 -- 172.21.15.131:0/3219272272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdd54069580 con 0x7fdd5410c250 2024-05-10T05:32:28.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.929+0000 7fdd50c00700 1 -- 172.21.15.131:0/3219272272 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdd4c0994c0 con 0x7fdd5410c250 2024-05-10T05:32:28.930 INFO:teuthology.orchestra.run.smithi131.stdout:55834574905 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 -- 172.21.15.131:0/3219272272 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd380a11b0 msgr2=0x7fdd380a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd380a11b0 0x7fdd380a36c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd540ffce0 tx=0x7fdd4401a040 comp rx=0 tx=0).stop 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 -- 172.21.15.131:0/3219272272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd5410c250 msgr2=0x7fdd5419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd5410c250 0x7fdd5419a590 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00ef30 tx=0x7fdd4c00c3a0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 -- 172.21.15.131:0/3219272272 shutdown_connections 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdd380a11b0 0x7fdd380a36c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd54102e20 0x7fdd541a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd5410c250 0x7fdd5419a590 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 --2- 172.21.15.131:0/3219272272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd54103770 0x7fdd541a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 -- 172.21.15.131:0/3219272272 >> 172.21.15.131:0/3219272272 conn(0x7fdd540fc7a0 msgr2=0x7fdd540fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 -- 172.21.15.131:0/3219272272 shutdown_connections 2024-05-10T05:32:28.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:28.932+0000 7fdd5ac00700 1 -- 172.21.15.131:0/3219272272 wait complete. 2024-05-10T05:32:29.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:28 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3219272272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T05:32:29.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:28 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3219272272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T05:32:29.303 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:29.353 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:29.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:28 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3219272272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T05:32:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:30.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:29 smithi131 ceph-mon[37264]: pgmap v259: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T05:32:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:29 smithi157 ceph-mon[78427]: pgmap v259: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T05:32:30.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:29 smithi142 ceph-mon[77667]: pgmap v259: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T05:32:31.151 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574904 got 55834574905 for osd.1 2024-05-10T05:32:31.152 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:31.211 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.953+0000 7f422d000700 1 -- 172.21.15.131:0/3483011112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4228101b00 msgr2=0x7f4228101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.953+0000 7f422d000700 1 --2- 172.21.15.131:0/3483011112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4228101b00 0x7f4228101f10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4214009990 tx=0x7f4214009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.953+0000 7f422d000700 1 -- 172.21.15.131:0/3483011112 shutdown_connections 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.953+0000 7f422d000700 1 --2- 172.21.15.131:0/3483011112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4228102450 0x7f422810e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.953+0000 7f422d000700 1 --2- 172.21.15.131:0/3483011112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4228107b20 0x7f4228107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.953+0000 7f422d000700 1 --2- 172.21.15.131:0/3483011112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4228101b00 0x7f4228101f10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.953+0000 7f422d000700 1 -- 172.21.15.131:0/3483011112 >> 172.21.15.131:0/3483011112 conn(0x7f4228077650 msgr2=0x7f4228077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.953+0000 7f422d000700 1 -- 172.21.15.131:0/3483011112 shutdown_connections 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.954+0000 7f422d000700 1 -- 172.21.15.131:0/3483011112 wait complete. 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.955+0000 7f422d000700 1 Processor -- start 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.955+0000 7f422d000700 1 -- start start 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.955+0000 7f422d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4228101b00 0x7f422819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.955+0000 7f422d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4228102450 0x7f422819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.955+0000 7f422d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4228107b20 0x7f422819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.955+0000 7f422d000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422819f3b0 con 0x7f4228107b20 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.955+0000 7f422d000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422819f520 con 0x7f4228102450 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.955+0000 7f422d000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422819f6e0 con 0x7f4228101b00 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4227400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4228101b00 0x7f422819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4226a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4228102450 0x7f422819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4227e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4228107b20 0x7f422819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4227400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4228101b00 0x7f422819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45270/0 (socket says 172.21.15.131:45270) 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4226a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4228102450 0x7f422819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:52116/0 (socket says 172.21.15.131:52116) 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4227400700 1 -- 172.21.15.131:0/2105458080 learned_addr learned my addr 172.21.15.131:0/2105458080 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4227400700 1 -- 172.21.15.131:0/2105458080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4228102450 msgr2=0x7f422819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4227400700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4228102450 0x7f422819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4227400700 1 -- 172.21.15.131:0/2105458080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4228107b20 msgr2=0x7f422819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4227400700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4228107b20 0x7f422819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4227400700 1 -- 172.21.15.131:0/2105458080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42140095f0 con 0x7f4228101b00 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.956+0000 7f4226a00700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4228102450 0x7f422819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.957+0000 7f4227400700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4228101b00 0x7f422819a5a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f421800e970 tx=0x7f421800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.957+0000 7f421fe00700 1 -- 172.21.15.131:0/2105458080 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218009790 con 0x7f4228101b00 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.957+0000 7f422d000700 1 -- 172.21.15.131:0/2105458080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422819f9c0 con 0x7f4228101b00 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.957+0000 7f421fe00700 1 -- 172.21.15.131:0/2105458080 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f421800c7e0 con 0x7f4228101b00 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.957+0000 7f421fe00700 1 -- 172.21.15.131:0/2105458080 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42180054e0 con 0x7f4228101b00 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.957+0000 7f422d000700 1 -- 172.21.15.131:0/2105458080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42281a72c0 con 0x7f4228101b00 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.958+0000 7f422d000700 1 -- 172.21.15.131:0/2105458080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4228103200 con 0x7f4228101b00 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.960+0000 7f421fe00700 1 -- 172.21.15.131:0/2105458080 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f421800c950 con 0x7f4228101b00 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.961+0000 7f421fe00700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f420c0a11b0 0x7f420c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.962+0000 7f421fe00700 1 -- 172.21.15.131:0/2105458080 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4218014070 con 0x7f4228101b00 2024-05-10T05:32:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.962+0000 7f4226a00700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f420c0a11b0 0x7f420c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:31.734 INFO:teuthology.orchestra.run.smithi131.stdout:188978561062 2024-05-10T05:32:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.962+0000 7f4226a00700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f420c0a11b0 0x7f420c0a36c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f42280ff770 tx=0x7f421401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:30.966+0000 7f421fe00700 1 -- 172.21.15.131:0/2105458080 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4218096260 con 0x7f4228101b00 2024-05-10T05:32:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.193+0000 7f422d000700 1 -- 172.21.15.131:0/2105458080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4228111a30 con 0x7f4228101b00 2024-05-10T05:32:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.193+0000 7f421fe00700 1 -- 172.21.15.131:0/2105458080 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4218099880 con 0x7f4228101b00 2024-05-10T05:32:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 -- 172.21.15.131:0/2105458080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f420c0a11b0 msgr2=0x7f420c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f420c0a11b0 0x7f420c0a36c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f42280ff770 tx=0x7f421401a040 comp rx=0 tx=0).stop 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 -- 172.21.15.131:0/2105458080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4228101b00 msgr2=0x7f422819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4228101b00 0x7f422819a5a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f421800e970 tx=0x7f421800ec80 comp rx=0 tx=0).stop 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 -- 172.21.15.131:0/2105458080 shutdown_connections 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f420c0a11b0 0x7f420c0a36c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4228107b20 0x7f422819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4228101b00 0x7f422819a5a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 --2- 172.21.15.131:0/2105458080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4228102450 0x7f422819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 -- 172.21.15.131:0/2105458080 >> 172.21.15.131:0/2105458080 conn(0x7f4228077650 msgr2=0x7f42280fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 -- 172.21.15.131:0/2105458080 shutdown_connections 2024-05-10T05:32:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:31.196+0000 7f422d000700 1 -- 172.21.15.131:0/2105458080 wait complete. 2024-05-10T05:32:32.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:31 smithi131 ceph-mon[37264]: pgmap v260: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T05:32:32.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:31 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2105458080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T05:32:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:31 smithi157 ceph-mon[78427]: pgmap v260: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T05:32:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:31 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2105458080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T05:32:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:31 smithi142 ceph-mon[77667]: pgmap v260: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T05:32:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:31 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2105458080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T05:32:32.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.713+0000 7fca59a00700 1 -- 172.21.15.131:0/2698072531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca541071c0 msgr2=0x7fca5410eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:32.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.713+0000 7fca59a00700 1 --2- 172.21.15.131:0/2698072531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca541071c0 0x7fca5410eb20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00b0a0 tx=0x7fca4c00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.713+0000 7fca59a00700 1 -- 172.21.15.131:0/2698072531 shutdown_connections 2024-05-10T05:32:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.713+0000 7fca59a00700 1 --2- 172.21.15.131:0/2698072531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca541071c0 0x7fca5410eb20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.713+0000 7fca59a00700 1 --2- 172.21.15.131:0/2698072531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca54101f40 0x7fca54102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.713+0000 7fca59a00700 1 --2- 172.21.15.131:0/2698072531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca54106870 0x7fca54106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.713+0000 7fca59a00700 1 -- 172.21.15.131:0/2698072531 >> 172.21.15.131:0/2698072531 conn(0x7fca540fd7f0 msgr2=0x7fca540ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.713+0000 7fca59a00700 1 -- 172.21.15.131:0/2698072531 shutdown_connections 2024-05-10T05:32:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.713+0000 7fca59a00700 1 -- 172.21.15.131:0/2698072531 wait complete. 2024-05-10T05:32:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.715+0000 7fca59a00700 1 Processor -- start 2024-05-10T05:32:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.715+0000 7fca59a00700 1 -- start start 2024-05-10T05:32:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.715+0000 7fca59a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca54101f40 0x7fca541a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.715+0000 7fca59a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca54106870 0x7fca541a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.715+0000 7fca59a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca541071c0 0x7fca5419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.715+0000 7fca59a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5419ad10 con 0x7fca54106870 2024-05-10T05:32:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.715+0000 7fca59a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5419ae80 con 0x7fca54101f40 2024-05-10T05:32:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.715+0000 7fca59a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5419b040 con 0x7fca541071c0 2024-05-10T05:32:32.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.715+0000 7fca52000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca54106870 0x7fca541a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:32.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca52000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca54106870 0x7fca541a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35964/0 (socket says 172.21.15.131:35964) 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca52000700 1 -- 172.21.15.131:0/3774726378 learned_addr learned my addr 172.21.15.131:0/3774726378 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca52a00700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca54101f40 0x7fca541a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca53400700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca541071c0 0x7fca5419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca52000700 1 -- 172.21.15.131:0/3774726378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca541071c0 msgr2=0x7fca5419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca52000700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca541071c0 0x7fca5419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca52000700 1 -- 172.21.15.131:0/3774726378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca54101f40 msgr2=0x7fca541a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca52000700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca54101f40 0x7fca541a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca52000700 1 -- 172.21.15.131:0/3774726378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c009d00 con 0x7fca54106870 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca52000700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca54106870 0x7fca541a0d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca4000e970 tx=0x7fca4000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.716+0000 7fca4b400700 1 -- 172.21.15.131:0/3774726378 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40009790 con 0x7fca54106870 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.717+0000 7fca59a00700 1 -- 172.21.15.131:0/3774726378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5419b320 con 0x7fca54106870 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.717+0000 7fca4b400700 1 -- 172.21.15.131:0/3774726378 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca4000eec0 con 0x7fca54106870 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.717+0000 7fca4b400700 1 -- 172.21.15.131:0/3774726378 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40005590 con 0x7fca54106870 2024-05-10T05:32:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.717+0000 7fca59a00700 1 -- 172.21.15.131:0/3774726378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca541a7780 con 0x7fca54106870 2024-05-10T05:32:32.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.718+0000 7fca59a00700 1 -- 172.21.15.131:0/3774726378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca541033f0 con 0x7fca54106870 2024-05-10T05:32:32.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.720+0000 7fca4b400700 1 -- 172.21.15.131:0/3774726378 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fca4000cbd0 con 0x7fca54106870 2024-05-10T05:32:32.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.721+0000 7fca4b400700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca380a1230 0x7fca380a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:32.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.721+0000 7fca4b400700 1 -- 172.21.15.131:0/3774726378 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fca40014070 con 0x7fca54106870 2024-05-10T05:32:32.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.721+0000 7fca52a00700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca380a1230 0x7fca380a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.722+0000 7fca52a00700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca380a1230 0x7fca380a3740 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca44009c60 tx=0x7fca44009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.726+0000 7fca4b400700 1 -- 172.21.15.131:0/3774726378 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fca40096230 con 0x7fca54106870 2024-05-10T05:32:32.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.956+0000 7fca59a00700 1 -- 172.21.15.131:0/3774726378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fca5404edc0 con 0x7fca54106870 2024-05-10T05:32:32.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.957+0000 7fca4b400700 1 -- 172.21.15.131:0/3774726378 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca40004ef0 con 0x7fca54106870 2024-05-10T05:32:32.957 INFO:teuthology.orchestra.run.smithi131.stdout:137438953518 2024-05-10T05:32:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.959+0000 7fca59a00700 1 -- 172.21.15.131:0/3774726378 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca380a1230 msgr2=0x7fca380a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.959+0000 7fca59a00700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca380a1230 0x7fca380a3740 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca44009c60 tx=0x7fca44009260 comp rx=0 tx=0).stop 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.959+0000 7fca59a00700 1 -- 172.21.15.131:0/3774726378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca54106870 msgr2=0x7fca541a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.959+0000 7fca59a00700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca54106870 0x7fca541a0d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca4000e970 tx=0x7fca4000ec80 comp rx=0 tx=0).stop 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.960+0000 7fca59a00700 1 -- 172.21.15.131:0/3774726378 shutdown_connections 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.960+0000 7fca59a00700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca380a1230 0x7fca380a3740 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.960+0000 7fca59a00700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca541071c0 0x7fca5419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.960+0000 7fca59a00700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca54101f40 0x7fca541a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.960+0000 7fca59a00700 1 --2- 172.21.15.131:0/3774726378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca54106870 0x7fca541a0d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.960+0000 7fca59a00700 1 -- 172.21.15.131:0/3774726378 >> 172.21.15.131:0/3774726378 conn(0x7fca540fd7f0 msgr2=0x7fca54108120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.960+0000 7fca59a00700 1 -- 172.21.15.131:0/3774726378 shutdown_connections 2024-05-10T05:32:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:32.960+0000 7fca59a00700 1 -- 172.21.15.131:0/3774726378 wait complete. 2024-05-10T05:32:33.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.187+0000 7fc5e2600700 1 -- 172.21.15.131:0/551814903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0740d0 msgr2=0x7fc5dc10dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:33.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.187+0000 7fc5e2600700 1 --2- 172.21.15.131:0/551814903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0740d0 0x7fc5dc10dfa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009990 tx=0x7fc5c8009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.188+0000 7fc5e2600700 1 -- 172.21.15.131:0/551814903 shutdown_connections 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.188+0000 7fc5e2600700 1 --2- 172.21.15.131:0/551814903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc073780 0x7fc5dc073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.188+0000 7fc5e2600700 1 --2- 172.21.15.131:0/551814903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc10e4e0 0x7fc5dc110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.188+0000 7fc5e2600700 1 --2- 172.21.15.131:0/551814903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0740d0 0x7fc5dc10dfa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.188+0000 7fc5e2600700 1 -- 172.21.15.131:0/551814903 >> 172.21.15.131:0/551814903 conn(0x7fc5dc0fd9c0 msgr2=0x7fc5dc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.188+0000 7fc5e2600700 1 -- 172.21.15.131:0/551814903 shutdown_connections 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.188+0000 7fc5e2600700 1 -- 172.21.15.131:0/551814903 wait complete. 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.189+0000 7fc5e2600700 1 Processor -- start 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.189+0000 7fc5e2600700 1 -- start start 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5e2600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc073780 0x7fc5dc1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5e2600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0740d0 0x7fc5dc1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5e2600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc10e4e0 0x7fc5dc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5e2600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc19ab30 con 0x7fc5dc10e4e0 2024-05-10T05:32:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5e2600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc19ac70 con 0x7fc5dc073780 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5e2600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc19ae30 con 0x7fc5dc0740d0 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5db400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc073780 0x7fc5dc1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5db400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc073780 0x7fc5dc1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:53992/0 (socket says 172.21.15.131:53992) 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5db400700 1 -- 172.21.15.131:0/4283932445 learned_addr learned my addr 172.21.15.131:0/4283932445 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5dbe00700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc10e4e0 0x7fc5dc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.190+0000 7fc5daa00700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0740d0 0x7fc5dc1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.191+0000 7fc5db400700 1 -- 172.21.15.131:0/4283932445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0740d0 msgr2=0x7fc5dc1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.191+0000 7fc5db400700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0740d0 0x7fc5dc1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.191+0000 7fc5db400700 1 -- 172.21.15.131:0/4283932445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc10e4e0 msgr2=0x7fc5dc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.191+0000 7fc5db400700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc10e4e0 0x7fc5dc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.191+0000 7fc5db400700 1 -- 172.21.15.131:0/4283932445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c80095f0 con 0x7fc5dc073780 2024-05-10T05:32:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.191+0000 7fc5db400700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc073780 0x7fc5dc1a0600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00e9a0 tx=0x7fc5cc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.191+0000 7fc5d3e00700 1 -- 172.21.15.131:0/4283932445 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc009790 con 0x7fc5dc073780 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.191+0000 7fc5e2600700 1 -- 172.21.15.131:0/4283932445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5dc19b110 con 0x7fc5dc073780 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.192+0000 7fc5d3e00700 1 -- 172.21.15.131:0/4283932445 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5cc010430 con 0x7fc5dc073780 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.192+0000 7fc5d3e00700 1 -- 172.21.15.131:0/4283932445 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc018680 con 0x7fc5dc073780 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.192+0000 7fc5e2600700 1 -- 172.21.15.131:0/4283932445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5dc1a75d0 con 0x7fc5dc073780 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.193+0000 7fc5e2600700 1 -- 172.21.15.131:0/4283932445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5dc1096f0 con 0x7fc5dc073780 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.195+0000 7fc5d3e00700 1 -- 172.21.15.131:0/4283932445 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc5cc0105a0 con 0x7fc5dc073780 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.196+0000 7fc5d3e00700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11b0 0x7fc5c00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.196+0000 7fc5d3e00700 1 -- 172.21.15.131:0/4283932445 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc5cc014070 con 0x7fc5dc073780 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.196+0000 7fc5daa00700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11b0 0x7fc5c00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.197+0000 7fc5daa00700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11b0 0x7fc5c00a36c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc19bed0 tx=0x7fc5c801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.198+0000 7fc5d3e00700 1 -- 172.21.15.131:0/4283932445 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc5cc096230 con 0x7fc5dc073780 2024-05-10T05:32:33.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.427+0000 7fc5e2600700 1 -- 172.21.15.131:0/4283932445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fc5dc101530 con 0x7fc5dc073780 2024-05-10T05:32:33.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.427+0000 7fc5d3e00700 1 -- 172.21.15.131:0/4283932445 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc5cc099850 con 0x7fc5dc073780 2024-05-10T05:32:33.427 INFO:teuthology.orchestra.run.smithi131.stdout:322122547219 2024-05-10T05:32:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.430+0000 7fc5e2600700 1 -- 172.21.15.131:0/4283932445 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11b0 msgr2=0x7fc5c00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.430+0000 7fc5e2600700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11b0 0x7fc5c00a36c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc19bed0 tx=0x7fc5c801a040 comp rx=0 tx=0).stop 2024-05-10T05:32:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.430+0000 7fc5e2600700 1 -- 172.21.15.131:0/4283932445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc073780 msgr2=0x7fc5dc1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.430+0000 7fc5e2600700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc073780 0x7fc5dc1a0600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00e9a0 tx=0x7fc5cc00ecb0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.431+0000 7fc5e2600700 1 -- 172.21.15.131:0/4283932445 shutdown_connections 2024-05-10T05:32:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.431+0000 7fc5e2600700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc5c00a11b0 0x7fc5c00a36c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.431+0000 7fc5e2600700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5dc10e4e0 0x7fc5dc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.431+0000 7fc5e2600700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5dc073780 0x7fc5dc1a0600 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.431+0000 7fc5e2600700 1 --2- 172.21.15.131:0/4283932445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0740d0 0x7fc5dc1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.431+0000 7fc5e2600700 1 -- 172.21.15.131:0/4283932445 >> 172.21.15.131:0/4283932445 conn(0x7fc5dc0fd9c0 msgr2=0x7fc5dc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.431+0000 7fc5e2600700 1 -- 172.21.15.131:0/4283932445 shutdown_connections 2024-05-10T05:32:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.431+0000 7fc5e2600700 1 -- 172.21.15.131:0/4283932445 wait complete. 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.574+0000 7f4297e00700 1 -- 172.21.15.131:0/974157805 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c00bb10 con 0x7f4298101f40 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.575+0000 7f429fe00700 1 -- 172.21.15.131:0/974157805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4298101f40 msgr2=0x7f4298102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.575+0000 7f429fe00700 1 --2- 172.21.15.131:0/974157805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4298101f40 0x7f4298102350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f428c009990 tx=0x7f428c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.575+0000 7f429fe00700 1 -- 172.21.15.131:0/974157805 shutdown_connections 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.575+0000 7f429fe00700 1 --2- 172.21.15.131:0/974157805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4298101f40 0x7f4298102350 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.575+0000 7f429fe00700 1 --2- 172.21.15.131:0/974157805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42981071c0 0x7f429810eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.575+0000 7f429fe00700 1 --2- 172.21.15.131:0/974157805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298106870 0x7f4298106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.575+0000 7f429fe00700 1 -- 172.21.15.131:0/974157805 >> 172.21.15.131:0/974157805 conn(0x7f42980fd7f0 msgr2=0x7f42980ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.575+0000 7f429fe00700 1 -- 172.21.15.131:0/974157805 shutdown_connections 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.575+0000 7f429fe00700 1 -- 172.21.15.131:0/974157805 wait complete. 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.576+0000 7f429fe00700 1 Processor -- start 2024-05-10T05:32:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.576+0000 7f429fe00700 1 -- start start 2024-05-10T05:32:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4298101f40 0x7f42981a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298106870 0x7f42981a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42981071c0 0x7f429819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:33.916 INFO:teuthology.orchestra.run.smithi131.stdout:240518168605 2024-05-10T05:32:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429fe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429819ab50 con 0x7f42981071c0 2024-05-10T05:32:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429fe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429819ac90 con 0x7f4298106870 2024-05-10T05:32:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429819ae50 con 0x7f4298101f40 2024-05-10T05:32:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4298101f40 0x7f42981a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42981071c0 0x7f429819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4298101f40 0x7f42981a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35774/0 (socket says 172.21.15.131:35774) 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429d400700 1 -- 172.21.15.131:0/1250263742 learned_addr learned my addr 172.21.15.131:0/1250263742 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.577+0000 7f429ca00700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298106870 0x7f42981a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.578+0000 7f429de00700 1 -- 172.21.15.131:0/1250263742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4298101f40 msgr2=0x7f42981a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.578+0000 7f429de00700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4298101f40 0x7f42981a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.578+0000 7f429de00700 1 -- 172.21.15.131:0/1250263742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298106870 msgr2=0x7f42981a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.578+0000 7f429de00700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298106870 0x7f42981a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.578+0000 7f429de00700 1 -- 172.21.15.131:0/1250263742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4284009520 con 0x7f42981071c0 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.578+0000 7f429de00700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42981071c0 0x7f429819a580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f429000efc0 tx=0x7f429000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.578+0000 7f4296000700 1 -- 172.21.15.131:0/1250263742 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290018070 con 0x7f42981071c0 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.578+0000 7f429fe00700 1 -- 172.21.15.131:0/1250263742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c0095f0 con 0x7f42981071c0 2024-05-10T05:32:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.578+0000 7f4296000700 1 -- 172.21.15.131:0/1250263742 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f429000f040 con 0x7f42981071c0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.578+0000 7f4296000700 1 -- 172.21.15.131:0/1250263742 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429000cd40 con 0x7f42981071c0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.579+0000 7f429fe00700 1 -- 172.21.15.131:0/1250263742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42981a7420 con 0x7f42981071c0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.580+0000 7f429fe00700 1 -- 172.21.15.131:0/1250263742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42981033f0 con 0x7f42981071c0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.582+0000 7f4296000700 1 -- 172.21.15.131:0/1250263742 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4290003a00 con 0x7f42981071c0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.583+0000 7f4296000700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f42800a12b0 0x7f42800a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.583+0000 7f4296000700 1 -- 172.21.15.131:0/1250263742 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f42900ccf60 con 0x7f42981071c0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.583+0000 7f429d400700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f42800a12b0 0x7f42800a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.584+0000 7f429d400700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f42800a12b0 0x7f42800a37c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f428c00b2e0 tx=0x7f428c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.586+0000 7f4296000700 1 -- 172.21.15.131:0/1250263742 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f42900fad60 con 0x7f42981071c0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.821+0000 7f429fe00700 1 -- 172.21.15.131:0/1250263742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f4298101410 con 0x7f42981071c0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.822+0000 7f4296000700 1 -- 172.21.15.131:0/1250263742 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f42900965c0 con 0x7f42981071c0 2024-05-10T05:32:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 -- 172.21.15.131:0/1250263742 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f42800a12b0 msgr2=0x7f42800a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f42800a12b0 0x7f42800a37c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f428c00b2e0 tx=0x7f428c01a040 comp rx=0 tx=0).stop 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 -- 172.21.15.131:0/1250263742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42981071c0 msgr2=0x7f429819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42981071c0 0x7f429819a580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f429000efc0 tx=0x7f429000c3a0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 -- 172.21.15.131:0/1250263742 shutdown_connections 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f42800a12b0 0x7f42800a37c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42981071c0 0x7f429819a580 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4298101f40 0x7f42981a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 --2- 172.21.15.131:0/1250263742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298106870 0x7f42981a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 -- 172.21.15.131:0/1250263742 >> 172.21.15.131:0/1250263742 conn(0x7f42980fd7f0 msgr2=0x7f4298108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 -- 172.21.15.131:0/1250263742 shutdown_connections 2024-05-10T05:32:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:33.824+0000 7f429fe00700 1 -- 172.21.15.131:0/1250263742 wait complete. 2024-05-10T05:32:34.108 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:33 smithi131 ceph-mon[37264]: pgmap v261: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:34.108 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:33 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3774726378' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T05:32:34.108 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:33 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4283932445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T05:32:34.108 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:33 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1250263742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T05:32:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:33 smithi157 ceph-mon[78427]: pgmap v261: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:33 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3774726378' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T05:32:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:33 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4283932445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T05:32:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:33 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1250263742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T05:32:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:33 smithi142 ceph-mon[77667]: pgmap v261: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:33 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3774726378' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T05:32:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:33 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4283932445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T05:32:34.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:33 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1250263742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T05:32:35.075 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561060 got 188978561062 for osd.6 2024-05-10T05:32:35.076 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:35.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953515 got 137438953518 for osd.4 2024-05-10T05:32:35.480 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.717+0000 7f6f8b200700 1 -- 172.21.15.131:0/3988425811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c0740d0 msgr2=0x7f6f8c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.717+0000 7f6f8b200700 1 --2- 172.21.15.131:0/3988425811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c0740d0 0x7f6f8c10de00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009990 tx=0x7f6f78009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.717+0000 7f6f8b200700 1 -- 172.21.15.131:0/3988425811 shutdown_connections 2024-05-10T05:32:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.717+0000 7f6f8b200700 1 --2- 172.21.15.131:0/3988425811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f8c10e340 0x7f6f8c1107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.717+0000 7f6f8b200700 1 --2- 172.21.15.131:0/3988425811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c073780 0x7f6f8c073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.717+0000 7f6f8b200700 1 --2- 172.21.15.131:0/3988425811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c0740d0 0x7f6f8c10de00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.717+0000 7f6f8b200700 1 -- 172.21.15.131:0/3988425811 >> 172.21.15.131:0/3988425811 conn(0x7f6f8c077650 msgr2=0x7f6f8c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.717+0000 7f6f8b200700 1 -- 172.21.15.131:0/3988425811 shutdown_connections 2024-05-10T05:32:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.717+0000 7f6f8b200700 1 -- 172.21.15.131:0/3988425811 wait complete. 2024-05-10T05:32:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.718+0000 7f6f8b200700 1 Processor -- start 2024-05-10T05:32:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.718+0000 7f6f8b200700 1 -- start start 2024-05-10T05:32:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f8b200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c073780 0x7f6f8c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f8b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c0740d0 0x7f6f8c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f8b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f8c10e340 0x7f6f8c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f8b200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c19f3b0 con 0x7f6f8c10e340 2024-05-10T05:32:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f8b200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c19f520 con 0x7f6f8c073780 2024-05-10T05:32:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f8b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c19f6e0 con 0x7f6f8c0740d0 2024-05-10T05:32:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f89e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c073780 0x7f6f8c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f89e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c073780 0x7f6f8c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:54050/0 (socket says 172.21.15.131:54050) 2024-05-10T05:32:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f89e00700 1 -- 172.21.15.131:0/4041470911 learned_addr learned my addr 172.21.15.131:0/4041470911 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f8a800700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f8c10e340 0x7f6f8c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f89400700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c0740d0 0x7f6f8c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f89e00700 1 -- 172.21.15.131:0/4041470911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c0740d0 msgr2=0x7f6f8c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f89e00700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c0740d0 0x7f6f8c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f89e00700 1 -- 172.21.15.131:0/4041470911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f8c10e340 msgr2=0x7f6f8c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f89e00700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f8c10e340 0x7f6f8c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.719+0000 7f6f89e00700 1 -- 172.21.15.131:0/4041470911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f780095f0 con 0x7f6f8c073780 2024-05-10T05:32:35.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.720+0000 7f6f89e00700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c073780 0x7f6f8c19a5a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00e9a0 tx=0x7f6f7c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:35.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.720+0000 7f6f82a00700 1 -- 172.21.15.131:0/4041470911 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c009790 con 0x7f6f8c073780 2024-05-10T05:32:35.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.720+0000 7f6f82a00700 1 -- 172.21.15.131:0/4041470911 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f7c010430 con 0x7f6f8c073780 2024-05-10T05:32:35.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.720+0000 7f6f8b200700 1 -- 172.21.15.131:0/4041470911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c19f9c0 con 0x7f6f8c073780 2024-05-10T05:32:35.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.720+0000 7f6f82a00700 1 -- 172.21.15.131:0/4041470911 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c018680 con 0x7f6f8c073780 2024-05-10T05:32:35.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.720+0000 7f6f8b200700 1 -- 172.21.15.131:0/4041470911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c1a72c0 con 0x7f6f8c073780 2024-05-10T05:32:35.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.722+0000 7f6f8b200700 1 -- 172.21.15.131:0/4041470911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c109550 con 0x7f6f8c073780 2024-05-10T05:32:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.724+0000 7f6f82a00700 1 -- 172.21.15.131:0/4041470911 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6f7c00ca10 con 0x7f6f8c073780 2024-05-10T05:32:35.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.725+0000 7f6f82a00700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6f700a1230 0x7f6f700a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:35.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.725+0000 7f6f82a00700 1 -- 172.21.15.131:0/4041470911 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6f7c014070 con 0x7f6f8c073780 2024-05-10T05:32:35.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.725+0000 7f6f89400700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6f700a1230 0x7f6f700a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:35.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.726+0000 7f6f89400700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6f700a1230 0x7f6f700a3740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c100fa0 tx=0x7f6f7801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:35.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.729+0000 7f6f82a00700 1 -- 172.21.15.131:0/4041470911 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6f7c052d60 con 0x7f6f8c073780 2024-05-10T05:32:35.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.752+0000 7f3d67e00700 1 -- 172.21.15.131:0/2326851136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680740d0 msgr2=0x7f3d6810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:35.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.752+0000 7f3d67e00700 1 --2- 172.21.15.131:0/2326851136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680740d0 0x7f3d6810de00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009990 tx=0x7f3d54009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.752+0000 7f3d67e00700 1 -- 172.21.15.131:0/2326851136 shutdown_connections 2024-05-10T05:32:35.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.752+0000 7f3d67e00700 1 --2- 172.21.15.131:0/2326851136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d6810e340 0x7f3d681107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.752+0000 7f3d67e00700 1 --2- 172.21.15.131:0/2326851136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d68073780 0x7f3d68073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.752+0000 7f3d67e00700 1 --2- 172.21.15.131:0/2326851136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680740d0 0x7f3d6810de00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.752+0000 7f3d67e00700 1 -- 172.21.15.131:0/2326851136 >> 172.21.15.131:0/2326851136 conn(0x7f3d68077650 msgr2=0x7f3d68077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.752+0000 7f3d67e00700 1 -- 172.21.15.131:0/2326851136 shutdown_connections 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.752+0000 7f3d67e00700 1 -- 172.21.15.131:0/2326851136 wait complete. 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.753+0000 7f3d67e00700 1 Processor -- start 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.754+0000 7f3d67e00700 1 -- start start 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.754+0000 7f3d67e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d68073780 0x7f3d6819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.754+0000 7f3d67e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d680740d0 0x7f3d6819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.754+0000 7f3d67e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d6810e340 0x7f3d6819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d67e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6819f3b0 con 0x7f3d680740d0 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d67e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6819f520 con 0x7f3d6810e340 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d67e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6819f6e0 con 0x7f3d68073780 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d66a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d68073780 0x7f3d6819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d66a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d68073780 0x7f3d6819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35820/0 (socket says 172.21.15.131:35820) 2024-05-10T05:32:35.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d66a00700 1 -- 172.21.15.131:0/2499392412 learned_addr learned my addr 172.21.15.131:0/2499392412 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d67400700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d6810e340 0x7f3d6819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d66a00700 1 -- 172.21.15.131:0/2499392412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d6810e340 msgr2=0x7f3d6819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d66a00700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d6810e340 0x7f3d6819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d66a00700 1 -- 172.21.15.131:0/2499392412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d680740d0 msgr2=0x7f3d6819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d66a00700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d680740d0 0x7f3d6819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d66a00700 1 -- 172.21.15.131:0/2499392412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d540095f0 con 0x7f3d68073780 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d67400700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d6810e340 0x7f3d6819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.755+0000 7f3d66a00700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d68073780 0x7f3d6819a5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800e9a0 tx=0x7f3d5800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.756+0000 7f3d5f400700 1 -- 172.21.15.131:0/2499392412 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58009790 con 0x7f3d68073780 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.756+0000 7f3d67e00700 1 -- 172.21.15.131:0/2499392412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6819f9c0 con 0x7f3d68073780 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.756+0000 7f3d5f400700 1 -- 172.21.15.131:0/2499392412 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d5800cbd0 con 0x7f3d68073780 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.756+0000 7f3d5f400700 1 -- 172.21.15.131:0/2499392412 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58018680 con 0x7f3d68073780 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.756+0000 7f3d67e00700 1 -- 172.21.15.131:0/2499392412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d681a72c0 con 0x7f3d68073780 2024-05-10T05:32:35.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.757+0000 7f3d67e00700 1 -- 172.21.15.131:0/2499392412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d68109550 con 0x7f3d68073780 2024-05-10T05:32:35.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.761+0000 7f3d5f400700 1 -- 172.21.15.131:0/2499392412 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3d5800cd40 con 0x7f3d68073780 2024-05-10T05:32:35.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.764+0000 7f3d5f400700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d4c0a1230 0x7f3d4c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:35.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.764+0000 7f3d5f400700 1 -- 172.21.15.131:0/2499392412 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3d58014070 con 0x7f3d68073780 2024-05-10T05:32:35.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.764+0000 7f3d5f400700 1 -- 172.21.15.131:0/2499392412 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3d580963b0 con 0x7f3d68073780 2024-05-10T05:32:35.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.765+0000 7f3d66000700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d4c0a1230 0x7f3d4c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:35.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.766+0000 7f3d66000700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d4c0a1230 0x7f3d4c0a3740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d540098e0 tx=0x7f3d5400b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:35.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.980+0000 7f6f8b200700 1 -- 172.21.15.131:0/4041470911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f6f8c074cf0 con 0x7f6f8c073780 2024-05-10T05:32:35.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.981+0000 7f6f82a00700 1 -- 172.21.15.131:0/4041470911 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f7c0961e0 con 0x7f6f8c073780 2024-05-10T05:32:35.981 INFO:teuthology.orchestra.run.smithi131.stdout:270582939674 2024-05-10T05:32:35.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.983+0000 7f6f8b200700 1 -- 172.21.15.131:0/4041470911 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6f700a1230 msgr2=0x7f6f700a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.983+0000 7f6f8b200700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6f700a1230 0x7f6f700a3740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c100fa0 tx=0x7f6f7801a040 comp rx=0 tx=0).stop 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.983+0000 7f6f8b200700 1 -- 172.21.15.131:0/4041470911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c073780 msgr2=0x7f6f8c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.983+0000 7f6f8b200700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c073780 0x7f6f8c19a5a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00e9a0 tx=0x7f6f7c00ecb0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.984+0000 7f6f8b200700 1 -- 172.21.15.131:0/4041470911 shutdown_connections 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.984+0000 7f6f8b200700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6f700a1230 0x7f6f700a3740 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.984+0000 7f6f8b200700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f8c10e340 0x7f6f8c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.984+0000 7f6f8b200700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c073780 0x7f6f8c19a5a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.984+0000 7f6f8b200700 1 --2- 172.21.15.131:0/4041470911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c0740d0 0x7f6f8c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.984+0000 7f6f8b200700 1 -- 172.21.15.131:0/4041470911 >> 172.21.15.131:0/4041470911 conn(0x7f6f8c077650 msgr2=0x7f6f8c0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.984+0000 7f6f8b200700 1 -- 172.21.15.131:0/4041470911 shutdown_connections 2024-05-10T05:32:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.984+0000 7f6f8b200700 1 -- 172.21.15.131:0/4041470911 wait complete. 2024-05-10T05:32:36.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.010+0000 7f3d67e00700 1 -- 172.21.15.131:0/2499392412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3d6819b900 con 0x7f3d68073780 2024-05-10T05:32:36.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.010+0000 7f3d5f400700 1 -- 172.21.15.131:0/2499392412 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3d580999d0 con 0x7f3d68073780 2024-05-10T05:32:36.010 INFO:teuthology.orchestra.run.smithi131.stdout:158913789994 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 -- 172.21.15.131:0/2499392412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d4c0a1230 msgr2=0x7f3d4c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d4c0a1230 0x7f3d4c0a3740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d540098e0 tx=0x7f3d5400b260 comp rx=0 tx=0).stop 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 -- 172.21.15.131:0/2499392412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d68073780 msgr2=0x7f3d6819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d68073780 0x7f3d6819a5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800e9a0 tx=0x7f3d5800ed60 comp rx=0 tx=0).stop 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 -- 172.21.15.131:0/2499392412 shutdown_connections 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d4c0a1230 0x7f3d4c0a3740 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d6810e340 0x7f3d6819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d68073780 0x7f3d6819a5a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 --2- 172.21.15.131:0/2499392412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d680740d0 0x7f3d6819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 -- 172.21.15.131:0/2499392412 >> 172.21.15.131:0/2499392412 conn(0x7f3d68077650 msgr2=0x7f3d680fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 -- 172.21.15.131:0/2499392412 shutdown_connections 2024-05-10T05:32:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.013+0000 7f3d67e00700 1 -- 172.21.15.131:0/2499392412 wait complete. 2024-05-10T05:32:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:35 smithi131 ceph-mon[37264]: pgmap v262: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-10T05:32:36.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:35 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3301307196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T05:32:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:35 smithi157 ceph-mon[78427]: pgmap v262: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-10T05:32:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:35 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3301307196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T05:32:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.335+0000 7fe288e00700 1 -- 172.21.15.131:0/2466686862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284073550 msgr2=0x7fe284073960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.335+0000 7fe288e00700 1 --2- 172.21.15.131:0/2466686862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284073550 0x7fe284073960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe274009990 tx=0x7fe274009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.336+0000 7fe288e00700 1 -- 172.21.15.131:0/2466686862 shutdown_connections 2024-05-10T05:32:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.336+0000 7fe288e00700 1 --2- 172.21.15.131:0/2466686862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284073550 0x7fe284073960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.336+0000 7fe288e00700 1 --2- 172.21.15.131:0/2466686862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe28410e540 0x7fe284110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.336+0000 7fe288e00700 1 --2- 172.21.15.131:0/2466686862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe284073ea0 0x7fe28410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.336+0000 7fe288e00700 1 -- 172.21.15.131:0/2466686862 >> 172.21.15.131:0/2466686862 conn(0x7fe2840fd9c0 msgr2=0x7fe2840ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.336+0000 7fe288e00700 1 -- 172.21.15.131:0/2466686862 shutdown_connections 2024-05-10T05:32:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.336+0000 7fe288e00700 1 -- 172.21.15.131:0/2466686862 wait complete. 2024-05-10T05:32:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.337+0000 7fe288e00700 1 Processor -- start 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.337+0000 7fe288e00700 1 -- start start 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.337+0000 7fe288e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284073550 0x7fe2841a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.337+0000 7fe288e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284073ea0 0x7fe2841a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.337+0000 7fe288e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe28410e540 0x7fe28419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.337+0000 7fe288e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28419ab60 con 0x7fe284073550 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.337+0000 7fe288e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28419aca0 con 0x7fe28410e540 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.337+0000 7fe288e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28419ae60 con 0x7fe284073ea0 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.337+0000 7fe281e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284073550 0x7fe2841a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe281400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284073ea0 0x7fe2841a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe281e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284073550 0x7fe2841a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47244/0 (socket says 172.21.15.131:47244) 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe281e00700 1 -- 172.21.15.131:0/3301307196 learned_addr learned my addr 172.21.15.131:0/3301307196 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe282800700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe28410e540 0x7fe28419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe281e00700 1 -- 172.21.15.131:0/3301307196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284073ea0 msgr2=0x7fe2841a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe281e00700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284073ea0 0x7fe2841a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe281e00700 1 -- 172.21.15.131:0/3301307196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe28410e540 msgr2=0x7fe28419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe281e00700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe28410e540 0x7fe28419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe281e00700 1 -- 172.21.15.131:0/3301307196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe270009520 con 0x7fe284073550 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe281400700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284073ea0 0x7fe2841a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe282800700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe28410e540 0x7fe28419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.338+0000 7fe281e00700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284073550 0x7fe2841a0630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe274000c00 tx=0x7fe274005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.339+0000 7fe27aa00700 1 -- 172.21.15.131:0/3301307196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27401d070 con 0x7fe284073550 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.339+0000 7fe27aa00700 1 -- 172.21.15.131:0/3301307196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe274022470 con 0x7fe284073550 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.339+0000 7fe288e00700 1 -- 172.21.15.131:0/3301307196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2740095f0 con 0x7fe284073550 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.339+0000 7fe27aa00700 1 -- 172.21.15.131:0/3301307196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274017650 con 0x7fe284073550 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.339+0000 7fe288e00700 1 -- 172.21.15.131:0/3301307196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28419b3f0 con 0x7fe284073550 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.340+0000 7fe288e00700 1 -- 172.21.15.131:0/3301307196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe284109750 con 0x7fe284073550 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.342+0000 7fe27aa00700 1 -- 172.21.15.131:0/3301307196 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe2740225e0 con 0x7fe284073550 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.344+0000 7fe27aa00700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2680a11b0 0x7fe2680a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.344+0000 7fe27aa00700 1 -- 172.21.15.131:0/3301307196 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe2740cde90 con 0x7fe284073550 2024-05-10T05:32:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.344+0000 7fe281400700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2680a11b0 0x7fe2680a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.345+0000 7fe281400700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2680a11b0 0x7fe2680a36c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe270009fd0 tx=0x7fe270009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.348+0000 7fe27aa00700 1 -- 172.21.15.131:0/3301307196 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe274097610 con 0x7fe284073550 2024-05-10T05:32:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.568+0000 7fe288e00700 1 -- 172.21.15.131:0/3301307196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe284074c90 con 0x7fe284073550 2024-05-10T05:32:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.569+0000 7fe27aa00700 1 -- 172.21.15.131:0/3301307196 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe274027090 con 0x7fe284073550 2024-05-10T05:32:36.315 INFO:teuthology.orchestra.run.smithi131.stdout:77309411382 2024-05-10T05:32:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 -- 172.21.15.131:0/3301307196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2680a11b0 msgr2=0x7fe2680a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2680a11b0 0x7fe2680a36c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe270009fd0 tx=0x7fe270009260 comp rx=0 tx=0).stop 2024-05-10T05:32:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 -- 172.21.15.131:0/3301307196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284073550 msgr2=0x7fe2841a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284073550 0x7fe2841a0630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe274000c00 tx=0x7fe274005740 comp rx=0 tx=0).stop 2024-05-10T05:32:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 -- 172.21.15.131:0/3301307196 shutdown_connections 2024-05-10T05:32:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2680a11b0 0x7fe2680a36c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe284073550 0x7fe2841a0630 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe28410e540 0x7fe28419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 --2- 172.21.15.131:0/3301307196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe284073ea0 0x7fe2841a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 -- 172.21.15.131:0/3301307196 >> 172.21.15.131:0/3301307196 conn(0x7fe2840fd9c0 msgr2=0x7fe2840fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.573+0000 7fe288e00700 1 -- 172.21.15.131:0/3301307196 shutdown_connections 2024-05-10T05:32:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:35.574+0000 7fe288e00700 1 -- 172.21.15.131:0/3301307196 wait complete. 2024-05-10T05:32:36.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:35 smithi142 ceph-mon[77667]: pgmap v262: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-10T05:32:36.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:35 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3301307196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T05:32:36.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.169+0000 7f9e5f600700 1 -- 172.21.15.131:0/2403806070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e58101f70 msgr2=0x7f9e58102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.169+0000 7f9e5f600700 1 --2- 172.21.15.131:0/2403806070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e58101f70 0x7f9e58102380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009990 tx=0x7f9e4c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.169+0000 7f9e5f600700 1 -- 172.21.15.131:0/2403806070 shutdown_connections 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.169+0000 7f9e5f600700 1 --2- 172.21.15.131:0/2403806070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e58101f70 0x7f9e58102380 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.170+0000 7f9e5f600700 1 --2- 172.21.15.131:0/2403806070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e581071f0 0x7f9e5810eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.170+0000 7f9e5f600700 1 --2- 172.21.15.131:0/2403806070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e581068a0 0x7f9e58106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.170+0000 7f9e5f600700 1 -- 172.21.15.131:0/2403806070 >> 172.21.15.131:0/2403806070 conn(0x7f9e580fd9c0 msgr2=0x7f9e580ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.170+0000 7f9e5f600700 1 -- 172.21.15.131:0/2403806070 shutdown_connections 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.170+0000 7f9e5f600700 1 -- 172.21.15.131:0/2403806070 wait complete. 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.171+0000 7f9e5f600700 1 Processor -- start 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.171+0000 7f9e5f600700 1 -- start start 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.171+0000 7f9e5f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e58101f70 0x7f9e581a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.171+0000 7f9e5f600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e581068a0 0x7f9e581a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:36.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.171+0000 7f9e5f600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e581071f0 0x7f9e5819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.171+0000 7f9e5f600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819ab60 con 0x7f9e581071f0 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.171+0000 7f9e5f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819aca0 con 0x7f9e58101f70 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.171+0000 7f9e5f600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819ae60 con 0x7f9e581068a0 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e5d600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e581071f0 0x7f9e5819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e5d600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e581071f0 0x7f9e5819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47288/0 (socket says 172.21.15.131:47288) 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e5d600700 1 -- 172.21.15.131:0/1610420814 learned_addr learned my addr 172.21.15.131:0/1610420814 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e5cc00700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e58101f70 0x7f9e581a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e57e00700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e581068a0 0x7f9e581a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e5d600700 1 -- 172.21.15.131:0/1610420814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e581068a0 msgr2=0x7f9e581a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e5d600700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e581068a0 0x7f9e581a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e5d600700 1 -- 172.21.15.131:0/1610420814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e58101f70 msgr2=0x7f9e581a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e5d600700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e58101f70 0x7f9e581a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e5d600700 1 -- 172.21.15.131:0/1610420814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e44009520 con 0x7f9e581071f0 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e57e00700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e581068a0 0x7f9e581a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e5d600700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e581071f0 0x7f9e5819a590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000ef30 tx=0x7f9e5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.172+0000 7f9e55600700 1 -- 172.21.15.131:0/1610420814 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50018070 con 0x7f9e581071f0 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.173+0000 7f9e55600700 1 -- 172.21.15.131:0/1610420814 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e5000f040 con 0x7f9e581071f0 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.173+0000 7f9e5f600700 1 -- 172.21.15.131:0/1610420814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c0095f0 con 0x7f9e581071f0 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.173+0000 7f9e55600700 1 -- 172.21.15.131:0/1610420814 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000cdb0 con 0x7f9e581071f0 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.173+0000 7f9e5f600700 1 -- 172.21.15.131:0/1610420814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e581a7430 con 0x7f9e581071f0 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.174+0000 7f9e5f600700 1 -- 172.21.15.131:0/1610420814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e58103420 con 0x7f9e581071f0 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.176+0000 7f9e55600700 1 -- 172.21.15.131:0/1610420814 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9e50003a00 con 0x7f9e581071f0 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.177+0000 7f9e55600700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e400a1230 0x7f9e400a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:36.531 INFO:teuthology.orchestra.run.smithi131.stdout:210453397538 2024-05-10T05:32:36.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.177+0000 7f9e55600700 1 -- 172.21.15.131:0/1610420814 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9e500cc5d0 con 0x7f9e581071f0 2024-05-10T05:32:36.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.177+0000 7f9e5cc00700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e400a1230 0x7f9e400a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:36.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.178+0000 7f9e5cc00700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e400a1230 0x7f9e400a3740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00b2e0 tx=0x7f9e4c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:36.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.181+0000 7f9e55600700 1 -- 172.21.15.131:0/1610420814 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9e50095c30 con 0x7f9e581071f0 2024-05-10T05:32:36.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.390+0000 7f9e5f600700 1 -- 172.21.15.131:0/1610420814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9e58101500 con 0x7f9e581071f0 2024-05-10T05:32:36.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.391+0000 7f9e55600700 1 -- 172.21.15.131:0/1610420814 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e50099250 con 0x7f9e581071f0 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.393+0000 7f9e5f600700 1 -- 172.21.15.131:0/1610420814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e400a1230 msgr2=0x7f9e400a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.393+0000 7f9e5f600700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e400a1230 0x7f9e400a3740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00b2e0 tx=0x7f9e4c01a040 comp rx=0 tx=0).stop 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.393+0000 7f9e5f600700 1 -- 172.21.15.131:0/1610420814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e581071f0 msgr2=0x7f9e5819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.393+0000 7f9e5f600700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e581071f0 0x7f9e5819a590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000ef30 tx=0x7f9e5000c3a0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.393+0000 7f9e5f600700 1 -- 172.21.15.131:0/1610420814 shutdown_connections 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.393+0000 7f9e5f600700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e400a1230 0x7f9e400a3740 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.393+0000 7f9e5f600700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e581071f0 0x7f9e5819a590 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.393+0000 7f9e5f600700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e58101f70 0x7f9e581a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.393+0000 7f9e5f600700 1 --2- 172.21.15.131:0/1610420814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e581068a0 0x7f9e581a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.393+0000 7f9e5f600700 1 -- 172.21.15.131:0/1610420814 >> 172.21.15.131:0/1610420814 conn(0x7f9e580fd9c0 msgr2=0x7f9e58106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.394+0000 7f9e5f600700 1 -- 172.21.15.131:0/1610420814 shutdown_connections 2024-05-10T05:32:36.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:36.394+0000 7f9e5f600700 1 -- 172.21.15.131:0/1610420814 wait complete. 2024-05-10T05:32:36.542 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168602 got 240518168605 for osd.8 2024-05-10T05:32:36.542 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:37.004 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789991 got 158913789994 for osd.5 2024-05-10T05:32:37.004 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:37.114 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:36 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4041470911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T05:32:37.114 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:36 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2499392412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T05:32:37.114 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:36 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1610420814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T05:32:37.136 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411380 got 77309411382 for osd.2 2024-05-10T05:32:37.136 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:36 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4041470911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T05:32:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:36 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2499392412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T05:32:37.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:36 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1610420814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T05:32:37.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:36 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4041470911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T05:32:37.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:36 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2499392412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T05:32:37.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:36 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1610420814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T05:32:37.396 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939671 got 270582939674 for osd.9 2024-05-10T05:32:37.396 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:37.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.644+0000 7f8bd9600700 1 -- 172.21.15.131:0/3077142051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bd4073550 msgr2=0x7f8bd4073960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:37.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.644+0000 7f8bd9600700 1 --2- 172.21.15.131:0/3077142051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bd4073550 0x7f8bd4073960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009990 tx=0x7f8bc4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.644+0000 7f8bd9600700 1 -- 172.21.15.131:0/3077142051 shutdown_connections 2024-05-10T05:32:37.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.644+0000 7f8bd9600700 1 --2- 172.21.15.131:0/3077142051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bd4073550 0x7f8bd4073960 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.644+0000 7f8bd9600700 1 --2- 172.21.15.131:0/3077142051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd410e3a0 0x7f8bd41107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.644+0000 7f8bd9600700 1 --2- 172.21.15.131:0/3077142051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd4073ea0 0x7f8bd410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.644+0000 7f8bd9600700 1 -- 172.21.15.131:0/3077142051 >> 172.21.15.131:0/3077142051 conn(0x7f8bd4077650 msgr2=0x7f8bd4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.644+0000 7f8bd9600700 1 -- 172.21.15.131:0/3077142051 shutdown_connections 2024-05-10T05:32:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.644+0000 7f8bd9600700 1 -- 172.21.15.131:0/3077142051 wait complete. 2024-05-10T05:32:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.645+0000 7f8bd9600700 1 Processor -- start 2024-05-10T05:32:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.645+0000 7f8bd9600700 1 -- start start 2024-05-10T05:32:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd9600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd4073550 0x7f8bd419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:37.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd9600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bd4073ea0 0x7f8bd419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:37.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410e3a0 0x7f8bd419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:37.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd9600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd419f3c0 con 0x7f8bd4073ea0 2024-05-10T05:32:37.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd9600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd419f530 con 0x7f8bd410e3a0 2024-05-10T05:32:37.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd9600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd419f6f0 con 0x7f8bd4073550 2024-05-10T05:32:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bd4073ea0 0x7f8bd419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bd4073ea0 0x7f8bd419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47304/0 (socket says 172.21.15.131:47304) 2024-05-10T05:32:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd3400700 1 -- 172.21.15.131:0/1232831332 learned_addr learned my addr 172.21.15.131:0/1232831332 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd8c00700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410e3a0 0x7f8bd419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.646+0000 7f8bd3e00700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd4073550 0x7f8bd419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd3400700 1 -- 172.21.15.131:0/1232831332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd4073550 msgr2=0x7f8bd419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd3400700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd4073550 0x7f8bd419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd3400700 1 -- 172.21.15.131:0/1232831332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410e3a0 msgr2=0x7f8bd419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd3400700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410e3a0 0x7f8bd419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd3400700 1 -- 172.21.15.131:0/1232831332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40095f0 con 0x7f8bd4073ea0 2024-05-10T05:32:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd8c00700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410e3a0 0x7f8bd419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd3400700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bd4073ea0 0x7f8bd419ab50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e970 tx=0x7f8bc000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd0c00700 1 -- 172.21.15.131:0/1232831332 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0009790 con 0x7f8bd4073ea0 2024-05-10T05:32:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd9600700 1 -- 172.21.15.131:0/1232831332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd419f9d0 con 0x7f8bd4073ea0 2024-05-10T05:32:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd0c00700 1 -- 172.21.15.131:0/1232831332 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bc000c7e0 con 0x7f8bd4073ea0 2024-05-10T05:32:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd0c00700 1 -- 172.21.15.131:0/1232831332 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00054e0 con 0x7f8bd4073ea0 2024-05-10T05:32:37.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.647+0000 7f8bd9600700 1 -- 172.21.15.131:0/1232831332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd41a72d0 con 0x7f8bd4073ea0 2024-05-10T05:32:37.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.649+0000 7f8bd9600700 1 -- 172.21.15.131:0/1232831332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd41095b0 con 0x7f8bd4073ea0 2024-05-10T05:32:37.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.651+0000 7f8bd0c00700 1 -- 172.21.15.131:0/1232831332 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f8bc000c950 con 0x7f8bd4073ea0 2024-05-10T05:32:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.652+0000 7f8bd0c00700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8bb80a1230 0x7f8bb80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:37.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.652+0000 7f8bd0c00700 1 -- 172.21.15.131:0/1232831332 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8bc0014070 con 0x7f8bd4073ea0 2024-05-10T05:32:37.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.653+0000 7f8bd3e00700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8bb80a1230 0x7f8bb80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.654+0000 7f8bd3e00700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8bb80a1230 0x7f8bb80a3740 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400b2e0 tx=0x7f8bc401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:37.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.659+0000 7f8bd0c00700 1 -- 172.21.15.131:0/1232831332 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8bc00962a0 con 0x7f8bd4073ea0 2024-05-10T05:32:37.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.873+0000 7f8bd9600700 1 -- 172.21.15.131:0/1232831332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8bd419b910 con 0x7f8bd4073ea0 2024-05-10T05:32:37.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.873+0000 7f8bd0c00700 1 -- 172.21.15.131:0/1232831332 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8bc0005740 con 0x7f8bd4073ea0 2024-05-10T05:32:37.873 INFO:teuthology.orchestra.run.smithi131.stdout:107374182450 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.875+0000 7f8bd9600700 1 -- 172.21.15.131:0/1232831332 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8bb80a1230 msgr2=0x7f8bb80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.875+0000 7f8bd9600700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8bb80a1230 0x7f8bb80a3740 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400b2e0 tx=0x7f8bc401a040 comp rx=0 tx=0).stop 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.875+0000 7f8bd9600700 1 -- 172.21.15.131:0/1232831332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bd4073ea0 msgr2=0x7f8bd419ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.875+0000 7f8bd9600700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bd4073ea0 0x7f8bd419ab50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e970 tx=0x7f8bc000ec80 comp rx=0 tx=0).stop 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.876+0000 7f8bd9600700 1 -- 172.21.15.131:0/1232831332 shutdown_connections 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.876+0000 7f8bd9600700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8bb80a1230 0x7f8bb80a3740 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.876+0000 7f8bd9600700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410e3a0 0x7f8bd419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.876+0000 7f8bd9600700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd4073550 0x7f8bd419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.876+0000 7f8bd9600700 1 --2- 172.21.15.131:0/1232831332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bd4073ea0 0x7f8bd419ab50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.876+0000 7f8bd9600700 1 -- 172.21.15.131:0/1232831332 >> 172.21.15.131:0/1232831332 conn(0x7f8bd4077650 msgr2=0x7f8bd40fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.876+0000 7f8bd9600700 1 -- 172.21.15.131:0/1232831332 shutdown_connections 2024-05-10T05:32:37.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.876+0000 7f8bd9600700 1 -- 172.21.15.131:0/1232831332 wait complete. 2024-05-10T05:32:37.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.880+0000 7f84aac00700 1 -- 172.21.15.131:0/4274505356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a41014e0 msgr2=0x7f84a4105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:37.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.880+0000 7f84aac00700 1 --2- 172.21.15.131:0/4274505356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a41014e0 0x7f84a4105a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8490009990 tx=0x7f8490009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.880+0000 7f84aac00700 1 -- 172.21.15.131:0/4274505356 shutdown_connections 2024-05-10T05:32:37.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.880+0000 7f84aac00700 1 --2- 172.21.15.131:0/4274505356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a4100b90 0x7f84a4100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.880+0000 7f84aac00700 1 --2- 172.21.15.131:0/4274505356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a4105fe0 0x7f84a41064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.880+0000 7f84aac00700 1 --2- 172.21.15.131:0/4274505356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a41014e0 0x7f84a4105a10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.880+0000 7f84aac00700 1 -- 172.21.15.131:0/4274505356 >> 172.21.15.131:0/4274505356 conn(0x7f84a40fc7a0 msgr2=0x7f84a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:37.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.881+0000 7f84aac00700 1 -- 172.21.15.131:0/4274505356 shutdown_connections 2024-05-10T05:32:37.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.881+0000 7f84aac00700 1 -- 172.21.15.131:0/4274505356 wait complete. 2024-05-10T05:32:37.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.882+0000 7f84aac00700 1 Processor -- start 2024-05-10T05:32:37.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.882+0000 7f84aac00700 1 -- start start 2024-05-10T05:32:37.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84aac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a4100b90 0x7f84a41a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:37.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84aac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a41014e0 0x7f84a41a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:37.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84aac00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a4105fe0 0x7f84a419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:37.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84aac00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a419f000 con 0x7f84a4105fe0 2024-05-10T05:32:37.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84aac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a419f170 con 0x7f84a4100b90 2024-05-10T05:32:37.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84aac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a419f330 con 0x7f84a41014e0 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84a3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a41014e0 0x7f84a41a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84a8c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a4105fe0 0x7f84a419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84a3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a41014e0 0x7f84a41a5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35868/0 (socket says 172.21.15.131:35868) 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84a3400700 1 -- 172.21.15.131:0/1454944538 learned_addr learned my addr 172.21.15.131:0/1454944538 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.883+0000 7f84a3e00700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a4100b90 0x7f84a41a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.884+0000 7f84a3400700 1 -- 172.21.15.131:0/1454944538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a4100b90 msgr2=0x7f84a41a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.884+0000 7f84a3400700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a4100b90 0x7f84a41a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.884+0000 7f84a3400700 1 -- 172.21.15.131:0/1454944538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a4105fe0 msgr2=0x7f84a419ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.884+0000 7f84a3400700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a4105fe0 0x7f84a419ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.884+0000 7f84a3400700 1 -- 172.21.15.131:0/1454944538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900095f0 con 0x7f84a41014e0 2024-05-10T05:32:37.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.884+0000 7f84a3400700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a41014e0 0x7f84a41a5030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8490000c00 tx=0x7f8490005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:37.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.885+0000 7f84a0c00700 1 -- 172.21.15.131:0/1454944538 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849001d070 con 0x7f84a41014e0 2024-05-10T05:32:37.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.885+0000 7f84aac00700 1 -- 172.21.15.131:0/1454944538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a419f5b0 con 0x7f84a41014e0 2024-05-10T05:32:37.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.885+0000 7f84a0c00700 1 -- 172.21.15.131:0/1454944538 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8490022470 con 0x7f84a41014e0 2024-05-10T05:32:37.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.885+0000 7f84a0c00700 1 -- 172.21.15.131:0/1454944538 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490017650 con 0x7f84a41014e0 2024-05-10T05:32:37.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.885+0000 7f84aac00700 1 -- 172.21.15.131:0/1454944538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a41aba70 con 0x7f84a41014e0 2024-05-10T05:32:37.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.886+0000 7f84aac00700 1 -- 172.21.15.131:0/1454944538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a4069600 con 0x7f84a41014e0 2024-05-10T05:32:37.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.888+0000 7f84a0c00700 1 -- 172.21.15.131:0/1454944538 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f849000fad0 con 0x7f84a41014e0 2024-05-10T05:32:37.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.890+0000 7f84a0c00700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f84880a11b0 0x7f84880a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:37.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.890+0000 7f84a0c00700 1 -- 172.21.15.131:0/1454944538 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f84900ce4c0 con 0x7f84a41014e0 2024-05-10T05:32:37.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.890+0000 7f84a3e00700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f84880a11b0 0x7f84880a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:37.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.891+0000 7f84a3e00700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f84880a11b0 0x7f84880a36c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f84a406ab80 tx=0x7f8494009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:37.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:37.892+0000 7f84a0c00700 1 -- 172.21.15.131:0/1454944538 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8490097b20 con 0x7f84a41014e0 2024-05-10T05:32:37.925 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397536 got 210453397538 for osd.7 2024-05-10T05:32:37.925 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:38.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.125+0000 7f84aac00700 1 -- 172.21.15.131:0/1454944538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f84a41a0120 con 0x7f84a41014e0 2024-05-10T05:32:38.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.126+0000 7f84a0c00700 1 -- 172.21.15.131:0/1454944538 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f849009b140 con 0x7f84a41014e0 2024-05-10T05:32:38.126 INFO:teuthology.orchestra.run.smithi131.stdout:34359738431 2024-05-10T05:32:38.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.128+0000 7f84aac00700 1 -- 172.21.15.131:0/1454944538 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f84880a11b0 msgr2=0x7f84880a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:38.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.128+0000 7f84aac00700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f84880a11b0 0x7f84880a36c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f84a406ab80 tx=0x7f8494009260 comp rx=0 tx=0).stop 2024-05-10T05:32:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.129+0000 7f84aac00700 1 -- 172.21.15.131:0/1454944538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a41014e0 msgr2=0x7f84a41a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.129+0000 7f84aac00700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a41014e0 0x7f84a41a5030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8490000c00 tx=0x7f8490005740 comp rx=0 tx=0).stop 2024-05-10T05:32:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.129+0000 7f84aac00700 1 -- 172.21.15.131:0/1454944538 shutdown_connections 2024-05-10T05:32:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.129+0000 7f84aac00700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f84880a11b0 0x7f84880a36c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.129+0000 7f84aac00700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a4105fe0 0x7f84a419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.129+0000 7f84aac00700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a4100b90 0x7f84a41a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.129+0000 7f84aac00700 1 --2- 172.21.15.131:0/1454944538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84a41014e0 0x7f84a41a5030 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.129+0000 7f84aac00700 1 -- 172.21.15.131:0/1454944538 >> 172.21.15.131:0/1454944538 conn(0x7f84a40fc7a0 msgr2=0x7f84a41039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.129+0000 7f84aac00700 1 -- 172.21.15.131:0/1454944538 shutdown_connections 2024-05-10T05:32:38.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.129+0000 7f84aac00700 1 -- 172.21.15.131:0/1454944538 wait complete. 2024-05-10T05:32:38.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:37 smithi131 ceph-mon[37264]: pgmap v263: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:38.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:37 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1232831332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T05:32:38.241 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547218 got 322122547219 for osd.11 2024-05-10T05:32:38.241 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:37 smithi157 ceph-mon[78427]: pgmap v263: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:37 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1232831332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T05:32:38.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.247+0000 7fca1b600700 1 -- 172.21.15.131:0/1570905595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca14106860 msgr2=0x7fca14106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.247+0000 7fca1b600700 1 --2- 172.21.15.131:0/1570905595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca14106860 0x7fca14106c70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fca00009990 tx=0x7fca00009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.247+0000 7fca1b600700 1 -- 172.21.15.131:0/1570905595 shutdown_connections 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.247+0000 7fca1b600700 1 --2- 172.21.15.131:0/1570905595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca141071b0 0x7fca1410eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.247+0000 7fca1b600700 1 --2- 172.21.15.131:0/1570905595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca14101f30 0x7fca14102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.247+0000 7fca1b600700 1 --2- 172.21.15.131:0/1570905595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca14106860 0x7fca14106c70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.247+0000 7fca1b600700 1 -- 172.21.15.131:0/1570905595 >> 172.21.15.131:0/1570905595 conn(0x7fca140fd9c0 msgr2=0x7fca140ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.248+0000 7fca1b600700 1 -- 172.21.15.131:0/1570905595 shutdown_connections 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.248+0000 7fca1b600700 1 -- 172.21.15.131:0/1570905595 wait complete. 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.249+0000 7fca1b600700 1 Processor -- start 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.249+0000 7fca1b600700 1 -- start start 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.249+0000 7fca1b600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca14101f30 0x7fca141a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.249+0000 7fca1b600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca14106860 0x7fca141a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.249+0000 7fca1b600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca141071b0 0x7fca1419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.249+0000 7fca1b600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1419abf0 con 0x7fca14106860 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.249+0000 7fca1b600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1419ad60 con 0x7fca14101f30 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.249+0000 7fca1b600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1419aed0 con 0x7fca141071b0 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca13e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca14106860 0x7fca141a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca13e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca14106860 0x7fca141a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47324/0 (socket says 172.21.15.131:47324) 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca13e00700 1 -- 172.21.15.131:0/4110016447 learned_addr learned my addr 172.21.15.131:0/4110016447 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca19600700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca141071b0 0x7fca1419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca18c00700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca14101f30 0x7fca141a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca13e00700 1 -- 172.21.15.131:0/4110016447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca141071b0 msgr2=0x7fca1419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca13e00700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca141071b0 0x7fca1419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca13e00700 1 -- 172.21.15.131:0/4110016447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca14101f30 msgr2=0x7fca141a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca13e00700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca14101f30 0x7fca141a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca13e00700 1 -- 172.21.15.131:0/4110016447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca000095f0 con 0x7fca14106860 2024-05-10T05:32:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.250+0000 7fca13e00700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca14106860 0x7fca141a0bc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fca00000c00 tx=0x7fca00009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.251+0000 7fca11600700 1 -- 172.21.15.131:0/4110016447 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0001d070 con 0x7fca14106860 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.251+0000 7fca11600700 1 -- 172.21.15.131:0/4110016447 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca00021410 con 0x7fca14106860 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.251+0000 7fca1b600700 1 -- 172.21.15.131:0/4110016447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1419b150 con 0x7fca14106860 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.251+0000 7fca11600700 1 -- 172.21.15.131:0/4110016447 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00017650 con 0x7fca14106860 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.251+0000 7fca1b600700 1 -- 172.21.15.131:0/4110016447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca141a7510 con 0x7fca14106860 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.253+0000 7fca1b600700 1 -- 172.21.15.131:0/4110016447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca141033e0 con 0x7fca14106860 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.254+0000 7fca11600700 1 -- 172.21.15.131:0/4110016447 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fca00021580 con 0x7fca14106860 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.255+0000 7fca11600700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc9fc0a11b0 0x7fc9fc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.256+0000 7fca11600700 1 -- 172.21.15.131:0/4110016447 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fca000cde60 con 0x7fca14106860 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.256+0000 7fca18c00700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc9fc0a11b0 0x7fc9fc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.256+0000 7fca18c00700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc9fc0a11b0 0x7fc9fc0a36c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fca1419b760 tx=0x7fca08009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:38.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.260+0000 7fca11600700 1 -- 172.21.15.131:0/4110016447 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fca000974c0 con 0x7fca14106860 2024-05-10T05:32:38.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:37 smithi142 ceph-mon[77667]: pgmap v263: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:38.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:37 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1232831332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T05:32:38.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.477+0000 7fca1b600700 1 -- 172.21.15.131:0/4110016447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fca1419bd60 con 0x7fca14106860 2024-05-10T05:32:38.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.478+0000 7fca11600700 1 -- 172.21.15.131:0/4110016447 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fca0009aae0 con 0x7fca14106860 2024-05-10T05:32:38.479 INFO:teuthology.orchestra.run.smithi131.stdout:292057776151 2024-05-10T05:32:38.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.480+0000 7fca1b600700 1 -- 172.21.15.131:0/4110016447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc9fc0a11b0 msgr2=0x7fc9fc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:38.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.480+0000 7fca1b600700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc9fc0a11b0 0x7fc9fc0a36c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fca1419b760 tx=0x7fca08009260 comp rx=0 tx=0).stop 2024-05-10T05:32:38.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.481+0000 7fca1b600700 1 -- 172.21.15.131:0/4110016447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca14106860 msgr2=0x7fca141a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:38.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.481+0000 7fca1b600700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca14106860 0x7fca141a0bc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fca00000c00 tx=0x7fca00009580 comp rx=0 tx=0).stop 2024-05-10T05:32:38.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.481+0000 7fca1b600700 1 -- 172.21.15.131:0/4110016447 shutdown_connections 2024-05-10T05:32:38.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.481+0000 7fca1b600700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc9fc0a11b0 0x7fc9fc0a36c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.481+0000 7fca1b600700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca141071b0 0x7fca1419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.481+0000 7fca1b600700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca14101f30 0x7fca141a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.481+0000 7fca1b600700 1 --2- 172.21.15.131:0/4110016447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca14106860 0x7fca141a0bc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:38.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.481+0000 7fca1b600700 1 -- 172.21.15.131:0/4110016447 >> 172.21.15.131:0/4110016447 conn(0x7fca140fd9c0 msgr2=0x7fca14108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:38.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.481+0000 7fca1b600700 1 -- 172.21.15.131:0/4110016447 shutdown_connections 2024-05-10T05:32:38.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:38.481+0000 7fca1b600700 1 -- 172.21.15.131:0/4110016447 wait complete. 2024-05-10T05:32:38.724 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738429 got 34359738431 for osd.0 2024-05-10T05:32:38.724 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:38.971 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182447 got 107374182450 for osd.3 2024-05-10T05:32:38.971 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:39.064 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:38 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1454944538' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T05:32:39.064 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:38 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4110016447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T05:32:39.270 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776148 got 292057776151 for osd.10 2024-05-10T05:32:39.270 DEBUG:teuthology.parallel:result is None 2024-05-10T05:32:39.271 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-10T05:32:39.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T05:32:39.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:38 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1454944538' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T05:32:39.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:38 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4110016447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T05:32:39.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:38 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1454944538' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T05:32:39.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:38 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4110016447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T05:32:39.643 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:40.164 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:39 smithi131 ceph-mon[37264]: pgmap v264: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:39 smithi157 ceph-mon[78427]: pgmap v264: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:39 smithi142 ceph-mon[77667]: pgmap v264: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.933+0000 7f0732200700 1 -- 172.21.15.131:0/1447214798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072c106860 msgr2=0x7f072c106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.933+0000 7f0732200700 1 --2- 172.21.15.131:0/1447214798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072c106860 0x7f072c106c70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0718009990 tx=0x7f0718009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.933+0000 7f0732200700 1 -- 172.21.15.131:0/1447214798 shutdown_connections 2024-05-10T05:32:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.933+0000 7f0732200700 1 --2- 172.21.15.131:0/1447214798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f072c1071b0 0x7f072c10eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.933+0000 7f0732200700 1 --2- 172.21.15.131:0/1447214798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c101f30 0x7f072c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.933+0000 7f0732200700 1 --2- 172.21.15.131:0/1447214798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072c106860 0x7f072c106c70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.933+0000 7f0732200700 1 -- 172.21.15.131:0/1447214798 >> 172.21.15.131:0/1447214798 conn(0x7f072c0fd9c0 msgr2=0x7f072c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.933+0000 7f0732200700 1 -- 172.21.15.131:0/1447214798 shutdown_connections 2024-05-10T05:32:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.933+0000 7f0732200700 1 -- 172.21.15.131:0/1447214798 wait complete. 2024-05-10T05:32:40.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.934+0000 7f0732200700 1 Processor -- start 2024-05-10T05:32:40.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.934+0000 7f0732200700 1 -- start start 2024-05-10T05:32:40.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f0732200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c101f30 0x7f072c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f0732200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f072c106860 0x7f072c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f0732200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072c1071b0 0x7f072c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f0732200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c19d4f0 con 0x7f072c1071b0 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f0732200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c19d660 con 0x7f072c106860 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f0732200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c19d820 con 0x7f072c101f30 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072c1071b0 0x7f072c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072c1071b0 0x7f072c19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47346/0 (socket says 172.21.15.131:47346) 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072be00700 1 -- 172.21.15.131:0/2469038014 learned_addr learned my addr 172.21.15.131:0/2469038014 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072b400700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c101f30 0x7f072c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072aa00700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f072c106860 0x7f072c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072b400700 1 -- 172.21.15.131:0/2469038014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f072c106860 msgr2=0x7f072c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072b400700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f072c106860 0x7f072c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072b400700 1 -- 172.21.15.131:0/2469038014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072c1071b0 msgr2=0x7f072c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072b400700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072c1071b0 0x7f072c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072b400700 1 -- 172.21.15.131:0/2469038014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07180095f0 con 0x7f072c101f30 2024-05-10T05:32:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.935+0000 7f072be00700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072c1071b0 0x7f072c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.936+0000 7f072b400700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c101f30 0x7f072c1984d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f071c00e9f0 tx=0x7f071c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.936+0000 7f0723e00700 1 -- 172.21.15.131:0/2469038014 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c00c9c0 con 0x7f072c101f30 2024-05-10T05:32:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.936+0000 7f0732200700 1 -- 172.21.15.131:0/2469038014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c19db00 con 0x7f072c101f30 2024-05-10T05:32:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.936+0000 7f0723e00700 1 -- 172.21.15.131:0/2469038014 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f071c010430 con 0x7f072c101f30 2024-05-10T05:32:40.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.936+0000 7f0723e00700 1 -- 172.21.15.131:0/2469038014 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c018680 con 0x7f072c101f30 2024-05-10T05:32:40.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.936+0000 7f0732200700 1 -- 172.21.15.131:0/2469038014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c1a5400 con 0x7f072c101f30 2024-05-10T05:32:40.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.937+0000 7f0732200700 1 -- 172.21.15.131:0/2469038014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c10a260 con 0x7f072c101f30 2024-05-10T05:32:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.939+0000 7f0723e00700 1 -- 172.21.15.131:0/2469038014 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f071c010860 con 0x7f072c101f30 2024-05-10T05:32:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.940+0000 7f0723e00700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f07100a11b0 0x7f07100a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.941+0000 7f0723e00700 1 -- 172.21.15.131:0/2469038014 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f071c014070 con 0x7f072c101f30 2024-05-10T05:32:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.941+0000 7f072aa00700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f07100a11b0 0x7f07100a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.941+0000 7f072aa00700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f07100a11b0 0x7f07100a36c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f071800b2e0 tx=0x7f071801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:40.943+0000 7f0723e00700 1 -- 172.21.15.131:0/2469038014 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f071c096360 con 0x7f072c101f30 2024-05-10T05:32:41.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.146+0000 7f0732200700 1 -- 172.21.15.131:0/2469038014 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f072c199a40 con 0x7f07100a11b0 2024-05-10T05:32:41.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.157+0000 7f0723e00700 1 -- 172.21.15.131:0/2469038014 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340314 (secure 0 0 0) 0x7f072c199a40 con 0x7f07100a11b0 2024-05-10T05:32:41.162 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:41.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 -- 172.21.15.131:0/2469038014 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f07100a11b0 msgr2=0x7f07100a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:41.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f07100a11b0 0x7f07100a36c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f071800b2e0 tx=0x7f071801a040 comp rx=0 tx=0).stop 2024-05-10T05:32:41.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 -- 172.21.15.131:0/2469038014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c101f30 msgr2=0x7f072c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:41.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c101f30 0x7f072c1984d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f071c00e9f0 tx=0x7f071c00ed00 comp rx=0 tx=0).stop 2024-05-10T05:32:41.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 -- 172.21.15.131:0/2469038014 shutdown_connections 2024-05-10T05:32:41.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f07100a11b0 0x7f07100a36c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:41.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072c1071b0 0x7f072c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:41.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f072c101f30 0x7f072c1984d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:41.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 --2- 172.21.15.131:0/2469038014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f072c106860 0x7f072c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:41.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 -- 172.21.15.131:0/2469038014 >> 172.21.15.131:0/2469038014 conn(0x7f072c0fd9c0 msgr2=0x7f072c108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:41.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.161+0000 7f0732200700 1 -- 172.21.15.131:0/2469038014 shutdown_connections 2024-05-10T05:32:41.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:41.162+0000 7f0732200700 1 -- 172.21.15.131:0/2469038014 wait complete. 2024-05-10T05:32:41.166 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-10T05:32:41.589 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-05-10T05:32:40.840736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657302,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":556,"num_read_kb":395,"num_write":417,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380356,"kb_used_data":11028,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124561468,"statfs":{"total":1151940427776,"available":1151550943232,"internally_reserved":0,"allocated":11292672,"data_stored":7117566,"data_compressed":113718,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19128,"internal_metadata":377992520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003323"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905479+0000","last_change":"2024-05-10T05:31:51.576054+0000","last_active":"2024-05-10T05:32:14.905479+0000","last_peered":"2024-05-10T05:32:14.905479+0000","last_clean":"2024-05-10T05:32:14.905479+0000","last_became_active":"2024-05-10T05:31:51.575186+0000","last_became_peered":"2024-05-10T05:31:51.575186+0000","last_unstale":"2024-05-10T05:32:14.905479+0000","last_undegraded":"2024-05-10T05:32:14.905479+0000","last_fullsized":"2024-05-10T05:32:14.905479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:16:27.654822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912479+0000","last_change":"2024-05-10T05:31:51.576483+0000","last_active":"2024-05-10T05:32:14.912479+0000","last_peered":"2024-05-10T05:32:14.912479+0000","last_clean":"2024-05-10T05:32:14.912479+0000","last_became_active":"2024-05-10T05:31:51.574936+0000","last_became_peered":"2024-05-10T05:31:51.574936+0000","last_unstale":"2024-05-10T05:32:14.912479+0000","last_undegraded":"2024-05-10T05:32:14.912479+0000","last_fullsized":"2024-05-10T05:32:14.912479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:56:51.436880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912069+0000","last_change":"2024-05-10T05:31:51.567114+0000","last_active":"2024-05-10T05:32:14.912069+0000","last_peered":"2024-05-10T05:32:14.912069+0000","last_clean":"2024-05-10T05:32:14.912069+0000","last_became_active":"2024-05-10T05:31:51.564552+0000","last_became_peered":"2024-05-10T05:31:51.564552+0000","last_unstale":"2024-05-10T05:32:14.912069+0000","last_undegraded":"2024-05-10T05:32:14.912069+0000","last_fullsized":"2024-05-10T05:32:14.912069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:40.268764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914315+0000","last_change":"2024-05-10T05:31:51.584471+0000","last_active":"2024-05-10T05:32:14.914315+0000","last_peered":"2024-05-10T05:32:14.914315+0000","last_clean":"2024-05-10T05:32:14.914315+0000","last_became_active":"2024-05-10T05:31:51.583749+0000","last_became_peered":"2024-05-10T05:31:51.583749+0000","last_unstale":"2024-05-10T05:32:14.914315+0000","last_undegraded":"2024-05-10T05:32:14.914315+0000","last_fullsized":"2024-05-10T05:32:14.914315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:30.775286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916284+0000","last_change":"2024-05-10T05:31:51.577698+0000","last_active":"2024-05-10T05:32:14.916284+0000","last_peered":"2024-05-10T05:32:14.916284+0000","last_clean":"2024-05-10T05:32:14.916284+0000","last_became_active":"2024-05-10T05:31:51.571469+0000","last_became_peered":"2024-05-10T05:31:51.571469+0000","last_unstale":"2024-05-10T05:32:14.916284+0000","last_undegraded":"2024-05-10T05:32:14.916284+0000","last_fullsized":"2024-05-10T05:32:14.916284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:35.161267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.911808+0000","last_change":"2024-05-10T05:31:51.573560+0000","last_active":"2024-05-10T05:32:14.911808+0000","last_peered":"2024-05-10T05:32:14.911808+0000","last_clean":"2024-05-10T05:32:14.911808+0000","last_became_active":"2024-05-10T05:31:51.573227+0000","last_became_peered":"2024-05-10T05:31:51.573227+0000","last_unstale":"2024-05-10T05:32:14.911808+0000","last_undegraded":"2024-05-10T05:32:14.911808+0000","last_fullsized":"2024-05-10T05:32:14.911808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:23.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919564+0000","last_change":"2024-05-10T05:31:51.573667+0000","last_active":"2024-05-10T05:32:14.919564+0000","last_peered":"2024-05-10T05:32:14.919564+0000","last_clean":"2024-05-10T05:32:14.919564+0000","last_became_active":"2024-05-10T05:31:51.573362+0000","last_became_peered":"2024-05-10T05:31:51.573362+0000","last_unstale":"2024-05-10T05:32:14.919564+0000","last_undegraded":"2024-05-10T05:32:14.919564+0000","last_fullsized":"2024-05-10T05:32:14.919564+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:00.300134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917768+0000","last_change":"2024-05-10T05:31:51.583596+0000","last_active":"2024-05-10T05:32:14.917768+0000","last_peered":"2024-05-10T05:32:14.917768+0000","last_clean":"2024-05-10T05:32:14.917768+0000","last_became_active":"2024-05-10T05:31:51.583235+0000","last_became_peered":"2024-05-10T05:31:51.583235+0000","last_unstale":"2024-05-10T05:32:14.917768+0000","last_undegraded":"2024-05-10T05:32:14.917768+0000","last_fullsized":"2024-05-10T05:32:14.917768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:05.575975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917655+0000","last_change":"2024-05-10T05:31:51.567627+0000","last_active":"2024-05-10T05:32:14.917655+0000","last_peered":"2024-05-10T05:32:14.917655+0000","last_clean":"2024-05-10T05:32:14.917655+0000","last_became_active":"2024-05-10T05:31:51.567276+0000","last_became_peered":"2024-05-10T05:31:51.567276+0000","last_unstale":"2024-05-10T05:32:14.917655+0000","last_undegraded":"2024-05-10T05:32:14.917655+0000","last_fullsized":"2024-05-10T05:32:14.917655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:16:27.914802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912121+0000","last_change":"2024-05-10T05:31:51.573859+0000","last_active":"2024-05-10T05:32:14.912121+0000","last_peered":"2024-05-10T05:32:14.912121+0000","last_clean":"2024-05-10T05:32:14.912121+0000","last_became_active":"2024-05-10T05:31:51.573493+0000","last_became_peered":"2024-05-10T05:31:51.573493+0000","last_unstale":"2024-05-10T05:32:14.912121+0000","last_undegraded":"2024-05-10T05:32:14.912121+0000","last_fullsized":"2024-05-10T05:32:14.912121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:21.024888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920160+0000","last_change":"2024-05-10T05:31:51.576056+0000","last_active":"2024-05-10T05:32:14.920160+0000","last_peered":"2024-05-10T05:32:14.920160+0000","last_clean":"2024-05-10T05:32:14.920160+0000","last_became_active":"2024-05-10T05:31:51.575146+0000","last_became_peered":"2024-05-10T05:31:51.575146+0000","last_unstale":"2024-05-10T05:32:14.920160+0000","last_undegraded":"2024-05-10T05:32:14.920160+0000","last_fullsized":"2024-05-10T05:32:14.920160+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:41.201052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914209+0000","last_change":"2024-05-10T05:31:51.584710+0000","last_active":"2024-05-10T05:32:14.914209+0000","last_peered":"2024-05-10T05:32:14.914209+0000","last_clean":"2024-05-10T05:32:14.914209+0000","last_became_active":"2024-05-10T05:31:51.584122+0000","last_became_peered":"2024-05-10T05:31:51.584122+0000","last_unstale":"2024-05-10T05:32:14.914209+0000","last_undegraded":"2024-05-10T05:32:14.914209+0000","last_fullsized":"2024-05-10T05:32:14.914209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:43.622981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.924380+0000","last_change":"2024-05-10T05:32:15.924380+0000","last_active":"2024-05-10T05:32:15.924380+0000","last_peered":"2024-05-10T05:32:15.924380+0000","last_clean":"2024-05-10T05:32:15.924380+0000","last_became_active":"2024-05-10T05:32:15.923952+0000","last_became_peered":"2024-05-10T05:32:15.923952+0000","last_unstale":"2024-05-10T05:32:15.924380+0000","last_undegraded":"2024-05-10T05:32:15.924380+0000","last_fullsized":"2024-05-10T05:32:15.924380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:04:14.087785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.911826+0000","last_change":"2024-05-10T05:31:51.574628+0000","last_active":"2024-05-10T05:32:14.911826+0000","last_peered":"2024-05-10T05:32:14.911826+0000","last_clean":"2024-05-10T05:32:14.911826+0000","last_became_active":"2024-05-10T05:31:51.574256+0000","last_became_peered":"2024-05-10T05:31:51.574256+0000","last_unstale":"2024-05-10T05:32:14.911826+0000","last_undegraded":"2024-05-10T05:32:14.911826+0000","last_fullsized":"2024-05-10T05:32:14.911826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:59.174727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.920243+0000","last_change":"2024-05-10T05:32:15.920243+0000","last_active":"2024-05-10T05:32:15.920243+0000","last_peered":"2024-05-10T05:32:15.920243+0000","last_clean":"2024-05-10T05:32:15.920243+0000","last_became_active":"2024-05-10T05:32:15.919822+0000","last_became_peered":"2024-05-10T05:32:15.919822+0000","last_unstale":"2024-05-10T05:32:15.920243+0000","last_undegraded":"2024-05-10T05:32:15.920243+0000","last_fullsized":"2024-05-10T05:32:15.920243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:59.392399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917143+0000","last_change":"2024-05-10T05:31:51.568919+0000","last_active":"2024-05-10T05:32:14.917143+0000","last_peered":"2024-05-10T05:32:14.917143+0000","last_clean":"2024-05-10T05:32:14.917143+0000","last_became_active":"2024-05-10T05:31:51.568582+0000","last_became_peered":"2024-05-10T05:31:51.568582+0000","last_unstale":"2024-05-10T05:32:14.917143+0000","last_undegraded":"2024-05-10T05:32:14.917143+0000","last_fullsized":"2024-05-10T05:32:14.917143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:08.377261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905717+0000","last_change":"2024-05-10T05:31:51.579025+0000","last_active":"2024-05-10T05:32:14.905717+0000","last_peered":"2024-05-10T05:32:14.905717+0000","last_clean":"2024-05-10T05:32:14.905717+0000","last_became_active":"2024-05-10T05:31:51.578704+0000","last_became_peered":"2024-05-10T05:31:51.578704+0000","last_unstale":"2024-05-10T05:32:14.905717+0000","last_undegraded":"2024-05-10T05:32:14.905717+0000","last_fullsized":"2024-05-10T05:32:14.905717+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:16.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913723+0000","last_change":"2024-05-10T05:31:51.573193+0000","last_active":"2024-05-10T05:32:14.913723+0000","last_peered":"2024-05-10T05:32:14.913723+0000","last_clean":"2024-05-10T05:32:14.913723+0000","last_became_active":"2024-05-10T05:31:51.572874+0000","last_became_peered":"2024-05-10T05:31:51.572874+0000","last_unstale":"2024-05-10T05:32:14.913723+0000","last_undegraded":"2024-05-10T05:32:14.913723+0000","last_fullsized":"2024-05-10T05:32:14.913723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:09.867661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919059+0000","last_change":"2024-05-10T05:31:51.581107+0000","last_active":"2024-05-10T05:32:14.919059+0000","last_peered":"2024-05-10T05:32:14.919059+0000","last_clean":"2024-05-10T05:32:14.919059+0000","last_became_active":"2024-05-10T05:31:51.580765+0000","last_became_peered":"2024-05-10T05:31:51.580765+0000","last_unstale":"2024-05-10T05:32:14.919059+0000","last_undegraded":"2024-05-10T05:32:14.919059+0000","last_fullsized":"2024-05-10T05:32:14.919059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:04.164486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:36.554881+0000","last_change":"2024-05-10T05:31:50.986534+0000","last_active":"2024-05-10T05:32:36.554881+0000","last_peered":"2024-05-10T05:32:36.554881+0000","last_clean":"2024-05-10T05:32:36.554881+0000","last_became_active":"2024-05-10T05:31:50.983493+0000","last_became_peered":"2024-05-10T05:31:50.983493+0000","last_unstale":"2024-05-10T05:32:36.554881+0000","last_undegraded":"2024-05-10T05:32:36.554881+0000","last_fullsized":"2024-05-10T05:32:36.554881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:18:56.926317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918423+0000","last_change":"2024-05-10T05:31:51.587121+0000","last_active":"2024-05-10T05:32:14.918423+0000","last_peered":"2024-05-10T05:32:14.918423+0000","last_clean":"2024-05-10T05:32:14.918423+0000","last_became_active":"2024-05-10T05:31:51.586641+0000","last_became_peered":"2024-05-10T05:31:51.586641+0000","last_unstale":"2024-05-10T05:32:14.918423+0000","last_undegraded":"2024-05-10T05:32:14.918423+0000","last_fullsized":"2024-05-10T05:32:14.918423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:52.654918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918650+0000","last_change":"2024-05-10T05:31:50.999755+0000","last_active":"2024-05-10T05:32:14.918650+0000","last_peered":"2024-05-10T05:32:14.918650+0000","last_clean":"2024-05-10T05:32:14.918650+0000","last_became_active":"2024-05-10T05:31:50.999336+0000","last_became_peered":"2024-05-10T05:31:50.999336+0000","last_unstale":"2024-05-10T05:32:14.918650+0000","last_undegraded":"2024-05-10T05:32:14.918650+0000","last_fullsized":"2024-05-10T05:32:14.918650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:43.755130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912608+0000","last_change":"2024-05-10T05:31:51.576938+0000","last_active":"2024-05-10T05:32:14.912608+0000","last_peered":"2024-05-10T05:32:14.912608+0000","last_clean":"2024-05-10T05:32:14.912608+0000","last_became_active":"2024-05-10T05:31:51.576517+0000","last_became_peered":"2024-05-10T05:31:51.576517+0000","last_unstale":"2024-05-10T05:32:14.912608+0000","last_undegraded":"2024-05-10T05:32:14.912608+0000","last_fullsized":"2024-05-10T05:32:14.912608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:52:38.709063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916132+0000","last_change":"2024-05-10T05:31:50.992925+0000","last_active":"2024-05-10T05:32:14.916132+0000","last_peered":"2024-05-10T05:32:14.916132+0000","last_clean":"2024-05-10T05:32:14.916132+0000","last_became_active":"2024-05-10T05:31:50.992574+0000","last_became_peered":"2024-05-10T05:31:50.992574+0000","last_unstale":"2024-05-10T05:32:14.916132+0000","last_undegraded":"2024-05-10T05:32:14.916132+0000","last_fullsized":"2024-05-10T05:32:14.916132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:14.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917334+0000","last_change":"2024-05-10T05:31:51.580145+0000","last_active":"2024-05-10T05:32:14.917334+0000","last_peered":"2024-05-10T05:32:14.917334+0000","last_clean":"2024-05-10T05:32:14.917334+0000","last_became_active":"2024-05-10T05:31:51.576701+0000","last_became_peered":"2024-05-10T05:31:51.576701+0000","last_unstale":"2024-05-10T05:32:14.917334+0000","last_undegraded":"2024-05-10T05:32:14.917334+0000","last_fullsized":"2024-05-10T05:32:14.917334+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:36:21.075571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915647+0000","last_change":"2024-05-10T05:31:50.987863+0000","last_active":"2024-05-10T05:32:14.915647+0000","last_peered":"2024-05-10T05:32:14.915647+0000","last_clean":"2024-05-10T05:32:14.915647+0000","last_became_active":"2024-05-10T05:31:50.987463+0000","last_became_peered":"2024-05-10T05:31:50.987463+0000","last_unstale":"2024-05-10T05:32:14.915647+0000","last_undegraded":"2024-05-10T05:32:14.915647+0000","last_fullsized":"2024-05-10T05:32:14.915647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:12:49.542288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:16.105675+0000","last_change":"2024-05-10T05:32:16.105284+0000","last_active":"2024-05-10T05:32:16.105675+0000","last_peered":"2024-05-10T05:32:16.105675+0000","last_clean":"2024-05-10T05:32:16.105675+0000","last_became_active":"2024-05-10T05:32:16.104915+0000","last_became_peered":"2024-05-10T05:32:16.104915+0000","last_unstale":"2024-05-10T05:32:16.105675+0000","last_undegraded":"2024-05-10T05:32:16.105675+0000","last_fullsized":"2024-05-10T05:32:16.105675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:25.628498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:37.557089+0000","last_change":"2024-05-10T05:31:50.985879+0000","last_active":"2024-05-10T05:32:37.557089+0000","last_peered":"2024-05-10T05:32:37.557089+0000","last_clean":"2024-05-10T05:32:37.557089+0000","last_became_active":"2024-05-10T05:31:50.985476+0000","last_became_peered":"2024-05-10T05:31:50.985476+0000","last_unstale":"2024-05-10T05:32:37.557089+0000","last_undegraded":"2024-05-10T05:32:37.557089+0000","last_fullsized":"2024-05-10T05:32:37.557089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:11:27.737972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915315+0000","last_change":"2024-05-10T05:31:51.570985+0000","last_active":"2024-05-10T05:32:14.915315+0000","last_peered":"2024-05-10T05:32:14.915315+0000","last_clean":"2024-05-10T05:32:14.915315+0000","last_became_active":"2024-05-10T05:31:51.570324+0000","last_became_peered":"2024-05-10T05:31:51.570324+0000","last_unstale":"2024-05-10T05:32:14.915315+0000","last_undegraded":"2024-05-10T05:32:14.915315+0000","last_fullsized":"2024-05-10T05:32:14.915315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:26.947574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.911748+0000","last_change":"2024-05-10T05:31:50.988483+0000","last_active":"2024-05-10T05:32:14.911748+0000","last_peered":"2024-05-10T05:32:14.911748+0000","last_clean":"2024-05-10T05:32:14.911748+0000","last_became_active":"2024-05-10T05:31:50.987897+0000","last_became_peered":"2024-05-10T05:31:50.987897+0000","last_unstale":"2024-05-10T05:32:14.911748+0000","last_undegraded":"2024-05-10T05:32:14.911748+0000","last_fullsized":"2024-05-10T05:32:14.911748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:30:23.474637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.922113+0000","last_change":"2024-05-10T05:32:15.922113+0000","last_active":"2024-05-10T05:32:15.922113+0000","last_peered":"2024-05-10T05:32:15.922113+0000","last_clean":"2024-05-10T05:32:15.922113+0000","last_became_active":"2024-05-10T05:32:15.921665+0000","last_became_peered":"2024-05-10T05:32:15.921665+0000","last_unstale":"2024-05-10T05:32:15.922113+0000","last_undegraded":"2024-05-10T05:32:15.922113+0000","last_fullsized":"2024-05-10T05:32:15.922113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:28.155216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6],"acting":[5,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.908880+0000","last_change":"2024-05-10T05:31:50.981877+0000","last_active":"2024-05-10T05:32:14.908880+0000","last_peered":"2024-05-10T05:32:14.908880+0000","last_clean":"2024-05-10T05:32:14.908880+0000","last_became_active":"2024-05-10T05:31:50.981527+0000","last_became_peered":"2024-05-10T05:31:50.981527+0000","last_unstale":"2024-05-10T05:32:14.908880+0000","last_undegraded":"2024-05-10T05:32:14.908880+0000","last_fullsized":"2024-05-10T05:32:14.908880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:40.735304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917899+0000","last_change":"2024-05-10T05:31:51.581027+0000","last_active":"2024-05-10T05:32:14.917899+0000","last_peered":"2024-05-10T05:32:14.917899+0000","last_clean":"2024-05-10T05:32:14.917899+0000","last_became_active":"2024-05-10T05:31:51.580620+0000","last_became_peered":"2024-05-10T05:31:51.580620+0000","last_unstale":"2024-05-10T05:32:14.917899+0000","last_undegraded":"2024-05-10T05:32:14.917899+0000","last_fullsized":"2024-05-10T05:32:14.917899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:30:15.378179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913340+0000","last_change":"2024-05-10T05:31:50.987633+0000","last_active":"2024-05-10T05:32:14.913340+0000","last_peered":"2024-05-10T05:32:14.913340+0000","last_clean":"2024-05-10T05:32:14.913340+0000","last_became_active":"2024-05-10T05:31:50.986750+0000","last_became_peered":"2024-05-10T05:31:50.986750+0000","last_unstale":"2024-05-10T05:32:14.913340+0000","last_undegraded":"2024-05-10T05:32:14.913340+0000","last_fullsized":"2024-05-10T05:32:14.913340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:17:47.705415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912961+0000","last_change":"2024-05-10T05:31:51.589484+0000","last_active":"2024-05-10T05:32:14.912961+0000","last_peered":"2024-05-10T05:32:14.912961+0000","last_clean":"2024-05-10T05:32:14.912961+0000","last_became_active":"2024-05-10T05:31:51.588992+0000","last_became_peered":"2024-05-10T05:31:51.588992+0000","last_unstale":"2024-05-10T05:32:14.912961+0000","last_undegraded":"2024-05-10T05:32:14.912961+0000","last_fullsized":"2024-05-10T05:32:14.912961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:32.360333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":94,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:35.553591+0000","last_change":"2024-05-10T05:31:50.988945+0000","last_active":"2024-05-10T05:32:35.553591+0000","last_peered":"2024-05-10T05:32:35.553591+0000","last_clean":"2024-05-10T05:32:35.553591+0000","last_became_active":"2024-05-10T05:31:50.988566+0000","last_became_peered":"2024-05-10T05:31:50.988566+0000","last_unstale":"2024-05-10T05:32:35.553591+0000","last_undegraded":"2024-05-10T05:32:35.553591+0000","last_fullsized":"2024-05-10T05:32:35.553591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:06:43.225346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917570+0000","last_change":"2024-05-10T05:31:51.577949+0000","last_active":"2024-05-10T05:32:14.917570+0000","last_peered":"2024-05-10T05:32:14.917570+0000","last_clean":"2024-05-10T05:32:14.917570+0000","last_became_active":"2024-05-10T05:31:51.577011+0000","last_became_peered":"2024-05-10T05:31:51.577011+0000","last_unstale":"2024-05-10T05:32:14.917570+0000","last_undegraded":"2024-05-10T05:32:14.917570+0000","last_fullsized":"2024-05-10T05:32:14.917570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:09.988700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920114+0000","last_change":"2024-05-10T05:31:50.984888+0000","last_active":"2024-05-10T05:32:14.920114+0000","last_peered":"2024-05-10T05:32:14.920114+0000","last_clean":"2024-05-10T05:32:14.920114+0000","last_became_active":"2024-05-10T05:31:50.984267+0000","last_became_peered":"2024-05-10T05:31:50.984267+0000","last_unstale":"2024-05-10T05:32:14.920114+0000","last_undegraded":"2024-05-10T05:32:14.920114+0000","last_fullsized":"2024-05-10T05:32:14.920114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:44.797469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912532+0000","last_change":"2024-05-10T05:31:51.576863+0000","last_active":"2024-05-10T05:32:14.912532+0000","last_peered":"2024-05-10T05:32:14.912532+0000","last_clean":"2024-05-10T05:32:14.912532+0000","last_became_active":"2024-05-10T05:31:51.576468+0000","last_became_peered":"2024-05-10T05:31:51.576468+0000","last_unstale":"2024-05-10T05:32:14.912532+0000","last_undegraded":"2024-05-10T05:32:14.912532+0000","last_fullsized":"2024-05-10T05:32:14.912532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:20.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913803+0000","last_change":"2024-05-10T05:31:51.001566+0000","last_active":"2024-05-10T05:32:14.913803+0000","last_peered":"2024-05-10T05:32:14.913803+0000","last_clean":"2024-05-10T05:32:14.913803+0000","last_became_active":"2024-05-10T05:31:51.001203+0000","last_became_peered":"2024-05-10T05:31:51.001203+0000","last_unstale":"2024-05-10T05:32:14.913803+0000","last_undegraded":"2024-05-10T05:32:14.913803+0000","last_fullsized":"2024-05-10T05:32:14.913803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:17:45.465039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917505+0000","last_change":"2024-05-10T05:31:51.579114+0000","last_active":"2024-05-10T05:32:14.917505+0000","last_peered":"2024-05-10T05:32:14.917505+0000","last_clean":"2024-05-10T05:32:14.917505+0000","last_became_active":"2024-05-10T05:31:51.577507+0000","last_became_peered":"2024-05-10T05:31:51.577507+0000","last_unstale":"2024-05-10T05:32:14.917505+0000","last_undegraded":"2024-05-10T05:32:14.917505+0000","last_fullsized":"2024-05-10T05:32:14.917505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:43.777983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":127,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:35.553530+0000","last_change":"2024-05-10T05:31:50.983558+0000","last_active":"2024-05-10T05:32:35.553530+0000","last_peered":"2024-05-10T05:32:35.553530+0000","last_clean":"2024-05-10T05:32:35.553530+0000","last_became_active":"2024-05-10T05:31:50.983087+0000","last_became_peered":"2024-05-10T05:31:50.983087+0000","last_unstale":"2024-05-10T05:32:35.553530+0000","last_undegraded":"2024-05-10T05:32:35.553530+0000","last_fullsized":"2024-05-10T05:32:35.553530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:08.753898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59684,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":100,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915844+0000","last_change":"2024-05-10T05:31:51.574024+0000","last_active":"2024-05-10T05:32:14.915844+0000","last_peered":"2024-05-10T05:32:14.915844+0000","last_clean":"2024-05-10T05:32:14.915844+0000","last_became_active":"2024-05-10T05:31:51.573703+0000","last_became_peered":"2024-05-10T05:31:51.573703+0000","last_unstale":"2024-05-10T05:32:14.915844+0000","last_undegraded":"2024-05-10T05:32:14.915844+0000","last_fullsized":"2024-05-10T05:32:14.915844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:55:23.435991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918608+0000","last_change":"2024-05-10T05:31:50.986795+0000","last_active":"2024-05-10T05:32:14.918608+0000","last_peered":"2024-05-10T05:32:14.918608+0000","last_clean":"2024-05-10T05:32:14.918608+0000","last_became_active":"2024-05-10T05:31:50.986428+0000","last_became_peered":"2024-05-10T05:31:50.986428+0000","last_unstale":"2024-05-10T05:32:14.918608+0000","last_undegraded":"2024-05-10T05:32:14.918608+0000","last_fullsized":"2024-05-10T05:32:14.918608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:11.039632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.906449+0000","last_change":"2024-05-10T05:31:51.585595+0000","last_active":"2024-05-10T05:32:14.906449+0000","last_peered":"2024-05-10T05:32:14.906449+0000","last_clean":"2024-05-10T05:32:14.906449+0000","last_became_active":"2024-05-10T05:31:51.585091+0000","last_became_peered":"2024-05-10T05:31:51.585091+0000","last_unstale":"2024-05-10T05:32:14.906449+0000","last_undegraded":"2024-05-10T05:32:14.906449+0000","last_fullsized":"2024-05-10T05:32:14.906449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:10:10.489430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913631+0000","last_change":"2024-05-10T05:31:50.984408+0000","last_active":"2024-05-10T05:32:14.913631+0000","last_peered":"2024-05-10T05:32:14.913631+0000","last_clean":"2024-05-10T05:32:14.913631+0000","last_became_active":"2024-05-10T05:31:50.983098+0000","last_became_peered":"2024-05-10T05:31:50.983098+0000","last_unstale":"2024-05-10T05:32:14.913631+0000","last_undegraded":"2024-05-10T05:32:14.913631+0000","last_fullsized":"2024-05-10T05:32:14.913631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:31.399649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919703+0000","last_change":"2024-05-10T05:31:51.576160+0000","last_active":"2024-05-10T05:32:14.919703+0000","last_peered":"2024-05-10T05:32:14.919703+0000","last_clean":"2024-05-10T05:32:14.919703+0000","last_became_active":"2024-05-10T05:31:51.575806+0000","last_became_peered":"2024-05-10T05:31:51.575806+0000","last_unstale":"2024-05-10T05:32:14.919703+0000","last_undegraded":"2024-05-10T05:32:14.919703+0000","last_fullsized":"2024-05-10T05:32:14.919703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:53:25.660988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:36.555572+0000","last_change":"2024-05-10T05:31:50.982752+0000","last_active":"2024-05-10T05:32:36.555572+0000","last_peered":"2024-05-10T05:32:36.555572+0000","last_clean":"2024-05-10T05:32:36.555572+0000","last_became_active":"2024-05-10T05:31:50.982299+0000","last_became_peered":"2024-05-10T05:31:50.982299+0000","last_unstale":"2024-05-10T05:32:36.555572+0000","last_undegraded":"2024-05-10T05:32:36.555572+0000","last_fullsized":"2024-05-10T05:32:36.555572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:52:40.206967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919413+0000","last_change":"2024-05-10T05:31:51.577081+0000","last_active":"2024-05-10T05:32:14.919413+0000","last_peered":"2024-05-10T05:32:14.919413+0000","last_clean":"2024-05-10T05:32:14.919413+0000","last_became_active":"2024-05-10T05:31:51.576729+0000","last_became_peered":"2024-05-10T05:31:51.576729+0000","last_unstale":"2024-05-10T05:32:14.919413+0000","last_undegraded":"2024-05-10T05:32:14.919413+0000","last_fullsized":"2024-05-10T05:32:14.919413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:14.072427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915219+0000","last_change":"2024-05-10T05:31:50.991868+0000","last_active":"2024-05-10T05:32:14.915219+0000","last_peered":"2024-05-10T05:32:14.915219+0000","last_clean":"2024-05-10T05:32:14.915219+0000","last_became_active":"2024-05-10T05:31:50.991471+0000","last_became_peered":"2024-05-10T05:31:50.991471+0000","last_unstale":"2024-05-10T05:32:14.915219+0000","last_undegraded":"2024-05-10T05:32:14.915219+0000","last_fullsized":"2024-05-10T05:32:14.915219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:22.346376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913564+0000","last_change":"2024-05-10T05:31:51.584726+0000","last_active":"2024-05-10T05:32:14.913564+0000","last_peered":"2024-05-10T05:32:14.913564+0000","last_clean":"2024-05-10T05:32:14.913564+0000","last_became_active":"2024-05-10T05:31:51.584327+0000","last_became_peered":"2024-05-10T05:31:51.584327+0000","last_unstale":"2024-05-10T05:32:14.913564+0000","last_undegraded":"2024-05-10T05:32:14.913564+0000","last_fullsized":"2024-05-10T05:32:14.913564+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:56:43.561717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915510+0000","last_change":"2024-05-10T05:31:50.989071+0000","last_active":"2024-05-10T05:32:14.915510+0000","last_peered":"2024-05-10T05:32:14.915510+0000","last_clean":"2024-05-10T05:32:14.915510+0000","last_became_active":"2024-05-10T05:31:50.988044+0000","last_became_peered":"2024-05-10T05:31:50.988044+0000","last_unstale":"2024-05-10T05:32:14.915510+0000","last_undegraded":"2024-05-10T05:32:14.915510+0000","last_fullsized":"2024-05-10T05:32:14.915510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:24.122564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905328+0000","last_change":"2024-05-10T05:31:51.576793+0000","last_active":"2024-05-10T05:32:14.905328+0000","last_peered":"2024-05-10T05:32:14.905328+0000","last_clean":"2024-05-10T05:32:14.905328+0000","last_became_active":"2024-05-10T05:31:51.576333+0000","last_became_peered":"2024-05-10T05:31:51.576333+0000","last_unstale":"2024-05-10T05:32:14.905328+0000","last_undegraded":"2024-05-10T05:32:14.905328+0000","last_fullsized":"2024-05-10T05:32:14.905328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:42.418736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913436+0000","last_change":"2024-05-10T05:31:50.997233+0000","last_active":"2024-05-10T05:32:14.913436+0000","last_peered":"2024-05-10T05:32:14.913436+0000","last_clean":"2024-05-10T05:32:14.913436+0000","last_became_active":"2024-05-10T05:31:50.996857+0000","last_became_peered":"2024-05-10T05:31:50.996857+0000","last_unstale":"2024-05-10T05:32:14.913436+0000","last_undegraded":"2024-05-10T05:32:14.913436+0000","last_fullsized":"2024-05-10T05:32:14.913436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:35.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.927682+0000","last_change":"2024-05-10T05:32:15.927682+0000","last_active":"2024-05-10T05:32:15.927682+0000","last_peered":"2024-05-10T05:32:15.927682+0000","last_clean":"2024-05-10T05:32:15.927682+0000","last_became_active":"2024-05-10T05:32:15.924197+0000","last_became_peered":"2024-05-10T05:32:15.924197+0000","last_unstale":"2024-05-10T05:32:15.927682+0000","last_undegraded":"2024-05-10T05:32:15.927682+0000","last_fullsized":"2024-05-10T05:32:15.927682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:14.747603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:34.552827+0000","last_change":"2024-05-10T05:31:50.991113+0000","last_active":"2024-05-10T05:32:34.552827+0000","last_peered":"2024-05-10T05:32:34.552827+0000","last_clean":"2024-05-10T05:32:34.552827+0000","last_became_active":"2024-05-10T05:31:50.990745+0000","last_became_peered":"2024-05-10T05:31:50.990745+0000","last_unstale":"2024-05-10T05:32:34.552827+0000","last_undegraded":"2024-05-10T05:32:34.552827+0000","last_fullsized":"2024-05-10T05:32:34.552827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:49:56.586231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912448+0000","last_change":"2024-05-10T05:31:51.582057+0000","last_active":"2024-05-10T05:32:14.912448+0000","last_peered":"2024-05-10T05:32:14.912448+0000","last_clean":"2024-05-10T05:32:14.912448+0000","last_became_active":"2024-05-10T05:31:51.581672+0000","last_became_peered":"2024-05-10T05:31:51.581672+0000","last_unstale":"2024-05-10T05:32:14.912448+0000","last_undegraded":"2024-05-10T05:32:14.912448+0000","last_fullsized":"2024-05-10T05:32:14.912448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:45:09.122256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914210+0000","last_change":"2024-05-10T05:31:51.585659+0000","last_active":"2024-05-10T05:32:14.914210+0000","last_peered":"2024-05-10T05:32:14.914210+0000","last_clean":"2024-05-10T05:32:14.914210+0000","last_became_active":"2024-05-10T05:31:51.584540+0000","last_became_peered":"2024-05-10T05:31:51.584540+0000","last_unstale":"2024-05-10T05:32:14.914210+0000","last_undegraded":"2024-05-10T05:32:14.914210+0000","last_fullsized":"2024-05-10T05:32:14.914210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:20.195369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:36.554882+0000","last_change":"2024-05-10T05:31:50.993547+0000","last_active":"2024-05-10T05:32:36.554882+0000","last_peered":"2024-05-10T05:32:36.554882+0000","last_clean":"2024-05-10T05:32:36.554882+0000","last_became_active":"2024-05-10T05:31:50.993190+0000","last_became_peered":"2024-05-10T05:31:50.993190+0000","last_unstale":"2024-05-10T05:32:36.554882+0000","last_undegraded":"2024-05-10T05:32:36.554882+0000","last_fullsized":"2024-05-10T05:32:36.554882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:06.159572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918191+0000","last_change":"2024-05-10T05:31:51.587678+0000","last_active":"2024-05-10T05:32:14.918191+0000","last_peered":"2024-05-10T05:32:14.918191+0000","last_clean":"2024-05-10T05:32:14.918191+0000","last_became_active":"2024-05-10T05:31:51.586999+0000","last_became_peered":"2024-05-10T05:31:51.586999+0000","last_unstale":"2024-05-10T05:32:14.918191+0000","last_undegraded":"2024-05-10T05:32:14.918191+0000","last_fullsized":"2024-05-10T05:32:14.918191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:18:00.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914737+0000","last_change":"2024-05-10T05:31:50.979131+0000","last_active":"2024-05-10T05:32:14.914737+0000","last_peered":"2024-05-10T05:32:14.914737+0000","last_clean":"2024-05-10T05:32:14.914737+0000","last_became_active":"2024-05-10T05:31:50.978715+0000","last_became_peered":"2024-05-10T05:31:50.978715+0000","last_unstale":"2024-05-10T05:32:14.914737+0000","last_undegraded":"2024-05-10T05:32:14.914737+0000","last_fullsized":"2024-05-10T05:32:14.914737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:29:11.953963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919607+0000","last_change":"2024-05-10T05:30:59.356688+0000","last_active":"2024-05-10T05:32:14.919607+0000","last_peered":"2024-05-10T05:32:14.919607+0000","last_clean":"2024-05-10T05:32:14.919607+0000","last_became_active":"2024-05-10T05:30:59.356313+0000","last_became_peered":"2024-05-10T05:30:59.356313+0000","last_unstale":"2024-05-10T05:32:14.919607+0000","last_undegraded":"2024-05-10T05:32:14.919607+0000","last_fullsized":"2024-05-10T05:32:14.919607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_clean_scrub_stamp":"2024-05-10T05:28:15.503591+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:43.011495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.908778+0000","last_change":"2024-05-10T05:31:51.580259+0000","last_active":"2024-05-10T05:32:14.908778+0000","last_peered":"2024-05-10T05:32:14.908778+0000","last_clean":"2024-05-10T05:32:14.908778+0000","last_became_active":"2024-05-10T05:31:51.578650+0000","last_became_peered":"2024-05-10T05:31:51.578650+0000","last_unstale":"2024-05-10T05:32:14.908778+0000","last_undegraded":"2024-05-10T05:32:14.908778+0000","last_fullsized":"2024-05-10T05:32:14.908778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:48.630183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919744+0000","last_change":"2024-05-10T05:31:50.987708+0000","last_active":"2024-05-10T05:32:14.919744+0000","last_peered":"2024-05-10T05:32:14.919744+0000","last_clean":"2024-05-10T05:32:14.919744+0000","last_became_active":"2024-05-10T05:31:50.987347+0000","last_became_peered":"2024-05-10T05:31:50.987347+0000","last_unstale":"2024-05-10T05:32:14.919744+0000","last_undegraded":"2024-05-10T05:32:14.919744+0000","last_fullsized":"2024-05-10T05:32:14.919744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:16:29.388690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915952+0000","last_change":"2024-05-10T05:31:50.999338+0000","last_active":"2024-05-10T05:32:14.915952+0000","last_peered":"2024-05-10T05:32:14.915952+0000","last_clean":"2024-05-10T05:32:14.915952+0000","last_became_active":"2024-05-10T05:31:50.999026+0000","last_became_peered":"2024-05-10T05:31:50.999026+0000","last_unstale":"2024-05-10T05:32:14.915952+0000","last_undegraded":"2024-05-10T05:32:14.915952+0000","last_fullsized":"2024-05-10T05:32:14.915952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:31.443409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.906875+0000","last_change":"2024-05-10T05:31:51.574201+0000","last_active":"2024-05-10T05:32:14.906875+0000","last_peered":"2024-05-10T05:32:14.906875+0000","last_clean":"2024-05-10T05:32:14.906875+0000","last_became_active":"2024-05-10T05:31:51.571180+0000","last_became_peered":"2024-05-10T05:31:51.571180+0000","last_unstale":"2024-05-10T05:32:14.906875+0000","last_undegraded":"2024-05-10T05:32:14.906875+0000","last_fullsized":"2024-05-10T05:32:14.906875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:33:16.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'10","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913628+0000","last_change":"2024-05-10T05:31:50.985164+0000","last_active":"2024-05-10T05:32:14.913628+0000","last_peered":"2024-05-10T05:32:14.913628+0000","last_clean":"2024-05-10T05:32:14.913628+0000","last_became_active":"2024-05-10T05:31:50.984760+0000","last_became_peered":"2024-05-10T05:31:50.984760+0000","last_unstale":"2024-05-10T05:32:14.913628+0000","last_undegraded":"2024-05-10T05:32:14.913628+0000","last_fullsized":"2024-05-10T05:32:14.913628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:19.827407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918649+0000","last_change":"2024-05-10T05:31:51.577445+0000","last_active":"2024-05-10T05:32:14.918649+0000","last_peered":"2024-05-10T05:32:14.918649+0000","last_clean":"2024-05-10T05:32:14.918649+0000","last_became_active":"2024-05-10T05:31:51.576671+0000","last_became_peered":"2024-05-10T05:31:51.576671+0000","last_unstale":"2024-05-10T05:32:14.918649+0000","last_undegraded":"2024-05-10T05:32:14.918649+0000","last_fullsized":"2024-05-10T05:32:14.918649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:38.774538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914419+0000","last_change":"2024-05-10T05:31:50.981495+0000","last_active":"2024-05-10T05:32:14.914419+0000","last_peered":"2024-05-10T05:32:14.914419+0000","last_clean":"2024-05-10T05:32:14.914419+0000","last_became_active":"2024-05-10T05:31:50.981127+0000","last_became_peered":"2024-05-10T05:31:50.981127+0000","last_unstale":"2024-05-10T05:32:14.914419+0000","last_undegraded":"2024-05-10T05:32:14.914419+0000","last_fullsized":"2024-05-10T05:32:14.914419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:22:32.257014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912554+0000","last_change":"2024-05-10T05:31:51.585926+0000","last_active":"2024-05-10T05:32:14.912554+0000","last_peered":"2024-05-10T05:32:14.912554+0000","last_clean":"2024-05-10T05:32:14.912554+0000","last_became_active":"2024-05-10T05:31:51.585576+0000","last_became_peered":"2024-05-10T05:31:51.585576+0000","last_unstale":"2024-05-10T05:32:14.912554+0000","last_undegraded":"2024-05-10T05:32:14.912554+0000","last_fullsized":"2024-05-10T05:32:14.912554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:22.344180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.926062+0000","last_change":"2024-05-10T05:32:15.926062+0000","last_active":"2024-05-10T05:32:15.926062+0000","last_peered":"2024-05-10T05:32:15.926062+0000","last_clean":"2024-05-10T05:32:15.926062+0000","last_became_active":"2024-05-10T05:32:15.923026+0000","last_became_peered":"2024-05-10T05:32:15.923026+0000","last_unstale":"2024-05-10T05:32:15.926062+0000","last_undegraded":"2024-05-10T05:32:15.926062+0000","last_fullsized":"2024-05-10T05:32:15.926062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:34.188115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915162+0000","last_change":"2024-05-10T05:31:51.573279+0000","last_active":"2024-05-10T05:32:14.915162+0000","last_peered":"2024-05-10T05:32:14.915162+0000","last_clean":"2024-05-10T05:32:14.915162+0000","last_became_active":"2024-05-10T05:31:51.572916+0000","last_became_peered":"2024-05-10T05:31:51.572916+0000","last_unstale":"2024-05-10T05:32:14.915162+0000","last_undegraded":"2024-05-10T05:32:14.915162+0000","last_fullsized":"2024-05-10T05:32:14.915162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:32:39.802463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918101+0000","last_change":"2024-05-10T05:31:50.983771+0000","last_active":"2024-05-10T05:32:14.918101+0000","last_peered":"2024-05-10T05:32:14.918101+0000","last_clean":"2024-05-10T05:32:14.918101+0000","last_became_active":"2024-05-10T05:31:50.981849+0000","last_became_peered":"2024-05-10T05:31:50.981849+0000","last_unstale":"2024-05-10T05:32:14.918101+0000","last_undegraded":"2024-05-10T05:32:14.918101+0000","last_fullsized":"2024-05-10T05:32:14.918101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:15.990869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905211+0000","last_change":"2024-05-10T05:31:51.568005+0000","last_active":"2024-05-10T05:32:14.905211+0000","last_peered":"2024-05-10T05:32:14.905211+0000","last_clean":"2024-05-10T05:32:14.905211+0000","last_became_active":"2024-05-10T05:31:51.567347+0000","last_became_peered":"2024-05-10T05:31:51.567347+0000","last_unstale":"2024-05-10T05:32:14.905211+0000","last_undegraded":"2024-05-10T05:32:14.905211+0000","last_fullsized":"2024-05-10T05:32:14.905211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:29.075816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.926267+0000","last_change":"2024-05-10T05:32:15.925869+0000","last_active":"2024-05-10T05:32:15.926267+0000","last_peered":"2024-05-10T05:32:15.926267+0000","last_clean":"2024-05-10T05:32:15.926267+0000","last_became_active":"2024-05-10T05:32:15.920591+0000","last_became_peered":"2024-05-10T05:32:15.920591+0000","last_unstale":"2024-05-10T05:32:15.926267+0000","last_undegraded":"2024-05-10T05:32:15.926267+0000","last_fullsized":"2024-05-10T05:32:15.926267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:13:20.398637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918979+0000","last_change":"2024-05-10T05:31:51.588255+0000","last_active":"2024-05-10T05:32:14.918979+0000","last_peered":"2024-05-10T05:32:14.918979+0000","last_clean":"2024-05-10T05:32:14.918979+0000","last_became_active":"2024-05-10T05:31:51.587912+0000","last_became_peered":"2024-05-10T05:31:51.587912+0000","last_unstale":"2024-05-10T05:32:14.918979+0000","last_undegraded":"2024-05-10T05:32:14.918979+0000","last_fullsized":"2024-05-10T05:32:14.918979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:18.515739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916229+0000","last_change":"2024-05-10T05:31:50.992392+0000","last_active":"2024-05-10T05:32:14.916229+0000","last_peered":"2024-05-10T05:32:14.916229+0000","last_clean":"2024-05-10T05:32:14.916229+0000","last_became_active":"2024-05-10T05:31:50.992119+0000","last_became_peered":"2024-05-10T05:31:50.992119+0000","last_unstale":"2024-05-10T05:32:14.916229+0000","last_undegraded":"2024-05-10T05:32:14.916229+0000","last_fullsized":"2024-05-10T05:32:14.916229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:11.515013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919747+0000","last_change":"2024-05-10T05:31:51.567780+0000","last_active":"2024-05-10T05:32:14.919747+0000","last_peered":"2024-05-10T05:32:14.919747+0000","last_clean":"2024-05-10T05:32:14.919747+0000","last_became_active":"2024-05-10T05:31:51.567455+0000","last_became_peered":"2024-05-10T05:31:51.567455+0000","last_unstale":"2024-05-10T05:32:14.919747+0000","last_undegraded":"2024-05-10T05:32:14.919747+0000","last_fullsized":"2024-05-10T05:32:14.919747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:47.108131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:38.558428+0000","last_change":"2024-05-10T05:31:50.997374+0000","last_active":"2024-05-10T05:32:38.558428+0000","last_peered":"2024-05-10T05:32:38.558428+0000","last_clean":"2024-05-10T05:32:38.558428+0000","last_became_active":"2024-05-10T05:31:50.997069+0000","last_became_peered":"2024-05-10T05:31:50.997069+0000","last_unstale":"2024-05-10T05:32:38.558428+0000","last_undegraded":"2024-05-10T05:32:38.558428+0000","last_fullsized":"2024-05-10T05:32:38.558428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:25:49.140006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920191+0000","last_change":"2024-05-10T05:31:51.576345+0000","last_active":"2024-05-10T05:32:14.920191+0000","last_peered":"2024-05-10T05:32:14.920191+0000","last_clean":"2024-05-10T05:32:14.920191+0000","last_became_active":"2024-05-10T05:31:51.575704+0000","last_became_peered":"2024-05-10T05:31:51.575704+0000","last_unstale":"2024-05-10T05:32:14.920191+0000","last_undegraded":"2024-05-10T05:32:14.920191+0000","last_fullsized":"2024-05-10T05:32:14.920191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:15.146591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913524+0000","last_change":"2024-05-10T05:31:50.984108+0000","last_active":"2024-05-10T05:32:14.913524+0000","last_peered":"2024-05-10T05:32:14.913524+0000","last_clean":"2024-05-10T05:32:14.913524+0000","last_became_active":"2024-05-10T05:31:50.983728+0000","last_became_peered":"2024-05-10T05:31:50.983728+0000","last_unstale":"2024-05-10T05:32:14.913524+0000","last_undegraded":"2024-05-10T05:32:14.913524+0000","last_fullsized":"2024-05-10T05:32:14.913524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:23:20.096286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919865+0000","last_change":"2024-05-10T05:31:51.572075+0000","last_active":"2024-05-10T05:32:14.919865+0000","last_peered":"2024-05-10T05:32:14.919865+0000","last_clean":"2024-05-10T05:32:14.919865+0000","last_became_active":"2024-05-10T05:31:51.571711+0000","last_became_peered":"2024-05-10T05:31:51.571711+0000","last_unstale":"2024-05-10T05:32:14.919865+0000","last_undegraded":"2024-05-10T05:32:14.919865+0000","last_fullsized":"2024-05-10T05:32:14.919865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:38:27.813001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.907701+0000","last_change":"2024-05-10T05:31:50.983985+0000","last_active":"2024-05-10T05:32:14.907701+0000","last_peered":"2024-05-10T05:32:14.907701+0000","last_clean":"2024-05-10T05:32:14.907701+0000","last_became_active":"2024-05-10T05:31:50.982931+0000","last_became_peered":"2024-05-10T05:31:50.982931+0000","last_unstale":"2024-05-10T05:32:14.907701+0000","last_undegraded":"2024-05-10T05:32:14.907701+0000","last_fullsized":"2024-05-10T05:32:14.907701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:35:17.158696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920269+0000","last_change":"2024-05-10T05:31:51.587250+0000","last_active":"2024-05-10T05:32:14.920269+0000","last_peered":"2024-05-10T05:32:14.920269+0000","last_clean":"2024-05-10T05:32:14.920269+0000","last_became_active":"2024-05-10T05:31:51.586925+0000","last_became_peered":"2024-05-10T05:31:51.586925+0000","last_unstale":"2024-05-10T05:32:14.920269+0000","last_undegraded":"2024-05-10T05:32:14.920269+0000","last_fullsized":"2024-05-10T05:32:14.920269+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:40.585803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905347+0000","last_change":"2024-05-10T05:31:51.567710+0000","last_active":"2024-05-10T05:32:14.905347+0000","last_peered":"2024-05-10T05:32:14.905347+0000","last_clean":"2024-05-10T05:32:14.905347+0000","last_became_active":"2024-05-10T05:31:51.566860+0000","last_became_peered":"2024-05-10T05:31:51.566860+0000","last_unstale":"2024-05-10T05:32:14.905347+0000","last_undegraded":"2024-05-10T05:32:14.905347+0000","last_fullsized":"2024-05-10T05:32:14.905347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:02:09.253345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918980+0000","last_change":"2024-05-10T05:31:51.577458+0000","last_active":"2024-05-10T05:32:14.918980+0000","last_peered":"2024-05-10T05:32:14.918980+0000","last_clean":"2024-05-10T05:32:14.918980+0000","last_became_active":"2024-05-10T05:31:51.577123+0000","last_became_peered":"2024-05-10T05:31:51.577123+0000","last_unstale":"2024-05-10T05:32:14.918980+0000","last_undegraded":"2024-05-10T05:32:14.918980+0000","last_fullsized":"2024-05-10T05:32:14.918980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:51.791090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919038+0000","last_change":"2024-05-10T05:31:51.589361+0000","last_active":"2024-05-10T05:32:14.919038+0000","last_peered":"2024-05-10T05:32:14.919038+0000","last_clean":"2024-05-10T05:32:14.919038+0000","last_became_active":"2024-05-10T05:31:51.589049+0000","last_became_peered":"2024-05-10T05:31:51.589049+0000","last_unstale":"2024-05-10T05:32:14.919038+0000","last_undegraded":"2024-05-10T05:32:14.919038+0000","last_fullsized":"2024-05-10T05:32:14.919038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:13.380505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.919725+0000","last_change":"2024-05-10T05:32:15.919725+0000","last_active":"2024-05-10T05:32:15.919725+0000","last_peered":"2024-05-10T05:32:15.919725+0000","last_clean":"2024-05-10T05:32:15.919725+0000","last_became_active":"2024-05-10T05:32:15.919342+0000","last_became_peered":"2024-05-10T05:32:15.919342+0000","last_unstale":"2024-05-10T05:32:15.919725+0000","last_undegraded":"2024-05-10T05:32:15.919725+0000","last_fullsized":"2024-05-10T05:32:15.919725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:33.785843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.906792+0000","last_change":"2024-05-10T05:31:51.579791+0000","last_active":"2024-05-10T05:32:14.906792+0000","last_peered":"2024-05-10T05:32:14.906792+0000","last_clean":"2024-05-10T05:32:14.906792+0000","last_became_active":"2024-05-10T05:31:51.577635+0000","last_became_peered":"2024-05-10T05:31:51.577635+0000","last_unstale":"2024-05-10T05:32:14.906792+0000","last_undegraded":"2024-05-10T05:32:14.906792+0000","last_fullsized":"2024-05-10T05:32:14.906792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:54.077805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905393+0000","last_change":"2024-05-10T05:31:51.584601+0000","last_active":"2024-05-10T05:32:14.905393+0000","last_peered":"2024-05-10T05:32:14.905393+0000","last_clean":"2024-05-10T05:32:14.905393+0000","last_became_active":"2024-05-10T05:31:51.584172+0000","last_became_peered":"2024-05-10T05:31:51.584172+0000","last_unstale":"2024-05-10T05:32:14.905393+0000","last_undegraded":"2024-05-10T05:32:14.905393+0000","last_fullsized":"2024-05-10T05:32:14.905393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:49:44.987379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919277+0000","last_change":"2024-05-10T05:31:51.573784+0000","last_active":"2024-05-10T05:32:14.919277+0000","last_peered":"2024-05-10T05:32:14.919277+0000","last_clean":"2024-05-10T05:32:14.919277+0000","last_became_active":"2024-05-10T05:31:51.573279+0000","last_became_peered":"2024-05-10T05:31:51.573279+0000","last_unstale":"2024-05-10T05:32:14.919277+0000","last_undegraded":"2024-05-10T05:32:14.919277+0000","last_fullsized":"2024-05-10T05:32:14.919277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:08.375964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918571+0000","last_change":"2024-05-10T05:31:51.585577+0000","last_active":"2024-05-10T05:32:14.918571+0000","last_peered":"2024-05-10T05:32:14.918571+0000","last_clean":"2024-05-10T05:32:14.918571+0000","last_became_active":"2024-05-10T05:31:51.585222+0000","last_became_peered":"2024-05-10T05:31:51.585222+0000","last_unstale":"2024-05-10T05:32:14.918571+0000","last_undegraded":"2024-05-10T05:32:14.918571+0000","last_fullsized":"2024-05-10T05:32:14.918571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:47:27.069513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913429+0000","last_change":"2024-05-10T05:31:51.571405+0000","last_active":"2024-05-10T05:32:14.913429+0000","last_peered":"2024-05-10T05:32:14.913429+0000","last_clean":"2024-05-10T05:32:14.913429+0000","last_became_active":"2024-05-10T05:31:51.571125+0000","last_became_peered":"2024-05-10T05:31:51.571125+0000","last_unstale":"2024-05-10T05:32:14.913429+0000","last_undegraded":"2024-05-10T05:32:14.913429+0000","last_fullsized":"2024-05-10T05:32:14.913429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:49:29.069121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918005+0000","last_change":"2024-05-10T05:31:51.578883+0000","last_active":"2024-05-10T05:32:14.918005+0000","last_peered":"2024-05-10T05:32:14.918005+0000","last_clean":"2024-05-10T05:32:14.918005+0000","last_became_active":"2024-05-10T05:31:51.578582+0000","last_became_peered":"2024-05-10T05:31:51.578582+0000","last_unstale":"2024-05-10T05:32:14.918005+0000","last_undegraded":"2024-05-10T05:32:14.918005+0000","last_fullsized":"2024-05-10T05:32:14.918005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:16.402282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917342+0000","last_change":"2024-05-10T05:31:51.589269+0000","last_active":"2024-05-10T05:32:14.917342+0000","last_peered":"2024-05-10T05:32:14.917342+0000","last_clean":"2024-05-10T05:32:14.917342+0000","last_became_active":"2024-05-10T05:31:51.588875+0000","last_became_peered":"2024-05-10T05:31:51.588875+0000","last_unstale":"2024-05-10T05:32:14.917342+0000","last_undegraded":"2024-05-10T05:32:14.917342+0000","last_fullsized":"2024-05-10T05:32:14.917342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:12.536555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.907976+0000","last_change":"2024-05-10T05:31:51.567789+0000","last_active":"2024-05-10T05:32:14.907976+0000","last_peered":"2024-05-10T05:32:14.907976+0000","last_clean":"2024-05-10T05:32:14.907976+0000","last_became_active":"2024-05-10T05:31:51.566625+0000","last_became_peered":"2024-05-10T05:31:51.566625+0000","last_unstale":"2024-05-10T05:32:14.907976+0000","last_undegraded":"2024-05-10T05:32:14.907976+0000","last_fullsized":"2024-05-10T05:32:14.907976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:43.042557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905276+0000","last_change":"2024-05-10T05:31:51.585838+0000","last_active":"2024-05-10T05:32:14.905276+0000","last_peered":"2024-05-10T05:32:14.905276+0000","last_clean":"2024-05-10T05:32:14.905276+0000","last_became_active":"2024-05-10T05:31:51.585473+0000","last_became_peered":"2024-05-10T05:31:51.585473+0000","last_unstale":"2024-05-10T05:32:14.905276+0000","last_undegraded":"2024-05-10T05:32:14.905276+0000","last_fullsized":"2024-05-10T05:32:14.905276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:11:11.282076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247530,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66700,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":430,"num_read_kb":286,"num_write":234,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":200100,"data_compressed":5520,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547221,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":429949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67700000000000005}]},{"osd":2,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.50600000000000001}]},{"osd":5,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.76900000000000002}]},{"osd":6,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.627}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":9,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.83199999999999996}]},{"osd":10,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.69199999999999995}]}]},{"osd":10,"up_from":68,"seq":292057776151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1081055,"data_compressed":37906,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.80300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":8,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.77800000000000002}]},{"osd":11,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.94499999999999995}]}]},{"osd":9,"up_from":63,"seq":270582939675,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":921600,"data_stored":430044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.85399999999999998}]},{"osd":3,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.39100000000000001}]},{"osd":4,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.82899999999999996}]},{"osd":10,"last update":"Fri May 10 05:31:58 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.78800000000000003}]}]},{"osd":8,"up_from":56,"seq":240518168607,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":430305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72399999999999998,"5min":0.995,"15min":0.995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.38900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72899999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.749},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61699999999999999,"15min":0.624},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73499999999999999}]},{"osd":2,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.502,"15min":0.497},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.753,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77400000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.622,"15min":0.622},"min":{"1min":0.47799999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.746,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91500000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.629}]},{"osd":5,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85199999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.375,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84899999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.44400000000000001}]},{"osd":6,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.496,"5min":0.495,"15min":0.495},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76900000000000002,"15min":0.77100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94199999999999995,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.749,"15min":0.751},"min":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99099999999999999,"5min":1.101,"15min":1.101},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.76600000000000001}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80100000000000005}]},{"osd":10,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.83299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397539,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":966656,"data_stored":1023074,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.435,"15min":0.432},"min":{"1min":0.32100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.45100000000000001,"15min":0.443},"min":{"1min":0.38100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64400000000000002}]},{"osd":3,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.86099999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.48699999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90600000000000003,"5min":1.302,"15min":1.302},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.35999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70599999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.497}]},{"osd":5,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.81000000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66900000000000004,"15min":0.66300000000000003},"min":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.754}]},{"osd":6,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.079,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85299999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri May 10 05:32:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.90400000000000003,"5min":1.002,"15min":1.002},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66700000000000004}]},{"osd":9,"last update":"Fri May 10 05:31:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.83299999999999996}]},{"osd":10,"last update":"Fri May 10 05:31:56 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":6,"up_from":44,"seq":188978561064,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":901120,"data_stored":429596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.76700000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.54200000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.85699999999999998}]},{"osd":2,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.65500000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.59499999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.871}]},{"osd":5,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.84499999999999997,"15min":0.83399999999999996},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.95999999999999996}]},{"osd":7,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83799999999999997,"15min":0.83199999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81899999999999995,"15min":0.81100000000000005},"min":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.21,"5min":1.252,"15min":1.252},"last":0.65600000000000003}]},{"osd":8,"last update":"Fri May 10 05:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.93799999999999994,"15min":0.95299999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1220000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.95099999999999996,"15min":0.96699999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.76800000000000002}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.0009999999999999}]},{"osd":10,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.02},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.98899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":958464,"data_stored":490193,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.68400000000000005,"15min":0.80800000000000005},"min":{"1min":0.40799999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91700000000000004,"5min":6.3810000000000002,"15min":6.3810000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81699999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56599999999999995,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.504},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri May 10 05:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.42999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.441,"15min":0.42899999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46899999999999997,"15min":0.443},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.56499999999999995,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.624,"15min":0.60399999999999998},"min":{"1min":0.44900000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.85499999999999998}]},{"osd":6,"last update":"Fri May 10 05:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.52400000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86499999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.753}]},{"osd":7,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.69499999999999995,"15min":0.68200000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.67400000000000004,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.874}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72999999999999998}]},{"osd":10,"last update":"Fri May 10 05:31:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.90200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":921600,"data_stored":430935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 05:32:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1890000000000001,"5min":1.256,"15min":1.256},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.64900000000000002,"15min":0.61099999999999999},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri May 10 05:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.68200000000000005,"15min":0.627},"min":{"1min":0.50800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.503,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.6920000000000002,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.625}]},{"osd":3,"last update":"Fri May 10 05:30:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.35199999999999998}]},{"osd":4,"last update":"Fri May 10 05:32:05 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.71199999999999997,"15min":0.66800000000000004},"min":{"1min":0.59599999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.69699999999999995,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri May 10 05:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.71999999999999997,"15min":0.66300000000000003},"min":{"1min":0.47499999999999998,"5min":0.44,"15min":0.44},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.73499999999999999,"15min":0.69499999999999995},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri May 10 05:31:39 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.75600000000000001,"15min":0.74199999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.347,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.746,"15min":0.73299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.077,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.40400000000000003}]},{"osd":7,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90700000000000003}]},{"osd":8,"last update":"Fri May 10 05:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.88700000000000001,"15min":0.875},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.087,"5min":0.88500000000000001,"15min":0.85099999999999998},"min":{"1min":0.68600000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":4.6559999999999997,"5min":4.6559999999999997,"15min":4.6559999999999997},"last":0.51000000000000001}]},{"osd":9,"last update":"Fri May 10 05:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.56799999999999995}]},{"osd":10,"last update":"Fri May 10 05:31:58 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.84399999999999997}]},{"osd":11,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.81000000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411384,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":978944,"data_stored":490758,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89300000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.55800000000000005,"15min":0.50700000000000001},"min":{"1min":0.52000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.85999999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56200000000000006,"15min":0.52400000000000002},"min":{"1min":0.46700000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79000000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.41999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.81100000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.439}]},{"osd":4,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.629}]},{"osd":5,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.46300000000000002,"15min":0.435},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.49199999999999999,"15min":0.46600000000000003},"min":{"1min":0.307,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.745}]},{"osd":8,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.91900000000000004}]},{"osd":10,"last update":"Fri May 10 05:31:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":429352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.42499999999999999,"15min":0.40300000000000002},"min":{"1min":0.38300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.436,"15min":0.40799999999999997},"min":{"1min":0.36499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":1.018}]},{"osd":1,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60599999999999998,"15min":0.59399999999999997},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.53300000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65700000000000003,"15min":0.63100000000000001},"min":{"1min":0.47399999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0589999999999999,"5min":1.206,"15min":1.206},"last":0.75}]},{"osd":4,"last update":"Fri May 10 05:32:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.65300000000000002,"15min":0.63200000000000001},"min":{"1min":0.54700000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.57599999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94899999999999995,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.72599999999999998,"15min":0.70399999999999996},"min":{"1min":0.48299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.02,"5min":1.109,"15min":1.109},"last":0.85099999999999998}]},{"osd":6,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.74399999999999999,"15min":0.72999999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.59099999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.095,"5min":1.119,"15min":1.119},"last":1.133}]},{"osd":7,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.78200000000000003,"15min":0.77000000000000002},"min":{"1min":0.60599999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.1359999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.84199999999999997,"15min":0.82799999999999996},"min":{"1min":0.72199999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2050000000000001,"5min":1.216,"15min":1.216},"last":0.92600000000000005}]},{"osd":8,"last update":"Fri May 10 05:31:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.753}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.71399999999999997}]},{"osd":10,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.319},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.95799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":901120,"data_stored":429850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri May 10 05:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.313,"5min":0.255,"15min":0.255},"max":{"1min":0.57899999999999996,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.39700000000000002},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66000000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri May 10 05:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59899999999999998,"15min":0.57599999999999996},"min":{"1min":0.46700000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83099999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri May 10 05:32:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":0.80800000000000005,"15min":0.64900000000000002},"min":{"1min":0.58999999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":13.608000000000001,"5min":13.608000000000001,"15min":13.608000000000001},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.621,"15min":0.61299999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82699999999999996,"5min":0.876,"15min":0.876},"last":0.53800000000000003}]},{"osd":5,"last update":"Fri May 10 05:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.63800000000000001,"15min":0.61399999999999999},"min":{"1min":0.51800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.622,"15min":0.60499999999999998},"min":{"1min":0.44900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88300000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri May 10 05:31:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.497,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.437}]},{"osd":7,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.51800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1,"5min":1.081,"15min":1.081},"last":0.56599999999999995}]},{"osd":8,"last update":"Fri May 10 05:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.58199999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.95499999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0049999999999999,"5min":1.012,"15min":1.012},"last":0.70299999999999996}]},{"osd":9,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59099999999999997}]},{"osd":10,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.79300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":5,"up_from":37,"seq":158913789995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":978944,"data_stored":1022455,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.55700000000000005},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.62,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":4.524,"15min":4.524},"last":0.61199999999999999}]},{"osd":1,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.438,"15min":0.434},"min":{"1min":0.316,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.441,"15min":0.437},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79200000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59899999999999998,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66000000000000003,"15min":0.65900000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.86799999999999999}]},{"osd":8,"last update":"Fri May 10 05:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.79000000000000004}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.90800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61388,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61953,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61765,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T05:32:41.596 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T05:32:41.959 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:41 smithi131 ceph-mon[37264]: pgmap v265: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:32:42.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:41 smithi157 ceph-mon[78427]: pgmap v265: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:32:42.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:41 smithi142 ceph-mon[77667]: pgmap v265: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:32:43.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:42 smithi131 ceph-mon[37264]: from='client.24901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:43.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.230+0000 7f4f27600700 1 -- 172.21.15.131:0/1802749623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f20106870 msgr2=0x7f4f20106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:43.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.230+0000 7f4f27600700 1 --2- 172.21.15.131:0/1802749623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f20106870 0x7f4f20106c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009990 tx=0x7f4f0c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.230+0000 7f4f27600700 1 -- 172.21.15.131:0/1802749623 shutdown_connections 2024-05-10T05:32:43.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.230+0000 7f4f27600700 1 --2- 172.21.15.131:0/1802749623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f201071c0 0x7f4f2010eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.230+0000 7f4f27600700 1 --2- 172.21.15.131:0/1802749623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f20101f40 0x7f4f20102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.230+0000 7f4f27600700 1 --2- 172.21.15.131:0/1802749623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f20106870 0x7f4f20106c80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.230+0000 7f4f27600700 1 -- 172.21.15.131:0/1802749623 >> 172.21.15.131:0/1802749623 conn(0x7f4f200fd7f0 msgr2=0x7f4f200ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:43.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.230+0000 7f4f27600700 1 -- 172.21.15.131:0/1802749623 shutdown_connections 2024-05-10T05:32:43.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.230+0000 7f4f27600700 1 -- 172.21.15.131:0/1802749623 wait complete. 2024-05-10T05:32:43.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.231+0000 7f4f27600700 1 Processor -- start 2024-05-10T05:32:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f27600700 1 -- start start 2024-05-10T05:32:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f27600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f20101f40 0x7f4f201a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f27600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f20106870 0x7f4f201a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f27600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f201071c0 0x7f4f2019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f27600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2019aba0 con 0x7f4f201071c0 2024-05-10T05:32:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f27600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2019ace0 con 0x7f4f20106870 2024-05-10T05:32:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f27600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2019aea0 con 0x7f4f20101f40 2024-05-10T05:32:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f24c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f20101f40 0x7f4f201a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f25600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f201071c0 0x7f4f2019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:43.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f24c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f20101f40 0x7f4f201a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:50850/0 (socket says 172.21.15.131:50850) 2024-05-10T05:32:43.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f24c00700 1 -- 172.21.15.131:0/981558341 learned_addr learned my addr 172.21.15.131:0/981558341 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:43.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.232+0000 7f4f1fe00700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f20106870 0x7f4f201a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:43.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.233+0000 7f4f25600700 1 -- 172.21.15.131:0/981558341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f20101f40 msgr2=0x7f4f201a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:43.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.233+0000 7f4f25600700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f20101f40 0x7f4f201a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.233+0000 7f4f25600700 1 -- 172.21.15.131:0/981558341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f20106870 msgr2=0x7f4f201a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:43.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.233+0000 7f4f25600700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f20106870 0x7f4f201a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.233+0000 7f4f25600700 1 -- 172.21.15.131:0/981558341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c0095f0 con 0x7f4f201071c0 2024-05-10T05:32:43.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.233+0000 7f4f25600700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f201071c0 0x7f4f2019a5d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f18007f10 tx=0x7f4f1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:43.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.233+0000 7f4f1d600700 1 -- 172.21.15.131:0/981558341 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18009ea0 con 0x7f4f201071c0 2024-05-10T05:32:43.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.233+0000 7f4f27600700 1 -- 172.21.15.131:0/981558341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2019b180 con 0x7f4f201071c0 2024-05-10T05:32:43.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.233+0000 7f4f1d600700 1 -- 172.21.15.131:0/981558341 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f1800f040 con 0x7f4f201071c0 2024-05-10T05:32:43.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.233+0000 7f4f1d600700 1 -- 172.21.15.131:0/981558341 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18004350 con 0x7f4f201071c0 2024-05-10T05:32:43.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.234+0000 7f4f27600700 1 -- 172.21.15.131:0/981558341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f201a7640 con 0x7f4f201071c0 2024-05-10T05:32:43.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.235+0000 7f4f27600700 1 -- 172.21.15.131:0/981558341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f201033f0 con 0x7f4f201071c0 2024-05-10T05:32:43.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.237+0000 7f4f1d600700 1 -- 172.21.15.131:0/981558341 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4f1800db20 con 0x7f4f201071c0 2024-05-10T05:32:43.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.238+0000 7f4f1d600700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a11b0 0x7f4f080a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:43.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.238+0000 7f4f1d600700 1 -- 172.21.15.131:0/981558341 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4f180cc330 con 0x7f4f201071c0 2024-05-10T05:32:43.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.238+0000 7f4f24c00700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a11b0 0x7f4f080a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:43.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.239+0000 7f4f24c00700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a11b0 0x7f4f080a36c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4f20105780 tx=0x7f4f14009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:43.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.243+0000 7f4f1d600700 1 -- 172.21.15.131:0/981558341 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4f18095900 con 0x7f4f201071c0 2024-05-10T05:32:43.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:42 smithi157 ceph-mon[78427]: from='client.24901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:43.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:42 smithi142 ceph-mon[77667]: from='client.24901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:43.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.443+0000 7f4f27600700 1 -- 172.21.15.131:0/981558341 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f2019bcf0 con 0x7f4f080a11b0 2024-05-10T05:32:43.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.453+0000 7f4f1d600700 1 -- 172.21.15.131:0/981558341 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340297 (secure 0 0 0) 0x7f4f2019bcf0 con 0x7f4f080a11b0 2024-05-10T05:32:43.459 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 -- 172.21.15.131:0/981558341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a11b0 msgr2=0x7f4f080a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a11b0 0x7f4f080a36c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4f20105780 tx=0x7f4f14009260 comp rx=0 tx=0).stop 2024-05-10T05:32:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 -- 172.21.15.131:0/981558341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f201071c0 msgr2=0x7f4f2019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f201071c0 0x7f4f2019a5d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f18007f10 tx=0x7f4f1800d3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 -- 172.21.15.131:0/981558341 shutdown_connections 2024-05-10T05:32:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a11b0 0x7f4f080a36c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f201071c0 0x7f4f2019a5d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f20101f40 0x7f4f201a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 --2- 172.21.15.131:0/981558341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f20106870 0x7f4f201a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 -- 172.21.15.131:0/981558341 >> 172.21.15.131:0/981558341 conn(0x7f4f200fd7f0 msgr2=0x7f4f20108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:43.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 -- 172.21.15.131:0/981558341 shutdown_connections 2024-05-10T05:32:43.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:43.458+0000 7f4f27600700 1 -- 172.21.15.131:0/981558341 wait complete. 2024-05-10T05:32:43.463 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-10T05:32:43.851 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-05-10T05:32:42.841330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657302,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":586,"num_read_kb":410,"num_write":417,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380356,"kb_used_data":11028,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124561468,"statfs":{"total":1151940427776,"available":1151550943232,"internally_reserved":0,"allocated":11292672,"data_stored":7117566,"data_compressed":113718,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19128,"internal_metadata":377992520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003452"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905479+0000","last_change":"2024-05-10T05:31:51.576054+0000","last_active":"2024-05-10T05:32:14.905479+0000","last_peered":"2024-05-10T05:32:14.905479+0000","last_clean":"2024-05-10T05:32:14.905479+0000","last_became_active":"2024-05-10T05:31:51.575186+0000","last_became_peered":"2024-05-10T05:31:51.575186+0000","last_unstale":"2024-05-10T05:32:14.905479+0000","last_undegraded":"2024-05-10T05:32:14.905479+0000","last_fullsized":"2024-05-10T05:32:14.905479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:16:27.654822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912479+0000","last_change":"2024-05-10T05:31:51.576483+0000","last_active":"2024-05-10T05:32:14.912479+0000","last_peered":"2024-05-10T05:32:14.912479+0000","last_clean":"2024-05-10T05:32:14.912479+0000","last_became_active":"2024-05-10T05:31:51.574936+0000","last_became_peered":"2024-05-10T05:31:51.574936+0000","last_unstale":"2024-05-10T05:32:14.912479+0000","last_undegraded":"2024-05-10T05:32:14.912479+0000","last_fullsized":"2024-05-10T05:32:14.912479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:56:51.436880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912069+0000","last_change":"2024-05-10T05:31:51.567114+0000","last_active":"2024-05-10T05:32:14.912069+0000","last_peered":"2024-05-10T05:32:14.912069+0000","last_clean":"2024-05-10T05:32:14.912069+0000","last_became_active":"2024-05-10T05:31:51.564552+0000","last_became_peered":"2024-05-10T05:31:51.564552+0000","last_unstale":"2024-05-10T05:32:14.912069+0000","last_undegraded":"2024-05-10T05:32:14.912069+0000","last_fullsized":"2024-05-10T05:32:14.912069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:40.268764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914315+0000","last_change":"2024-05-10T05:31:51.584471+0000","last_active":"2024-05-10T05:32:14.914315+0000","last_peered":"2024-05-10T05:32:14.914315+0000","last_clean":"2024-05-10T05:32:14.914315+0000","last_became_active":"2024-05-10T05:31:51.583749+0000","last_became_peered":"2024-05-10T05:31:51.583749+0000","last_unstale":"2024-05-10T05:32:14.914315+0000","last_undegraded":"2024-05-10T05:32:14.914315+0000","last_fullsized":"2024-05-10T05:32:14.914315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:30.775286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916284+0000","last_change":"2024-05-10T05:31:51.577698+0000","last_active":"2024-05-10T05:32:14.916284+0000","last_peered":"2024-05-10T05:32:14.916284+0000","last_clean":"2024-05-10T05:32:14.916284+0000","last_became_active":"2024-05-10T05:31:51.571469+0000","last_became_peered":"2024-05-10T05:31:51.571469+0000","last_unstale":"2024-05-10T05:32:14.916284+0000","last_undegraded":"2024-05-10T05:32:14.916284+0000","last_fullsized":"2024-05-10T05:32:14.916284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:35.161267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.911808+0000","last_change":"2024-05-10T05:31:51.573560+0000","last_active":"2024-05-10T05:32:14.911808+0000","last_peered":"2024-05-10T05:32:14.911808+0000","last_clean":"2024-05-10T05:32:14.911808+0000","last_became_active":"2024-05-10T05:31:51.573227+0000","last_became_peered":"2024-05-10T05:31:51.573227+0000","last_unstale":"2024-05-10T05:32:14.911808+0000","last_undegraded":"2024-05-10T05:32:14.911808+0000","last_fullsized":"2024-05-10T05:32:14.911808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:23.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919564+0000","last_change":"2024-05-10T05:31:51.573667+0000","last_active":"2024-05-10T05:32:14.919564+0000","last_peered":"2024-05-10T05:32:14.919564+0000","last_clean":"2024-05-10T05:32:14.919564+0000","last_became_active":"2024-05-10T05:31:51.573362+0000","last_became_peered":"2024-05-10T05:31:51.573362+0000","last_unstale":"2024-05-10T05:32:14.919564+0000","last_undegraded":"2024-05-10T05:32:14.919564+0000","last_fullsized":"2024-05-10T05:32:14.919564+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:00.300134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917768+0000","last_change":"2024-05-10T05:31:51.583596+0000","last_active":"2024-05-10T05:32:14.917768+0000","last_peered":"2024-05-10T05:32:14.917768+0000","last_clean":"2024-05-10T05:32:14.917768+0000","last_became_active":"2024-05-10T05:31:51.583235+0000","last_became_peered":"2024-05-10T05:31:51.583235+0000","last_unstale":"2024-05-10T05:32:14.917768+0000","last_undegraded":"2024-05-10T05:32:14.917768+0000","last_fullsized":"2024-05-10T05:32:14.917768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:05.575975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917655+0000","last_change":"2024-05-10T05:31:51.567627+0000","last_active":"2024-05-10T05:32:14.917655+0000","last_peered":"2024-05-10T05:32:14.917655+0000","last_clean":"2024-05-10T05:32:14.917655+0000","last_became_active":"2024-05-10T05:31:51.567276+0000","last_became_peered":"2024-05-10T05:31:51.567276+0000","last_unstale":"2024-05-10T05:32:14.917655+0000","last_undegraded":"2024-05-10T05:32:14.917655+0000","last_fullsized":"2024-05-10T05:32:14.917655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:16:27.914802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912121+0000","last_change":"2024-05-10T05:31:51.573859+0000","last_active":"2024-05-10T05:32:14.912121+0000","last_peered":"2024-05-10T05:32:14.912121+0000","last_clean":"2024-05-10T05:32:14.912121+0000","last_became_active":"2024-05-10T05:31:51.573493+0000","last_became_peered":"2024-05-10T05:31:51.573493+0000","last_unstale":"2024-05-10T05:32:14.912121+0000","last_undegraded":"2024-05-10T05:32:14.912121+0000","last_fullsized":"2024-05-10T05:32:14.912121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:21.024888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920160+0000","last_change":"2024-05-10T05:31:51.576056+0000","last_active":"2024-05-10T05:32:14.920160+0000","last_peered":"2024-05-10T05:32:14.920160+0000","last_clean":"2024-05-10T05:32:14.920160+0000","last_became_active":"2024-05-10T05:31:51.575146+0000","last_became_peered":"2024-05-10T05:31:51.575146+0000","last_unstale":"2024-05-10T05:32:14.920160+0000","last_undegraded":"2024-05-10T05:32:14.920160+0000","last_fullsized":"2024-05-10T05:32:14.920160+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:41.201052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914209+0000","last_change":"2024-05-10T05:31:51.584710+0000","last_active":"2024-05-10T05:32:14.914209+0000","last_peered":"2024-05-10T05:32:14.914209+0000","last_clean":"2024-05-10T05:32:14.914209+0000","last_became_active":"2024-05-10T05:31:51.584122+0000","last_became_peered":"2024-05-10T05:31:51.584122+0000","last_unstale":"2024-05-10T05:32:14.914209+0000","last_undegraded":"2024-05-10T05:32:14.914209+0000","last_fullsized":"2024-05-10T05:32:14.914209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:43.622981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.924380+0000","last_change":"2024-05-10T05:32:15.924380+0000","last_active":"2024-05-10T05:32:15.924380+0000","last_peered":"2024-05-10T05:32:15.924380+0000","last_clean":"2024-05-10T05:32:15.924380+0000","last_became_active":"2024-05-10T05:32:15.923952+0000","last_became_peered":"2024-05-10T05:32:15.923952+0000","last_unstale":"2024-05-10T05:32:15.924380+0000","last_undegraded":"2024-05-10T05:32:15.924380+0000","last_fullsized":"2024-05-10T05:32:15.924380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:04:14.087785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.911826+0000","last_change":"2024-05-10T05:31:51.574628+0000","last_active":"2024-05-10T05:32:14.911826+0000","last_peered":"2024-05-10T05:32:14.911826+0000","last_clean":"2024-05-10T05:32:14.911826+0000","last_became_active":"2024-05-10T05:31:51.574256+0000","last_became_peered":"2024-05-10T05:31:51.574256+0000","last_unstale":"2024-05-10T05:32:14.911826+0000","last_undegraded":"2024-05-10T05:32:14.911826+0000","last_fullsized":"2024-05-10T05:32:14.911826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:59.174727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.920243+0000","last_change":"2024-05-10T05:32:15.920243+0000","last_active":"2024-05-10T05:32:15.920243+0000","last_peered":"2024-05-10T05:32:15.920243+0000","last_clean":"2024-05-10T05:32:15.920243+0000","last_became_active":"2024-05-10T05:32:15.919822+0000","last_became_peered":"2024-05-10T05:32:15.919822+0000","last_unstale":"2024-05-10T05:32:15.920243+0000","last_undegraded":"2024-05-10T05:32:15.920243+0000","last_fullsized":"2024-05-10T05:32:15.920243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:59.392399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917143+0000","last_change":"2024-05-10T05:31:51.568919+0000","last_active":"2024-05-10T05:32:14.917143+0000","last_peered":"2024-05-10T05:32:14.917143+0000","last_clean":"2024-05-10T05:32:14.917143+0000","last_became_active":"2024-05-10T05:31:51.568582+0000","last_became_peered":"2024-05-10T05:31:51.568582+0000","last_unstale":"2024-05-10T05:32:14.917143+0000","last_undegraded":"2024-05-10T05:32:14.917143+0000","last_fullsized":"2024-05-10T05:32:14.917143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:08.377261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905717+0000","last_change":"2024-05-10T05:31:51.579025+0000","last_active":"2024-05-10T05:32:14.905717+0000","last_peered":"2024-05-10T05:32:14.905717+0000","last_clean":"2024-05-10T05:32:14.905717+0000","last_became_active":"2024-05-10T05:31:51.578704+0000","last_became_peered":"2024-05-10T05:31:51.578704+0000","last_unstale":"2024-05-10T05:32:14.905717+0000","last_undegraded":"2024-05-10T05:32:14.905717+0000","last_fullsized":"2024-05-10T05:32:14.905717+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:16.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913723+0000","last_change":"2024-05-10T05:31:51.573193+0000","last_active":"2024-05-10T05:32:14.913723+0000","last_peered":"2024-05-10T05:32:14.913723+0000","last_clean":"2024-05-10T05:32:14.913723+0000","last_became_active":"2024-05-10T05:31:51.572874+0000","last_became_peered":"2024-05-10T05:31:51.572874+0000","last_unstale":"2024-05-10T05:32:14.913723+0000","last_undegraded":"2024-05-10T05:32:14.913723+0000","last_fullsized":"2024-05-10T05:32:14.913723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:09.867661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919059+0000","last_change":"2024-05-10T05:31:51.581107+0000","last_active":"2024-05-10T05:32:14.919059+0000","last_peered":"2024-05-10T05:32:14.919059+0000","last_clean":"2024-05-10T05:32:14.919059+0000","last_became_active":"2024-05-10T05:31:51.580765+0000","last_became_peered":"2024-05-10T05:31:51.580765+0000","last_unstale":"2024-05-10T05:32:14.919059+0000","last_undegraded":"2024-05-10T05:32:14.919059+0000","last_fullsized":"2024-05-10T05:32:14.919059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:04.164486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:36.554881+0000","last_change":"2024-05-10T05:31:50.986534+0000","last_active":"2024-05-10T05:32:36.554881+0000","last_peered":"2024-05-10T05:32:36.554881+0000","last_clean":"2024-05-10T05:32:36.554881+0000","last_became_active":"2024-05-10T05:31:50.983493+0000","last_became_peered":"2024-05-10T05:31:50.983493+0000","last_unstale":"2024-05-10T05:32:36.554881+0000","last_undegraded":"2024-05-10T05:32:36.554881+0000","last_fullsized":"2024-05-10T05:32:36.554881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:18:56.926317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918423+0000","last_change":"2024-05-10T05:31:51.587121+0000","last_active":"2024-05-10T05:32:14.918423+0000","last_peered":"2024-05-10T05:32:14.918423+0000","last_clean":"2024-05-10T05:32:14.918423+0000","last_became_active":"2024-05-10T05:31:51.586641+0000","last_became_peered":"2024-05-10T05:31:51.586641+0000","last_unstale":"2024-05-10T05:32:14.918423+0000","last_undegraded":"2024-05-10T05:32:14.918423+0000","last_fullsized":"2024-05-10T05:32:14.918423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:52.654918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918650+0000","last_change":"2024-05-10T05:31:50.999755+0000","last_active":"2024-05-10T05:32:14.918650+0000","last_peered":"2024-05-10T05:32:14.918650+0000","last_clean":"2024-05-10T05:32:14.918650+0000","last_became_active":"2024-05-10T05:31:50.999336+0000","last_became_peered":"2024-05-10T05:31:50.999336+0000","last_unstale":"2024-05-10T05:32:14.918650+0000","last_undegraded":"2024-05-10T05:32:14.918650+0000","last_fullsized":"2024-05-10T05:32:14.918650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:43.755130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912608+0000","last_change":"2024-05-10T05:31:51.576938+0000","last_active":"2024-05-10T05:32:14.912608+0000","last_peered":"2024-05-10T05:32:14.912608+0000","last_clean":"2024-05-10T05:32:14.912608+0000","last_became_active":"2024-05-10T05:31:51.576517+0000","last_became_peered":"2024-05-10T05:31:51.576517+0000","last_unstale":"2024-05-10T05:32:14.912608+0000","last_undegraded":"2024-05-10T05:32:14.912608+0000","last_fullsized":"2024-05-10T05:32:14.912608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:52:38.709063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916132+0000","last_change":"2024-05-10T05:31:50.992925+0000","last_active":"2024-05-10T05:32:14.916132+0000","last_peered":"2024-05-10T05:32:14.916132+0000","last_clean":"2024-05-10T05:32:14.916132+0000","last_became_active":"2024-05-10T05:31:50.992574+0000","last_became_peered":"2024-05-10T05:31:50.992574+0000","last_unstale":"2024-05-10T05:32:14.916132+0000","last_undegraded":"2024-05-10T05:32:14.916132+0000","last_fullsized":"2024-05-10T05:32:14.916132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:14.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917334+0000","last_change":"2024-05-10T05:31:51.580145+0000","last_active":"2024-05-10T05:32:14.917334+0000","last_peered":"2024-05-10T05:32:14.917334+0000","last_clean":"2024-05-10T05:32:14.917334+0000","last_became_active":"2024-05-10T05:31:51.576701+0000","last_became_peered":"2024-05-10T05:31:51.576701+0000","last_unstale":"2024-05-10T05:32:14.917334+0000","last_undegraded":"2024-05-10T05:32:14.917334+0000","last_fullsized":"2024-05-10T05:32:14.917334+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:36:21.075571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915647+0000","last_change":"2024-05-10T05:31:50.987863+0000","last_active":"2024-05-10T05:32:14.915647+0000","last_peered":"2024-05-10T05:32:14.915647+0000","last_clean":"2024-05-10T05:32:14.915647+0000","last_became_active":"2024-05-10T05:31:50.987463+0000","last_became_peered":"2024-05-10T05:31:50.987463+0000","last_unstale":"2024-05-10T05:32:14.915647+0000","last_undegraded":"2024-05-10T05:32:14.915647+0000","last_fullsized":"2024-05-10T05:32:14.915647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:12:49.542288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:16.105675+0000","last_change":"2024-05-10T05:32:16.105284+0000","last_active":"2024-05-10T05:32:16.105675+0000","last_peered":"2024-05-10T05:32:16.105675+0000","last_clean":"2024-05-10T05:32:16.105675+0000","last_became_active":"2024-05-10T05:32:16.104915+0000","last_became_peered":"2024-05-10T05:32:16.104915+0000","last_unstale":"2024-05-10T05:32:16.105675+0000","last_undegraded":"2024-05-10T05:32:16.105675+0000","last_fullsized":"2024-05-10T05:32:16.105675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:25.628498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:37.557089+0000","last_change":"2024-05-10T05:31:50.985879+0000","last_active":"2024-05-10T05:32:37.557089+0000","last_peered":"2024-05-10T05:32:37.557089+0000","last_clean":"2024-05-10T05:32:37.557089+0000","last_became_active":"2024-05-10T05:31:50.985476+0000","last_became_peered":"2024-05-10T05:31:50.985476+0000","last_unstale":"2024-05-10T05:32:37.557089+0000","last_undegraded":"2024-05-10T05:32:37.557089+0000","last_fullsized":"2024-05-10T05:32:37.557089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:11:27.737972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915315+0000","last_change":"2024-05-10T05:31:51.570985+0000","last_active":"2024-05-10T05:32:14.915315+0000","last_peered":"2024-05-10T05:32:14.915315+0000","last_clean":"2024-05-10T05:32:14.915315+0000","last_became_active":"2024-05-10T05:31:51.570324+0000","last_became_peered":"2024-05-10T05:31:51.570324+0000","last_unstale":"2024-05-10T05:32:14.915315+0000","last_undegraded":"2024-05-10T05:32:14.915315+0000","last_fullsized":"2024-05-10T05:32:14.915315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:26.947574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.911748+0000","last_change":"2024-05-10T05:31:50.988483+0000","last_active":"2024-05-10T05:32:14.911748+0000","last_peered":"2024-05-10T05:32:14.911748+0000","last_clean":"2024-05-10T05:32:14.911748+0000","last_became_active":"2024-05-10T05:31:50.987897+0000","last_became_peered":"2024-05-10T05:31:50.987897+0000","last_unstale":"2024-05-10T05:32:14.911748+0000","last_undegraded":"2024-05-10T05:32:14.911748+0000","last_fullsized":"2024-05-10T05:32:14.911748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:30:23.474637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.922113+0000","last_change":"2024-05-10T05:32:15.922113+0000","last_active":"2024-05-10T05:32:15.922113+0000","last_peered":"2024-05-10T05:32:15.922113+0000","last_clean":"2024-05-10T05:32:15.922113+0000","last_became_active":"2024-05-10T05:32:15.921665+0000","last_became_peered":"2024-05-10T05:32:15.921665+0000","last_unstale":"2024-05-10T05:32:15.922113+0000","last_undegraded":"2024-05-10T05:32:15.922113+0000","last_fullsized":"2024-05-10T05:32:15.922113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:28.155216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6],"acting":[5,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.908880+0000","last_change":"2024-05-10T05:31:50.981877+0000","last_active":"2024-05-10T05:32:14.908880+0000","last_peered":"2024-05-10T05:32:14.908880+0000","last_clean":"2024-05-10T05:32:14.908880+0000","last_became_active":"2024-05-10T05:31:50.981527+0000","last_became_peered":"2024-05-10T05:31:50.981527+0000","last_unstale":"2024-05-10T05:32:14.908880+0000","last_undegraded":"2024-05-10T05:32:14.908880+0000","last_fullsized":"2024-05-10T05:32:14.908880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:40.735304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917899+0000","last_change":"2024-05-10T05:31:51.581027+0000","last_active":"2024-05-10T05:32:14.917899+0000","last_peered":"2024-05-10T05:32:14.917899+0000","last_clean":"2024-05-10T05:32:14.917899+0000","last_became_active":"2024-05-10T05:31:51.580620+0000","last_became_peered":"2024-05-10T05:31:51.580620+0000","last_unstale":"2024-05-10T05:32:14.917899+0000","last_undegraded":"2024-05-10T05:32:14.917899+0000","last_fullsized":"2024-05-10T05:32:14.917899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:30:15.378179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913340+0000","last_change":"2024-05-10T05:31:50.987633+0000","last_active":"2024-05-10T05:32:14.913340+0000","last_peered":"2024-05-10T05:32:14.913340+0000","last_clean":"2024-05-10T05:32:14.913340+0000","last_became_active":"2024-05-10T05:31:50.986750+0000","last_became_peered":"2024-05-10T05:31:50.986750+0000","last_unstale":"2024-05-10T05:32:14.913340+0000","last_undegraded":"2024-05-10T05:32:14.913340+0000","last_fullsized":"2024-05-10T05:32:14.913340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:17:47.705415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912961+0000","last_change":"2024-05-10T05:31:51.589484+0000","last_active":"2024-05-10T05:32:14.912961+0000","last_peered":"2024-05-10T05:32:14.912961+0000","last_clean":"2024-05-10T05:32:14.912961+0000","last_became_active":"2024-05-10T05:31:51.588992+0000","last_became_peered":"2024-05-10T05:31:51.588992+0000","last_unstale":"2024-05-10T05:32:14.912961+0000","last_undegraded":"2024-05-10T05:32:14.912961+0000","last_fullsized":"2024-05-10T05:32:14.912961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:32.360333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:40.560472+0000","last_change":"2024-05-10T05:31:50.988945+0000","last_active":"2024-05-10T05:32:40.560472+0000","last_peered":"2024-05-10T05:32:40.560472+0000","last_clean":"2024-05-10T05:32:40.560472+0000","last_became_active":"2024-05-10T05:31:50.988566+0000","last_became_peered":"2024-05-10T05:31:50.988566+0000","last_unstale":"2024-05-10T05:32:40.560472+0000","last_undegraded":"2024-05-10T05:32:40.560472+0000","last_fullsized":"2024-05-10T05:32:40.560472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:06:43.225346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917570+0000","last_change":"2024-05-10T05:31:51.577949+0000","last_active":"2024-05-10T05:32:14.917570+0000","last_peered":"2024-05-10T05:32:14.917570+0000","last_clean":"2024-05-10T05:32:14.917570+0000","last_became_active":"2024-05-10T05:31:51.577011+0000","last_became_peered":"2024-05-10T05:31:51.577011+0000","last_unstale":"2024-05-10T05:32:14.917570+0000","last_undegraded":"2024-05-10T05:32:14.917570+0000","last_fullsized":"2024-05-10T05:32:14.917570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:09.988700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920114+0000","last_change":"2024-05-10T05:31:50.984888+0000","last_active":"2024-05-10T05:32:14.920114+0000","last_peered":"2024-05-10T05:32:14.920114+0000","last_clean":"2024-05-10T05:32:14.920114+0000","last_became_active":"2024-05-10T05:31:50.984267+0000","last_became_peered":"2024-05-10T05:31:50.984267+0000","last_unstale":"2024-05-10T05:32:14.920114+0000","last_undegraded":"2024-05-10T05:32:14.920114+0000","last_fullsized":"2024-05-10T05:32:14.920114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:44.797469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912532+0000","last_change":"2024-05-10T05:31:51.576863+0000","last_active":"2024-05-10T05:32:14.912532+0000","last_peered":"2024-05-10T05:32:14.912532+0000","last_clean":"2024-05-10T05:32:14.912532+0000","last_became_active":"2024-05-10T05:31:51.576468+0000","last_became_peered":"2024-05-10T05:31:51.576468+0000","last_unstale":"2024-05-10T05:32:14.912532+0000","last_undegraded":"2024-05-10T05:32:14.912532+0000","last_fullsized":"2024-05-10T05:32:14.912532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:20.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913803+0000","last_change":"2024-05-10T05:31:51.001566+0000","last_active":"2024-05-10T05:32:14.913803+0000","last_peered":"2024-05-10T05:32:14.913803+0000","last_clean":"2024-05-10T05:32:14.913803+0000","last_became_active":"2024-05-10T05:31:51.001203+0000","last_became_peered":"2024-05-10T05:31:51.001203+0000","last_unstale":"2024-05-10T05:32:14.913803+0000","last_undegraded":"2024-05-10T05:32:14.913803+0000","last_fullsized":"2024-05-10T05:32:14.913803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:17:45.465039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917505+0000","last_change":"2024-05-10T05:31:51.579114+0000","last_active":"2024-05-10T05:32:14.917505+0000","last_peered":"2024-05-10T05:32:14.917505+0000","last_clean":"2024-05-10T05:32:14.917505+0000","last_became_active":"2024-05-10T05:31:51.577507+0000","last_became_peered":"2024-05-10T05:31:51.577507+0000","last_unstale":"2024-05-10T05:32:14.917505+0000","last_undegraded":"2024-05-10T05:32:14.917505+0000","last_fullsized":"2024-05-10T05:32:14.917505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:43.777983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":137,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:40.560356+0000","last_change":"2024-05-10T05:31:50.983558+0000","last_active":"2024-05-10T05:32:40.560356+0000","last_peered":"2024-05-10T05:32:40.560356+0000","last_clean":"2024-05-10T05:32:40.560356+0000","last_became_active":"2024-05-10T05:31:50.983087+0000","last_became_peered":"2024-05-10T05:31:50.983087+0000","last_unstale":"2024-05-10T05:32:40.560356+0000","last_undegraded":"2024-05-10T05:32:40.560356+0000","last_fullsized":"2024-05-10T05:32:40.560356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:08.753898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59684,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":105,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915844+0000","last_change":"2024-05-10T05:31:51.574024+0000","last_active":"2024-05-10T05:32:14.915844+0000","last_peered":"2024-05-10T05:32:14.915844+0000","last_clean":"2024-05-10T05:32:14.915844+0000","last_became_active":"2024-05-10T05:31:51.573703+0000","last_became_peered":"2024-05-10T05:31:51.573703+0000","last_unstale":"2024-05-10T05:32:14.915844+0000","last_undegraded":"2024-05-10T05:32:14.915844+0000","last_fullsized":"2024-05-10T05:32:14.915844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:55:23.435991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918608+0000","last_change":"2024-05-10T05:31:50.986795+0000","last_active":"2024-05-10T05:32:14.918608+0000","last_peered":"2024-05-10T05:32:14.918608+0000","last_clean":"2024-05-10T05:32:14.918608+0000","last_became_active":"2024-05-10T05:31:50.986428+0000","last_became_peered":"2024-05-10T05:31:50.986428+0000","last_unstale":"2024-05-10T05:32:14.918608+0000","last_undegraded":"2024-05-10T05:32:14.918608+0000","last_fullsized":"2024-05-10T05:32:14.918608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:11.039632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.906449+0000","last_change":"2024-05-10T05:31:51.585595+0000","last_active":"2024-05-10T05:32:14.906449+0000","last_peered":"2024-05-10T05:32:14.906449+0000","last_clean":"2024-05-10T05:32:14.906449+0000","last_became_active":"2024-05-10T05:31:51.585091+0000","last_became_peered":"2024-05-10T05:31:51.585091+0000","last_unstale":"2024-05-10T05:32:14.906449+0000","last_undegraded":"2024-05-10T05:32:14.906449+0000","last_fullsized":"2024-05-10T05:32:14.906449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:10:10.489430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913631+0000","last_change":"2024-05-10T05:31:50.984408+0000","last_active":"2024-05-10T05:32:14.913631+0000","last_peered":"2024-05-10T05:32:14.913631+0000","last_clean":"2024-05-10T05:32:14.913631+0000","last_became_active":"2024-05-10T05:31:50.983098+0000","last_became_peered":"2024-05-10T05:31:50.983098+0000","last_unstale":"2024-05-10T05:32:14.913631+0000","last_undegraded":"2024-05-10T05:32:14.913631+0000","last_fullsized":"2024-05-10T05:32:14.913631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:31.399649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919703+0000","last_change":"2024-05-10T05:31:51.576160+0000","last_active":"2024-05-10T05:32:14.919703+0000","last_peered":"2024-05-10T05:32:14.919703+0000","last_clean":"2024-05-10T05:32:14.919703+0000","last_became_active":"2024-05-10T05:31:51.575806+0000","last_became_peered":"2024-05-10T05:31:51.575806+0000","last_unstale":"2024-05-10T05:32:14.919703+0000","last_undegraded":"2024-05-10T05:32:14.919703+0000","last_fullsized":"2024-05-10T05:32:14.919703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:53:25.660988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:41.562518+0000","last_change":"2024-05-10T05:31:50.982752+0000","last_active":"2024-05-10T05:32:41.562518+0000","last_peered":"2024-05-10T05:32:41.562518+0000","last_clean":"2024-05-10T05:32:41.562518+0000","last_became_active":"2024-05-10T05:31:50.982299+0000","last_became_peered":"2024-05-10T05:31:50.982299+0000","last_unstale":"2024-05-10T05:32:41.562518+0000","last_undegraded":"2024-05-10T05:32:41.562518+0000","last_fullsized":"2024-05-10T05:32:41.562518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:52:40.206967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919413+0000","last_change":"2024-05-10T05:31:51.577081+0000","last_active":"2024-05-10T05:32:14.919413+0000","last_peered":"2024-05-10T05:32:14.919413+0000","last_clean":"2024-05-10T05:32:14.919413+0000","last_became_active":"2024-05-10T05:31:51.576729+0000","last_became_peered":"2024-05-10T05:31:51.576729+0000","last_unstale":"2024-05-10T05:32:14.919413+0000","last_undegraded":"2024-05-10T05:32:14.919413+0000","last_fullsized":"2024-05-10T05:32:14.919413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:14.072427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915219+0000","last_change":"2024-05-10T05:31:50.991868+0000","last_active":"2024-05-10T05:32:14.915219+0000","last_peered":"2024-05-10T05:32:14.915219+0000","last_clean":"2024-05-10T05:32:14.915219+0000","last_became_active":"2024-05-10T05:31:50.991471+0000","last_became_peered":"2024-05-10T05:31:50.991471+0000","last_unstale":"2024-05-10T05:32:14.915219+0000","last_undegraded":"2024-05-10T05:32:14.915219+0000","last_fullsized":"2024-05-10T05:32:14.915219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:22.346376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913564+0000","last_change":"2024-05-10T05:31:51.584726+0000","last_active":"2024-05-10T05:32:14.913564+0000","last_peered":"2024-05-10T05:32:14.913564+0000","last_clean":"2024-05-10T05:32:14.913564+0000","last_became_active":"2024-05-10T05:31:51.584327+0000","last_became_peered":"2024-05-10T05:31:51.584327+0000","last_unstale":"2024-05-10T05:32:14.913564+0000","last_undegraded":"2024-05-10T05:32:14.913564+0000","last_fullsized":"2024-05-10T05:32:14.913564+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:56:43.561717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915510+0000","last_change":"2024-05-10T05:31:50.989071+0000","last_active":"2024-05-10T05:32:14.915510+0000","last_peered":"2024-05-10T05:32:14.915510+0000","last_clean":"2024-05-10T05:32:14.915510+0000","last_became_active":"2024-05-10T05:31:50.988044+0000","last_became_peered":"2024-05-10T05:31:50.988044+0000","last_unstale":"2024-05-10T05:32:14.915510+0000","last_undegraded":"2024-05-10T05:32:14.915510+0000","last_fullsized":"2024-05-10T05:32:14.915510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:24.122564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905328+0000","last_change":"2024-05-10T05:31:51.576793+0000","last_active":"2024-05-10T05:32:14.905328+0000","last_peered":"2024-05-10T05:32:14.905328+0000","last_clean":"2024-05-10T05:32:14.905328+0000","last_became_active":"2024-05-10T05:31:51.576333+0000","last_became_peered":"2024-05-10T05:31:51.576333+0000","last_unstale":"2024-05-10T05:32:14.905328+0000","last_undegraded":"2024-05-10T05:32:14.905328+0000","last_fullsized":"2024-05-10T05:32:14.905328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:42.418736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913436+0000","last_change":"2024-05-10T05:31:50.997233+0000","last_active":"2024-05-10T05:32:14.913436+0000","last_peered":"2024-05-10T05:32:14.913436+0000","last_clean":"2024-05-10T05:32:14.913436+0000","last_became_active":"2024-05-10T05:31:50.996857+0000","last_became_peered":"2024-05-10T05:31:50.996857+0000","last_unstale":"2024-05-10T05:32:14.913436+0000","last_undegraded":"2024-05-10T05:32:14.913436+0000","last_fullsized":"2024-05-10T05:32:14.913436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:35.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.927682+0000","last_change":"2024-05-10T05:32:15.927682+0000","last_active":"2024-05-10T05:32:15.927682+0000","last_peered":"2024-05-10T05:32:15.927682+0000","last_clean":"2024-05-10T05:32:15.927682+0000","last_became_active":"2024-05-10T05:32:15.924197+0000","last_became_peered":"2024-05-10T05:32:15.924197+0000","last_unstale":"2024-05-10T05:32:15.927682+0000","last_undegraded":"2024-05-10T05:32:15.927682+0000","last_fullsized":"2024-05-10T05:32:15.927682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:14.747603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:39.559649+0000","last_change":"2024-05-10T05:31:50.991113+0000","last_active":"2024-05-10T05:32:39.559649+0000","last_peered":"2024-05-10T05:32:39.559649+0000","last_clean":"2024-05-10T05:32:39.559649+0000","last_became_active":"2024-05-10T05:31:50.990745+0000","last_became_peered":"2024-05-10T05:31:50.990745+0000","last_unstale":"2024-05-10T05:32:39.559649+0000","last_undegraded":"2024-05-10T05:32:39.559649+0000","last_fullsized":"2024-05-10T05:32:39.559649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:49:56.586231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912448+0000","last_change":"2024-05-10T05:31:51.582057+0000","last_active":"2024-05-10T05:32:14.912448+0000","last_peered":"2024-05-10T05:32:14.912448+0000","last_clean":"2024-05-10T05:32:14.912448+0000","last_became_active":"2024-05-10T05:31:51.581672+0000","last_became_peered":"2024-05-10T05:31:51.581672+0000","last_unstale":"2024-05-10T05:32:14.912448+0000","last_undegraded":"2024-05-10T05:32:14.912448+0000","last_fullsized":"2024-05-10T05:32:14.912448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:45:09.122256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914210+0000","last_change":"2024-05-10T05:31:51.585659+0000","last_active":"2024-05-10T05:32:14.914210+0000","last_peered":"2024-05-10T05:32:14.914210+0000","last_clean":"2024-05-10T05:32:14.914210+0000","last_became_active":"2024-05-10T05:31:51.584540+0000","last_became_peered":"2024-05-10T05:31:51.584540+0000","last_unstale":"2024-05-10T05:32:14.914210+0000","last_undegraded":"2024-05-10T05:32:14.914210+0000","last_fullsized":"2024-05-10T05:32:14.914210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:20.195369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:36.554882+0000","last_change":"2024-05-10T05:31:50.993547+0000","last_active":"2024-05-10T05:32:36.554882+0000","last_peered":"2024-05-10T05:32:36.554882+0000","last_clean":"2024-05-10T05:32:36.554882+0000","last_became_active":"2024-05-10T05:31:50.993190+0000","last_became_peered":"2024-05-10T05:31:50.993190+0000","last_unstale":"2024-05-10T05:32:36.554882+0000","last_undegraded":"2024-05-10T05:32:36.554882+0000","last_fullsized":"2024-05-10T05:32:36.554882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:06.159572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918191+0000","last_change":"2024-05-10T05:31:51.587678+0000","last_active":"2024-05-10T05:32:14.918191+0000","last_peered":"2024-05-10T05:32:14.918191+0000","last_clean":"2024-05-10T05:32:14.918191+0000","last_became_active":"2024-05-10T05:31:51.586999+0000","last_became_peered":"2024-05-10T05:31:51.586999+0000","last_unstale":"2024-05-10T05:32:14.918191+0000","last_undegraded":"2024-05-10T05:32:14.918191+0000","last_fullsized":"2024-05-10T05:32:14.918191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:18:00.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914737+0000","last_change":"2024-05-10T05:31:50.979131+0000","last_active":"2024-05-10T05:32:14.914737+0000","last_peered":"2024-05-10T05:32:14.914737+0000","last_clean":"2024-05-10T05:32:14.914737+0000","last_became_active":"2024-05-10T05:31:50.978715+0000","last_became_peered":"2024-05-10T05:31:50.978715+0000","last_unstale":"2024-05-10T05:32:14.914737+0000","last_undegraded":"2024-05-10T05:32:14.914737+0000","last_fullsized":"2024-05-10T05:32:14.914737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:29:11.953963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919607+0000","last_change":"2024-05-10T05:30:59.356688+0000","last_active":"2024-05-10T05:32:14.919607+0000","last_peered":"2024-05-10T05:32:14.919607+0000","last_clean":"2024-05-10T05:32:14.919607+0000","last_became_active":"2024-05-10T05:30:59.356313+0000","last_became_peered":"2024-05-10T05:30:59.356313+0000","last_unstale":"2024-05-10T05:32:14.919607+0000","last_undegraded":"2024-05-10T05:32:14.919607+0000","last_fullsized":"2024-05-10T05:32:14.919607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_clean_scrub_stamp":"2024-05-10T05:28:15.503591+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:43.011495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.908778+0000","last_change":"2024-05-10T05:31:51.580259+0000","last_active":"2024-05-10T05:32:14.908778+0000","last_peered":"2024-05-10T05:32:14.908778+0000","last_clean":"2024-05-10T05:32:14.908778+0000","last_became_active":"2024-05-10T05:31:51.578650+0000","last_became_peered":"2024-05-10T05:31:51.578650+0000","last_unstale":"2024-05-10T05:32:14.908778+0000","last_undegraded":"2024-05-10T05:32:14.908778+0000","last_fullsized":"2024-05-10T05:32:14.908778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:48.630183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919744+0000","last_change":"2024-05-10T05:31:50.987708+0000","last_active":"2024-05-10T05:32:14.919744+0000","last_peered":"2024-05-10T05:32:14.919744+0000","last_clean":"2024-05-10T05:32:14.919744+0000","last_became_active":"2024-05-10T05:31:50.987347+0000","last_became_peered":"2024-05-10T05:31:50.987347+0000","last_unstale":"2024-05-10T05:32:14.919744+0000","last_undegraded":"2024-05-10T05:32:14.919744+0000","last_fullsized":"2024-05-10T05:32:14.919744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:16:29.388690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915952+0000","last_change":"2024-05-10T05:31:50.999338+0000","last_active":"2024-05-10T05:32:14.915952+0000","last_peered":"2024-05-10T05:32:14.915952+0000","last_clean":"2024-05-10T05:32:14.915952+0000","last_became_active":"2024-05-10T05:31:50.999026+0000","last_became_peered":"2024-05-10T05:31:50.999026+0000","last_unstale":"2024-05-10T05:32:14.915952+0000","last_undegraded":"2024-05-10T05:32:14.915952+0000","last_fullsized":"2024-05-10T05:32:14.915952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:31.443409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.906875+0000","last_change":"2024-05-10T05:31:51.574201+0000","last_active":"2024-05-10T05:32:14.906875+0000","last_peered":"2024-05-10T05:32:14.906875+0000","last_clean":"2024-05-10T05:32:14.906875+0000","last_became_active":"2024-05-10T05:31:51.571180+0000","last_became_peered":"2024-05-10T05:31:51.571180+0000","last_unstale":"2024-05-10T05:32:14.906875+0000","last_undegraded":"2024-05-10T05:32:14.906875+0000","last_fullsized":"2024-05-10T05:32:14.906875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:33:16.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'10","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913628+0000","last_change":"2024-05-10T05:31:50.985164+0000","last_active":"2024-05-10T05:32:14.913628+0000","last_peered":"2024-05-10T05:32:14.913628+0000","last_clean":"2024-05-10T05:32:14.913628+0000","last_became_active":"2024-05-10T05:31:50.984760+0000","last_became_peered":"2024-05-10T05:31:50.984760+0000","last_unstale":"2024-05-10T05:32:14.913628+0000","last_undegraded":"2024-05-10T05:32:14.913628+0000","last_fullsized":"2024-05-10T05:32:14.913628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:19.827407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918649+0000","last_change":"2024-05-10T05:31:51.577445+0000","last_active":"2024-05-10T05:32:14.918649+0000","last_peered":"2024-05-10T05:32:14.918649+0000","last_clean":"2024-05-10T05:32:14.918649+0000","last_became_active":"2024-05-10T05:31:51.576671+0000","last_became_peered":"2024-05-10T05:31:51.576671+0000","last_unstale":"2024-05-10T05:32:14.918649+0000","last_undegraded":"2024-05-10T05:32:14.918649+0000","last_fullsized":"2024-05-10T05:32:14.918649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:38.774538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914419+0000","last_change":"2024-05-10T05:31:50.981495+0000","last_active":"2024-05-10T05:32:14.914419+0000","last_peered":"2024-05-10T05:32:14.914419+0000","last_clean":"2024-05-10T05:32:14.914419+0000","last_became_active":"2024-05-10T05:31:50.981127+0000","last_became_peered":"2024-05-10T05:31:50.981127+0000","last_unstale":"2024-05-10T05:32:14.914419+0000","last_undegraded":"2024-05-10T05:32:14.914419+0000","last_fullsized":"2024-05-10T05:32:14.914419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:22:32.257014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.912554+0000","last_change":"2024-05-10T05:31:51.585926+0000","last_active":"2024-05-10T05:32:14.912554+0000","last_peered":"2024-05-10T05:32:14.912554+0000","last_clean":"2024-05-10T05:32:14.912554+0000","last_became_active":"2024-05-10T05:31:51.585576+0000","last_became_peered":"2024-05-10T05:31:51.585576+0000","last_unstale":"2024-05-10T05:32:14.912554+0000","last_undegraded":"2024-05-10T05:32:14.912554+0000","last_fullsized":"2024-05-10T05:32:14.912554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:22.344180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.926062+0000","last_change":"2024-05-10T05:32:15.926062+0000","last_active":"2024-05-10T05:32:15.926062+0000","last_peered":"2024-05-10T05:32:15.926062+0000","last_clean":"2024-05-10T05:32:15.926062+0000","last_became_active":"2024-05-10T05:32:15.923026+0000","last_became_peered":"2024-05-10T05:32:15.923026+0000","last_unstale":"2024-05-10T05:32:15.926062+0000","last_undegraded":"2024-05-10T05:32:15.926062+0000","last_fullsized":"2024-05-10T05:32:15.926062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:34.188115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915162+0000","last_change":"2024-05-10T05:31:51.573279+0000","last_active":"2024-05-10T05:32:14.915162+0000","last_peered":"2024-05-10T05:32:14.915162+0000","last_clean":"2024-05-10T05:32:14.915162+0000","last_became_active":"2024-05-10T05:31:51.572916+0000","last_became_peered":"2024-05-10T05:31:51.572916+0000","last_unstale":"2024-05-10T05:32:14.915162+0000","last_undegraded":"2024-05-10T05:32:14.915162+0000","last_fullsized":"2024-05-10T05:32:14.915162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:32:39.802463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918101+0000","last_change":"2024-05-10T05:31:50.983771+0000","last_active":"2024-05-10T05:32:14.918101+0000","last_peered":"2024-05-10T05:32:14.918101+0000","last_clean":"2024-05-10T05:32:14.918101+0000","last_became_active":"2024-05-10T05:31:50.981849+0000","last_became_peered":"2024-05-10T05:31:50.981849+0000","last_unstale":"2024-05-10T05:32:14.918101+0000","last_undegraded":"2024-05-10T05:32:14.918101+0000","last_fullsized":"2024-05-10T05:32:14.918101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:15.990869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905211+0000","last_change":"2024-05-10T05:31:51.568005+0000","last_active":"2024-05-10T05:32:14.905211+0000","last_peered":"2024-05-10T05:32:14.905211+0000","last_clean":"2024-05-10T05:32:14.905211+0000","last_became_active":"2024-05-10T05:31:51.567347+0000","last_became_peered":"2024-05-10T05:31:51.567347+0000","last_unstale":"2024-05-10T05:32:14.905211+0000","last_undegraded":"2024-05-10T05:32:14.905211+0000","last_fullsized":"2024-05-10T05:32:14.905211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:29.075816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.926267+0000","last_change":"2024-05-10T05:32:15.925869+0000","last_active":"2024-05-10T05:32:15.926267+0000","last_peered":"2024-05-10T05:32:15.926267+0000","last_clean":"2024-05-10T05:32:15.926267+0000","last_became_active":"2024-05-10T05:32:15.920591+0000","last_became_peered":"2024-05-10T05:32:15.920591+0000","last_unstale":"2024-05-10T05:32:15.926267+0000","last_undegraded":"2024-05-10T05:32:15.926267+0000","last_fullsized":"2024-05-10T05:32:15.926267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:13:20.398637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918979+0000","last_change":"2024-05-10T05:31:51.588255+0000","last_active":"2024-05-10T05:32:14.918979+0000","last_peered":"2024-05-10T05:32:14.918979+0000","last_clean":"2024-05-10T05:32:14.918979+0000","last_became_active":"2024-05-10T05:31:51.587912+0000","last_became_peered":"2024-05-10T05:31:51.587912+0000","last_unstale":"2024-05-10T05:32:14.918979+0000","last_undegraded":"2024-05-10T05:32:14.918979+0000","last_fullsized":"2024-05-10T05:32:14.918979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:18.515739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916229+0000","last_change":"2024-05-10T05:31:50.992392+0000","last_active":"2024-05-10T05:32:14.916229+0000","last_peered":"2024-05-10T05:32:14.916229+0000","last_clean":"2024-05-10T05:32:14.916229+0000","last_became_active":"2024-05-10T05:31:50.992119+0000","last_became_peered":"2024-05-10T05:31:50.992119+0000","last_unstale":"2024-05-10T05:32:14.916229+0000","last_undegraded":"2024-05-10T05:32:14.916229+0000","last_fullsized":"2024-05-10T05:32:14.916229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:11.515013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919747+0000","last_change":"2024-05-10T05:31:51.567780+0000","last_active":"2024-05-10T05:32:14.919747+0000","last_peered":"2024-05-10T05:32:14.919747+0000","last_clean":"2024-05-10T05:32:14.919747+0000","last_became_active":"2024-05-10T05:31:51.567455+0000","last_became_peered":"2024-05-10T05:31:51.567455+0000","last_unstale":"2024-05-10T05:32:14.919747+0000","last_undegraded":"2024-05-10T05:32:14.919747+0000","last_fullsized":"2024-05-10T05:32:14.919747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:47.108131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:38.558428+0000","last_change":"2024-05-10T05:31:50.997374+0000","last_active":"2024-05-10T05:32:38.558428+0000","last_peered":"2024-05-10T05:32:38.558428+0000","last_clean":"2024-05-10T05:32:38.558428+0000","last_became_active":"2024-05-10T05:31:50.997069+0000","last_became_peered":"2024-05-10T05:31:50.997069+0000","last_unstale":"2024-05-10T05:32:38.558428+0000","last_undegraded":"2024-05-10T05:32:38.558428+0000","last_fullsized":"2024-05-10T05:32:38.558428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:25:49.140006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920191+0000","last_change":"2024-05-10T05:31:51.576345+0000","last_active":"2024-05-10T05:32:14.920191+0000","last_peered":"2024-05-10T05:32:14.920191+0000","last_clean":"2024-05-10T05:32:14.920191+0000","last_became_active":"2024-05-10T05:31:51.575704+0000","last_became_peered":"2024-05-10T05:31:51.575704+0000","last_unstale":"2024-05-10T05:32:14.920191+0000","last_undegraded":"2024-05-10T05:32:14.920191+0000","last_fullsized":"2024-05-10T05:32:14.920191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:15.146591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913524+0000","last_change":"2024-05-10T05:31:50.984108+0000","last_active":"2024-05-10T05:32:14.913524+0000","last_peered":"2024-05-10T05:32:14.913524+0000","last_clean":"2024-05-10T05:32:14.913524+0000","last_became_active":"2024-05-10T05:31:50.983728+0000","last_became_peered":"2024-05-10T05:31:50.983728+0000","last_unstale":"2024-05-10T05:32:14.913524+0000","last_undegraded":"2024-05-10T05:32:14.913524+0000","last_fullsized":"2024-05-10T05:32:14.913524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:23:20.096286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919865+0000","last_change":"2024-05-10T05:31:51.572075+0000","last_active":"2024-05-10T05:32:14.919865+0000","last_peered":"2024-05-10T05:32:14.919865+0000","last_clean":"2024-05-10T05:32:14.919865+0000","last_became_active":"2024-05-10T05:31:51.571711+0000","last_became_peered":"2024-05-10T05:31:51.571711+0000","last_unstale":"2024-05-10T05:32:14.919865+0000","last_undegraded":"2024-05-10T05:32:14.919865+0000","last_fullsized":"2024-05-10T05:32:14.919865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:38:27.813001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.907701+0000","last_change":"2024-05-10T05:31:50.983985+0000","last_active":"2024-05-10T05:32:14.907701+0000","last_peered":"2024-05-10T05:32:14.907701+0000","last_clean":"2024-05-10T05:32:14.907701+0000","last_became_active":"2024-05-10T05:31:50.982931+0000","last_became_peered":"2024-05-10T05:31:50.982931+0000","last_unstale":"2024-05-10T05:32:14.907701+0000","last_undegraded":"2024-05-10T05:32:14.907701+0000","last_fullsized":"2024-05-10T05:32:14.907701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:35:17.158696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920269+0000","last_change":"2024-05-10T05:31:51.587250+0000","last_active":"2024-05-10T05:32:14.920269+0000","last_peered":"2024-05-10T05:32:14.920269+0000","last_clean":"2024-05-10T05:32:14.920269+0000","last_became_active":"2024-05-10T05:31:51.586925+0000","last_became_peered":"2024-05-10T05:31:51.586925+0000","last_unstale":"2024-05-10T05:32:14.920269+0000","last_undegraded":"2024-05-10T05:32:14.920269+0000","last_fullsized":"2024-05-10T05:32:14.920269+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:40.585803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905347+0000","last_change":"2024-05-10T05:31:51.567710+0000","last_active":"2024-05-10T05:32:14.905347+0000","last_peered":"2024-05-10T05:32:14.905347+0000","last_clean":"2024-05-10T05:32:14.905347+0000","last_became_active":"2024-05-10T05:31:51.566860+0000","last_became_peered":"2024-05-10T05:31:51.566860+0000","last_unstale":"2024-05-10T05:32:14.905347+0000","last_undegraded":"2024-05-10T05:32:14.905347+0000","last_fullsized":"2024-05-10T05:32:14.905347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:02:09.253345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918980+0000","last_change":"2024-05-10T05:31:51.577458+0000","last_active":"2024-05-10T05:32:14.918980+0000","last_peered":"2024-05-10T05:32:14.918980+0000","last_clean":"2024-05-10T05:32:14.918980+0000","last_became_active":"2024-05-10T05:31:51.577123+0000","last_became_peered":"2024-05-10T05:31:51.577123+0000","last_unstale":"2024-05-10T05:32:14.918980+0000","last_undegraded":"2024-05-10T05:32:14.918980+0000","last_fullsized":"2024-05-10T05:32:14.918980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:51.791090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919038+0000","last_change":"2024-05-10T05:31:51.589361+0000","last_active":"2024-05-10T05:32:14.919038+0000","last_peered":"2024-05-10T05:32:14.919038+0000","last_clean":"2024-05-10T05:32:14.919038+0000","last_became_active":"2024-05-10T05:31:51.589049+0000","last_became_peered":"2024-05-10T05:31:51.589049+0000","last_unstale":"2024-05-10T05:32:14.919038+0000","last_undegraded":"2024-05-10T05:32:14.919038+0000","last_fullsized":"2024-05-10T05:32:14.919038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:13.380505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.919725+0000","last_change":"2024-05-10T05:32:15.919725+0000","last_active":"2024-05-10T05:32:15.919725+0000","last_peered":"2024-05-10T05:32:15.919725+0000","last_clean":"2024-05-10T05:32:15.919725+0000","last_became_active":"2024-05-10T05:32:15.919342+0000","last_became_peered":"2024-05-10T05:32:15.919342+0000","last_unstale":"2024-05-10T05:32:15.919725+0000","last_undegraded":"2024-05-10T05:32:15.919725+0000","last_fullsized":"2024-05-10T05:32:15.919725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:33.785843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.906792+0000","last_change":"2024-05-10T05:31:51.579791+0000","last_active":"2024-05-10T05:32:14.906792+0000","last_peered":"2024-05-10T05:32:14.906792+0000","last_clean":"2024-05-10T05:32:14.906792+0000","last_became_active":"2024-05-10T05:31:51.577635+0000","last_became_peered":"2024-05-10T05:31:51.577635+0000","last_unstale":"2024-05-10T05:32:14.906792+0000","last_undegraded":"2024-05-10T05:32:14.906792+0000","last_fullsized":"2024-05-10T05:32:14.906792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:54.077805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905393+0000","last_change":"2024-05-10T05:31:51.584601+0000","last_active":"2024-05-10T05:32:14.905393+0000","last_peered":"2024-05-10T05:32:14.905393+0000","last_clean":"2024-05-10T05:32:14.905393+0000","last_became_active":"2024-05-10T05:31:51.584172+0000","last_became_peered":"2024-05-10T05:31:51.584172+0000","last_unstale":"2024-05-10T05:32:14.905393+0000","last_undegraded":"2024-05-10T05:32:14.905393+0000","last_fullsized":"2024-05-10T05:32:14.905393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:49:44.987379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919277+0000","last_change":"2024-05-10T05:31:51.573784+0000","last_active":"2024-05-10T05:32:14.919277+0000","last_peered":"2024-05-10T05:32:14.919277+0000","last_clean":"2024-05-10T05:32:14.919277+0000","last_became_active":"2024-05-10T05:31:51.573279+0000","last_became_peered":"2024-05-10T05:31:51.573279+0000","last_unstale":"2024-05-10T05:32:14.919277+0000","last_undegraded":"2024-05-10T05:32:14.919277+0000","last_fullsized":"2024-05-10T05:32:14.919277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:08.375964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918571+0000","last_change":"2024-05-10T05:31:51.585577+0000","last_active":"2024-05-10T05:32:14.918571+0000","last_peered":"2024-05-10T05:32:14.918571+0000","last_clean":"2024-05-10T05:32:14.918571+0000","last_became_active":"2024-05-10T05:31:51.585222+0000","last_became_peered":"2024-05-10T05:31:51.585222+0000","last_unstale":"2024-05-10T05:32:14.918571+0000","last_undegraded":"2024-05-10T05:32:14.918571+0000","last_fullsized":"2024-05-10T05:32:14.918571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:47:27.069513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913429+0000","last_change":"2024-05-10T05:31:51.571405+0000","last_active":"2024-05-10T05:32:14.913429+0000","last_peered":"2024-05-10T05:32:14.913429+0000","last_clean":"2024-05-10T05:32:14.913429+0000","last_became_active":"2024-05-10T05:31:51.571125+0000","last_became_peered":"2024-05-10T05:31:51.571125+0000","last_unstale":"2024-05-10T05:32:14.913429+0000","last_undegraded":"2024-05-10T05:32:14.913429+0000","last_fullsized":"2024-05-10T05:32:14.913429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:49:29.069121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918005+0000","last_change":"2024-05-10T05:31:51.578883+0000","last_active":"2024-05-10T05:32:14.918005+0000","last_peered":"2024-05-10T05:32:14.918005+0000","last_clean":"2024-05-10T05:32:14.918005+0000","last_became_active":"2024-05-10T05:31:51.578582+0000","last_became_peered":"2024-05-10T05:31:51.578582+0000","last_unstale":"2024-05-10T05:32:14.918005+0000","last_undegraded":"2024-05-10T05:32:14.918005+0000","last_fullsized":"2024-05-10T05:32:14.918005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:16.402282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.917342+0000","last_change":"2024-05-10T05:31:51.589269+0000","last_active":"2024-05-10T05:32:14.917342+0000","last_peered":"2024-05-10T05:32:14.917342+0000","last_clean":"2024-05-10T05:32:14.917342+0000","last_became_active":"2024-05-10T05:31:51.588875+0000","last_became_peered":"2024-05-10T05:31:51.588875+0000","last_unstale":"2024-05-10T05:32:14.917342+0000","last_undegraded":"2024-05-10T05:32:14.917342+0000","last_fullsized":"2024-05-10T05:32:14.917342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:12.536555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.907976+0000","last_change":"2024-05-10T05:31:51.567789+0000","last_active":"2024-05-10T05:32:14.907976+0000","last_peered":"2024-05-10T05:32:14.907976+0000","last_clean":"2024-05-10T05:32:14.907976+0000","last_became_active":"2024-05-10T05:31:51.566625+0000","last_became_peered":"2024-05-10T05:31:51.566625+0000","last_unstale":"2024-05-10T05:32:14.907976+0000","last_undegraded":"2024-05-10T05:32:14.907976+0000","last_fullsized":"2024-05-10T05:32:14.907976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:43.042557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.905276+0000","last_change":"2024-05-10T05:31:51.585838+0000","last_active":"2024-05-10T05:32:14.905276+0000","last_peered":"2024-05-10T05:32:14.905276+0000","last_clean":"2024-05-10T05:32:14.905276+0000","last_became_active":"2024-05-10T05:31:51.585473+0000","last_became_peered":"2024-05-10T05:31:51.585473+0000","last_unstale":"2024-05-10T05:32:14.905276+0000","last_undegraded":"2024-05-10T05:32:14.905276+0000","last_fullsized":"2024-05-10T05:32:14.905276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:11:11.282076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247530,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66700,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":460,"num_read_kb":301,"num_write":234,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":200100,"data_compressed":5520,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547221,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":429949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67700000000000005}]},{"osd":2,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.50600000000000001}]},{"osd":5,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.76900000000000002}]},{"osd":6,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.627}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":9,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.83199999999999996}]},{"osd":10,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.69199999999999995}]}]},{"osd":10,"up_from":68,"seq":292057776152,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1081055,"data_compressed":37906,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.625}]},{"osd":2,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.504}]},{"osd":4,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":8,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.755}]},{"osd":9,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.60599999999999998}]},{"osd":11,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.80600000000000005}]}]},{"osd":9,"up_from":63,"seq":270582939675,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":921600,"data_stored":430044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.85399999999999998}]},{"osd":3,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.39100000000000001}]},{"osd":4,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri May 10 05:31:37 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.82899999999999996}]},{"osd":10,"last update":"Fri May 10 05:31:58 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.78800000000000003}]}]},{"osd":8,"up_from":56,"seq":240518168607,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":430305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72399999999999998,"5min":0.995,"15min":0.995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.38900000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72899999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.749},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61699999999999999,"15min":0.624},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73499999999999999}]},{"osd":2,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.502,"15min":0.497},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.753,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77400000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.622,"15min":0.622},"min":{"1min":0.47799999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.746,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91500000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.629}]},{"osd":5,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85199999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.375,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84899999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.44400000000000001}]},{"osd":6,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.496,"5min":0.495,"15min":0.495},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76900000000000002,"15min":0.77100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94199999999999995,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri May 10 05:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.749,"15min":0.751},"min":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99099999999999999,"5min":1.101,"15min":1.101},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.76600000000000001}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80100000000000005}]},{"osd":10,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.83299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397540,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":966656,"data_stored":1023074,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.435,"15min":0.432},"min":{"1min":0.32100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.626},{"interface":"front","average":{"1min":0.501,"5min":0.45100000000000001,"15min":0.443},"min":{"1min":0.38100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.79200000000000004}]},{"osd":3,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.86099999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.48699999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90600000000000003,"5min":1.302,"15min":1.302},"last":0.81699999999999995}]},{"osd":4,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.35999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70599999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.81000000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66900000000000004,"15min":0.66300000000000003},"min":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.93799999999999994}]},{"osd":6,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.079,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85299999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.84299999999999997}]},{"osd":8,"last update":"Fri May 10 05:32:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.90400000000000003,"5min":1.002,"15min":1.002},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.91000000000000003}]},{"osd":9,"last update":"Fri May 10 05:31:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.022},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.875}]},{"osd":10,"last update":"Fri May 10 05:31:56 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.97399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561064,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":901120,"data_stored":429596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.76700000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.54200000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.85699999999999998}]},{"osd":2,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.65500000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.59499999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.871}]},{"osd":5,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.84499999999999997,"15min":0.83399999999999996},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.95999999999999996}]},{"osd":7,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83799999999999997,"15min":0.83199999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81899999999999995,"15min":0.81100000000000005},"min":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.21,"5min":1.252,"15min":1.252},"last":0.65600000000000003}]},{"osd":8,"last update":"Fri May 10 05:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.93799999999999994,"15min":0.95299999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1220000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.95099999999999996,"15min":0.96699999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.76800000000000002}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.0009999999999999}]},{"osd":10,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.02},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.98899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":958464,"data_stored":490193,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.68400000000000005,"15min":0.80800000000000005},"min":{"1min":0.40799999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91700000000000004,"5min":6.3810000000000002,"15min":6.3810000000000002},"last":0.749},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81699999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.78400000000000003}]},{"osd":2,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56599999999999995,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.89400000000000002}]},{"osd":3,"last update":"Fri May 10 05:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.42999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54800000000000004}]},{"osd":4,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.441,"15min":0.42899999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46899999999999997,"15min":0.443},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.56499999999999995,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.624,"15min":0.60399999999999998},"min":{"1min":0.44900000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.95399999999999996}]},{"osd":6,"last update":"Fri May 10 05:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.52400000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86499999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71599999999999997}]},{"osd":8,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.69499999999999995,"15min":0.68200000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.67400000000000004,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.90900000000000003}]},{"osd":9,"last update":"Fri May 10 05:32:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.79700000000000004,"15min":0.79100000000000004},"min":{"1min":0.59499999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.1379999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.88600000000000001}]},{"osd":10,"last update":"Fri May 10 05:31:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.87},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":1.0189999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":921600,"data_stored":430935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 05:32:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1890000000000001,"5min":1.256,"15min":1.256},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.64900000000000002,"15min":0.61099999999999999},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri May 10 05:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.68200000000000005,"15min":0.627},"min":{"1min":0.50800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.503,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.6920000000000002,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.625}]},{"osd":3,"last update":"Fri May 10 05:30:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.35199999999999998}]},{"osd":4,"last update":"Fri May 10 05:32:05 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.71199999999999997,"15min":0.66800000000000004},"min":{"1min":0.59599999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.69699999999999995,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri May 10 05:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.71999999999999997,"15min":0.66300000000000003},"min":{"1min":0.47499999999999998,"5min":0.44,"15min":0.44},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.73499999999999999,"15min":0.69499999999999995},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri May 10 05:31:39 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.75600000000000001,"15min":0.74199999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.347,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.746,"15min":0.73299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.077,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.40400000000000003}]},{"osd":7,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90700000000000003}]},{"osd":8,"last update":"Fri May 10 05:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.88700000000000001,"15min":0.875},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.087,"5min":0.88500000000000001,"15min":0.85099999999999998},"min":{"1min":0.68600000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":4.6559999999999997,"5min":4.6559999999999997,"15min":4.6559999999999997},"last":0.51000000000000001}]},{"osd":9,"last update":"Fri May 10 05:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.56799999999999995}]},{"osd":10,"last update":"Fri May 10 05:31:58 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.84399999999999997}]},{"osd":11,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.81000000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411384,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":978944,"data_stored":490758,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89300000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.55800000000000005,"15min":0.50700000000000001},"min":{"1min":0.52000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.85999999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56200000000000006,"15min":0.52400000000000002},"min":{"1min":0.46700000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79000000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.41999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.81100000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.439}]},{"osd":4,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.629}]},{"osd":5,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.46300000000000002,"15min":0.435},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.49199999999999999,"15min":0.46600000000000003},"min":{"1min":0.307,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.745}]},{"osd":8,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri May 10 05:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.91900000000000004}]},{"osd":10,"last update":"Fri May 10 05:31:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":429352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.42499999999999999,"15min":0.40300000000000002},"min":{"1min":0.38300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.436,"15min":0.40799999999999997},"min":{"1min":0.36499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":1.018}]},{"osd":1,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60599999999999998,"15min":0.59399999999999997},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.53300000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65700000000000003,"15min":0.63100000000000001},"min":{"1min":0.47399999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0589999999999999,"5min":1.206,"15min":1.206},"last":0.75}]},{"osd":4,"last update":"Fri May 10 05:32:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.65300000000000002,"15min":0.63200000000000001},"min":{"1min":0.54700000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri May 10 05:32:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.57599999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94899999999999995,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.72599999999999998,"15min":0.70399999999999996},"min":{"1min":0.48299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.02,"5min":1.109,"15min":1.109},"last":0.85099999999999998}]},{"osd":6,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.74399999999999999,"15min":0.72999999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.59099999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.095,"5min":1.119,"15min":1.119},"last":1.133}]},{"osd":7,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.78200000000000003,"15min":0.77000000000000002},"min":{"1min":0.60599999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.1359999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.84199999999999997,"15min":0.82799999999999996},"min":{"1min":0.72199999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2050000000000001,"5min":1.216,"15min":1.216},"last":0.92600000000000005}]},{"osd":8,"last update":"Fri May 10 05:31:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.753}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.71399999999999997}]},{"osd":10,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.319},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.95799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":901120,"data_stored":429850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri May 10 05:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39100000000000001,"15min":0.36499999999999999},"min":{"1min":0.313,"5min":0.255,"15min":0.255},"max":{"1min":0.57899999999999996,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.39700000000000002},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66000000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri May 10 05:32:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59899999999999998,"15min":0.57599999999999996},"min":{"1min":0.46700000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83099999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri May 10 05:32:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":0.80800000000000005,"15min":0.64900000000000002},"min":{"1min":0.58999999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":13.608000000000001,"5min":13.608000000000001,"15min":13.608000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.621,"15min":0.61299999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82699999999999996,"5min":0.876,"15min":0.876},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri May 10 05:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.63800000000000001,"15min":0.61399999999999999},"min":{"1min":0.51800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.622,"15min":0.60499999999999998},"min":{"1min":0.44900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88300000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri May 10 05:31:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.497,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.51800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1,"5min":1.081,"15min":1.081},"last":0.64500000000000002}]},{"osd":8,"last update":"Fri May 10 05:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.58199999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.95499999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0049999999999999,"5min":1.012,"15min":1.012},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri May 10 05:32:41 2024","interfaces":[{"interface":"back","average":{"1min":1.8220000000000001,"5min":1.048,"15min":0.91900000000000004},"min":{"1min":0.65700000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":13.648,"5min":13.648,"15min":13.648},"last":2.2360000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":2.2639999999999998}]},{"osd":10,"last update":"Fri May 10 05:32:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.78600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":5,"up_from":37,"seq":158913789995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":978944,"data_stored":1022455,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.55700000000000005},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.62,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":4.524,"15min":4.524},"last":0.61199999999999999}]},{"osd":1,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.438,"15min":0.434},"min":{"1min":0.316,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.441,"15min":0.437},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79200000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri May 10 05:32:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59899999999999998,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66000000000000003,"15min":0.65900000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.86799999999999999}]},{"osd":8,"last update":"Fri May 10 05:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.79000000000000004}]},{"osd":9,"last update":"Fri May 10 05:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri May 10 05:31:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.90800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61388,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61953,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61765,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T05:32:43.857 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-10T05:32:43.857 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-10T05:32:43.857 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-10T05:32:43.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-10T05:32:44.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:43 smithi131 ceph-mon[37264]: pgmap v266: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:44.226 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:43 smithi157 ceph-mon[78427]: pgmap v266: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:43 smithi142 ceph-mon[77667]: pgmap v266: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:45.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:44 smithi157 ceph-mon[78427]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:45.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:44 smithi142 ceph-mon[77667]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:45.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:44 smithi131 ceph-mon[37264]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:32:45.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.480+0000 7f19fb200700 1 -- 172.21.15.131:0/2571497562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19fc101b00 msgr2=0x7f19fc101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:45.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.480+0000 7f19fb200700 1 --2- 172.21.15.131:0/2571497562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19fc101b00 0x7f19fc101f10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009990 tx=0x7f19e8009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:45.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.480+0000 7f19fb200700 1 -- 172.21.15.131:0/2571497562 shutdown_connections 2024-05-10T05:32:45.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.480+0000 7f19fb200700 1 --2- 172.21.15.131:0/2571497562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc102450 0x7f19fc10e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:45.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.480+0000 7f19fb200700 1 --2- 172.21.15.131:0/2571497562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19fc107b20 0x7f19fc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:45.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.480+0000 7f19fb200700 1 --2- 172.21.15.131:0/2571497562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19fc101b00 0x7f19fc101f10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:45.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.480+0000 7f19fb200700 1 -- 172.21.15.131:0/2571497562 >> 172.21.15.131:0/2571497562 conn(0x7f19fc077650 msgr2=0x7f19fc077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:45.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.480+0000 7f19fb200700 1 -- 172.21.15.131:0/2571497562 shutdown_connections 2024-05-10T05:32:45.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.481+0000 7f19fb200700 1 -- 172.21.15.131:0/2571497562 wait complete. 2024-05-10T05:32:45.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19fb200700 1 Processor -- start 2024-05-10T05:32:45.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19fb200700 1 -- start start 2024-05-10T05:32:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19fb200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19fc101b00 0x7f19fc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19fb200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19fc102450 0x7f19fc198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19fb200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc107b20 0x7f19fc19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19fb200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc19d190 con 0x7f19fc101b00 2024-05-10T05:32:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19fb200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc19d300 con 0x7f19fc107b20 2024-05-10T05:32:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19fb200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc19d4c0 con 0x7f19fc102450 2024-05-10T05:32:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19f9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19fc101b00 0x7f19fc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19f9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19fc101b00 0x7f19fc198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50814/0 (socket says 172.21.15.131:50814) 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.482+0000 7f19f9e00700 1 -- 172.21.15.131:0/3418657535 learned_addr learned my addr 172.21.15.131:0/3418657535 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.483+0000 7f19f9400700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19fc102450 0x7f19fc198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.483+0000 7f19fa800700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc107b20 0x7f19fc19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.483+0000 7f19f9e00700 1 -- 172.21.15.131:0/3418657535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19fc102450 msgr2=0x7f19fc198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.483+0000 7f19f9e00700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19fc102450 0x7f19fc198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.483+0000 7f19f9e00700 1 -- 172.21.15.131:0/3418657535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc107b20 msgr2=0x7f19fc19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.483+0000 7f19f9e00700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc107b20 0x7f19fc19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.483+0000 7f19f9e00700 1 -- 172.21.15.131:0/3418657535 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80095f0 con 0x7f19fc101b00 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.483+0000 7f19f9400700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19fc102450 0x7f19fc198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.483+0000 7f19f9e00700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19fc101b00 0x7f19fc198380 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00e970 tx=0x7f19ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:45.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.483+0000 7f19f2a00700 1 -- 172.21.15.131:0/3418657535 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec009790 con 0x7f19fc101b00 2024-05-10T05:32:45.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.484+0000 7f19f2a00700 1 -- 172.21.15.131:0/3418657535 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19ec00c7e0 con 0x7f19fc101b00 2024-05-10T05:32:45.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.484+0000 7f19fb200700 1 -- 172.21.15.131:0/3418657535 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19fc19d7a0 con 0x7f19fc101b00 2024-05-10T05:32:45.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.484+0000 7f19f2a00700 1 -- 172.21.15.131:0/3418657535 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0054e0 con 0x7f19fc101b00 2024-05-10T05:32:45.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.484+0000 7f19fb200700 1 -- 172.21.15.131:0/3418657535 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19fc1a50a0 con 0x7f19fc101b00 2024-05-10T05:32:45.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.485+0000 7f19fb200700 1 -- 172.21.15.131:0/3418657535 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19fc10a010 con 0x7f19fc101b00 2024-05-10T05:32:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.487+0000 7f19f2a00700 1 -- 172.21.15.131:0/3418657535 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f19ec00c950 con 0x7f19fc101b00 2024-05-10T05:32:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.488+0000 7f19f2a00700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f19e00a1230 0x7f19e00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.488+0000 7f19f2a00700 1 -- 172.21.15.131:0/3418657535 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f19ec014070 con 0x7f19fc101b00 2024-05-10T05:32:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.488+0000 7f19f9400700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f19e00a1230 0x7f19e00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:45.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.489+0000 7f19f9400700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f19e00a1230 0x7f19e00a3740 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0ff780 tx=0x7f19e801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:45.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.491+0000 7f19f2a00700 1 -- 172.21.15.131:0/3418657535 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f19ec096440 con 0x7f19fc101b00 2024-05-10T05:32:45.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.738+0000 7f19fb200700 1 -- 172.21.15.131:0/3418657535 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f19fc1996e0 con 0x7f19fc101b00 2024-05-10T05:32:45.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.738+0000 7f19f2a00700 1 -- 172.21.15.131:0/3418657535 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f19ec099a60 con 0x7f19fc101b00 2024-05-10T05:32:45.739 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:45.739 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 -- 172.21.15.131:0/3418657535 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f19e00a1230 msgr2=0x7f19e00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f19e00a1230 0x7f19e00a3740 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0ff780 tx=0x7f19e801a040 comp rx=0 tx=0).stop 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 -- 172.21.15.131:0/3418657535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19fc101b00 msgr2=0x7f19fc198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19fc101b00 0x7f19fc198380 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00e970 tx=0x7f19ec00ec80 comp rx=0 tx=0).stop 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 -- 172.21.15.131:0/3418657535 shutdown_connections 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f19e00a1230 0x7f19e00a3740 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19fc101b00 0x7f19fc198380 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc107b20 0x7f19fc19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 --2- 172.21.15.131:0/3418657535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19fc102450 0x7f19fc198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 -- 172.21.15.131:0/3418657535 >> 172.21.15.131:0/3418657535 conn(0x7f19fc077650 msgr2=0x7f19fc100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:45.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 -- 172.21.15.131:0/3418657535 shutdown_connections 2024-05-10T05:32:45.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:45.741+0000 7f19fb200700 1 -- 172.21.15.131:0/3418657535 wait complete. 2024-05-10T05:32:46.129 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:45 smithi131 ceph-mon[37264]: pgmap v267: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:46.130 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:45 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3418657535' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T05:32:46.153 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-10T05:32:46.153 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:32:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:45 smithi157 ceph-mon[78427]: pgmap v267: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:45 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3418657535' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T05:32:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:45 smithi142 ceph-mon[77667]: pgmap v267: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:45 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3418657535' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T05:32:46.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.419+0000 7f1428e00700 1 -- 172.21.15.131:0/1999452396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1424101be0 msgr2=0x7f1424101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:46.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.419+0000 7f1428e00700 1 --2- 172.21.15.131:0/1999452396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1424101be0 0x7f1424101ff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14140099c0 tx=0x7f1414009cd0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.419+0000 7f1428e00700 1 -- 172.21.15.131:0/1999452396 shutdown_connections 2024-05-10T05:32:46.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.419+0000 7f1428e00700 1 --2- 172.21.15.131:0/1999452396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14241036a0 0x7f1424109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.419+0000 7f1428e00700 1 --2- 172.21.15.131:0/1999452396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1424101be0 0x7f1424101ff0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.419+0000 7f1428e00700 1 --2- 172.21.15.131:0/1999452396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1424102d00 0x7f1424103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.419+0000 7f1428e00700 1 -- 172.21.15.131:0/1999452396 >> 172.21.15.131:0/1999452396 conn(0x7f14240fd1d0 msgr2=0x7f14240ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:46.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.419+0000 7f1428e00700 1 -- 172.21.15.131:0/1999452396 shutdown_connections 2024-05-10T05:32:46.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.419+0000 7f1428e00700 1 -- 172.21.15.131:0/1999452396 wait complete. 2024-05-10T05:32:46.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1428e00700 1 Processor -- start 2024-05-10T05:32:46.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1428e00700 1 -- start start 2024-05-10T05:32:46.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1428e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1424101be0 0x7f142419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:46.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1428e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1424102d00 0x7f142419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:46.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1428e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14241036a0 0x7f14241a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1428e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14241a1cb0 con 0x7f1424102d00 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1428e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14241a1e20 con 0x7f1424101be0 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1428e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14241a1fe0 con 0x7f14241036a0 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1421400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1424102d00 0x7f142419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1421400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1424102d00 0x7f142419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50834/0 (socket says 172.21.15.131:50834) 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.421+0000 7f1421400700 1 -- 172.21.15.131:0/3492657104 learned_addr learned my addr 172.21.15.131:0/3492657104 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1421e00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1424101be0 0x7f142419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1422800700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14241036a0 0x7f14241a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1421400700 1 -- 172.21.15.131:0/3492657104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14241036a0 msgr2=0x7f14241a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1421400700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14241036a0 0x7f14241a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1421400700 1 -- 172.21.15.131:0/3492657104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1424101be0 msgr2=0x7f142419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:46.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1421400700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1424101be0 0x7f142419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1421400700 1 -- 172.21.15.131:0/3492657104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c009c40 con 0x7f1424102d00 2024-05-10T05:32:46.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1422800700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14241036a0 0x7f14241a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:46.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1421e00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1424101be0 0x7f142419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:46.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1421400700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1424102d00 0x7f142419a710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f141000e9a0 tx=0x7f141000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:46.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f141aa00700 1 -- 172.21.15.131:0/3492657104 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410009790 con 0x7f1424102d00 2024-05-10T05:32:46.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1428e00700 1 -- 172.21.15.131:0/3492657104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14140095f0 con 0x7f1424102d00 2024-05-10T05:32:46.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f141aa00700 1 -- 172.21.15.131:0/3492657104 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f141000cbd0 con 0x7f1424102d00 2024-05-10T05:32:46.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f141aa00700 1 -- 172.21.15.131:0/3492657104 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410018680 con 0x7f1424102d00 2024-05-10T05:32:46.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.422+0000 7f1428e00700 1 -- 172.21.15.131:0/3492657104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14241a2640 con 0x7f1424102d00 2024-05-10T05:32:46.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.424+0000 7f1428e00700 1 -- 172.21.15.131:0/3492657104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14240694b0 con 0x7f1424102d00 2024-05-10T05:32:46.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.425+0000 7f141aa00700 1 -- 172.21.15.131:0/3492657104 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f141000cd40 con 0x7f1424102d00 2024-05-10T05:32:46.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.427+0000 7f141aa00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14080a11b0 0x7f14080a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:46.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.427+0000 7f141aa00700 1 -- 172.21.15.131:0/3492657104 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1410014070 con 0x7f1424102d00 2024-05-10T05:32:46.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.427+0000 7f1421e00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14080a11b0 0x7f14080a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:46.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.427+0000 7f1421e00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14080a11b0 0x7f14080a36c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1414009fd0 tx=0x7f1414005e80 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:46.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.431+0000 7f141aa00700 1 -- 172.21.15.131:0/3492657104 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1410052d60 con 0x7f1424102d00 2024-05-10T05:32:46.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.643+0000 7f1428e00700 1 -- 172.21.15.131:0/3492657104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f142410d410 con 0x7f1424102d00 2024-05-10T05:32:46.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.644+0000 7f141aa00700 1 -- 172.21.15.131:0/3492657104 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7f1410099e60 con 0x7f1424102d00 2024-05-10T05:32:46.646 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:46.646 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:32:46.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 -- 172.21.15.131:0/3492657104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14080a11b0 msgr2=0x7f14080a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:46.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14080a11b0 0x7f14080a36c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1414009fd0 tx=0x7f1414005e80 comp rx=0 tx=0).stop 2024-05-10T05:32:46.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 -- 172.21.15.131:0/3492657104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1424102d00 msgr2=0x7f142419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:46.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1424102d00 0x7f142419a710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f141000e9a0 tx=0x7f141000ed60 comp rx=0 tx=0).stop 2024-05-10T05:32:46.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 -- 172.21.15.131:0/3492657104 shutdown_connections 2024-05-10T05:32:46.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14080a11b0 0x7f14080a36c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14241036a0 0x7f14241a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1424101be0 0x7f142419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 --2- 172.21.15.131:0/3492657104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1424102d00 0x7f142419a710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 -- 172.21.15.131:0/3492657104 >> 172.21.15.131:0/3492657104 conn(0x7f14240fd1d0 msgr2=0x7f1424073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:46.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 -- 172.21.15.131:0/3492657104 shutdown_connections 2024-05-10T05:32:46.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.647+0000 7f1428e00700 1 -- 172.21.15.131:0/3492657104 wait complete. 2024-05-10T05:32:46.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T05:31:49.481667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T05:31:50.960751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:32:46.675 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:32:46.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.943+0000 7f4f29400700 1 -- 172.21.15.131:0/1157065785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24107880 msgr2=0x7f4f24109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:46.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.943+0000 7f4f29400700 1 --2- 172.21.15.131:0/1157065785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24107880 0x7f4f24109d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00b0a0 tx=0x7f4f1c00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.943+0000 7f4f29400700 1 -- 172.21.15.131:0/1157065785 shutdown_connections 2024-05-10T05:32:46.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.943+0000 7f4f29400700 1 --2- 172.21.15.131:0/1157065785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24107880 0x7f4f24109d90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.943+0000 7f4f29400700 1 --2- 172.21.15.131:0/1157065785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f24069090 0x7f4f24104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.943+0000 7f4f29400700 1 --2- 172.21.15.131:0/1157065785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f24104d10 0x7f4f241071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.943+0000 7f4f29400700 1 -- 172.21.15.131:0/1157065785 >> 172.21.15.131:0/1157065785 conn(0x7f4f240fc0d0 msgr2=0x7f4f240fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:46.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.943+0000 7f4f29400700 1 -- 172.21.15.131:0/1157065785 shutdown_connections 2024-05-10T05:32:46.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.943+0000 7f4f29400700 1 -- 172.21.15.131:0/1157065785 wait complete. 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.944+0000 7f4f29400700 1 Processor -- start 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.944+0000 7f4f29400700 1 -- start start 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.944+0000 7f4f29400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f24069090 0x7f4f2419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f29400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f24104d10 0x7f4f2419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f29400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24107880 0x7f4f241a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f29400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f241a1cc0 con 0x7f4f24107880 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f29400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f241a1e30 con 0x7f4f24069090 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f29400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f241a1ff0 con 0x7f4f24104d10 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f22800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f24069090 0x7f4f2419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f23200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24107880 0x7f4f241a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f22800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f24069090 0x7f4f2419a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:52792/0 (socket says 172.21.15.131:52792) 2024-05-10T05:32:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f22800700 1 -- 172.21.15.131:0/1641887412 learned_addr learned my addr 172.21.15.131:0/1641887412 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f21e00700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f24104d10 0x7f4f2419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f23200700 1 -- 172.21.15.131:0/1641887412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f24104d10 msgr2=0x7f4f2419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f23200700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f24104d10 0x7f4f2419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f23200700 1 -- 172.21.15.131:0/1641887412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f24069090 msgr2=0x7f4f2419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f23200700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f24069090 0x7f4f2419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f23200700 1 -- 172.21.15.131:0/1641887412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c009d00 con 0x7f4f24107880 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.945+0000 7f4f22800700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f24069090 0x7f4f2419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.946+0000 7f4f23200700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24107880 0x7f4f241a16b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c007b90 tx=0x7f4f1c0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.946+0000 7f4f1b400700 1 -- 172.21.15.131:0/1641887412 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c00e040 con 0x7f4f24107880 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.946+0000 7f4f1b400700 1 -- 172.21.15.131:0/1641887412 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f1c00be40 con 0x7f4f24107880 2024-05-10T05:32:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.946+0000 7f4f1b400700 1 -- 172.21.15.131:0/1641887412 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c01b6e0 con 0x7f4f24107880 2024-05-10T05:32:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.946+0000 7f4f29400700 1 -- 172.21.15.131:0/1641887412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f241a2270 con 0x7f4f24107880 2024-05-10T05:32:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.946+0000 7f4f29400700 1 -- 172.21.15.131:0/1641887412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f241a2640 con 0x7f4f24107880 2024-05-10T05:32:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.947+0000 7f4f29400700 1 -- 172.21.15.131:0/1641887412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f240694d0 con 0x7f4f24107880 2024-05-10T05:32:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.949+0000 7f4f1b400700 1 -- 172.21.15.131:0/1641887412 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4f1c01d470 con 0x7f4f24107880 2024-05-10T05:32:46.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.950+0000 7f4f1b400700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a1260 0x7f4f080a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:46.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.950+0000 7f4f1b400700 1 -- 172.21.15.131:0/1641887412 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4f1c0cdc20 con 0x7f4f24107880 2024-05-10T05:32:46.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.950+0000 7f4f22800700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a1260 0x7f4f080a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.951+0000 7f4f22800700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a1260 0x7f4f080a3770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f14011550 tx=0x7f4f14009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:46.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:46.952+0000 7f4f1b400700 1 -- 172.21.15.131:0/1641887412 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4f1c097280 con 0x7f4f24107880 2024-05-10T05:32:47.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.164+0000 7f4f29400700 1 -- 172.21.15.131:0/1641887412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f241a29a0 con 0x7f4f24107880 2024-05-10T05:32:47.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.165+0000 7f4f1b400700 1 -- 172.21.15.131:0/1641887412 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f1c09a8a0 con 0x7f4f24107880 2024-05-10T05:32:47.165 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:32:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 -- 172.21.15.131:0/1641887412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a1260 msgr2=0x7f4f080a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a1260 0x7f4f080a3770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f14011550 tx=0x7f4f14009260 comp rx=0 tx=0).stop 2024-05-10T05:32:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 -- 172.21.15.131:0/1641887412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24107880 msgr2=0x7f4f241a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24107880 0x7f4f241a16b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c007b90 tx=0x7f4f1c0078e0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 -- 172.21.15.131:0/1641887412 shutdown_connections 2024-05-10T05:32:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4f080a1260 0x7f4f080a3770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24107880 0x7f4f241a16b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f24069090 0x7f4f2419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 --2- 172.21.15.131:0/1641887412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f24104d10 0x7f4f2419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 -- 172.21.15.131:0/1641887412 >> 172.21.15.131:0/1641887412 conn(0x7f4f240fc0d0 msgr2=0x7f4f240fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:47.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 -- 172.21.15.131:0/1641887412 shutdown_connections 2024-05-10T05:32:47.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.167+0000 7f4f29400700 1 -- 172.21.15.131:0/1641887412 wait complete. 2024-05-10T05:32:47.195 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T05:32:47.252 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:46 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3492657104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:47.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:46 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3492657104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:47.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:46 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3492657104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:47.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.459+0000 7f851f200700 1 -- 172.21.15.131:0/3995609073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520102b10 msgr2=0x7f8520102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:47.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.459+0000 7f851f200700 1 --2- 172.21.15.131:0/3995609073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520102b10 0x7f8520102f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f850c009990 tx=0x7f850c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.459+0000 7f851f200700 1 -- 172.21.15.131:0/3995609073 shutdown_connections 2024-05-10T05:32:47.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.459+0000 7f851f200700 1 --2- 172.21.15.131:0/3995609073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85201034b0 0x7f8520109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.459+0000 7f851f200700 1 --2- 172.21.15.131:0/3995609073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85201019f0 0x7f8520101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.459+0000 7f851f200700 1 --2- 172.21.15.131:0/3995609073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520102b10 0x7f8520102f20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.460+0000 7f851f200700 1 -- 172.21.15.131:0/3995609073 >> 172.21.15.131:0/3995609073 conn(0x7f85200fd000 msgr2=0x7f85200ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:47.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.460+0000 7f851f200700 1 -- 172.21.15.131:0/3995609073 shutdown_connections 2024-05-10T05:32:47.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.460+0000 7f851f200700 1 -- 172.21.15.131:0/3995609073 wait complete. 2024-05-10T05:32:47.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851f200700 1 Processor -- start 2024-05-10T05:32:47.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851f200700 1 -- start start 2024-05-10T05:32:47.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85201019f0 0x7f8520197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:47.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520102b10 0x7f85201983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851f200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85201034b0 0x7f852019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851f200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852019f900 con 0x7f8520102b10 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851f200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852019fa40 con 0x7f85201034b0 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852019fbd0 con 0x7f85201019f0 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520102b10 0x7f85201983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520102b10 0x7f85201983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50866/0 (socket says 172.21.15.131:50866) 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851d400700 1 -- 172.21.15.131:0/3560322797 learned_addr learned my addr 172.21.15.131:0/3560322797 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851de00700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85201019f0 0x7f8520197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.461+0000 7f851e800700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85201034b0 0x7f852019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.462+0000 7f851d400700 1 -- 172.21.15.131:0/3560322797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85201019f0 msgr2=0x7f8520197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.462+0000 7f851d400700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85201019f0 0x7f8520197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.462+0000 7f851d400700 1 -- 172.21.15.131:0/3560322797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85201034b0 msgr2=0x7f852019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.462+0000 7f851d400700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85201034b0 0x7f852019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.462+0000 7f851d400700 1 -- 172.21.15.131:0/3560322797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c0095f0 con 0x7f8520102b10 2024-05-10T05:32:47.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.462+0000 7f851d400700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520102b10 0x7f85201983a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f850c005910 tx=0x7f850c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:47.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.462+0000 7f8516a00700 1 -- 172.21.15.131:0/3560322797 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c01d070 con 0x7f8520102b10 2024-05-10T05:32:47.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.462+0000 7f851f200700 1 -- 172.21.15.131:0/3560322797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852019fe50 con 0x7f8520102b10 2024-05-10T05:32:47.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.462+0000 7f8516a00700 1 -- 172.21.15.131:0/3560322797 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850c005030 con 0x7f8520102b10 2024-05-10T05:32:47.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.462+0000 7f8516a00700 1 -- 172.21.15.131:0/3560322797 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c004b00 con 0x7f8520102b10 2024-05-10T05:32:47.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.463+0000 7f851f200700 1 -- 172.21.15.131:0/3560322797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85201a01c0 con 0x7f8520102b10 2024-05-10T05:32:47.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.464+0000 7f851f200700 1 -- 172.21.15.131:0/3560322797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8520108c00 con 0x7f8520102b10 2024-05-10T05:32:47.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.466+0000 7f8516a00700 1 -- 172.21.15.131:0/3560322797 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f850c0051a0 con 0x7f8520102b10 2024-05-10T05:32:47.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.467+0000 7f8516a00700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85040a11b0 0x7f85040a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:47.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.467+0000 7f8516a00700 1 -- 172.21.15.131:0/3560322797 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f850c0cddd0 con 0x7f8520102b10 2024-05-10T05:32:47.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.467+0000 7f851de00700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85040a11b0 0x7f85040a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:47.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.468+0000 7f851de00700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85040a11b0 0x7f85040a36c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8520102a10 tx=0x7f8510009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:47.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.471+0000 7f8516a00700 1 -- 172.21.15.131:0/3560322797 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f850c097430 con 0x7f8520102b10 2024-05-10T05:32:47.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.683+0000 7f851f200700 1 -- 172.21.15.131:0/3560322797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f852010d240 con 0x7f8520102b10 2024-05-10T05:32:47.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.684+0000 7f8516a00700 1 -- 172.21.15.131:0/3560322797 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f850c09aa50 con 0x7f8520102b10 2024-05-10T05:32:47.684 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-10T05:32:47.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.686+0000 7f851f200700 1 -- 172.21.15.131:0/3560322797 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85040a11b0 msgr2=0x7f85040a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:47.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.686+0000 7f851f200700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85040a11b0 0x7f85040a36c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8520102a10 tx=0x7f8510009260 comp rx=0 tx=0).stop 2024-05-10T05:32:47.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.686+0000 7f851f200700 1 -- 172.21.15.131:0/3560322797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520102b10 msgr2=0x7f85201983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:47.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.686+0000 7f851f200700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520102b10 0x7f85201983a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f850c005910 tx=0x7f850c004730 comp rx=0 tx=0).stop 2024-05-10T05:32:47.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.686+0000 7f851f200700 1 -- 172.21.15.131:0/3560322797 shutdown_connections 2024-05-10T05:32:47.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.686+0000 7f851f200700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85040a11b0 0x7f85040a36c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.686+0000 7f851f200700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85201034b0 0x7f852019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.686+0000 7f851f200700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85201019f0 0x7f8520197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.686+0000 7f851f200700 1 --2- 172.21.15.131:0/3560322797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520102b10 0x7f85201983a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.686+0000 7f851f200700 1 -- 172.21.15.131:0/3560322797 >> 172.21.15.131:0/3560322797 conn(0x7f85200fd000 msgr2=0x7f8520105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:47.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.687+0000 7f851f200700 1 -- 172.21.15.131:0/3560322797 shutdown_connections 2024-05-10T05:32:47.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.687+0000 7f851f200700 1 -- 172.21.15.131:0/3560322797 wait complete. 2024-05-10T05:32:47.714 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T05:32:47.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.981+0000 7f8a40a00700 1 -- 172.21.15.131:0/3035739940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a3c1036a0 msgr2=0x7f8a3c109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:47.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.981+0000 7f8a40a00700 1 --2- 172.21.15.131:0/3035739940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a3c1036a0 0x7f8a3c109f50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400b0a0 tx=0x7f8a3400b3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.981+0000 7f8a40a00700 1 -- 172.21.15.131:0/3035739940 shutdown_connections 2024-05-10T05:32:47.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.981+0000 7f8a40a00700 1 --2- 172.21.15.131:0/3035739940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a3c1036a0 0x7f8a3c109f50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.981+0000 7f8a40a00700 1 --2- 172.21.15.131:0/3035739940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3c101be0 0x7f8a3c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.981+0000 7f8a40a00700 1 --2- 172.21.15.131:0/3035739940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c102d00 0x7f8a3c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.981+0000 7f8a40a00700 1 -- 172.21.15.131:0/3035739940 >> 172.21.15.131:0/3035739940 conn(0x7f8a3c0fd1d0 msgr2=0x7f8a3c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:47.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.981+0000 7f8a40a00700 1 -- 172.21.15.131:0/3035739940 shutdown_connections 2024-05-10T05:32:47.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.981+0000 7f8a40a00700 1 -- 172.21.15.131:0/3035739940 wait complete. 2024-05-10T05:32:47.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.982+0000 7f8a40a00700 1 Processor -- start 2024-05-10T05:32:47.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.982+0000 7f8a40a00700 1 -- start start 2024-05-10T05:32:47.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.982+0000 7f8a40a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3c101be0 0x7f8a3c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.982+0000 7f8a40a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c102d00 0x7f8a3c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.982+0000 7f8a40a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a3c1036a0 0x7f8a3c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.982+0000 7f8a40a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c19fb00 con 0x7f8a3c1036a0 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.982+0000 7f8a40a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c19fc70 con 0x7f8a3c102d00 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.982+0000 7f8a40a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c19fe30 con 0x7f8a3c101be0 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a3a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a3c1036a0 0x7f8a3c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a3a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a3c1036a0 0x7f8a3c19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50892/0 (socket says 172.21.15.131:50892) 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a39400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c102d00 0x7f8a3c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a3a800700 1 -- 172.21.15.131:0/1901717740 learned_addr learned my addr 172.21.15.131:0/1901717740 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a39e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3c101be0 0x7f8a3c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a3a800700 1 -- 172.21.15.131:0/1901717740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3c101be0 msgr2=0x7f8a3c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a3a800700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3c101be0 0x7f8a3c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a3a800700 1 -- 172.21.15.131:0/1901717740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c102d00 msgr2=0x7f8a3c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a3a800700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c102d00 0x7f8a3c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:47.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a3a800700 1 -- 172.21.15.131:0/1901717740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009d00 con 0x7f8a3c1036a0 2024-05-10T05:32:47.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a39400700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c102d00 0x7f8a3c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:47.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a3a800700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a3c1036a0 0x7f8a3c19f4f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400b070 tx=0x7f8a340078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:47.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a32a00700 1 -- 172.21.15.131:0/1901717740 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400e040 con 0x7f8a3c1036a0 2024-05-10T05:32:47.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.983+0000 7f8a32a00700 1 -- 172.21.15.131:0/1901717740 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a3401d470 con 0x7f8a3c1036a0 2024-05-10T05:32:47.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.984+0000 7f8a40a00700 1 -- 172.21.15.131:0/1901717740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c1a00b0 con 0x7f8a3c1036a0 2024-05-10T05:32:47.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.984+0000 7f8a32a00700 1 -- 172.21.15.131:0/1901717740 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a340052a0 con 0x7f8a3c1036a0 2024-05-10T05:32:47.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.984+0000 7f8a40a00700 1 -- 172.21.15.131:0/1901717740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c1a0580 con 0x7f8a3c1036a0 2024-05-10T05:32:47.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.985+0000 7f8a40a00700 1 -- 172.21.15.131:0/1901717740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a3c108df0 con 0x7f8a3c1036a0 2024-05-10T05:32:47.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.987+0000 7f8a32a00700 1 -- 172.21.15.131:0/1901717740 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f8a3401d5e0 con 0x7f8a3c1036a0 2024-05-10T05:32:47.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.988+0000 7f8a32a00700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a200a12e0 0x7f8a200a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:47.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.988+0000 7f8a32a00700 1 -- 172.21.15.131:0/1901717740 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8a340ce0a0 con 0x7f8a3c1036a0 2024-05-10T05:32:47.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.988+0000 7f8a39e00700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a200a12e0 0x7f8a200a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:47.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.989+0000 7f8a39e00700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a200a12e0 0x7f8a200a37f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c102c00 tx=0x7f8a2c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:47.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:47.990+0000 7f8a32a00700 1 -- 172.21.15.131:0/1901717740 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8a340977e0 con 0x7f8a3c1036a0 2024-05-10T05:32:48.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.198+0000 7f8a40a00700 1 -- 172.21.15.131:0/1901717740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a3c10d440 con 0x7f8a3c1036a0 2024-05-10T05:32:48.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.199+0000 7f8a32a00700 1 -- 172.21.15.131:0/1901717740 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a34021020 con 0x7f8a3c1036a0 2024-05-10T05:32:48.199 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 -- 172.21.15.131:0/1901717740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a200a12e0 msgr2=0x7f8a200a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a200a12e0 0x7f8a200a37f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c102c00 tx=0x7f8a2c009260 comp rx=0 tx=0).stop 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 -- 172.21.15.131:0/1901717740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a3c1036a0 msgr2=0x7f8a3c19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a3c1036a0 0x7f8a3c19f4f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400b070 tx=0x7f8a340078a0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 -- 172.21.15.131:0/1901717740 shutdown_connections 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a200a12e0 0x7f8a200a37f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a3c1036a0 0x7f8a3c19f4f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a3c101be0 0x7f8a3c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 --2- 172.21.15.131:0/1901717740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a3c102d00 0x7f8a3c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 -- 172.21.15.131:0/1901717740 >> 172.21.15.131:0/1901717740 conn(0x7f8a3c0fd1d0 msgr2=0x7f8a3c105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:48.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 -- 172.21.15.131:0/1901717740 shutdown_connections 2024-05-10T05:32:48.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.201+0000 7f8a40a00700 1 -- 172.21.15.131:0/1901717740 wait complete. 2024-05-10T05:32:48.229 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:32:48.286 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:47 smithi131 ceph-mon[37264]: pgmap v268: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:48.286 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:47 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1641887412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:32:48.286 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:47 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3560322797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:32:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:47 smithi157 ceph-mon[78427]: pgmap v268: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:47 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1641887412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:32:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:47 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3560322797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:32:48.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:47 smithi142 ceph-mon[77667]: pgmap v268: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:48.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:47 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1641887412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:32:48.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:47 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3560322797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:32:48.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.490+0000 7f1cbd600700 1 -- 172.21.15.131:0/2395936553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8101be0 msgr2=0x7f1cb8101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:48.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.490+0000 7f1cbd600700 1 --2- 172.21.15.131:0/2395936553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8101be0 0x7f1cb8101ff0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80099c0 tx=0x7f1ca8009cd0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.490+0000 7f1cbd600700 1 -- 172.21.15.131:0/2395936553 shutdown_connections 2024-05-10T05:32:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.490+0000 7f1cbd600700 1 --2- 172.21.15.131:0/2395936553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb81036a0 0x7f1cb8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.490+0000 7f1cbd600700 1 --2- 172.21.15.131:0/2395936553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8101be0 0x7f1cb8101ff0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.490+0000 7f1cbd600700 1 --2- 172.21.15.131:0/2395936553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb8102d00 0x7f1cb8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.490+0000 7f1cbd600700 1 -- 172.21.15.131:0/2395936553 >> 172.21.15.131:0/2395936553 conn(0x7f1cb80fd1d0 msgr2=0x7f1cb80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.490+0000 7f1cbd600700 1 -- 172.21.15.131:0/2395936553 shutdown_connections 2024-05-10T05:32:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.490+0000 7f1cbd600700 1 -- 172.21.15.131:0/2395936553 wait complete. 2024-05-10T05:32:48.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.491+0000 7f1cbd600700 1 Processor -- start 2024-05-10T05:32:48.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.491+0000 7f1cbd600700 1 -- start start 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cbd600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8101be0 0x7f1cb8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cbd600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb8102d00 0x7f1cb81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cbd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb81036a0 0x7f1cb819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cbd600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb819fa90 con 0x7f1cb81036a0 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cbd600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb819fc00 con 0x7f1cb8102d00 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cbd600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb819fdc0 con 0x7f1cb8101be0 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cb7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb81036a0 0x7f1cb819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cb7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb81036a0 0x7f1cb819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50902/0 (socket says 172.21.15.131:50902) 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cb7400700 1 -- 172.21.15.131:0/3815571135 learned_addr learned my addr 172.21.15.131:0/3815571135 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cb6a00700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8101be0 0x7f1cb8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cb6000700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb8102d00 0x7f1cb81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cb6a00700 1 -- 172.21.15.131:0/3815571135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb8102d00 msgr2=0x7f1cb81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.492+0000 7f1cb6a00700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb8102d00 0x7f1cb81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.493+0000 7f1cb6a00700 1 -- 172.21.15.131:0/3815571135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb81036a0 msgr2=0x7f1cb819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.493+0000 7f1cb6a00700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb81036a0 0x7f1cb819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.493+0000 7f1cb6a00700 1 -- 172.21.15.131:0/3815571135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca80095f0 con 0x7f1cb8101be0 2024-05-10T05:32:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.493+0000 7f1cb6a00700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8101be0 0x7f1cb8197fb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80058e0 tx=0x7f1ca800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.493+0000 7f1caf400700 1 -- 172.21.15.131:0/3815571135 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca801d070 con 0x7f1cb8101be0 2024-05-10T05:32:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.493+0000 7f1cbd600700 1 -- 172.21.15.131:0/3815571135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb81a0040 con 0x7f1cb8101be0 2024-05-10T05:32:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.493+0000 7f1caf400700 1 -- 172.21.15.131:0/3815571135 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca8005050 con 0x7f1cb8101be0 2024-05-10T05:32:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.493+0000 7f1caf400700 1 -- 172.21.15.131:0/3815571135 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8004970 con 0x7f1cb8101be0 2024-05-10T05:32:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.493+0000 7f1cbd600700 1 -- 172.21.15.131:0/3815571135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb81a0460 con 0x7f1cb8101be0 2024-05-10T05:32:48.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.494+0000 7f1cbd600700 1 -- 172.21.15.131:0/3815571135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb8108df0 con 0x7f1cb8101be0 2024-05-10T05:32:48.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.497+0000 7f1caf400700 1 -- 172.21.15.131:0/3815571135 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1ca8005450 con 0x7f1cb8101be0 2024-05-10T05:32:48.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.498+0000 7f1caf400700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c9c0a11b0 0x7f1c9c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:48.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.498+0000 7f1caf400700 1 -- 172.21.15.131:0/3815571135 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1ca80cd9a0 con 0x7f1cb8101be0 2024-05-10T05:32:48.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.498+0000 7f1cb6000700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c9c0a11b0 0x7f1c9c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:48.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.498+0000 7f1cb6000700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c9c0a11b0 0x7f1c9c0a36c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8199050 tx=0x7f1ca4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:48.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.500+0000 7f1caf400700 1 -- 172.21.15.131:0/3815571135 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ca8097000 con 0x7f1cb8101be0 2024-05-10T05:32:48.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.753+0000 7f1cbd600700 1 -- 172.21.15.131:0/3815571135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cb80694b0 con 0x7f1cb8101be0 2024-05-10T05:32:48.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.755+0000 7f1caf400700 1 -- 172.21.15.131:0/3815571135 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f1ca809a620 con 0x7f1cb8101be0 2024-05-10T05:32:48.755 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:48.756 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:31:58.482427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:32:48.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 -- 172.21.15.131:0/3815571135 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c9c0a11b0 msgr2=0x7f1c9c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:48.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c9c0a11b0 0x7f1c9c0a36c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8199050 tx=0x7f1ca4009260 comp rx=0 tx=0).stop 2024-05-10T05:32:48.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 -- 172.21.15.131:0/3815571135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8101be0 msgr2=0x7f1cb8197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:48.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8101be0 0x7f1cb8197fb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80058e0 tx=0x7f1ca800f690 comp rx=0 tx=0).stop 2024-05-10T05:32:48.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 -- 172.21.15.131:0/3815571135 shutdown_connections 2024-05-10T05:32:48.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c9c0a11b0 0x7f1c9c0a36c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cb81036a0 0x7f1cb819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8101be0 0x7f1cb8197fb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 --2- 172.21.15.131:0/3815571135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb8102d00 0x7f1cb81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:48.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 -- 172.21.15.131:0/3815571135 >> 172.21.15.131:0/3815571135 conn(0x7f1cb80fd1d0 msgr2=0x7f1cb8105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:48.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 -- 172.21.15.131:0/3815571135 shutdown_connections 2024-05-10T05:32:48.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:48.757+0000 7f1cbd600700 1 -- 172.21.15.131:0/3815571135 wait complete. 2024-05-10T05:32:48.759 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-05-10T05:32:48.784 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:32:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.047+0000 7f3d56000700 1 -- 172.21.15.131:0/3475227828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d50104e30 msgr2=0x7f3d50107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.047+0000 7f3d56000700 1 --2- 172.21.15.131:0/3475227828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d50104e30 0x7f3d50107250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0098f0 tx=0x7f3d3c009c00 comp rx=0 tx=0).stop 2024-05-10T05:32:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.047+0000 7f3d56000700 1 -- 172.21.15.131:0/3475227828 shutdown_connections 2024-05-10T05:32:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.047+0000 7f3d56000700 1 --2- 172.21.15.131:0/3475227828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d50107790 0x7f3d50109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.047+0000 7f3d56000700 1 --2- 172.21.15.131:0/3475227828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d501024d0 0x7f3d501048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.047+0000 7f3d56000700 1 --2- 172.21.15.131:0/3475227828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d50104e30 0x7f3d50107250 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.047+0000 7f3d56000700 1 -- 172.21.15.131:0/3475227828 >> 172.21.15.131:0/3475227828 conn(0x7f3d500fbef0 msgr2=0x7f3d500fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.047+0000 7f3d56000700 1 -- 172.21.15.131:0/3475227828 shutdown_connections 2024-05-10T05:32:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.048+0000 7f3d56000700 1 -- 172.21.15.131:0/3475227828 wait complete. 2024-05-10T05:32:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d56000700 1 Processor -- start 2024-05-10T05:32:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d56000700 1 -- start start 2024-05-10T05:32:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d56000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d501024d0 0x7f3d50101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d56000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d50104e30 0x7f3d501003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d56000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d50107790 0x7f3d50100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d56000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d50100f80 con 0x7f3d501024d0 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d56000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d501010f0 con 0x7f3d50104e30 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d56000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d501012b0 con 0x7f3d50107790 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d54c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d501024d0 0x7f3d50101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d54c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d501024d0 0x7f3d50101d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50928/0 (socket says 172.21.15.131:50928) 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d54c00700 1 -- 172.21.15.131:0/3028530807 learned_addr learned my addr 172.21.15.131:0/3028530807 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d55600700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d50107790 0x7f3d50100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.049+0000 7f3d4fe00700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d50104e30 0x7f3d501003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d54c00700 1 -- 172.21.15.131:0/3028530807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d50107790 msgr2=0x7f3d50100920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d54c00700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d50107790 0x7f3d50100920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d54c00700 1 -- 172.21.15.131:0/3028530807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d50104e30 msgr2=0x7f3d501003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d54c00700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d50104e30 0x7f3d501003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d54c00700 1 -- 172.21.15.131:0/3028530807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c009520 con 0x7f3d501024d0 2024-05-10T05:32:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d4fe00700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d50104e30 0x7f3d501003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d54c00700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d501024d0 0x7f3d50101d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400e9f0 tx=0x7f3d4400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d4d600700 1 -- 172.21.15.131:0/3028530807 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d44009690 con 0x7f3d501024d0 2024-05-10T05:32:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d56000700 1 -- 172.21.15.131:0/3028530807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d501a1fc0 con 0x7f3d501024d0 2024-05-10T05:32:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d4d600700 1 -- 172.21.15.131:0/3028530807 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d4400ce80 con 0x7f3d501024d0 2024-05-10T05:32:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d4d600700 1 -- 172.21.15.131:0/3028530807 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d44018680 con 0x7f3d501024d0 2024-05-10T05:32:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.050+0000 7f3d56000700 1 -- 172.21.15.131:0/3028530807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d501a2470 con 0x7f3d501024d0 2024-05-10T05:32:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.051+0000 7f3d56000700 1 -- 172.21.15.131:0/3028530807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d500fdaa0 con 0x7f3d501024d0 2024-05-10T05:32:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.053+0000 7f3d4d600700 1 -- 172.21.15.131:0/3028530807 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3d44010430 con 0x7f3d501024d0 2024-05-10T05:32:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.054+0000 7f3d4d600700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d380a1260 0x7f3d380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.055+0000 7f3d4d600700 1 -- 172.21.15.131:0/3028530807 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3d44014070 con 0x7f3d501024d0 2024-05-10T05:32:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.055+0000 7f3d4fe00700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d380a1260 0x7f3d380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:49.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.055+0000 7f3d4fe00700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d380a1260 0x7f3d380a3770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0098c0 tx=0x7f3d3c019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:49.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.059+0000 7f3d4d600700 1 -- 172.21.15.131:0/3028530807 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3d44052d60 con 0x7f3d501024d0 2024-05-10T05:32:49.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:49 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1901717740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:32:49.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:49 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3815571135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:49.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.272+0000 7f3d56000700 1 -- 172.21.15.131:0/3028530807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d5010d250 con 0x7f3d501024d0 2024-05-10T05:32:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.274+0000 7f3d4d600700 1 -- 172.21.15.131:0/3028530807 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7f3d44099ed0 con 0x7f3d501024d0 2024-05-10T05:32:49.275 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:49.275 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:32:49.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.276+0000 7f3d56000700 1 -- 172.21.15.131:0/3028530807 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d380a1260 msgr2=0x7f3d380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:49.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.276+0000 7f3d56000700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d380a1260 0x7f3d380a3770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0098c0 tx=0x7f3d3c019040 comp rx=0 tx=0).stop 2024-05-10T05:32:49.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.276+0000 7f3d56000700 1 -- 172.21.15.131:0/3028530807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d501024d0 msgr2=0x7f3d50101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:49.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.276+0000 7f3d56000700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d501024d0 0x7f3d50101d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400e9f0 tx=0x7f3d4400ed00 comp rx=0 tx=0).stop 2024-05-10T05:32:49.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.276+0000 7f3d56000700 1 -- 172.21.15.131:0/3028530807 shutdown_connections 2024-05-10T05:32:49.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.276+0000 7f3d56000700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3d380a1260 0x7f3d380a3770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.276+0000 7f3d56000700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d501024d0 0x7f3d50101d40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.276+0000 7f3d56000700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d50107790 0x7f3d50100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.276+0000 7f3d56000700 1 --2- 172.21.15.131:0/3028530807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d50104e30 0x7f3d501003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.277+0000 7f3d56000700 1 -- 172.21.15.131:0/3028530807 >> 172.21.15.131:0/3028530807 conn(0x7f3d500fbef0 msgr2=0x7f3d50103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:49.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.277+0000 7f3d56000700 1 -- 172.21.15.131:0/3028530807 shutdown_connections 2024-05-10T05:32:49.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.277+0000 7f3d56000700 1 -- 172.21.15.131:0/3028530807 wait complete. 2024-05-10T05:32:49.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:49 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1901717740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:32:49.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:49 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3815571135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:49.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T05:31:49.481667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T05:31:50.960751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:32:49.306 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:32:49.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:49 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1901717740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:32:49.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:49 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3815571135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:49.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.576+0000 7fbc8ec00700 1 -- 172.21.15.131:0/2734710921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc88069090 msgr2=0x7fbc88104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:49.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.576+0000 7fbc8ec00700 1 --2- 172.21.15.131:0/2734710921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc88069090 0x7fbc88104520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009990 tx=0x7fbc7c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.576+0000 7fbc8ec00700 1 -- 172.21.15.131:0/2734710921 shutdown_connections 2024-05-10T05:32:49.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.576+0000 7fbc8ec00700 1 --2- 172.21.15.131:0/2734710921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc88069090 0x7fbc88104520 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.576+0000 7fbc8ec00700 1 --2- 172.21.15.131:0/2734710921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc88107680 0x7fbc88109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.576+0000 7fbc8ec00700 1 --2- 172.21.15.131:0/2734710921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc88104b10 0x7fbc88106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.576+0000 7fbc8ec00700 1 -- 172.21.15.131:0/2734710921 >> 172.21.15.131:0/2734710921 conn(0x7fbc880fbef0 msgr2=0x7fbc880fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:49.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.576+0000 7fbc8ec00700 1 -- 172.21.15.131:0/2734710921 shutdown_connections 2024-05-10T05:32:49.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.576+0000 7fbc8ec00700 1 -- 172.21.15.131:0/2734710921 wait complete. 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.577+0000 7fbc8ec00700 1 Processor -- start 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.577+0000 7fbc8ec00700 1 -- start start 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.577+0000 7fbc8ec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc88069090 0x7fbc88199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.577+0000 7fbc8ec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc88104b10 0x7fbc8819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.577+0000 7fbc8ec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc88107680 0x7fbc881a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.577+0000 7fbc8ec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc881a1a50 con 0x7fbc88104b10 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8ec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc881a1b90 con 0x7fbc88107680 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8ec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc881a1d20 con 0x7fbc88069090 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc88104b10 0x7fbc8819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc88104b10 0x7fbc8819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50944/0 (socket says 172.21.15.131:50944) 2024-05-10T05:32:49.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8ce00700 1 -- 172.21.15.131:0/511962736 learned_addr learned my addr 172.21.15.131:0/511962736 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8d800700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc88069090 0x7fbc88199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8e200700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc88107680 0x7fbc881a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8ce00700 1 -- 172.21.15.131:0/511962736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc88069090 msgr2=0x7fbc88199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8ce00700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc88069090 0x7fbc88199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8ce00700 1 -- 172.21.15.131:0/511962736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc88107680 msgr2=0x7fbc881a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8ce00700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc88107680 0x7fbc881a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.578+0000 7fbc8ce00700 1 -- 172.21.15.131:0/511962736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c0095f0 con 0x7fbc88104b10 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.579+0000 7fbc8ce00700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc88104b10 0x7fbc8819a4f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e9d0 tx=0x7fbc7400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.579+0000 7fbc86000700 1 -- 172.21.15.131:0/511962736 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74009790 con 0x7fbc88104b10 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.579+0000 7fbc86000700 1 -- 172.21.15.131:0/511962736 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc7400cbd0 con 0x7fbc88104b10 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.579+0000 7fbc8ec00700 1 -- 172.21.15.131:0/511962736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc881a1fa0 con 0x7fbc88104b10 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.579+0000 7fbc86000700 1 -- 172.21.15.131:0/511962736 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74018680 con 0x7fbc88104b10 2024-05-10T05:32:49.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.579+0000 7fbc8ec00700 1 -- 172.21.15.131:0/511962736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc881a2420 con 0x7fbc88104b10 2024-05-10T05:32:49.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.580+0000 7fbc8ec00700 1 -- 172.21.15.131:0/511962736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc88068110 con 0x7fbc88104b10 2024-05-10T05:32:49.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.582+0000 7fbc86000700 1 -- 172.21.15.131:0/511962736 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fbc7400cd40 con 0x7fbc88104b10 2024-05-10T05:32:49.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.583+0000 7fbc86000700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc700a1230 0x7fbc700a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:49.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.583+0000 7fbc86000700 1 -- 172.21.15.131:0/511962736 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbc74014070 con 0x7fbc88104b10 2024-05-10T05:32:49.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.584+0000 7fbc8d800700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc700a1230 0x7fbc700a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:49.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.584+0000 7fbc8d800700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc700a1230 0x7fbc700a3740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c006010 tx=0x7fbc7c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:49.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.588+0000 7fbc86000700 1 -- 172.21.15.131:0/511962736 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbc74052d60 con 0x7fbc88104b10 2024-05-10T05:32:49.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.795+0000 7fbc8ec00700 1 -- 172.21.15.131:0/511962736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc8810d3b0 con 0x7fbc88104b10 2024-05-10T05:32:49.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.796+0000 7fbc86000700 1 -- 172.21.15.131:0/511962736 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc74096310 con 0x7fbc88104b10 2024-05-10T05:32:49.796 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:32:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 -- 172.21.15.131:0/511962736 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc700a1230 msgr2=0x7fbc700a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc700a1230 0x7fbc700a3740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c006010 tx=0x7fbc7c01a040 comp rx=0 tx=0).stop 2024-05-10T05:32:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 -- 172.21.15.131:0/511962736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc88104b10 msgr2=0x7fbc8819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc88104b10 0x7fbc8819a4f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e9d0 tx=0x7fbc7400ece0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 -- 172.21.15.131:0/511962736 shutdown_connections 2024-05-10T05:32:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc700a1230 0x7fbc700a3740 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc88107680 0x7fbc881a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc88069090 0x7fbc88199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 --2- 172.21.15.131:0/511962736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc88104b10 0x7fbc8819a4f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 -- 172.21.15.131:0/511962736 >> 172.21.15.131:0/511962736 conn(0x7fbc880fbef0 msgr2=0x7fbc88100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 -- 172.21.15.131:0/511962736 shutdown_connections 2024-05-10T05:32:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:49.798+0000 7fbc8ec00700 1 -- 172.21.15.131:0/511962736 wait complete. 2024-05-10T05:32:49.829 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T05:32:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.103+0000 7f6abe400700 1 -- 172.21.15.131:0/252498550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab810e220 msgr2=0x7f6ab8110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.103+0000 7f6abe400700 1 --2- 172.21.15.131:0/252498550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab810e220 0x7f6ab8110640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000b0a0 tx=0x7f6ab000b3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.103+0000 7f6abe400700 1 -- 172.21.15.131:0/252498550 shutdown_connections 2024-05-10T05:32:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.103+0000 7f6abe400700 1 --2- 172.21.15.131:0/252498550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ab8075140 0x7f6ab8073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.103+0000 7f6abe400700 1 --2- 172.21.15.131:0/252498550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab810e220 0x7f6ab8110640 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.103+0000 7f6abe400700 1 --2- 172.21.15.131:0/252498550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ab8073b40 0x7f6ab8074010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.103+0000 7f6abe400700 1 -- 172.21.15.131:0/252498550 >> 172.21.15.131:0/252498550 conn(0x7f6ab80fd210 msgr2=0x7f6ab80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.103+0000 7f6abe400700 1 -- 172.21.15.131:0/252498550 shutdown_connections 2024-05-10T05:32:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.103+0000 7f6abe400700 1 -- 172.21.15.131:0/252498550 wait complete. 2024-05-10T05:32:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.104+0000 7f6abe400700 1 Processor -- start 2024-05-10T05:32:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.104+0000 7f6abe400700 1 -- start start 2024-05-10T05:32:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6abe400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8073b40 0x7f6ab819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6abe400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ab8075140 0x7f6ab819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6abe400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ab810e220 0x7f6ab81a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6abe400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81a60b0 con 0x7f6ab810e220 2024-05-10T05:32:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6abe400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81a6220 con 0x7f6ab8075140 2024-05-10T05:32:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6abe400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81a63e0 con 0x7f6ab8073b40 2024-05-10T05:32:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6ab7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8073b40 0x7f6ab819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:50.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6ab7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8073b40 0x7f6ab819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:50990/0 (socket says 172.21.15.131:50990) 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6ab7400700 1 -- 172.21.15.131:0/3539169439 learned_addr learned my addr 172.21.15.131:0/3539169439 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6ab6a00700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ab8075140 0x7f6ab819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6ab7e00700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ab810e220 0x7f6ab81a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6ab7400700 1 -- 172.21.15.131:0/3539169439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ab8075140 msgr2=0x7f6ab819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.105+0000 7f6ab7400700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ab8075140 0x7f6ab819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.106+0000 7f6ab7400700 1 -- 172.21.15.131:0/3539169439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ab810e220 msgr2=0x7f6ab81a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.106+0000 7f6ab7400700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ab810e220 0x7f6ab81a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.106+0000 7f6ab7400700 1 -- 172.21.15.131:0/3539169439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa8009520 con 0x7f6ab8073b40 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.106+0000 7f6ab6a00700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ab8075140 0x7f6ab819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.106+0000 7f6ab7400700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8073b40 0x7f6ab819e5d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800ea30 tx=0x7f6aa800ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.106+0000 7f6aafe00700 1 -- 172.21.15.131:0/3539169439 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa800cbd0 con 0x7f6ab8073b40 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.106+0000 7f6aafe00700 1 -- 172.21.15.131:0/3539169439 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aa800eed0 con 0x7f6ab8073b40 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.106+0000 7f6abe400700 1 -- 172.21.15.131:0/3539169439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab0009d00 con 0x7f6ab8073b40 2024-05-10T05:32:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.106+0000 7f6aafe00700 1 -- 172.21.15.131:0/3539169439 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8003d20 con 0x7f6ab8073b40 2024-05-10T05:32:50.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.106+0000 7f6abe400700 1 -- 172.21.15.131:0/3539169439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab81a6a40 con 0x7f6ab8073b40 2024-05-10T05:32:50.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.108+0000 7f6abe400700 1 -- 172.21.15.131:0/3539169439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab8103d70 con 0x7f6ab8073b40 2024-05-10T05:32:50.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.109+0000 7f6aafe00700 1 -- 172.21.15.131:0/3539169439 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6aa8003ee0 con 0x7f6ab8073b40 2024-05-10T05:32:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.110+0000 7f6aafe00700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a9c0a1260 0x7f6a9c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.111+0000 7f6aafe00700 1 -- 172.21.15.131:0/3539169439 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6aa8014070 con 0x7f6ab8073b40 2024-05-10T05:32:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.111+0000 7f6ab6a00700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a9c0a1260 0x7f6a9c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:50.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.111+0000 7f6ab6a00700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a9c0a1260 0x7f6a9c0a3770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6ab819f670 tx=0x7f6aa4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:50.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.115+0000 7f6aafe00700 1 -- 172.21.15.131:0/3539169439 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6aa80963c0 con 0x7f6ab8073b40 2024-05-10T05:32:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:50 smithi131 ceph-mon[37264]: pgmap v269: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:50 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3028530807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:50 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/511962736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:32:50.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:50 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:32:50.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:50 smithi157 ceph-mon[78427]: pgmap v269: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:50 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3028530807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:50 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/511962736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:32:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:50 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:32:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.352+0000 7f6abe400700 1 -- 172.21.15.131:0/3539169439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ab8113a20 con 0x7f6ab8073b40 2024-05-10T05:32:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.353+0000 7f6aafe00700 1 -- 172.21.15.131:0/3539169439 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6aa80999e0 con 0x7f6ab8073b40 2024-05-10T05:32:50.354 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-10T05:32:50.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 -- 172.21.15.131:0/3539169439 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a9c0a1260 msgr2=0x7f6a9c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:50.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a9c0a1260 0x7f6a9c0a3770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6ab819f670 tx=0x7f6aa4009260 comp rx=0 tx=0).stop 2024-05-10T05:32:50.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 -- 172.21.15.131:0/3539169439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8073b40 msgr2=0x7f6ab819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:50.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8073b40 0x7f6ab819e5d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800ea30 tx=0x7f6aa800ed40 comp rx=0 tx=0).stop 2024-05-10T05:32:50.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 -- 172.21.15.131:0/3539169439 shutdown_connections 2024-05-10T05:32:50.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a9c0a1260 0x7f6a9c0a3770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ab810e220 0x7f6ab81a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ab8073b40 0x7f6ab819e5d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 --2- 172.21.15.131:0/3539169439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ab8075140 0x7f6ab819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 -- 172.21.15.131:0/3539169439 >> 172.21.15.131:0/3539169439 conn(0x7f6ab80fd210 msgr2=0x7f6ab80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:50.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 -- 172.21.15.131:0/3539169439 shutdown_connections 2024-05-10T05:32:50.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.356+0000 7f6abe400700 1 -- 172.21.15.131:0/3539169439 wait complete. 2024-05-10T05:32:50.386 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T05:32:50.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:50 smithi142 ceph-mon[77667]: pgmap v269: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:50.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:50 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3028530807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:50.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:50 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/511962736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:32:50.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:50 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:32:50.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.669+0000 7fcac8600700 1 -- 172.21.15.131:0/4075882990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcac0073b40 msgr2=0x7fcac0074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:50.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.669+0000 7fcac8600700 1 --2- 172.21.15.131:0/4075882990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcac0073b40 0x7fcac0074010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009990 tx=0x7fcaac009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.669+0000 7fcac8600700 1 -- 172.21.15.131:0/4075882990 shutdown_connections 2024-05-10T05:32:50.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.669+0000 7fcac8600700 1 --2- 172.21.15.131:0/4075882990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac010e220 0x7fcac0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.669+0000 7fcac8600700 1 --2- 172.21.15.131:0/4075882990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac0075140 0x7fcac0073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.669+0000 7fcac8600700 1 --2- 172.21.15.131:0/4075882990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcac0073b40 0x7fcac0074010 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.669+0000 7fcac8600700 1 -- 172.21.15.131:0/4075882990 >> 172.21.15.131:0/4075882990 conn(0x7fcac00fd1d0 msgr2=0x7fcac00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:50.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.669+0000 7fcac8600700 1 -- 172.21.15.131:0/4075882990 shutdown_connections 2024-05-10T05:32:50.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.669+0000 7fcac8600700 1 -- 172.21.15.131:0/4075882990 wait complete. 2024-05-10T05:32:50.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.671+0000 7fcac8600700 1 Processor -- start 2024-05-10T05:32:50.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.671+0000 7fcac8600700 1 -- start start 2024-05-10T05:32:50.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.671+0000 7fcac8600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0073b40 0x7fcac019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:50.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.671+0000 7fcac8600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcac0075140 0x7fcac019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac8600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac010e220 0x7fcac01a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac8600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01a6020 con 0x7fcac0073b40 2024-05-10T05:32:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac8600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01a6190 con 0x7fcac0075140 2024-05-10T05:32:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac8600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01a6350 con 0x7fcac010e220 2024-05-10T05:32:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0073b40 0x7fcac019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:50.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac6600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac010e220 0x7fcac01a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:50.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0073b40 0x7fcac019e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50968/0 (socket says 172.21.15.131:50968) 2024-05-10T05:32:50.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcac0075140 0x7fcac019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:50.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac5c00700 1 -- 172.21.15.131:0/3481907291 learned_addr learned my addr 172.21.15.131:0/3481907291 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:50.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac6600700 1 -- 172.21.15.131:0/3481907291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcac0075140 msgr2=0x7fcac019ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:50.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac6600700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcac0075140 0x7fcac019ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac6600700 1 -- 172.21.15.131:0/3481907291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0073b40 msgr2=0x7fcac019e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:50.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac6600700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0073b40 0x7fcac019e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac6600700 1 -- 172.21.15.131:0/3481907291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac0095f0 con 0x7fcac010e220 2024-05-10T05:32:50.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac5c00700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0073b40 0x7fcac019e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:50.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.672+0000 7fcac6600700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac010e220 0x7fcac01a5a10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcab800efc0 tx=0x7fcab800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:50.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.673+0000 7fcabe000700 1 -- 172.21.15.131:0/3481907291 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8018070 con 0x7fcac010e220 2024-05-10T05:32:50.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.673+0000 7fcac8600700 1 -- 172.21.15.131:0/3481907291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac01a65d0 con 0x7fcac010e220 2024-05-10T05:32:50.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.673+0000 7fcabe000700 1 -- 172.21.15.131:0/3481907291 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcab800f040 con 0x7fcac010e220 2024-05-10T05:32:50.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.673+0000 7fcabe000700 1 -- 172.21.15.131:0/3481907291 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab800cd40 con 0x7fcac010e220 2024-05-10T05:32:50.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.673+0000 7fcac8600700 1 -- 172.21.15.131:0/3481907291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01a6a90 con 0x7fcac010e220 2024-05-10T05:32:50.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.674+0000 7fcac8600700 1 -- 172.21.15.131:0/3481907291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac0104480 con 0x7fcac010e220 2024-05-10T05:32:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.676+0000 7fcabe000700 1 -- 172.21.15.131:0/3481907291 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fcab8003a00 con 0x7fcac010e220 2024-05-10T05:32:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.677+0000 7fcabe000700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcaa80a1260 0x7fcaa80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.677+0000 7fcabe000700 1 -- 172.21.15.131:0/3481907291 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fcab80cc640 con 0x7fcac010e220 2024-05-10T05:32:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.678+0000 7fcac5c00700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcaa80a1260 0x7fcaa80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.678+0000 7fcac5c00700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcaa80a1260 0x7fcaa80a3770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcab4005950 tx=0x7fcab4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:50.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.682+0000 7fcabe000700 1 -- 172.21.15.131:0/3481907291 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcab8095ca0 con 0x7fcac010e220 2024-05-10T05:32:50.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.901+0000 7fcac8600700 1 -- 172.21.15.131:0/3481907291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcac0113c50 con 0x7fcac010e220 2024-05-10T05:32:50.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.902+0000 7fcabe000700 1 -- 172.21.15.131:0/3481907291 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcab801b320 con 0x7fcac010e220 2024-05-10T05:32:50.903 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-10T05:32:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 -- 172.21.15.131:0/3481907291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcaa80a1260 msgr2=0x7fcaa80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcaa80a1260 0x7fcaa80a3770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcab4005950 tx=0x7fcab4009260 comp rx=0 tx=0).stop 2024-05-10T05:32:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 -- 172.21.15.131:0/3481907291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac010e220 msgr2=0x7fcac01a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac010e220 0x7fcac01a5a10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcab800efc0 tx=0x7fcab800c3a0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 -- 172.21.15.131:0/3481907291 shutdown_connections 2024-05-10T05:32:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcaa80a1260 0x7fcaa80a3770 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcac0073b40 0x7fcac019e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac010e220 0x7fcac01a5a10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 --2- 172.21.15.131:0/3481907291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcac0075140 0x7fcac019ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:50.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 -- 172.21.15.131:0/3481907291 >> 172.21.15.131:0/3481907291 conn(0x7fcac00fd1d0 msgr2=0x7fcac01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:50.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 -- 172.21.15.131:0/3481907291 shutdown_connections 2024-05-10T05:32:50.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:50.905+0000 7fcac8600700 1 -- 172.21.15.131:0/3481907291 wait complete. 2024-05-10T05:32:50.938 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:32:51.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:51 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3539169439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:32:51.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:51 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3481907291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:32:51.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:51.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:51 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:51.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.229+0000 7f8302200700 1 -- 172.21.15.131:0/1868341790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82fc101be0 msgr2=0x7f82fc101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:51.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.229+0000 7f8302200700 1 --2- 172.21.15.131:0/1868341790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82fc101be0 0x7f82fc101ff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f82e80099c0 tx=0x7f82e8009cd0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.229+0000 7f8302200700 1 -- 172.21.15.131:0/1868341790 shutdown_connections 2024-05-10T05:32:51.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.229+0000 7f8302200700 1 --2- 172.21.15.131:0/1868341790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc1036a0 0x7f82fc109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.229+0000 7f8302200700 1 --2- 172.21.15.131:0/1868341790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82fc101be0 0x7f82fc101ff0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.229+0000 7f8302200700 1 --2- 172.21.15.131:0/1868341790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82fc102d00 0x7f82fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.229+0000 7f8302200700 1 -- 172.21.15.131:0/1868341790 >> 172.21.15.131:0/1868341790 conn(0x7f82fc0fd210 msgr2=0x7f82fc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:51.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.229+0000 7f8302200700 1 -- 172.21.15.131:0/1868341790 shutdown_connections 2024-05-10T05:32:51.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.229+0000 7f8302200700 1 -- 172.21.15.131:0/1868341790 wait complete. 2024-05-10T05:32:51.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.230+0000 7f8302200700 1 Processor -- start 2024-05-10T05:32:51.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.230+0000 7f8302200700 1 -- start start 2024-05-10T05:32:51.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.230+0000 7f8302200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82fc101be0 0x7f82fc074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:51.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.230+0000 7f8302200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82fc102d00 0x7f82fc073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:51.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.230+0000 7f8302200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc1036a0 0x7f82fc073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.230+0000 7f8302200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc074180 con 0x7f82fc1036a0 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.230+0000 7f8302200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0742f0 con 0x7f82fc101be0 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.230+0000 7f8302200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc1a2080 con 0x7f82fc102d00 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82fc101be0 0x7f82fc074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82faa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82fc102d00 0x7f82fc073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82fc101be0 0x7f82fc074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:52950/0 (socket says 172.21.15.131:52950) 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fb400700 1 -- 172.21.15.131:0/3010139640 learned_addr learned my addr 172.21.15.131:0/3010139640 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fbe00700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc1036a0 0x7f82fc073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fb400700 1 -- 172.21.15.131:0/3010139640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82fc102d00 msgr2=0x7f82fc073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fb400700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82fc102d00 0x7f82fc073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fb400700 1 -- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc1036a0 msgr2=0x7f82fc073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fb400700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc1036a0 0x7f82fc073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fb400700 1 -- 172.21.15.131:0/3010139640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80095f0 con 0x7f82fc101be0 2024-05-10T05:32:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fbe00700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc1036a0 0x7f82fc073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82faa00700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82fc102d00 0x7f82fc073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.231+0000 7f82fb400700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82fc101be0 0x7f82fc074ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009990 tx=0x7f82e800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.232+0000 7f82f3e00700 1 -- 172.21.15.131:0/3010139640 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e801d070 con 0x7f82fc101be0 2024-05-10T05:32:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.232+0000 7f8302200700 1 -- 172.21.15.131:0/3010139640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc1a2280 con 0x7f82fc101be0 2024-05-10T05:32:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.232+0000 7f82f3e00700 1 -- 172.21.15.131:0/3010139640 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e80054f0 con 0x7f82fc101be0 2024-05-10T05:32:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.232+0000 7f82f3e00700 1 -- 172.21.15.131:0/3010139640 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8004a90 con 0x7f82fc101be0 2024-05-10T05:32:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.232+0000 7f8302200700 1 -- 172.21.15.131:0/3010139640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc1a26b0 con 0x7f82fc101be0 2024-05-10T05:32:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.233+0000 7f8302200700 1 -- 172.21.15.131:0/3010139640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc108df0 con 0x7f82fc101be0 2024-05-10T05:32:51.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.236+0000 7f82f3e00700 1 -- 172.21.15.131:0/3010139640 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f82e800fb80 con 0x7f82fc101be0 2024-05-10T05:32:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.237+0000 7f82f3e00700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f82e00a11b0 0x7f82e00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.237+0000 7f82f3e00700 1 -- 172.21.15.131:0/3010139640 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f82e80cdb10 con 0x7f82fc101be0 2024-05-10T05:32:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.237+0000 7f82faa00700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f82e00a11b0 0x7f82e00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.238+0000 7f82faa00700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f82e00a11b0 0x7f82e00a36c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82fc074860 tx=0x7f82ec009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:51.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.243+0000 7f82f3e00700 1 -- 172.21.15.131:0/3010139640 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f82e80971d0 con 0x7f82fc101be0 2024-05-10T05:32:51.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:51 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3539169439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:32:51.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:51 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3481907291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:32:51.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:51.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:51 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:51.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:51 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3539169439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:32:51.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:51 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3481907291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:32:51.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:51.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:51 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:51.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.518+0000 7f8302200700 1 -- 172.21.15.131:0/3010139640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82fc0694b0 con 0x7f82fc101be0 2024-05-10T05:32:51.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.519+0000 7f82f3e00700 1 -- 172.21.15.131:0/3010139640 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f82e809a7f0 con 0x7f82fc101be0 2024-05-10T05:32:51.520 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:51.520 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:31:58.482427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.521+0000 7f8302200700 1 -- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f82e00a11b0 msgr2=0x7f82e00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.521+0000 7f8302200700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f82e00a11b0 0x7f82e00a36c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82fc074860 tx=0x7f82ec009260 comp rx=0 tx=0).stop 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.521+0000 7f8302200700 1 -- 172.21.15.131:0/3010139640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82fc101be0 msgr2=0x7f82fc074ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.521+0000 7f8302200700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82fc101be0 0x7f82fc074ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009990 tx=0x7f82e800f690 comp rx=0 tx=0).stop 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.522+0000 7f8302200700 1 -- 172.21.15.131:0/3010139640 shutdown_connections 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.522+0000 7f8302200700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f82e00a11b0 0x7f82e00a36c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.522+0000 7f8302200700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82fc1036a0 0x7f82fc073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.522+0000 7f8302200700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82fc101be0 0x7f82fc074ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.522+0000 7f8302200700 1 --2- 172.21.15.131:0/3010139640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82fc102d00 0x7f82fc073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.522+0000 7f8302200700 1 -- 172.21.15.131:0/3010139640 >> 172.21.15.131:0/3010139640 conn(0x7f82fc0fd210 msgr2=0x7f82fc105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.522+0000 7f8302200700 1 -- 172.21.15.131:0/3010139640 shutdown_connections 2024-05-10T05:32:51.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.522+0000 7f8302200700 1 -- 172.21.15.131:0/3010139640 wait complete. 2024-05-10T05:32:51.524 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-05-10T05:32:51.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:32:51.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.866+0000 7f4f0f600700 1 -- 172.21.15.131:0/1640961847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f081036a0 msgr2=0x7f4f08109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:51.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.866+0000 7f4f0f600700 1 --2- 172.21.15.131:0/1640961847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f081036a0 0x7f4f08109f50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000b0d0 tx=0x7f4f0000b3e0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.866+0000 7f4f0f600700 1 -- 172.21.15.131:0/1640961847 shutdown_connections 2024-05-10T05:32:51.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.866+0000 7f4f0f600700 1 --2- 172.21.15.131:0/1640961847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f08101be0 0x7f4f08101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.867+0000 7f4f0f600700 1 --2- 172.21.15.131:0/1640961847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f081036a0 0x7f4f08109f50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.867+0000 7f4f0f600700 1 --2- 172.21.15.131:0/1640961847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f08102d00 0x7f4f08103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.867+0000 7f4f0f600700 1 -- 172.21.15.131:0/1640961847 >> 172.21.15.131:0/1640961847 conn(0x7f4f080fd1d0 msgr2=0x7f4f080ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:51.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.867+0000 7f4f0f600700 1 -- 172.21.15.131:0/1640961847 shutdown_connections 2024-05-10T05:32:51.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.867+0000 7f4f0f600700 1 -- 172.21.15.131:0/1640961847 wait complete. 2024-05-10T05:32:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.868+0000 7f4f0f600700 1 Processor -- start 2024-05-10T05:32:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.868+0000 7f4f0f600700 1 -- start start 2024-05-10T05:32:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.868+0000 7f4f0f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f08101be0 0x7f4f081980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.868+0000 7f4f0f600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08102d00 0x7f4f081985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.868+0000 7f4f0f600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f081036a0 0x7f4f0819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.868+0000 7f4f0f600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0819fb80 con 0x7f4f081036a0 2024-05-10T05:32:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.868+0000 7f4f0f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0819fcf0 con 0x7f4f08101be0 2024-05-10T05:32:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.868+0000 7f4f0f600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0819feb0 con 0x7f4f08102d00 2024-05-10T05:32:51.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f08101be0 0x7f4f081980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f08101be0 0x7f4f081980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:52972/0 (socket says 172.21.15.131:52972) 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0cc00700 1 -- 172.21.15.131:0/765763680 learned_addr learned my addr 172.21.15.131:0/765763680 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0d600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f081036a0 0x7f4f0819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f07e00700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08102d00 0x7f4f081985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0cc00700 1 -- 172.21.15.131:0/765763680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08102d00 msgr2=0x7f4f081985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0cc00700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08102d00 0x7f4f081985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0cc00700 1 -- 172.21.15.131:0/765763680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f081036a0 msgr2=0x7f4f0819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0cc00700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f081036a0 0x7f4f0819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0cc00700 1 -- 172.21.15.131:0/765763680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc009520 con 0x7f4f08101be0 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0d600700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f081036a0 0x7f4f0819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:51.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.869+0000 7f4f0cc00700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f08101be0 0x7f4f081980a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00ea10 tx=0x7f4efc00ed20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:51.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.870+0000 7f4f05600700 1 -- 172.21.15.131:0/765763680 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00cbd0 con 0x7f4f08101be0 2024-05-10T05:32:51.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.870+0000 7f4f05600700 1 -- 172.21.15.131:0/765763680 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4efc00eeb0 con 0x7f4f08101be0 2024-05-10T05:32:51.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.870+0000 7f4f0f600700 1 -- 172.21.15.131:0/765763680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f00009d00 con 0x7f4f08101be0 2024-05-10T05:32:51.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.870+0000 7f4f05600700 1 -- 172.21.15.131:0/765763680 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc018680 con 0x7f4f08101be0 2024-05-10T05:32:51.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.870+0000 7f4f0f600700 1 -- 172.21.15.131:0/765763680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f081a0560 con 0x7f4f08101be0 2024-05-10T05:32:51.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.871+0000 7f4f0f600700 1 -- 172.21.15.131:0/765763680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f08108df0 con 0x7f4f08101be0 2024-05-10T05:32:51.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.873+0000 7f4f05600700 1 -- 172.21.15.131:0/765763680 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4efc010430 con 0x7f4f08101be0 2024-05-10T05:32:51.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.874+0000 7f4f05600700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4ef00a1230 0x7f4ef00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:51.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.875+0000 7f4f05600700 1 -- 172.21.15.131:0/765763680 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4efc014070 con 0x7f4f08101be0 2024-05-10T05:32:51.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.875+0000 7f4f07e00700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4ef00a1230 0x7f4ef00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:51.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.875+0000 7f4f07e00700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4ef00a1230 0x7f4ef00a3740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4f08199140 tx=0x7f4ef4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:51.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:51.879+0000 7f4f05600700 1 -- 172.21.15.131:0/765763680 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4efc0963a0 con 0x7f4f08101be0 2024-05-10T05:32:52.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.106+0000 7f4f0f600700 1 -- 172.21.15.131:0/765763680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f080694b0 con 0x7f4f08101be0 2024-05-10T05:32:52.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.108+0000 7f4f05600700 1 -- 172.21.15.131:0/765763680 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7f4efc0999c0 con 0x7f4f08101be0 2024-05-10T05:32:52.110 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:52.110 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:32:52.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 -- 172.21.15.131:0/765763680 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4ef00a1230 msgr2=0x7f4ef00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:52.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4ef00a1230 0x7f4ef00a3740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4f08199140 tx=0x7f4ef4009260 comp rx=0 tx=0).stop 2024-05-10T05:32:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 -- 172.21.15.131:0/765763680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f08101be0 msgr2=0x7f4f081980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f08101be0 0x7f4f081980a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00ea10 tx=0x7f4efc00ed20 comp rx=0 tx=0).stop 2024-05-10T05:32:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 -- 172.21.15.131:0/765763680 shutdown_connections 2024-05-10T05:32:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4ef00a1230 0x7f4ef00a3740 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f081036a0 0x7f4f0819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f08101be0 0x7f4f081980a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 --2- 172.21.15.131:0/765763680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f08102d00 0x7f4f081985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 -- 172.21.15.131:0/765763680 >> 172.21.15.131:0/765763680 conn(0x7f4f080fd1d0 msgr2=0x7f4f08105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 -- 172.21.15.131:0/765763680 shutdown_connections 2024-05-10T05:32:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.113+0000 7f4f0f600700 1 -- 172.21.15.131:0/765763680 wait complete. 2024-05-10T05:32:52.155 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:32:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:52 smithi157 ceph-mon[78427]: pgmap v270: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:32:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:52 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:52 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:52 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3010139640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:52 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:52 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:52 smithi142 ceph-mon[77667]: pgmap v270: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:32:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:52 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:52 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:52 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3010139640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:52.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:52 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:52 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.424+0000 7fba76400700 1 -- 172.21.15.131:0/544821323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba70073b40 msgr2=0x7fba70074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:52.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.424+0000 7fba76400700 1 --2- 172.21.15.131:0/544821323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba70073b40 0x7fba70074010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009990 tx=0x7fba5c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.425+0000 7fba76400700 1 -- 172.21.15.131:0/544821323 shutdown_connections 2024-05-10T05:32:52.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.425+0000 7fba76400700 1 --2- 172.21.15.131:0/544821323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba7010e220 0x7fba70110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.425+0000 7fba76400700 1 --2- 172.21.15.131:0/544821323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba70075140 0x7fba70073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.425+0000 7fba76400700 1 --2- 172.21.15.131:0/544821323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba70073b40 0x7fba70074010 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.425+0000 7fba76400700 1 -- 172.21.15.131:0/544821323 >> 172.21.15.131:0/544821323 conn(0x7fba700fd210 msgr2=0x7fba700ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:52.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.425+0000 7fba76400700 1 -- 172.21.15.131:0/544821323 shutdown_connections 2024-05-10T05:32:52.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.425+0000 7fba76400700 1 -- 172.21.15.131:0/544821323 wait complete. 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba76400700 1 Processor -- start 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba76400700 1 -- start start 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba76400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba70073b40 0x7fba7019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba76400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba70075140 0x7fba7019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba76400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7010e220 0x7fba701a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba76400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba701a6050 con 0x7fba70073b40 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba76400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba701a61c0 con 0x7fba7010e220 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba76400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba701a6380 con 0x7fba70075140 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba6f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba70073b40 0x7fba7019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba6f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba70073b40 0x7fba7019e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51022/0 (socket says 172.21.15.131:51022) 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.426+0000 7fba6f400700 1 -- 172.21.15.131:0/3277301979 learned_addr learned my addr 172.21.15.131:0/3277301979 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.427+0000 7fba6ea00700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba70075140 0x7fba7019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.427+0000 7fba6fe00700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7010e220 0x7fba701a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.427+0000 7fba6f400700 1 -- 172.21.15.131:0/3277301979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba70075140 msgr2=0x7fba7019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.427+0000 7fba6f400700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba70075140 0x7fba7019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.427+0000 7fba6f400700 1 -- 172.21.15.131:0/3277301979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7010e220 msgr2=0x7fba701a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.427+0000 7fba6f400700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7010e220 0x7fba701a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.427+0000 7fba6f400700 1 -- 172.21.15.131:0/3277301979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba60009520 con 0x7fba70073b40 2024-05-10T05:32:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.427+0000 7fba6f400700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba70073b40 0x7fba7019e570 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba6000ea30 tx=0x7fba6000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:52.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.427+0000 7fba67e00700 1 -- 172.21.15.131:0/3277301979 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000cbd0 con 0x7fba70073b40 2024-05-10T05:32:52.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.428+0000 7fba67e00700 1 -- 172.21.15.131:0/3277301979 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba6000eed0 con 0x7fba70073b40 2024-05-10T05:32:52.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.428+0000 7fba76400700 1 -- 172.21.15.131:0/3277301979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5c0095f0 con 0x7fba70073b40 2024-05-10T05:32:52.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.428+0000 7fba67e00700 1 -- 172.21.15.131:0/3277301979 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60003d50 con 0x7fba70073b40 2024-05-10T05:32:52.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.428+0000 7fba76400700 1 -- 172.21.15.131:0/3277301979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba701a6a30 con 0x7fba70073b40 2024-05-10T05:32:52.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.429+0000 7fba76400700 1 -- 172.21.15.131:0/3277301979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba70104480 con 0x7fba70073b40 2024-05-10T05:32:52.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.431+0000 7fba67e00700 1 -- 172.21.15.131:0/3277301979 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fba6000cd30 con 0x7fba70073b40 2024-05-10T05:32:52.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.432+0000 7fba67e00700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fba540a1230 0x7fba540a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:52.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.433+0000 7fba67e00700 1 -- 172.21.15.131:0/3277301979 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fba60014070 con 0x7fba70073b40 2024-05-10T05:32:52.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.433+0000 7fba6ea00700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fba540a1230 0x7fba540a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:52.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.433+0000 7fba6ea00700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fba540a1230 0x7fba540a3740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00b2e0 tx=0x7fba5c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:52.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.435+0000 7fba67e00700 1 -- 172.21.15.131:0/3277301979 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fba600964e0 con 0x7fba70073b40 2024-05-10T05:32:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:52 smithi131 ceph-mon[37264]: pgmap v270: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:32:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:52 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:52 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:52 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3010139640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:52 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:52 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.704+0000 7fba76400700 1 -- 172.21.15.131:0/3277301979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba70113c50 con 0x7fba70073b40 2024-05-10T05:32:52.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.705+0000 7fba67e00700 1 -- 172.21.15.131:0/3277301979 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fba60099b00 con 0x7fba70073b40 2024-05-10T05:32:52.706 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:32:52.706 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:31:58.482427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:32:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 -- 172.21.15.131:0/3277301979 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fba540a1230 msgr2=0x7fba540a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fba540a1230 0x7fba540a3740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00b2e0 tx=0x7fba5c01a040 comp rx=0 tx=0).stop 2024-05-10T05:32:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 -- 172.21.15.131:0/3277301979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba70073b40 msgr2=0x7fba7019e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba70073b40 0x7fba7019e570 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba6000ea30 tx=0x7fba6000ed40 comp rx=0 tx=0).stop 2024-05-10T05:32:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 -- 172.21.15.131:0/3277301979 shutdown_connections 2024-05-10T05:32:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fba540a1230 0x7fba540a3740 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba70073b40 0x7fba7019e570 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7010e220 0x7fba701a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 --2- 172.21.15.131:0/3277301979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba70075140 0x7fba7019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 -- 172.21.15.131:0/3277301979 >> 172.21.15.131:0/3277301979 conn(0x7fba700fd210 msgr2=0x7fba701015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:52.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 -- 172.21.15.131:0/3277301979 shutdown_connections 2024-05-10T05:32:52.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:52.708+0000 7fba76400700 1 -- 172.21.15.131:0/3277301979 wait complete. 2024-05-10T05:32:52.710 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-05-10T05:32:52.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T05:31:51.547495+0000', 'modified': '2024-05-10T05:31:58.482427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24581, 'mds_1': 24571, 'mds_2': 24569, 'mds_3': 24565, 'mds_4': 24541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24476': {'gid': 24476, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6833/3035299290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 3035299290}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 3035299290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'd', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6837/3356790716', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 3356790716}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 3356790716}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/3168061272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 3168061272}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 3168061272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/72787128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 72787128}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 72787128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/2645029376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 2645029376}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2645029376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.157:6837/707351222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 707351222}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 707351222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.157:6839/1794088540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 1794088540}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 1794088540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24569': {'gid': 24569, 'name': 'a', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.131:6841/2396760252', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6840', 'nonce': 2396760252}, {'type': 'v1', 'addr': '172.21.15.131:6841', 'nonce': 2396760252}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24571': {'gid': 24571, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.131:6839/2980869223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 2980869223}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 2980869223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6839/2348955278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 2348955278}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 2348955278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T05:32:52.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-10T05:32:52.736 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T05:32:53.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.003+0000 7f3f08600700 1 -- 172.21.15.131:0/1429490989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f00101be0 msgr2=0x7f3f00101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.003+0000 7f3f08600700 1 --2- 172.21.15.131:0/1429490989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f00101be0 0x7f3f00101ff0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009920 tx=0x7f3ef4009c30 comp rx=0 tx=0).stop 2024-05-10T05:32:53.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.003+0000 7f3f08600700 1 -- 172.21.15.131:0/1429490989 shutdown_connections 2024-05-10T05:32:53.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.003+0000 7f3f08600700 1 --2- 172.21.15.131:0/1429490989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f001036a0 0x7f3f00109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.003+0000 7f3f08600700 1 --2- 172.21.15.131:0/1429490989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f00101be0 0x7f3f00101ff0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.003+0000 7f3f08600700 1 --2- 172.21.15.131:0/1429490989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f00102d00 0x7f3f00103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.003+0000 7f3f08600700 1 -- 172.21.15.131:0/1429490989 >> 172.21.15.131:0/1429490989 conn(0x7f3f000fd210 msgr2=0x7f3f000ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.003+0000 7f3f08600700 1 -- 172.21.15.131:0/1429490989 shutdown_connections 2024-05-10T05:32:53.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.004+0000 7f3f08600700 1 -- 172.21.15.131:0/1429490989 wait complete. 2024-05-10T05:32:53.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.004+0000 7f3f08600700 1 Processor -- start 2024-05-10T05:32:53.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f08600700 1 -- start start 2024-05-10T05:32:53.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f08600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f00101be0 0x7f3f0019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f08600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f00102d00 0x7f3f0019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f08600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f001036a0 0x7f3f001a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f08600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f001a1c60 con 0x7f3f00101be0 2024-05-10T05:32:53.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f08600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f001a1dd0 con 0x7f3f00102d00 2024-05-10T05:32:53.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f08600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f001a1f90 con 0x7f3f001036a0 2024-05-10T05:32:53.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f05c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f00101be0 0x7f3f0019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f05c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f00101be0 0x7f3f0019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53796/0 (socket says 172.21.15.131:53796) 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f05c00700 1 -- 172.21.15.131:0/110435921 learned_addr learned my addr 172.21.15.131:0/110435921 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f06600700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f001036a0 0x7f3f001a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.005+0000 7f3f05200700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f00102d00 0x7f3f0019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3f05c00700 1 -- 172.21.15.131:0/110435921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f001036a0 msgr2=0x7f3f001a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3f05c00700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f001036a0 0x7f3f001a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3f05c00700 1 -- 172.21.15.131:0/110435921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f00102d00 msgr2=0x7f3f0019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3f05c00700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f00102d00 0x7f3f0019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3f05c00700 1 -- 172.21.15.131:0/110435921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8009d00 con 0x7f3f00101be0 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3f05c00700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f00101be0 0x7f3f0019a180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4000c00 tx=0x7f3ef400f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3efe000700 1 -- 172.21.15.131:0/110435921 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef401c070 con 0x7f3f00101be0 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3efe000700 1 -- 172.21.15.131:0/110435921 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef400bbe0 con 0x7f3f00101be0 2024-05-10T05:32:53.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3efe000700 1 -- 172.21.15.131:0/110435921 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40038e0 con 0x7f3f00101be0 2024-05-10T05:32:53.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3f08600700 1 -- 172.21.15.131:0/110435921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef4009520 con 0x7f3f00101be0 2024-05-10T05:32:53.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.006+0000 7f3f08600700 1 -- 172.21.15.131:0/110435921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f001a2450 con 0x7f3f00101be0 2024-05-10T05:32:53.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.009+0000 7f3efe000700 1 -- 172.21.15.131:0/110435921 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3ef4020030 con 0x7f3f00101be0 2024-05-10T05:32:53.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.011+0000 7f3efe000700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3ee80a12e0 0x7f3ee80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.011+0000 7f3efe000700 1 -- 172.21.15.131:0/110435921 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3ef40cd980 con 0x7f3f00101be0 2024-05-10T05:32:53.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.011+0000 7f3f20495b80 1 Processor -- start 2024-05-10T05:32:53.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.011+0000 7f3f05200700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3ee80a12e0 0x7f3ee80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.011+0000 7f3f20495b80 1 -- start start 2024-05-10T05:32:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.011+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b686f650 0x5602b686faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.011+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6870930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.011+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b6870e70 0x5602b6877770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.011+0000 7f3f20495b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b6796100 con 0x5602b6870520 2024-05-10T05:32:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f20495b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b6793ac0 con 0x5602b6870e70 2024-05-10T05:32:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f20495b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b678f3b0 con 0x5602b686f650 2024-05-10T05:32:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f05200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6870930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f05200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6870930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53812/0 (socket says 172.21.15.131:53812) 2024-05-10T05:32:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f05200700 1 -- 172.21.15.131:0/3055089130 learned_addr learned my addr 172.21.15.131:0/3055089130 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f06600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b6870e70 0x5602b6877770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f05c00700 1 --2- 172.21.15.131:0/3055089130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b686f650 0x5602b686faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f05200700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3ee80a12e0 0x7f3ee80a37f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f0019b220 tx=0x7f3eec009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f05c00700 1 -- 172.21.15.131:0/3055089130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b6870e70 msgr2=0x5602b6877770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f05c00700 1 --2- 172.21.15.131:0/3055089130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b6870e70 0x5602b6877770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f05c00700 1 -- 172.21.15.131:0/3055089130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 msgr2=0x5602b6870930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f05c00700 1 --2- 172.21.15.131:0/3055089130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6870930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.012+0000 7f3f05c00700 1 -- 172.21.15.131:0/3055089130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602b6610360 con 0x5602b686f650 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.013+0000 7f3f05c00700 1 --2- 172.21.15.131:0/3055089130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b686f650 0x5602b686faa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40cb2d0 tx=0x7f3ef4005530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12e5a12777d820f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.013+0000 7f3ef2a00700 1 -- 172.21.15.131:0/3055089130 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40a3090 con 0x5602b686f650 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.013+0000 7f3ef2a00700 1 -- 172.21.15.131:0/3055089130 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef40cb520 con 0x5602b686f650 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.013+0000 7f3ef2a00700 1 -- 172.21.15.131:0/3055089130 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef409af00 con 0x5602b686f650 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.014+0000 7f3f20495b80 1 -- 172.21.15.131:0/3055089130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b686f650 msgr2=0x5602b686faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.014+0000 7f3f20495b80 1 --2- 172.21.15.131:0/3055089130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b686f650 0x5602b686faa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40cb2d0 tx=0x7f3ef4005530 comp rx=0 tx=0).stop 2024-05-10T05:32:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.014+0000 7f3f20495b80 1 -- 172.21.15.131:0/3055089130 shutdown_connections 2024-05-10T05:32:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.014+0000 7f3f20495b80 1 --2- 172.21.15.131:0/3055089130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b6870e70 0x5602b6877770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.014+0000 7f3f20495b80 1 --2- 172.21.15.131:0/3055089130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b686f650 0x5602b686faa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.014+0000 7f3f20495b80 1 --2- 172.21.15.131:0/3055089130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6870930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.014+0000 7f3f20495b80 1 -- 172.21.15.131:0/3055089130 >> 172.21.15.131:0/3055089130 conn(0x5602b67d1a10 msgr2=0x5602b6869f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.014+0000 7f3f20495b80 1 -- 172.21.15.131:0/3055089130 shutdown_connections 2024-05-10T05:32:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.014+0000 7f3f20495b80 1 -- 172.21.15.131:0/3055089130 wait complete. 2024-05-10T05:32:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.014+0000 7f3f20495b80 1 -- start start 2024-05-10T05:32:53.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f20495b80 1 Processor -- start 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b686f650 0x5602b68792e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b6870520 0x5602b6879820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b6870e70 0x5602b6880780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f20495b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b672b510 con 0x5602b686f650 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f20495b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b672b1b0 con 0x5602b6870e70 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f20495b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b6612650 con 0x5602b6870520 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f20495b80 10 client.0 _ll_register_callbacks cb 0x5602b6734ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f05200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b6870520 0x5602b6879820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f05c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b686f650 0x5602b68792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f05200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b6870520 0x5602b6879820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59022/0 (socket says 172.21.15.131:59022) 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.015+0000 7f3f05200700 1 -- 172.21.15.131:0/303606497 learned_addr learned my addr 172.21.15.131:0/303606497 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3f06600700 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b6870e70 0x5602b6880780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3f05200700 1 -- 172.21.15.131:0/303606497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b6870e70 msgr2=0x5602b6880780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3f05200700 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b6870e70 0x5602b6880780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3f05200700 1 -- 172.21.15.131:0/303606497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b686f650 msgr2=0x5602b68792e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3f05200700 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b686f650 0x5602b68792e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3f05200700 1 -- 172.21.15.131:0/303606497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f001a2450 con 0x5602b6870520 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3f05c00700 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b686f650 0x5602b68792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3f05200700 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b6870520 0x5602b6879820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3eec012900 tx=0x7f3eec0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d544642e250bea8b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3ee7400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3ee7400700 1 -- 172.21.15.131:0/303606497 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec020940 con 0x5602b6870520 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3f20495b80 1 -- 172.21.15.131:0/303606497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602b66e8220 con 0x5602b6870520 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.016+0000 7f3ee7400700 1 -- 172.21.15.131:0/303606497 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3eec018070 con 0x5602b6870520 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.017+0000 7f3ee7400700 1 -- 172.21.15.131:0/303606497 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eec020aa0 con 0x5602b6870520 2024-05-10T05:32:53.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.017+0000 7f3f20495b80 10 client.24997 fetch_fsmap learned FSMap version 26 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.017+0000 7f3f20495b80 1 -- 172.21.15.131:0/303606497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5602b65bcb10 con 0x5602b6870520 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.017+0000 7f3ee7400700 1 -- 172.21.15.131:0/303606497 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602b66e8220 con 0x5602b6870520 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.017+0000 7f3ee7400700 1 -- 172.21.15.131:0/303606497 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3eec033400 con 0x5602b6870520 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.017+0000 7f3f20495b80 10 client.24997 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.017+0000 7f3f20495b80 10 client.24997 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.018+0000 7f3f20495b80 20 client.24997 populate_metadata read hostname 'smithi131' 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.018+0000 7f3f20495b80 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5602b6884980 0x5602b68896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.018+0000 7f3f20495b80 4 client.24997 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.018+0000 7f3f20495b80 1 -- 172.21.15.131:0/303606497 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602b683e730 con 0x5602b6884980 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.018+0000 7f3f05c00700 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5602b6884980 0x5602b68896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.018+0000 7f3ee7400700 10 client.24997 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.018+0000 7f3f05c00700 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5602b6884980 0x5602b68896b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=39b5358f68caa879 server_cookie=65d3fca369906275 in_seq=0 out_seq=0 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.019+0000 7f3ee7400700 1 -- 172.21.15.131:0/303606497 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602b683e730 con 0x5602b6884980 2024-05-10T05:32:53.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.019+0000 7f3ee7400700 10 client.24997 handle_command_reply: tid=0 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.019+0000 7f3f20495b80 1 client.24997 shutdown 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 1 -- 172.21.15.131:0/303606497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b6870520 msgr2=0x5602b6879820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b6870520 0x5602b6879820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3eec012900 tx=0x7f3eec0047c0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 1 -- 172.21.15.131:0/303606497 shutdown_connections 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5602b6884980 0x5602b68896b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b686f650 0x5602b68792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3ee7400700 0 client.24997 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b6870e70 0x5602b6880780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 1 --2- 172.21.15.131:0/303606497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b6870520 0x5602b6879820 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 1 -- 172.21.15.131:0/303606497 >> 172.21.15.131:0/303606497 conn(0x5602b67d1a10 msgr2=0x5602b6869f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 1 -- 172.21.15.131:0/303606497 shutdown_connections 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 1 -- 172.21.15.131:0/303606497 wait complete. 2024-05-10T05:32:53.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.020+0000 7f3f20495b80 20 client.24997 trim_cache size 0 max 16384 2024-05-10T05:32:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.039+0000 7f3f20495b80 1 Processor -- start 2024-05-10T05:32:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.039+0000 7f3f20495b80 1 -- start start 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b686f650 0x5602b687ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6878eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f20495b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b6788150 con 0x5602b6870520 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f20495b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b678c010 con 0x5602b686f650 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f20495b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b672e5a0 con 0x5602b67d1a10 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f06600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6878eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f05c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f06600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6878eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53822/0 (socket says 172.21.15.131:53822) 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f05c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59038/0 (socket says 172.21.15.131:59038) 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f06600700 1 -- 172.21.15.131:0/875563190 learned_addr learned my addr 172.21.15.131:0/875563190 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f05200700 1 --2- 172.21.15.131:0/875563190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b686f650 0x5602b687ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f05c00700 1 -- 172.21.15.131:0/875563190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b686f650 msgr2=0x5602b687ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f05c00700 1 --2- 172.21.15.131:0/875563190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b686f650 0x5602b687ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f05c00700 1 -- 172.21.15.131:0/875563190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 msgr2=0x5602b6878eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f05c00700 1 --2- 172.21.15.131:0/875563190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6878eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f05c00700 1 -- 172.21.15.131:0/875563190 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602b6755510 con 0x5602b67d1a10 2024-05-10T05:32:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.040+0000 7f3f06600700 1 --2- 172.21.15.131:0/875563190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6878eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.041+0000 7f3f05c00700 1 --2- 172.21.15.131:0/875563190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687e730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5602b686f330 tx=0x7f3ef40973b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd1cbc89decbed10 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.041+0000 7f3ef3400700 1 -- 172.21.15.131:0/875563190 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40a3090 con 0x5602b67d1a10 2024-05-10T05:32:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.041+0000 7f3ef3400700 1 -- 172.21.15.131:0/875563190 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef40cb690 con 0x5602b67d1a10 2024-05-10T05:32:53.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.041+0000 7f3ef3400700 1 -- 172.21.15.131:0/875563190 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef409b160 con 0x5602b67d1a10 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.042+0000 7f3f20495b80 1 -- 172.21.15.131:0/875563190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 msgr2=0x5602b687e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.042+0000 7f3f20495b80 1 --2- 172.21.15.131:0/875563190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687e730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5602b686f330 tx=0x7f3ef40973b0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.042+0000 7f3f20495b80 1 -- 172.21.15.131:0/875563190 shutdown_connections 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.042+0000 7f3f20495b80 1 --2- 172.21.15.131:0/875563190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b6878eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.042+0000 7f3f20495b80 1 --2- 172.21.15.131:0/875563190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687e730 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.042+0000 7f3f20495b80 1 --2- 172.21.15.131:0/875563190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b686f650 0x5602b687ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.042+0000 7f3f20495b80 1 -- 172.21.15.131:0/875563190 >> 172.21.15.131:0/875563190 conn(0x5602b6866400 msgr2=0x5602b6877890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.042+0000 7f3f20495b80 1 -- 172.21.15.131:0/875563190 shutdown_connections 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.042+0000 7f3f20495b80 1 -- 172.21.15.131:0/875563190 wait complete. 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.042+0000 7f3f20495b80 1 -- start start 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f20495b80 1 Processor -- start 2024-05-10T05:32:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b686f650 0x5602b687db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f20495b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b687e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f20495b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b68db370 con 0x5602b6870520 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f20495b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b68db4b0 con 0x5602b686f650 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f20495b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b68db670 con 0x5602b67d1a10 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f20495b80 10 client.0 _ll_register_callbacks cb 0x5602b6734ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f05c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f06600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b687e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f05c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59048/0 (socket says 172.21.15.131:59048) 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f05c00700 1 -- 172.21.15.131:0/3181740110 learned_addr learned my addr 172.21.15.131:0/3181740110 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.043+0000 7f3f05200700 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b686f650 0x5602b687db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3f05c00700 1 -- 172.21.15.131:0/3181740110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b686f650 msgr2=0x5602b687db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3f05c00700 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b686f650 0x5602b687db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3f05c00700 1 -- 172.21.15.131:0/3181740110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 msgr2=0x5602b687e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3f05c00700 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b687e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3f05c00700 1 -- 172.21.15.131:0/3181740110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef8003e40 con 0x5602b67d1a10 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3f05c00700 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687d5f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x5602b674d400 tx=0x7f3ef4009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c0a4c3ca004292 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3ee7400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3ee7400700 1 -- 172.21.15.131:0/3181740110 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4003c40 con 0x5602b67d1a10 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3f20495b80 1 -- 172.21.15.131:0/3181740110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ef40cb520 con 0x5602b67d1a10 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3ee7400700 1 -- 172.21.15.131:0/3181740110 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3ef409a3e0 con 0x5602b67d1a10 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3ee7400700 1 -- 172.21.15.131:0/3181740110 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ef4097500 con 0x5602b67d1a10 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3f20495b80 10 client.25003 fetch_fsmap learned FSMap version 26 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3f20495b80 1 -- 172.21.15.131:0/3181740110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5602b6610360 con 0x5602b67d1a10 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.044+0000 7f3ee7400700 1 -- 172.21.15.131:0/3181740110 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef40cb520 con 0x5602b67d1a10 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.045+0000 7f3ee7400700 1 -- 172.21.15.131:0/3181740110 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ef40a2c90 con 0x5602b67d1a10 2024-05-10T05:32:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.045+0000 7f3f20495b80 10 client.25003 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.045+0000 7f3f20495b80 10 client.25003 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.045+0000 7f3f20495b80 20 client.25003 populate_metadata read hostname 'smithi131' 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.045+0000 7f3f20495b80 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5602b6870e70 0x5602b68dc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.045+0000 7f3f20495b80 4 client.25003 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.045+0000 7f3f20495b80 1 -- 172.21.15.131:0/3181740110 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5602b6787e70 con 0x5602b6870e70 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.045+0000 7f3f05200700 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5602b6870e70 0x5602b68dc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3ee7400700 10 client.25003 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f05200700 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5602b6870e70 0x5602b68dc8e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6e93b83e0dfcb597 server_cookie=260bccc689336f34 in_seq=0 out_seq=0 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3ee7400700 1 -- 172.21.15.131:0/3181740110 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5602b6787e70 con 0x5602b6870e70 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3ee7400700 10 client.25003 handle_command_reply: tid=0 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 client.25003 shutdown 2024-05-10T05:32:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 -- 172.21.15.131:0/3181740110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 msgr2=0x5602b687d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687d5f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x5602b674d400 tx=0x7f3ef4009400 comp rx=0 tx=0).stop 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 -- 172.21.15.131:0/3181740110 shutdown_connections 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5602b6870e70 0x5602b68dc8e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5602b6870520 0x5602b687e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602b67d1a10 0x5602b687d5f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 --2- 172.21.15.131:0/3181740110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b686f650 0x5602b687db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3ee7400700 0 client.25003 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 -- 172.21.15.131:0/3181740110 >> 172.21.15.131:0/3181740110 conn(0x5602b6866400 msgr2=0x5602b6877480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 -- 172.21.15.131:0/3181740110 shutdown_connections 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 1 -- 172.21.15.131:0/3181740110 wait complete. 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.046+0000 7f3f20495b80 20 client.25003 trim_cache size 0 max 16384 2024-05-10T05:32:53.052 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 4, 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24541, 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 55.008248154999997, 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 78.001468009000007 2024-05-10T05:32:53.053 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:32:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 -- 172.21.15.131:0/110435921 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3ee80a12e0 msgr2=0x7f3ee80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3ee80a12e0 0x7f3ee80a37f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f0019b220 tx=0x7f3eec009260 comp rx=0 tx=0).stop 2024-05-10T05:32:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 -- 172.21.15.131:0/110435921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f00101be0 msgr2=0x7f3f0019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f00101be0 0x7f3f0019a180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4000c00 tx=0x7f3ef400f950 comp rx=0 tx=0).stop 2024-05-10T05:32:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 -- 172.21.15.131:0/110435921 shutdown_connections 2024-05-10T05:32:53.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3ee80a12e0 0x7f3ee80a37f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f00101be0 0x7f3f0019a180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f001036a0 0x7f3f001a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 --2- 172.21.15.131:0/110435921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f00102d00 0x7f3f0019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 -- 172.21.15.131:0/110435921 >> 172.21.15.131:0/110435921 conn(0x7f3f000fd210 msgr2=0x7f3f00073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 -- 172.21.15.131:0/110435921 shutdown_connections 2024-05-10T05:32:53.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.047+0000 7f3f08600700 1 -- 172.21.15.131:0/110435921 wait complete. 2024-05-10T05:32:53.076 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-10T05:32:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:53 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/765763680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:32:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:32:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:53 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3277301979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:53.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.339+0000 7fdf12c00700 1 -- 172.21.15.131:0/2865002458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c102d00 msgr2=0x7fdf0c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.339+0000 7fdf12c00700 1 --2- 172.21.15.131:0/2865002458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c102d00 0x7fdf0c103110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdef40099c0 tx=0x7fdef4009cd0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.339+0000 7fdf12c00700 1 -- 172.21.15.131:0/2865002458 shutdown_connections 2024-05-10T05:32:53.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.339+0000 7fdf12c00700 1 --2- 172.21.15.131:0/2865002458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf0c1036a0 0x7fdf0c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.339+0000 7fdf12c00700 1 --2- 172.21.15.131:0/2865002458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c101be0 0x7fdf0c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.339+0000 7fdf12c00700 1 --2- 172.21.15.131:0/2865002458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c102d00 0x7fdf0c103110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.339+0000 7fdf12c00700 1 -- 172.21.15.131:0/2865002458 >> 172.21.15.131:0/2865002458 conn(0x7fdf0c0fd1d0 msgr2=0x7fdf0c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.339+0000 7fdf12c00700 1 -- 172.21.15.131:0/2865002458 shutdown_connections 2024-05-10T05:32:53.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.339+0000 7fdf12c00700 1 -- 172.21.15.131:0/2865002458 wait complete. 2024-05-10T05:32:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.340+0000 7fdf12c00700 1 Processor -- start 2024-05-10T05:32:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf12c00700 1 -- start start 2024-05-10T05:32:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf12c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c101be0 0x7fdf0c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf12c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf0c102d00 0x7fdf0c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf12c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c1036a0 0x7fdf0c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf12c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1a1bc0 con 0x7fdf0c102d00 2024-05-10T05:32:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf12c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1a1d30 con 0x7fdf0c101be0 2024-05-10T05:32:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf12c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1a1ef0 con 0x7fdf0c1036a0 2024-05-10T05:32:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf03e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf0c102d00 0x7fdf0c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf03e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf0c102d00 0x7fdf0c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53848/0 (socket says 172.21.15.131:53848) 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf03e00700 1 -- 172.21.15.131:0/3984530305 learned_addr learned my addr 172.21.15.131:0/3984530305 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf0be00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c101be0 0x7fdf0c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.341+0000 7fdf10c00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c1036a0 0x7fdf0c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf03e00700 1 -- 172.21.15.131:0/3984530305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c1036a0 msgr2=0x7fdf0c1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf03e00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c1036a0 0x7fdf0c1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf03e00700 1 -- 172.21.15.131:0/3984530305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c101be0 msgr2=0x7fdf0c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf03e00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c101be0 0x7fdf0c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf03e00700 1 -- 172.21.15.131:0/3984530305 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef40095f0 con 0x7fdf0c102d00 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf03e00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf0c102d00 0x7fdf0c19a620 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009fd0 tx=0x7fdef40036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf09600700 1 -- 172.21.15.131:0/3984530305 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef401d070 con 0x7fdf0c102d00 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf09600700 1 -- 172.21.15.131:0/3984530305 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef4005030 con 0x7fdf0c102d00 2024-05-10T05:32:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf12c00700 1 -- 172.21.15.131:0/3984530305 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c1a2170 con 0x7fdf0c102d00 2024-05-10T05:32:53.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf09600700 1 -- 172.21.15.131:0/3984530305 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4004950 con 0x7fdf0c102d00 2024-05-10T05:32:53.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.342+0000 7fdf12c00700 1 -- 172.21.15.131:0/3984530305 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c1a2590 con 0x7fdf0c102d00 2024-05-10T05:32:53.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.346+0000 7fdf09600700 1 -- 172.21.15.131:0/3984530305 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fdef40051a0 con 0x7fdf0c102d00 2024-05-10T05:32:53.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.347+0000 7fdf2ac0cb80 1 Processor -- start 2024-05-10T05:32:53.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf2ac0cb80 1 -- start start 2024-05-10T05:32:53.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf09600700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdef00a1260 0x7fdef00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf09600700 1 -- 172.21.15.131:0/3984530305 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fdef40cd800 con 0x7fdf0c102d00 2024-05-10T05:32:53.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b75aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 0x557239b76930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b7d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239a9c100 con 0x557239b76520 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239a99ac0 con 0x557239b76e70 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239a953b0 con 0x557239b75650 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf10c00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdef00a1260 0x7fdef00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.348+0000 7fdf0be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b75aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf0be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b75aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59068/0 (socket says 172.21.15.131:59068) 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf0be00700 1 -- 172.21.15.131:0/4020063748 learned_addr learned my addr 172.21.15.131:0/4020063748 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf03e00700 1 --2- 172.21.15.131:0/4020063748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 0x557239b76930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf10c00700 1 --2- 172.21.15.131:0/4020063748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b7d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf0be00700 1 -- 172.21.15.131:0/4020063748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 msgr2=0x557239b7d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf0be00700 1 --2- 172.21.15.131:0/4020063748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b7d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf0be00700 1 -- 172.21.15.131:0/4020063748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 msgr2=0x557239b76930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf0be00700 1 --2- 172.21.15.131:0/4020063748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 0x557239b76930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf0be00700 1 -- 172.21.15.131:0/4020063748 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557239916360 con 0x557239b75650 2024-05-10T05:32:53.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf03e00700 1 --2- 172.21.15.131:0/4020063748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 0x557239b76930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:53.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf10c00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdef00a1260 0x7fdef00a3770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009fd0 tx=0x7fdf0400fba0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.349+0000 7fdf0be00700 1 --2- 172.21.15.131:0/4020063748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b75aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009cf0 tx=0x7fdefc011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba6f75db6d8a3060 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.350+0000 7fdf01200700 1 -- 172.21.15.131:0/4020063748 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc011d00 con 0x557239b75650 2024-05-10T05:32:53.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.350+0000 7fdf01200700 1 -- 172.21.15.131:0/4020063748 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc011e60 con 0x557239b75650 2024-05-10T05:32:53.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.350+0000 7fdf01200700 1 -- 172.21.15.131:0/4020063748 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc005280 con 0x557239b75650 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.350+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/4020063748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 msgr2=0x557239b75aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.350+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/4020063748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b75aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009cf0 tx=0x7fdefc011440 comp rx=0 tx=0).stop 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.351+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/4020063748 shutdown_connections 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.351+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/4020063748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b7d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.351+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/4020063748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b75aa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.351+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/4020063748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 0x557239b76930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.351+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/4020063748 >> 172.21.15.131:0/4020063748 conn(0x557239ad7a10 msgr2=0x557239b6ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.351+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/4020063748 shutdown_connections 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.351+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/4020063748 wait complete. 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.351+0000 7fdf2ac0cb80 1 -- start start 2024-05-10T05:32:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.351+0000 7fdf2ac0cb80 1 Processor -- start 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b7f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 0x557239b7fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b86a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239a31510 con 0x557239b76520 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239a311b0 con 0x557239b76e70 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239918650 con 0x557239b75650 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf2ac0cb80 10 client.0 _ll_register_callbacks cb 0x557239a3aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf03e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 0x557239b7fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf0be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b7f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf0be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b7f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59082/0 (socket says 172.21.15.131:59082) 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf0be00700 1 -- 172.21.15.131:0/1041205299 learned_addr learned my addr 172.21.15.131:0/1041205299 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf10c00700 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b86a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf0be00700 1 -- 172.21.15.131:0/1041205299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 msgr2=0x557239b86a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf0be00700 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b86a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf0be00700 1 -- 172.21.15.131:0/1041205299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 msgr2=0x557239b7fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.352+0000 7fdf0be00700 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 0x557239b7fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdf0be00700 1 -- 172.21.15.131:0/1041205299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c1a2590 con 0x557239b75650 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdf10c00700 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b86a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdf03e00700 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 0x557239b7fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdf0be00700 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b7f5f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdefc01b040 tx=0x7fdefc005000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef83c740ab1161f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdef9600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdef9600700 1 -- 172.21.15.131:0/1041205299 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc011cc0 con 0x557239b75650 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdef9600700 1 -- 172.21.15.131:0/1041205299 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fdefc01f040 con 0x557239b75650 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/1041205299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572399ee220 con 0x557239b75650 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdef9600700 1 -- 172.21.15.131:0/1041205299 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdefc005430 con 0x557239b75650 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdf2ac0cb80 10 client.25015 fetch_fsmap learned FSMap version 26 2024-05-10T05:32:53.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.353+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/1041205299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fdefc009920 con 0x557239b75650 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.354+0000 7fdef9600700 1 -- 172.21.15.131:0/1041205299 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572399ee220 con 0x557239b75650 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.354+0000 7fdef9600700 1 -- 172.21.15.131:0/1041205299 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdefc02f430 con 0x557239b75650 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.354+0000 7fdf2ac0cb80 10 client.25015 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.354+0000 7fdf2ac0cb80 10 client.25015 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.354+0000 7fdf2ac0cb80 20 client.25015 populate_metadata read hostname 'smithi131' 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.354+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557239b8a610 0x557239b8f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.354+0000 7fdf2ac0cb80 4 client.25015 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.354+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/1041205299 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557239b44730 con 0x557239b8a610 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.354+0000 7fdf03e00700 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557239b8a610 0x557239b8f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.355+0000 7fdf03e00700 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557239b8a610 0x557239b8f930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d9a1d6a6b9a604d server_cookie=e4166b29e754f297 in_seq=0 out_seq=0 2024-05-10T05:32:53.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.355+0000 7fdef9600700 10 client.25015 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:32:53.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdef9600700 1 -- 172.21.15.131:0/1041205299 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557239b44730 con 0x557239b8a610 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdef9600700 10 client.25015 handle_command_reply: tid=0 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 client.25015 shutdown 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/1041205299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 msgr2=0x557239b7f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b7f5f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdefc01b040 tx=0x7fdefc005000 comp rx=0 tx=0).stop 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/1041205299 shutdown_connections 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557239b8a610 0x557239b8f930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b86a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b75650 0x557239b7f5f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/1041205299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b76520 0x557239b7fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdef9600700 0 client.25015 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/1041205299 >> 172.21.15.131:0/1041205299 conn(0x557239ad7a10 msgr2=0x557239b6ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/1041205299 shutdown_connections 2024-05-10T05:32:53.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/1041205299 wait complete. 2024-05-10T05:32:53.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.356+0000 7fdf2ac0cb80 20 client.25015 trim_cache size 0 max 16384 2024-05-10T05:32:53.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf2ac0cb80 1 Processor -- start 2024-05-10T05:32:53.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf2ac0cb80 1 -- start start 2024-05-10T05:32:53.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b848d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b76520 0x557239b84e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b7dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239a345e0 con 0x557239b75650 2024-05-10T05:32:53.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239b6c440 con 0x557239b76e70 2024-05-10T05:32:53.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239b8c520 con 0x557239b76520 2024-05-10T05:32:53.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf0be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf0be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b848d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53874/0 (socket says 172.21.15.131:53874) 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf0be00700 1 -- 172.21.15.131:0/3482408895 learned_addr learned my addr 172.21.15.131:0/3482408895 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf03e00700 1 --2- 172.21.15.131:0/3482408895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b76520 0x557239b84e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.375+0000 7fdf10c00700 1 --2- 172.21.15.131:0/3482408895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b7dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.376+0000 7fdf0be00700 1 -- 172.21.15.131:0/3482408895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b76520 msgr2=0x557239b84e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.376+0000 7fdf0be00700 1 --2- 172.21.15.131:0/3482408895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b76520 0x557239b84e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.376+0000 7fdf0be00700 1 -- 172.21.15.131:0/3482408895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 msgr2=0x557239b7dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.376+0000 7fdf0be00700 1 --2- 172.21.15.131:0/3482408895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b7dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.376+0000 7fdf0be00700 1 -- 172.21.15.131:0/3482408895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572398c2b10 con 0x557239b75650 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.376+0000 7fdf0be00700 1 --2- 172.21.15.131:0/3482408895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b848d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000c00 tx=0x7fdefc011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebaf1d48bcb4e57 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.376+0000 7fdf01c00700 1 -- 172.21.15.131:0/3482408895 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc011cc0 con 0x557239b75650 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.376+0000 7fdf01c00700 1 -- 172.21.15.131:0/3482408895 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc004c00 con 0x557239b75650 2024-05-10T05:32:53.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.376+0000 7fdf01c00700 1 -- 172.21.15.131:0/3482408895 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0248f0 con 0x557239b75650 2024-05-10T05:32:53.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.377+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3482408895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 msgr2=0x557239b848d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.377+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/3482408895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b848d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000c00 tx=0x7fdefc011440 comp rx=0 tx=0).stop 2024-05-10T05:32:53.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.377+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3482408895 shutdown_connections 2024-05-10T05:32:53.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.377+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/3482408895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b848d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.377+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/3482408895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76e70 0x557239b7dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.377+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/3482408895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b76520 0x557239b84e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.377+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3482408895 >> 172.21.15.131:0/3482408895 conn(0x557239ad7a10 msgr2=0x557239b75cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.377+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3482408895 shutdown_connections 2024-05-10T05:32:53.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.377+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3482408895 wait complete. 2024-05-10T05:32:53.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.377+0000 7fdf2ac0cb80 1 -- start start 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf2ac0cb80 1 Processor -- start 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b82ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76520 0x557239b83400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf2ac0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b76e70 0x557239b84330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239b84870 con 0x557239b75650 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239b849e0 con 0x557239b76520 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf2ac0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239b84ba0 con 0x557239b76e70 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf2ac0cb80 10 client.0 _ll_register_callbacks cb 0x557239a3aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf0be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b82ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf0be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b82ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53884/0 (socket says 172.21.15.131:53884) 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf0be00700 1 -- 172.21.15.131:0/3001421772 learned_addr learned my addr 172.21.15.131:0/3001421772 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf03e00700 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76520 0x557239b83400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.378+0000 7fdf10c00700 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b76e70 0x557239b84330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdf0be00700 1 -- 172.21.15.131:0/3001421772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b76e70 msgr2=0x557239b84330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdf0be00700 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b76e70 0x557239b84330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdf0be00700 1 -- 172.21.15.131:0/3001421772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76520 msgr2=0x557239b83400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdf0be00700 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76520 0x557239b83400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdf0be00700 1 -- 172.21.15.131:0/3001421772 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc009920 con 0x557239b75650 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdf0be00700 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b82ec0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdefc02f760 tx=0x7fdefc024a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c5fd896ffa6959 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdef9600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdef9600700 1 -- 172.21.15.131:0/3001421772 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0056e0 con 0x557239b75650 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdef9600700 1 -- 172.21.15.131:0/3001421772 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fdefc000e40 con 0x557239b75650 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3001421772 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdefc011e60 con 0x557239b75650 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdef9600700 1 -- 172.21.15.131:0/3001421772 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdefc011c30 con 0x557239b75650 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdf2ac0cb80 10 client.14994 fetch_fsmap learned FSMap version 26 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.379+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3001421772 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557239a5b510 con 0x557239b75650 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.380+0000 7fdef9600700 1 -- 172.21.15.131:0/3001421772 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc011e60 con 0x557239b75650 2024-05-10T05:32:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.380+0000 7fdef9600700 1 -- 172.21.15.131:0/3001421772 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdefc038b20 con 0x557239b75650 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.380+0000 7fdf2ac0cb80 10 client.14994 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.380+0000 7fdf2ac0cb80 10 client.14994 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.380+0000 7fdf2ac0cb80 20 client.14994 populate_metadata read hostname 'smithi131' 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.380+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557239be3ac0 0x557239be2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.380+0000 7fdf2ac0cb80 4 client.14994 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.380+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3001421772 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557239a8de70 con 0x557239be3ac0 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.380+0000 7fdf03e00700 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557239be3ac0 0x557239be2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdf03e00700 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557239be3ac0 0x557239be2050 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bd6921410c6af0ea server_cookie=53848761f597a498 in_seq=0 out_seq=0 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdef9600700 10 client.14994 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdef9600700 1 -- 172.21.15.131:0/3001421772 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557239a8de70 con 0x557239be3ac0 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdef9600700 10 client.14994 handle_command_reply: tid=0 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdf2ac0cb80 1 client.14994 shutdown 2024-05-10T05:32:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3001421772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 msgr2=0x557239b82ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b82ec0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdefc02f760 tx=0x7fdefc024a00 comp rx=0 tx=0).stop 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3001421772 shutdown_connections 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557239be3ac0 0x557239be2050 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557239b75650 0x557239b82ec0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557239b76e70 0x557239b84330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdef9600700 0 client.14994 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdf2ac0cb80 1 --2- 172.21.15.131:0/3001421772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557239b76520 0x557239b83400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.381+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3001421772 >> 172.21.15.131:0/3001421772 conn(0x557239ad7a10 msgr2=0x557239b93950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3001421772 shutdown_connections 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf2ac0cb80 1 -- 172.21.15.131:0/3001421772 wait complete. 2024-05-10T05:32:53.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf2ac0cb80 20 client.14994 trim_cache size 0 max 16384 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 3, 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24565, 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 56.350661473000002, 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T05:32:53.388 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 74.751392812999995 2024-05-10T05:32:53.389 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:32:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf12c00700 1 -- 172.21.15.131:0/3984530305 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdef00a1260 msgr2=0x7fdef00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf12c00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdef00a1260 0x7fdef00a3770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009fd0 tx=0x7fdf0400fba0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf12c00700 1 -- 172.21.15.131:0/3984530305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf0c102d00 msgr2=0x7fdf0c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf12c00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf0c102d00 0x7fdf0c19a620 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009fd0 tx=0x7fdef40036f0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf12c00700 1 -- 172.21.15.131:0/3984530305 shutdown_connections 2024-05-10T05:32:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf12c00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdef00a1260 0x7fdef00a3770 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf12c00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf0c1036a0 0x7fdf0c1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf12c00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c101be0 0x7fdf0c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf12c00700 1 --2- 172.21.15.131:0/3984530305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf0c102d00 0x7fdf0c19a620 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.382+0000 7fdf12c00700 1 -- 172.21.15.131:0/3984530305 >> 172.21.15.131:0/3984530305 conn(0x7fdf0c0fd1d0 msgr2=0x7fdf0c073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.383+0000 7fdf12c00700 1 -- 172.21.15.131:0/3984530305 shutdown_connections 2024-05-10T05:32:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.383+0000 7fdf12c00700 1 -- 172.21.15.131:0/3984530305 wait complete. 2024-05-10T05:32:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:53 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/765763680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:32:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:32:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:53 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3277301979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:53.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:53 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/765763680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:32:53.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:32:53.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:32:53.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:32:53.409 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:53 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3277301979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:32:53.412 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T05:32:53.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.678+0000 7fcb5b800700 1 -- 172.21.15.131:0/1112097801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb541036a0 msgr2=0x7fcb54109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.678+0000 7fcb5b800700 1 --2- 172.21.15.131:0/1112097801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb541036a0 0x7fcb54109f50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009f90 tx=0x7fcb4c00b330 comp rx=0 tx=0).stop 2024-05-10T05:32:53.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.678+0000 7fcb5b800700 1 -- 172.21.15.131:0/1112097801 shutdown_connections 2024-05-10T05:32:53.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.678+0000 7fcb5b800700 1 --2- 172.21.15.131:0/1112097801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb541036a0 0x7fcb54109f50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.678+0000 7fcb5b800700 1 --2- 172.21.15.131:0/1112097801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb54101be0 0x7fcb54101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.678+0000 7fcb5b800700 1 --2- 172.21.15.131:0/1112097801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb54102d00 0x7fcb54103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.678+0000 7fcb5b800700 1 -- 172.21.15.131:0/1112097801 >> 172.21.15.131:0/1112097801 conn(0x7fcb540fd1d0 msgr2=0x7fcb540ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.678+0000 7fcb5b800700 1 -- 172.21.15.131:0/1112097801 shutdown_connections 2024-05-10T05:32:53.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.678+0000 7fcb5b800700 1 -- 172.21.15.131:0/1112097801 wait complete. 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.679+0000 7fcb5b800700 1 Processor -- start 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.679+0000 7fcb5b800700 1 -- start start 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.679+0000 7fcb5b800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb54101be0 0x7fcb5419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.679+0000 7fcb5b800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb54102d00 0x7fcb5419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.679+0000 7fcb5b800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb541036a0 0x7fcb541a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb5b800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541a1d20 con 0x7fcb541036a0 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb5b800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541a1e90 con 0x7fcb54101be0 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb5b800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541a2050 con 0x7fcb54102d00 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb59800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb541036a0 0x7fcb541a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb58e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb54101be0 0x7fcb5419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb58e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb54101be0 0x7fcb5419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:52552/0 (socket says 172.21.15.131:52552) 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb59800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb541036a0 0x7fcb541a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53912/0 (socket says 172.21.15.131:53912) 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb58e00700 1 -- 172.21.15.131:0/1558605628 learned_addr learned my addr 172.21.15.131:0/1558605628 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb53e00700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb54102d00 0x7fcb5419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb58e00700 1 -- 172.21.15.131:0/1558605628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb54102d00 msgr2=0x7fcb5419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb58e00700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb54102d00 0x7fcb5419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb58e00700 1 -- 172.21.15.131:0/1558605628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb541036a0 msgr2=0x7fcb541a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb58e00700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb541036a0 0x7fcb541a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb58e00700 1 -- 172.21.15.131:0/1558605628 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c009c40 con 0x7fcb54101be0 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb59800700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb541036a0 0x7fcb541a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.680+0000 7fcb58e00700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb54101be0 0x7fcb5419a240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800e9c0 tx=0x7fcb4800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.681+0000 7fcb51600700 1 -- 172.21.15.131:0/1558605628 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48004220 con 0x7fcb54101be0 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.681+0000 7fcb5b800700 1 -- 172.21.15.131:0/1558605628 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb541a2330 con 0x7fcb54101be0 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.681+0000 7fcb51600700 1 -- 172.21.15.131:0/1558605628 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb48004380 con 0x7fcb54101be0 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.681+0000 7fcb51600700 1 -- 172.21.15.131:0/1558605628 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48003b20 con 0x7fcb54101be0 2024-05-10T05:32:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.681+0000 7fcb5b800700 1 -- 172.21.15.131:0/1558605628 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb541a2890 con 0x7fcb54101be0 2024-05-10T05:32:53.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.684+0000 7fcb51600700 1 -- 172.21.15.131:0/1558605628 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fcb48003cc0 con 0x7fcb54101be0 2024-05-10T05:32:53.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.685+0000 7fcb736a4b80 1 Processor -- start 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.685+0000 7fcb736a4b80 1 -- start start 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.685+0000 7fcb51600700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcb3c0a1260 0x7fcb3c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb51600700 1 -- 172.21.15.131:0/1558605628 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fcb48014070 con 0x7fcb54101be0 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94db650 0x5617f94dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94dc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94dce70 0x5617f94e3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f9402100 con 0x5617f94dce70 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f93ffac0 con 0x5617f94db650 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f93fb3b0 con 0x5617f94dc520 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb59800700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcb3c0a1260 0x7fcb3c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb58e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94db650 0x5617f94dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb58e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94db650 0x5617f94dbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:52566/0 (socket says 172.21.15.131:52566) 2024-05-10T05:32:53.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb58e00700 1 -- 172.21.15.131:0/2329406251 learned_addr learned my addr 172.21.15.131:0/2329406251 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb59800700 1 --2- 172.21.15.131:0/2329406251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94dce70 0x5617f94e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb53e00700 1 --2- 172.21.15.131:0/2329406251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94dc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb58e00700 1 -- 172.21.15.131:0/2329406251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 msgr2=0x5617f94dc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb58e00700 1 --2- 172.21.15.131:0/2329406251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94dc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb58e00700 1 -- 172.21.15.131:0/2329406251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94dce70 msgr2=0x5617f94e3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb58e00700 1 --2- 172.21.15.131:0/2329406251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94dce70 0x5617f94e3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.686+0000 7fcb58e00700 1 -- 172.21.15.131:0/2329406251 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f927c360 con 0x5617f94db650 2024-05-10T05:32:53.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.687+0000 7fcb58e00700 1 --2- 172.21.15.131:0/2329406251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94db650 0x5617f94dbaa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb480ca500 tx=0x7fcb48099800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6f7bba108d105 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.687+0000 7fcb59800700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcb3c0a1260 0x7fcb3c0a3770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009840 tx=0x7fcb4c009470 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.687+0000 7fcb46000700 1 -- 172.21.15.131:0/2329406251 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480b1070 con 0x5617f94db650 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.687+0000 7fcb46000700 1 -- 172.21.15.131:0/2329406251 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb48099ea0 con 0x5617f94db650 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.687+0000 7fcb46000700 1 -- 172.21.15.131:0/2329406251 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480ca7b0 con 0x5617f94db650 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.688+0000 7fcb736a4b80 1 -- 172.21.15.131:0/2329406251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94db650 msgr2=0x5617f94dbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.688+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/2329406251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94db650 0x5617f94dbaa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb480ca500 tx=0x7fcb48099800 comp rx=0 tx=0).stop 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.688+0000 7fcb736a4b80 1 -- 172.21.15.131:0/2329406251 shutdown_connections 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.688+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/2329406251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94dce70 0x5617f94e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.688+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/2329406251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94db650 0x5617f94dbaa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.688+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/2329406251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94dc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.688+0000 7fcb736a4b80 1 -- 172.21.15.131:0/2329406251 >> 172.21.15.131:0/2329406251 conn(0x5617f943da10 msgr2=0x5617f94d5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.688+0000 7fcb736a4b80 1 -- 172.21.15.131:0/2329406251 shutdown_connections 2024-05-10T05:32:53.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.688+0000 7fcb736a4b80 1 -- 172.21.15.131:0/2329406251 wait complete. 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.688+0000 7fcb736a4b80 1 -- start start 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb736a4b80 1 Processor -- start 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dc520 0x5617f94e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dce70 0x5617f94ecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f9397510 con 0x5617f94db650 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f93971b0 con 0x5617f94dc520 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f927e650 con 0x5617f94dce70 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb736a4b80 10 client.0 _ll_register_callbacks cb 0x5617f93a0ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb58e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb58e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53924/0 (socket says 172.21.15.131:53924) 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb58e00700 1 -- 172.21.15.131:0/3063465711 learned_addr learned my addr 172.21.15.131:0/3063465711 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.689+0000 7fcb53e00700 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dc520 0x5617f94e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb59800700 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dce70 0x5617f94ecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb58e00700 1 -- 172.21.15.131:0/3063465711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dce70 msgr2=0x5617f94ecad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb58e00700 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dce70 0x5617f94ecad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb58e00700 1 -- 172.21.15.131:0/3063465711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dc520 msgr2=0x5617f94e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb58e00700 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dc520 0x5617f94e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb58e00700 1 -- 172.21.15.131:0/3063465711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb541a2890 con 0x5617f94db650 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb58e00700 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e5630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb480963a0 tx=0x7fcb48098400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff43201922d8af3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb3aa00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb3aa00700 1 -- 172.21.15.131:0/3063465711 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480b1070 con 0x5617f94db650 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb3aa00700 1 -- 172.21.15.131:0/3063465711 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fcb4809dcd0 con 0x5617f94db650 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb736a4b80 1 -- 172.21.15.131:0/3063465711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617f9354220 con 0x5617f94db650 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb3aa00700 1 -- 172.21.15.131:0/3063465711 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb480b82a0 con 0x5617f94db650 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb736a4b80 10 client.15006 fetch_fsmap learned FSMap version 26 2024-05-10T05:32:53.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.690+0000 7fcb736a4b80 1 -- 172.21.15.131:0/3063465711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5617f9228b10 con 0x5617f94db650 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.691+0000 7fcb3aa00700 1 -- 172.21.15.131:0/3063465711 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb48065020 con 0x5617f94db650 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.691+0000 7fcb3aa00700 1 -- 172.21.15.131:0/3063465711 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb480cab70 con 0x5617f94db650 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.691+0000 7fcb736a4b80 10 client.15006 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.691+0000 7fcb736a4b80 10 client.15006 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.691+0000 7fcb736a4b80 20 client.15006 populate_metadata read hostname 'smithi131' 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.691+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5617f94f0980 0x5617f94f5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.691+0000 7fcb736a4b80 4 client.15006 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.691+0000 7fcb736a4b80 1 -- 172.21.15.131:0/3063465711 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617f94aa730 con 0x5617f94f0980 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.691+0000 7fcb53e00700 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5617f94f0980 0x5617f94f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.692+0000 7fcb53e00700 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5617f94f0980 0x5617f94f5690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a47279f41d81d267 server_cookie=8654039bd223f2c7 in_seq=0 out_seq=0 2024-05-10T05:32:53.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.692+0000 7fcb3aa00700 10 client.15006 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:32:53.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.692+0000 7fcb3aa00700 1 -- 172.21.15.131:0/3063465711 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5617f94aa730 con 0x5617f94f0980 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.692+0000 7fcb3aa00700 10 client.15006 handle_command_reply: tid=0 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.692+0000 7fcb736a4b80 1 client.15006 shutdown 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 1 -- 172.21.15.131:0/3063465711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 msgr2=0x5617f94e5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e5630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb480963a0 tx=0x7fcb48098400 comp rx=0 tx=0).stop 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 1 -- 172.21.15.131:0/3063465711 shutdown_connections 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5617f94f0980 0x5617f94f5690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e5630 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb3aa00700 0 client.15006 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dce70 0x5617f94ecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/3063465711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dc520 0x5617f94e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 1 -- 172.21.15.131:0/3063465711 >> 172.21.15.131:0/3063465711 conn(0x5617f943da10 msgr2=0x5617f94d5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 1 -- 172.21.15.131:0/3063465711 shutdown_connections 2024-05-10T05:32:53.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 1 -- 172.21.15.131:0/3063465711 wait complete. 2024-05-10T05:32:53.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.693+0000 7fcb736a4b80 20 client.15006 trim_cache size 0 max 16384 2024-05-10T05:32:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.712+0000 7fcb736a4b80 1 Processor -- start 2024-05-10T05:32:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.712+0000 7fcb736a4b80 1 -- start start 2024-05-10T05:32:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94dc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94eca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 0x5617f94d6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f93f4150 con 0x5617f94db650 2024-05-10T05:32:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f93f8010 con 0x5617f94dce70 2024-05-10T05:32:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f94d2ce0 con 0x5617f94dc520 2024-05-10T05:32:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb58e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94dc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb58e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94dc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53936/0 (socket says 172.21.15.131:53936) 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb58e00700 1 -- 172.21.15.131:0/653595876 learned_addr learned my addr 172.21.15.131:0/653595876 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb53e00700 1 --2- 172.21.15.131:0/653595876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94eca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb59800700 1 --2- 172.21.15.131:0/653595876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 0x5617f94d6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb58e00700 1 -- 172.21.15.131:0/653595876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 msgr2=0x5617f94eca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb58e00700 1 --2- 172.21.15.131:0/653595876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94eca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb58e00700 1 -- 172.21.15.131:0/653595876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 msgr2=0x5617f94d6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb58e00700 1 --2- 172.21.15.131:0/653595876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 0x5617f94d6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb58e00700 1 -- 172.21.15.131:0/653595876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f93c1510 con 0x5617f94db650 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.713+0000 7fcb59800700 1 --2- 172.21.15.131:0/653595876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 0x5617f94d6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.714+0000 7fcb58e00700 1 --2- 172.21.15.131:0/653595876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94dc930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb48099ba0 tx=0x7fcb48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e70b371d86b5e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.714+0000 7fcb46a00700 1 -- 172.21.15.131:0/653595876 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480b1070 con 0x5617f94db650 2024-05-10T05:32:53.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.714+0000 7fcb46a00700 1 -- 172.21.15.131:0/653595876 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb48065050 con 0x5617f94db650 2024-05-10T05:32:53.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.714+0000 7fcb46a00700 1 -- 172.21.15.131:0/653595876 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480cab70 con 0x5617f94db650 2024-05-10T05:32:53.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 -- 172.21.15.131:0/653595876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 msgr2=0x5617f94dc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/653595876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94dc930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb48099ba0 tx=0x7fcb48004750 comp rx=0 tx=0).stop 2024-05-10T05:32:53.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 -- 172.21.15.131:0/653595876 shutdown_connections 2024-05-10T05:32:53.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/653595876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94dc930 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/653595876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 0x5617f94d6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/653595876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94eca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 -- 172.21.15.131:0/653595876 >> 172.21.15.131:0/653595876 conn(0x5617f943da10 msgr2=0x5617f94e4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 -- 172.21.15.131:0/653595876 shutdown_connections 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 -- 172.21.15.131:0/653595876 wait complete. 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 -- start start 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.715+0000 7fcb736a4b80 1 Processor -- start 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb736a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 0x5617f94ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f94ec000 con 0x5617f94db650 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f94ec170 con 0x5617f94dce70 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb736a4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f94ec330 con 0x5617f94dc520 2024-05-10T05:32:53.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb736a4b80 10 client.0 _ll_register_callbacks cb 0x5617f93a0ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb58e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb58e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53946/0 (socket says 172.21.15.131:53946) 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb58e00700 1 -- 172.21.15.131:0/4062717964 learned_addr learned my addr 172.21.15.131:0/4062717964 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb53e00700 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb59800700 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 0x5617f94ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb58e00700 1 -- 172.21.15.131:0/4062717964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 msgr2=0x5617f94e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb58e00700 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94e8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb58e00700 1 -- 172.21.15.131:0/4062717964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 msgr2=0x5617f94ebac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb58e00700 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 0x5617f94ebac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.716+0000 7fcb58e00700 1 -- 172.21.15.131:0/4062717964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617f9228b10 con 0x5617f94db650 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb58e00700 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e8640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb48099e30 tx=0x7fcb4800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0e78822ddcbe67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb3aa00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb3aa00700 1 -- 172.21.15.131:0/4062717964 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480b1020 con 0x5617f94db650 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb3aa00700 1 -- 172.21.15.131:0/4062717964 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fcb48098e80 con 0x5617f94db650 2024-05-10T05:32:53.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb736a4b80 1 -- 172.21.15.131:0/4062717964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb48099ea0 con 0x5617f94db650 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb3aa00700 1 -- 172.21.15.131:0/4062717964 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb480b82a0 con 0x5617f94db650 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb736a4b80 10 client.15012 fetch_fsmap learned FSMap version 26 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb736a4b80 1 -- 172.21.15.131:0/4062717964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5617f9280c80 con 0x5617f94db650 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb3aa00700 1 -- 172.21.15.131:0/4062717964 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb480a6020 con 0x5617f94db650 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb3aa00700 1 -- 172.21.15.131:0/4062717964 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb4809ebf0 con 0x5617f94db650 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.717+0000 7fcb736a4b80 10 client.15012 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.718+0000 7fcb736a4b80 10 client.15012 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.718+0000 7fcb736a4b80 20 client.15012 populate_metadata read hostname 'smithi131' 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.718+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5617f9549150 0x5617f9547860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.718+0000 7fcb736a4b80 4 client.15012 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.718+0000 7fcb736a4b80 1 -- 172.21.15.131:0/4062717964 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5617f93f3e70 con 0x5617f9549150 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.718+0000 7fcb53e00700 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5617f9549150 0x5617f9547860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.718+0000 7fcb3aa00700 10 client.15012 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:32:53.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.718+0000 7fcb53e00700 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5617f9549150 0x5617f9547860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d51e52d99f2caa77 server_cookie=2bcea9ff6abb9297 in_seq=0 out_seq=0 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb3aa00700 1 -- 172.21.15.131:0/4062717964 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5617f93f3e70 con 0x5617f9549150 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb3aa00700 10 client.15012 handle_command_reply: tid=0 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 client.15012 shutdown 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 -- 172.21.15.131:0/4062717964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 msgr2=0x5617f94e8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e8640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb48099e30 tx=0x7fcb4800ef50 comp rx=0 tx=0).stop 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 -- 172.21.15.131:0/4062717964 shutdown_connections 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5617f9549150 0x5617f9547860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f94db650 0x5617f94e8640 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617f94dce70 0x5617f94ebac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb3aa00700 0 client.15012 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 --2- 172.21.15.131:0/4062717964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617f94dc520 0x5617f94e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 -- 172.21.15.131:0/4062717964 >> 172.21.15.131:0/4062717964 conn(0x5617f943da10 msgr2=0x5617f94f96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 -- 172.21.15.131:0/4062717964 shutdown_connections 2024-05-10T05:32:53.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 1 -- 172.21.15.131:0/4062717964 wait complete. 2024-05-10T05:32:53.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.719+0000 7fcb736a4b80 20 client.15012 trim_cache size 0 max 16384 2024-05-10T05:32:53.725 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:32:53.725 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:32:53.725 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-10T05:32:53.725 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24569, 2024-05-10T05:32:53.725 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-10T05:32:53.725 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-10T05:32:53.725 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-10T05:32:53.725 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 57.706961343000003, 2024-05-10T05:32:53.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-05-10T05:32:53.726 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-10T05:32:53.726 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T05:32:53.726 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 70.063459049000002 2024-05-10T05:32:53.726 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 -- 172.21.15.131:0/1558605628 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcb3c0a1260 msgr2=0x7fcb3c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcb3c0a1260 0x7fcb3c0a3770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009840 tx=0x7fcb4c009470 comp rx=0 tx=0).stop 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 -- 172.21.15.131:0/1558605628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb54101be0 msgr2=0x7fcb5419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb54101be0 0x7fcb5419a240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800e9c0 tx=0x7fcb4800ecd0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 -- 172.21.15.131:0/1558605628 shutdown_connections 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcb3c0a1260 0x7fcb3c0a3770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb541036a0 0x7fcb541a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb54101be0 0x7fcb5419a240 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 --2- 172.21.15.131:0/1558605628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb54102d00 0x7fcb5419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 -- 172.21.15.131:0/1558605628 >> 172.21.15.131:0/1558605628 conn(0x7fcb540fd1d0 msgr2=0x7fcb54073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 -- 172.21.15.131:0/1558605628 shutdown_connections 2024-05-10T05:32:53.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:53.720+0000 7fcb5b800700 1 -- 172.21.15.131:0/1558605628 wait complete. 2024-05-10T05:32:53.746 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-05-10T05:32:54.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.009+0000 7fcb05c00700 1 -- 172.21.15.131:0/1402891621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00102d00 msgr2=0x7fcb00103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.009+0000 7fcb05c00700 1 --2- 172.21.15.131:0/1402891621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00102d00 0x7fcb00103110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0099c0 tx=0x7fcaec009cd0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.009+0000 7fcb05c00700 1 -- 172.21.15.131:0/1402891621 shutdown_connections 2024-05-10T05:32:54.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.009+0000 7fcb05c00700 1 --2- 172.21.15.131:0/1402891621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb00101be0 0x7fcb00101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.009+0000 7fcb05c00700 1 --2- 172.21.15.131:0/1402891621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb001036a0 0x7fcb00109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.009+0000 7fcb05c00700 1 --2- 172.21.15.131:0/1402891621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb00102d00 0x7fcb00103110 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.009+0000 7fcb05c00700 1 -- 172.21.15.131:0/1402891621 >> 172.21.15.131:0/1402891621 conn(0x7fcb000fd210 msgr2=0x7fcb000ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.009+0000 7fcb05c00700 1 -- 172.21.15.131:0/1402891621 shutdown_connections 2024-05-10T05:32:54.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.009+0000 7fcb05c00700 1 -- 172.21.15.131:0/1402891621 wait complete. 2024-05-10T05:32:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.010+0000 7fcb05c00700 1 Processor -- start 2024-05-10T05:32:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.010+0000 7fcb05c00700 1 -- start start 2024-05-10T05:32:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcb05c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00101be0 0x7fcb00197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcb05c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb00102d00 0x7fcb001983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcb05c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb001036a0 0x7fcb0019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcb05c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0019f970 con 0x7fcb00102d00 2024-05-10T05:32:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcb05c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0019fae0 con 0x7fcb00101be0 2024-05-10T05:32:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcb05c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0019fca0 con 0x7fcb001036a0 2024-05-10T05:32:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcafe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb00102d00 0x7fcb001983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcafe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb00102d00 0x7fcb001983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53954/0 (socket says 172.21.15.131:53954) 2024-05-10T05:32:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcafe000700 1 -- 172.21.15.131:0/3295983933 learned_addr learned my addr 172.21.15.131:0/3295983933 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcafea00700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00101be0 0x7fcb00197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcaff400700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb001036a0 0x7fcb0019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcafe000700 1 -- 172.21.15.131:0/3295983933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb001036a0 msgr2=0x7fcb0019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcafe000700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb001036a0 0x7fcb0019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcafe000700 1 -- 172.21.15.131:0/3295983933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00101be0 msgr2=0x7fcb00197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcafe000700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00101be0 0x7fcb00197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.011+0000 7fcafe000700 1 -- 172.21.15.131:0/3295983933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec0095f0 con 0x7fcb00102d00 2024-05-10T05:32:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.012+0000 7fcafe000700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb00102d00 0x7fcb001983d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009fd0 tx=0x7fcaec0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.012+0000 7fcaf7400700 1 -- 172.21.15.131:0/3295983933 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec01d070 con 0x7fcb00102d00 2024-05-10T05:32:54.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.012+0000 7fcaf7400700 1 -- 172.21.15.131:0/3295983933 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaec005030 con 0x7fcb00102d00 2024-05-10T05:32:54.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.012+0000 7fcb05c00700 1 -- 172.21.15.131:0/3295983933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0019ff20 con 0x7fcb00102d00 2024-05-10T05:32:54.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.012+0000 7fcaf7400700 1 -- 172.21.15.131:0/3295983933 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec004950 con 0x7fcb00102d00 2024-05-10T05:32:54.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.012+0000 7fcb05c00700 1 -- 172.21.15.131:0/3295983933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb001a0290 con 0x7fcb00102d00 2024-05-10T05:32:54.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.017+0000 7fcaf7400700 1 -- 172.21.15.131:0/3295983933 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fcaec004b50 con 0x7fcb00102d00 2024-05-10T05:32:54.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.017+0000 7fcb1db0cb80 1 Processor -- start 2024-05-10T05:32:54.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcb1db0cb80 1 -- start start 2024-05-10T05:32:54.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d86eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 0x559571d87d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d8eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571ca9100 con 0x559571d88280 2024-05-10T05:32:54.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571ca6ac0 con 0x559571d87930 2024-05-10T05:32:54.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571ca23b0 con 0x559571d86a60 2024-05-10T05:32:54.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcaf7400700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcae40a1260 0x7fcae40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcafe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d8eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcaff400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d86eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcafea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 0x559571d87d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcafea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 0x559571d87d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:52610/0 (socket says 172.21.15.131:52610) 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcaff400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d86eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59186/0 (socket says 172.21.15.131:59186) 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcafea00700 1 -- 172.21.15.131:0/3187796934 learned_addr learned my addr 172.21.15.131:0/3187796934 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.018+0000 7fcafe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d8eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53962/0 (socket says 172.21.15.131:53962) 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcaf7400700 1 -- 172.21.15.131:0/3295983933 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fcaec0cd820 con 0x7fcb00102d00 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcaff400700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcae40a1260 0x7fcae40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcafea00700 1 -- 172.21.15.131:0/3187796934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 msgr2=0x559571d86eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcafea00700 1 --2- 172.21.15.131:0/3187796934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d86eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcafea00700 1 -- 172.21.15.131:0/3187796934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 msgr2=0x559571d8eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcafea00700 1 --2- 172.21.15.131:0/3187796934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d8eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcafea00700 1 -- 172.21.15.131:0/3187796934 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559571b23360 con 0x559571d87930 2024-05-10T05:32:54.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcaff400700 1 --2- 172.21.15.131:0/3187796934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d86eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:54.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcaff400700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcae40a1260 0x7fcae40a3770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x559571d86740 tx=0x7fcaf800b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcafea00700 1 --2- 172.21.15.131:0/3187796934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 0x559571d87d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009cf0 tx=0x7fcaf0011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cda34e0c024345a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.019+0000 7fcaebe00700 1 -- 172.21.15.131:0/3187796934 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0011d80 con 0x559571d87930 2024-05-10T05:32:54.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.020+0000 7fcaebe00700 1 -- 172.21.15.131:0/3187796934 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf00044e0 con 0x559571d87930 2024-05-10T05:32:54.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.020+0000 7fcaebe00700 1 -- 172.21.15.131:0/3187796934 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0005280 con 0x559571d87930 2024-05-10T05:32:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.020+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3187796934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 msgr2=0x559571d87d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.020+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/3187796934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 0x559571d87d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009cf0 tx=0x7fcaf0011440 comp rx=0 tx=0).stop 2024-05-10T05:32:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.020+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3187796934 shutdown_connections 2024-05-10T05:32:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.020+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/3187796934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d8eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.020+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/3187796934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d86eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.020+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/3187796934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 0x559571d87d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.020+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3187796934 >> 172.21.15.131:0/3187796934 conn(0x559571ce4a10 msgr2=0x559571d7cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.021+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3187796934 shutdown_connections 2024-05-10T05:32:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.021+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3187796934 wait complete. 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.021+0000 7fcb1db0cb80 1 -- start start 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.021+0000 7fcb1db0cb80 1 Processor -- start 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d90a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 0x559571d90fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d97f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571c3e510 con 0x559571d88280 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571c3e1b0 con 0x559571d87930 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571b25650 con 0x559571d86a60 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcb1db0cb80 10 client.0 _ll_register_callbacks cb 0x559571c47ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcaff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d97f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcafea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d90a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcafea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d90a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59190/0 (socket says 172.21.15.131:59190) 2024-05-10T05:32:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcafea00700 1 -- 172.21.15.131:0/3360718041 learned_addr learned my addr 172.21.15.131:0/3360718041 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcafe000700 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 0x559571d90fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcafea00700 1 -- 172.21.15.131:0/3360718041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 msgr2=0x559571d90fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcafea00700 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 0x559571d90fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcafea00700 1 -- 172.21.15.131:0/3360718041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 msgr2=0x559571d97f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcafea00700 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d97f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcafea00700 1 -- 172.21.15.131:0/3360718041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf80089c0 con 0x559571d86a60 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.022+0000 7fcaff400700 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d97f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.023+0000 7fcafea00700 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d90a60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0011790 tx=0x7fcaf0011e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9ffce36ea7e44d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.023+0000 7fcae8c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.023+0000 7fcae8c00700 1 -- 172.21.15.131:0/3360718041 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00050d0 con 0x559571d86a60 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.023+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3360718041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559571bfb220 con 0x559571d86a60 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.023+0000 7fcae8c00700 1 -- 172.21.15.131:0/3360718041 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fcaf001f040 con 0x559571d86a60 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.023+0000 7fcae8c00700 1 -- 172.21.15.131:0/3360718041 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf0005230 con 0x559571d86a60 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.023+0000 7fcb1db0cb80 10 client.25051 fetch_fsmap learned FSMap version 26 2024-05-10T05:32:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.023+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3360718041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fcaf0009920 con 0x559571d86a60 2024-05-10T05:32:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.024+0000 7fcae8c00700 1 -- 172.21.15.131:0/3360718041 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf0031030 con 0x559571d86a60 2024-05-10T05:32:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.024+0000 7fcae8c00700 1 -- 172.21.15.131:0/3360718041 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcaf0024cc0 con 0x559571d86a60 2024-05-10T05:32:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.024+0000 7fcb1db0cb80 10 client.25051 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:32:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.024+0000 7fcb1db0cb80 10 client.25051 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-10T05:32:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.024+0000 7fcb1db0cb80 20 client.25051 populate_metadata read hostname 'smithi131' 2024-05-10T05:32:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.024+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x559571d9bae0 0x559571da0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.024+0000 7fcb1db0cb80 4 client.25051 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:32:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.024+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3360718041 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559571d51730 con 0x559571d9bae0 2024-05-10T05:32:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.024+0000 7fcafe000700 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x559571d9bae0 0x559571da0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.025+0000 7fcae8c00700 10 client.25051 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:32:54.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.025+0000 7fcafe000700 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x559571d9bae0 0x559571da0e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73ab8d6db8f7eab1 server_cookie=115d1c7eaa7f4b87 in_seq=0 out_seq=0 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcae8c00700 1 -- 172.21.15.131:0/3360718041 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559571d51730 con 0x559571d9bae0 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcae8c00700 10 client.25051 handle_command_reply: tid=0 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 client.25051 shutdown 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3360718041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 msgr2=0x559571d90a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d90a60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0011790 tx=0x7fcaf0011e00 comp rx=0 tx=0).stop 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3360718041 shutdown_connections 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x559571d9bae0 0x559571da0e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d97f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcae8c00700 0 client.25051 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d86a60 0x559571d90a60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/3360718041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d87930 0x559571d90fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3360718041 >> 172.21.15.131:0/3360718041 conn(0x559571ce4a10 msgr2=0x559571d7cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3360718041 shutdown_connections 2024-05-10T05:32:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/3360718041 wait complete. 2024-05-10T05:32:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.026+0000 7fcb1db0cb80 20 client.25051 trim_cache size 0 max 16384 2024-05-10T05:32:54.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.047+0000 7fcb1db0cb80 1 Processor -- start 2024-05-10T05:32:54.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.047+0000 7fcb1db0cb80 1 -- start start 2024-05-10T05:32:54.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d86a60 0x559571d98490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d87930 0x559571d95940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d95e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571c9f010 con 0x559571d88280 2024-05-10T05:32:54.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571c415a0 con 0x559571d86a60 2024-05-10T05:32:54.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571d79ce0 con 0x559571d87930 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcaff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d95e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcaff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d95e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53980/0 (socket says 172.21.15.131:53980) 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcaff400700 1 -- 172.21.15.131:0/4052792632 learned_addr learned my addr 172.21.15.131:0/4052792632 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcafe000700 1 --2- 172.21.15.131:0/4052792632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d87930 0x559571d95940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.048+0000 7fcafea00700 1 --2- 172.21.15.131:0/4052792632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d86a60 0x559571d98490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.049+0000 7fcaff400700 1 -- 172.21.15.131:0/4052792632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d87930 msgr2=0x559571d95940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.049+0000 7fcaff400700 1 --2- 172.21.15.131:0/4052792632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d87930 0x559571d95940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.049+0000 7fcaff400700 1 -- 172.21.15.131:0/4052792632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d86a60 msgr2=0x559571d98490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.049+0000 7fcaff400700 1 --2- 172.21.15.131:0/4052792632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d86a60 0x559571d98490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.049+0000 7fcaff400700 1 -- 172.21.15.131:0/4052792632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559571acfb10 con 0x559571d88280 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.049+0000 7fcaff400700 1 --2- 172.21.15.131:0/4052792632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d95e80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80060b0 tx=0x7fcaf8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8764234f6105f841 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.049+0000 7fcaf4c00700 1 -- 172.21.15.131:0/4052792632 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf800e070 con 0x559571d88280 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.049+0000 7fcaf4c00700 1 -- 172.21.15.131:0/4052792632 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf8021040 con 0x559571d88280 2024-05-10T05:32:54.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.049+0000 7fcaf4c00700 1 -- 172.21.15.131:0/4052792632 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8004290 con 0x559571d88280 2024-05-10T05:32:54.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.050+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/4052792632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 msgr2=0x559571d95e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.050+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/4052792632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d95e80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80060b0 tx=0x7fcaf8009920 comp rx=0 tx=0).stop 2024-05-10T05:32:54.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.050+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/4052792632 shutdown_connections 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.050+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/4052792632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d88280 0x559571d95e80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.050+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/4052792632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d86a60 0x559571d98490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.050+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/4052792632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d87930 0x559571d95940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.050+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/4052792632 >> 172.21.15.131:0/4052792632 conn(0x559571ce4a10 msgr2=0x559571d870c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.050+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/4052792632 shutdown_connections 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.050+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/4052792632 wait complete. 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.051+0000 7fcb1db0cb80 1 -- start start 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.051+0000 7fcb1db0cb80 1 Processor -- start 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.051+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d86a60 0x559571d93f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.051+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d87930 0x559571d944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.051+0000 7fcb1db0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d88280 0x559571d95210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.051+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571d95750 con 0x559571d86a60 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.051+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571d958c0 con 0x559571d88280 2024-05-10T05:32:54.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.051+0000 7fcb1db0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559571d95a80 con 0x559571d87930 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.051+0000 7fcb1db0cb80 10 client.0 _ll_register_callbacks cb 0x559571c47ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcafea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d86a60 0x559571d93f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcafea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d86a60 0x559571d93f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53982/0 (socket says 172.21.15.131:53982) 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcafea00700 1 -- 172.21.15.131:0/2582949264 learned_addr learned my addr 172.21.15.131:0/2582949264 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcaff400700 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d88280 0x559571d95210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcafe000700 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d87930 0x559571d944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcafea00700 1 -- 172.21.15.131:0/2582949264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d87930 msgr2=0x559571d944c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcafea00700 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d87930 0x559571d944c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcafea00700 1 -- 172.21.15.131:0/2582949264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d88280 msgr2=0x559571d95210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcafea00700 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d88280 0x559571d95210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcafea00700 1 -- 172.21.15.131:0/2582949264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf8008e40 con 0x559571d86a60 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcafea00700 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d86a60 0x559571d93f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0014040 tx=0x7fcaf0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d8d238a2435e66 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcae8c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:32:54.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcae8c00700 1 -- 172.21.15.131:0/2582949264 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf001aee0 con 0x559571d86a60 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcae8c00700 1 -- 172.21.15.131:0/2582949264 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fcaf0019820 con 0x559571d86a60 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.052+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/2582949264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaf00044e0 con 0x559571d86a60 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcae8c00700 1 -- 172.21.15.131:0/2582949264 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf00052e0 con 0x559571d86a60 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcb1db0cb80 10 client.15030 fetch_fsmap learned FSMap version 26 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/2582949264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559571c68510 con 0x559571d86a60 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcae8c00700 1 -- 172.21.15.131:0/2582949264 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf0047020 con 0x559571d86a60 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcae8c00700 1 -- 172.21.15.131:0/2582949264 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcaf0036420 con 0x559571d86a60 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcb1db0cb80 10 client.15030 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcb1db0cb80 10 client.15030 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcb1db0cb80 20 client.15030 populate_metadata read hostname 'smithi131' 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x559571df4ec0 0x559571df3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcb1db0cb80 4 client.15030 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.053+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/2582949264 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559571c9ae70 con 0x559571df4ec0 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.054+0000 7fcafe000700 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x559571df4ec0 0x559571df3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.054+0000 7fcafe000700 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x559571df4ec0 0x559571df3420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e03a315698153ea4 server_cookie=da36772472b98e5f in_seq=0 out_seq=0 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.054+0000 7fcae8c00700 10 client.15030 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.054+0000 7fcae8c00700 1 -- 172.21.15.131:0/2582949264 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x559571c9ae70 con 0x559571df4ec0 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.054+0000 7fcae8c00700 10 client.15030 handle_command_reply: tid=0 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.054+0000 7fcb1db0cb80 1 client.15030 shutdown 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.054+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/2582949264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d86a60 msgr2=0x559571d93f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.054+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d86a60 0x559571d93f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0014040 tx=0x7fcaf0001e50 comp rx=0 tx=0).stop 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/2582949264 shutdown_connections 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x559571df4ec0 0x559571df3420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559571d86a60 0x559571d93f80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559571d88280 0x559571d95210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb1db0cb80 1 --2- 172.21.15.131:0/2582949264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559571d87930 0x559571d944c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcae8c00700 0 client.15030 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:32:54.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/2582949264 >> 172.21.15.131:0/2582949264 conn(0x559571ce4a10 msgr2=0x559571d8f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/2582949264 shutdown_connections 2024-05-10T05:32:54.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb1db0cb80 1 -- 172.21.15.131:0/2582949264 wait complete. 2024-05-10T05:32:54.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb1db0cb80 20 client.15030 trim_cache size 0 max 16384 2024-05-10T05:32:54.060 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:32:54.060 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:32:54.060 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-10T05:32:54.060 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24571, 2024-05-10T05:32:54.060 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-10T05:32:54.061 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-10T05:32:54.061 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-10T05:32:54.061 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 59.048651053999997, 2024-05-10T05:32:54.061 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-05-10T05:32:54.061 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-10T05:32:54.061 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T05:32:54.061 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 72.986466843000002 2024-05-10T05:32:54.061 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb05c00700 1 -- 172.21.15.131:0/3295983933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcae40a1260 msgr2=0x7fcae40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.055+0000 7fcb05c00700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcae40a1260 0x7fcae40a3770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x559571d86740 tx=0x7fcaf800b040 comp rx=0 tx=0).stop 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.056+0000 7fcb05c00700 1 -- 172.21.15.131:0/3295983933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb00102d00 msgr2=0x7fcb001983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.056+0000 7fcb05c00700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb00102d00 0x7fcb001983d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009fd0 tx=0x7fcaec0036f0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.056+0000 7fcb05c00700 1 -- 172.21.15.131:0/3295983933 shutdown_connections 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.056+0000 7fcb05c00700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcae40a1260 0x7fcae40a3770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.056+0000 7fcb05c00700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb001036a0 0x7fcb0019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.056+0000 7fcb05c00700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00101be0 0x7fcb00197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.056+0000 7fcb05c00700 1 --2- 172.21.15.131:0/3295983933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb00102d00 0x7fcb001983d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.056+0000 7fcb05c00700 1 -- 172.21.15.131:0/3295983933 >> 172.21.15.131:0/3295983933 conn(0x7fcb000fd210 msgr2=0x7fcb00105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.056+0000 7fcb05c00700 1 -- 172.21.15.131:0/3295983933 shutdown_connections 2024-05-10T05:32:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.056+0000 7fcb05c00700 1 -- 172.21.15.131:0/3295983933 wait complete. 2024-05-10T05:32:54.081 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-10T05:32:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:54 smithi157 ceph-mon[78427]: pgmap v271: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:54.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.343+0000 7fd17cc00700 1 -- 172.21.15.131:0/333255176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd178102b10 msgr2=0x7fd178102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.343+0000 7fd17cc00700 1 --2- 172.21.15.131:0/333255176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd178102b10 0x7fd178102f20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd164009990 tx=0x7fd164009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.343+0000 7fd17cc00700 1 -- 172.21.15.131:0/333255176 shutdown_connections 2024-05-10T05:32:54.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.343+0000 7fd17cc00700 1 --2- 172.21.15.131:0/333255176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781034b0 0x7fd178109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.343+0000 7fd17cc00700 1 --2- 172.21.15.131:0/333255176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1781019f0 0x7fd178101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.343+0000 7fd17cc00700 1 --2- 172.21.15.131:0/333255176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd178102b10 0x7fd178102f20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.343+0000 7fd17cc00700 1 -- 172.21.15.131:0/333255176 >> 172.21.15.131:0/333255176 conn(0x7fd1780fd000 msgr2=0x7fd1780ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.343+0000 7fd17cc00700 1 -- 172.21.15.131:0/333255176 shutdown_connections 2024-05-10T05:32:54.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.343+0000 7fd17cc00700 1 -- 172.21.15.131:0/333255176 wait complete. 2024-05-10T05:32:54.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.345+0000 7fd17cc00700 1 Processor -- start 2024-05-10T05:32:54.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.345+0000 7fd17cc00700 1 -- start start 2024-05-10T05:32:54.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.345+0000 7fd17cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781019f0 0x7fd178197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.345+0000 7fd17cc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd178102b10 0x7fd178198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.345+0000 7fd17cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1781034b0 0x7fd17819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.345+0000 7fd17cc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17819f850 con 0x7fd1781034b0 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.345+0000 7fd17cc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17819f990 con 0x7fd1781019f0 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd17cc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17819fb50 con 0x7fd178102b10 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd177e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1781034b0 0x7fd17819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd177e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1781034b0 0x7fd17819f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53998/0 (socket says 172.21.15.131:53998) 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd177e00700 1 -- 172.21.15.131:0/3599013170 learned_addr learned my addr 172.21.15.131:0/3599013170 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd177400700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781019f0 0x7fd178197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd176a00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd178102b10 0x7fd178198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd177e00700 1 -- 172.21.15.131:0/3599013170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd178102b10 msgr2=0x7fd178198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd177e00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd178102b10 0x7fd178198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd177e00700 1 -- 172.21.15.131:0/3599013170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781019f0 msgr2=0x7fd178197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd177e00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781019f0 0x7fd178197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd177e00700 1 -- 172.21.15.131:0/3599013170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009520 con 0x7fd1781034b0 2024-05-10T05:32:54.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.346+0000 7fd177e00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1781034b0 0x7fd17819f280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd170007ee0 tx=0x7fd17000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.347+0000 7fd16fe00700 1 -- 172.21.15.131:0/3599013170 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170017070 con 0x7fd1781034b0 2024-05-10T05:32:54.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.347+0000 7fd17cc00700 1 -- 172.21.15.131:0/3599013170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1640095f0 con 0x7fd1781034b0 2024-05-10T05:32:54.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.347+0000 7fd16fe00700 1 -- 172.21.15.131:0/3599013170 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd17000f040 con 0x7fd1781034b0 2024-05-10T05:32:54.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.347+0000 7fd16fe00700 1 -- 172.21.15.131:0/3599013170 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170005610 con 0x7fd1781034b0 2024-05-10T05:32:54.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.347+0000 7fd17cc00700 1 -- 172.21.15.131:0/3599013170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1781a0160 con 0x7fd1781034b0 2024-05-10T05:32:54.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.350+0000 7fd16fe00700 1 -- 172.21.15.131:0/3599013170 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd170004c40 con 0x7fd1781034b0 2024-05-10T05:32:54.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.351+0000 7fd19609ab80 1 Processor -- start 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.351+0000 7fd16fe00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd15c0a11b0 0x7fd15c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.351+0000 7fd19609ab80 1 -- start start 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.351+0000 7fd16fe00700 1 -- 172.21.15.131:0/3599013170 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd1700cc1c0 con 0x7fd1781034b0 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd176a00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd15c0a11b0 0x7fd15c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e4f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 0x55ef85e50b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e510a0 0x55ef85e579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd19609ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85d76100 con 0x55ef85e4f4e0 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd19609ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85d73ac0 con 0x55ef85e50750 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd19609ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85d6f3b0 con 0x55ef85e510a0 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd177400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e4f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd177400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e4f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54004/0 (socket says 172.21.15.131:54004) 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd177400700 1 -- 172.21.15.131:0/1225362465 learned_addr learned my addr 172.21.15.131:0/1225362465 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd177e00700 1 --2- 172.21.15.131:0/1225362465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e510a0 0x55ef85e579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd176a00700 1 --2- 172.21.15.131:0/1225362465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 0x55ef85e50b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.352+0000 7fd176a00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd15c0a11b0 0x7fd15c0a36c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd16400b2e0 tx=0x7fd16401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.353+0000 7fd177400700 1 -- 172.21.15.131:0/1225362465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e510a0 msgr2=0x55ef85e579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.353+0000 7fd177400700 1 --2- 172.21.15.131:0/1225362465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e510a0 0x55ef85e579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.353+0000 7fd177400700 1 -- 172.21.15.131:0/1225362465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 msgr2=0x55ef85e50b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.353+0000 7fd177400700 1 --2- 172.21.15.131:0/1225362465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 0x55ef85e50b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.353+0000 7fd177400700 1 -- 172.21.15.131:0/1225362465 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef85bf0360 con 0x55ef85e4f4e0 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.353+0000 7fd176a00700 1 --2- 172.21.15.131:0/1225362465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 0x55ef85e50b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.353+0000 7fd177400700 1 --2- 172.21.15.131:0/1225362465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e4f910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd168009e40 tx=0x7fd168011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf88e2030952d45 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.353+0000 7fd163e00700 1 -- 172.21.15.131:0/1225362465 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168004310 con 0x55ef85e4f4e0 2024-05-10T05:32:54.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.353+0000 7fd163e00700 1 -- 172.21.15.131:0/1225362465 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd168004470 con 0x55ef85e4f4e0 2024-05-10T05:32:54.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.353+0000 7fd163e00700 1 -- 172.21.15.131:0/1225362465 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168019940 con 0x55ef85e4f4e0 2024-05-10T05:32:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.354+0000 7fd19609ab80 1 -- 172.21.15.131:0/1225362465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 msgr2=0x55ef85e4f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.354+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1225362465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e4f910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd168009e40 tx=0x7fd168011720 comp rx=0 tx=0).stop 2024-05-10T05:32:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.354+0000 7fd19609ab80 1 -- 172.21.15.131:0/1225362465 shutdown_connections 2024-05-10T05:32:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.354+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1225362465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e4f910 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.354+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1225362465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e510a0 0x55ef85e579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.354+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1225362465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 0x55ef85e50b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.354+0000 7fd19609ab80 1 -- 172.21.15.131:0/1225362465 >> 172.21.15.131:0/1225362465 conn(0x55ef85db1a10 msgr2=0x55ef85e4a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.354+0000 7fd19609ab80 1 -- 172.21.15.131:0/1225362465 shutdown_connections 2024-05-10T05:32:54.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.354+0000 7fd19609ab80 1 -- 172.21.15.131:0/1225362465 wait complete. 2024-05-10T05:32:54.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.355+0000 7fd19609ab80 1 -- start start 2024-05-10T05:32:54.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.355+0000 7fd19609ab80 1 Processor -- start 2024-05-10T05:32:54.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.355+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e4f4e0 0x55ef85e59890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.355+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 0x55ef85e59dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.355+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e510a0 0x55ef85e60d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.355+0000 7fd19609ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85d0b510 con 0x55ef85e510a0 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.355+0000 7fd19609ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85d0b1b0 con 0x55ef85e50750 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.355+0000 7fd19609ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85bf2650 con 0x55ef85e4f4e0 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.355+0000 7fd19609ab80 10 client.0 _ll_register_callbacks cb 0x55ef85d14ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd177e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e510a0 0x55ef85e60d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd177e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e510a0 0x55ef85e60d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54006/0 (socket says 172.21.15.131:54006) 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd177400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e4f4e0 0x55ef85e59890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd177e00700 1 -- 172.21.15.131:0/1799473816 learned_addr learned my addr 172.21.15.131:0/1799473816 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd176a00700 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 0x55ef85e59dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd177e00700 1 -- 172.21.15.131:0/1799473816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e4f4e0 msgr2=0x55ef85e59890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd177e00700 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e4f4e0 0x55ef85e59890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd177e00700 1 -- 172.21.15.131:0/1799473816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 msgr2=0x55ef85e59dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd177e00700 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 0x55ef85e59dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd177e00700 1 -- 172.21.15.131:0/1799473816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd168009a70 con 0x55ef85e510a0 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd177e00700 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e510a0 0x55ef85e60d30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd1700c9a80 tx=0x7fd170098e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236000e65a369c11 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd160c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd160c00700 1 -- 172.21.15.131:0/1799473816 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170099450 con 0x55ef85e510a0 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd160c00700 1 -- 172.21.15.131:0/1799473816 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd17009d280 con 0x55ef85e510a0 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.356+0000 7fd19609ab80 1 -- 172.21.15.131:0/1799473816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef85cc8220 con 0x55ef85e510a0 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd160c00700 1 -- 172.21.15.131:0/1799473816 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1700a18d0 con 0x55ef85e510a0 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd19609ab80 10 client.15042 fetch_fsmap learned FSMap version 26 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd19609ab80 1 -- 172.21.15.131:0/1799473816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ef85b9cb10 con 0x55ef85e510a0 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd160c00700 1 -- 172.21.15.131:0/1799473816 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef85cc8220 con 0x55ef85e510a0 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd160c00700 1 -- 172.21.15.131:0/1799473816 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd1700a1440 con 0x55ef85e510a0 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd19609ab80 10 client.15042 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd19609ab80 10 client.15042 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T05:32:54.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd19609ab80 20 client.15042 populate_metadata read hostname 'smithi131' 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ef85e64970 0x55ef85e69c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd19609ab80 4 client.15042 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.357+0000 7fd19609ab80 1 -- 172.21.15.131:0/1799473816 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef85e1e730 con 0x55ef85e64970 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.358+0000 7fd177400700 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ef85e64970 0x55ef85e69c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.358+0000 7fd160c00700 10 client.15042 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.358+0000 7fd177400700 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ef85e64970 0x55ef85e69c70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde3a0ee5f296069 server_cookie=769f5df557278b63 in_seq=0 out_seq=0 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd160c00700 1 -- 172.21.15.131:0/1799473816 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef85e1e730 con 0x55ef85e64970 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd160c00700 10 client.15042 handle_command_reply: tid=0 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 client.15042 shutdown 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 -- 172.21.15.131:0/1799473816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e510a0 msgr2=0x55ef85e60d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e510a0 0x55ef85e60d30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd1700c9a80 tx=0x7fd170098e80 comp rx=0 tx=0).stop 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 -- 172.21.15.131:0/1799473816 shutdown_connections 2024-05-10T05:32:54.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ef85e64970 0x55ef85e69c70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e510a0 0x55ef85e60d30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd160c00700 0 client.15042 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:32:54.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e4f4e0 0x55ef85e59890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1799473816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e50750 0x55ef85e59dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 -- 172.21.15.131:0/1799473816 >> 172.21.15.131:0/1799473816 conn(0x55ef85db1a10 msgr2=0x55ef85e4e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 -- 172.21.15.131:0/1799473816 shutdown_connections 2024-05-10T05:32:54.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 1 -- 172.21.15.131:0/1799473816 wait complete. 2024-05-10T05:32:54.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.359+0000 7fd19609ab80 20 client.15042 trim_cache size 0 max 16384 2024-05-10T05:32:54.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.378+0000 7fd19609ab80 1 Processor -- start 2024-05-10T05:32:54.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.378+0000 7fd19609ab80 1 -- start start 2024-05-10T05:32:54.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.378+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e50b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.378+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e50750 0x55ef85e58e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.378+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 0x55ef85e4a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.378+0000 7fd19609ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85cc8220 con 0x55ef85e4f4e0 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.378+0000 7fd19609ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85d68150 con 0x55ef85e510a0 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.378+0000 7fd19609ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85d0e5a0 con 0x55ef85e50750 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e50b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e50b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54022/0 (socket says 172.21.15.131:54022) 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177400700 1 -- 172.21.15.131:0/1460975298 learned_addr learned my addr 172.21.15.131:0/1460975298 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177e00700 1 --2- 172.21.15.131:0/1460975298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 0x55ef85e4a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd176a00700 1 --2- 172.21.15.131:0/1460975298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e50750 0x55ef85e58e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177400700 1 -- 172.21.15.131:0/1460975298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e50750 msgr2=0x55ef85e58e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177400700 1 --2- 172.21.15.131:0/1460975298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e50750 0x55ef85e58e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177400700 1 -- 172.21.15.131:0/1460975298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 msgr2=0x55ef85e4a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177400700 1 --2- 172.21.15.131:0/1460975298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 0x55ef85e4a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177400700 1 -- 172.21.15.131:0/1460975298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef85d35510 con 0x55ef85e4f4e0 2024-05-10T05:32:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177e00700 1 --2- 172.21.15.131:0/1460975298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 0x55ef85e4a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:32:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd177400700 1 --2- 172.21.15.131:0/1460975298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e50b60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd168009d60 tx=0x7fd168005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262734e41766a057 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.379+0000 7fd16d200700 1 -- 172.21.15.131:0/1460975298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16802b070 con 0x55ef85e4f4e0 2024-05-10T05:32:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd16d200700 1 -- 172.21.15.131:0/1460975298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd168004190 con 0x55ef85e4f4e0 2024-05-10T05:32:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd16d200700 1 -- 172.21.15.131:0/1460975298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1680049b0 con 0x55ef85e4f4e0 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd19609ab80 1 -- 172.21.15.131:0/1460975298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 msgr2=0x55ef85e50b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1460975298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e50b60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd168009d60 tx=0x7fd168005dc0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd19609ab80 1 -- 172.21.15.131:0/1460975298 shutdown_connections 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1460975298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e4f4e0 0x55ef85e50b60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1460975298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 0x55ef85e4a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd19609ab80 1 --2- 172.21.15.131:0/1460975298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e50750 0x55ef85e58e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd19609ab80 1 -- 172.21.15.131:0/1460975298 >> 172.21.15.131:0/1460975298 conn(0x55ef85db1a10 msgr2=0x55ef85e49b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd19609ab80 1 -- 172.21.15.131:0/1460975298 shutdown_connections 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.380+0000 7fd19609ab80 1 -- 172.21.15.131:0/1460975298 wait complete. 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.381+0000 7fd19609ab80 1 -- start start 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.381+0000 7fd19609ab80 1 Processor -- start 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.381+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e4f4e0 0x55ef85e5cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.381+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e50750 0x55ef85e5d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.381+0000 7fd19609ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 0x55ef85e5e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.381+0000 7fd19609ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85e5e8f0 con 0x55ef85e50750 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.381+0000 7fd19609ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85e5ea60 con 0x55ef85e510a0 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.381+0000 7fd19609ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef85e5ec20 con 0x55ef85e4f4e0 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd19609ab80 10 client.0 _ll_register_callbacks cb 0x7fd170098ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd176a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e50750 0x55ef85e5d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd176a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e50750 0x55ef85e5d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54032/0 (socket says 172.21.15.131:54032) 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd176a00700 1 -- 172.21.15.131:0/2690934539 learned_addr learned my addr 172.21.15.131:0/2690934539 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd177400700 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e4f4e0 0x55ef85e5cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd177e00700 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 0x55ef85e5e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd176a00700 1 -- 172.21.15.131:0/2690934539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e4f4e0 msgr2=0x55ef85e5cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd176a00700 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e4f4e0 0x55ef85e5cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd176a00700 1 -- 172.21.15.131:0/2690934539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 msgr2=0x55ef85e5e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:32:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd176a00700 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 0x55ef85e5e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd176a00700 1 -- 172.21.15.131:0/2690934539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef85b9cb10 con 0x55ef85e50750 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd176a00700 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e50750 0x55ef85e5d4d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd164015000 tx=0x7fd164009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab65113e430c72eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.382+0000 7fd160c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd160c00700 1 -- 172.21.15.131:0/2690934539 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16402b040 con 0x55ef85e50750 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd160c00700 1 -- 172.21.15.131:0/2690934539 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd164024820 con 0x55ef85e50750 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd19609ab80 1 -- 172.21.15.131:0/2690934539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd168004470 con 0x55ef85e50750 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd160c00700 1 -- 172.21.15.131:0/2690934539 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1640249e0 con 0x55ef85e50750 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd19609ab80 10 client.15048 fetch_fsmap learned FSMap version 26 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd19609ab80 1 -- 172.21.15.131:0/2690934539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fd16801e040 con 0x55ef85e50750 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd160c00700 1 -- 172.21.15.131:0/2690934539 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd168004470 con 0x55ef85e50750 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd160c00700 1 -- 172.21.15.131:0/2690934539 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd164032c00 con 0x55ef85e50750 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd19609ab80 10 client.15048 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd19609ab80 10 client.15048 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T05:32:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd19609ab80 20 client.15048 populate_metadata read hostname 'smithi131' 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd19609ab80 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ef85ebe0d0 0x55ef85ebc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd19609ab80 4 client.15048 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.383+0000 7fd19609ab80 1 -- 172.21.15.131:0/2690934539 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ef85d67e70 con 0x55ef85ebe0d0 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.384+0000 7fd177400700 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ef85ebe0d0 0x55ef85ebc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.384+0000 7fd177400700 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ef85ebe0d0 0x55ef85ebc630 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c364f380a3cc7d8f server_cookie=a62305b4f20dfced in_seq=0 out_seq=0 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.384+0000 7fd160c00700 10 client.15048 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.384+0000 7fd160c00700 1 -- 172.21.15.131:0/2690934539 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ef85d67e70 con 0x55ef85ebe0d0 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.384+0000 7fd160c00700 10 client.15048 handle_command_reply: tid=0 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.384+0000 7fd19609ab80 1 client.15048 shutdown 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.384+0000 7fd19609ab80 1 -- 172.21.15.131:0/2690934539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e50750 msgr2=0x55ef85e5d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.384+0000 7fd19609ab80 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e50750 0x55ef85e5d4d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd164015000 tx=0x7fd164009310 comp rx=0 tx=0).stop 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd19609ab80 1 -- 172.21.15.131:0/2690934539 shutdown_connections 2024-05-10T05:32:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd19609ab80 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ef85ebe0d0 0x55ef85ebc630 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd19609ab80 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef85e510a0 0x55ef85e5e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd19609ab80 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ef85e4f4e0 0x55ef85e5cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd160c00700 0 client.15048 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd19609ab80 1 --2- 172.21.15.131:0/2690934539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ef85e50750 0x55ef85e5d4d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd19609ab80 1 -- 172.21.15.131:0/2690934539 >> 172.21.15.131:0/2690934539 conn(0x55ef85db1a10 msgr2=0x55ef85e582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd19609ab80 1 -- 172.21.15.131:0/2690934539 shutdown_connections 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd19609ab80 1 -- 172.21.15.131:0/2690934539 wait complete. 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd19609ab80 20 client.15048 trim_cache size 0 max 16384 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-10T05:32:54.391 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24581, 2024-05-10T05:32:54.392 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-10T05:32:54.392 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-10T05:32:54.392 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-10T05:32:54.392 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 62.834151372000001, 2024-05-10T05:32:54.392 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-05-10T05:32:54.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-10T05:32:54.392 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T05:32:54.392 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 69.055492651999998 2024-05-10T05:32:54.392 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd17cc00700 1 -- 172.21.15.131:0/3599013170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd15c0a11b0 msgr2=0x7fd15c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd17cc00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd15c0a11b0 0x7fd15c0a36c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd16400b2e0 tx=0x7fd16401a040 comp rx=0 tx=0).stop 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd17cc00700 1 -- 172.21.15.131:0/3599013170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1781034b0 msgr2=0x7fd17819f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.385+0000 7fd17cc00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1781034b0 0x7fd17819f280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd170007ee0 tx=0x7fd17000d3b0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.386+0000 7fd17cc00700 1 -- 172.21.15.131:0/3599013170 shutdown_connections 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.386+0000 7fd17cc00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd15c0a11b0 0x7fd15c0a36c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.386+0000 7fd17cc00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1781034b0 0x7fd17819f280 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.386+0000 7fd17cc00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781019f0 0x7fd178197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.386+0000 7fd17cc00700 1 --2- 172.21.15.131:0/3599013170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd178102b10 0x7fd178198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.386+0000 7fd17cc00700 1 -- 172.21.15.131:0/3599013170 >> 172.21.15.131:0/3599013170 conn(0x7fd1780fd000 msgr2=0x7fd178105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.386+0000 7fd17cc00700 1 -- 172.21.15.131:0/3599013170 shutdown_connections 2024-05-10T05:32:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:54.386+0000 7fd17cc00700 1 -- 172.21.15.131:0/3599013170 wait complete. 2024-05-10T05:32:54.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:54 smithi142 ceph-mon[77667]: pgmap v271: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:54.411 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:54 smithi131 ceph-mon[37264]: pgmap v271: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:54.414 INFO:tasks.cephadm:Setup complete, yielding 2024-05-10T05:32:54.418 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T05:32:54.427 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-05-10T05:32:54.428 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-10T05:32:54.784 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:56.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:55.998+0000 7fc162e00700 1 -- 172.21.15.131:0/3265717286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc15c103dc0 msgr2=0x7fc15c1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:56.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:55.998+0000 7fc162e00700 1 --2- 172.21.15.131:0/3265717286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc15c103dc0 0x7fc15c1041d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009990 tx=0x7fc14c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:55.998+0000 7fc162e00700 1 -- 172.21.15.131:0/3265717286 shutdown_connections 2024-05-10T05:32:56.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:55.998+0000 7fc162e00700 1 --2- 172.21.15.131:0/3265717286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc15c103dc0 0x7fc15c1041d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:55.998+0000 7fc162e00700 1 --2- 172.21.15.131:0/3265717286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc15c1058f0 0x7fc15c10c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:55.998+0000 7fc162e00700 1 --2- 172.21.15.131:0/3265717286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc15c104fa0 0x7fc15c1053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:55.998+0000 7fc162e00700 1 -- 172.21.15.131:0/3265717286 >> 172.21.15.131:0/3265717286 conn(0x7fc15c0ff530 msgr2=0x7fc15c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:56.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:55.999+0000 7fc162e00700 1 -- 172.21.15.131:0/3265717286 shutdown_connections 2024-05-10T05:32:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:55.999+0000 7fc162e00700 1 -- 172.21.15.131:0/3265717286 wait complete. 2024-05-10T05:32:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.000+0000 7fc162e00700 1 Processor -- start 2024-05-10T05:32:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.000+0000 7fc162e00700 1 -- start start 2024-05-10T05:32:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.000+0000 7fc162e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc15c103dc0 0x7fc15c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.000+0000 7fc162e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc15c104fa0 0x7fc15c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.000+0000 7fc162e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc15c1058f0 0x7fc15c19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.000+0000 7fc162e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c19bce0 con 0x7fc15c104fa0 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.000+0000 7fc162e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c19be20 con 0x7fc15c103dc0 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.000+0000 7fc162e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c19bfe0 con 0x7fc15c1058f0 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.000+0000 7fc15b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc15c104fa0 0x7fc15c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc15b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc15c104fa0 0x7fc15c19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54064/0 (socket says 172.21.15.131:54064) 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc15b400700 1 -- 172.21.15.131:0/3285362764 learned_addr learned my addr 172.21.15.131:0/3285362764 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc160e00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc15c1058f0 0x7fc15c19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc15be00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc15c103dc0 0x7fc15c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc15b400700 1 -- 172.21.15.131:0/3285362764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc15c1058f0 msgr2=0x7fc15c19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc15b400700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc15c1058f0 0x7fc15c19b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc15b400700 1 -- 172.21.15.131:0/3285362764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc15c103dc0 msgr2=0x7fc15c19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc15b400700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc15c103dc0 0x7fc15c19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc15b400700 1 -- 172.21.15.131:0/3285362764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c0095f0 con 0x7fc15c104fa0 2024-05-10T05:32:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc15b400700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc15c104fa0 0x7fc15c19a8d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc14800e9a0 tx=0x7fc14800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc158c00700 1 -- 172.21.15.131:0/3285362764 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148009790 con 0x7fc15c104fa0 2024-05-10T05:32:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.001+0000 7fc158c00700 1 -- 172.21.15.131:0/3285362764 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc148010430 con 0x7fc15c104fa0 2024-05-10T05:32:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.002+0000 7fc162e00700 1 -- 172.21.15.131:0/3285362764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c19c2c0 con 0x7fc15c104fa0 2024-05-10T05:32:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.002+0000 7fc158c00700 1 -- 172.21.15.131:0/3285362764 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148018680 con 0x7fc15c104fa0 2024-05-10T05:32:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.002+0000 7fc162e00700 1 -- 172.21.15.131:0/3285362764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c1a2aa0 con 0x7fc15c104fa0 2024-05-10T05:32:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.003+0000 7fc162e00700 1 -- 172.21.15.131:0/3285362764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c10b120 con 0x7fc15c104fa0 2024-05-10T05:32:56.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.005+0000 7fc158c00700 1 -- 172.21.15.131:0/3285362764 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc1480105a0 con 0x7fc15c104fa0 2024-05-10T05:32:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.006+0000 7fc158c00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc1400a11b0 0x7fc1400a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.006+0000 7fc158c00700 1 -- 172.21.15.131:0/3285362764 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc148014070 con 0x7fc15c104fa0 2024-05-10T05:32:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.006+0000 7fc15be00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc1400a11b0 0x7fc1400a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.007+0000 7fc15be00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc1400a11b0 0x7fc1400a36c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0098e0 tx=0x7fc14c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:56.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.011+0000 7fc158c00700 1 -- 172.21.15.131:0/3285362764 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc148052d60 con 0x7fc15c104fa0 2024-05-10T05:32:56.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.238+0000 7fc162e00700 1 -- 172.21.15.131:0/3285362764 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc15c0008d0 con 0x7fc1400a11b0 2024-05-10T05:32:56.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.239+0000 7fc158c00700 1 -- 172.21.15.131:0/3285362764 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc15c0008d0 con 0x7fc1400a11b0 2024-05-10T05:32:56.240 INFO:teuthology.orchestra.run.smithi131.stdout:Backend: cephadm 2024-05-10T05:32:56.240 INFO:teuthology.orchestra.run.smithi131.stdout:Available: Yes 2024-05-10T05:32:56.240 INFO:teuthology.orchestra.run.smithi131.stdout:Paused: No 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 -- 172.21.15.131:0/3285362764 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc1400a11b0 msgr2=0x7fc1400a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc1400a11b0 0x7fc1400a36c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0098e0 tx=0x7fc14c01a040 comp rx=0 tx=0).stop 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 -- 172.21.15.131:0/3285362764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc15c104fa0 msgr2=0x7fc15c19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc15c104fa0 0x7fc15c19a8d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc14800e9a0 tx=0x7fc14800ecb0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 -- 172.21.15.131:0/3285362764 shutdown_connections 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc1400a11b0 0x7fc1400a36c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc15c1058f0 0x7fc15c19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc15c103dc0 0x7fc15c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 --2- 172.21.15.131:0/3285362764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc15c104fa0 0x7fc15c19a8d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 -- 172.21.15.131:0/3285362764 >> 172.21.15.131:0/3285362764 conn(0x7fc15c0ff530 msgr2=0x7fc15c108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 -- 172.21.15.131:0/3285362764 shutdown_connections 2024-05-10T05:32:56.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:56.242+0000 7fc162e00700 1 -- 172.21.15.131:0/3285362764 wait complete. 2024-05-10T05:32:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:56 smithi157 ceph-mon[78427]: pgmap v272: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:56 smithi142 ceph-mon[77667]: pgmap v272: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:56.444 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:56 smithi131 ceph-mon[37264]: pgmap v272: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:32:56.653 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-10T05:32:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:58 smithi131 ceph-mon[37264]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:32:58 smithi131 ceph-mon[37264]: pgmap v273: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:58.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.273+0000 7fab9be00700 1 -- 172.21.15.131:0/1993573810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab94075130 msgr2=0x7fab94073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:58.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.273+0000 7fab9be00700 1 --2- 172.21.15.131:0/1993573810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab94075130 0x7fab94073600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab88009990 tx=0x7fab88009ca0 comp rx=0 tx=0).stop 2024-05-10T05:32:58.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.273+0000 7fab9be00700 1 -- 172.21.15.131:0/1993573810 shutdown_connections 2024-05-10T05:32:58.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.273+0000 7fab9be00700 1 --2- 172.21.15.131:0/1993573810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab94075130 0x7fab94073600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:58.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.273+0000 7fab9be00700 1 --2- 172.21.15.131:0/1993573810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab94109b50 0x7fab9410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:58.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.273+0000 7fab9be00700 1 --2- 172.21.15.131:0/1993573810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab94073b40 0x7fab94073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:58.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.273+0000 7fab9be00700 1 -- 172.21.15.131:0/1993573810 >> 172.21.15.131:0/1993573810 conn(0x7fab940fd2c0 msgr2=0x7fab940ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:58.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.273+0000 7fab9be00700 1 -- 172.21.15.131:0/1993573810 shutdown_connections 2024-05-10T05:32:58.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.274+0000 7fab9be00700 1 -- 172.21.15.131:0/1993573810 wait complete. 2024-05-10T05:32:58.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab9be00700 1 Processor -- start 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab9be00700 1 -- start start 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab9be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab94073b40 0x7fab9419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab9be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab94075130 0x7fab9419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab9be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab94109b50 0x7fab9419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab9be00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9419bc70 con 0x7fab94073b40 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab9be00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9419bdb0 con 0x7fab94109b50 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab9be00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9419bf70 con 0x7fab94075130 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab99400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab94073b40 0x7fab9419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab99400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab94073b40 0x7fab9419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54084/0 (socket says 172.21.15.131:54084) 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab99400700 1 -- 172.21.15.131:0/3630428743 learned_addr learned my addr 172.21.15.131:0/3630428743 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.275+0000 7fab98a00700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab94075130 0x7fab9419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab99e00700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab94109b50 0x7fab9419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab99400700 1 -- 172.21.15.131:0/3630428743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab94075130 msgr2=0x7fab9419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:58.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab99400700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab94075130 0x7fab9419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:58.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab99400700 1 -- 172.21.15.131:0/3630428743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab94109b50 msgr2=0x7fab9419b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:58.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab99400700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab94109b50 0x7fab9419b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:58.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab99400700 1 -- 172.21.15.131:0/3630428743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab880095f0 con 0x7fab94073b40 2024-05-10T05:32:58.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab99400700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab94073b40 0x7fab9419a370 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab880098e0 tx=0x7fab88009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:58.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab92000700 1 -- 172.21.15.131:0/3630428743 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8801d070 con 0x7fab94073b40 2024-05-10T05:32:58.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab92000700 1 -- 172.21.15.131:0/3630428743 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab88021410 con 0x7fab94073b40 2024-05-10T05:32:58.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab9be00700 1 -- 172.21.15.131:0/3630428743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9419c1f0 con 0x7fab94073b40 2024-05-10T05:32:58.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.276+0000 7fab92000700 1 -- 172.21.15.131:0/3630428743 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88017650 con 0x7fab94073b40 2024-05-10T05:32:58.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.277+0000 7fab9be00700 1 -- 172.21.15.131:0/3630428743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab941a2a30 con 0x7fab94073b40 2024-05-10T05:32:58.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.278+0000 7fab9be00700 1 -- 172.21.15.131:0/3630428743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab941089a0 con 0x7fab94073b40 2024-05-10T05:32:58.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.280+0000 7fab92000700 1 -- 172.21.15.131:0/3630428743 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fab88021580 con 0x7fab94073b40 2024-05-10T05:32:58.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.281+0000 7fab92000700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fab7c0a11b0 0x7fab7c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:32:58.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.281+0000 7fab92000700 1 -- 172.21.15.131:0/3630428743 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fab880ce520 con 0x7fab94073b40 2024-05-10T05:32:58.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.281+0000 7fab98a00700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fab7c0a11b0 0x7fab7c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:32:58.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.282+0000 7fab98a00700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fab7c0a11b0 0x7fab7c0a36c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab9419b2f0 tx=0x7fab80009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:32:58.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.283+0000 7fab92000700 1 -- 172.21.15.131:0/3630428743 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fab88097b80 con 0x7fab94073b40 2024-05-10T05:32:58.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:58 smithi157 ceph-mon[78427]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:58.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:32:58 smithi157 ceph-mon[78427]: pgmap v273: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:58 smithi142 ceph-mon[77667]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:32:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:32:58 smithi142 ceph-mon[77667]: pgmap v273: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:32:58.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.494+0000 7fab9be00700 1 -- 172.21.15.131:0/3630428743 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fab9410f770 con 0x7fab7c0a11b0 2024-05-10T05:32:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.507+0000 7fab92000700 1 -- 172.21.15.131:0/3630428743 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fab9410f770 con 0x7fab7c0a11b0 2024-05-10T05:32:58.508 INFO:teuthology.orchestra.run.smithi131.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-10T05:32:58.508 INFO:teuthology.orchestra.run.smithi131.stdout:mds.a smithi131 running (75s) 6s ago 74s 85.3M - 18.2.2-1330-g50174015 0e8066d08b37 48c990417b3c 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.b smithi142 running (73s) 7s ago 73s 98.2M - 18.2.2-1330-g50174015 0e8066d08b37 b7f8069e25b1 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.c smithi157 running (83s) 7s ago 83s 86.3M - 18.2.2-1330-g50174015 0e8066d08b37 861c5f373185 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.d smithi131 running (89s) 6s ago 88s 84.8M - 18.2.2-1330-g50174015 0e8066d08b37 3800de5cf8b2 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.e smithi142 running (87s) 7s ago 87s 88.8M - 18.2.2-1330-g50174015 0e8066d08b37 e53cf021c4c0 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.f smithi157 running (85s) 7s ago 85s 85.6M - 18.2.2-1330-g50174015 0e8066d08b37 632df6c0ff83 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.g smithi131 running (77s) 6s ago 77s 85.5M - 18.2.2-1330-g50174015 0e8066d08b37 23194a56e2d4 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.h smithi142 running (81s) 7s ago 81s 88.7M - 18.2.2-1330-g50174015 0e8066d08b37 fe535d287283 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.i smithi157 running (80s) 7s ago 79s 87.9M - 18.2.2-1330-g50174015 0e8066d08b37 58c40dc5938e 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.j smithi131 running (91s) 6s ago 91s 81.3M - 18.2.2-1330-g50174015 0e8066d08b37 b59369d96419 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.k smithi142 running (94s) 7s ago 93s 88.5M - 18.2.2-1330-g50174015 0e8066d08b37 f874d82e7854 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mds.l smithi157 running (95s) 7s ago 95s 82.4M - 18.2.2-1330-g50174015 0e8066d08b37 2b8a05138a28 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mgr.x smithi131 *:9283,8765 running (7m) 6s ago 7m 604M - 18.2.2-1330-g50174015 0e8066d08b37 ea4ddccd9275 2024-05-10T05:32:58.509 INFO:teuthology.orchestra.run.smithi131.stdout:mgr.y smithi142 *:8443,8765 running (5m) 7s ago 5m 525M - 18.2.2-1330-g50174015 0e8066d08b37 c657d2af24d8 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:mgr.z smithi157 *:8443,8765 running (5m) 7s ago 5m 526M - 18.2.2-1330-g50174015 0e8066d08b37 a8a1ead4ea42 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:mon.a smithi131 running (7m) 6s ago 7m 158M 2048M 18.2.2-1330-g50174015 0e8066d08b37 33ae1f0f9704 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:mon.b smithi142 running (6m) 7s ago 6m 129M 2048M 18.2.2-1330-g50174015 0e8066d08b37 5ec92ab4fbf1 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:mon.c smithi157 running (6m) 7s ago 6m 130M 2048M 18.2.2-1330-g50174015 0e8066d08b37 6e00848200d8 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 smithi131 running (5m) 6s ago 5m 206M 4317M 18.2.2-1330-g50174015 0e8066d08b37 be5e11e1345a 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 smithi142 running (5m) 7s ago 5m 210M 4278M 18.2.2-1330-g50174015 0e8066d08b37 711c10717a22 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 smithi157 running (4m) 7s ago 4m 212M 4278M 18.2.2-1330-g50174015 0e8066d08b37 e9f102e1fc86 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 smithi131 running (4m) 6s ago 4m 129M 4317M 18.2.2-1330-g50174015 0e8066d08b37 15909db36db6 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 smithi142 running (4m) 7s ago 4m 195M 4278M 18.2.2-1330-g50174015 0e8066d08b37 956a61dbd14c 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 smithi157 running (3m) 7s ago 3m 203M 4278M 18.2.2-1330-g50174015 0e8066d08b37 0a2666a84b8a 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 smithi131 running (3m) 6s ago 3m 193M 4317M 18.2.2-1330-g50174015 0e8066d08b37 107f8a46b03f 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 smithi142 running (3m) 7s ago 3m 202M 4278M 18.2.2-1330-g50174015 0e8066d08b37 465d73b1c3c2 2024-05-10T05:32:58.510 INFO:teuthology.orchestra.run.smithi131.stdout:osd.8 smithi157 running (2m) 7s ago 2m 127M 4278M 18.2.2-1330-g50174015 0e8066d08b37 c9c9434145fd 2024-05-10T05:32:58.511 INFO:teuthology.orchestra.run.smithi131.stdout:osd.9 smithi131 running (2m) 6s ago 2m 191M 4317M 18.2.2-1330-g50174015 0e8066d08b37 91461fce19ec 2024-05-10T05:32:58.511 INFO:teuthology.orchestra.run.smithi131.stdout:osd.10 smithi142 running (2m) 7s ago 2m 128M 4278M 18.2.2-1330-g50174015 0e8066d08b37 9e29df21a80a 2024-05-10T05:32:58.511 INFO:teuthology.orchestra.run.smithi131.stdout:osd.11 smithi157 running (111s) 7s ago 111s 121M 4278M 18.2.2-1330-g50174015 0e8066d08b37 34efc931278e 2024-05-10T05:32:58.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 -- 172.21.15.131:0/3630428743 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fab7c0a11b0 msgr2=0x7fab7c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fab7c0a11b0 0x7fab7c0a36c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab9419b2f0 tx=0x7fab80009260 comp rx=0 tx=0).stop 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 -- 172.21.15.131:0/3630428743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab94073b40 msgr2=0x7fab9419a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab94073b40 0x7fab9419a370 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab880098e0 tx=0x7fab88009580 comp rx=0 tx=0).stop 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 -- 172.21.15.131:0/3630428743 shutdown_connections 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fab7c0a11b0 0x7fab7c0a36c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab94073b40 0x7fab9419a370 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab94109b50 0x7fab9419b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 --2- 172.21.15.131:0/3630428743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab94075130 0x7fab9419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 -- 172.21.15.131:0/3630428743 >> 172.21.15.131:0/3630428743 conn(0x7fab940fd2c0 msgr2=0x7fab94105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 -- 172.21.15.131:0/3630428743 shutdown_connections 2024-05-10T05:32:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:32:58.510+0000 7fab9be00700 1 -- 172.21.15.131:0/3630428743 wait complete. 2024-05-10T05:32:58.891 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-10T05:32:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:33:00.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:00 smithi157 ceph-mon[78427]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:00.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:00 smithi157 ceph-mon[78427]: pgmap v274: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:00 smithi142 ceph-mon[77667]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:00 smithi142 ceph-mon[77667]: pgmap v274: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.447+0000 7f9ce8e00700 1 -- 172.21.15.131:0/868870022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ce41058f0 msgr2=0x7f9ce410c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.447+0000 7f9ce8e00700 1 --2- 172.21.15.131:0/868870022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ce41058f0 0x7f9ce410c280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b0a0 tx=0x7f9cdc00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.447+0000 7f9ce8e00700 1 -- 172.21.15.131:0/868870022 shutdown_connections 2024-05-10T05:33:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.447+0000 7f9ce8e00700 1 --2- 172.21.15.131:0/868870022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ce41058f0 0x7f9ce410c280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.447+0000 7f9ce8e00700 1 --2- 172.21.15.131:0/868870022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4103dc0 0x7f9ce41041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.447+0000 7f9ce8e00700 1 --2- 172.21.15.131:0/868870022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce4104fa0 0x7f9ce41053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.447+0000 7f9ce8e00700 1 -- 172.21.15.131:0/868870022 >> 172.21.15.131:0/868870022 conn(0x7f9ce40ff510 msgr2=0x7f9ce41019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.447+0000 7f9ce8e00700 1 -- 172.21.15.131:0/868870022 shutdown_connections 2024-05-10T05:33:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.448+0000 7f9ce8e00700 1 -- 172.21.15.131:0/868870022 wait complete. 2024-05-10T05:33:00.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.448+0000 7f9ce8e00700 1 Processor -- start 2024-05-10T05:33:00.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce8e00700 1 -- start start 2024-05-10T05:33:00.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce8e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4103dc0 0x7f9ce419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:00.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce8e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ce4104fa0 0x7f9ce419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:00.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce8e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce41058f0 0x7f9ce419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:00.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce8e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce419bdc0 con 0x7f9ce4104fa0 2024-05-10T05:33:00.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce8e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce419bf00 con 0x7f9ce4103dc0 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce8e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce419c0c0 con 0x7f9ce41058f0 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ce4104fa0 0x7f9ce419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4103dc0 0x7f9ce419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ce4104fa0 0x7f9ce419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54106/0 (socket says 172.21.15.131:54106) 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce1400700 1 -- 172.21.15.131:0/2974874047 learned_addr learned my addr 172.21.15.131:0/2974874047 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.449+0000 7f9ce2800700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce41058f0 0x7f9ce419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9ce1400700 1 -- 172.21.15.131:0/2974874047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce41058f0 msgr2=0x7f9ce419b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9ce1400700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce41058f0 0x7f9ce419b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9ce1400700 1 -- 172.21.15.131:0/2974874047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4103dc0 msgr2=0x7f9ce419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9ce1400700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4103dc0 0x7f9ce419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9ce1400700 1 -- 172.21.15.131:0/2974874047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc009d00 con 0x7f9ce4104fa0 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9ce1e00700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4103dc0 0x7f9ce419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9ce1400700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ce4104fa0 0x7f9ce419a9b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000e9a0 tx=0x7f9cd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:00.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9cdaa00700 1 -- 172.21.15.131:0/2974874047 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0009790 con 0x7f9ce4104fa0 2024-05-10T05:33:00.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9cdaa00700 1 -- 172.21.15.131:0/2974874047 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cd0010430 con 0x7f9ce4104fa0 2024-05-10T05:33:00.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9cdaa00700 1 -- 172.21.15.131:0/2974874047 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0018680 con 0x7f9ce4104fa0 2024-05-10T05:33:00.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9ce8e00700 1 -- 172.21.15.131:0/2974874047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce419c3a0 con 0x7f9ce4104fa0 2024-05-10T05:33:00.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.450+0000 7f9ce8e00700 1 -- 172.21.15.131:0/2974874047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce41a2b80 con 0x7f9ce4104fa0 2024-05-10T05:33:00.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.452+0000 7f9ce8e00700 1 -- 172.21.15.131:0/2974874047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce410b120 con 0x7f9ce4104fa0 2024-05-10T05:33:00.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.454+0000 7f9cdaa00700 1 -- 172.21.15.131:0/2974874047 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9cd00105a0 con 0x7f9ce4104fa0 2024-05-10T05:33:00.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.455+0000 7f9cdaa00700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9cc80a11b0 0x7f9cc80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:00.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.455+0000 7f9cdaa00700 1 -- 172.21.15.131:0/2974874047 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9cd0014070 con 0x7f9ce4104fa0 2024-05-10T05:33:00.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.455+0000 7f9ce1e00700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9cc80a11b0 0x7f9cc80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:00.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.456+0000 7f9ce1e00700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9cc80a11b0 0x7f9cc80a36c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4103b80 tx=0x7f9cd4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:00.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:00 smithi131 ceph-mon[37264]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:00.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:00 smithi131 ceph-mon[37264]: pgmap v274: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:00.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.459+0000 7f9cdaa00700 1 -- 172.21.15.131:0/2974874047 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9cd0052d60 con 0x7f9ce4104fa0 2024-05-10T05:33:00.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.672+0000 7f9ce8e00700 1 -- 172.21.15.131:0/2974874047 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ce410f9b0 con 0x7f9cc80a11b0 2024-05-10T05:33:00.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.677+0000 7f9cdaa00700 1 -- 172.21.15.131:0/2974874047 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f9ce410f9b0 con 0x7f9cc80a11b0 2024-05-10T05:33:00.678 INFO:teuthology.orchestra.run.smithi131.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-10T05:33:00.678 INFO:teuthology.orchestra.run.smithi131.stdout:mds.all 12/12 9s ago 99s smithi131=a;smithi131=d;smithi131=g;smithi131=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-10T05:33:00.678 INFO:teuthology.orchestra.run.smithi131.stdout:mgr 3/3 9s ago 5m smithi131=x;smithi142=y;smithi157=z;count:3 2024-05-10T05:33:00.679 INFO:teuthology.orchestra.run.smithi131.stdout:mon 3/3 9s ago 6m smithi131:172.21.15.131=a;smithi142:172.21.15.142=b;smithi157:172.21.15.157=c;count:3 2024-05-10T05:33:00.679 INFO:teuthology.orchestra.run.smithi131.stdout:osd 12 9s ago - 2024-05-10T05:33:00.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.680+0000 7f9ce8e00700 1 -- 172.21.15.131:0/2974874047 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9cc80a11b0 msgr2=0x7f9cc80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:00.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.680+0000 7f9ce8e00700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9cc80a11b0 0x7f9cc80a36c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4103b80 tx=0x7f9cd4009260 comp rx=0 tx=0).stop 2024-05-10T05:33:00.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.680+0000 7f9ce8e00700 1 -- 172.21.15.131:0/2974874047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ce4104fa0 msgr2=0x7f9ce419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.680+0000 7f9ce8e00700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ce4104fa0 0x7f9ce419a9b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000e9a0 tx=0x7f9cd000ecb0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.680+0000 7f9ce8e00700 1 -- 172.21.15.131:0/2974874047 shutdown_connections 2024-05-10T05:33:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.680+0000 7f9ce8e00700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9cc80a11b0 0x7f9cc80a36c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.680+0000 7f9ce8e00700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce41058f0 0x7f9ce419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.680+0000 7f9ce8e00700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4103dc0 0x7f9ce419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.680+0000 7f9ce8e00700 1 --2- 172.21.15.131:0/2974874047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ce4104fa0 0x7f9ce419a9b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.680+0000 7f9ce8e00700 1 -- 172.21.15.131:0/2974874047 >> 172.21.15.131:0/2974874047 conn(0x7f9ce40ff510 msgr2=0x7f9ce4108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.681+0000 7f9ce8e00700 1 -- 172.21.15.131:0/2974874047 shutdown_connections 2024-05-10T05:33:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:00.681+0000 7f9ce8e00700 1 -- 172.21.15.131:0/2974874047 wait complete. 2024-05-10T05:33:01.086 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-10T05:33:01.448 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:33:02.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:02 smithi157 ceph-mon[78427]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:02.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:02 smithi157 ceph-mon[78427]: pgmap v275: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:33:02.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:02 smithi142 ceph-mon[77667]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:02.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:02 smithi142 ceph-mon[77667]: pgmap v275: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:33:02.449 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:02 smithi131 ceph-mon[37264]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:02.449 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:02 smithi131 ceph-mon[37264]: pgmap v275: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:33:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.749+0000 7f1730200700 1 -- 172.21.15.131:0/3140825496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1728101ca0 msgr2=0x7f17281020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.749+0000 7f1730200700 1 --2- 172.21.15.131:0/3140825496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1728101ca0 0x7f17281020b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f171c009990 tx=0x7f171c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.749+0000 7f1730200700 1 -- 172.21.15.131:0/3140825496 shutdown_connections 2024-05-10T05:33:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.749+0000 7f1730200700 1 --2- 172.21.15.131:0/3140825496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1728101ca0 0x7f17281020b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.749+0000 7f1730200700 1 --2- 172.21.15.131:0/3140825496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728103770 0x7f172810a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.749+0000 7f1730200700 1 --2- 172.21.15.131:0/3140825496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1728102dd0 0x7f17281031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.749+0000 7f1730200700 1 -- 172.21.15.131:0/3140825496 >> 172.21.15.131:0/3140825496 conn(0x7f17280fd300 msgr2=0x7f17280ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.749+0000 7f1730200700 1 -- 172.21.15.131:0/3140825496 shutdown_connections 2024-05-10T05:33:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.749+0000 7f1730200700 1 -- 172.21.15.131:0/3140825496 wait complete. 2024-05-10T05:33:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.750+0000 7f1730200700 1 Processor -- start 2024-05-10T05:33:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.750+0000 7f1730200700 1 -- start start 2024-05-10T05:33:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f1730200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728101ca0 0x7f172819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f1730200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1728102dd0 0x7f172819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f1730200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1728103770 0x7f172819b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f1730200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172819bcc0 con 0x7f1728103770 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f1730200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172819be00 con 0x7f1728102dd0 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f1730200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172819bfc0 con 0x7f1728101ca0 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f172e200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1728103770 0x7f172819b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f172d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728101ca0 0x7f172819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f172d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728101ca0 0x7f172819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59366/0 (socket says 172.21.15.131:59366) 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f172e200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1728103770 0x7f172819b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54122/0 (socket says 172.21.15.131:54122) 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f172d800700 1 -- 172.21.15.131:0/1257452335 learned_addr learned my addr 172.21.15.131:0/1257452335 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.751+0000 7f172ce00700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1728102dd0 0x7f172819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f172d800700 1 -- 172.21.15.131:0/1257452335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1728102dd0 msgr2=0x7f172819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f172d800700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1728102dd0 0x7f172819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f172d800700 1 -- 172.21.15.131:0/1257452335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1728103770 msgr2=0x7f172819b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f172d800700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1728103770 0x7f172819b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:02.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f172d800700 1 -- 172.21.15.131:0/1257452335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c0095f0 con 0x7f1728101ca0 2024-05-10T05:33:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f172e200700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1728103770 0x7f172819b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f172d800700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728101ca0 0x7f172819a370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f171c0098e0 tx=0x7f171c005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f1726000700 1 -- 172.21.15.131:0/1257452335 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c01d070 con 0x7f1728101ca0 2024-05-10T05:33:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f1730200700 1 -- 172.21.15.131:0/1257452335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172819c240 con 0x7f1728101ca0 2024-05-10T05:33:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f1726000700 1 -- 172.21.15.131:0/1257452335 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171c021410 con 0x7f1728101ca0 2024-05-10T05:33:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f1726000700 1 -- 172.21.15.131:0/1257452335 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c017650 con 0x7f1728101ca0 2024-05-10T05:33:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.752+0000 7f1730200700 1 -- 172.21.15.131:0/1257452335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17281a2a80 con 0x7f1728101ca0 2024-05-10T05:33:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.753+0000 7f1730200700 1 -- 172.21.15.131:0/1257452335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1728100fa0 con 0x7f1728101ca0 2024-05-10T05:33:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.756+0000 7f1726000700 1 -- 172.21.15.131:0/1257452335 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f171c021580 con 0x7f1728101ca0 2024-05-10T05:33:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.757+0000 7f1726000700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17100a11b0 0x7f17100a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.757+0000 7f1726000700 1 -- 172.21.15.131:0/1257452335 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f171c0ce330 con 0x7f1728101ca0 2024-05-10T05:33:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.757+0000 7f172ce00700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17100a11b0 0x7f17100a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.758+0000 7f172ce00700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17100a11b0 0x7f17100a36c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f172819b340 tx=0x7f1714009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.759+0000 7f1726000700 1 -- 172.21.15.131:0/1257452335 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f171c0979f0 con 0x7f1728101ca0 2024-05-10T05:33:02.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.971+0000 7f1730200700 1 -- 172.21.15.131:0/1257452335 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f172810d670 con 0x7f17100a11b0 2024-05-10T05:33:02.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.973+0000 7f1726000700 1 -- 172.21.15.131:0/1257452335 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f172810d670 con 0x7f17100a11b0 2024-05-10T05:33:02.974 INFO:teuthology.orchestra.run.smithi131.stdout:HOST ADDR LABELS STATUS 2024-05-10T05:33:02.974 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131 172.21.15.131 2024-05-10T05:33:02.974 INFO:teuthology.orchestra.run.smithi131.stdout:smithi142 172.21.15.142 2024-05-10T05:33:02.974 INFO:teuthology.orchestra.run.smithi131.stdout:smithi157 172.21.15.157 2024-05-10T05:33:02.974 INFO:teuthology.orchestra.run.smithi131.stdout:3 hosts in cluster 2024-05-10T05:33:02.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.975+0000 7f1730200700 1 -- 172.21.15.131:0/1257452335 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17100a11b0 msgr2=0x7f17100a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.975+0000 7f1730200700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17100a11b0 0x7f17100a36c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f172819b340 tx=0x7f1714009260 comp rx=0 tx=0).stop 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.975+0000 7f1730200700 1 -- 172.21.15.131:0/1257452335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728101ca0 msgr2=0x7f172819a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.975+0000 7f1730200700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728101ca0 0x7f172819a370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f171c0098e0 tx=0x7f171c005b00 comp rx=0 tx=0).stop 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.975+0000 7f1730200700 1 -- 172.21.15.131:0/1257452335 shutdown_connections 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.975+0000 7f1730200700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f17100a11b0 0x7f17100a36c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.975+0000 7f1730200700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1728103770 0x7f172819b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.975+0000 7f1730200700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728101ca0 0x7f172819a370 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.975+0000 7f1730200700 1 --2- 172.21.15.131:0/1257452335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1728102dd0 0x7f172819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.975+0000 7f1730200700 1 -- 172.21.15.131:0/1257452335 >> 172.21.15.131:0/1257452335 conn(0x7f17280fd300 msgr2=0x7f1728073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.976+0000 7f1730200700 1 -- 172.21.15.131:0/1257452335 shutdown_connections 2024-05-10T05:33:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:02.976+0000 7f1730200700 1 -- 172.21.15.131:0/1257452335 wait complete. 2024-05-10T05:33:03.418 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-10T05:33:03.783 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:33:04.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:04 smithi131 ceph-mon[37264]: pgmap v276: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:04.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:04 smithi131 ceph-mon[37264]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:04 smithi157 ceph-mon[78427]: pgmap v276: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:04 smithi157 ceph-mon[78427]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:04.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:04 smithi142 ceph-mon[77667]: pgmap v276: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:04.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:04 smithi142 ceph-mon[77667]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.034+0000 7f0dd9e00700 1 -- 172.21.15.131:0/2471199800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd4103770 msgr2=0x7f0dd410a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.034+0000 7f0dd9e00700 1 --2- 172.21.15.131:0/2471199800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd4103770 0x7f0dd410a020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00b0a0 tx=0x7f0dcc00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.034+0000 7f0dd9e00700 1 -- 172.21.15.131:0/2471199800 shutdown_connections 2024-05-10T05:33:05.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.034+0000 7f0dd9e00700 1 --2- 172.21.15.131:0/2471199800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd4103770 0x7f0dd410a020 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.034+0000 7f0dd9e00700 1 --2- 172.21.15.131:0/2471199800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd4101ca0 0x7f0dd41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.034+0000 7f0dd9e00700 1 --2- 172.21.15.131:0/2471199800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4102dd0 0x7f0dd41031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.034+0000 7f0dd9e00700 1 -- 172.21.15.131:0/2471199800 >> 172.21.15.131:0/2471199800 conn(0x7f0dd40fd300 msgr2=0x7f0dd40ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:05.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.034+0000 7f0dd9e00700 1 -- 172.21.15.131:0/2471199800 shutdown_connections 2024-05-10T05:33:05.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.034+0000 7f0dd9e00700 1 -- 172.21.15.131:0/2471199800 wait complete. 2024-05-10T05:33:05.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.035+0000 7f0dd9e00700 1 Processor -- start 2024-05-10T05:33:05.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd9e00700 1 -- start start 2024-05-10T05:33:05.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd4101ca0 0x7f0dd41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:05.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4102dd0 0x7f0dd4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:05.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd4103770 0x7f0dd41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:05.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd9e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4199b40 con 0x7f0dd4101ca0 2024-05-10T05:33:05.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd9e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4199c80 con 0x7f0dd4102dd0 2024-05-10T05:33:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd9e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4199e40 con 0x7f0dd4103770 2024-05-10T05:33:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd4101ca0 0x7f0dd41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd4101ca0 0x7f0dd41981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49478/0 (socket says 172.21.15.131:49478) 2024-05-10T05:33:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd2a00700 1 -- 172.21.15.131:0/3861313946 learned_addr learned my addr 172.21.15.131:0/3861313946 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd3400700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd4103770 0x7f0dd41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.036+0000 7f0dd2000700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4102dd0 0x7f0dd4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd2a00700 1 -- 172.21.15.131:0/3861313946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd4103770 msgr2=0x7f0dd41994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd2a00700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd4103770 0x7f0dd41994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd2a00700 1 -- 172.21.15.131:0/3861313946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4102dd0 msgr2=0x7f0dd4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd2a00700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4102dd0 0x7f0dd4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd2a00700 1 -- 172.21.15.131:0/3861313946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc009d00 con 0x7f0dd4101ca0 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd2a00700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd4101ca0 0x7f0dd41981f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400e940 tx=0x7f0dc400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd8a00700 1 -- 172.21.15.131:0/3861313946 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4004220 con 0x7f0dd4101ca0 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd8a00700 1 -- 172.21.15.131:0/3861313946 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dc400cca0 con 0x7f0dd4101ca0 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd9e00700 1 -- 172.21.15.131:0/3861313946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd419a120 con 0x7f0dd4101ca0 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd8a00700 1 -- 172.21.15.131:0/3861313946 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4003c00 con 0x7f0dd4101ca0 2024-05-10T05:33:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.037+0000 7f0dd9e00700 1 -- 172.21.15.131:0/3861313946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd41a0900 con 0x7f0dd4101ca0 2024-05-10T05:33:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.039+0000 7f0dd9e00700 1 -- 172.21.15.131:0/3861313946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd4108ec0 con 0x7f0dd4101ca0 2024-05-10T05:33:05.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.040+0000 7f0dd8a00700 1 -- 172.21.15.131:0/3861313946 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0dc400c7e0 con 0x7f0dd4101ca0 2024-05-10T05:33:05.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.042+0000 7f0dd8a00700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0db80a1230 0x7f0db80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:05.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.042+0000 7f0dd8a00700 1 -- 172.21.15.131:0/3861313946 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0dc4014070 con 0x7f0dd4101ca0 2024-05-10T05:33:05.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.042+0000 7f0dd2000700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0db80a1230 0x7f0db80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:05.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.043+0000 7f0dd2000700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0db80a1230 0x7f0db80a3740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0dd41991c0 tx=0x7f0dc00073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:05.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.046+0000 7f0dd8a00700 1 -- 172.21.15.131:0/3861313946 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0dc4096260 con 0x7f0dd4101ca0 2024-05-10T05:33:05.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.260+0000 7f0dd9e00700 1 -- 172.21.15.131:0/3861313946 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dd410d6a0 con 0x7f0db80a1230 2024-05-10T05:33:05.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.262+0000 7f0dd8a00700 1 -- 172.21.15.131:0/3861313946 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f0dd410d6a0 con 0x7f0db80a1230 2024-05-10T05:33:05.262 INFO:teuthology.orchestra.run.smithi131.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-10T05:33:05.263 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300N4400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-10T05:33:05.263 INFO:teuthology.orchestra.run.smithi131.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-10T05:33:05.263 INFO:teuthology.orchestra.run.smithi131.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 113s ago Has a FileSystem, LVM detected 2024-05-10T05:33:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 -- 172.21.15.131:0/3861313946 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0db80a1230 msgr2=0x7f0db80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0db80a1230 0x7f0db80a3740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0dd41991c0 tx=0x7f0dc00073d0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 -- 172.21.15.131:0/3861313946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd4101ca0 msgr2=0x7f0dd41981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd4101ca0 0x7f0dd41981f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400e940 tx=0x7f0dc400ec50 comp rx=0 tx=0).stop 2024-05-10T05:33:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 -- 172.21.15.131:0/3861313946 shutdown_connections 2024-05-10T05:33:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0db80a1230 0x7f0db80a3740 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd4101ca0 0x7f0dd41981f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd4103770 0x7f0dd41994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 --2- 172.21.15.131:0/3861313946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4102dd0 0x7f0dd4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:05.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 -- 172.21.15.131:0/3861313946 >> 172.21.15.131:0/3861313946 conn(0x7f0dd40fd300 msgr2=0x7f0dd4106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:05.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 -- 172.21.15.131:0/3861313946 shutdown_connections 2024-05-10T05:33:05.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:05.264+0000 7f0dd9e00700 1 -- 172.21.15.131:0/3861313946 wait complete. 2024-05-10T05:33:05.651 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T05:33:05.660 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-05-10T05:33:05.660 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-10T05:33:06.022 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:33:06.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:06 smithi157 ceph-mon[78427]: pgmap v277: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:06.371 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:06 smithi131 ceph-mon[37264]: pgmap v277: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:06 smithi142 ceph-mon[77667]: pgmap v277: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:07.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.281+0000 7f7baa000700 1 -- 172.21.15.131:0/66257198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ba4073b40 msgr2=0x7f7ba4073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.281+0000 7f7baa000700 1 --2- 172.21.15.131:0/66257198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ba4073b40 0x7f7ba4073fd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009990 tx=0x7f7b90009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.281+0000 7f7baa000700 1 -- 172.21.15.131:0/66257198 shutdown_connections 2024-05-10T05:33:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.281+0000 7f7baa000700 1 --2- 172.21.15.131:0/66257198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba4109b50 0x7f7ba410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.281+0000 7f7baa000700 1 --2- 172.21.15.131:0/66257198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4075130 0x7f7ba4073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.281+0000 7f7baa000700 1 --2- 172.21.15.131:0/66257198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ba4073b40 0x7f7ba4073fd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.281+0000 7f7baa000700 1 -- 172.21.15.131:0/66257198 >> 172.21.15.131:0/66257198 conn(0x7f7ba40fd2c0 msgr2=0x7f7ba40ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.281+0000 7f7baa000700 1 -- 172.21.15.131:0/66257198 shutdown_connections 2024-05-10T05:33:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.281+0000 7f7baa000700 1 -- 172.21.15.131:0/66257198 wait complete. 2024-05-10T05:33:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.282+0000 7f7baa000700 1 Processor -- start 2024-05-10T05:33:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7baa000700 1 -- start start 2024-05-10T05:33:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7baa000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba4073b40 0x7f7ba419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7baa000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4075130 0x7f7ba419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7baa000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ba4109b50 0x7f7ba419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7baa000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba419bc40 con 0x7f7ba4109b50 2024-05-10T05:33:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7baa000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba419bd80 con 0x7f7ba4075130 2024-05-10T05:33:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7baa000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba419bf40 con 0x7f7ba4073b40 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7ba3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ba4109b50 0x7f7ba419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7ba3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ba4109b50 0x7f7ba419b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49484/0 (socket says 172.21.15.131:49484) 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7ba3e00700 1 -- 172.21.15.131:0/3478195356 learned_addr learned my addr 172.21.15.131:0/3478195356 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7ba3400700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba4073b40 0x7f7ba419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.283+0000 7f7ba2a00700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4075130 0x7f7ba419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7ba3e00700 1 -- 172.21.15.131:0/3478195356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba4073b40 msgr2=0x7f7ba419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7ba3e00700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba4073b40 0x7f7ba419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7ba3e00700 1 -- 172.21.15.131:0/3478195356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4075130 msgr2=0x7f7ba419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7ba3e00700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4075130 0x7f7ba419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7ba3e00700 1 -- 172.21.15.131:0/3478195356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900095f0 con 0x7f7ba4109b50 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7ba3e00700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ba4109b50 0x7f7ba419b5e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00ef30 tx=0x7f7b9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7b9be00700 1 -- 172.21.15.131:0/3478195356 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c018070 con 0x7f7ba4109b50 2024-05-10T05:33:07.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7b9be00700 1 -- 172.21.15.131:0/3478195356 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b9c00f040 con 0x7f7ba4109b50 2024-05-10T05:33:07.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7baa000700 1 -- 172.21.15.131:0/3478195356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba419c1c0 con 0x7f7ba4109b50 2024-05-10T05:33:07.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7b9be00700 1 -- 172.21.15.131:0/3478195356 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00cd00 con 0x7f7ba4109b50 2024-05-10T05:33:07.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.284+0000 7f7baa000700 1 -- 172.21.15.131:0/3478195356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba41a2a00 con 0x7f7ba4109b50 2024-05-10T05:33:07.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.285+0000 7f7baa000700 1 -- 172.21.15.131:0/3478195356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba41089a0 con 0x7f7ba4109b50 2024-05-10T05:33:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.287+0000 7f7b9be00700 1 -- 172.21.15.131:0/3478195356 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7b9c003a00 con 0x7f7ba4109b50 2024-05-10T05:33:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.289+0000 7f7b9be00700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b880a1230 0x7f7b880a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.289+0000 7f7b9be00700 1 -- 172.21.15.131:0/3478195356 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f7b9c0cc5a0 con 0x7f7ba4109b50 2024-05-10T05:33:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.289+0000 7f7ba3400700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b880a1230 0x7f7b880a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.290+0000 7f7ba3400700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b880a1230 0x7f7b880a3740 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4074e30 tx=0x7f7b94009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:07.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.293+0000 7f7b9be00700 1 -- 172.21.15.131:0/3478195356 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7b9c095c00 con 0x7f7ba4109b50 2024-05-10T05:33:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:07 smithi142 ceph-mon[77667]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:07.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:07 smithi131 ceph-mon[37264]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.539+0000 7f7baa000700 1 -- 172.21.15.131:0/3478195356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7ba4100fa0 con 0x7f7ba4109b50 2024-05-10T05:33:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.540+0000 7f7b9be00700 1 -- 172.21.15.131:0/3478195356 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2935 (secure 0 0 0) 0x7f7b9c099220 con 0x7f7ba4109b50 2024-05-10T05:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:e26 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:legacy client fscid: 1 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:Filesystem 'cephfs' (1) 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:fs_name cephfs 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 26 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-05-10T05:31:51.547495+0000 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:modified 2024-05-10T05:31:58.482427+0000 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:tableserver 0 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:root 0 2024-05-10T05:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:session_timeout 300 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:session_autoclose 300 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:max_file_size 1099511627776 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:max_xattr_size 65536 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:required_client_features {} 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:last_failure 0 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:last_failure_osd_epoch 0 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:max_mds 5 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:in 0,1,2,3,4 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:up {0=24581,1=24571,2=24569,3=24565,4=24541} 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:failed 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:damaged 2024-05-10T05:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:stopped 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:data_pools [3] 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:metadata_pool 2 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:inline_data disabled 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:balancer 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:bal_rank_mask -1 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:standby_count_wanted 0 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b{0:24581} state up:active seq 3 addr [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.h{0:24539} state up:standby-replay seq 1 addr [v2:172.21.15.142:6836/2645029376,v1:172.21.15.142:6837/2645029376] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.g{1:24571} state up:active seq 5 addr [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.f{1:24529} state up:standby-replay seq 1 addr [v2:172.21.15.157:6834/72787128,v1:172.21.15.157:6835/72787128] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a{2:24569} state up:active seq 5 addr [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.e{2:24512} state up:standby-replay seq 1 addr [v2:172.21.15.142:6834/3168061272,v1:172.21.15.142:6835/3168061272] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.i{3:24565} state up:active seq 6 addr [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d{3:24500} state up:standby-replay seq 1 addr [v2:172.21.15.131:6836/3356790716,v1:172.21.15.131:6837/3356790716] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c{4:24541} state up:active seq 7 addr [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.544 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.k{4:24476} state up:standby-replay seq 1 addr [v2:172.21.15.142:6832/3035299290,v1:172.21.15.142:6833/3035299290] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:07.544 INFO:teuthology.orchestra.run.smithi131.stdout:Standby daemons: 2024-05-10T05:33:07.544 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:07.544 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.j{-1:14475} state up:standby seq 1 addr [v2:172.21.15.131:6834/2077120233,v1:172.21.15.131:6835/2077120233] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.544 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.l{-1:24475} state up:standby seq 1 addr [v2:172.21.15.157:6832/3274595788,v1:172.21.15.157:6833/3274595788] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 -- 172.21.15.131:0/3478195356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b880a1230 msgr2=0x7f7b880a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b880a1230 0x7f7b880a3740 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4074e30 tx=0x7f7b94009260 comp rx=0 tx=0).stop 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 -- 172.21.15.131:0/3478195356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ba4109b50 msgr2=0x7f7ba419b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ba4109b50 0x7f7ba419b5e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00ef30 tx=0x7f7b9c00c3a0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 -- 172.21.15.131:0/3478195356 shutdown_connections 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b880a1230 0x7f7b880a3740 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ba4109b50 0x7f7ba419b5e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba4073b40 0x7f7ba419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 --2- 172.21.15.131:0/3478195356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4075130 0x7f7ba419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 -- 172.21.15.131:0/3478195356 >> 172.21.15.131:0/3478195356 conn(0x7f7ba40fd2c0 msgr2=0x7f7ba4105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 -- 172.21.15.131:0/3478195356 shutdown_connections 2024-05-10T05:33:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:07.542+0000 7f7baa000700 1 -- 172.21.15.131:0/3478195356 wait complete. 2024-05-10T05:33:07.546 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-05-10T05:33:07.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:07 smithi157 ceph-mon[78427]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T05:33:07.930 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-10T05:33:08.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:08 smithi131 ceph-mon[37264]: pgmap v278: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:08.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:08 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3478195356' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T05:33:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:33:08.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:08 smithi142 ceph-mon[77667]: pgmap v278: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:08.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:08 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3478195356' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T05:33:08.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:08 smithi157 ceph-mon[78427]: pgmap v278: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:08.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:08 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3478195356' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T05:33:09.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:09 smithi142 ceph-mon[77667]: pgmap v279: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:09.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:09 smithi131 ceph-mon[37264]: pgmap v279: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:09.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.530+0000 7f3238c00700 1 -- 172.21.15.131:0/3562241477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3234075130 msgr2=0x7f3234073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.530+0000 7f3238c00700 1 --2- 172.21.15.131:0/3562241477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3234075130 0x7f3234073600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3224009990 tx=0x7f3224009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.530+0000 7f3238c00700 1 -- 172.21.15.131:0/3562241477 shutdown_connections 2024-05-10T05:33:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.530+0000 7f3238c00700 1 --2- 172.21.15.131:0/3562241477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3234075130 0x7f3234073600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.530+0000 7f3238c00700 1 --2- 172.21.15.131:0/3562241477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3234109b50 0x7f323410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.530+0000 7f3238c00700 1 --2- 172.21.15.131:0/3562241477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3234073b40 0x7f3234073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.530+0000 7f3238c00700 1 -- 172.21.15.131:0/3562241477 >> 172.21.15.131:0/3562241477 conn(0x7f32340fd2c0 msgr2=0x7f32340ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.530+0000 7f3238c00700 1 -- 172.21.15.131:0/3562241477 shutdown_connections 2024-05-10T05:33:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.530+0000 7f3238c00700 1 -- 172.21.15.131:0/3562241477 wait complete. 2024-05-10T05:33:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.531+0000 7f3238c00700 1 Processor -- start 2024-05-10T05:33:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3238c00700 1 -- start start 2024-05-10T05:33:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3238c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3234073b40 0x7f323419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3238c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3234075130 0x7f323419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3238c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3234109b50 0x7f323419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3238c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323419bc80 con 0x7f3234075130 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3238c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323419bdc0 con 0x7f3234109b50 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3238c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323419bf80 con 0x7f3234073b40 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3231400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3234075130 0x7f323419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3231e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3234073b40 0x7f323419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3231e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3234073b40 0x7f323419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:38684/0 (socket says 172.21.15.131:38684) 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3231400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3234075130 0x7f323419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49494/0 (socket says 172.21.15.131:49494) 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3231e00700 1 -- 172.21.15.131:0/3550730011 learned_addr learned my addr 172.21.15.131:0/3550730011 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.532+0000 7f3232800700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3234109b50 0x7f323419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f3231e00700 1 -- 172.21.15.131:0/3550730011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3234109b50 msgr2=0x7f323419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f3231e00700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3234109b50 0x7f323419b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f3231e00700 1 -- 172.21.15.131:0/3550730011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3234075130 msgr2=0x7f323419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f3231e00700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3234075130 0x7f323419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f3231e00700 1 -- 172.21.15.131:0/3550730011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240095f0 con 0x7f3234073b40 2024-05-10T05:33:09.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f3231e00700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3234073b40 0x7f323419a330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32240098e0 tx=0x7f3224004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:09.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f322aa00700 1 -- 172.21.15.131:0/3550730011 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322401d070 con 0x7f3234073b40 2024-05-10T05:33:09.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f322aa00700 1 -- 172.21.15.131:0/3550730011 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3224021410 con 0x7f3234073b40 2024-05-10T05:33:09.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f3238c00700 1 -- 172.21.15.131:0/3550730011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323419c200 con 0x7f3234073b40 2024-05-10T05:33:09.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f322aa00700 1 -- 172.21.15.131:0/3550730011 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224017650 con 0x7f3234073b40 2024-05-10T05:33:09.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.533+0000 7f3238c00700 1 -- 172.21.15.131:0/3550730011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32341a2a40 con 0x7f3234073b40 2024-05-10T05:33:09.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.535+0000 7f3238c00700 1 -- 172.21.15.131:0/3550730011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32341089a0 con 0x7f3234073b40 2024-05-10T05:33:09.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.536+0000 7f322aa00700 1 -- 172.21.15.131:0/3550730011 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f32240217f0 con 0x7f3234073b40 2024-05-10T05:33:09.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.538+0000 7f322aa00700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32180a11b0 0x7f32180a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.538+0000 7f322aa00700 1 -- 172.21.15.131:0/3550730011 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f32240cde10 con 0x7f3234073b40 2024-05-10T05:33:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.538+0000 7f3231400700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32180a11b0 0x7f32180a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.538+0000 7f3231400700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32180a11b0 0x7f32180a36c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f323419b300 tx=0x7f3220009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.542+0000 7f322aa00700 1 -- 172.21.15.131:0/3550730011 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f32240d1030 con 0x7f3234073b40 2024-05-10T05:33:09.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:09 smithi157 ceph-mon[78427]: pgmap v279: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:09.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.745+0000 7f3238c00700 1 -- 172.21.15.131:0/3550730011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f3234100fa0 con 0x7f3234073b40 2024-05-10T05:33:09.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.747+0000 7f322aa00700 1 -- 172.21.15.131:0/3550730011 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) v1 ==== 56+0+5322 (secure 0 0 0) 0x7f3224097470 con 0x7f3234073b40 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 82 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-05-10T05:25:18.809652+0000 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:modified 2024-05-10T05:32:15.903245+0000 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:crush_version 26 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:full_ratio 0.95 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:backfillfull_ratio 0.9 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:nearfull_ratio 0.85 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:require_min_compat_client luminous 2024-05-10T05:33:09.748 INFO:teuthology.orchestra.run.smithi131.stdout:min_compat_client luminous 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:require_osd_release reef 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:stretch_mode_enabled false 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:max_osd 12 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6802/3101739550,v1:172.21.15.131:6803/3101739550] [v2:172.21.15.131:6804/3101739550,v1:172.21.15.131:6805/3101739550] exists,up 5ad2b260-d7ae-4270-ab31-8855bf4b056a 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/451743952,v1:172.21.15.142:6801/451743952] [v2:172.21.15.142:6802/451743952,v1:172.21.15.142:6803/451743952] exists,up 908f4583-6e05-4832-add9-714bf647e807 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 up in weight 1 up_from 18 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6800/3652465947,v1:172.21.15.157:6801/3652465947] [v2:172.21.15.157:6802/3652465947,v1:172.21.15.157:6803/3652465947] exists,up c2da9e21-2747-4101-8566-feb07d70e809 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 up in weight 1 up_from 25 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6810/2743036600,v1:172.21.15.131:6811/2743036600] [v2:172.21.15.131:6812/2743036600,v1:172.21.15.131:6813/2743036600] exists,up 665394bb-71dc-422c-bebc-5b0557311dec 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 up in weight 1 up_from 32 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/1165225030,v1:172.21.15.142:6809/1165225030] [v2:172.21.15.142:6810/1165225030,v1:172.21.15.142:6811/1165225030] exists,up a346957d-a63e-4865-b454-204ba17320f3 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 up in weight 1 up_from 37 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6808/1157754257,v1:172.21.15.157:6809/1157754257] [v2:172.21.15.157:6810/1157754257,v1:172.21.15.157:6811/1157754257] exists,up 33496762-ef3a-4ac9-99a6-0b85e46fd5df 2024-05-10T05:33:09.749 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 up in weight 1 up_from 44 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6818/1462313866,v1:172.21.15.131:6819/1462313866] [v2:172.21.15.131:6820/1462313866,v1:172.21.15.131:6821/1462313866] exists,up 4f5e4557-11d2-40d7-9cee-aa6e36659940 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 up in weight 1 up_from 49 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/1027002285,v1:172.21.15.142:6817/1027002285] [v2:172.21.15.142:6818/1027002285,v1:172.21.15.142:6819/1027002285] exists,up bfcb0014-f452-4449-98aa-2f041be39249 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:osd.8 up in weight 1 up_from 56 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6816/2904635811,v1:172.21.15.157:6817/2904635811] [v2:172.21.15.157:6818/2904635811,v1:172.21.15.157:6819/2904635811] exists,up 1fb1c9d3-d3b2-47db-9e17-16b16513a568 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:osd.9 up in weight 1 up_from 63 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6826/558554742,v1:172.21.15.131:6827/558554742] [v2:172.21.15.131:6828/558554742,v1:172.21.15.131:6829/558554742] exists,up 31e98635-37d4-43eb-a609-a43df9f89afd 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:osd.10 up in weight 1 up_from 68 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/2763887998,v1:172.21.15.142:6825/2763887998] [v2:172.21.15.142:6826/2763887998,v1:172.21.15.142:6827/2763887998] exists,up a073aaf2-1cff-4d5e-8cf7-20f41deeab54 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:osd.11 up in weight 1 up_from 75 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6824/1016688620,v1:172.21.15.157:6825/1016688620] [v2:172.21.15.157:6826/1016688620,v1:172.21.15.157:6827/1016688620] exists,up 6d1207f7-17c7-4e22-85e7-a589fd87f906 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 2.8 [2,11] 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 2.a [2,11] 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.5 [3,9] 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.1b [4,8,3,6] 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.1d [4,7,5,9] 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.23 [4,8] 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.31 [4,6] 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:pg_upmap_items 3.33 [5,9] 2024-05-10T05:33:09.750 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6801/3754674139 expires 2024-05-11T05:26:14.753805+0000 2024-05-10T05:33:09.751 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/1436248426 expires 2024-05-11T05:26:14.753805+0000 2024-05-10T05:33:09.751 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/2071559003 expires 2024-05-11T05:26:14.753805+0000 2024-05-10T05:33:09.751 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6800/3754674139 expires 2024-05-11T05:26:14.753805+0000 2024-05-10T05:33:09.751 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/3122376975 expires 2024-05-11T05:25:49.184015+0000 2024-05-10T05:33:09.751 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6800/2225571046 expires 2024-05-11T05:25:49.184015+0000 2024-05-10T05:33:09.751 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:6801/2225571046 expires 2024-05-11T05:25:49.184015+0000 2024-05-10T05:33:09.751 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/1953348051 expires 2024-05-11T05:25:49.184015+0000 2024-05-10T05:33:09.751 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/2872216821 expires 2024-05-11T05:25:49.184015+0000 2024-05-10T05:33:09.751 INFO:teuthology.orchestra.run.smithi131.stdout:blocklist 172.21.15.131:0/3302654208 expires 2024-05-11T05:26:14.753805+0000 2024-05-10T05:33:09.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.749+0000 7f3238c00700 1 -- 172.21.15.131:0/3550730011 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32180a11b0 msgr2=0x7f32180a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:09.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.749+0000 7f3238c00700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32180a11b0 0x7f32180a36c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f323419b300 tx=0x7f3220009260 comp rx=0 tx=0).stop 2024-05-10T05:33:09.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.749+0000 7f3238c00700 1 -- 172.21.15.131:0/3550730011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3234073b40 msgr2=0x7f323419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:09.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.749+0000 7f3238c00700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3234073b40 0x7f323419a330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32240098e0 tx=0x7f3224004730 comp rx=0 tx=0).stop 2024-05-10T05:33:09.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.749+0000 7f3238c00700 1 -- 172.21.15.131:0/3550730011 shutdown_connections 2024-05-10T05:33:09.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.749+0000 7f3238c00700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f32180a11b0 0x7f32180a36c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:09.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.749+0000 7f3238c00700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3234109b50 0x7f323419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:09.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.749+0000 7f3238c00700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3234073b40 0x7f323419a330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:09.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.749+0000 7f3238c00700 1 --2- 172.21.15.131:0/3550730011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3234075130 0x7f323419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:09.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.749+0000 7f3238c00700 1 -- 172.21.15.131:0/3550730011 >> 172.21.15.131:0/3550730011 conn(0x7f32340fd2c0 msgr2=0x7f3234105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:09.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.750+0000 7f3238c00700 1 -- 172.21.15.131:0/3550730011 shutdown_connections 2024-05-10T05:33:09.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:09.750+0000 7f3238c00700 1 -- 172.21.15.131:0/3550730011 wait complete. 2024-05-10T05:33:10.171 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-10T05:33:10.183 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:33:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:10 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3550730011' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T05:33:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.450+0000 7f9dbb400700 1 -- 172.21.15.131:0/734260203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db4102d00 msgr2=0x7f9db4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.450+0000 7f9dbb400700 1 --2- 172.21.15.131:0/734260203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db4102d00 0x7f9db4103110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9da00098f0 tx=0x7f9da0009c00 comp rx=0 tx=0).stop 2024-05-10T05:33:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.450+0000 7f9dbb400700 1 -- 172.21.15.131:0/734260203 shutdown_connections 2024-05-10T05:33:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.450+0000 7f9dbb400700 1 --2- 172.21.15.131:0/734260203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db41036a0 0x7f9db4109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.450+0000 7f9dbb400700 1 --2- 172.21.15.131:0/734260203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db4101be0 0x7f9db4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.450+0000 7f9dbb400700 1 --2- 172.21.15.131:0/734260203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db4102d00 0x7f9db4103110 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.450+0000 7f9dbb400700 1 -- 172.21.15.131:0/734260203 >> 172.21.15.131:0/734260203 conn(0x7f9db40fd210 msgr2=0x7f9db40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.450+0000 7f9dbb400700 1 -- 172.21.15.131:0/734260203 shutdown_connections 2024-05-10T05:33:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.450+0000 7f9dbb400700 1 -- 172.21.15.131:0/734260203 wait complete. 2024-05-10T05:33:10.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.451+0000 7f9dbb400700 1 Processor -- start 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.451+0000 7f9dbb400700 1 -- start start 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.451+0000 7f9dbb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db4101be0 0x7f9db419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.451+0000 7f9dbb400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db4102d00 0x7f9db419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.451+0000 7f9dbb400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db41036a0 0x7f9db41a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.451+0000 7f9dbb400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db41a1dc0 con 0x7f9db4102d00 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.451+0000 7f9dbb400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db41a1f30 con 0x7f9db4101be0 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.451+0000 7f9dbb400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db41a20f0 con 0x7f9db41036a0 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db4102d00 0x7f9db419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db8a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db4101be0 0x7f9db419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db4102d00 0x7f9db419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49510/0 (socket says 172.21.15.131:49510) 2024-05-10T05:33:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db3e00700 1 -- 172.21.15.131:0/1614037674 learned_addr learned my addr 172.21.15.131:0/1614037674 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db9400700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db41036a0 0x7f9db41a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db3e00700 1 -- 172.21.15.131:0/1614037674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db41036a0 msgr2=0x7f9db41a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db3e00700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db41036a0 0x7f9db41a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db3e00700 1 -- 172.21.15.131:0/1614037674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db4101be0 msgr2=0x7f9db419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db3e00700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db4101be0 0x7f9db419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db3e00700 1 -- 172.21.15.131:0/1614037674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac009c40 con 0x7f9db4102d00 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db8a00700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db4101be0 0x7f9db419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.452+0000 7f9db3e00700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db4102d00 0x7f9db419a820 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009490 tx=0x7f9da000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.453+0000 7f9db1600700 1 -- 172.21.15.131:0/1614037674 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da001c070 con 0x7f9db4102d00 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.453+0000 7f9dbb400700 1 -- 172.21.15.131:0/1614037674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da0009520 con 0x7f9db4102d00 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.453+0000 7f9db1600700 1 -- 172.21.15.131:0/1614037674 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9da0005040 con 0x7f9db4102d00 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.453+0000 7f9db1600700 1 -- 172.21.15.131:0/1614037674 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0004910 con 0x7f9db4102d00 2024-05-10T05:33:10.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.453+0000 7f9dbb400700 1 -- 172.21.15.131:0/1614037674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db41a2600 con 0x7f9db4102d00 2024-05-10T05:33:10.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.454+0000 7f9dbb400700 1 -- 172.21.15.131:0/1614037674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db40694b0 con 0x7f9db4102d00 2024-05-10T05:33:10.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.456+0000 7f9db1600700 1 -- 172.21.15.131:0/1614037674 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9da0020070 con 0x7f9db4102d00 2024-05-10T05:33:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.457+0000 7f9db1600700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9d9c0a11b0 0x7f9d9c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.457+0000 7f9db1600700 1 -- 172.21.15.131:0/1614037674 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9da00cdb40 con 0x7f9db4102d00 2024-05-10T05:33:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.457+0000 7f9db8a00700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9d9c0a11b0 0x7f9d9c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.458+0000 7f9db8a00700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9d9c0a11b0 0x7f9d9c0a36c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9db4102c00 tx=0x7f9da8009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:10.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:10 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3550730011' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T05:33:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.460+0000 7f9db1600700 1 -- 172.21.15.131:0/1614037674 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9da00971a0 con 0x7f9db4102d00 2024-05-10T05:33:10.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:10 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3550730011' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T05:33:10.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.669+0000 7f9dbb400700 1 -- 172.21.15.131:0/1614037674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9db410d430 con 0x7f9db4102d00 2024-05-10T05:33:10.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.670+0000 7f9db1600700 1 -- 172.21.15.131:0/1614037674 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7f9da00051b0 con 0x7f9db4102d00 2024-05-10T05:33:10.671 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:10.672 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:33:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 -- 172.21.15.131:0/1614037674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9d9c0a11b0 msgr2=0x7f9d9c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9d9c0a11b0 0x7f9d9c0a36c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9db4102c00 tx=0x7f9da8009310 comp rx=0 tx=0).stop 2024-05-10T05:33:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 -- 172.21.15.131:0/1614037674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db4102d00 msgr2=0x7f9db419a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db4102d00 0x7f9db419a820 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009490 tx=0x7f9da000bfa0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 -- 172.21.15.131:0/1614037674 shutdown_connections 2024-05-10T05:33:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9d9c0a11b0 0x7f9d9c0a36c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db41036a0 0x7f9db41a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9db4101be0 0x7f9db419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 --2- 172.21.15.131:0/1614037674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db4102d00 0x7f9db419a820 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 -- 172.21.15.131:0/1614037674 >> 172.21.15.131:0/1614037674 conn(0x7f9db40fd210 msgr2=0x7f9db4073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:10.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 -- 172.21.15.131:0/1614037674 shutdown_connections 2024-05-10T05:33:10.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.673+0000 7f9dbb400700 1 -- 172.21.15.131:0/1614037674 wait complete. 2024-05-10T05:33:10.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T05:31:49.481667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T05:31:50.960751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:33:10.700 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:33:10.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.967+0000 7f6837a00700 1 -- 172.21.15.131:0/518369092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830101be0 msgr2=0x7f6830101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:10.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.967+0000 7f6837a00700 1 --2- 172.21.15.131:0/518369092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830101be0 0x7f6830101ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6824009990 tx=0x7f6824009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.967+0000 7f6837a00700 1 -- 172.21.15.131:0/518369092 shutdown_connections 2024-05-10T05:33:10.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.967+0000 7f6837a00700 1 --2- 172.21.15.131:0/518369092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830101be0 0x7f6830101ff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.967+0000 7f6837a00700 1 --2- 172.21.15.131:0/518369092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68301036a0 0x7f6830109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.967+0000 7f6837a00700 1 --2- 172.21.15.131:0/518369092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6830102d00 0x7f6830103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.967+0000 7f6837a00700 1 -- 172.21.15.131:0/518369092 >> 172.21.15.131:0/518369092 conn(0x7f68300fd1d0 msgr2=0x7f68300ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:10.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.967+0000 7f6837a00700 1 -- 172.21.15.131:0/518369092 shutdown_connections 2024-05-10T05:33:10.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.967+0000 7f6837a00700 1 -- 172.21.15.131:0/518369092 wait complete. 2024-05-10T05:33:10.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.968+0000 7f6837a00700 1 Processor -- start 2024-05-10T05:33:10.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f6837a00700 1 -- start start 2024-05-10T05:33:10.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f6837a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6830101be0 0x7f6830074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:10.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f6837a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830102d00 0x7f6830073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:10.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f6837a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68301036a0 0x7f6830073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:10.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f6837a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830074180 con 0x7f6830102d00 2024-05-10T05:33:10.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f6837a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68300742f0 con 0x7f68301036a0 2024-05-10T05:33:10.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f6837a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301a1fe0 con 0x7f6830101be0 2024-05-10T05:33:10.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f682fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830102d00 0x7f6830073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:10.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f6835000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6830101be0 0x7f6830074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:10.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f682fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830102d00 0x7f6830073560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49526/0 (socket says 172.21.15.131:49526) 2024-05-10T05:33:10.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f682fe00700 1 -- 172.21.15.131:0/1387174082 learned_addr learned my addr 172.21.15.131:0/1387174082 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:10.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.969+0000 7f6835a00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68301036a0 0x7f6830073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:10.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f682fe00700 1 -- 172.21.15.131:0/1387174082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6830101be0 msgr2=0x7f6830074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:10.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f682fe00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6830101be0 0x7f6830074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f682fe00700 1 -- 172.21.15.131:0/1387174082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68301036a0 msgr2=0x7f6830073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:10.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f682fe00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68301036a0 0x7f6830073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:10.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f682fe00700 1 -- 172.21.15.131:0/1387174082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68240095f0 con 0x7f6830102d00 2024-05-10T05:33:10.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f6835a00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68301036a0 0x7f6830073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:33:10.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f682fe00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830102d00 0x7f6830073560 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f681c00e9a0 tx=0x7f681c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:10.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f682d600700 1 -- 172.21.15.131:0/1387174082 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c009790 con 0x7f6830102d00 2024-05-10T05:33:10.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f6837a00700 1 -- 172.21.15.131:0/1387174082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68301a2240 con 0x7f6830102d00 2024-05-10T05:33:10.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f682d600700 1 -- 172.21.15.131:0/1387174082 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f681c00cbd0 con 0x7f6830102d00 2024-05-10T05:33:10.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.970+0000 7f682d600700 1 -- 172.21.15.131:0/1387174082 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c018680 con 0x7f6830102d00 2024-05-10T05:33:10.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.971+0000 7f6837a00700 1 -- 172.21.15.131:0/1387174082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68301a2720 con 0x7f6830102d00 2024-05-10T05:33:10.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.972+0000 7f6837a00700 1 -- 172.21.15.131:0/1387174082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6830108df0 con 0x7f6830102d00 2024-05-10T05:33:10.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.973+0000 7f682d600700 1 -- 172.21.15.131:0/1387174082 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f681c00cd40 con 0x7f6830102d00 2024-05-10T05:33:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.975+0000 7f682d600700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f68180a1260 0x7f68180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.975+0000 7f682d600700 1 -- 172.21.15.131:0/1387174082 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f681c014070 con 0x7f6830102d00 2024-05-10T05:33:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.975+0000 7f6835000700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f68180a1260 0x7f68180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.976+0000 7f6835000700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f68180a1260 0x7f68180a3770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6824006010 tx=0x7f682401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:10.979+0000 7f682d600700 1 -- 172.21.15.131:0/1387174082 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f681c052d60 con 0x7f6830102d00 2024-05-10T05:33:11.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.191+0000 7f6837a00700 1 -- 172.21.15.131:0/1387174082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f683010d410 con 0x7f6830102d00 2024-05-10T05:33:11.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.191+0000 7f682d600700 1 -- 172.21.15.131:0/1387174082 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f681c096300 con 0x7f6830102d00 2024-05-10T05:33:11.192 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:33:11.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 -- 172.21.15.131:0/1387174082 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f68180a1260 msgr2=0x7f68180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f68180a1260 0x7f68180a3770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6824006010 tx=0x7f682401a040 comp rx=0 tx=0).stop 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 -- 172.21.15.131:0/1387174082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830102d00 msgr2=0x7f6830073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830102d00 0x7f6830073560 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f681c00e9a0 tx=0x7f681c00ecb0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 -- 172.21.15.131:0/1387174082 shutdown_connections 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f68180a1260 0x7f68180a3770 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68301036a0 0x7f6830073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6830101be0 0x7f6830074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 --2- 172.21.15.131:0/1387174082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6830102d00 0x7f6830073560 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 -- 172.21.15.131:0/1387174082 >> 172.21.15.131:0/1387174082 conn(0x7f68300fd1d0 msgr2=0x7f6830105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 -- 172.21.15.131:0/1387174082 shutdown_connections 2024-05-10T05:33:11.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.194+0000 7f6837a00700 1 -- 172.21.15.131:0/1387174082 wait complete. 2024-05-10T05:33:11.223 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T05:33:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:11 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1614037674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:11 smithi142 ceph-mon[77667]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:33:11.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:11 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1614037674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:11.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:11 smithi131 ceph-mon[37264]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:33:11.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.489+0000 7fde31600700 1 -- 172.21.15.131:0/3639209735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde2c102b10 msgr2=0x7fde2c102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.489+0000 7fde31600700 1 --2- 172.21.15.131:0/3639209735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde2c102b10 0x7fde2c102f20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fde18009990 tx=0x7fde18009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.489+0000 7fde31600700 1 -- 172.21.15.131:0/3639209735 shutdown_connections 2024-05-10T05:33:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.489+0000 7fde31600700 1 --2- 172.21.15.131:0/3639209735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde2c1034b0 0x7fde2c109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.489+0000 7fde31600700 1 --2- 172.21.15.131:0/3639209735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c1019f0 0x7fde2c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.489+0000 7fde31600700 1 --2- 172.21.15.131:0/3639209735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde2c102b10 0x7fde2c102f20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.489+0000 7fde31600700 1 -- 172.21.15.131:0/3639209735 >> 172.21.15.131:0/3639209735 conn(0x7fde2c0fd000 msgr2=0x7fde2c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.489+0000 7fde31600700 1 -- 172.21.15.131:0/3639209735 shutdown_connections 2024-05-10T05:33:11.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.489+0000 7fde31600700 1 -- 172.21.15.131:0/3639209735 wait complete. 2024-05-10T05:33:11.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.490+0000 7fde31600700 1 Processor -- start 2024-05-10T05:33:11.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.490+0000 7fde31600700 1 -- start start 2024-05-10T05:33:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde31600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c1019f0 0x7fde2c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde31600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde2c102b10 0x7fde2c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde31600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde2c1034b0 0x7fde2c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde31600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c1a1af0 con 0x7fde2c1034b0 2024-05-10T05:33:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde31600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c1a1c30 con 0x7fde2c102b10 2024-05-10T05:33:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde31600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c1a1dc0 con 0x7fde2c1019f0 2024-05-10T05:33:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde30c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde2c1034b0 0x7fde2c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde30c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde2c1034b0 0x7fde2c1a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49552/0 (socket says 172.21.15.131:49552) 2024-05-10T05:33:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde30c00700 1 -- 172.21.15.131:0/2229770813 learned_addr learned my addr 172.21.15.131:0/2229770813 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde2b400700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde2c102b10 0x7fde2c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde2be00700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c1019f0 0x7fde2c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde30c00700 1 -- 172.21.15.131:0/2229770813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c1019f0 msgr2=0x7fde2c19a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde30c00700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c1019f0 0x7fde2c19a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde30c00700 1 -- 172.21.15.131:0/2229770813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde2c102b10 msgr2=0x7fde2c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.491+0000 7fde30c00700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde2c102b10 0x7fde2c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.492+0000 7fde30c00700 1 -- 172.21.15.131:0/2229770813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde180095f0 con 0x7fde2c1034b0 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.492+0000 7fde30c00700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde2c1034b0 0x7fde2c1a1520 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fde24007f10 tx=0x7fde2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.492+0000 7fde28c00700 1 -- 172.21.15.131:0/2229770813 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24017070 con 0x7fde2c1034b0 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.492+0000 7fde28c00700 1 -- 172.21.15.131:0/2229770813 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde2400f040 con 0x7fde2c1034b0 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.492+0000 7fde28c00700 1 -- 172.21.15.131:0/2229770813 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24005610 con 0x7fde2c1034b0 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.492+0000 7fde31600700 1 -- 172.21.15.131:0/2229770813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2c1a20a0 con 0x7fde2c1034b0 2024-05-10T05:33:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.492+0000 7fde31600700 1 -- 172.21.15.131:0/2229770813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2c1a2560 con 0x7fde2c1034b0 2024-05-10T05:33:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.493+0000 7fde31600700 1 -- 172.21.15.131:0/2229770813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2c068110 con 0x7fde2c1034b0 2024-05-10T05:33:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.495+0000 7fde28c00700 1 -- 172.21.15.131:0/2229770813 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fde24004ad0 con 0x7fde2c1034b0 2024-05-10T05:33:11.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.497+0000 7fde28c00700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fde100a1230 0x7fde100a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:11.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.497+0000 7fde28c00700 1 -- 172.21.15.131:0/2229770813 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fde240cc780 con 0x7fde2c1034b0 2024-05-10T05:33:11.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.497+0000 7fde2be00700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fde100a1230 0x7fde100a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:11.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.497+0000 7fde2be00700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fde100a1230 0x7fde100a3740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fde2c102a10 tx=0x7fde1c007400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:11.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.499+0000 7fde28c00700 1 -- 172.21.15.131:0/2229770813 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fde24095d50 con 0x7fde2c1034b0 2024-05-10T05:33:11.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:11 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1614037674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:11.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:11 smithi157 ceph-mon[78427]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:33:11.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.715+0000 7fde31600700 1 -- 172.21.15.131:0/2229770813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde2c10d3d0 con 0x7fde2c1034b0 2024-05-10T05:33:11.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.715+0000 7fde28c00700 1 -- 172.21.15.131:0/2229770813 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde2c10d3d0 con 0x7fde2c1034b0 2024-05-10T05:33:11.716 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 -- 172.21.15.131:0/2229770813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fde100a1230 msgr2=0x7fde100a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fde100a1230 0x7fde100a3740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fde2c102a10 tx=0x7fde1c007400 comp rx=0 tx=0).stop 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 -- 172.21.15.131:0/2229770813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde2c1034b0 msgr2=0x7fde2c1a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde2c1034b0 0x7fde2c1a1520 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fde24007f10 tx=0x7fde2400d3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 -- 172.21.15.131:0/2229770813 shutdown_connections 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fde100a1230 0x7fde100a3740 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde2c1034b0 0x7fde2c1a1520 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde2c1019f0 0x7fde2c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 --2- 172.21.15.131:0/2229770813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde2c102b10 0x7fde2c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 -- 172.21.15.131:0/2229770813 >> 172.21.15.131:0/2229770813 conn(0x7fde2c0fd000 msgr2=0x7fde2c073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 -- 172.21.15.131:0/2229770813 shutdown_connections 2024-05-10T05:33:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:11.718+0000 7fde31600700 1 -- 172.21.15.131:0/2229770813 wait complete. 2024-05-10T05:33:11.744 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T05:33:12.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.012+0000 7f5a94a00700 1 -- 172.21.15.131:0/3573020446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a901036a0 msgr2=0x7f5a90109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:12.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.012+0000 7f5a94a00700 1 --2- 172.21.15.131:0/3573020446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a901036a0 0x7f5a90109f50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800b0a0 tx=0x7f5a8800b3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.012+0000 7f5a94a00700 1 -- 172.21.15.131:0/3573020446 shutdown_connections 2024-05-10T05:33:12.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.012+0000 7f5a94a00700 1 --2- 172.21.15.131:0/3573020446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a901036a0 0x7f5a90109f50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.012+0000 7f5a94a00700 1 --2- 172.21.15.131:0/3573020446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a90101be0 0x7f5a90101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.012+0000 7f5a94a00700 1 --2- 172.21.15.131:0/3573020446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a90102d00 0x7f5a90103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.012+0000 7f5a94a00700 1 -- 172.21.15.131:0/3573020446 >> 172.21.15.131:0/3573020446 conn(0x7f5a900fd210 msgr2=0x7f5a900ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:12.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.013+0000 7f5a94a00700 1 -- 172.21.15.131:0/3573020446 shutdown_connections 2024-05-10T05:33:12.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.013+0000 7f5a94a00700 1 -- 172.21.15.131:0/3573020446 wait complete. 2024-05-10T05:33:12.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a94a00700 1 Processor -- start 2024-05-10T05:33:12.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a94a00700 1 -- start start 2024-05-10T05:33:12.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a94a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a90101be0 0x7f5a90107750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a94a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a90102d00 0x7f5a90107c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a94a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a901036a0 0x7f5a90104360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a94a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a90104a60 con 0x7f5a901036a0 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a94a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a90104bd0 con 0x7f5a90101be0 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a94a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a90104d90 con 0x7f5a90102d00 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a8e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a901036a0 0x7f5a90104360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a8e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a901036a0 0x7f5a90104360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49570/0 (socket says 172.21.15.131:49570) 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a8e800700 1 -- 172.21.15.131:0/2238655832 learned_addr learned my addr 172.21.15.131:0/2238655832 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a8d400700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a90102d00 0x7f5a90107c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.014+0000 7f5a8de00700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a90101be0 0x7f5a90107750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a8e800700 1 -- 172.21.15.131:0/2238655832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a90102d00 msgr2=0x7f5a90107c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a8e800700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a90102d00 0x7f5a90107c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a8e800700 1 -- 172.21.15.131:0/2238655832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a90101be0 msgr2=0x7f5a90107750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:12.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a8e800700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a90101be0 0x7f5a90107750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a8e800700 1 -- 172.21.15.131:0/2238655832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a88009d00 con 0x7f5a901036a0 2024-05-10T05:33:12.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a8e800700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a901036a0 0x7f5a90104360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800b070 tx=0x7f5a880078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:12.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a86a00700 1 -- 172.21.15.131:0/2238655832 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800e040 con 0x7f5a901036a0 2024-05-10T05:33:12.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a86a00700 1 -- 172.21.15.131:0/2238655832 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a8801d470 con 0x7f5a901036a0 2024-05-10T05:33:12.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a94a00700 1 -- 172.21.15.131:0/2238655832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a901a66c0 con 0x7f5a901036a0 2024-05-10T05:33:12.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a86a00700 1 -- 172.21.15.131:0/2238655832 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a880052a0 con 0x7f5a901036a0 2024-05-10T05:33:12.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.015+0000 7f5a94a00700 1 -- 172.21.15.131:0/2238655832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a901a6b40 con 0x7f5a901036a0 2024-05-10T05:33:12.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.016+0000 7f5a94a00700 1 -- 172.21.15.131:0/2238655832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a900694d0 con 0x7f5a901036a0 2024-05-10T05:33:12.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.018+0000 7f5a86a00700 1 -- 172.21.15.131:0/2238655832 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5a8801d5e0 con 0x7f5a901036a0 2024-05-10T05:33:12.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.020+0000 7f5a86a00700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5a740a1260 0x7f5a740a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:12.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.020+0000 7f5a86a00700 1 -- 172.21.15.131:0/2238655832 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5a880ce180 con 0x7f5a901036a0 2024-05-10T05:33:12.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.020+0000 7f5a8de00700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5a740a1260 0x7f5a740a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:12.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.021+0000 7f5a8de00700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5a740a1260 0x7f5a740a3770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a90102c00 tx=0x7f5a80009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:12.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.022+0000 7f5a86a00700 1 -- 172.21.15.131:0/2238655832 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5a880977e0 con 0x7f5a901036a0 2024-05-10T05:33:12.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.235+0000 7f5a94a00700 1 -- 172.21.15.131:0/2238655832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a901a6e30 con 0x7f5a901036a0 2024-05-10T05:33:12.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.236+0000 7f5a86a00700 1 -- 172.21.15.131:0/2238655832 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a88025e90 con 0x7f5a901036a0 2024-05-10T05:33:12.237 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-10T05:33:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.238+0000 7f5a94a00700 1 -- 172.21.15.131:0/2238655832 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5a740a1260 msgr2=0x7f5a740a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.238+0000 7f5a94a00700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5a740a1260 0x7f5a740a3770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a90102c00 tx=0x7f5a80009260 comp rx=0 tx=0).stop 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.238+0000 7f5a94a00700 1 -- 172.21.15.131:0/2238655832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a901036a0 msgr2=0x7f5a90104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.238+0000 7f5a94a00700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a901036a0 0x7f5a90104360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800b070 tx=0x7f5a880078a0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.238+0000 7f5a94a00700 1 -- 172.21.15.131:0/2238655832 shutdown_connections 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.238+0000 7f5a94a00700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5a740a1260 0x7f5a740a3770 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.238+0000 7f5a94a00700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a901036a0 0x7f5a90104360 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.238+0000 7f5a94a00700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a90101be0 0x7f5a90107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.238+0000 7f5a94a00700 1 --2- 172.21.15.131:0/2238655832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a90102d00 0x7f5a90107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.238+0000 7f5a94a00700 1 -- 172.21.15.131:0/2238655832 >> 172.21.15.131:0/2238655832 conn(0x7f5a900fd210 msgr2=0x7f5a90073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.239+0000 7f5a94a00700 1 -- 172.21.15.131:0/2238655832 shutdown_connections 2024-05-10T05:33:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.239+0000 7f5a94a00700 1 -- 172.21.15.131:0/2238655832 wait complete. 2024-05-10T05:33:12.268 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:33:12.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:12 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1387174082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:12.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:12 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2229770813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:12.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:12 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1387174082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:12.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:12 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2229770813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:12.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.530+0000 7f5705200700 1 -- 172.21.15.131:0/2255019467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5700069090 msgr2=0x7f5700104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:12.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.530+0000 7f5705200700 1 --2- 172.21.15.131:0/2255019467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5700069090 0x7f5700104720 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009990 tx=0x7f56f0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.530+0000 7f5705200700 1 -- 172.21.15.131:0/2255019467 shutdown_connections 2024-05-10T05:33:12.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.530+0000 7f5705200700 1 --2- 172.21.15.131:0/2255019467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5700069090 0x7f5700104720 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.530+0000 7f5705200700 1 --2- 172.21.15.131:0/2255019467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5700107880 0x7f5700109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.530+0000 7f5705200700 1 --2- 172.21.15.131:0/2255019467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5700104d10 0x7f57001071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.530+0000 7f5705200700 1 -- 172.21.15.131:0/2255019467 >> 172.21.15.131:0/2255019467 conn(0x7f57000fc0d0 msgr2=0x7f57000fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:12.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.530+0000 7f5705200700 1 -- 172.21.15.131:0/2255019467 shutdown_connections 2024-05-10T05:33:12.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.530+0000 7f5705200700 1 -- 172.21.15.131:0/2255019467 wait complete. 2024-05-10T05:33:12.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.531+0000 7f5705200700 1 Processor -- start 2024-05-10T05:33:12.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.531+0000 7f5705200700 1 -- start start 2024-05-10T05:33:12.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f5705200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5700069090 0x7f5700101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:12.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f5705200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5700104d10 0x7f5700100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:12.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f5705200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5700107880 0x7f5700100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:12.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f5705200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5700101210 con 0x7f5700104d10 2024-05-10T05:33:12.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f5705200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5700101350 con 0x7f5700069090 2024-05-10T05:33:12.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f5705200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57001a2110 con 0x7f5700107880 2024-05-10T05:33:12.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f56f7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5700104d10 0x7f5700100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:12.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f56f7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5700104d10 0x7f5700100620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49580/0 (socket says 172.21.15.131:49580) 2024-05-10T05:33:12.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f56f7e00700 1 -- 172.21.15.131:0/826727885 learned_addr learned my addr 172.21.15.131:0/826727885 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:12.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f56fde00700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5700069090 0x7f5700101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:12.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f56fe800700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5700107880 0x7f5700100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:12.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f56f7e00700 1 -- 172.21.15.131:0/826727885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5700107880 msgr2=0x7f5700100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:12.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f56f7e00700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5700107880 0x7f5700100b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f56f7e00700 1 -- 172.21.15.131:0/826727885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5700069090 msgr2=0x7f5700101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:12.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f56f7e00700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5700069090 0x7f5700101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.532+0000 7f56f7e00700 1 -- 172.21.15.131:0/826727885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00095f0 con 0x7f5700104d10 2024-05-10T05:33:12.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.533+0000 7f56f7e00700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5700104d10 0x7f5700100620 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56e800e9a0 tx=0x7f56e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:12.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.533+0000 7f56f6a00700 1 -- 172.21.15.131:0/826727885 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8009790 con 0x7f5700104d10 2024-05-10T05:33:12.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.533+0000 7f56f6a00700 1 -- 172.21.15.131:0/826727885 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56e8010430 con 0x7f5700104d10 2024-05-10T05:33:12.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.533+0000 7f5705200700 1 -- 172.21.15.131:0/826727885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57001a2310 con 0x7f5700104d10 2024-05-10T05:33:12.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.533+0000 7f56f6a00700 1 -- 172.21.15.131:0/826727885 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8018680 con 0x7f5700104d10 2024-05-10T05:33:12.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.533+0000 7f5705200700 1 -- 172.21.15.131:0/826727885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57001a2840 con 0x7f5700104d10 2024-05-10T05:33:12.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.534+0000 7f5705200700 1 -- 172.21.15.131:0/826727885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57000fdc80 con 0x7f5700104d10 2024-05-10T05:33:12.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.536+0000 7f56f6a00700 1 -- 172.21.15.131:0/826727885 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f56e80105a0 con 0x7f5700104d10 2024-05-10T05:33:12.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.537+0000 7f56f6a00700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56e40a1230 0x7f56e40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:12.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.537+0000 7f56f6a00700 1 -- 172.21.15.131:0/826727885 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f56e8014070 con 0x7f5700104d10 2024-05-10T05:33:12.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.538+0000 7f56fde00700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56e40a1230 0x7f56e40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:12.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.538+0000 7f56fde00700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56e40a1230 0x7f56e40a3740 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f56f0006010 tx=0x7f56f001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:12.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.541+0000 7f56f6a00700 1 -- 172.21.15.131:0/826727885 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f56e8052d60 con 0x7f5700104d10 2024-05-10T05:33:12.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:12 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1387174082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:12.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:12 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2229770813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.798+0000 7f5705200700 1 -- 172.21.15.131:0/826727885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57000694d0 con 0x7f5700104d10 2024-05-10T05:33:12.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.798+0000 7f56f6a00700 1 -- 172.21.15.131:0/826727885 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f56e80961e0 con 0x7f5700104d10 2024-05-10T05:33:12.799 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:12.799 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:31:58.482427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 -- 172.21.15.131:0/826727885 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56e40a1230 msgr2=0x7f56e40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56e40a1230 0x7f56e40a3740 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f56f0006010 tx=0x7f56f001a040 comp rx=0 tx=0).stop 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 -- 172.21.15.131:0/826727885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5700104d10 msgr2=0x7f5700100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5700104d10 0x7f5700100620 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56e800e9a0 tx=0x7f56e800ecb0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 -- 172.21.15.131:0/826727885 shutdown_connections 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56e40a1230 0x7f56e40a3740 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5700107880 0x7f5700100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5700069090 0x7f5700101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 --2- 172.21.15.131:0/826727885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5700104d10 0x7f5700100620 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 -- 172.21.15.131:0/826727885 >> 172.21.15.131:0/826727885 conn(0x7f57000fc0d0 msgr2=0x7f57001043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 -- 172.21.15.131:0/826727885 shutdown_connections 2024-05-10T05:33:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:12.801+0000 7f5705200700 1 -- 172.21.15.131:0/826727885 wait complete. 2024-05-10T05:33:12.803 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-05-10T05:33:12.828 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:33:13.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.092+0000 7f9e8c600700 1 -- 172.21.15.131:0/2774923943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e84069090 msgr2=0x7f9e84104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:13.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.092+0000 7f9e8c600700 1 --2- 172.21.15.131:0/2774923943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e84069090 0x7f9e84104720 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009990 tx=0x7f9e78009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.092+0000 7f9e8c600700 1 -- 172.21.15.131:0/2774923943 shutdown_connections 2024-05-10T05:33:13.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.092+0000 7f9e8c600700 1 --2- 172.21.15.131:0/2774923943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e84069090 0x7f9e84104720 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.092+0000 7f9e8c600700 1 --2- 172.21.15.131:0/2774923943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e84107880 0x7f9e84109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.092+0000 7f9e8c600700 1 --2- 172.21.15.131:0/2774923943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e84104d10 0x7f9e841071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.092+0000 7f9e8c600700 1 -- 172.21.15.131:0/2774923943 >> 172.21.15.131:0/2774923943 conn(0x7f9e840fc0d0 msgr2=0x7f9e840fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:13.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.092+0000 7f9e8c600700 1 -- 172.21.15.131:0/2774923943 shutdown_connections 2024-05-10T05:33:13.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.092+0000 7f9e8c600700 1 -- 172.21.15.131:0/2774923943 wait complete. 2024-05-10T05:33:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.093+0000 7f9e8c600700 1 Processor -- start 2024-05-10T05:33:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.093+0000 7f9e8c600700 1 -- start start 2024-05-10T05:33:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e8c600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e84069090 0x7f9e84197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e8c600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e84104d10 0x7f9e84198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e8c600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e84107880 0x7f9e8419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e8c600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8419fac0 con 0x7f9e84069090 2024-05-10T05:33:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e8c600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8419fc30 con 0x7f9e84104d10 2024-05-10T05:33:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e8c600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8419fdf0 con 0x7f9e84107880 2024-05-10T05:33:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e89c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e84069090 0x7f9e84197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e89200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e84104d10 0x7f9e84198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e89c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e84069090 0x7f9e84197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52066/0 (socket says 172.21.15.131:52066) 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e89c00700 1 -- 172.21.15.131:0/3234145545 learned_addr learned my addr 172.21.15.131:0/3234145545 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e8a600700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e84107880 0x7f9e8419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e89200700 1 -- 172.21.15.131:0/3234145545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e84107880 msgr2=0x7f9e8419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e89200700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e84107880 0x7f9e8419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e89200700 1 -- 172.21.15.131:0/3234145545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e84069090 msgr2=0x7f9e84197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e89200700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e84069090 0x7f9e84197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e89200700 1 -- 172.21.15.131:0/3234145545 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c009c40 con 0x7f9e84104d10 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.094+0000 7f9e89c00700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e84069090 0x7f9e84197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.095+0000 7f9e89200700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e84104d10 0x7f9e84198520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000e9f0 tx=0x7f9e7000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.095+0000 7f9e82000700 1 -- 172.21.15.131:0/3234145545 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7000c9c0 con 0x7f9e84104d10 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.095+0000 7f9e8c600700 1 -- 172.21.15.131:0/3234145545 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e780095f0 con 0x7f9e84104d10 2024-05-10T05:33:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.095+0000 7f9e82000700 1 -- 172.21.15.131:0/3234145545 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e70010430 con 0x7f9e84104d10 2024-05-10T05:33:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.095+0000 7f9e82000700 1 -- 172.21.15.131:0/3234145545 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70018680 con 0x7f9e84104d10 2024-05-10T05:33:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.095+0000 7f9e8c600700 1 -- 172.21.15.131:0/3234145545 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e841a0450 con 0x7f9e84104d10 2024-05-10T05:33:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.096+0000 7f9e8c600700 1 -- 172.21.15.131:0/3234145545 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e840fdc80 con 0x7f9e84104d10 2024-05-10T05:33:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.098+0000 7f9e82000700 1 -- 172.21.15.131:0/3234145545 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9e70010860 con 0x7f9e84104d10 2024-05-10T05:33:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.099+0000 7f9e82000700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e6c0a11b0 0x7f9e6c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.099+0000 7f9e82000700 1 -- 172.21.15.131:0/3234145545 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9e70014070 con 0x7f9e84104d10 2024-05-10T05:33:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.100+0000 7f9e89c00700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e6c0a11b0 0x7f9e6c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.100+0000 7f9e89c00700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e6c0a11b0 0x7f9e6c0a36c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800b2e0 tx=0x7f9e7801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:13.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.102+0000 7f9e82000700 1 -- 172.21.15.131:0/3234145545 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9e70096560 con 0x7f9e84104d10 2024-05-10T05:33:13.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.316+0000 7f9e8c600700 1 -- 172.21.15.131:0/3234145545 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e840694d0 con 0x7f9e84104d10 2024-05-10T05:33:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.319+0000 7f9e82000700 1 -- 172.21.15.131:0/3234145545 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7f9e70099b80 con 0x7f9e84104d10 2024-05-10T05:33:13.319 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:13.319 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:33:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 -- 172.21.15.131:0/3234145545 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e6c0a11b0 msgr2=0x7f9e6c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e6c0a11b0 0x7f9e6c0a36c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800b2e0 tx=0x7f9e7801a040 comp rx=0 tx=0).stop 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 -- 172.21.15.131:0/3234145545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e84104d10 msgr2=0x7f9e84198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e84104d10 0x7f9e84198520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000e9f0 tx=0x7f9e7000ed00 comp rx=0 tx=0).stop 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 -- 172.21.15.131:0/3234145545 shutdown_connections 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9e6c0a11b0 0x7f9e6c0a36c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e84069090 0x7f9e84197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e84107880 0x7f9e8419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 --2- 172.21.15.131:0/3234145545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e84104d10 0x7f9e84198520 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 -- 172.21.15.131:0/3234145545 >> 172.21.15.131:0/3234145545 conn(0x7f9e840fc0d0 msgr2=0x7f9e84103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 -- 172.21.15.131:0/3234145545 shutdown_connections 2024-05-10T05:33:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.321+0000 7f9e8c600700 1 -- 172.21.15.131:0/3234145545 wait complete. 2024-05-10T05:33:13.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T05:31:49.481667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T05:31:50.960751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:33:13.349 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:33:13.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:13 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2238655832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:13.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:13 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/826727885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:13.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:13 smithi142 ceph-mon[77667]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:13.406 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:13 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2238655832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:13.406 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:13 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/826727885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:13.407 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:13 smithi131 ceph-mon[37264]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:13 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2238655832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:13 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/826727885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:13.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:13 smithi157 ceph-mon[78427]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:13.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.616+0000 7f1c49e00700 1 -- 172.21.15.131:0/1171159806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c44101be0 msgr2=0x7f1c44101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:13.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.616+0000 7f1c49e00700 1 --2- 172.21.15.131:0/1171159806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c44101be0 0x7f1c44101ff0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c340099c0 tx=0x7f1c34009cd0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.616+0000 7f1c49e00700 1 -- 172.21.15.131:0/1171159806 shutdown_connections 2024-05-10T05:33:13.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.616+0000 7f1c49e00700 1 --2- 172.21.15.131:0/1171159806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c441036a0 0x7f1c44109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.616+0000 7f1c49e00700 1 --2- 172.21.15.131:0/1171159806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c44101be0 0x7f1c44101ff0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.616+0000 7f1c49e00700 1 --2- 172.21.15.131:0/1171159806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c44102d00 0x7f1c44103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.616+0000 7f1c49e00700 1 -- 172.21.15.131:0/1171159806 >> 172.21.15.131:0/1171159806 conn(0x7f1c440fd1d0 msgr2=0x7f1c440ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:13.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.616+0000 7f1c49e00700 1 -- 172.21.15.131:0/1171159806 shutdown_connections 2024-05-10T05:33:13.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.616+0000 7f1c49e00700 1 -- 172.21.15.131:0/1171159806 wait complete. 2024-05-10T05:33:13.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.617+0000 7f1c49e00700 1 Processor -- start 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.617+0000 7f1c49e00700 1 -- start start 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.617+0000 7f1c49e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c44101be0 0x7f1c4419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.617+0000 7f1c49e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c44102d00 0x7f1c4419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.617+0000 7f1c49e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c441036a0 0x7f1c441a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.617+0000 7f1c49e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c441a1c20 con 0x7f1c44102d00 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.617+0000 7f1c49e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c441a1d90 con 0x7f1c441036a0 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.617+0000 7f1c49e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c441a1f50 con 0x7f1c44101be0 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c42a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c44101be0 0x7f1c4419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c42a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c44101be0 0x7f1c4419a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45074/0 (socket says 172.21.15.131:45074) 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c42a00700 1 -- 172.21.15.131:0/44819448 learned_addr learned my addr 172.21.15.131:0/44819448 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c43400700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c441036a0 0x7f1c441a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c42000700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c44102d00 0x7f1c4419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c43400700 1 -- 172.21.15.131:0/44819448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c44101be0 msgr2=0x7f1c4419a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c43400700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c44101be0 0x7f1c4419a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c43400700 1 -- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c44102d00 msgr2=0x7f1c4419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c43400700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c44102d00 0x7f1c4419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c43400700 1 -- 172.21.15.131:0/44819448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c340095f0 con 0x7f1c441036a0 2024-05-10T05:33:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c42a00700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c44101be0 0x7f1c4419a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c42000700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c44102d00 0x7f1c4419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.618+0000 7f1c43400700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c441036a0 0x7f1c441a1610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00efc0 tx=0x7f1c3c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.619+0000 7f1c48a00700 1 -- 172.21.15.131:0/44819448 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c018070 con 0x7f1c441036a0 2024-05-10T05:33:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.619+0000 7f1c49e00700 1 -- 172.21.15.131:0/44819448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c441a21d0 con 0x7f1c441036a0 2024-05-10T05:33:13.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.619+0000 7f1c48a00700 1 -- 172.21.15.131:0/44819448 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c3c00f040 con 0x7f1c441036a0 2024-05-10T05:33:13.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.619+0000 7f1c48a00700 1 -- 172.21.15.131:0/44819448 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00cd40 con 0x7f1c441036a0 2024-05-10T05:33:13.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.619+0000 7f1c49e00700 1 -- 172.21.15.131:0/44819448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c441a2690 con 0x7f1c441036a0 2024-05-10T05:33:13.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.620+0000 7f1c49e00700 1 -- 172.21.15.131:0/44819448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c440694d0 con 0x7f1c441036a0 2024-05-10T05:33:13.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.622+0000 7f1c48a00700 1 -- 172.21.15.131:0/44819448 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1c3c003a00 con 0x7f1c441036a0 2024-05-10T05:33:13.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.623+0000 7f1c48a00700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c280a1260 0x7f1c280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:13.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.623+0000 7f1c48a00700 1 -- 172.21.15.131:0/44819448 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1c3c0ccfd0 con 0x7f1c441036a0 2024-05-10T05:33:13.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.623+0000 7f1c42a00700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c280a1260 0x7f1c280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:13.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.624+0000 7f1c42a00700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c280a1260 0x7f1c280a3770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1c34004800 tx=0x7f1c34005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:13.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.626+0000 7f1c48a00700 1 -- 172.21.15.131:0/44819448 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1c3c096630 con 0x7f1c441036a0 2024-05-10T05:33:13.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.839+0000 7f1c49e00700 1 -- 172.21.15.131:0/44819448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c441a29b0 con 0x7f1c441036a0 2024-05-10T05:33:13.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.840+0000 7f1c48a00700 1 -- 172.21.15.131:0/44819448 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c3c099c50 con 0x7f1c441036a0 2024-05-10T05:33:13.841 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:33:13.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 -- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c280a1260 msgr2=0x7f1c280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:13.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c280a1260 0x7f1c280a3770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1c34004800 tx=0x7f1c34005fb0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 -- 172.21.15.131:0/44819448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c441036a0 msgr2=0x7f1c441a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:13.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c441036a0 0x7f1c441a1610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00efc0 tx=0x7f1c3c00c3a0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 -- 172.21.15.131:0/44819448 shutdown_connections 2024-05-10T05:33:13.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1c280a1260 0x7f1c280a3770 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c441036a0 0x7f1c441a1610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c44101be0 0x7f1c4419a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 --2- 172.21.15.131:0/44819448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c44102d00 0x7f1c4419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:13.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 -- 172.21.15.131:0/44819448 >> 172.21.15.131:0/44819448 conn(0x7f1c440fd1d0 msgr2=0x7f1c44073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:13.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 -- 172.21.15.131:0/44819448 shutdown_connections 2024-05-10T05:33:13.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:13.843+0000 7f1c49e00700 1 -- 172.21.15.131:0/44819448 wait complete. 2024-05-10T05:33:13.871 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T05:33:14.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.140+0000 7fe971200700 1 -- 172.21.15.131:0/128206270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96c1024b0 msgr2=0x7fe96c1048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.140+0000 7fe971200700 1 --2- 172.21.15.131:0/128206270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96c1024b0 0x7fe96c1048d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0098f0 tx=0x7fe95c009c00 comp rx=0 tx=0).stop 2024-05-10T05:33:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.141+0000 7fe971200700 1 -- 172.21.15.131:0/128206270 shutdown_connections 2024-05-10T05:33:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.141+0000 7fe971200700 1 --2- 172.21.15.131:0/128206270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96c107770 0x7fe96c109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.141+0000 7fe971200700 1 --2- 172.21.15.131:0/128206270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96c1024b0 0x7fe96c1048d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.141+0000 7fe971200700 1 --2- 172.21.15.131:0/128206270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c104e10 0x7fe96c107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.141+0000 7fe971200700 1 -- 172.21.15.131:0/128206270 >> 172.21.15.131:0/128206270 conn(0x7fe96c0fbef0 msgr2=0x7fe96c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.141+0000 7fe971200700 1 -- 172.21.15.131:0/128206270 shutdown_connections 2024-05-10T05:33:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.141+0000 7fe971200700 1 -- 172.21.15.131:0/128206270 wait complete. 2024-05-10T05:33:14.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe971200700 1 Processor -- start 2024-05-10T05:33:14.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe971200700 1 -- start start 2024-05-10T05:33:14.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe971200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96c1024b0 0x7fe96c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:14.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe971200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c104e10 0x7fe96c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe971200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96c107770 0x7fe96c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe971200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c19f900 con 0x7fe96c104e10 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe971200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c19fa40 con 0x7fe96c107770 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe971200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c19fbd0 con 0x7fe96c1024b0 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe96b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96c1024b0 0x7fe96c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe96aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c104e10 0x7fe96c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe96b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96c1024b0 0x7fe96c197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45094/0 (socket says 172.21.15.131:45094) 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe96b400700 1 -- 172.21.15.131:0/3043288189 learned_addr learned my addr 172.21.15.131:0/3043288189 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.142+0000 7fe96be00700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96c107770 0x7fe96c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe96b400700 1 -- 172.21.15.131:0/3043288189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96c107770 msgr2=0x7fe96c19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe96b400700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96c107770 0x7fe96c19f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe96b400700 1 -- 172.21.15.131:0/3043288189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c104e10 msgr2=0x7fe96c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe96b400700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c104e10 0x7fe96c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe96b400700 1 -- 172.21.15.131:0/3043288189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580095f0 con 0x7fe96c1024b0 2024-05-10T05:33:14.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe96b400700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96c1024b0 0x7fe96c197e60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00fb70 tx=0x7fe95c00bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:14.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe963e00700 1 -- 172.21.15.131:0/3043288189 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c01c070 con 0x7fe96c1024b0 2024-05-10T05:33:14.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe971200700 1 -- 172.21.15.131:0/3043288189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c009520 con 0x7fe96c1024b0 2024-05-10T05:33:14.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe963e00700 1 -- 172.21.15.131:0/3043288189 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe95c005040 con 0x7fe96c1024b0 2024-05-10T05:33:14.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe963e00700 1 -- 172.21.15.131:0/3043288189 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c003b20 con 0x7fe96c1024b0 2024-05-10T05:33:14.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.143+0000 7fe971200700 1 -- 172.21.15.131:0/3043288189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c1a0140 con 0x7fe96c1024b0 2024-05-10T05:33:14.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.144+0000 7fe971200700 1 -- 172.21.15.131:0/3043288189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96c0fdaa0 con 0x7fe96c1024b0 2024-05-10T05:33:14.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.146+0000 7fe963e00700 1 -- 172.21.15.131:0/3043288189 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe95c0051b0 con 0x7fe96c1024b0 2024-05-10T05:33:14.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.147+0000 7fe963e00700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9500a11b0 0x7fe9500a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:14.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.148+0000 7fe963e00700 1 -- 172.21.15.131:0/3043288189 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe95c0cd6c0 con 0x7fe96c1024b0 2024-05-10T05:33:14.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.148+0000 7fe96aa00700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9500a11b0 0x7fe9500a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:14.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.148+0000 7fe96aa00700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9500a11b0 0x7fe9500a36c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe96c198f00 tx=0x7fe958009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:14.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.152+0000 7fe963e00700 1 -- 172.21.15.131:0/3043288189 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe95c096d80 con 0x7fe96c1024b0 2024-05-10T05:33:14.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.362+0000 7fe971200700 1 -- 172.21.15.131:0/3043288189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe96c10d3a0 con 0x7fe96c1024b0 2024-05-10T05:33:14.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.364+0000 7fe963e00700 1 -- 172.21.15.131:0/3043288189 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe95c09a3a0 con 0x7fe96c1024b0 2024-05-10T05:33:14.364 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-10T05:33:14.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 -- 172.21.15.131:0/3043288189 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9500a11b0 msgr2=0x7fe9500a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:14.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9500a11b0 0x7fe9500a36c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe96c198f00 tx=0x7fe958009310 comp rx=0 tx=0).stop 2024-05-10T05:33:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 -- 172.21.15.131:0/3043288189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96c1024b0 msgr2=0x7fe96c197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96c1024b0 0x7fe96c197e60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00fb70 tx=0x7fe95c00bef0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 -- 172.21.15.131:0/3043288189 shutdown_connections 2024-05-10T05:33:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9500a11b0 0x7fe9500a36c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96c107770 0x7fe96c19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96c1024b0 0x7fe96c197e60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 --2- 172.21.15.131:0/3043288189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe96c104e10 0x7fe96c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 -- 172.21.15.131:0/3043288189 >> 172.21.15.131:0/3043288189 conn(0x7fe96c0fbef0 msgr2=0x7fe96c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:14.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 -- 172.21.15.131:0/3043288189 shutdown_connections 2024-05-10T05:33:14.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.366+0000 7fe971200700 1 -- 172.21.15.131:0/3043288189 wait complete. 2024-05-10T05:33:14.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:14 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3234145545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:14.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:14 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/44819448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:14.393 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T05:33:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:14 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3234145545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:14 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/44819448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:14.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:14 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3234145545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:14.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:14 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/44819448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:14.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.658+0000 7fb72d200700 1 -- 172.21.15.131:0/1171609428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728104d10 msgr2=0x7fb7281071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.658+0000 7fb72d200700 1 --2- 172.21.15.131:0/1171609428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728104d10 0x7fb7281071f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb7140098f0 tx=0x7fb714009c00 comp rx=0 tx=0).stop 2024-05-10T05:33:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.658+0000 7fb72d200700 1 -- 172.21.15.131:0/1171609428 shutdown_connections 2024-05-10T05:33:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.658+0000 7fb72d200700 1 --2- 172.21.15.131:0/1171609428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb728069090 0x7fb728104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.658+0000 7fb72d200700 1 --2- 172.21.15.131:0/1171609428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb728107880 0x7fb728109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.658+0000 7fb72d200700 1 --2- 172.21.15.131:0/1171609428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728104d10 0x7fb7281071f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.658+0000 7fb72d200700 1 -- 172.21.15.131:0/1171609428 >> 172.21.15.131:0/1171609428 conn(0x7fb7280fc0d0 msgr2=0x7fb7280fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.659+0000 7fb72d200700 1 -- 172.21.15.131:0/1171609428 shutdown_connections 2024-05-10T05:33:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.659+0000 7fb72d200700 1 -- 172.21.15.131:0/1171609428 wait complete. 2024-05-10T05:33:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.659+0000 7fb72d200700 1 Processor -- start 2024-05-10T05:33:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb72d200700 1 -- start start 2024-05-10T05:33:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb72d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb728069090 0x7fb72819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb72d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728104d10 0x7fb72819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb72d200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb728107880 0x7fb7281a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb72d200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7281a1c90 con 0x7fb728107880 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb72d200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7281a1e00 con 0x7fb728069090 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb72d200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7281a1fc0 con 0x7fb728104d10 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb726800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb728107880 0x7fb7281a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb725400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728104d10 0x7fb72819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb726800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb728107880 0x7fb7281a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52110/0 (socket says 172.21.15.131:52110) 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb726800700 1 -- 172.21.15.131:0/1234960431 learned_addr learned my addr 172.21.15.131:0/1234960431 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.660+0000 7fb725e00700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb728069090 0x7fb72819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb725400700 1 -- 172.21.15.131:0/1234960431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb728069090 msgr2=0x7fb72819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb725400700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb728069090 0x7fb72819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb725400700 1 -- 172.21.15.131:0/1234960431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb728107880 msgr2=0x7fb7281a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb725400700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb728107880 0x7fb7281a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb725400700 1 -- 172.21.15.131:0/1234960431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb714009520 con 0x7fb728104d10 2024-05-10T05:33:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb725400700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728104d10 0x7fb72819a6f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb714000c00 tx=0x7fb71400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb71ea00700 1 -- 172.21.15.131:0/1234960431 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71401c070 con 0x7fb728104d10 2024-05-10T05:33:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb71ea00700 1 -- 172.21.15.131:0/1234960431 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb71400bbc0 con 0x7fb728104d10 2024-05-10T05:33:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb72d200700 1 -- 172.21.15.131:0/1234960431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7281a2240 con 0x7fb728104d10 2024-05-10T05:33:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb71ea00700 1 -- 172.21.15.131:0/1234960431 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714003b40 con 0x7fb728104d10 2024-05-10T05:33:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.661+0000 7fb72d200700 1 -- 172.21.15.131:0/1234960431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7281a26c0 con 0x7fb728104d10 2024-05-10T05:33:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.662+0000 7fb72d200700 1 -- 172.21.15.131:0/1234960431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7280694d0 con 0x7fb728104d10 2024-05-10T05:33:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.664+0000 7fb71ea00700 1 -- 172.21.15.131:0/1234960431 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb714003ca0 con 0x7fb728104d10 2024-05-10T05:33:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.666+0000 7fb71ea00700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb70c0a12e0 0x7fb70c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.666+0000 7fb71ea00700 1 -- 172.21.15.131:0/1234960431 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb7140cda70 con 0x7fb728104d10 2024-05-10T05:33:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.666+0000 7fb725e00700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb70c0a12e0 0x7fb70c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.666+0000 7fb725e00700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb70c0a12e0 0x7fb70c0a37f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb7280694a0 tx=0x7fb718009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:14.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.668+0000 7fb71ea00700 1 -- 172.21.15.131:0/1234960431 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb7140970d0 con 0x7fb728104d10 2024-05-10T05:33:14.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.909+0000 7fb72d200700 1 -- 172.21.15.131:0/1234960431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb72810d660 con 0x7fb728104d10 2024-05-10T05:33:14.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.910+0000 7fb71ea00700 1 -- 172.21.15.131:0/1234960431 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb71409a6f0 con 0x7fb728104d10 2024-05-10T05:33:14.911 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 -- 172.21.15.131:0/1234960431 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb70c0a12e0 msgr2=0x7fb70c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb70c0a12e0 0x7fb70c0a37f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb7280694a0 tx=0x7fb718009310 comp rx=0 tx=0).stop 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 -- 172.21.15.131:0/1234960431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728104d10 msgr2=0x7fb72819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728104d10 0x7fb72819a6f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb714000c00 tx=0x7fb71400b790 comp rx=0 tx=0).stop 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 -- 172.21.15.131:0/1234960431 shutdown_connections 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb70c0a12e0 0x7fb70c0a37f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb728107880 0x7fb7281a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb728069090 0x7fb72819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 --2- 172.21.15.131:0/1234960431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728104d10 0x7fb72819a6f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 -- 172.21.15.131:0/1234960431 >> 172.21.15.131:0/1234960431 conn(0x7fb7280fc0d0 msgr2=0x7fb7280fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:14.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 -- 172.21.15.131:0/1234960431 shutdown_connections 2024-05-10T05:33:14.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:14.913+0000 7fb72d200700 1 -- 172.21.15.131:0/1234960431 wait complete. 2024-05-10T05:33:14.941 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:33:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:15 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3043288189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:15 smithi131 ceph-mon[37264]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:15 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1234960431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:15.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.210+0000 7ff552200700 1 -- 172.21.15.131:0/624953877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c101be0 msgr2=0x7ff54c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:15.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.210+0000 7ff552200700 1 --2- 172.21.15.131:0/624953877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c101be0 0x7ff54c101ff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009990 tx=0x7ff53c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.211+0000 7ff552200700 1 -- 172.21.15.131:0/624953877 shutdown_connections 2024-05-10T05:33:15.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.211+0000 7ff552200700 1 --2- 172.21.15.131:0/624953877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c101be0 0x7ff54c101ff0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.211+0000 7ff552200700 1 --2- 172.21.15.131:0/624953877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c1036a0 0x7ff54c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.211+0000 7ff552200700 1 --2- 172.21.15.131:0/624953877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c102d00 0x7ff54c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.211+0000 7ff552200700 1 -- 172.21.15.131:0/624953877 >> 172.21.15.131:0/624953877 conn(0x7ff54c0fd1d0 msgr2=0x7ff54c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:15.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.211+0000 7ff552200700 1 -- 172.21.15.131:0/624953877 shutdown_connections 2024-05-10T05:33:15.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.211+0000 7ff552200700 1 -- 172.21.15.131:0/624953877 wait complete. 2024-05-10T05:33:15.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.212+0000 7ff552200700 1 Processor -- start 2024-05-10T05:33:15.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.212+0000 7ff552200700 1 -- start start 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff552200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c101be0 0x7ff54c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff552200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c102d00 0x7ff54c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff552200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff552200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1a1d70 con 0x7ff54c1036a0 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff552200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1a1ee0 con 0x7ff54c102d00 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff552200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1a20a0 con 0x7ff54c101be0 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c102d00 0x7ff54c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c102d00 0x7ff54c19a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39926/0 (socket says 172.21.15.131:39926) 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54aa00700 1 -- 172.21.15.131:0/2926399974 learned_addr learned my addr 172.21.15.131:0/2926399974 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:15.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54b400700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c101be0 0x7ff54c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54be00700 1 -- 172.21.15.131:0/2926399974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c101be0 msgr2=0x7ff54c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54be00700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c101be0 0x7ff54c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54be00700 1 -- 172.21.15.131:0/2926399974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c102d00 msgr2=0x7ff54c19a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54be00700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c102d00 0x7ff54c19a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54be00700 1 -- 172.21.15.131:0/2926399974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0095f0 con 0x7ff54c1036a0 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.213+0000 7ff54aa00700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c102d00 0x7ff54c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.214+0000 7ff54be00700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c1a1760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff54400efc0 tx=0x7ff54400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.214+0000 7ff543e00700 1 -- 172.21.15.131:0/2926399974 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544018070 con 0x7ff54c1036a0 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.214+0000 7ff543e00700 1 -- 172.21.15.131:0/2926399974 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff54400f040 con 0x7ff54c1036a0 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.214+0000 7ff552200700 1 -- 172.21.15.131:0/2926399974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c1a2320 con 0x7ff54c1036a0 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.214+0000 7ff543e00700 1 -- 172.21.15.131:0/2926399974 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400cd40 con 0x7ff54c1036a0 2024-05-10T05:33:15.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.214+0000 7ff552200700 1 -- 172.21.15.131:0/2926399974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c1a27e0 con 0x7ff54c1036a0 2024-05-10T05:33:15.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.215+0000 7ff552200700 1 -- 172.21.15.131:0/2926399974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c0694d0 con 0x7ff54c1036a0 2024-05-10T05:33:15.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.217+0000 7ff543e00700 1 -- 172.21.15.131:0/2926399974 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff544003a00 con 0x7ff54c1036a0 2024-05-10T05:33:15.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.218+0000 7ff543e00700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 0x7ff5300a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:15.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.218+0000 7ff543e00700 1 -- 172.21.15.131:0/2926399974 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff5440cc380 con 0x7ff54c1036a0 2024-05-10T05:33:15.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.218+0000 7ff54b400700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 0x7ff5300a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:15.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.219+0000 7ff54b400700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 0x7ff5300a3630 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006010 tx=0x7ff53c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:15.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.223+0000 7ff543e00700 1 -- 172.21.15.131:0/2926399974 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff544095ba0 con 0x7ff54c1036a0 2024-05-10T05:33:15.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:15 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3043288189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:15.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:15 smithi142 ceph-mon[77667]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:15.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:15 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1234960431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.479+0000 7ff552200700 1 -- 172.21.15.131:0/2926399974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff54c10d410 con 0x7ff54c1036a0 2024-05-10T05:33:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.480+0000 7ff543e00700 1 -- 172.21.15.131:0/2926399974 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10930 (secure 0 0 0) 0x7ff5440991c0 con 0x7ff54c1036a0 2024-05-10T05:33:15.481 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:15.481 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:31:58.482427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:33:15.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 -- 172.21.15.131:0/2926399974 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 msgr2=0x7ff5300a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:15.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 0x7ff5300a3630 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006010 tx=0x7ff53c01a040 comp rx=0 tx=0).stop 2024-05-10T05:33:15.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 -- 172.21.15.131:0/2926399974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 msgr2=0x7ff54c1a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:15.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c1a1760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff54400efc0 tx=0x7ff54400c3a0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 -- 172.21.15.131:0/2926399974 shutdown_connections 2024-05-10T05:33:15.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 0x7ff5300a3630 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c1a1760 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c101be0 0x7ff54c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 --2- 172.21.15.131:0/2926399974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c102d00 0x7ff54c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 -- 172.21.15.131:0/2926399974 >> 172.21.15.131:0/2926399974 conn(0x7ff54c0fd1d0 msgr2=0x7ff54c073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:15.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 -- 172.21.15.131:0/2926399974 shutdown_connections 2024-05-10T05:33:15.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.482+0000 7ff552200700 1 -- 172.21.15.131:0/2926399974 wait complete. 2024-05-10T05:33:15.484 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 26 2024-05-10T05:33:15.513 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:33:15.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:15 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3043288189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:15.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:15 smithi157 ceph-mon[78427]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:15.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:15 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1234960431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:15.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.774+0000 7f5c82e00700 1 -- 172.21.15.131:0/4277188694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c7c1019f0 msgr2=0x7f5c7c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:15.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.774+0000 7f5c82e00700 1 --2- 172.21.15.131:0/4277188694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c7c1019f0 0x7f5c7c101e00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5c700099c0 tx=0x7f5c70009cd0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.774+0000 7f5c82e00700 1 -- 172.21.15.131:0/4277188694 shutdown_connections 2024-05-10T05:33:15.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.774+0000 7f5c82e00700 1 --2- 172.21.15.131:0/4277188694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c1034b0 0x7f5c7c109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.774+0000 7f5c82e00700 1 --2- 172.21.15.131:0/4277188694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c7c1019f0 0x7f5c7c101e00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.774+0000 7f5c82e00700 1 --2- 172.21.15.131:0/4277188694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c7c102b10 0x7f5c7c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.774+0000 7f5c82e00700 1 -- 172.21.15.131:0/4277188694 >> 172.21.15.131:0/4277188694 conn(0x7f5c7c0fcfe0 msgr2=0x7f5c7c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:15.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.775+0000 7f5c82e00700 1 -- 172.21.15.131:0/4277188694 shutdown_connections 2024-05-10T05:33:15.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.775+0000 7f5c82e00700 1 -- 172.21.15.131:0/4277188694 wait complete. 2024-05-10T05:33:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c82e00700 1 Processor -- start 2024-05-10T05:33:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c82e00700 1 -- start start 2024-05-10T05:33:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c82e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c7c1019f0 0x7f5c7c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c82e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c7c102b10 0x7f5c7c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c82e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c1034b0 0x7f5c7c1a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c82e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1a19f0 con 0x7f5c7c102b10 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c82e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1a1b30 con 0x7f5c7c1034b0 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c82e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1a1cc0 con 0x7f5c7c1019f0 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c81000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c7c102b10 0x7f5c7c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c81000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c7c102b10 0x7f5c7c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52160/0 (socket says 172.21.15.131:52160) 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c81000700 1 -- 172.21.15.131:0/373620947 learned_addr learned my addr 172.21.15.131:0/373620947 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c81a00700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c7c1019f0 0x7f5c7c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.776+0000 7f5c82400700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c1034b0 0x7f5c7c1a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c81000700 1 -- 172.21.15.131:0/373620947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c7c1019f0 msgr2=0x7f5c7c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c81000700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c7c1019f0 0x7f5c7c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c81000700 1 -- 172.21.15.131:0/373620947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c1034b0 msgr2=0x7f5c7c1a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c81000700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c1034b0 0x7f5c7c1a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c81000700 1 -- 172.21.15.131:0/373620947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68009520 con 0x7f5c7c102b10 2024-05-10T05:33:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c82400700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c1034b0 0x7f5c7c1a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c81000700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c7c102b10 0x7f5c7c19a490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800eb60 tx=0x7f5c6800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c7a000700 1 -- 172.21.15.131:0/373620947 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800cc90 con 0x7f5c7c102b10 2024-05-10T05:33:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c7a000700 1 -- 172.21.15.131:0/373620947 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c680044e0 con 0x7f5c7c102b10 2024-05-10T05:33:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c82e00700 1 -- 172.21.15.131:0/373620947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c700095f0 con 0x7f5c7c102b10 2024-05-10T05:33:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c7a000700 1 -- 172.21.15.131:0/373620947 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68010430 con 0x7f5c7c102b10 2024-05-10T05:33:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.777+0000 7f5c82e00700 1 -- 172.21.15.131:0/373620947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c1a2230 con 0x7f5c7c102b10 2024-05-10T05:33:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.778+0000 7f5c82e00700 1 -- 172.21.15.131:0/373620947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c068110 con 0x7f5c7c102b10 2024-05-10T05:33:15.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.780+0000 7f5c7a000700 1 -- 172.21.15.131:0/373620947 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5c68003f40 con 0x7f5c7c102b10 2024-05-10T05:33:15.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.782+0000 7f5c7a000700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c640a11c0 0x7f5c640a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:15.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.782+0000 7f5c7a000700 1 -- 172.21.15.131:0/373620947 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5c68014070 con 0x7f5c7c102b10 2024-05-10T05:33:15.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.782+0000 7f5c81a00700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c640a11c0 0x7f5c640a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:15.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.782+0000 7f5c81a00700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c640a11c0 0x7f5c640a36d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5c700098e0 tx=0x7f5c70000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:15.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.784+0000 7f5c7a000700 1 -- 172.21.15.131:0/373620947 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5c68096610 con 0x7f5c7c102b10 2024-05-10T05:33:15.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.993+0000 7f5c82e00700 1 -- 172.21.15.131:0/373620947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c7c10d220 con 0x7f5c7c102b10 2024-05-10T05:33:15.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.994+0000 7f5c7a000700 1 -- 172.21.15.131:0/373620947 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7f5c68099c30 con 0x7f5c7c102b10 2024-05-10T05:33:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:15.996 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:33:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 -- 172.21.15.131:0/373620947 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c640a11c0 msgr2=0x7f5c640a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c640a11c0 0x7f5c640a36d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5c700098e0 tx=0x7f5c70000bc0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 -- 172.21.15.131:0/373620947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c7c102b10 msgr2=0x7f5c7c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c7c102b10 0x7f5c7c19a490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800eb60 tx=0x7f5c6800ee70 comp rx=0 tx=0).stop 2024-05-10T05:33:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 -- 172.21.15.131:0/373620947 shutdown_connections 2024-05-10T05:33:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c640a11c0 0x7f5c640a36d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c1034b0 0x7f5c7c1a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c7c1019f0 0x7f5c7c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 --2- 172.21.15.131:0/373620947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c7c102b10 0x7f5c7c19a490 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 -- 172.21.15.131:0/373620947 >> 172.21.15.131:0/373620947 conn(0x7f5c7c0fcfe0 msgr2=0x7f5c7c073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 -- 172.21.15.131:0/373620947 shutdown_connections 2024-05-10T05:33:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:15.997+0000 7f5c82e00700 1 -- 172.21.15.131:0/373620947 wait complete. 2024-05-10T05:33:16.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T05:31:51.547495+0000', 'modified': '2024-05-10T05:31:58.482427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24581, 'mds_1': 24571, 'mds_2': 24569, 'mds_3': 24565, 'mds_4': 24541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24476': {'gid': 24476, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6833/3035299290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 3035299290}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 3035299290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'd', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6837/3356790716', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 3356790716}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 3356790716}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/3168061272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 3168061272}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 3168061272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/72787128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 72787128}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 72787128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/2645029376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 2645029376}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2645029376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.157:6837/707351222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 707351222}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 707351222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.157:6839/1794088540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 1794088540}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 1794088540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24569': {'gid': 24569, 'name': 'a', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.131:6841/2396760252', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6840', 'nonce': 2396760252}, {'type': 'v1', 'addr': '172.21.15.131:6841', 'nonce': 2396760252}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24571': {'gid': 24571, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.131:6839/2980869223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 2980869223}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 2980869223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6839/2348955278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 2348955278}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 2348955278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T05:33:16.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-10T05:33:16.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T05:33:16.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.290+0000 7f7b9be00700 1 -- 172.21.15.131:0/2057834047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b9c0690b0 msgr2=0x7f7b9c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.290+0000 7f7b9be00700 1 --2- 172.21.15.131:0/2057834047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b9c0690b0 0x7f7b9c104500 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009990 tx=0x7f7b8c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.290+0000 7f7b9be00700 1 -- 172.21.15.131:0/2057834047 shutdown_connections 2024-05-10T05:33:16.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.290+0000 7f7b9be00700 1 --2- 172.21.15.131:0/2057834047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b9c0690b0 0x7f7b9c104500 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.290+0000 7f7b9be00700 1 --2- 172.21.15.131:0/2057834047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b9c107680 0x7f7b9c109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.290+0000 7f7b9be00700 1 --2- 172.21.15.131:0/2057834047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b9c104b10 0x7f7b9c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.290+0000 7f7b9be00700 1 -- 172.21.15.131:0/2057834047 >> 172.21.15.131:0/2057834047 conn(0x7f7b9c0fbef0 msgr2=0x7f7b9c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.290+0000 7f7b9be00700 1 -- 172.21.15.131:0/2057834047 shutdown_connections 2024-05-10T05:33:16.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.291+0000 7f7b9be00700 1 -- 172.21.15.131:0/2057834047 wait complete. 2024-05-10T05:33:16.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.291+0000 7f7b9be00700 1 Processor -- start 2024-05-10T05:33:16.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9be00700 1 -- start start 2024-05-10T05:33:16.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b9c104b10 0x7f7b9c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b9c107680 0x7f7b9c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b9c19f070 0x7f7b9c1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9be00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c1a1a60 con 0x7f7b9c19f070 2024-05-10T05:33:16.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9be00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c1a1ba0 con 0x7f7b9c104b10 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9be00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c1a1d30 con 0x7f7b9c107680 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b9c19f070 0x7f7b9c1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b9c19f070 0x7f7b9c1a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52186/0 (socket says 172.21.15.131:52186) 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9b400700 1 -- 172.21.15.131:0/3331371909 learned_addr learned my addr 172.21.15.131:0/3331371909 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9a000700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b9c107680 0x7f7b9c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.292+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b9c104b10 0x7f7b9c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7b9b400700 1 -- 172.21.15.131:0/3331371909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b9c107680 msgr2=0x7f7b9c19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7b9b400700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b9c107680 0x7f7b9c19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7b9b400700 1 -- 172.21.15.131:0/3331371909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b9c104b10 msgr2=0x7f7b9c199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7b9b400700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b9c104b10 0x7f7b9c199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7b9b400700 1 -- 172.21.15.131:0/3331371909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c0095f0 con 0x7f7b9c19f070 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7b9a000700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b9c107680 0x7f7b9c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7b9b400700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b9c19f070 0x7f7b9c1a1490 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b94007f10 tx=0x7f7b9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7ba0a00700 1 -- 172.21.15.131:0/3331371909 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400ddd0 con 0x7f7b9c19f070 2024-05-10T05:33:16.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7ba0a00700 1 -- 172.21.15.131:0/3331371909 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b9400f040 con 0x7f7b9c19f070 2024-05-10T05:33:16.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7ba0a00700 1 -- 172.21.15.131:0/3331371909 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94015400 con 0x7f7b9c19f070 2024-05-10T05:33:16.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7b9be00700 1 -- 172.21.15.131:0/3331371909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c1a1fb0 con 0x7f7b9c19f070 2024-05-10T05:33:16.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.293+0000 7f7b9be00700 1 -- 172.21.15.131:0/3331371909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c1a2410 con 0x7f7b9c19f070 2024-05-10T05:33:16.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.297+0000 7f7ba0a00700 1 -- 172.21.15.131:0/3331371909 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7b94004020 con 0x7f7b9c19f070 2024-05-10T05:33:16.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.298+0000 7f7bb9c4ab80 1 Processor -- start 2024-05-10T05:33:16.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.298+0000 7f7bb9c4ab80 1 -- start start 2024-05-10T05:33:16.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.298+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c2c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.298+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 0x560886c2db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.298+0000 7f7ba0a00700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b800a11a0 0x7f7b800a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.298+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.298+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886b53100 con 0x560886c2e0a0 2024-05-10T05:33:16.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.298+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886b50ac0 con 0x560886c2d750 2024-05-10T05:33:16.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.298+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886b4c3b0 con 0x560886c2c4e0 2024-05-10T05:33:16.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7ba0a00700 1 -- 172.21.15.131:0/3331371909 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f7b940cc1d0 con 0x7f7b9c19f070 2024-05-10T05:33:16.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b800a11a0 0x7f7b800a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 0x560886c2db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 0x560886c2db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39960/0 (socket says 172.21.15.131:39960) 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9a000700 1 -- 172.21.15.131:0/3989543194 learned_addr learned my addr 172.21.15.131:0/3989543194 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9a000700 1 --2- 172.21.15.131:0/3989543194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c2c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9a000700 1 -- 172.21.15.131:0/3989543194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 msgr2=0x560886c2c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9a000700 1 --2- 172.21.15.131:0/3989543194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c2c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9a000700 1 -- 172.21.15.131:0/3989543194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 msgr2=0x560886c349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9a000700 1 --2- 172.21.15.131:0/3989543194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9a000700 1 -- 172.21.15.131:0/3989543194 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608869cd360 con 0x560886c2d750 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.299+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b800a11a0 0x7f7b800a36b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c003730 tx=0x7f7b8c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.300+0000 7f7b9a000700 1 --2- 172.21.15.131:0/3989543194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 0x560886c2db60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005c50 tx=0x7f7b8800b5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7211a2097f05707f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.300+0000 7f7b87e00700 1 -- 172.21.15.131:0/3989543194 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88014310 con 0x560886c2d750 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.300+0000 7f7b87e00700 1 -- 172.21.15.131:0/3989543194 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b8801a070 con 0x560886c2d750 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.300+0000 7f7b87e00700 1 -- 172.21.15.131:0/3989543194 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88003b50 con 0x560886c2d750 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.301+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3989543194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 msgr2=0x560886c2db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.301+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3989543194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 0x560886c2db60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005c50 tx=0x7f7b8800b5f0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.301+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3989543194 shutdown_connections 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.301+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3989543194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.301+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3989543194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c2c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.301+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3989543194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 0x560886c2db60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.301+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3989543194 >> 172.21.15.131:0/3989543194 conn(0x560886b8ea10 msgr2=0x560886c272e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.301+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3989543194 shutdown_connections 2024-05-10T05:33:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.301+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3989543194 wait complete. 2024-05-10T05:33:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.301+0000 7f7bb9c4ab80 1 -- start start 2024-05-10T05:33:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7bb9c4ab80 1 Processor -- start 2024-05-10T05:33:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2c4e0 0x560886c36950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2d750 0x560886c36e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c3ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886ae8510 con 0x560886c2e0a0 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886ae81b0 con 0x560886c2c4e0 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608869cf650 con 0x560886c2d750 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7bb9c4ab80 10 client.0 _ll_register_callbacks cb 0x560886af1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7b9b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c3ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7b9a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2d750 0x560886c36e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7b9b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c3ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52208/0 (socket says 172.21.15.131:52208) 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7b9b400700 1 -- 172.21.15.131:0/3105493478 learned_addr learned my addr 172.21.15.131:0/3105493478 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.302+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2c4e0 0x560886c36950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b9b400700 1 -- 172.21.15.131:0/3105493478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2d750 msgr2=0x560886c36e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b9b400700 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2d750 0x560886c36e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b9b400700 1 -- 172.21.15.131:0/3105493478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2c4e0 msgr2=0x560886c36950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b9b400700 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2c4e0 0x560886c36950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b9b400700 1 -- 172.21.15.131:0/3105493478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b880147b0 con 0x560886c2e0a0 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b9a000700 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2d750 0x560886c36e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2c4e0 0x560886c36950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b9b400700 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c3ddf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b94099550 tx=0x7f7b94099100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62a1fa5166252ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b84c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b84c00700 1 -- 172.21.15.131:0/3105493478 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b940cb040 con 0x560886c2e0a0 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b84c00700 1 -- 172.21.15.131:0/3105493478 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f7b94099cc0 con 0x560886c2e0a0 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3105493478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560886aa5220 con 0x560886c2e0a0 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7b84c00700 1 -- 172.21.15.131:0/3105493478 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b940b9070 con 0x560886c2e0a0 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7bb9c4ab80 10 client.15168 fetch_fsmap learned FSMap version 26 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.303+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3105493478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560886979b10 con 0x560886c2e0a0 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.304+0000 7f7b84c00700 1 -- 172.21.15.131:0/3105493478 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560886aa5220 con 0x560886c2e0a0 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.304+0000 7f7b84c00700 1 -- 172.21.15.131:0/3105493478 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7b94099820 con 0x560886c2e0a0 2024-05-10T05:33:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.304+0000 7f7bb9c4ab80 10 client.15168 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.304+0000 7f7bb9c4ab80 10 client.15168 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.304+0000 7f7bb9c4ab80 20 client.15168 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.304+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560886c41f60 0x560886c46c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.304+0000 7f7bb9c4ab80 4 client.15168 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.304+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3105493478 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560886bfb730 con 0x560886c41f60 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.304+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560886c41f60 0x560886c46c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.305+0000 7f7b84c00700 10 client.15168 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.305+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560886c41f60 0x560886c46c90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ecb357ae31bc7cb server_cookie=fcf290e8be48f3f3 in_seq=0 out_seq=0 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7b84c00700 1 -- 172.21.15.131:0/3105493478 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560886bfb730 con 0x560886c41f60 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7b84c00700 10 client.15168 handle_command_reply: tid=0 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 client.15168 shutdown 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3105493478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 msgr2=0x560886c3ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c3ddf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b94099550 tx=0x7f7b94099100 comp rx=0 tx=0).stop 2024-05-10T05:33:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3105493478 shutdown_connections 2024-05-10T05:33:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560886c41f60 0x560886c46c90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c3ddf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2c4e0 0x560886c36950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3105493478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2d750 0x560886c36e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3105493478 >> 172.21.15.131:0/3105493478 conn(0x560886b8ea10 msgr2=0x560886c2b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7b84c00700 0 client.15168 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:33:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3105493478 shutdown_connections 2024-05-10T05:33:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3105493478 wait complete. 2024-05-10T05:33:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.306+0000 7f7bb9c4ab80 20 client.15168 trim_cache size 0 max 16384 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.324+0000 7f7bb9c4ab80 1 Processor -- start 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7bb9c4ab80 1 -- start start 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c2db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 0x560886c26b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c35ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886b45190 con 0x560886c2e0a0 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886b49040 con 0x560886c2d750 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886c21630 con 0x560886c2c4e0 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7b9b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c35ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7b9b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c35ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52218/0 (socket says 172.21.15.131:52218) 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7b9b400700 1 -- 172.21.15.131:0/4190815347 learned_addr learned my addr 172.21.15.131:0/4190815347 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/4190815347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c2db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7b9a000700 1 --2- 172.21.15.131:0/4190815347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 0x560886c26b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7b9b400700 1 -- 172.21.15.131:0/4190815347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 msgr2=0x560886c2db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7b9b400700 1 --2- 172.21.15.131:0/4190815347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c2db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7b9b400700 1 -- 172.21.15.131:0/4190815347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 msgr2=0x560886c26b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.325+0000 7f7b9b400700 1 --2- 172.21.15.131:0/4190815347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 0x560886c26b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.326+0000 7f7b9b400700 1 -- 172.21.15.131:0/4190815347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560886b12510 con 0x560886c2e0a0 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.326+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/4190815347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c2db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.326+0000 7f7b9b400700 1 --2- 172.21.15.131:0/4190815347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c35ec0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400d7b0 tx=0x7f7b940c9cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5064768aa5e784b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.326+0000 7f7b91200700 1 -- 172.21.15.131:0/4190815347 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b940cb070 con 0x560886c2e0a0 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.326+0000 7f7b91200700 1 -- 172.21.15.131:0/4190815347 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b94099cc0 con 0x560886c2e0a0 2024-05-10T05:33:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.326+0000 7f7b91200700 1 -- 172.21.15.131:0/4190815347 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9409d120 con 0x560886c2e0a0 2024-05-10T05:33:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/4190815347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 msgr2=0x560886c35ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/4190815347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c35ec0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400d7b0 tx=0x7f7b940c9cf0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/4190815347 shutdown_connections 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/4190815347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2e0a0 0x560886c35ec0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/4190815347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c2db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/4190815347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2d750 0x560886c26b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/4190815347 >> 172.21.15.131:0/4190815347 conn(0x560886b8ea10 msgr2=0x560886c34d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/4190815347 shutdown_connections 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/4190815347 wait complete. 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 -- start start 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.327+0000 7f7bb9c4ab80 1 Processor -- start 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c39830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2d750 0x560886c39d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7bb9c4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2e0a0 0x560886c3ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886c3b190 con 0x560886c2d750 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886c3b300 con 0x560886c2e0a0 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7bb9c4ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886c3b4c0 con 0x560886c2c4e0 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7bb9c4ab80 10 client.0 _ll_register_callbacks cb 0x560886af1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7b9a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2d750 0x560886c39d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7b9b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2e0a0 0x560886c3ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7b9b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2e0a0 0x560886c3ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39980/0 (socket says 172.21.15.131:39980) 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7b9b400700 1 -- 172.21.15.131:0/3140826069 learned_addr learned my addr 172.21.15.131:0/3140826069 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7b9a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2d750 0x560886c39d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52224/0 (socket says 172.21.15.131:52224) 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.328+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c39830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b9b400700 1 -- 172.21.15.131:0/3140826069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 msgr2=0x560886c39830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b9b400700 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c39830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b9b400700 1 -- 172.21.15.131:0/3140826069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2d750 msgr2=0x560886c39d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b9b400700 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2d750 0x560886c39d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b9b400700 1 -- 172.21.15.131:0/3140826069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560886979b10 con 0x560886c2e0a0 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b9a000700 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2d750 0x560886c39d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b9b400700 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2e0a0 0x560886c3ac50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7b94095240 tx=0x7f7b94099230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ccf7eff41f93ca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b84c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b84c00700 1 -- 172.21.15.131:0/3140826069 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b940cb070 con 0x560886c2e0a0 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b84c00700 1 -- 172.21.15.131:0/3140826069 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f7b94099820 con 0x560886c2e0a0 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3140826069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b8801a070 con 0x560886c2e0a0 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b84c00700 1 -- 172.21.15.131:0/3140826069 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b94004bd0 con 0x560886c2e0a0 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7bb9c4ab80 10 client.25184 fetch_fsmap learned FSMap version 26 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3140826069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5608869d1c80 con 0x560886c2e0a0 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.329+0000 7f7b84c00700 1 -- 172.21.15.131:0/3140826069 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b8801a070 con 0x560886c2e0a0 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.330+0000 7f7b84c00700 1 -- 172.21.15.131:0/3140826069 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7b94098520 con 0x560886c2e0a0 2024-05-10T05:33:16.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.330+0000 7f7bb9c4ab80 10 client.25184 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.330+0000 7f7bb9c4ab80 10 client.25184 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.330+0000 7f7bb9c4ab80 20 client.25184 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.330+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560886c9a270 0x560886c98950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.330+0000 7f7bb9c4ab80 4 client.25184 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.330+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3140826069 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560886b44e70 con 0x560886c9a270 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.330+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560886c9a270 0x560886c98950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7b84c00700 10 client.25184 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7b9aa00700 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560886c9a270 0x560886c98950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9be189e878c64225 server_cookie=2c9b7d3b760856c1 in_seq=0 out_seq=0 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7b84c00700 1 -- 172.21.15.131:0/3140826069 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560886b44e70 con 0x560886c9a270 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7b84c00700 10 client.25184 handle_command_reply: tid=0 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 client.25184 shutdown 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3140826069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2e0a0 msgr2=0x560886c3ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2e0a0 0x560886c3ac50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7b94095240 tx=0x7f7b94099230 comp rx=0 tx=0).stop 2024-05-10T05:33:16.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3140826069 shutdown_connections 2024-05-10T05:33:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560886c9a270 0x560886c98950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560886c2e0a0 0x560886c3ac50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560886c2c4e0 0x560886c39830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 --2- 172.21.15.131:0/3140826069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560886c2d750 0x560886c39d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3140826069 >> 172.21.15.131:0/3140826069 conn(0x560886b8ea10 msgr2=0x560886c34cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7b84c00700 0 client.25184 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:33:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3140826069 shutdown_connections 2024-05-10T05:33:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 1 -- 172.21.15.131:0/3140826069 wait complete. 2024-05-10T05:33:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.331+0000 7f7bb9c4ab80 20 client.25184 trim_cache size 0 max 16384 2024-05-10T05:33:16.337 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 4, 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24541, 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 78.293302381999993, 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 101.28652187100001 2024-05-10T05:33:16.338 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:33:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 -- 172.21.15.131:0/3331371909 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b800a11a0 msgr2=0x7f7b800a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b800a11a0 0x7f7b800a36b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c003730 tx=0x7f7b8c005fb0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 -- 172.21.15.131:0/3331371909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b9c19f070 msgr2=0x7f7b9c1a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b9c19f070 0x7f7b9c1a1490 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b94007f10 tx=0x7f7b9400d3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 -- 172.21.15.131:0/3331371909 shutdown_connections 2024-05-10T05:33:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7b800a11a0 0x7f7b800a36b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b9c19f070 0x7f7b9c1a1490 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b9c104b10 0x7f7b9c199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 --2- 172.21.15.131:0/3331371909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b9c107680 0x7f7b9c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 -- 172.21.15.131:0/3331371909 >> 172.21.15.131:0/3331371909 conn(0x7f7b9c0fbef0 msgr2=0x7f7b9c1059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 -- 172.21.15.131:0/3331371909 shutdown_connections 2024-05-10T05:33:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.332+0000 7f7b9be00700 1 -- 172.21.15.131:0/3331371909 wait complete. 2024-05-10T05:33:16.356 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:16 smithi131 ceph-mon[37264]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-10T05:33:16.356 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:16 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2926399974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:16.356 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:16 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/373620947' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:16.358 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-10T05:33:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:16 smithi142 ceph-mon[77667]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-10T05:33:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:16 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2926399974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:16 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/373620947' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:16.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:16 smithi157 ceph-mon[78427]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-10T05:33:16.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:16 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2926399974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:16.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:16 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/373620947' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:16.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.620+0000 7f11dfc00700 1 -- 172.21.15.131:0/2013579924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11d8102d20 msgr2=0x7f11d8103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.620+0000 7f11dfc00700 1 --2- 172.21.15.131:0/2013579924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11d8102d20 0x7f11d8103130 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009990 tx=0x7f11c4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.621+0000 7f11dfc00700 1 -- 172.21.15.131:0/2013579924 shutdown_connections 2024-05-10T05:33:16.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.621+0000 7f11dfc00700 1 --2- 172.21.15.131:0/2013579924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11d81036c0 0x7f11d8109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.621+0000 7f11dfc00700 1 --2- 172.21.15.131:0/2013579924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d8101c00 0x7f11d8102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.621+0000 7f11dfc00700 1 --2- 172.21.15.131:0/2013579924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11d8102d20 0x7f11d8103130 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.621+0000 7f11dfc00700 1 -- 172.21.15.131:0/2013579924 >> 172.21.15.131:0/2013579924 conn(0x7f11d80fd210 msgr2=0x7f11d80ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.621+0000 7f11dfc00700 1 -- 172.21.15.131:0/2013579924 shutdown_connections 2024-05-10T05:33:16.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.621+0000 7f11dfc00700 1 -- 172.21.15.131:0/2013579924 wait complete. 2024-05-10T05:33:16.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dfc00700 1 Processor -- start 2024-05-10T05:33:16.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dfc00700 1 -- start start 2024-05-10T05:33:16.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dfc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11d8101c00 0x7f11d819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dfc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11d8102d20 0x7f11d819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dfc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d81036c0 0x7f11d81a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dfc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d81a1c10 con 0x7f11d8101c00 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dfc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d81a1d80 con 0x7f11d8102d20 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dfc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d81a1f40 con 0x7f11d81036c0 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11d8101c00 0x7f11d819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11d8101c00 0x7f11d819a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52240/0 (socket says 172.21.15.131:52240) 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11dd200700 1 -- 172.21.15.131:0/1992461053 learned_addr learned my addr 172.21.15.131:0/1992461053 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.622+0000 7f11ddc00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d81036c0 0x7f11d81a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11d7e00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11d8102d20 0x7f11d819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11dd200700 1 -- 172.21.15.131:0/1992461053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d81036c0 msgr2=0x7f11d81a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11dd200700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d81036c0 0x7f11d81a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11dd200700 1 -- 172.21.15.131:0/1992461053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11d8102d20 msgr2=0x7f11d819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11dd200700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11d8102d20 0x7f11d819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11dd200700 1 -- 172.21.15.131:0/1992461053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40095f0 con 0x7f11d8101c00 2024-05-10T05:33:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11d7e00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11d8102d20 0x7f11d819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11dd200700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11d8101c00 0x7f11d819a130 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00e940 tx=0x7f11cc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11d5600700 1 -- 172.21.15.131:0/1992461053 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00cbd0 con 0x7f11d8101c00 2024-05-10T05:33:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11d5600700 1 -- 172.21.15.131:0/1992461053 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11cc00ee90 con 0x7f11d8101c00 2024-05-10T05:33:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11dfc00700 1 -- 172.21.15.131:0/1992461053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d81a2220 con 0x7f11d8101c00 2024-05-10T05:33:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11d5600700 1 -- 172.21.15.131:0/1992461053 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc003cb0 con 0x7f11d8101c00 2024-05-10T05:33:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.623+0000 7f11dfc00700 1 -- 172.21.15.131:0/1992461053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d81a2780 con 0x7f11d8101c00 2024-05-10T05:33:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.627+0000 7f11d5600700 1 -- 172.21.15.131:0/1992461053 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f11cc00cd30 con 0x7f11d8101c00 2024-05-10T05:33:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11f7ab3b80 1 Processor -- start 2024-05-10T05:33:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11d5600700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11c00a1240 0x7f11c00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11f7ab3b80 1 -- start start 2024-05-10T05:33:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11d5600700 1 -- 172.21.15.131:0/1992461053 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f11cc014070 con 0x7f11d8101c00 2024-05-10T05:33:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11ddc00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11c00a1240 0x7f11c00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b3650 0x561a9a8b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4520 0x561a9a8b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8bb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a7da100 con 0x561a9a8b4520 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a7d7ac0 con 0x561a9a8b3650 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.628+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a7d33b0 con 0x561a9a8b4e70 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11d7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4520 0x561a9a8b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11dd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b3650 0x561a9a8b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11d7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4520 0x561a9a8b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52256/0 (socket says 172.21.15.131:52256) 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11d7e00700 1 -- 172.21.15.131:0/1352741102 learned_addr learned my addr 172.21.15.131:0/1352741102 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11ddc00700 1 --2- 172.21.15.131:0/1352741102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8bb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11dd200700 1 -- 172.21.15.131:0/1352741102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 msgr2=0x561a9a8bb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11dd200700 1 --2- 172.21.15.131:0/1352741102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8bb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11dd200700 1 -- 172.21.15.131:0/1352741102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4520 msgr2=0x561a9a8b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11dd200700 1 --2- 172.21.15.131:0/1352741102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4520 0x561a9a8b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11dd200700 1 -- 172.21.15.131:0/1352741102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a9a654360 con 0x561a9a8b3650 2024-05-10T05:33:16.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11ddc00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11c00a1240 0x7f11c00a3750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009fd0 tx=0x7f11d000b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11dd200700 1 --2- 172.21.15.131:0/1352741102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b3650 0x561a9a8b3aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0ca5a0 tx=0x7f11cc099a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f663f1ef5db54c9a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.629+0000 7f11ca000700 1 -- 172.21.15.131:0/1352741102 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0b1070 con 0x561a9a8b3650 2024-05-10T05:33:16.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.630+0000 7f11ca000700 1 -- 172.21.15.131:0/1352741102 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11cc099ef0 con 0x561a9a8b3650 2024-05-10T05:33:16.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.630+0000 7f11ca000700 1 -- 172.21.15.131:0/1352741102 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc09a150 con 0x561a9a8b3650 2024-05-10T05:33:16.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.630+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/1352741102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b3650 msgr2=0x561a9a8b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.630+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/1352741102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b3650 0x561a9a8b3aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0ca5a0 tx=0x7f11cc099a30 comp rx=0 tx=0).stop 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.630+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/1352741102 shutdown_connections 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.630+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/1352741102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8bb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.630+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/1352741102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b3650 0x561a9a8b3aa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.630+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/1352741102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4520 0x561a9a8b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.630+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/1352741102 >> 172.21.15.131:0/1352741102 conn(0x561a9a815a10 msgr2=0x561a9a8adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.631+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/1352741102 shutdown_connections 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.631+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/1352741102 wait complete. 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.631+0000 7f11f7ab3b80 1 -- start start 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.631+0000 7f11f7ab3b80 1 Processor -- start 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b3650 0x561a9a8bd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4e70 0x561a9a8c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a76f510 con 0x561a9a8b4e70 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a76f1b0 con 0x561a9a8b4520 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a656650 con 0x561a9a8b3650 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11f7ab3b80 10 client.0 _ll_register_callbacks cb 0x561a9a778ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11ddc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4e70 0x561a9a8c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11d7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11ddc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4e70 0x561a9a8c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52264/0 (socket says 172.21.15.131:52264) 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11d7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8bdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:40010/0 (socket says 172.21.15.131:40010) 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11ddc00700 1 -- 172.21.15.131:0/392124576 learned_addr learned my addr 172.21.15.131:0/392124576 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11dd200700 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b3650 0x561a9a8bd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11ddc00700 1 -- 172.21.15.131:0/392124576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b3650 msgr2=0x561a9a8bd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11ddc00700 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b3650 0x561a9a8bd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11ddc00700 1 -- 172.21.15.131:0/392124576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 msgr2=0x561a9a8bdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11ddc00700 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8bdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11ddc00700 1 -- 172.21.15.131:0/392124576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d81a2780 con 0x561a9a8b4e70 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.632+0000 7f11dd200700 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b3650 0x561a9a8bd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.633+0000 7f11ddc00700 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4e70 0x561a9a8c4b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f11d00045f0 tx=0x7f11d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c1734f71ef5940 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.633+0000 7f11bea00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.633+0000 7f11bea00700 1 -- 172.21.15.131:0/392124576 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d000e040 con 0x561a9a8b4e70 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.633+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/392124576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a9a72c220 con 0x561a9a8b4e70 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.633+0000 7f11bea00700 1 -- 172.21.15.131:0/392124576 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f11d0017ce0 con 0x561a9a8b4e70 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.633+0000 7f11bea00700 1 -- 172.21.15.131:0/392124576 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11d000fe70 con 0x561a9a8b4e70 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.633+0000 7f11f7ab3b80 10 client.15186 fetch_fsmap learned FSMap version 26 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.633+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/392124576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561a9a600b10 con 0x561a9a8b4e70 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.634+0000 7f11bea00700 1 -- 172.21.15.131:0/392124576 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a9a72c220 con 0x561a9a8b4e70 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.634+0000 7f11bea00700 1 -- 172.21.15.131:0/392124576 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f11d0027030 con 0x561a9a8b4e70 2024-05-10T05:33:16.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.634+0000 7f11f7ab3b80 10 client.15186 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:33:16.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.634+0000 7f11f7ab3b80 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T05:33:16.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.634+0000 7f11f7ab3b80 20 client.15186 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:16.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.634+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561a9a8c7d10 0x561a9a8cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.634+0000 7f11f7ab3b80 4 client.15186 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:33:16.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.634+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/392124576 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a9a882730 con 0x561a9a8c7d10 2024-05-10T05:33:16.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.634+0000 7f11dd200700 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561a9a8c7d10 0x561a9a8cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.635+0000 7f11bea00700 10 client.15186 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:33:16.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.635+0000 7f11dd200700 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561a9a8c7d10 0x561a9a8cdb60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=119839c17b8f0a22 server_cookie=9de5318bec9d63f9 in_seq=0 out_seq=0 2024-05-10T05:33:16.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11bea00700 1 -- 172.21.15.131:0/392124576 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a9a882730 con 0x561a9a8c7d10 2024-05-10T05:33:16.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11bea00700 10 client.15186 handle_command_reply: tid=0 2024-05-10T05:33:16.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 client.15186 shutdown 2024-05-10T05:33:16.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/392124576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4e70 msgr2=0x561a9a8c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4e70 0x561a9a8c4b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f11d00045f0 tx=0x7f11d0004ad0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/392124576 shutdown_connections 2024-05-10T05:33:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561a9a8c7d10 0x561a9a8cdb60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b4e70 0x561a9a8c4b90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b3650 0x561a9a8bd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/392124576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8bdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11bea00700 0 client.15186 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:33:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/392124576 >> 172.21.15.131:0/392124576 conn(0x561a9a815a10 msgr2=0x561a9a8adf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/392124576 shutdown_connections 2024-05-10T05:33:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/392124576 wait complete. 2024-05-10T05:33:16.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.636+0000 7f11f7ab3b80 20 client.15186 trim_cache size 0 max 16384 2024-05-10T05:33:16.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.654+0000 7f11f7ab3b80 1 Processor -- start 2024-05-10T05:33:16.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.654+0000 7f11f7ab3b80 1 -- start start 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a7d0010 con 0x561a9a8b3650 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a7725a0 con 0x561a9a8b4520 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a8aace0 con 0x561a9a8b4e70 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11dd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11dd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52278/0 (socket says 172.21.15.131:52278) 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11dd200700 1 -- 172.21.15.131:0/3889187781 learned_addr learned my addr 172.21.15.131:0/3889187781 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11ddc00700 1 --2- 172.21.15.131:0/3889187781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11d7e00700 1 --2- 172.21.15.131:0/3889187781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11dd200700 1 -- 172.21.15.131:0/3889187781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 msgr2=0x561a9a8adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11dd200700 1 --2- 172.21.15.131:0/3889187781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11dd200700 1 -- 172.21.15.131:0/3889187781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 msgr2=0x561a9a8bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11dd200700 1 --2- 172.21.15.131:0/3889187781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8bcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11dd200700 1 -- 172.21.15.131:0/3889187781 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a9a799510 con 0x561a9a8b3650 2024-05-10T05:33:16.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.655+0000 7f11d7e00700 1 --2- 172.21.15.131:0/3889187781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:16.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.656+0000 7f11dd200700 1 --2- 172.21.15.131:0/3889187781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8b4930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c5b0 tx=0x7f11cc003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6a78aabe87d62f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.656+0000 7f11caa00700 1 -- 172.21.15.131:0/3889187781 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0b1070 con 0x561a9a8b3650 2024-05-10T05:33:16.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.656+0000 7f11caa00700 1 -- 172.21.15.131:0/3889187781 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11cc0980b0 con 0x561a9a8b3650 2024-05-10T05:33:16.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.656+0000 7f11caa00700 1 -- 172.21.15.131:0/3889187781 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc09a150 con 0x561a9a8b3650 2024-05-10T05:33:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/3889187781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 msgr2=0x561a9a8b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/3889187781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8b4930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c5b0 tx=0x7f11cc003780 comp rx=0 tx=0).stop 2024-05-10T05:33:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/3889187781 shutdown_connections 2024-05-10T05:33:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/3889187781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8b4930 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/3889187781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/3889187781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/3889187781 >> 172.21.15.131:0/3889187781 conn(0x561a9a815a10 msgr2=0x561a9a8b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/3889187781 shutdown_connections 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/3889187781 wait complete. 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 -- start start 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.657+0000 7f11f7ab3b80 1 Processor -- start 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11f7ab3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a8c2190 con 0x561a9a8b3650 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a8c2300 con 0x561a9a8b4520 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11f7ab3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a8c24c0 con 0x561a9a8b4e70 2024-05-10T05:33:16.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11f7ab3b80 10 client.0 _ll_register_callbacks cb 0x561a9a778ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11dd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11dd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8c07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52292/0 (socket says 172.21.15.131:52292) 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11dd200700 1 -- 172.21.15.131:0/11621095 learned_addr learned my addr 172.21.15.131:0/11621095 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11ddc00700 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11d7e00700 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11dd200700 1 -- 172.21.15.131:0/11621095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 msgr2=0x561a9a8c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11dd200700 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8c1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11dd200700 1 -- 172.21.15.131:0/11621095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 msgr2=0x561a9a8c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11dd200700 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8c0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.658+0000 7f11dd200700 1 -- 172.21.15.131:0/11621095 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a9a654360 con 0x561a9a8b3650 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11dd200700 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8c07e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00e910 tx=0x7f11cc099b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54c9667f91e6f67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11bea00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:33:16.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11bea00700 1 -- 172.21.15.131:0/11621095 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0b1030 con 0x561a9a8b3650 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11bea00700 1 -- 172.21.15.131:0/11621095 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f11cc0b2070 con 0x561a9a8b3650 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/11621095 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11cc099ef0 con 0x561a9a8b3650 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11bea00700 1 -- 172.21.15.131:0/11621095 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11cc0833e0 con 0x561a9a8b3650 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11f7ab3b80 10 client.15192 fetch_fsmap learned FSMap version 26 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/11621095 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561a9a658c80 con 0x561a9a8b3650 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11bea00700 1 -- 172.21.15.131:0/11621095 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11cc0a6020 con 0x561a9a8b3650 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11bea00700 1 -- 172.21.15.131:0/11621095 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f11cc0a1fe0 con 0x561a9a8b3650 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.659+0000 7f11f7ab3b80 10 client.15192 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.660+0000 7f11f7ab3b80 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.660+0000 7f11f7ab3b80 20 client.15192 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.660+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561a9a9219e0 0x561a9a91fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.660+0000 7f11f7ab3b80 4 client.15192 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.660+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/11621095 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561a9a7cbe70 con 0x561a9a9219e0 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.660+0000 7f11d7e00700 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561a9a9219e0 0x561a9a91fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.660+0000 7f11d7e00700 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561a9a9219e0 0x561a9a91fa30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7feb9e5e0c27ee47 server_cookie=97664b451c18363 in_seq=0 out_seq=0 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.660+0000 7f11bea00700 10 client.15192 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11bea00700 1 -- 172.21.15.131:0/11621095 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561a9a7cbe70 con 0x561a9a9219e0 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11bea00700 10 client.15192 handle_command_reply: tid=0 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 client.15192 shutdown 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/11621095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 msgr2=0x561a9a8c07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8c07e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00e910 tx=0x7f11cc099b30 comp rx=0 tx=0).stop 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/11621095 shutdown_connections 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561a9a9219e0 0x561a9a91fa30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a9a8b3650 0x561a9a8c07e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a9a8b4e70 0x561a9a8c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11bea00700 0 client.15192 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 --2- 172.21.15.131:0/11621095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a9a8b4520 0x561a9a8c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/11621095 >> 172.21.15.131:0/11621095 conn(0x561a9a815a10 msgr2=0x561a9a8b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/11621095 shutdown_connections 2024-05-10T05:33:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 1 -- 172.21.15.131:0/11621095 wait complete. 2024-05-10T05:33:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.661+0000 7f11f7ab3b80 20 client.15192 trim_cache size 0 max 16384 2024-05-10T05:33:16.667 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:33:16.667 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:33:16.667 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 3, 2024-05-10T05:33:16.667 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24565, 2024-05-10T05:33:16.667 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-10T05:33:16.667 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-10T05:33:16.667 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-10T05:33:16.667 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 79.630234434000002, 2024-05-10T05:33:16.668 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-05-10T05:33:16.668 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-10T05:33:16.668 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T05:33:16.668 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 98.030968329000004 2024-05-10T05:33:16.668 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:33:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 -- 172.21.15.131:0/1992461053 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11c00a1240 msgr2=0x7f11c00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11c00a1240 0x7f11c00a3750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009fd0 tx=0x7f11d000b040 comp rx=0 tx=0).stop 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 -- 172.21.15.131:0/1992461053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11d8101c00 msgr2=0x7f11d819a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11d8101c00 0x7f11d819a130 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00e940 tx=0x7f11cc00ec50 comp rx=0 tx=0).stop 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 -- 172.21.15.131:0/1992461053 shutdown_connections 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11c00a1240 0x7f11c00a3750 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11d8101c00 0x7f11d819a130 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d81036c0 0x7f11d81a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 --2- 172.21.15.131:0/1992461053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11d8102d20 0x7f11d819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 -- 172.21.15.131:0/1992461053 >> 172.21.15.131:0/1992461053 conn(0x7f11d80fd210 msgr2=0x7f11d8073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 -- 172.21.15.131:0/1992461053 shutdown_connections 2024-05-10T05:33:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.662+0000 7f11dfc00700 1 -- 172.21.15.131:0/1992461053 wait complete. 2024-05-10T05:33:16.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T05:33:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.952+0000 7f0d9ec00700 1 -- 172.21.15.131:0/2792910488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d98102d00 msgr2=0x7f0d98103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.952+0000 7f0d9ec00700 1 --2- 172.21.15.131:0/2792910488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d98102d00 0x7f0d98103110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d840099c0 tx=0x7f0d84009cd0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.953+0000 7f0d9ec00700 1 -- 172.21.15.131:0/2792910488 shutdown_connections 2024-05-10T05:33:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.953+0000 7f0d9ec00700 1 --2- 172.21.15.131:0/2792910488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d981036a0 0x7f0d98109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.953+0000 7f0d9ec00700 1 --2- 172.21.15.131:0/2792910488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d98101be0 0x7f0d98101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.953+0000 7f0d9ec00700 1 --2- 172.21.15.131:0/2792910488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d98102d00 0x7f0d98103110 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.953+0000 7f0d9ec00700 1 -- 172.21.15.131:0/2792910488 >> 172.21.15.131:0/2792910488 conn(0x7f0d980fd210 msgr2=0x7f0d980ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.953+0000 7f0d9ec00700 1 -- 172.21.15.131:0/2792910488 shutdown_connections 2024-05-10T05:33:16.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.953+0000 7f0d9ec00700 1 -- 172.21.15.131:0/2792910488 wait complete. 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.954+0000 7f0d9ec00700 1 Processor -- start 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.954+0000 7f0d9ec00700 1 -- start start 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d9ec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d98101be0 0x7f0d9819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d9ec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d98102d00 0x7f0d9819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d9ec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d981036a0 0x7f0d981a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d9ec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d981a1c20 con 0x7f0d98101be0 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d9ec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d981a1d90 con 0x7f0d981036a0 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d9ec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d981a1f50 con 0x7f0d98102d00 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d97e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d98101be0 0x7f0d9819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d97e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d98101be0 0x7f0d9819a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52304/0 (socket says 172.21.15.131:52304) 2024-05-10T05:33:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d97e00700 1 -- 172.21.15.131:0/1503369232 learned_addr learned my addr 172.21.15.131:0/1503369232 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d97400700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d98102d00 0x7f0d9819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d9cc00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d981036a0 0x7f0d981a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d97e00700 1 -- 172.21.15.131:0/1503369232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d98102d00 msgr2=0x7f0d9819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.955+0000 7f0d97e00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d98102d00 0x7f0d9819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.956+0000 7f0d97e00700 1 -- 172.21.15.131:0/1503369232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d981036a0 msgr2=0x7f0d981a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.956+0000 7f0d97e00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d981036a0 0x7f0d981a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.956+0000 7f0d97e00700 1 -- 172.21.15.131:0/1503369232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d840095f0 con 0x7f0d98101be0 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.956+0000 7f0d9cc00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d981036a0 0x7f0d981a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.956+0000 7f0d97e00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d98101be0 0x7f0d9819a140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800e940 tx=0x7f0d8800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.956+0000 7f0d94c00700 1 -- 172.21.15.131:0/1503369232 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800cbd0 con 0x7f0d98101be0 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.956+0000 7f0d94c00700 1 -- 172.21.15.131:0/1503369232 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8800ee90 con 0x7f0d98101be0 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.956+0000 7f0d9ec00700 1 -- 172.21.15.131:0/1503369232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d981a2230 con 0x7f0d98101be0 2024-05-10T05:33:16.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.956+0000 7f0d94c00700 1 -- 172.21.15.131:0/1503369232 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88003cb0 con 0x7f0d98101be0 2024-05-10T05:33:16.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.956+0000 7f0d9ec00700 1 -- 172.21.15.131:0/1503369232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d981a2740 con 0x7f0d98101be0 2024-05-10T05:33:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.959+0000 7f0d94c00700 1 -- 172.21.15.131:0/1503369232 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0d8800cd30 con 0x7f0d98101be0 2024-05-10T05:33:16.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0d94c00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0d7c0a11c0 0x7f0d7c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0db6abab80 1 Processor -- start 2024-05-10T05:33:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0d94c00700 1 -- 172.21.15.131:0/1503369232 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0d88014070 con 0x7f0d98101be0 2024-05-10T05:33:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0d97400700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0d7c0a11c0 0x7f0d7c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0db6abab80 1 -- start start 2024-05-10T05:33:16.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68157e60 0x55fb681582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb6815ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0db6abab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb68076100 con 0x55fb68158d30 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0db6abab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb68073ac0 con 0x55fb68159680 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.961+0000 7f0db6abab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6806f3b0 con 0x55fb68157e60 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68157e60 0x55fb681582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52314/0 (socket says 172.21.15.131:52314) 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97400700 1 -- 172.21.15.131:0/4228455381 learned_addr learned my addr 172.21.15.131:0/4228455381 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97400700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0d7c0a11c0 0x7f0d7c0a36d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0d9819b1e0 tx=0x7f0d84005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d9cc00700 1 --2- 172.21.15.131:0/4228455381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb6815ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97e00700 1 -- 172.21.15.131:0/4228455381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 msgr2=0x55fb6815ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97e00700 1 --2- 172.21.15.131:0/4228455381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb6815ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97e00700 1 -- 172.21.15.131:0/4228455381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 msgr2=0x55fb68159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97e00700 1 --2- 172.21.15.131:0/4228455381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97e00700 1 -- 172.21.15.131:0/4228455381 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb67ef0360 con 0x55fb68157e60 2024-05-10T05:33:16.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d9cc00700 1 --2- 172.21.15.131:0/4228455381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb6815ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:16.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d97e00700 1 --2- 172.21.15.131:0/4228455381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68157e60 0x55fb681582b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d880ca690 tx=0x7f0d88099e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e91a5e55e00d309 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.962+0000 7f0d8d200700 1 -- 172.21.15.131:0/4228455381 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88099f90 con 0x55fb68157e60 2024-05-10T05:33:16.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.963+0000 7f0d8d200700 1 -- 172.21.15.131:0/4228455381 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8809ddc0 con 0x55fb68157e60 2024-05-10T05:33:16.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.963+0000 7f0d8d200700 1 -- 172.21.15.131:0/4228455381 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8809a4c0 con 0x55fb68157e60 2024-05-10T05:33:16.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.963+0000 7f0db6abab80 1 -- 172.21.15.131:0/4228455381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68157e60 msgr2=0x55fb681582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.963+0000 7f0db6abab80 1 --2- 172.21.15.131:0/4228455381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68157e60 0x55fb681582b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d880ca690 tx=0x7f0d88099e80 comp rx=0 tx=0).stop 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.963+0000 7f0db6abab80 1 -- 172.21.15.131:0/4228455381 shutdown_connections 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.963+0000 7f0db6abab80 1 --2- 172.21.15.131:0/4228455381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb6815ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.963+0000 7f0db6abab80 1 --2- 172.21.15.131:0/4228455381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68157e60 0x55fb681582b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.963+0000 7f0db6abab80 1 --2- 172.21.15.131:0/4228455381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.963+0000 7f0db6abab80 1 -- 172.21.15.131:0/4228455381 >> 172.21.15.131:0/4228455381 conn(0x55fb680b1a10 msgr2=0x55fb68149f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.964+0000 7f0db6abab80 1 -- 172.21.15.131:0/4228455381 shutdown_connections 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.964+0000 7f0db6abab80 1 -- 172.21.15.131:0/4228455381 wait complete. 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.964+0000 7f0db6abab80 1 -- start start 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.965+0000 7f0db6abab80 1 Processor -- start 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.965+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.965+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68158d30 0x55fb68162340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.965+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb681692a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.965+0000 7f0db6abab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6800b510 con 0x55fb68157e60 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.965+0000 7f0db6abab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6800b1b0 con 0x55fb68159680 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.965+0000 7f0db6abab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb67ef2650 con 0x55fb68158d30 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.965+0000 7f0db6abab80 10 client.0 _ll_register_callbacks cb 0x55fb68014ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.965+0000 7f0d97e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d97e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52330/0 (socket says 172.21.15.131:52330) 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d97e00700 1 -- 172.21.15.131:0/2137931311 learned_addr learned my addr 172.21.15.131:0/2137931311 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d97400700 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68158d30 0x55fb68162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d9cc00700 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb681692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d97e00700 1 -- 172.21.15.131:0/2137931311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68158d30 msgr2=0x55fb68162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d97e00700 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68158d30 0x55fb68162340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d97e00700 1 -- 172.21.15.131:0/2137931311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 msgr2=0x55fb681692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d97e00700 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb681692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d97e00700 1 -- 172.21.15.131:0/2137931311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d981a2740 con 0x55fb68157e60 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d97e00700 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68161e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d88099f20 tx=0x7f0d880ca5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab40622972da5959 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d81600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d81600700 1 -- 172.21.15.131:0/2137931311 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d880b8070 con 0x55fb68157e60 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0d81600700 1 -- 172.21.15.131:0/2137931311 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0d88099f90 con 0x55fb68157e60 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.966+0000 7f0db6abab80 1 -- 172.21.15.131:0/2137931311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb67fc8220 con 0x55fb68157e60 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0d81600700 1 -- 172.21.15.131:0/2137931311 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d88099480 con 0x55fb68157e60 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0db6abab80 10 client.15204 fetch_fsmap learned FSMap version 26 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0db6abab80 1 -- 172.21.15.131:0/2137931311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fb67e9cb10 con 0x55fb68157e60 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0d81600700 1 -- 172.21.15.131:0/2137931311 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8807b070 con 0x55fb68157e60 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0d81600700 1 -- 172.21.15.131:0/2137931311 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0d880cab00 con 0x55fb68157e60 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0db6abab80 10 client.15204 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:33:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0db6abab80 10 client.15204 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T05:33:16.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0db6abab80 20 client.15204 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:16.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55fb6816d630 0x55fb68171d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0db6abab80 4 client.15204 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:33:16.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.967+0000 7f0db6abab80 1 -- 172.21.15.131:0/2137931311 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb6811e730 con 0x55fb6816d630 2024-05-10T05:33:16.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.968+0000 7f0d97400700 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55fb6816d630 0x55fb68171d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.968+0000 7f0d81600700 10 client.15204 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:33:16.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.968+0000 7f0d97400700 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55fb6816d630 0x55fb68171d50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b59edafbc9fd0855 server_cookie=1e2d080cc4235b5d in_seq=0 out_seq=0 2024-05-10T05:33:16.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0d81600700 1 -- 172.21.15.131:0/2137931311 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb6811e730 con 0x55fb6816d630 2024-05-10T05:33:16.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0d81600700 10 client.15204 handle_command_reply: tid=0 2024-05-10T05:33:16.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 client.15204 shutdown 2024-05-10T05:33:16.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 -- 172.21.15.131:0/2137931311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 msgr2=0x55fb68161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68161e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d88099f20 tx=0x7f0d880ca5e0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 -- 172.21.15.131:0/2137931311 shutdown_connections 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55fb6816d630 0x55fb68171d50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68161e00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb681692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0d81600700 0 client.15204 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2137931311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68158d30 0x55fb68162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 -- 172.21.15.131:0/2137931311 >> 172.21.15.131:0/2137931311 conn(0x55fb680b1a10 msgr2=0x55fb68149f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 -- 172.21.15.131:0/2137931311 shutdown_connections 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 1 -- 172.21.15.131:0/2137931311 wait complete. 2024-05-10T05:33:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.969+0000 7f0db6abab80 20 client.15204 trim_cache size 0 max 16384 2024-05-10T05:33:16.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.987+0000 7f0db6abab80 1 Processor -- start 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0db6abab80 1 -- start start 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68167250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68158d30 0x55fb68167790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb68175d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0db6abab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6811e730 con 0x55fb68157e60 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0db6abab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6802d2b0 con 0x55fb68159680 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0db6abab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6806c010 con 0x55fb68158d30 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0d97e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68167250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0d97e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68167250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52340/0 (socket says 172.21.15.131:52340) 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0d97e00700 1 -- 172.21.15.131:0/3318662936 learned_addr learned my addr 172.21.15.131:0/3318662936 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0d97400700 1 --2- 172.21.15.131:0/3318662936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68158d30 0x55fb68167790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0d9cc00700 1 --2- 172.21.15.131:0/3318662936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb68175d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.988+0000 7f0d97e00700 1 -- 172.21.15.131:0/3318662936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68158d30 msgr2=0x55fb68167790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.989+0000 7f0d97e00700 1 --2- 172.21.15.131:0/3318662936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68158d30 0x55fb68167790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.989+0000 7f0d97e00700 1 -- 172.21.15.131:0/3318662936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 msgr2=0x55fb68175d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.989+0000 7f0d97e00700 1 --2- 172.21.15.131:0/3318662936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb68175d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.989+0000 7f0d97e00700 1 -- 172.21.15.131:0/3318662936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb68035510 con 0x55fb68157e60 2024-05-10T05:33:16.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.989+0000 7f0d97e00700 1 --2- 172.21.15.131:0/3318662936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68167250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009800 tx=0x7f0d88003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9d893260c03ad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.989+0000 7f0d8dc00700 1 -- 172.21.15.131:0/3318662936 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d880b8040 con 0x55fb68157e60 2024-05-10T05:33:16.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.989+0000 7f0d8dc00700 1 -- 172.21.15.131:0/3318662936 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8807b070 con 0x55fb68157e60 2024-05-10T05:33:16.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.989+0000 7f0d8dc00700 1 -- 172.21.15.131:0/3318662936 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d880cab00 con 0x55fb68157e60 2024-05-10T05:33:16.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 -- 172.21.15.131:0/3318662936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 msgr2=0x55fb68167250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 --2- 172.21.15.131:0/3318662936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68167250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009800 tx=0x7f0d88003680 comp rx=0 tx=0).stop 2024-05-10T05:33:16.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 -- 172.21.15.131:0/3318662936 shutdown_connections 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 --2- 172.21.15.131:0/3318662936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68157e60 0x55fb68167250 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 --2- 172.21.15.131:0/3318662936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68159680 0x55fb68175d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 --2- 172.21.15.131:0/3318662936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68158d30 0x55fb68167790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 -- 172.21.15.131:0/3318662936 >> 172.21.15.131:0/3318662936 conn(0x55fb680b1a10 msgr2=0x55fb6815cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 -- 172.21.15.131:0/3318662936 shutdown_connections 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 -- 172.21.15.131:0/3318662936 wait complete. 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 -- start start 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.990+0000 7f0db6abab80 1 Processor -- start 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68157e60 0x55fb681656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68165c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0db6abab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68159680 0x55fb68166a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0db6abab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb68166fb0 con 0x55fb68158d30 2024-05-10T05:33:16.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0db6abab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb68167120 con 0x55fb68157e60 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0db6abab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb681672e0 con 0x55fb68159680 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0db6abab80 10 client.0 _ll_register_callbacks cb 0x55fb68014ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0d97400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68165c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0d97400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68165c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52352/0 (socket says 172.21.15.131:52352) 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0d97e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68157e60 0x55fb681656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0d97400700 1 -- 172.21.15.131:0/2079091361 learned_addr learned my addr 172.21.15.131:0/2079091361 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0d9cc00700 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68159680 0x55fb68166a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0d97400700 1 -- 172.21.15.131:0/2079091361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68159680 msgr2=0x55fb68166a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.991+0000 7f0d97400700 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68159680 0x55fb68166a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0d97400700 1 -- 172.21.15.131:0/2079091361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68157e60 msgr2=0x55fb681656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0d97400700 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68157e60 0x55fb681656f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0d97400700 1 -- 172.21.15.131:0/2079091361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb67e9cb10 con 0x55fb68158d30 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0d97e00700 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68157e60 0x55fb681656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:16.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0d97400700 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68165c30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009990 tx=0x7f0d84009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf281ac1674c256a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0d81600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0d81600700 1 -- 172.21.15.131:0/2079091361 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8402b040 con 0x55fb68158d30 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0d81600700 1 -- 172.21.15.131:0/2079091361 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0d840323f0 con 0x55fb68158d30 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0db6abab80 1 -- 172.21.15.131:0/2079091361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d8809ddc0 con 0x55fb68158d30 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0d81600700 1 -- 172.21.15.131:0/2079091361 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d8401d040 con 0x55fb68158d30 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0db6abab80 10 client.15210 fetch_fsmap learned FSMap version 26 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.992+0000 7f0db6abab80 1 -- 172.21.15.131:0/2079091361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fb67ef4c80 con 0x55fb68158d30 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0d81600700 1 -- 172.21.15.131:0/2079091361 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8809ddc0 con 0x55fb68158d30 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0d81600700 1 -- 172.21.15.131:0/2079091361 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0d8403bc90 con 0x55fb68158d30 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0db6abab80 10 client.15210 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0db6abab80 10 client.15210 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0db6abab80 20 client.15210 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:16.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55fb681c6190 0x55fb681c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0db6abab80 4 client.15210 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0db6abab80 1 -- 172.21.15.131:0/2079091361 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb68067e70 con 0x55fb681c6190 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0d97e00700 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55fb681c6190 0x55fb681c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0d97e00700 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55fb681c6190 0x55fb681c46f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b598c38f2a3dbd72 server_cookie=b448e21ce8c8f980 in_seq=0 out_seq=0 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.993+0000 7f0d81600700 10 client.15210 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0d81600700 1 -- 172.21.15.131:0/2079091361 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fb68067e70 con 0x55fb681c6190 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0d81600700 10 client.15210 handle_command_reply: tid=0 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 client.15210 shutdown 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 -- 172.21.15.131:0/2079091361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 msgr2=0x55fb68165c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68165c30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009990 tx=0x7f0d84009310 comp rx=0 tx=0).stop 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 -- 172.21.15.131:0/2079091361 shutdown_connections 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55fb681c6190 0x55fb681c46f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb68159680 0x55fb68166a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb68157e60 0x55fb681656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 --2- 172.21.15.131:0/2079091361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb68158d30 0x55fb68165c30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:16.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 -- 172.21.15.131:0/2079091361 >> 172.21.15.131:0/2079091361 conn(0x55fb680b1a10 msgr2=0x55fb68175d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0d81600700 0 client.15210 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:33:17.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 -- 172.21.15.131:0/2079091361 shutdown_connections 2024-05-10T05:33:17.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 1 -- 172.21.15.131:0/2079091361 wait complete. 2024-05-10T05:33:17.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.994+0000 7f0db6abab80 20 client.15210 trim_cache size 0 max 16384 2024-05-10T05:33:17.000 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:33:17.000 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:33:17.000 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-10T05:33:17.000 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24569, 2024-05-10T05:33:17.000 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-10T05:33:17.001 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-10T05:33:17.001 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-10T05:33:17.001 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 80.982107920000004, 2024-05-10T05:33:17.001 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-05-10T05:33:17.001 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-10T05:33:17.001 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T05:33:17.001 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 93.338606119000005 2024-05-10T05:33:17.001 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 -- 172.21.15.131:0/1503369232 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0d7c0a11c0 msgr2=0x7f0d7c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0d7c0a11c0 0x7f0d7c0a36d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0d9819b1e0 tx=0x7f0d84005fb0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 -- 172.21.15.131:0/1503369232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d98101be0 msgr2=0x7f0d9819a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d98101be0 0x7f0d9819a140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800e940 tx=0x7f0d8800ec50 comp rx=0 tx=0).stop 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 -- 172.21.15.131:0/1503369232 shutdown_connections 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0d7c0a11c0 0x7f0d7c0a36d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d98101be0 0x7f0d9819a140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d981036a0 0x7f0d981a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 --2- 172.21.15.131:0/1503369232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d98102d00 0x7f0d9819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 -- 172.21.15.131:0/1503369232 >> 172.21.15.131:0/1503369232 conn(0x7f0d980fd210 msgr2=0x7f0d98073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 -- 172.21.15.131:0/1503369232 shutdown_connections 2024-05-10T05:33:17.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:16.995+0000 7f0d9ec00700 1 -- 172.21.15.131:0/1503369232 wait complete. 2024-05-10T05:33:17.024 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-05-10T05:33:17.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.289+0000 7f7977a00700 1 -- 172.21.15.131:0/3321127376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7970101be0 msgr2=0x7f7970101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.289+0000 7f7977a00700 1 --2- 172.21.15.131:0/3321127376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7970101be0 0x7f7970101ff0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f795c009990 tx=0x7f795c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.289+0000 7f7977a00700 1 -- 172.21.15.131:0/3321127376 shutdown_connections 2024-05-10T05:33:17.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.290+0000 7f7977a00700 1 --2- 172.21.15.131:0/3321127376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970102580 0x7f7970108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.290+0000 7f7977a00700 1 --2- 172.21.15.131:0/3321127376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7970100ac0 0x7f7970100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.290+0000 7f7977a00700 1 --2- 172.21.15.131:0/3321127376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7970101be0 0x7f7970101ff0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.290+0000 7f7977a00700 1 -- 172.21.15.131:0/3321127376 >> 172.21.15.131:0/3321127376 conn(0x7f79700fc0d0 msgr2=0x7f79700fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.290+0000 7f7977a00700 1 -- 172.21.15.131:0/3321127376 shutdown_connections 2024-05-10T05:33:17.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.290+0000 7f7977a00700 1 -- 172.21.15.131:0/3321127376 wait complete. 2024-05-10T05:33:17.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.291+0000 7f7977a00700 1 Processor -- start 2024-05-10T05:33:17.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.291+0000 7f7977a00700 1 -- start start 2024-05-10T05:33:17.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.291+0000 7f7977a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100ac0 0x7f797019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.291+0000 7f7977a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7970101be0 0x7f797019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.291+0000 7f7977a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7970102580 0x7f79701a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.291+0000 7f7977a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79701a1c30 con 0x7f7970102580 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.291+0000 7f7977a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79701a1da0 con 0x7f7970100ac0 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.291+0000 7f7977a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79701a1f60 con 0x7f7970101be0 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f796fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7970101be0 0x7f797019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f7975a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7970102580 0x7f79701a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f7975000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100ac0 0x7f797019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f796fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7970101be0 0x7f797019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45336/0 (socket says 172.21.15.131:45336) 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f796fe00700 1 -- 172.21.15.131:0/3611137035 learned_addr learned my addr 172.21.15.131:0/3611137035 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f7975a00700 1 -- 172.21.15.131:0/3611137035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7970101be0 msgr2=0x7f797019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f7975a00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7970101be0 0x7f797019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f7975a00700 1 -- 172.21.15.131:0/3611137035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100ac0 msgr2=0x7f797019a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f7975a00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100ac0 0x7f797019a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f7975a00700 1 -- 172.21.15.131:0/3611137035 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c0095f0 con 0x7f7970102580 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f7975a00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7970102580 0x7f79701a1620 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f796800efc0 tx=0x7f796800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f796d600700 1 -- 172.21.15.131:0/3611137035 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968018070 con 0x7f7970102580 2024-05-10T05:33:17.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f796d600700 1 -- 172.21.15.131:0/3611137035 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f796800f040 con 0x7f7970102580 2024-05-10T05:33:17.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f796d600700 1 -- 172.21.15.131:0/3611137035 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800cd40 con 0x7f7970102580 2024-05-10T05:33:17.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.292+0000 7f7977a00700 1 -- 172.21.15.131:0/3611137035 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79701a21e0 con 0x7f7970102580 2024-05-10T05:33:17.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.293+0000 7f7977a00700 1 -- 172.21.15.131:0/3611137035 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79701a26a0 con 0x7f7970102580 2024-05-10T05:33:17.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.296+0000 7f796d600700 1 -- 172.21.15.131:0/3611137035 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7968003a00 con 0x7f7970102580 2024-05-10T05:33:17.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f798fa04b80 1 Processor -- start 2024-05-10T05:33:17.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f798fa04b80 1 -- start start 2024-05-10T05:33:17.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f796d600700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f79580a1240 0x7f79580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa68caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68d520 0x563eaa68d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa694770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f798fa04b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa5b3100 con 0x563eaa68de70 2024-05-10T05:33:17.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f798fa04b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa5b0ac0 con 0x563eaa68d520 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f798fa04b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa5ac3b0 con 0x563eaa68c650 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f796d600700 1 -- 172.21.15.131:0/3611137035 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f79680cc3d0 con 0x7f7970102580 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.297+0000 7f7975000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa68caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f7975000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa68caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45342/0 (socket says 172.21.15.131:45342) 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f796fe00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f79580a1240 0x7f79580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f7975000700 1 -- 172.21.15.131:0/1538704805 learned_addr learned my addr 172.21.15.131:0/1538704805 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f796fe00700 1 --2- 172.21.15.131:0/1538704805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68d520 0x563eaa68d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f7975a00700 1 --2- 172.21.15.131:0/1538704805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa694770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f7975000700 1 -- 172.21.15.131:0/1538704805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68d520 msgr2=0x563eaa68d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f7975000700 1 --2- 172.21.15.131:0/1538704805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68d520 0x563eaa68d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f7975000700 1 -- 172.21.15.131:0/1538704805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 msgr2=0x563eaa694770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f7975000700 1 --2- 172.21.15.131:0/1538704805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa694770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f7975000700 1 -- 172.21.15.131:0/1538704805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563eaa42d360 con 0x563eaa68c650 2024-05-10T05:33:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f796fe00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f79580a1240 0x7f79580a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f795c006010 tx=0x7f795c004580 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.298+0000 7f7975000700 1 --2- 172.21.15.131:0/1538704805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa68caa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7964009cf0 tx=0x7f7964011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40aa980afd1e9b76 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.299+0000 7f7962000700 1 -- 172.21.15.131:0/1538704805 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964011d80 con 0x563eaa68c650 2024-05-10T05:33:17.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.299+0000 7f7962000700 1 -- 172.21.15.131:0/1538704805 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7964004cf0 con 0x563eaa68c650 2024-05-10T05:33:17.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.299+0000 7f7962000700 1 -- 172.21.15.131:0/1538704805 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964019430 con 0x563eaa68c650 2024-05-10T05:33:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.299+0000 7f798fa04b80 1 -- 172.21.15.131:0/1538704805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 msgr2=0x563eaa68caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.299+0000 7f798fa04b80 1 --2- 172.21.15.131:0/1538704805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa68caa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7964009cf0 tx=0x7f7964011440 comp rx=0 tx=0).stop 2024-05-10T05:33:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.300+0000 7f798fa04b80 1 -- 172.21.15.131:0/1538704805 shutdown_connections 2024-05-10T05:33:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.300+0000 7f798fa04b80 1 --2- 172.21.15.131:0/1538704805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa694770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.300+0000 7f798fa04b80 1 --2- 172.21.15.131:0/1538704805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa68caa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.300+0000 7f798fa04b80 1 --2- 172.21.15.131:0/1538704805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68d520 0x563eaa68d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.300+0000 7f798fa04b80 1 -- 172.21.15.131:0/1538704805 >> 172.21.15.131:0/1538704805 conn(0x563eaa5eea10 msgr2=0x563eaa686f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.300+0000 7f798fa04b80 1 -- 172.21.15.131:0/1538704805 shutdown_connections 2024-05-10T05:33:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.300+0000 7f798fa04b80 1 -- 172.21.15.131:0/1538704805 wait complete. 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.300+0000 7f798fa04b80 1 -- start start 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.300+0000 7f798fa04b80 1 Processor -- start 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 0x563eaa696650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68d520 0x563eaa696b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68de70 0x563eaa69daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f798fa04b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa548510 con 0x563eaa68d520 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f798fa04b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa5481b0 con 0x563eaa68c650 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f798fa04b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa42f650 con 0x563eaa68de70 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f798fa04b80 10 client.0 _ll_register_callbacks cb 0x563eaa551ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f796fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68d520 0x563eaa696b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f796fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68d520 0x563eaa696b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52380/0 (socket says 172.21.15.131:52380) 2024-05-10T05:33:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f796fe00700 1 -- 172.21.15.131:0/2493597854 learned_addr learned my addr 172.21.15.131:0/2493597854 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f7975000700 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 0x563eaa696650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.301+0000 7f7975a00700 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68de70 0x563eaa69daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f796fe00700 1 -- 172.21.15.131:0/2493597854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68de70 msgr2=0x563eaa69daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f796fe00700 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68de70 0x563eaa69daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f796fe00700 1 -- 172.21.15.131:0/2493597854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 msgr2=0x563eaa696650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f796fe00700 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 0x563eaa696650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f796fe00700 1 -- 172.21.15.131:0/2493597854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7964009920 con 0x563eaa68d520 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f796fe00700 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68d520 0x563eaa696b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f795c005910 tx=0x7f795c00b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372cb52dd3a3a531 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f7956a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f7956a00700 1 -- 172.21.15.131:0/2493597854 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c02e070 con 0x563eaa68d520 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f7956a00700 1 -- 172.21.15.131:0/2493597854 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f795c004e00 con 0x563eaa68d520 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f798fa04b80 1 -- 172.21.15.131:0/2493597854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563eaa505220 con 0x563eaa68d520 2024-05-10T05:33:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f7956a00700 1 -- 172.21.15.131:0/2493597854 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f795c005560 con 0x563eaa68d520 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f798fa04b80 10 client.15222 fetch_fsmap learned FSMap version 26 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f798fa04b80 1 -- 172.21.15.131:0/2493597854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563eaa3d9b10 con 0x563eaa68d520 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.302+0000 7f7956a00700 1 -- 172.21.15.131:0/2493597854 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563eaa505220 con 0x563eaa68d520 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.303+0000 7f7956a00700 1 -- 172.21.15.131:0/2493597854 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f795c028460 con 0x563eaa68d520 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.303+0000 7f798fa04b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.303+0000 7f798fa04b80 10 client.15222 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.303+0000 7f798fa04b80 20 client.15222 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.303+0000 7f798fa04b80 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563eaa6a1100 0x563eaa6a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.303+0000 7f798fa04b80 4 client.15222 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.303+0000 7f798fa04b80 1 -- 172.21.15.131:0/2493597854 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563eaa65b730 con 0x563eaa6a1100 2024-05-10T05:33:17.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.303+0000 7f7975000700 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563eaa6a1100 0x563eaa6a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.304+0000 7f7956a00700 10 client.15222 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:33:17.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.304+0000 7f7975000700 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563eaa6a1100 0x563eaa6a69f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4735368b504c34d1 server_cookie=85eb7c193694ff23 in_seq=0 out_seq=0 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.304+0000 7f7956a00700 1 -- 172.21.15.131:0/2493597854 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563eaa65b730 con 0x563eaa6a1100 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.304+0000 7f7956a00700 10 client.15222 handle_command_reply: tid=0 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.304+0000 7f798fa04b80 1 client.15222 shutdown 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 1 -- 172.21.15.131:0/2493597854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68d520 msgr2=0x563eaa696b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68d520 0x563eaa696b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f795c005910 tx=0x7f795c00b750 comp rx=0 tx=0).stop 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 1 -- 172.21.15.131:0/2493597854 shutdown_connections 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563eaa6a1100 0x563eaa6a69f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68de70 0x563eaa69daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 0x563eaa696650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f7956a00700 0 client.15222 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 1 --2- 172.21.15.131:0/2493597854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68d520 0x563eaa696b90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 1 -- 172.21.15.131:0/2493597854 >> 172.21.15.131:0/2493597854 conn(0x563eaa5eea10 msgr2=0x563eaa686f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 1 -- 172.21.15.131:0/2493597854 shutdown_connections 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 1 -- 172.21.15.131:0/2493597854 wait complete. 2024-05-10T05:33:17.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.305+0000 7f798fa04b80 20 client.15222 trim_cache size 0 max 16384 2024-05-10T05:33:17.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.323+0000 7f798fa04b80 1 Processor -- start 2024-05-10T05:33:17.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.323+0000 7f798fa04b80 1 -- start start 2024-05-10T05:33:17.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa6953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68d520 0x563eaa69baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa69bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f798fa04b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa6a0e10 con 0x563eaa68de70 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f798fa04b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa56a2b0 con 0x563eaa68d520 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f798fa04b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa5a9010 con 0x563eaa68c650 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa6953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa69bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa6953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45362/0 (socket says 172.21.15.131:45362) 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975000700 1 -- 172.21.15.131:0/350304032 learned_addr learned my addr 172.21.15.131:0/350304032 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975a00700 1 --2- 172.21.15.131:0/350304032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa69bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52392/0 (socket says 172.21.15.131:52392) 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f796fe00700 1 --2- 172.21.15.131:0/350304032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68d520 0x563eaa69baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975000700 1 -- 172.21.15.131:0/350304032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68d520 msgr2=0x563eaa69baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975000700 1 --2- 172.21.15.131:0/350304032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68d520 0x563eaa69baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975000700 1 -- 172.21.15.131:0/350304032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 msgr2=0x563eaa69bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975000700 1 --2- 172.21.15.131:0/350304032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa69bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975000700 1 -- 172.21.15.131:0/350304032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563eaa572510 con 0x563eaa68c650 2024-05-10T05:33:17.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.324+0000 7f7975a00700 1 --2- 172.21.15.131:0/350304032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa69bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:17.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.325+0000 7f7975000700 1 --2- 172.21.15.131:0/350304032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa6953c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7964009cc0 tx=0x7f7964011860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8bcb60897b6565a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.325+0000 7f7962a00700 1 -- 172.21.15.131:0/350304032 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796402b070 con 0x563eaa68c650 2024-05-10T05:33:17.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.325+0000 7f7962a00700 1 -- 172.21.15.131:0/350304032 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79640056a0 con 0x563eaa68c650 2024-05-10T05:33:17.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.325+0000 7f7962a00700 1 -- 172.21.15.131:0/350304032 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964003c40 con 0x563eaa68c650 2024-05-10T05:33:17.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.326+0000 7f798fa04b80 1 -- 172.21.15.131:0/350304032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 msgr2=0x563eaa6953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.326+0000 7f798fa04b80 1 --2- 172.21.15.131:0/350304032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa6953c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7964009cc0 tx=0x7f7964011860 comp rx=0 tx=0).stop 2024-05-10T05:33:17.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.326+0000 7f798fa04b80 1 -- 172.21.15.131:0/350304032 shutdown_connections 2024-05-10T05:33:17.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.326+0000 7f798fa04b80 1 --2- 172.21.15.131:0/350304032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa69bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.326+0000 7f798fa04b80 1 --2- 172.21.15.131:0/350304032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68c650 0x563eaa6953c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.326+0000 7f798fa04b80 1 --2- 172.21.15.131:0/350304032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68d520 0x563eaa69baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.326+0000 7f798fa04b80 1 -- 172.21.15.131:0/350304032 >> 172.21.15.131:0/350304032 conn(0x563eaa5eea10 msgr2=0x563eaa694cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.326+0000 7f798fa04b80 1 -- 172.21.15.131:0/350304032 shutdown_connections 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.326+0000 7f798fa04b80 1 -- 172.21.15.131:0/350304032 wait complete. 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.326+0000 7f798fa04b80 1 -- start start 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f798fa04b80 1 Processor -- start 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 0x563eaa69b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68d520 0x563eaa69bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f798fa04b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa699ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f798fa04b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa69a460 con 0x563eaa68de70 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f798fa04b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa69a5a0 con 0x563eaa68c650 2024-05-10T05:33:17.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f798fa04b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eaa69a760 con 0x563eaa68d520 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f798fa04b80 10 client.0 _ll_register_callbacks cb 0x563eaa551ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f7975a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa699ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f7975a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa699ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52398/0 (socket says 172.21.15.131:52398) 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.327+0000 7f7975a00700 1 -- 172.21.15.131:0/4132279954 learned_addr learned my addr 172.21.15.131:0/4132279954 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7975000700 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 0x563eaa69b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f796fe00700 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68d520 0x563eaa69bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7975a00700 1 -- 172.21.15.131:0/4132279954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68d520 msgr2=0x563eaa69bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7975a00700 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68d520 0x563eaa69bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7975a00700 1 -- 172.21.15.131:0/4132279954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 msgr2=0x563eaa69b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7975a00700 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 0x563eaa69b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7975a00700 1 -- 172.21.15.131:0/4132279954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79701a26a0 con 0x563eaa68de70 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7975000700 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 0x563eaa69b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7975a00700 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa699ee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7968004d70 tx=0x7f796801b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd4a5edbc61d388 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7956a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7956a00700 1 -- 172.21.15.131:0/4132279954 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968099fb0 con 0x563eaa68de70 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f7956a00700 1 -- 172.21.15.131:0/4132279954 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f796801b570 con 0x563eaa68de70 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.328+0000 7f798fa04b80 1 -- 172.21.15.131:0/4132279954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7964004cf0 con 0x563eaa68de70 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f7956a00700 1 -- 172.21.15.131:0/4132279954 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79680998b0 con 0x563eaa68de70 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f798fa04b80 10 client.15228 fetch_fsmap learned FSMap version 26 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f798fa04b80 1 -- 172.21.15.131:0/4132279954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563eaa42d360 con 0x563eaa68de70 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f7956a00700 1 -- 172.21.15.131:0/4132279954 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7964004cf0 con 0x563eaa68de70 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f7956a00700 1 -- 172.21.15.131:0/4132279954 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7968099c30 con 0x563eaa68de70 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f798fa04b80 10 client.15228 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f798fa04b80 10 client.15228 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f798fa04b80 20 client.15228 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f798fa04b80 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563eaa6faff0 0x563eaa6f9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f798fa04b80 4 client.15228 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f798fa04b80 1 -- 172.21.15.131:0/4132279954 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563eaa5a4e70 con 0x563eaa6faff0 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.329+0000 7f7975000700 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563eaa6faff0 0x563eaa6f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f7975000700 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563eaa6faff0 0x563eaa6f9550 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0bac14ba6e94b63 server_cookie=98fa2c143ce1a3f in_seq=0 out_seq=0 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f7956a00700 10 client.15228 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f7956a00700 1 -- 172.21.15.131:0/4132279954 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563eaa5a4e70 con 0x563eaa6faff0 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f7956a00700 10 client.15228 handle_command_reply: tid=0 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f798fa04b80 1 client.15228 shutdown 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f798fa04b80 1 -- 172.21.15.131:0/4132279954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 msgr2=0x563eaa699ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f798fa04b80 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa699ee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7968004d70 tx=0x7f796801b320 comp rx=0 tx=0).stop 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f798fa04b80 1 -- 172.21.15.131:0/4132279954 shutdown_connections 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f798fa04b80 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563eaa6faff0 0x563eaa6f9550 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f798fa04b80 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563eaa68de70 0x563eaa699ee0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f798fa04b80 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563eaa68c650 0x563eaa69b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f798fa04b80 1 --2- 172.21.15.131:0/4132279954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563eaa68d520 0x563eaa69bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f7956a00700 0 client.15228 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.330+0000 7f798fa04b80 1 -- 172.21.15.131:0/4132279954 >> 172.21.15.131:0/4132279954 conn(0x563eaa5eea10 msgr2=0x563eaa6aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f798fa04b80 1 -- 172.21.15.131:0/4132279954 shutdown_connections 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f798fa04b80 1 -- 172.21.15.131:0/4132279954 wait complete. 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f798fa04b80 20 client.15228 trim_cache size 0 max 16384 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24571, 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-10T05:33:17.336 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-10T05:33:17.337 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-10T05:33:17.337 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 82.324504462999997, 2024-05-10T05:33:17.337 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-05-10T05:33:17.337 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-10T05:33:17.337 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T05:33:17.337 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 96.262325802000007 2024-05-10T05:33:17.337 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f7977a00700 1 -- 172.21.15.131:0/3611137035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f79580a1240 msgr2=0x7f79580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f7977a00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f79580a1240 0x7f79580a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f795c006010 tx=0x7f795c004580 comp rx=0 tx=0).stop 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f7977a00700 1 -- 172.21.15.131:0/3611137035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7970102580 msgr2=0x7f79701a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f7977a00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7970102580 0x7f79701a1620 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f796800efc0 tx=0x7f796800c3a0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f7977a00700 1 -- 172.21.15.131:0/3611137035 shutdown_connections 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f7977a00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f79580a1240 0x7f79580a3750 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f7977a00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7970102580 0x7f79701a1620 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f7977a00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100ac0 0x7f797019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f7977a00700 1 --2- 172.21.15.131:0/3611137035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7970101be0 0x7f797019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.331+0000 7f7977a00700 1 -- 172.21.15.131:0/3611137035 >> 172.21.15.131:0/3611137035 conn(0x7f79700fc0d0 msgr2=0x7f79701098c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.332+0000 7f7977a00700 1 -- 172.21.15.131:0/3611137035 shutdown_connections 2024-05-10T05:33:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.332+0000 7f7977a00700 1 -- 172.21.15.131:0/3611137035 wait complete. 2024-05-10T05:33:17.355 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:17 smithi131 ceph-mon[37264]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-10T05:33:17.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-10T05:33:17.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:17 smithi157 ceph-mon[78427]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-10T05:33:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.620+0000 7fbc99600700 1 -- 172.21.15.131:0/899907878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc94102d00 msgr2=0x7fbc94103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.620+0000 7fbc99600700 1 --2- 172.21.15.131:0/899907878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc94102d00 0x7fbc94103110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009990 tx=0x7fbc80009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.620+0000 7fbc99600700 1 -- 172.21.15.131:0/899907878 shutdown_connections 2024-05-10T05:33:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.620+0000 7fbc99600700 1 --2- 172.21.15.131:0/899907878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc941036a0 0x7fbc94109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.620+0000 7fbc99600700 1 --2- 172.21.15.131:0/899907878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc94101be0 0x7fbc94101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.620+0000 7fbc99600700 1 --2- 172.21.15.131:0/899907878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc94102d00 0x7fbc94103110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.620+0000 7fbc99600700 1 -- 172.21.15.131:0/899907878 >> 172.21.15.131:0/899907878 conn(0x7fbc940fd1d0 msgr2=0x7fbc940ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.620+0000 7fbc99600700 1 -- 172.21.15.131:0/899907878 shutdown_connections 2024-05-10T05:33:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.620+0000 7fbc99600700 1 -- 172.21.15.131:0/899907878 wait complete. 2024-05-10T05:33:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.621+0000 7fbc99600700 1 Processor -- start 2024-05-10T05:33:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc99600700 1 -- start start 2024-05-10T05:33:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc99600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc94101be0 0x7fbc9419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc99600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc94102d00 0x7fbc9419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc99600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc941036a0 0x7fbc941a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc99600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc941a1c20 con 0x7fbc94102d00 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc99600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc941a1d90 con 0x7fbc941036a0 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc99600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc941a1f50 con 0x7fbc94101be0 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc92000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc94102d00 0x7fbc9419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc92a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc94101be0 0x7fbc9419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc92000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc94102d00 0x7fbc9419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52418/0 (socket says 172.21.15.131:52418) 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.622+0000 7fbc92000700 1 -- 172.21.15.131:0/3654712206 learned_addr learned my addr 172.21.15.131:0/3654712206 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc93400700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc941036a0 0x7fbc941a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc92000700 1 -- 172.21.15.131:0/3654712206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc94101be0 msgr2=0x7fbc9419a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc92000700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc94101be0 0x7fbc9419a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc92000700 1 -- 172.21.15.131:0/3654712206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc941036a0 msgr2=0x7fbc941a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc92000700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc941036a0 0x7fbc941a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc92000700 1 -- 172.21.15.131:0/3654712206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc800095f0 con 0x7fbc94102d00 2024-05-10T05:33:17.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc92000700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc94102d00 0x7fbc9419a680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbc80000c00 tx=0x7fbc80005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc8b400700 1 -- 172.21.15.131:0/3654712206 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8001d070 con 0x7fbc94102d00 2024-05-10T05:33:17.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc8b400700 1 -- 172.21.15.131:0/3654712206 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc80021410 con 0x7fbc94102d00 2024-05-10T05:33:17.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc99600700 1 -- 172.21.15.131:0/3654712206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc941a21d0 con 0x7fbc94102d00 2024-05-10T05:33:17.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc8b400700 1 -- 172.21.15.131:0/3654712206 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80017650 con 0x7fbc94102d00 2024-05-10T05:33:17.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.623+0000 7fbc99600700 1 -- 172.21.15.131:0/3654712206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc941a25f0 con 0x7fbc94102d00 2024-05-10T05:33:17.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.626+0000 7fbc8b400700 1 -- 172.21.15.131:0/3654712206 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbc80021580 con 0x7fbc94102d00 2024-05-10T05:33:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.628+0000 7fbcb1449b80 1 Processor -- start 2024-05-10T05:33:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.628+0000 7fbcb1449b80 1 -- start start 2024-05-10T05:33:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.628+0000 7fbc8b400700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc780a11c0 0x7fbc780a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.628+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b17aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.628+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18520 0x559909b18930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.628+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18e70 0x559909b1f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.628+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559909a3e100 con 0x559909b18520 2024-05-10T05:33:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.628+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559909a3bac0 con 0x559909b18e70 2024-05-10T05:33:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.628+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559909a373b0 con 0x559909b17650 2024-05-10T05:33:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.628+0000 7fbc8b400700 1 -- 172.21.15.131:0/3654712206 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbc800cdc80 con 0x7fbc94102d00 2024-05-10T05:33:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc92a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b17aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc92a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b17aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45414/0 (socket says 172.21.15.131:45414) 2024-05-10T05:33:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc92a00700 1 -- 172.21.15.131:0/2589408216 learned_addr learned my addr 172.21.15.131:0/2589408216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc92000700 1 --2- 172.21.15.131:0/2589408216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18520 0x559909b18930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc93400700 1 --2- 172.21.15.131:0/2589408216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18e70 0x559909b1f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc93400700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc780a11c0 0x7fbc780a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc92a00700 1 -- 172.21.15.131:0/2589408216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18e70 msgr2=0x559909b1f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc92a00700 1 --2- 172.21.15.131:0/2589408216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18e70 0x559909b1f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc92a00700 1 -- 172.21.15.131:0/2589408216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18520 msgr2=0x559909b18930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc92a00700 1 --2- 172.21.15.131:0/2589408216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18520 0x559909b18930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc92a00700 1 -- 172.21.15.131:0/2589408216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599098b8360 con 0x559909b17650 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc92a00700 1 --2- 172.21.15.131:0/2589408216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b17aa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbc840115c0 tx=0x7fbc84009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d2bd0e8ba93b90e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.629+0000 7fbc93400700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc780a11c0 0x7fbc780a36d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0097a0 tx=0x7fbc8c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.630+0000 7fbc7f400700 1 -- 172.21.15.131:0/2589408216 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400e040 con 0x559909b17650 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.630+0000 7fbc7f400700 1 -- 172.21.15.131:0/2589408216 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc8400f040 con 0x559909b17650 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.630+0000 7fbc7f400700 1 -- 172.21.15.131:0/2589408216 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84007500 con 0x559909b17650 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.630+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2589408216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 msgr2=0x559909b17aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.630+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/2589408216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b17aa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbc840115c0 tx=0x7fbc84009920 comp rx=0 tx=0).stop 2024-05-10T05:33:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.631+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2589408216 shutdown_connections 2024-05-10T05:33:17.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.631+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/2589408216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18e70 0x559909b1f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.631+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/2589408216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b17aa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.631+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/2589408216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18520 0x559909b18930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.631+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2589408216 >> 172.21.15.131:0/2589408216 conn(0x559909a79a10 msgr2=0x559909b11f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.631+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2589408216 shutdown_connections 2024-05-10T05:33:17.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.631+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2589408216 wait complete. 2024-05-10T05:33:17.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.631+0000 7fbcb1449b80 1 -- start start 2024-05-10T05:33:17.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.632+0000 7fbcb1449b80 1 Processor -- start 2024-05-10T05:33:17.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.632+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b21650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.632+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b21b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.632+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b28af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.632+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599099d3510 con 0x559909b18e70 2024-05-10T05:33:17.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.632+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599099d31b0 con 0x559909b18520 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.632+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599098ba650 con 0x559909b17650 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbcb1449b80 10 client.0 _ll_register_callbacks cb 0x5599099dcae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b21650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b21b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc93400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b28af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b21650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45430/0 (socket says 172.21.15.131:45430) 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b21b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:40174/0 (socket says 172.21.15.131:40174) 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92a00700 1 -- 172.21.15.131:0/370208685 learned_addr learned my addr 172.21.15.131:0/370208685 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92000700 1 -- 172.21.15.131:0/370208685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 msgr2=0x559909b21650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92000700 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b21650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92000700 1 -- 172.21.15.131:0/370208685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 msgr2=0x559909b28af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92000700 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b28af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92000700 1 -- 172.21.15.131:0/370208685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc84011270 con 0x559909b18520 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc93400700 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b28af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc92000700 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b21b90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc800cb720 tx=0x7fbc8009ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad26904cc90fc25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc73e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.633+0000 7fbc73e00700 1 -- 172.21.15.131:0/370208685 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800cbde0 con 0x559909b18520 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbc73e00700 1 -- 172.21.15.131:0/370208685 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbc8009ef30 con 0x559909b18520 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbcb1449b80 1 -- 172.21.15.131:0/370208685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559909990220 con 0x559909b18520 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbc73e00700 1 -- 172.21.15.131:0/370208685 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc800a34f0 con 0x559909b18520 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbcb1449b80 10 client.25250 fetch_fsmap learned FSMap version 26 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbcb1449b80 1 -- 172.21.15.131:0/370208685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559909864b10 con 0x559909b18520 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbc73e00700 1 -- 172.21.15.131:0/370208685 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc80064080 con 0x559909b18520 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbc73e00700 1 -- 172.21.15.131:0/370208685 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbc8009a710 con 0x559909b18520 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbcb1449b80 10 client.25250 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbcb1449b80 10 client.25250 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbcb1449b80 20 client.25250 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559909b2c2f0 0x559909b315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbcb1449b80 4 client.25250 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.634+0000 7fbcb1449b80 1 -- 172.21.15.131:0/370208685 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559909ae6730 con 0x559909b2c2f0 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.635+0000 7fbc92a00700 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559909b2c2f0 0x559909b315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.635+0000 7fbc92a00700 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559909b2c2f0 0x559909b315f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c3a1ed506ac729 server_cookie=59e056a5c73160c6 in_seq=0 out_seq=0 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.635+0000 7fbc73e00700 10 client.25250 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbc73e00700 1 -- 172.21.15.131:0/370208685 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559909ae6730 con 0x559909b2c2f0 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbc73e00700 10 client.25250 handle_command_reply: tid=0 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 client.25250 shutdown 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 -- 172.21.15.131:0/370208685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 msgr2=0x559909b21b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b21b90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc800cb720 tx=0x7fbc8009ab90 comp rx=0 tx=0).stop 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 -- 172.21.15.131:0/370208685 shutdown_connections 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559909b2c2f0 0x559909b315f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b28af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbc73e00700 0 client.25250 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b21650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/370208685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b21b90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 -- 172.21.15.131:0/370208685 >> 172.21.15.131:0/370208685 conn(0x559909a79a10 msgr2=0x559909b11f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 -- 172.21.15.131:0/370208685 shutdown_connections 2024-05-10T05:33:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.636+0000 7fbcb1449b80 1 -- 172.21.15.131:0/370208685 wait complete. 2024-05-10T05:33:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.637+0000 7fbcb1449b80 20 client.25250 trim_cache size 0 max 16384 2024-05-10T05:33:17.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:17 smithi142 ceph-mon[77667]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-10T05:33:17.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbcb1449b80 1 Processor -- start 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbcb1449b80 1 -- start start 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b17650 0x559909b21930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b18520 0x559909b26aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18e70 0x559909b26fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559909a30150 con 0x559909b17650 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559909a34010 con 0x559909b18e70 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559909b2ba40 con 0x559909b18520 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbc92a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b17650 0x559909b21930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbc92a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b17650 0x559909b21930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52448/0 (socket says 172.21.15.131:52448) 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.655+0000 7fbc92a00700 1 -- 172.21.15.131:0/804758193 learned_addr learned my addr 172.21.15.131:0/804758193 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc93400700 1 --2- 172.21.15.131:0/804758193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18e70 0x559909b26fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc92000700 1 --2- 172.21.15.131:0/804758193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b18520 0x559909b26aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc92a00700 1 -- 172.21.15.131:0/804758193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b18520 msgr2=0x559909b26aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc92a00700 1 --2- 172.21.15.131:0/804758193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b18520 0x559909b26aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc92a00700 1 -- 172.21.15.131:0/804758193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18e70 msgr2=0x559909b26fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc92a00700 1 --2- 172.21.15.131:0/804758193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18e70 0x559909b26fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc92a00700 1 -- 172.21.15.131:0/804758193 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599099fd510 con 0x559909b17650 2024-05-10T05:33:17.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc92a00700 1 --2- 172.21.15.131:0/804758193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b17650 0x559909b21930 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbc84011590 tx=0x7fbc840099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4b83f71c5d3257 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc7fe00700 1 -- 172.21.15.131:0/804758193 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84023040 con 0x559909b17650 2024-05-10T05:33:17.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc7fe00700 1 -- 172.21.15.131:0/804758193 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc840275f0 con 0x559909b17650 2024-05-10T05:33:17.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.656+0000 7fbc7fe00700 1 -- 172.21.15.131:0/804758193 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84027a90 con 0x559909b17650 2024-05-10T05:33:17.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.657+0000 7fbcb1449b80 1 -- 172.21.15.131:0/804758193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b17650 msgr2=0x559909b21930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.657+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/804758193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b17650 0x559909b21930 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbc84011590 tx=0x7fbc840099b0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.657+0000 7fbcb1449b80 1 -- 172.21.15.131:0/804758193 shutdown_connections 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.657+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/804758193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b17650 0x559909b21930 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.657+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/804758193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18e70 0x559909b26fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.657+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/804758193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b18520 0x559909b26aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.657+0000 7fbcb1449b80 1 -- 172.21.15.131:0/804758193 >> 172.21.15.131:0/804758193 conn(0x559909a79a10 msgr2=0x559909b21220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.657+0000 7fbcb1449b80 1 -- 172.21.15.131:0/804758193 shutdown_connections 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.657+0000 7fbcb1449b80 1 -- 172.21.15.131:0/804758193 wait complete. 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.658+0000 7fbcb1449b80 1 -- start start 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.658+0000 7fbcb1449b80 1 Processor -- start 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.658+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b26890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.658+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b26dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.658+0000 7fbcb1449b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b25040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.658+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559909b25610 con 0x559909b18e70 2024-05-10T05:33:17.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.658+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559909b25780 con 0x559909b18520 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.658+0000 7fbcb1449b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559909b25940 con 0x559909b17650 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.658+0000 7fbcb1449b80 10 client.0 _ll_register_callbacks cb 0x5599099dcae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc93400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b25040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc93400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b25040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52464/0 (socket says 172.21.15.131:52464) 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc93400700 1 -- 172.21.15.131:0/2479923819 learned_addr learned my addr 172.21.15.131:0/2479923819 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc92000700 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b26dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc92a00700 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b26890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc93400700 1 -- 172.21.15.131:0/2479923819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 msgr2=0x559909b26890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc93400700 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b26890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc93400700 1 -- 172.21.15.131:0/2479923819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 msgr2=0x559909b26dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc93400700 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b26dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc93400700 1 -- 172.21.15.131:0/2479923819 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599098b8360 con 0x559909b18e70 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc92000700 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b26dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:17.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc93400700 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b25040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0046e0 tx=0x7fbc8c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3701f057e669b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.659+0000 7fbc73e00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbc73e00700 1 -- 172.21.15.131:0/2479923819 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c027070 con 0x559909b18e70 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbc73e00700 1 -- 172.21.15.131:0/2479923819 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbc8c00fe80 con 0x559909b18e70 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2479923819 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc8400f040 con 0x559909b18e70 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbc73e00700 1 -- 172.21.15.131:0/2479923819 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc8c003890 con 0x559909b18e70 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbcb1449b80 10 client.15246 fetch_fsmap learned FSMap version 26 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2479923819 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5599098bcc80 con 0x559909b18e70 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbc73e00700 1 -- 172.21.15.131:0/2479923819 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc8400f040 con 0x559909b18e70 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbc73e00700 1 -- 172.21.15.131:0/2479923819 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbc8c005020 con 0x559909b18e70 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbcb1449b80 10 client.15246 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbcb1449b80 10 client.15246 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbcb1449b80 20 client.15246 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559909b85dc0 0x559909b84320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbcb1449b80 4 client.15246 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.660+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2479923819 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559909a2fe70 con 0x559909b85dc0 2024-05-10T05:33:17.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.661+0000 7fbc92a00700 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559909b85dc0 0x559909b84320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.661+0000 7fbc73e00700 10 client.15246 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.661+0000 7fbc92a00700 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559909b85dc0 0x559909b84320 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67a815c7f9bd5662 server_cookie=df638d79042f8741 in_seq=0 out_seq=0 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.661+0000 7fbc73e00700 1 -- 172.21.15.131:0/2479923819 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x559909a2fe70 con 0x559909b85dc0 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.661+0000 7fbc73e00700 10 client.15246 handle_command_reply: tid=0 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.661+0000 7fbcb1449b80 1 client.15246 shutdown 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.661+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2479923819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 msgr2=0x559909b25040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.661+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b25040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0046e0 tx=0x7fbc8c0047c0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2479923819 shutdown_connections 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559909b85dc0 0x559909b84320 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559909b18e70 0x559909b25040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559909b17650 0x559909b26890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbcb1449b80 1 --2- 172.21.15.131:0/2479923819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559909b18520 0x559909b26dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbc73e00700 0 client.15246 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:17.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2479923819 >> 172.21.15.131:0/2479923819 conn(0x559909a79a10 msgr2=0x559909b35610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2479923819 shutdown_connections 2024-05-10T05:33:17.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbcb1449b80 1 -- 172.21.15.131:0/2479923819 wait complete. 2024-05-10T05:33:17.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbcb1449b80 20 client.15246 trim_cache size 0 max 16384 2024-05-10T05:33:17.667 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:33:17.667 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "6e8de474-0e8d-11ef-bc98-c7b262605968", 2024-05-10T05:33:17.667 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-10T05:33:17.667 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 24581, 2024-05-10T05:33:17.667 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-10T05:33:17.667 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-10T05:33:17.668 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-10T05:33:17.668 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 86.111158919999994, 2024-05-10T05:33:17.668 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 26, 2024-05-10T05:33:17.668 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-10T05:33:17.668 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T05:33:17.668 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 92.332500558999996 2024-05-10T05:33:17.668 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbc99600700 1 -- 172.21.15.131:0/3654712206 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc780a11c0 msgr2=0x7fbc780a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbc99600700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc780a11c0 0x7fbc780a36d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0097a0 tx=0x7fbc8c009260 comp rx=0 tx=0).stop 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbc99600700 1 -- 172.21.15.131:0/3654712206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc94102d00 msgr2=0x7fbc9419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.662+0000 7fbc99600700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc94102d00 0x7fbc9419a680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbc80000c00 tx=0x7fbc80005720 comp rx=0 tx=0).stop 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.663+0000 7fbc99600700 1 -- 172.21.15.131:0/3654712206 shutdown_connections 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.663+0000 7fbc99600700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbc780a11c0 0x7fbc780a36d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.663+0000 7fbc99600700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc941036a0 0x7fbc941a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.663+0000 7fbc99600700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc94101be0 0x7fbc9419a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.663+0000 7fbc99600700 1 --2- 172.21.15.131:0/3654712206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc94102d00 0x7fbc9419a680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.663+0000 7fbc99600700 1 -- 172.21.15.131:0/3654712206 >> 172.21.15.131:0/3654712206 conn(0x7fbc940fd1d0 msgr2=0x7fbc94073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.663+0000 7fbc99600700 1 -- 172.21.15.131:0/3654712206 shutdown_connections 2024-05-10T05:33:17.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:17.663+0000 7fbc99600700 1 -- 172.21.15.131:0/3654712206 wait complete. 2024-05-10T05:33:17.689 INFO:teuthology.run_tasks:Running task sequential... 2024-05-10T05:33:17.699 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-10T05:33:17.699 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-10T05:33:17.699 INFO:teuthology.task.print:Enabling mgr modules 2024-05-10T05:33:17.699 INFO:teuthology.run_tasks:Running task kclient... 2024-05-10T05:33:17.758 INFO:tasks.kclient:Mounting kernel clients... 2024-05-10T05:33:17.758 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-10T05:33:17.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:17.759 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-10T05:33:17.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:17.802 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-05-10T05:33:17.911 INFO:teuthology.orchestra.run.smithi131.stderr:Cannot find device "ceph-brx" 2024-05-10T05:33:17.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:33:17.915 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-10T05:33:17.915 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T05:33:17.915 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T05:33:17.916 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-10T05:33:18.277 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:33:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.585+0000 7fbec3200700 1 -- 172.21.15.131:0/1882826346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc102450 msgr2=0x7fbebc10e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.585+0000 7fbec3200700 1 --2- 172.21.15.131:0/1882826346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc102450 0x7fbebc10e930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800b0a0 tx=0x7fbeb800b3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.585+0000 7fbec3200700 1 -- 172.21.15.131:0/1882826346 shutdown_connections 2024-05-10T05:33:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.585+0000 7fbec3200700 1 --2- 172.21.15.131:0/1882826346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc107b20 0x7fbebc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:19.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.585+0000 7fbec3200700 1 --2- 172.21.15.131:0/1882826346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc102450 0x7fbebc10e930 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:19.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.585+0000 7fbec3200700 1 --2- 172.21.15.131:0/1882826346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc101b00 0x7fbebc101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:19.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.585+0000 7fbec3200700 1 -- 172.21.15.131:0/1882826346 >> 172.21.15.131:0/1882826346 conn(0x7fbebc077650 msgr2=0x7fbebc077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:19.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.586+0000 7fbec3200700 1 -- 172.21.15.131:0/1882826346 shutdown_connections 2024-05-10T05:33:19.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.586+0000 7fbec3200700 1 -- 172.21.15.131:0/1882826346 wait complete. 2024-05-10T05:33:19.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.587+0000 7fbec3200700 1 Processor -- start 2024-05-10T05:33:19.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.587+0000 7fbec3200700 1 -- start start 2024-05-10T05:33:19.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec3200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc101b00 0x7fbebc19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:19.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec3200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc102450 0x7fbebc19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:19.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec3200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc107b20 0x7fbebc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec3200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc19f530 con 0x7fbebc101b00 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec3200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc19f6a0 con 0x7fbebc102450 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec3200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc19f860 con 0x7fbebc107b20 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc101b00 0x7fbebc19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc101b00 0x7fbebc19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52474/0 (socket says 172.21.15.131:52474) 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec1e00700 1 -- 172.21.15.131:0/1014787557 learned_addr learned my addr 172.21.15.131:0/1014787557 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec1400700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc102450 0x7fbebc19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec2800700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc107b20 0x7fbebc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec1e00700 1 -- 172.21.15.131:0/1014787557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc107b20 msgr2=0x7fbebc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec1e00700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc107b20 0x7fbebc19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec1e00700 1 -- 172.21.15.131:0/1014787557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc102450 msgr2=0x7fbebc19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec1e00700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc102450 0x7fbebc19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.588+0000 7fbec1e00700 1 -- 172.21.15.131:0/1014787557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb0009520 con 0x7fbebc101b00 2024-05-10T05:33:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.589+0000 7fbec1e00700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc101b00 0x7fbebc19a720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000ea30 tx=0x7fbeb000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.589+0000 7fbeb6a00700 1 -- 172.21.15.131:0/1014787557 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb000cbd0 con 0x7fbebc101b00 2024-05-10T05:33:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.589+0000 7fbeb6a00700 1 -- 172.21.15.131:0/1014787557 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb000eed0 con 0x7fbebc101b00 2024-05-10T05:33:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.589+0000 7fbec3200700 1 -- 172.21.15.131:0/1014787557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb8009d00 con 0x7fbebc101b00 2024-05-10T05:33:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.589+0000 7fbeb6a00700 1 -- 172.21.15.131:0/1014787557 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0003d20 con 0x7fbebc101b00 2024-05-10T05:33:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.589+0000 7fbec3200700 1 -- 172.21.15.131:0/1014787557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc1a7270 con 0x7fbebc101b00 2024-05-10T05:33:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.590+0000 7fbec3200700 1 -- 172.21.15.131:0/1014787557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbebc103200 con 0x7fbebc101b00 2024-05-10T05:33:19.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.593+0000 7fbeb6a00700 1 -- 172.21.15.131:0/1014787557 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbeb0003ea0 con 0x7fbebc101b00 2024-05-10T05:33:19.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.594+0000 7fbeb6a00700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbea40a1120 0x7fbea40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:19.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.594+0000 7fbeb6a00700 1 -- 172.21.15.131:0/1014787557 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbeb0014070 con 0x7fbebc101b00 2024-05-10T05:33:19.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.594+0000 7fbec1400700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbea40a1120 0x7fbea40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:19.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.595+0000 7fbec1400700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbea40a1120 0x7fbea40a3630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbebc10d980 tx=0x7fbeac009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:19.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.597+0000 7fbeb6a00700 1 -- 172.21.15.131:0/1014787557 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbeb0096450 con 0x7fbebc101b00 2024-05-10T05:33:19.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.847+0000 7fbec3200700 1 -- 172.21.15.131:0/1014787557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbebc068a70 con 0x7fbebc101b00 2024-05-10T05:33:19.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.848+0000 7fbeb6a00700 1 -- 172.21.15.131:0/1014787557 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbeb009ee60 con 0x7fbebc101b00 2024-05-10T05:33:19.848 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T05:33:19.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 -- 172.21.15.131:0/1014787557 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbea40a1120 msgr2=0x7fbea40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:19.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbea40a1120 0x7fbea40a3630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbebc10d980 tx=0x7fbeac009260 comp rx=0 tx=0).stop 2024-05-10T05:33:19.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 -- 172.21.15.131:0/1014787557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc101b00 msgr2=0x7fbebc19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:19.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc101b00 0x7fbebc19a720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000ea30 tx=0x7fbeb000ed40 comp rx=0 tx=0).stop 2024-05-10T05:33:19.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 -- 172.21.15.131:0/1014787557 shutdown_connections 2024-05-10T05:33:19.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbea40a1120 0x7fbea40a3630 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:19.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc101b00 0x7fbebc19a720 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:19.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbebc107b20 0x7fbebc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:19.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 --2- 172.21.15.131:0/1014787557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbebc102450 0x7fbebc19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:19.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 -- 172.21.15.131:0/1014787557 >> 172.21.15.131:0/1014787557 conn(0x7fbebc077650 msgr2=0x7fbebc10e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 -- 172.21.15.131:0/1014787557 shutdown_connections 2024-05-10T05:33:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:19.850+0000 7fbec3200700 1 -- 172.21.15.131:0/1014787557 wait complete. 2024-05-10T05:33:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:19 smithi131 ceph-mon[37264]: pgmap v284: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-10T05:33:20.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:19 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1014787557' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T05:33:20.283 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T05:33:20.283 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T05:33:20.283 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-10T05:33:20.284 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T05:33:20.284 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T05:33:20.284 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T05:33:20.284 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T05:33:20.284 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T05:33:20.284 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T05:33:20.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:20.284 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-10T05:33:20.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:19 smithi157 ceph-mon[78427]: pgmap v284: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-10T05:33:20.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:19 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1014787557' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T05:33:20.328 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T05:33:20.328 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T05:33:20.328 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T05:33:20.328 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-10T05:33:20.328 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-10T05:33:20.328 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-10T05:33:20.328 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T05:33:20.328 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T05:33:20.328 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-10T05:33:20.328 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T05:33:20.329 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T05:33:20.329 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-10T05:33:20.329 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T05:33:20.329 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T05:33:20.329 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T05:33:20.329 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T05:33:20.329 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-10T05:33:20.329 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41634sec preferred_lft 41634sec 2024-05-10T05:33:20.329 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-10T05:33:20.329 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-10T05:33:20.330 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T05:33:20.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:20.330 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-10T05:33:20.331 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-10T05:33:20.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2024-05-10T05:33:20.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2024-05-10T05:33:20.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2024-05-10T05:33:20.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T05:33:20.331 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-10T05:33:20.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:19 smithi142 ceph-mon[77667]: pgmap v284: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-10T05:33:20.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:19 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1014787557' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T05:33:20.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:20.820 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T05:33:20.889 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-10T05:33:20.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:20.893 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-10T05:33:20.957 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-10T05:33:20.958 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T05:33:20.958 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T05:33:20.958 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T05:33:20.958 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T05:33:20.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:20.959 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-10T05:33:20.959 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-10T05:33:20.959 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T05:33:20.959 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T05:33:20.959 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T05:33:20.960 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-10T05:33:21.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:21.629 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-10T05:33:21.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:21.673 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-05-10T05:33:21.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:21.755 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-10T05:33:21.755 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-10T05:33:21.755 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T05:33:21.756 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-10T05:33:21.756 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-10T05:33:21.933 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-10T05:33:21.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:21.934 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-10T05:33:21.934 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-10T05:33:21.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-10T05:33:21.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T05:33:21.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-10T05:33:21.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-10T05:33:21.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-10T05:33:21.934 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-10T05:33:22.161 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:21 smithi131 ceph-mon[37264]: pgmap v285: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-10T05:33:22.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:22.205 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2024-05-10T05:33:22.205 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-10T05:33:22.206 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-10T05:33:22.206 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T05:33:22.206 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-10T05:33:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:21 smithi157 ceph-mon[78427]: pgmap v285: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-10T05:33:22.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T05:33:22.374 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T05:33:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:21 smithi142 ceph-mon[77667]: pgmap v285: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-10T05:33:22.415 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T05:33:22.416 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T05:33:22.416 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T05:33:22.498 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-10T05:33:22.498 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-10T05:33:22.498 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/dce0b8c1-4e28-48fa-a70b-816ec396fce0 2024-05-10T05:33:22.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:22.498 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/dce0b8c1-4e28-48fa-a70b-816ec396fce0 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-10T05:33:22.710 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-10T05:33:22.710 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-05-10T05:33:23.421 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-10T05:33:23.421 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-05-10T05:33:23.421 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:6789,172.21.15.142:6789,172.21.15.157:6789" 2024-05-10T05:33:23.421 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@6e8de474-0e8d-11ef-bc98-c7b262605968.cephfs=/volumes/qa/sv_1/dce0b8c1-4e28-48fa-a70b-816ec396fce0 2024-05-10T05:33:23.421 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.131:6789/172.21.15.142:6789/172.21.15.157:6789" will pass to kernel 2024-05-10T05:33:23.427 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T05:33:23.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:23.427 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T05:33:23.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:23.496 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T05:33:23.587 INFO:teuthology.orchestra.run.smithi131.stdout:client15255 2024-05-10T05:33:23.587 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:33:23.588 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T05:33:23.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:23.630 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T05:33:23.753 INFO:teuthology.orchestra.run.smithi131.stdout:client15255 2024-05-10T05:33:23.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:23.754 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T05:33:23.836 INFO:teuthology.orchestra.run.smithi131.stdout:client15255 2024-05-10T05:33:23.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:33:23.836 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968.client15255/status') 2024-05-10T05:33:23.940 INFO:teuthology.orchestra.run.smithi131.stdout:instance: client.15255 (3)192.168.0.1:0/563565219 2024-05-10T05:33:23.940 INFO:teuthology.orchestra.run.smithi131.stdout:blocklisted: false 2024-05-10T05:33:23.940 INFO:teuthology.orchestra.run.smithi131.stderr:0+1 records in 2024-05-10T05:33:23.940 INFO:teuthology.orchestra.run.smithi131.stderr:0+1 records out 2024-05-10T05:33:23.940 INFO:teuthology.orchestra.run.smithi131.stderr:69 bytes copied, 5.2483e-05 s, 1.3 MB/s 2024-05-10T05:33:23.943 INFO:teuthology.run_tasks:Running task exec... 2024-05-10T05:33:23.956 INFO:teuthology.task.exec:Executing custom commands... 2024-05-10T05:33:23.956 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-05-10T05:33:23.956 DEBUG:teuthology.orchestra.run.smithi131:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-10T05:33:24.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:23 smithi131 ceph-mon[37264]: pgmap v286: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-10T05:33:24.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.218+0000 7ff552200700 1 -- 172.21.15.131:0/3979497188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c101be0 msgr2=0x7ff54c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:24.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.218+0000 7ff552200700 1 --2- 172.21.15.131:0/3979497188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c101be0 0x7ff54c101ff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009990 tx=0x7ff53c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.218+0000 7ff552200700 1 -- 172.21.15.131:0/3979497188 shutdown_connections 2024-05-10T05:33:24.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.218+0000 7ff552200700 1 --2- 172.21.15.131:0/3979497188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c101be0 0x7ff54c101ff0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.218+0000 7ff552200700 1 --2- 172.21.15.131:0/3979497188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c1036a0 0x7ff54c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.218+0000 7ff552200700 1 --2- 172.21.15.131:0/3979497188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c102d00 0x7ff54c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.218+0000 7ff552200700 1 -- 172.21.15.131:0/3979497188 >> 172.21.15.131:0/3979497188 conn(0x7ff54c0fd1f0 msgr2=0x7ff54c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:24.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.218+0000 7ff552200700 1 -- 172.21.15.131:0/3979497188 shutdown_connections 2024-05-10T05:33:24.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.218+0000 7ff552200700 1 -- 172.21.15.131:0/3979497188 wait complete. 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.219+0000 7ff552200700 1 Processor -- start 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.219+0000 7ff552200700 1 -- start start 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.219+0000 7ff552200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c101be0 0x7ff54c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff552200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c102d00 0x7ff54c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff552200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff552200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19fa80 con 0x7ff54c1036a0 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff552200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19fbc0 con 0x7ff54c101be0 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff552200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19fd50 con 0x7ff54c102d00 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c101be0 0x7ff54c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c19f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33220/0 (socket says 172.21.15.131:33220) 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54be00700 1 -- 172.21.15.131:0/1243154415 learned_addr learned my addr 172.21.15.131:0/1243154415 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54aa00700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c102d00 0x7ff54c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54be00700 1 -- 172.21.15.131:0/1243154415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c102d00 msgr2=0x7ff54c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54be00700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c102d00 0x7ff54c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54be00700 1 -- 172.21.15.131:0/1243154415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c101be0 msgr2=0x7ff54c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54be00700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c101be0 0x7ff54c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54be00700 1 -- 172.21.15.131:0/1243154415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544009c40 con 0x7ff54c1036a0 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.220+0000 7ff54b400700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c101be0 0x7ff54c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.221+0000 7ff54be00700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c19f4b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c3a0 tx=0x7ff54400c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.221+0000 7ff543e00700 1 -- 172.21.15.131:0/1243154415 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544017070 con 0x7ff54c1036a0 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.221+0000 7ff543e00700 1 -- 172.21.15.131:0/1243154415 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff54400f040 con 0x7ff54c1036a0 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.221+0000 7ff552200700 1 -- 172.21.15.131:0/1243154415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53c0095f0 con 0x7ff54c1036a0 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.221+0000 7ff543e00700 1 -- 172.21.15.131:0/1243154415 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400e050 con 0x7ff54c1036a0 2024-05-10T05:33:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.221+0000 7ff552200700 1 -- 172.21.15.131:0/1243154415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c1a0210 con 0x7ff54c1036a0 2024-05-10T05:33:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.222+0000 7ff552200700 1 -- 172.21.15.131:0/1243154415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c108df0 con 0x7ff54c1036a0 2024-05-10T05:33:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.224+0000 7ff543e00700 1 -- 172.21.15.131:0/1243154415 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff5440044c0 con 0x7ff54c1036a0 2024-05-10T05:33:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.225+0000 7ff543e00700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 0x7ff5300a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.225+0000 7ff543e00700 1 -- 172.21.15.131:0/1243154415 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff5440cc240 con 0x7ff54c1036a0 2024-05-10T05:33:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.226+0000 7ff54b400700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 0x7ff5300a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.226+0000 7ff54b400700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 0x7ff5300a3630 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0098e0 tx=0x7ff53c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.230+0000 7ff543e00700 1 -- 172.21.15.131:0/1243154415 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff544095970 con 0x7ff54c1036a0 2024-05-10T05:33:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:23 smithi157 ceph-mon[78427]: pgmap v286: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-10T05:33:24.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:23 smithi142 ceph-mon[77667]: pgmap v286: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-10T05:33:24.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.487+0000 7ff552200700 1 -- 172.21.15.131:0/1243154415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7ff54c0683a0 con 0x7ff54c1036a0 2024-05-10T05:33:24.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.924+0000 7ff543e00700 1 -- 172.21.15.131:0/1243154415 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) v1 ==== 117+0+0 (secure 0 0 0) 0x7ff544098f90 con 0x7ff54c1036a0 2024-05-10T05:33:24.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 -- 172.21.15.131:0/1243154415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 msgr2=0x7ff5300a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:24.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 0x7ff5300a3630 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0098e0 tx=0x7ff53c01a040 comp rx=0 tx=0).stop 2024-05-10T05:33:24.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 -- 172.21.15.131:0/1243154415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 msgr2=0x7ff54c19f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:24.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c19f4b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c3a0 tx=0x7ff54400c6b0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 -- 172.21.15.131:0/1243154415 shutdown_connections 2024-05-10T05:33:24.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5300a1120 0x7ff5300a3630 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff54c1036a0 0x7ff54c19f4b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c101be0 0x7ff54c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 --2- 172.21.15.131:0/1243154415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54c102d00 0x7ff54c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:24.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 -- 172.21.15.131:0/1243154415 >> 172.21.15.131:0/1243154415 conn(0x7ff54c0fd1f0 msgr2=0x7ff54c105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:24.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 -- 172.21.15.131:0/1243154415 shutdown_connections 2024-05-10T05:33:24.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:24.927+0000 7ff552200700 1 -- 172.21.15.131:0/1243154415 wait complete. 2024-05-10T05:33:24.954 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-10T05:33:24.965 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-10T05:33:25.028 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:33:25.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:24 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1243154415' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T05:33:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.294+0000 7fd696e00700 1 -- 172.21.15.131:0/767731889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd690069070 msgr2=0x7fd690104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.294+0000 7fd696e00700 1 --2- 172.21.15.131:0/767731889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd690069070 0x7fd690104740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd680009990 tx=0x7fd680009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.294+0000 7fd696e00700 1 -- 172.21.15.131:0/767731889 shutdown_connections 2024-05-10T05:33:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.294+0000 7fd696e00700 1 --2- 172.21.15.131:0/767731889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd690069070 0x7fd690104740 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.294+0000 7fd696e00700 1 --2- 172.21.15.131:0/767731889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690107880 0x7fd690109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.294+0000 7fd696e00700 1 --2- 172.21.15.131:0/767731889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd690104d10 0x7fd6901071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.294+0000 7fd696e00700 1 -- 172.21.15.131:0/767731889 >> 172.21.15.131:0/767731889 conn(0x7fd6900fc0d0 msgr2=0x7fd6900fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:25.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.294+0000 7fd696e00700 1 -- 172.21.15.131:0/767731889 shutdown_connections 2024-05-10T05:33:25.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.294+0000 7fd696e00700 1 -- 172.21.15.131:0/767731889 wait complete. 2024-05-10T05:33:25.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.295+0000 7fd696e00700 1 Processor -- start 2024-05-10T05:33:25.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.295+0000 7fd696e00700 1 -- start start 2024-05-10T05:33:25.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd696e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd690069070 0x7fd69019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:25.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd696e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690104d10 0x7fd69019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:25.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd696e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd690107880 0x7fd6901a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:25.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd696e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6901a1c80 con 0x7fd690069070 2024-05-10T05:33:25.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd696e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6901a1df0 con 0x7fd690107880 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd696e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6901a1fb0 con 0x7fd690104d10 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd68fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd690069070 0x7fd69019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd68fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd690069070 0x7fd69019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33232/0 (socket says 172.21.15.131:33232) 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd68fe00700 1 -- 172.21.15.131:0/3798408907 learned_addr learned my addr 172.21.15.131:0/3798408907 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd694e00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd690107880 0x7fd6901a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd68f400700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690104d10 0x7fd69019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd68fe00700 1 -- 172.21.15.131:0/3798408907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690104d10 msgr2=0x7fd69019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd68fe00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690104d10 0x7fd69019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd68fe00700 1 -- 172.21.15.131:0/3798408907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd690107880 msgr2=0x7fd6901a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd68fe00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd690107880 0x7fd6901a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.296+0000 7fd68fe00700 1 -- 172.21.15.131:0/3798408907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6800095f0 con 0x7fd690069070 2024-05-10T05:33:25.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.297+0000 7fd68fe00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd690069070 0x7fd69019a1a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd680000c00 tx=0x7fd680009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:25.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.297+0000 7fd68cc00700 1 -- 172.21.15.131:0/3798408907 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68001d070 con 0x7fd690069070 2024-05-10T05:33:25.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.297+0000 7fd68cc00700 1 -- 172.21.15.131:0/3798408907 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd680021410 con 0x7fd690069070 2024-05-10T05:33:25.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.297+0000 7fd68cc00700 1 -- 172.21.15.131:0/3798408907 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd680017650 con 0x7fd690069070 2024-05-10T05:33:25.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.297+0000 7fd696e00700 1 -- 172.21.15.131:0/3798408907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6901a2230 con 0x7fd690069070 2024-05-10T05:33:25.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.297+0000 7fd696e00700 1 -- 172.21.15.131:0/3798408907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6901a2600 con 0x7fd690069070 2024-05-10T05:33:25.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.298+0000 7fd696e00700 1 -- 172.21.15.131:0/3798408907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6900694d0 con 0x7fd690069070 2024-05-10T05:33:25.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.300+0000 7fd68cc00700 1 -- 172.21.15.131:0/3798408907 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd680021580 con 0x7fd690069070 2024-05-10T05:33:25.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.301+0000 7fd68cc00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd6740a11a0 0x7fd6740a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:25.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.301+0000 7fd68cc00700 1 -- 172.21.15.131:0/3798408907 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd6800ce4d0 con 0x7fd690069070 2024-05-10T05:33:25.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.301+0000 7fd68f400700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd6740a11a0 0x7fd6740a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:25.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.302+0000 7fd68f400700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd6740a11a0 0x7fd6740a36b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd69019b240 tx=0x7fd67c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:24 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1243154415' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T05:33:25.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.304+0000 7fd68cc00700 1 -- 172.21.15.131:0/3798408907 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd680097c90 con 0x7fd690069070 2024-05-10T05:33:25.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:24 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1243154415' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T05:33:25.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.516+0000 7fd696e00700 1 -- 172.21.15.131:0/3798408907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd69010d380 con 0x7fd690069070 2024-05-10T05:33:25.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.517+0000 7fd68cc00700 1 -- 172.21.15.131:0/3798408907 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7fd68009b2b0 con 0x7fd690069070 2024-05-10T05:33:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:25.518 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:33:25.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 -- 172.21.15.131:0/3798408907 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd6740a11a0 msgr2=0x7fd6740a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:25.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd6740a11a0 0x7fd6740a36b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd69019b240 tx=0x7fd67c009260 comp rx=0 tx=0).stop 2024-05-10T05:33:25.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 -- 172.21.15.131:0/3798408907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd690069070 msgr2=0x7fd69019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:25.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd690069070 0x7fd69019a1a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd680000c00 tx=0x7fd680009580 comp rx=0 tx=0).stop 2024-05-10T05:33:25.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 -- 172.21.15.131:0/3798408907 shutdown_connections 2024-05-10T05:33:25.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd6740a11a0 0x7fd6740a36b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd690069070 0x7fd69019a1a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd690107880 0x7fd6901a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 --2- 172.21.15.131:0/3798408907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd690104d10 0x7fd69019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 -- 172.21.15.131:0/3798408907 >> 172.21.15.131:0/3798408907 conn(0x7fd6900fc0d0 msgr2=0x7fd6900fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:25.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 -- 172.21.15.131:0/3798408907 shutdown_connections 2024-05-10T05:33:25.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.520+0000 7fd696e00700 1 -- 172.21.15.131:0/3798408907 wait complete. 2024-05-10T05:33:25.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T05:31:49.481667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T05:31:50.960751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:33:25.550 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:33:25.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.820+0000 7f5d1f400700 1 -- 172.21.15.131:0/2800040650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d181036a0 msgr2=0x7f5d18109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:25.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.820+0000 7f5d1f400700 1 --2- 172.21.15.131:0/2800040650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d181036a0 0x7f5d18109f50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000b0a0 tx=0x7f5d1000b3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.820+0000 7f5d1f400700 1 -- 172.21.15.131:0/2800040650 shutdown_connections 2024-05-10T05:33:25.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.820+0000 7f5d1f400700 1 --2- 172.21.15.131:0/2800040650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d181036a0 0x7f5d18109f50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.820+0000 7f5d1f400700 1 --2- 172.21.15.131:0/2800040650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d18101be0 0x7f5d18101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.820+0000 7f5d1f400700 1 --2- 172.21.15.131:0/2800040650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d18102d00 0x7f5d18103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.820+0000 7f5d1f400700 1 -- 172.21.15.131:0/2800040650 >> 172.21.15.131:0/2800040650 conn(0x7f5d180fd210 msgr2=0x7f5d180ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:25.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.820+0000 7f5d1f400700 1 -- 172.21.15.131:0/2800040650 shutdown_connections 2024-05-10T05:33:25.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.820+0000 7f5d1f400700 1 -- 172.21.15.131:0/2800040650 wait complete. 2024-05-10T05:33:25.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.821+0000 7f5d1f400700 1 Processor -- start 2024-05-10T05:33:25.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d1f400700 1 -- start start 2024-05-10T05:33:25.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d1f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d18101be0 0x7f5d1819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:25.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d1f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d18102d00 0x7f5d1819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:25.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d1f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d181036a0 0x7f5d181a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d1f400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d181a1cd0 con 0x7f5d18102d00 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d1f400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d181a1e40 con 0x7f5d18101be0 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d1f400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d181a2000 con 0x7f5d181036a0 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d17e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d18102d00 0x7f5d1819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d17e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d18102d00 0x7f5d1819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33248/0 (socket says 172.21.15.131:33248) 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d17e00700 1 -- 172.21.15.131:0/4131685008 learned_addr learned my addr 172.21.15.131:0/4131685008 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d1d400700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d181036a0 0x7f5d181a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.822+0000 7f5d1ca00700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d18101be0 0x7f5d1819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d17e00700 1 -- 172.21.15.131:0/4131685008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d181036a0 msgr2=0x7f5d181a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d17e00700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d181036a0 0x7f5d181a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d17e00700 1 -- 172.21.15.131:0/4131685008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d18101be0 msgr2=0x7f5d1819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d17e00700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d18101be0 0x7f5d1819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d17e00700 1 -- 172.21.15.131:0/4131685008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d10009d00 con 0x7f5d18102d00 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d17e00700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d18102d00 0x7f5d1819a730 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400e9a0 tx=0x7f5d0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:25.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d15600700 1 -- 172.21.15.131:0/4131685008 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04009790 con 0x7f5d18102d00 2024-05-10T05:33:25.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d15600700 1 -- 172.21.15.131:0/4131685008 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d04010430 con 0x7f5d18102d00 2024-05-10T05:33:25.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d1f400700 1 -- 172.21.15.131:0/4131685008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d181a22e0 con 0x7f5d18102d00 2024-05-10T05:33:25.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d15600700 1 -- 172.21.15.131:0/4131685008 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04018680 con 0x7f5d18102d00 2024-05-10T05:33:25.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.823+0000 7f5d1f400700 1 -- 172.21.15.131:0/4131685008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d181a2840 con 0x7f5d18102d00 2024-05-10T05:33:25.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.824+0000 7f5d1f400700 1 -- 172.21.15.131:0/4131685008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d180694b0 con 0x7f5d18102d00 2024-05-10T05:33:25.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.826+0000 7f5d15600700 1 -- 172.21.15.131:0/4131685008 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5d040105a0 con 0x7f5d18102d00 2024-05-10T05:33:25.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.828+0000 7f5d15600700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d000a1240 0x7f5d000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:25.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.828+0000 7f5d15600700 1 -- 172.21.15.131:0/4131685008 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5d04014070 con 0x7f5d18102d00 2024-05-10T05:33:25.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.828+0000 7f5d1ca00700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d000a1240 0x7f5d000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:25.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.829+0000 7f5d1ca00700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d000a1240 0x7f5d000a3750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d18102c00 tx=0x7f5d0c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:25.830+0000 7f5d15600700 1 -- 172.21.15.131:0/4131685008 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5d04096430 con 0x7f5d18102d00 2024-05-10T05:33:26.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.041+0000 7f5d1f400700 1 -- 172.21.15.131:0/4131685008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d181a2b60 con 0x7f5d18102d00 2024-05-10T05:33:26.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.041+0000 7f5d15600700 1 -- 172.21.15.131:0/4131685008 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d04026020 con 0x7f5d18102d00 2024-05-10T05:33:26.041 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 -- 172.21.15.131:0/4131685008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d000a1240 msgr2=0x7f5d000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d000a1240 0x7f5d000a3750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d18102c00 tx=0x7f5d0c009260 comp rx=0 tx=0).stop 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 -- 172.21.15.131:0/4131685008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d18102d00 msgr2=0x7f5d1819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d18102d00 0x7f5d1819a730 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400e9a0 tx=0x7f5d0400ecb0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 -- 172.21.15.131:0/4131685008 shutdown_connections 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d000a1240 0x7f5d000a3750 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d181036a0 0x7f5d181a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d18101be0 0x7f5d1819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 --2- 172.21.15.131:0/4131685008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d18102d00 0x7f5d1819a730 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 -- 172.21.15.131:0/4131685008 >> 172.21.15.131:0/4131685008 conn(0x7f5d180fd210 msgr2=0x7f5d18073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:26.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 -- 172.21.15.131:0/4131685008 shutdown_connections 2024-05-10T05:33:26.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.044+0000 7f5d1f400700 1 -- 172.21.15.131:0/4131685008 wait complete. 2024-05-10T05:33:26.073 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T05:33:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:25 smithi131 ceph-mon[37264]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-10T05:33:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:25 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1243154415' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T05:33:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:25 smithi131 ceph-mon[37264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:33:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:25 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3798408907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:25 smithi157 ceph-mon[78427]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-10T05:33:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:25 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1243154415' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T05:33:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:25 smithi157 ceph-mon[78427]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:33:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:25 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3798408907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:26.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.339+0000 7f9509e00700 1 -- 172.21.15.131:0/159671558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9504102d00 msgr2=0x7f9504103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:26.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.339+0000 7f9509e00700 1 --2- 172.21.15.131:0/159671558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9504102d00 0x7f9504103110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009990 tx=0x7f94f0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.339+0000 7f9509e00700 1 -- 172.21.15.131:0/159671558 shutdown_connections 2024-05-10T05:33:26.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.339+0000 7f9509e00700 1 --2- 172.21.15.131:0/159671558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95041036a0 0x7f9504109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.339+0000 7f9509e00700 1 --2- 172.21.15.131:0/159671558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9504101be0 0x7f9504101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.339+0000 7f9509e00700 1 --2- 172.21.15.131:0/159671558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9504102d00 0x7f9504103110 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.339+0000 7f9509e00700 1 -- 172.21.15.131:0/159671558 >> 172.21.15.131:0/159671558 conn(0x7f95040fd210 msgr2=0x7f95040ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:26.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.340+0000 7f9509e00700 1 -- 172.21.15.131:0/159671558 shutdown_connections 2024-05-10T05:33:26.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.340+0000 7f9509e00700 1 -- 172.21.15.131:0/159671558 wait complete. 2024-05-10T05:33:26.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9509e00700 1 Processor -- start 2024-05-10T05:33:26.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9509e00700 1 -- start start 2024-05-10T05:33:26.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9509e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9504101be0 0x7f9504197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:26.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9509e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9504102d00 0x7f9504198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:26.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9509e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95041036a0 0x7f950419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:26.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9509e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950419f9c0 con 0x7f9504102d00 2024-05-10T05:33:26.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9509e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950419fb30 con 0x7f95041036a0 2024-05-10T05:33:26.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9509e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950419fcf0 con 0x7f9504101be0 2024-05-10T05:33:26.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9502000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9504102d00 0x7f9504198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:26.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9502000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9504102d00 0x7f9504198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33262/0 (socket says 172.21.15.131:33262) 2024-05-10T05:33:26.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9502000700 1 -- 172.21.15.131:0/2732388650 learned_addr learned my addr 172.21.15.131:0/2732388650 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:26.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9502a00700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9504101be0 0x7f9504197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:26.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.341+0000 7f9503400700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95041036a0 0x7f950419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:26.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9502000700 1 -- 172.21.15.131:0/2732388650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9504101be0 msgr2=0x7f9504197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:26.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9502000700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9504101be0 0x7f9504197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9502000700 1 -- 172.21.15.131:0/2732388650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95041036a0 msgr2=0x7f950419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:26.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9502000700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95041036a0 0x7f950419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9502000700 1 -- 172.21.15.131:0/2732388650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00095f0 con 0x7f9504102d00 2024-05-10T05:33:26.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9503400700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95041036a0 0x7f950419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:26.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9502000700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9504102d00 0x7f9504198420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94f0000c00 tx=0x7f94f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:26.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9508a00700 1 -- 172.21.15.131:0/2732388650 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f001d070 con 0x7f9504102d00 2024-05-10T05:33:26.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9508a00700 1 -- 172.21.15.131:0/2732388650 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94f0021410 con 0x7f9504102d00 2024-05-10T05:33:26.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9508a00700 1 -- 172.21.15.131:0/2732388650 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0017650 con 0x7f9504102d00 2024-05-10T05:33:26.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9509e00700 1 -- 172.21.15.131:0/2732388650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950419ff70 con 0x7f9504102d00 2024-05-10T05:33:26.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.342+0000 7f9509e00700 1 -- 172.21.15.131:0/2732388650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95041a02e0 con 0x7f9504102d00 2024-05-10T05:33:26.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.343+0000 7f9509e00700 1 -- 172.21.15.131:0/2732388650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9504108df0 con 0x7f9504102d00 2024-05-10T05:33:26.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.345+0000 7f9508a00700 1 -- 172.21.15.131:0/2732388650 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f94f00217f0 con 0x7f9504102d00 2024-05-10T05:33:26.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.347+0000 7f9508a00700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f94e80a11c0 0x7f94e80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:26.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.347+0000 7f9508a00700 1 -- 172.21.15.131:0/2732388650 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f94f00ce380 con 0x7f9504102d00 2024-05-10T05:33:26.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.347+0000 7f9502a00700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f94e80a11c0 0x7f94e80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:26.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.348+0000 7f9502a00700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f94e80a11c0 0x7f94e80a36d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9504102c00 tx=0x7f94f4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:26.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.349+0000 7f9508a00700 1 -- 172.21.15.131:0/2732388650 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f94f0097b40 con 0x7f9504102d00 2024-05-10T05:33:26.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:25 smithi142 ceph-mon[77667]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-10T05:33:26.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:25 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1243154415' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T05:33:26.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:25 smithi142 ceph-mon[77667]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:33:26.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:25 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3798408907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:26.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.557+0000 7f9509e00700 1 -- 172.21.15.131:0/2732388650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f950410d430 con 0x7f9504102d00 2024-05-10T05:33:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.558+0000 7f9508a00700 1 -- 172.21.15.131:0/2732388650 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94f009b160 con 0x7f9504102d00 2024-05-10T05:33:26.558 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-10T05:33:26.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.560+0000 7f9509e00700 1 -- 172.21.15.131:0/2732388650 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f94e80a11c0 msgr2=0x7f94e80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.560+0000 7f9509e00700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f94e80a11c0 0x7f94e80a36d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9504102c00 tx=0x7f94f4009260 comp rx=0 tx=0).stop 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.560+0000 7f9509e00700 1 -- 172.21.15.131:0/2732388650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9504102d00 msgr2=0x7f9504198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.560+0000 7f9509e00700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9504102d00 0x7f9504198420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94f0000c00 tx=0x7f94f0004730 comp rx=0 tx=0).stop 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.561+0000 7f9509e00700 1 -- 172.21.15.131:0/2732388650 shutdown_connections 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.561+0000 7f9509e00700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f94e80a11c0 0x7f94e80a36d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.561+0000 7f9509e00700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95041036a0 0x7f950419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.561+0000 7f9509e00700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9504101be0 0x7f9504197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.561+0000 7f9509e00700 1 --2- 172.21.15.131:0/2732388650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9504102d00 0x7f9504198420 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.561+0000 7f9509e00700 1 -- 172.21.15.131:0/2732388650 >> 172.21.15.131:0/2732388650 conn(0x7f95040fd210 msgr2=0x7f9504105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.561+0000 7f9509e00700 1 -- 172.21.15.131:0/2732388650 shutdown_connections 2024-05-10T05:33:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.561+0000 7f9509e00700 1 -- 172.21.15.131:0/2732388650 wait complete. 2024-05-10T05:33:26.589 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T05:33:26.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.857+0000 7f5c21400700 1 -- 172.21.15.131:0/65550489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c1c100ac0 msgr2=0x7f5c1c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:26.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.857+0000 7f5c21400700 1 --2- 172.21.15.131:0/65550489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c1c100ac0 0x7f5c1c100ed0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009990 tx=0x7f5c0c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.857+0000 7f5c21400700 1 -- 172.21.15.131:0/65550489 shutdown_connections 2024-05-10T05:33:26.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.857+0000 7f5c21400700 1 --2- 172.21.15.131:0/65550489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c1c100ac0 0x7f5c1c100ed0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.857+0000 7f5c21400700 1 --2- 172.21.15.131:0/65550489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1c102580 0x7f5c1c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.857+0000 7f5c21400700 1 --2- 172.21.15.131:0/65550489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c101be0 0x7f5c1c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.857+0000 7f5c21400700 1 -- 172.21.15.131:0/65550489 >> 172.21.15.131:0/65550489 conn(0x7f5c1c0fc0d0 msgr2=0x7f5c1c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:26.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.857+0000 7f5c21400700 1 -- 172.21.15.131:0/65550489 shutdown_connections 2024-05-10T05:33:26.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.857+0000 7f5c21400700 1 -- 172.21.15.131:0/65550489 wait complete. 2024-05-10T05:33:26.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.859+0000 7f5c21400700 1 Processor -- start 2024-05-10T05:33:26.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.859+0000 7f5c21400700 1 -- start start 2024-05-10T05:33:26.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.859+0000 7f5c21400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1c100ac0 0x7f5c1c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:26.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.859+0000 7f5c21400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c1c101be0 0x7f5c1c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:26.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.859+0000 7f5c21400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c102580 0x7f5c1c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.859+0000 7f5c21400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c19f9b0 con 0x7f5c1c101be0 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.859+0000 7f5c21400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c19fb20 con 0x7f5c1c102580 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.859+0000 7f5c21400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c19fce0 con 0x7f5c1c100ac0 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c19e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c1c101be0 0x7f5c1c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c19e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c1c101be0 0x7f5c1c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33274/0 (socket says 172.21.15.131:33274) 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c19e00700 1 -- 172.21.15.131:0/1680217996 learned_addr learned my addr 172.21.15.131:0/1680217996 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c1a800700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1c100ac0 0x7f5c1c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c1b200700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c102580 0x7f5c1c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c19e00700 1 -- 172.21.15.131:0/1680217996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1c100ac0 msgr2=0x7f5c1c197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c19e00700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1c100ac0 0x7f5c1c197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c19e00700 1 -- 172.21.15.131:0/1680217996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c102580 msgr2=0x7f5c1c19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c19e00700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c102580 0x7f5c1c19f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:26.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c19e00700 1 -- 172.21.15.131:0/1680217996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c0095f0 con 0x7f5c1c101be0 2024-05-10T05:33:26.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.860+0000 7f5c19e00700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c1c101be0 0x7f5c1c198410 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800e9a0 tx=0x7f5c0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:26.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.861+0000 7f5c13400700 1 -- 172.21.15.131:0/1680217996 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08009790 con 0x7f5c1c101be0 2024-05-10T05:33:26.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.861+0000 7f5c13400700 1 -- 172.21.15.131:0/1680217996 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c08010430 con 0x7f5c1c101be0 2024-05-10T05:33:26.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.861+0000 7f5c21400700 1 -- 172.21.15.131:0/1680217996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c19ffc0 con 0x7f5c1c101be0 2024-05-10T05:33:26.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.861+0000 7f5c13400700 1 -- 172.21.15.131:0/1680217996 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08018680 con 0x7f5c1c101be0 2024-05-10T05:33:26.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.861+0000 7f5c21400700 1 -- 172.21.15.131:0/1680217996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c1a03e0 con 0x7f5c1c101be0 2024-05-10T05:33:26.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.862+0000 7f5c21400700 1 -- 172.21.15.131:0/1680217996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c0694d0 con 0x7f5c1c101be0 2024-05-10T05:33:26.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.864+0000 7f5c13400700 1 -- 172.21.15.131:0/1680217996 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5c080105a0 con 0x7f5c1c101be0 2024-05-10T05:33:26.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.865+0000 7f5c13400700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c000a11c0 0x7f5c000a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:26.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.865+0000 7f5c13400700 1 -- 172.21.15.131:0/1680217996 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5c08014070 con 0x7f5c1c101be0 2024-05-10T05:33:26.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.865+0000 7f5c1a800700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c000a11c0 0x7f5c000a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:26.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.866+0000 7f5c1a800700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c000a11c0 0x7f5c000a36d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00b7b0 tx=0x7f5c0c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:26.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:26.869+0000 7f5c13400700 1 -- 172.21.15.131:0/1680217996 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5c08052d60 con 0x7f5c1c101be0 2024-05-10T05:33:27.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.081+0000 7f5c21400700 1 -- 172.21.15.131:0/1680217996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c1c1a07d0 con 0x7f5c1c101be0 2024-05-10T05:33:27.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.082+0000 7f5c13400700 1 -- 172.21.15.131:0/1680217996 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c08096240 con 0x7f5c1c101be0 2024-05-10T05:33:27.082 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.084+0000 7f5c21400700 1 -- 172.21.15.131:0/1680217996 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c000a11c0 msgr2=0x7f5c000a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.084+0000 7f5c21400700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c000a11c0 0x7f5c000a36d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00b7b0 tx=0x7f5c0c01a040 comp rx=0 tx=0).stop 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.084+0000 7f5c21400700 1 -- 172.21.15.131:0/1680217996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c1c101be0 msgr2=0x7f5c1c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.084+0000 7f5c21400700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c1c101be0 0x7f5c1c198410 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800e9a0 tx=0x7f5c0800ecb0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.085+0000 7f5c21400700 1 -- 172.21.15.131:0/1680217996 shutdown_connections 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.085+0000 7f5c21400700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c000a11c0 0x7f5c000a36d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.085+0000 7f5c21400700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c102580 0x7f5c1c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.085+0000 7f5c21400700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c1c100ac0 0x7f5c1c197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.085+0000 7f5c21400700 1 --2- 172.21.15.131:0/1680217996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c1c101be0 0x7f5c1c198410 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.085+0000 7f5c21400700 1 -- 172.21.15.131:0/1680217996 >> 172.21.15.131:0/1680217996 conn(0x7f5c1c0fc0d0 msgr2=0x7f5c1c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.085+0000 7f5c21400700 1 -- 172.21.15.131:0/1680217996 shutdown_connections 2024-05-10T05:33:27.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.085+0000 7f5c21400700 1 -- 172.21.15.131:0/1680217996 wait complete. 2024-05-10T05:33:27.112 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:33:27.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:26 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4131685008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:27.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:26 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2732388650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:27.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:26 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4131685008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:27.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:26 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2732388650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:27.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:26 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4131685008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:27.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:26 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2732388650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:27.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.376+0000 7f0514a00700 1 -- 172.21.15.131:0/1995049234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0510101be0 msgr2=0x7f0510101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:27.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.376+0000 7f0514a00700 1 --2- 172.21.15.131:0/1995049234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0510101be0 0x7f0510101ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f05000099c0 tx=0x7f0500009cd0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.376+0000 7f0514a00700 1 -- 172.21.15.131:0/1995049234 shutdown_connections 2024-05-10T05:33:27.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.376+0000 7f0514a00700 1 --2- 172.21.15.131:0/1995049234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05101036a0 0x7f0510109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.376+0000 7f0514a00700 1 --2- 172.21.15.131:0/1995049234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0510101be0 0x7f0510101ff0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.376+0000 7f0514a00700 1 --2- 172.21.15.131:0/1995049234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0510102d00 0x7f0510103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.376+0000 7f0514a00700 1 -- 172.21.15.131:0/1995049234 >> 172.21.15.131:0/1995049234 conn(0x7f05100fd1d0 msgr2=0x7f05100ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:27.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.376+0000 7f0514a00700 1 -- 172.21.15.131:0/1995049234 shutdown_connections 2024-05-10T05:33:27.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.376+0000 7f0514a00700 1 -- 172.21.15.131:0/1995049234 wait complete. 2024-05-10T05:33:27.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.377+0000 7f0514a00700 1 Processor -- start 2024-05-10T05:33:27.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f0514a00700 1 -- start start 2024-05-10T05:33:27.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f0514a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0510101be0 0x7f051019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:27.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f0514a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0510102d00 0x7f051019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:27.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f0514a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05101036a0 0x7f05101a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:27.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f0514a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05101a1ca0 con 0x7f0510101be0 2024-05-10T05:33:27.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f0514a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05101a1e10 con 0x7f05101036a0 2024-05-10T05:33:27.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f0514a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05101a1fd0 con 0x7f0510102d00 2024-05-10T05:33:27.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f050de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0510101be0 0x7f051019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:27.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f050de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0510101be0 0x7f051019a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33306/0 (socket says 172.21.15.131:33306) 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f050de00700 1 -- 172.21.15.131:0/1970196873 learned_addr learned my addr 172.21.15.131:0/1970196873 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f0507e00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0510102d00 0x7f051019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.378+0000 7f050e800700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05101036a0 0x7f05101a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f050de00700 1 -- 172.21.15.131:0/1970196873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0510102d00 msgr2=0x7f051019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f050de00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0510102d00 0x7f051019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f050de00700 1 -- 172.21.15.131:0/1970196873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05101036a0 msgr2=0x7f05101a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f050de00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05101036a0 0x7f05101a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f050de00700 1 -- 172.21.15.131:0/1970196873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05000095f0 con 0x7f0510101be0 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f050e800700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05101036a0 0x7f05101a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f050de00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0510101be0 0x7f051019a1c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0500009990 tx=0x7f050000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f0506a00700 1 -- 172.21.15.131:0/1970196873 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050001d070 con 0x7f0510101be0 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f0514a00700 1 -- 172.21.15.131:0/1970196873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05101a2250 con 0x7f0510101be0 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f0506a00700 1 -- 172.21.15.131:0/1970196873 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05000054f0 con 0x7f0510101be0 2024-05-10T05:33:27.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f0514a00700 1 -- 172.21.15.131:0/1970196873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05101a2680 con 0x7f0510101be0 2024-05-10T05:33:27.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.379+0000 7f0506a00700 1 -- 172.21.15.131:0/1970196873 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0500004a30 con 0x7f0510101be0 2024-05-10T05:33:27.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.380+0000 7f0514a00700 1 -- 172.21.15.131:0/1970196873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05100694b0 con 0x7f0510101be0 2024-05-10T05:33:27.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.382+0000 7f0506a00700 1 -- 172.21.15.131:0/1970196873 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f050000fb80 con 0x7f0510101be0 2024-05-10T05:33:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.384+0000 7f0506a00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04f40a1120 0x7f04f40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:27.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.384+0000 7f0506a00700 1 -- 172.21.15.131:0/1970196873 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f05000cd630 con 0x7f0510101be0 2024-05-10T05:33:27.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.384+0000 7f0507e00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04f40a1120 0x7f04f40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:27.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.385+0000 7f0507e00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04f40a1120 0x7f04f40a3630 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f051019b260 tx=0x7f04f8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:27.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.388+0000 7f0506a00700 1 -- 172.21.15.131:0/1970196873 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0500096df0 con 0x7f0510101be0 2024-05-10T05:33:27.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.605+0000 7f0514a00700 1 -- 172.21.15.131:0/1970196873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05101a2990 con 0x7f0510101be0 2024-05-10T05:33:27.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.607+0000 7f0506a00700 1 -- 172.21.15.131:0/1970196873 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7f050009a410 con 0x7f0510101be0 2024-05-10T05:33:27.607 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:27.607 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:33:27.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.609+0000 7f0514a00700 1 -- 172.21.15.131:0/1970196873 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04f40a1120 msgr2=0x7f04f40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:27.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.609+0000 7f0514a00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04f40a1120 0x7f04f40a3630 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f051019b260 tx=0x7f04f8009260 comp rx=0 tx=0).stop 2024-05-10T05:33:27.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.609+0000 7f0514a00700 1 -- 172.21.15.131:0/1970196873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0510101be0 msgr2=0x7f051019a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:27.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.609+0000 7f0514a00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0510101be0 0x7f051019a1c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0500009990 tx=0x7f050000f690 comp rx=0 tx=0).stop 2024-05-10T05:33:27.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.610+0000 7f0514a00700 1 -- 172.21.15.131:0/1970196873 shutdown_connections 2024-05-10T05:33:27.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.610+0000 7f0514a00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f04f40a1120 0x7f04f40a3630 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.610+0000 7f0514a00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0510101be0 0x7f051019a1c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.610+0000 7f0514a00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05101036a0 0x7f05101a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.610+0000 7f0514a00700 1 --2- 172.21.15.131:0/1970196873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0510102d00 0x7f051019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.610+0000 7f0514a00700 1 -- 172.21.15.131:0/1970196873 >> 172.21.15.131:0/1970196873 conn(0x7f05100fd1d0 msgr2=0x7f0510073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:27.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.610+0000 7f0514a00700 1 -- 172.21.15.131:0/1970196873 shutdown_connections 2024-05-10T05:33:27.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.610+0000 7f0514a00700 1 -- 172.21.15.131:0/1970196873 wait complete. 2024-05-10T05:33:27.637 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T05:31:49.481667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T05:31:50.960751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:33:27.637 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:33:27.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.905+0000 7f086e200700 1 -- 172.21.15.131:0/4206519261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08681034a0 msgr2=0x7f0868109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.905+0000 7f086e200700 1 --2- 172.21.15.131:0/4206519261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08681034a0 0x7f0868109d50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f086000b0a0 tx=0x7f086000b3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.906+0000 7f086e200700 1 -- 172.21.15.131:0/4206519261 shutdown_connections 2024-05-10T05:33:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.906+0000 7f086e200700 1 --2- 172.21.15.131:0/4206519261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08681034a0 0x7f0868109d50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.906+0000 7f086e200700 1 --2- 172.21.15.131:0/4206519261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08681019e0 0x7f0868101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.906+0000 7f086e200700 1 --2- 172.21.15.131:0/4206519261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0868102b00 0x7f0868102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.906+0000 7f086e200700 1 -- 172.21.15.131:0/4206519261 >> 172.21.15.131:0/4206519261 conn(0x7f08680fcfd0 msgr2=0x7f08680ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.906+0000 7f086e200700 1 -- 172.21.15.131:0/4206519261 shutdown_connections 2024-05-10T05:33:27.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.906+0000 7f086e200700 1 -- 172.21.15.131:0/4206519261 wait complete. 2024-05-10T05:33:27.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.907+0000 7f086e200700 1 Processor -- start 2024-05-10T05:33:27.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.907+0000 7f086e200700 1 -- start start 2024-05-10T05:33:27.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f086e200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08681019e0 0x7f086819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:27.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f086e200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0868102b00 0x7f086819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:27.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f086e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08681034a0 0x7f08681a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:27.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f086e200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681a1be0 con 0x7f08681019e0 2024-05-10T05:33:27.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f086e200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681a1d20 con 0x7f08681034a0 2024-05-10T05:33:27.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f086e200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681a1eb0 con 0x7f0868102b00 2024-05-10T05:33:27.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f086ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08681019e0 0x7f086819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:27.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f086ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08681019e0 0x7f086819a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33314/0 (socket says 172.21.15.131:33314) 2024-05-10T05:33:27.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f086ce00700 1 -- 172.21.15.131:0/3135872034 learned_addr learned my addr 172.21.15.131:0/3135872034 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f0867e00700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0868102b00 0x7f086819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.908+0000 7f086d800700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08681034a0 0x7f08681a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f086ce00700 1 -- 172.21.15.131:0/3135872034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0868102b00 msgr2=0x7f086819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f086ce00700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0868102b00 0x7f086819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f086ce00700 1 -- 172.21.15.131:0/3135872034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08681034a0 msgr2=0x7f08681a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f086ce00700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08681034a0 0x7f08681a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f086ce00700 1 -- 172.21.15.131:0/3135872034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0860009d00 con 0x7f08681019e0 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f086d800700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08681034a0 0x7f08681a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f086ce00700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08681019e0 0x7f086819a140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f085c00e970 tx=0x7f085c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f0865600700 1 -- 172.21.15.131:0/3135872034 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c00cbd0 con 0x7f08681019e0 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f086e200700 1 -- 172.21.15.131:0/3135872034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08681a2190 con 0x7f08681019e0 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f0865600700 1 -- 172.21.15.131:0/3135872034 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f085c00eec0 con 0x7f08681019e0 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f0865600700 1 -- 172.21.15.131:0/3135872034 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c003d20 con 0x7f08681019e0 2024-05-10T05:33:27.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.909+0000 7f086e200700 1 -- 172.21.15.131:0/3135872034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08681a26f0 con 0x7f08681019e0 2024-05-10T05:33:27.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.910+0000 7f086e200700 1 -- 172.21.15.131:0/3135872034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0868068110 con 0x7f08681019e0 2024-05-10T05:33:27.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.912+0000 7f0865600700 1 -- 172.21.15.131:0/3135872034 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f085c003e80 con 0x7f08681019e0 2024-05-10T05:33:27.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.914+0000 7f0865600700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08500a1120 0x7f08500a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:27.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.914+0000 7f0865600700 1 -- 172.21.15.131:0/3135872034 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f085c014070 con 0x7f08681019e0 2024-05-10T05:33:27.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.914+0000 7f0867e00700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08500a1120 0x7f08500a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:27.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.914+0000 7f0867e00700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08500a1120 0x7f08500a3630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f086819b1e0 tx=0x7f0854009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:27.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:27.918+0000 7f0865600700 1 -- 172.21.15.131:0/3135872034 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f085c0962f0 con 0x7f08681019e0 2024-05-10T05:33:28.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.131+0000 7f086e200700 1 -- 172.21.15.131:0/3135872034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f086810d240 con 0x7f08681019e0 2024-05-10T05:33:28.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.132+0000 7f0865600700 1 -- 172.21.15.131:0/3135872034 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f085c099910 con 0x7f08681019e0 2024-05-10T05:33:28.133 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:33:28.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.134+0000 7f086e200700 1 -- 172.21.15.131:0/3135872034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08500a1120 msgr2=0x7f08500a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:28.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.134+0000 7f086e200700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08500a1120 0x7f08500a3630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f086819b1e0 tx=0x7f0854009260 comp rx=0 tx=0).stop 2024-05-10T05:33:28.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.134+0000 7f086e200700 1 -- 172.21.15.131:0/3135872034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08681019e0 msgr2=0x7f086819a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.134+0000 7f086e200700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08681019e0 0x7f086819a140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f085c00e970 tx=0x7f085c00ec80 comp rx=0 tx=0).stop 2024-05-10T05:33:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.134+0000 7f086e200700 1 -- 172.21.15.131:0/3135872034 shutdown_connections 2024-05-10T05:33:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.134+0000 7f086e200700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08500a1120 0x7f08500a3630 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.134+0000 7f086e200700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08681019e0 0x7f086819a140 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.134+0000 7f086e200700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08681034a0 0x7f08681a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.134+0000 7f086e200700 1 --2- 172.21.15.131:0/3135872034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0868102b00 0x7f086819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.134+0000 7f086e200700 1 -- 172.21.15.131:0/3135872034 >> 172.21.15.131:0/3135872034 conn(0x7f08680fcfd0 msgr2=0x7f0868073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.135+0000 7f086e200700 1 -- 172.21.15.131:0/3135872034 shutdown_connections 2024-05-10T05:33:28.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.135+0000 7f086e200700 1 -- 172.21.15.131:0/3135872034 wait complete. 2024-05-10T05:33:28.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T05:33:28.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:27 smithi131 ceph-mon[37264]: pgmap v288: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-10T05:33:28.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:27 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1680217996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:28.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:27 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1970196873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:27 smithi157 ceph-mon[78427]: pgmap v288: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-10T05:33:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:27 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1680217996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:27 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1970196873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:27 smithi142 ceph-mon[77667]: pgmap v288: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-10T05:33:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:27 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1680217996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:27 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1970196873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.428+0000 7f97d7000700 1 -- 172.21.15.131:0/4163612952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d01034b0 msgr2=0x7f97d0109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.428+0000 7f97d7000700 1 --2- 172.21.15.131:0/4163612952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d01034b0 0x7f97d0109d60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f97c800b0d0 tx=0x7f97c800b3e0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.428+0000 7f97d7000700 1 -- 172.21.15.131:0/4163612952 shutdown_connections 2024-05-10T05:33:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.428+0000 7f97d7000700 1 --2- 172.21.15.131:0/4163612952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d01034b0 0x7f97d0109d60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.428+0000 7f97d7000700 1 --2- 172.21.15.131:0/4163612952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01019f0 0x7f97d0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.428+0000 7f97d7000700 1 --2- 172.21.15.131:0/4163612952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97d0102b10 0x7f97d0102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.428+0000 7f97d7000700 1 -- 172.21.15.131:0/4163612952 >> 172.21.15.131:0/4163612952 conn(0x7f97d00fd000 msgr2=0x7f97d00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.428+0000 7f97d7000700 1 -- 172.21.15.131:0/4163612952 shutdown_connections 2024-05-10T05:33:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.428+0000 7f97d7000700 1 -- 172.21.15.131:0/4163612952 wait complete. 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.429+0000 7f97d7000700 1 Processor -- start 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.429+0000 7f97d7000700 1 -- start start 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.429+0000 7f97d7000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01019f0 0x7f97d019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.429+0000 7f97d7000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97d0102b10 0x7f97d019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.429+0000 7f97d7000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d01034b0 0x7f97d01a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.429+0000 7f97d7000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01a1c10 con 0x7f97d0102b10 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.429+0000 7f97d7000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01a1d50 con 0x7f97d01034b0 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.429+0000 7f97d7000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01a1ee0 con 0x7f97d01019f0 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97d0102b10 0x7f97d019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d5c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01019f0 0x7f97d019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97d0102b10 0x7f97d019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33320/0 (socket says 172.21.15.131:33320) 2024-05-10T05:33:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d5200700 1 -- 172.21.15.131:0/3870057939 learned_addr learned my addr 172.21.15.131:0/3870057939 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d6600700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d01034b0 0x7f97d01a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d5c00700 1 -- 172.21.15.131:0/3870057939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d01034b0 msgr2=0x7f97d01a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d5c00700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d01034b0 0x7f97d01a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d5c00700 1 -- 172.21.15.131:0/3870057939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97d0102b10 msgr2=0x7f97d019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d5c00700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97d0102b10 0x7f97d019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d5c00700 1 -- 172.21.15.131:0/3870057939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc009520 con 0x7f97d01019f0 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.430+0000 7f97d5c00700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01019f0 0x7f97d019a170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f97c400e9c0 tx=0x7f97c400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.431+0000 7f97ce000700 1 -- 172.21.15.131:0/3870057939 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4004220 con 0x7f97d01019f0 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.431+0000 7f97ce000700 1 -- 172.21.15.131:0/3870057939 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97c4004380 con 0x7f97d01019f0 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.431+0000 7f97d7000700 1 -- 172.21.15.131:0/3870057939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c8009d00 con 0x7f97d01019f0 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.431+0000 7f97ce000700 1 -- 172.21.15.131:0/3870057939 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4003be0 con 0x7f97d01019f0 2024-05-10T05:33:28.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.431+0000 7f97d7000700 1 -- 172.21.15.131:0/3870057939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d01a2540 con 0x7f97d01019f0 2024-05-10T05:33:28.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.432+0000 7f97d7000700 1 -- 172.21.15.131:0/3870057939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d0068110 con 0x7f97d01019f0 2024-05-10T05:33:28.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.434+0000 7f97ce000700 1 -- 172.21.15.131:0/3870057939 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f97c4003d40 con 0x7f97d01019f0 2024-05-10T05:33:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.435+0000 7f97ce000700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97b80a1120 0x7f97b80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.435+0000 7f97ce000700 1 -- 172.21.15.131:0/3870057939 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f97c4014070 con 0x7f97d01019f0 2024-05-10T05:33:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.435+0000 7f97d5200700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97b80a1120 0x7f97b80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.436+0000 7f97d5200700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97b80a1120 0x7f97b80a3630 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f97d019b210 tx=0x7f97bc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:28.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.438+0000 7f97ce000700 1 -- 172.21.15.131:0/3870057939 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97c40969b0 con 0x7f97d01019f0 2024-05-10T05:33:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.646+0000 7f97d7000700 1 -- 172.21.15.131:0/3870057939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97d010d250 con 0x7f97d01019f0 2024-05-10T05:33:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.647+0000 7f97ce000700 1 -- 172.21.15.131:0/3870057939 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97c4099fd0 con 0x7f97d01019f0 2024-05-10T05:33:28.648 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.649+0000 7f97d7000700 1 -- 172.21.15.131:0/3870057939 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97b80a1120 msgr2=0x7f97b80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.649+0000 7f97d7000700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97b80a1120 0x7f97b80a3630 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f97d019b210 tx=0x7f97bc009260 comp rx=0 tx=0).stop 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.649+0000 7f97d7000700 1 -- 172.21.15.131:0/3870057939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01019f0 msgr2=0x7f97d019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.649+0000 7f97d7000700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01019f0 0x7f97d019a170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f97c400e9c0 tx=0x7f97c400ecd0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.650+0000 7f97d7000700 1 -- 172.21.15.131:0/3870057939 shutdown_connections 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.650+0000 7f97d7000700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97b80a1120 0x7f97b80a3630 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.650+0000 7f97d7000700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d01034b0 0x7f97d01a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.650+0000 7f97d7000700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01019f0 0x7f97d019a170 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.650+0000 7f97d7000700 1 --2- 172.21.15.131:0/3870057939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97d0102b10 0x7f97d019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.650+0000 7f97d7000700 1 -- 172.21.15.131:0/3870057939 >> 172.21.15.131:0/3870057939 conn(0x7f97d00fd000 msgr2=0x7f97d0073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.650+0000 7f97d7000700 1 -- 172.21.15.131:0/3870057939 shutdown_connections 2024-05-10T05:33:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.650+0000 7f97d7000700 1 -- 172.21.15.131:0/3870057939 wait complete. 2024-05-10T05:33:28.680 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T05:33:28.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.948+0000 7f2a61a00700 1 -- 172.21.15.131:0/2291457007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5c069090 msgr2=0x7f2a5c104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:28.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.948+0000 7f2a61a00700 1 --2- 172.21.15.131:0/2291457007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5c069090 0x7f2a5c104720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009990 tx=0x7f2a4c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.948+0000 7f2a61a00700 1 -- 172.21.15.131:0/2291457007 shutdown_connections 2024-05-10T05:33:28.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.948+0000 7f2a61a00700 1 --2- 172.21.15.131:0/2291457007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c107880 0x7f2a5c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.948+0000 7f2a61a00700 1 --2- 172.21.15.131:0/2291457007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5c069090 0x7f2a5c104720 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.948+0000 7f2a61a00700 1 --2- 172.21.15.131:0/2291457007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c104d10 0x7f2a5c1071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.948+0000 7f2a61a00700 1 -- 172.21.15.131:0/2291457007 >> 172.21.15.131:0/2291457007 conn(0x7f2a5c0fc0d0 msgr2=0x7f2a5c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:28.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.948+0000 7f2a61a00700 1 -- 172.21.15.131:0/2291457007 shutdown_connections 2024-05-10T05:33:28.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.948+0000 7f2a61a00700 1 -- 172.21.15.131:0/2291457007 wait complete. 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.949+0000 7f2a61a00700 1 Processor -- start 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.949+0000 7f2a61a00700 1 -- start start 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.949+0000 7f2a61a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5c069090 0x7f2a5c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.949+0000 7f2a61a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c104d10 0x7f2a5c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.949+0000 7f2a61a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c107880 0x7f2a5c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.949+0000 7f2a61a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c19fa40 con 0x7f2a5c107880 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.949+0000 7f2a61a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c19fbb0 con 0x7f2a5c069090 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.949+0000 7f2a61a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c19fd70 con 0x7f2a5c104d10 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5c069090 0x7f2a5c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5c069090 0x7f2a5c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43346/0 (socket says 172.21.15.131:43346) 2024-05-10T05:33:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5aa00700 1 -- 172.21.15.131:0/1677322719 learned_addr learned my addr 172.21.15.131:0/1677322719 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5a000700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c104d10 0x7f2a5c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5b400700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c107880 0x7f2a5c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5aa00700 1 -- 172.21.15.131:0/1677322719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c104d10 msgr2=0x7f2a5c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5aa00700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c104d10 0x7f2a5c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5aa00700 1 -- 172.21.15.131:0/1677322719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c107880 msgr2=0x7f2a5c19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5aa00700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c107880 0x7f2a5c19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5aa00700 1 -- 172.21.15.131:0/1677322719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a54009c40 con 0x7f2a5c069090 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5a000700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c104d10 0x7f2a5c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.950+0000 7f2a5aa00700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5c069090 0x7f2a5c197f60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c000c00 tx=0x7f2a4c005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.951+0000 7f2a53400700 1 -- 172.21.15.131:0/1677322719 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c01d070 con 0x7f2a5c069090 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.951+0000 7f2a53400700 1 -- 172.21.15.131:0/1677322719 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a4c021410 con 0x7f2a5c069090 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.951+0000 7f2a53400700 1 -- 172.21.15.131:0/1677322719 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c017650 con 0x7f2a5c069090 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.951+0000 7f2a61a00700 1 -- 172.21.15.131:0/1677322719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c0095f0 con 0x7f2a5c069090 2024-05-10T05:33:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.951+0000 7f2a61a00700 1 -- 172.21.15.131:0/1677322719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c1a01d0 con 0x7f2a5c069090 2024-05-10T05:33:28.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.952+0000 7f2a61a00700 1 -- 172.21.15.131:0/1677322719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c0fdc80 con 0x7f2a5c069090 2024-05-10T05:33:28.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.954+0000 7f2a53400700 1 -- 172.21.15.131:0/1677322719 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2a4c021580 con 0x7f2a5c069090 2024-05-10T05:33:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.955+0000 7f2a53400700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a400a1240 0x7f2a400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.955+0000 7f2a53400700 1 -- 172.21.15.131:0/1677322719 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2a4c0ce110 con 0x7f2a5c069090 2024-05-10T05:33:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.955+0000 7f2a5a000700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a400a1240 0x7f2a400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.956+0000 7f2a5a000700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a400a1240 0x7f2a400a3750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c199000 tx=0x7f2a48009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:28.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:28.958+0000 7f2a53400700 1 -- 172.21.15.131:0/1677322719 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2a4c097930 con 0x7f2a5c069090 2024-05-10T05:33:29.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.167+0000 7f2a61a00700 1 -- 172.21.15.131:0/1677322719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a5c0694d0 con 0x7f2a5c069090 2024-05-10T05:33:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.167+0000 7f2a53400700 1 -- 172.21.15.131:0/1677322719 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a4c09af50 con 0x7f2a5c069090 2024-05-10T05:33:29.168 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-10T05:33:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 -- 172.21.15.131:0/1677322719 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a400a1240 msgr2=0x7f2a400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a400a1240 0x7f2a400a3750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c199000 tx=0x7f2a48009260 comp rx=0 tx=0).stop 2024-05-10T05:33:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 -- 172.21.15.131:0/1677322719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5c069090 msgr2=0x7f2a5c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5c069090 0x7f2a5c197f60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c000c00 tx=0x7f2a4c005b00 comp rx=0 tx=0).stop 2024-05-10T05:33:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 -- 172.21.15.131:0/1677322719 shutdown_connections 2024-05-10T05:33:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a400a1240 0x7f2a400a3750 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c107880 0x7f2a5c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5c069090 0x7f2a5c197f60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 --2- 172.21.15.131:0/1677322719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c104d10 0x7f2a5c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 -- 172.21.15.131:0/1677322719 >> 172.21.15.131:0/1677322719 conn(0x7f2a5c0fc0d0 msgr2=0x7f2a5c103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 -- 172.21.15.131:0/1677322719 shutdown_connections 2024-05-10T05:33:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.170+0000 7f2a61a00700 1 -- 172.21.15.131:0/1677322719 wait complete. 2024-05-10T05:33:29.194 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:28 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3135872034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:29.194 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:28 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3870057939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:29.196 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-10T05:33:29.197 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:33:29.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:28 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3135872034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:29.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:28 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3870057939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:29.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:28 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3135872034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:29.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:28 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3870057939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:29.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.457+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1197383824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a80102b20 msgr2=0x7f0a80102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:29.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.457+0000 7f0a7fe00700 1 --2- 172.21.15.131:0/1197383824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a80102b20 0x7f0a80102f30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009990 tx=0x7f0a68009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.457+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1197383824 shutdown_connections 2024-05-10T05:33:29.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.457+0000 7f0a7fe00700 1 --2- 172.21.15.131:0/1197383824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a801034c0 0x7f0a80109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.457+0000 7f0a7fe00700 1 --2- 172.21.15.131:0/1197383824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a80101a00 0x7f0a80101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.457+0000 7f0a7fe00700 1 --2- 172.21.15.131:0/1197383824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a80102b20 0x7f0a80102f30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.457+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1197383824 >> 172.21.15.131:0/1197383824 conn(0x7f0a800fd010 msgr2=0x7f0a800ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:29.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.458+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1197383824 shutdown_connections 2024-05-10T05:33:29.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.458+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1197383824 wait complete. 2024-05-10T05:33:29.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7fe00700 1 Processor -- start 2024-05-10T05:33:29.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7fe00700 1 -- start start 2024-05-10T05:33:29.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a80101a00 0x7f0a80074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:29.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a80102b20 0x7f0a80073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a801034c0 0x7f0a80073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7fe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80074100 con 0x7f0a80101a00 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7fe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80074270 con 0x7f0a80102b20 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80074430 con 0x7f0a801034c0 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a80101a00 0x7f0a80074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a80101a00 0x7f0a80074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33366/0 (socket says 172.21.15.131:33366) 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.459+0000 7f0a7ea00700 1 -- 172.21.15.131:0/1799367146 learned_addr learned my addr 172.21.15.131:0/1799367146 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a76000700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a80102b20 0x7f0a80073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a7f400700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a801034c0 0x7f0a80073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a7ea00700 1 -- 172.21.15.131:0/1799367146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a801034c0 msgr2=0x7f0a80073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a7ea00700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a801034c0 0x7f0a80073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a7ea00700 1 -- 172.21.15.131:0/1799367146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a80102b20 msgr2=0x7f0a80073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:29.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a7ea00700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a80102b20 0x7f0a80073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a7ea00700 1 -- 172.21.15.131:0/1799367146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70009520 con 0x7f0a80101a00 2024-05-10T05:33:29.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a76000700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a80102b20 0x7f0a80073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:29.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a7ea00700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a80101a00 0x7f0a80074ec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000ea10 tx=0x7f0a7000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:29.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a77e00700 1 -- 172.21.15.131:0/1799367146 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000cbd0 con 0x7f0a80101a00 2024-05-10T05:33:29.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a77e00700 1 -- 172.21.15.131:0/1799367146 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a7000eeb0 con 0x7f0a80101a00 2024-05-10T05:33:29.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.460+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1799367146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a680095f0 con 0x7f0a80101a00 2024-05-10T05:33:29.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.461+0000 7f0a77e00700 1 -- 172.21.15.131:0/1799367146 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70018680 con 0x7f0a80101a00 2024-05-10T05:33:29.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.461+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1799367146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a801a2240 con 0x7f0a80101a00 2024-05-10T05:33:29.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.462+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1799367146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a80108c10 con 0x7f0a80101a00 2024-05-10T05:33:29.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.464+0000 7f0a77e00700 1 -- 172.21.15.131:0/1799367146 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0a70010430 con 0x7f0a80101a00 2024-05-10T05:33:29.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.465+0000 7f0a77e00700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0a640a11c0 0x7f0a640a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:29.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.465+0000 7f0a77e00700 1 -- 172.21.15.131:0/1799367146 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0a70014070 con 0x7f0a80101a00 2024-05-10T05:33:29.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.465+0000 7f0a76000700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0a640a11c0 0x7f0a640a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:29.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.466+0000 7f0a76000700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0a640a11c0 0x7f0a640a36d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0a80074860 tx=0x7f0a6801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:29.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.470+0000 7f0a77e00700 1 -- 172.21.15.131:0/1799367146 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0a700cee60 con 0x7f0a80101a00 2024-05-10T05:33:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.722+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1799367146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a80068110 con 0x7f0a80101a00 2024-05-10T05:33:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.723+0000 7f0a77e00700 1 -- 172.21.15.131:0/1799367146 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f0a700963f0 con 0x7f0a80101a00 2024-05-10T05:33:29.723 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:29.723 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:33:24.915360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:33:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1799367146 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0a640a11c0 msgr2=0x7f0a640a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0a640a11c0 0x7f0a640a36d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0a80074860 tx=0x7f0a6801a040 comp rx=0 tx=0).stop 2024-05-10T05:33:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1799367146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a80101a00 msgr2=0x7f0a80074ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a80101a00 0x7f0a80074ec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000ea10 tx=0x7f0a7000ed20 comp rx=0 tx=0).stop 2024-05-10T05:33:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1799367146 shutdown_connections 2024-05-10T05:33:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0a640a11c0 0x7f0a640a36d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a80101a00 0x7f0a80074ec0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a801034c0 0x7f0a80073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 --2- 172.21.15.131:0/1799367146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a80102b20 0x7f0a80073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1799367146 >> 172.21.15.131:0/1799367146 conn(0x7f0a800fd010 msgr2=0x7f0a80105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1799367146 shutdown_connections 2024-05-10T05:33:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:29.725+0000 7f0a7fe00700 1 -- 172.21.15.131:0/1799367146 wait complete. 2024-05-10T05:33:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-05-10T05:33:29.752 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-10T05:33:29.752 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-10T05:33:29.752 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T05:33:30.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.015+0000 7f7d28e00700 1 -- 172.21.15.131:0/292724247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d24102d00 msgr2=0x7f7d24103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:30.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.015+0000 7f7d28e00700 1 --2- 172.21.15.131:0/292724247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d24102d00 0x7f7d24103110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009990 tx=0x7f7d10009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.015+0000 7f7d28e00700 1 -- 172.21.15.131:0/292724247 shutdown_connections 2024-05-10T05:33:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.015+0000 7f7d28e00700 1 --2- 172.21.15.131:0/292724247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d241036a0 0x7f7d24109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.015+0000 7f7d28e00700 1 --2- 172.21.15.131:0/292724247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d24101be0 0x7f7d24101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.015+0000 7f7d28e00700 1 --2- 172.21.15.131:0/292724247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d24102d00 0x7f7d24103110 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.015+0000 7f7d28e00700 1 -- 172.21.15.131:0/292724247 >> 172.21.15.131:0/292724247 conn(0x7f7d240fd210 msgr2=0x7f7d240ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.015+0000 7f7d28e00700 1 -- 172.21.15.131:0/292724247 shutdown_connections 2024-05-10T05:33:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.015+0000 7f7d28e00700 1 -- 172.21.15.131:0/292724247 wait complete. 2024-05-10T05:33:30.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.016+0000 7f7d28e00700 1 Processor -- start 2024-05-10T05:33:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.017+0000 7f7d28e00700 1 -- start start 2024-05-10T05:33:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.017+0000 7f7d28e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d24101be0 0x7f7d24197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.017+0000 7f7d28e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d24102d00 0x7f7d24198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.017+0000 7f7d28e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d241036a0 0x7f7d2419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.017+0000 7f7d28e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2419fab0 con 0x7f7d24102d00 2024-05-10T05:33:30.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.017+0000 7f7d28e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2419fc20 con 0x7f7d241036a0 2024-05-10T05:33:30.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.017+0000 7f7d28e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2419fde0 con 0x7f7d24101be0 2024-05-10T05:33:30.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d21400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d24102d00 0x7f7d24198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:30.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d21e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d24101be0 0x7f7d24197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:30.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d21e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d24101be0 0x7f7d24197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54708/0 (socket says 172.21.15.131:54708) 2024-05-10T05:33:30.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d21e00700 1 -- 172.21.15.131:0/886224130 learned_addr learned my addr 172.21.15.131:0/886224130 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:30.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d22800700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d241036a0 0x7f7d2419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:30.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d21e00700 1 -- 172.21.15.131:0/886224130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d241036a0 msgr2=0x7f7d2419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d21e00700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d241036a0 0x7f7d2419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d21e00700 1 -- 172.21.15.131:0/886224130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d24102d00 msgr2=0x7f7d24198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d21e00700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d24102d00 0x7f7d24198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d21e00700 1 -- 172.21.15.131:0/886224130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14009520 con 0x7f7d24101be0 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.018+0000 7f7d21e00700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d24101be0 0x7f7d24197fd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400ea10 tx=0x7f7d1400ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.019+0000 7f7d1aa00700 1 -- 172.21.15.131:0/886224130 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400cbd0 con 0x7f7d24101be0 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.019+0000 7f7d1aa00700 1 -- 172.21.15.131:0/886224130 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1400eeb0 con 0x7f7d24101be0 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.019+0000 7f7d28e00700 1 -- 172.21.15.131:0/886224130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d100095f0 con 0x7f7d24101be0 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.019+0000 7f7d1aa00700 1 -- 172.21.15.131:0/886224130 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14018680 con 0x7f7d24101be0 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.019+0000 7f7d28e00700 1 -- 172.21.15.131:0/886224130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d241a0490 con 0x7f7d24101be0 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.020+0000 7f7d28e00700 1 -- 172.21.15.131:0/886224130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d24108df0 con 0x7f7d24101be0 2024-05-10T05:33:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.022+0000 7f7d1aa00700 1 -- 172.21.15.131:0/886224130 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7d14010430 con 0x7f7d24101be0 2024-05-10T05:33:30.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.023+0000 7f7d1aa00700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7d080a11a0 0x7f7d080a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:30.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.023+0000 7f7d1aa00700 1 -- 172.21.15.131:0/886224130 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f7d14014070 con 0x7f7d24101be0 2024-05-10T05:33:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.023+0000 7f7d21400700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7d080a11a0 0x7f7d080a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.024+0000 7f7d21400700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7d080a11a0 0x7f7d080a36b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000b2e0 tx=0x7f7d1001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.026+0000 7f7d1aa00700 1 -- 172.21.15.131:0/886224130 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7d14096380 con 0x7f7d24101be0 2024-05-10T05:33:30.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:29 smithi131 ceph-mon[37264]: pgmap v289: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-10T05:33:30.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:29 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1677322719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:30.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:29 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1799367146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:30.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.230+0000 7f7d28e00700 1 -- 172.21.15.131:0/886224130 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d24061650 con 0x7f7d080a11a0 2024-05-10T05:33:30.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.242+0000 7f7d1aa00700 1 -- 172.21.15.131:0/886224130 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344599 (secure 0 0 0) 0x7f7d24061650 con 0x7f7d080a11a0 2024-05-10T05:33:30.245 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:30.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.247+0000 7f7d28e00700 1 -- 172.21.15.131:0/886224130 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7d080a11a0 msgr2=0x7f7d080a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.247+0000 7f7d28e00700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7d080a11a0 0x7f7d080a36b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000b2e0 tx=0x7f7d1001a040 comp rx=0 tx=0).stop 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.247+0000 7f7d28e00700 1 -- 172.21.15.131:0/886224130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d24101be0 msgr2=0x7f7d24197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.247+0000 7f7d28e00700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d24101be0 0x7f7d24197fd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400ea10 tx=0x7f7d1400ed20 comp rx=0 tx=0).stop 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.247+0000 7f7d28e00700 1 -- 172.21.15.131:0/886224130 shutdown_connections 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.247+0000 7f7d28e00700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7d080a11a0 0x7f7d080a36b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.247+0000 7f7d28e00700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d241036a0 0x7f7d2419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.248+0000 7f7d28e00700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d24101be0 0x7f7d24197fd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.248+0000 7f7d28e00700 1 --2- 172.21.15.131:0/886224130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d24102d00 0x7f7d24198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.248+0000 7f7d28e00700 1 -- 172.21.15.131:0/886224130 >> 172.21.15.131:0/886224130 conn(0x7f7d240fd210 msgr2=0x7f7d24105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.248+0000 7f7d28e00700 1 -- 172.21.15.131:0/886224130 shutdown_connections 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.248+0000 7f7d28e00700 1 -- 172.21.15.131:0/886224130 wait complete. 2024-05-10T05:33:30.252 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-10T05:33:30.276 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-05-10T05:33:28.857234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663516,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1051,"num_read_kb":655,"num_write":430,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":854,"ondisk_log_size":854,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380284,"kb_used_data":11036,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124561540,"statfs":{"total":1151940427776,"available":1151551016960,"internally_reserved":0,"allocated":11300864,"data_stored":7130835,"data_compressed":113718,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19128,"internal_metadata":377992520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5965,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":69,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003157"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897159+0000","last_change":"2024-05-10T05:31:51.576054+0000","last_active":"2024-05-10T05:33:14.897159+0000","last_peered":"2024-05-10T05:33:14.897159+0000","last_clean":"2024-05-10T05:33:14.897159+0000","last_became_active":"2024-05-10T05:31:51.575186+0000","last_became_peered":"2024-05-10T05:31:51.575186+0000","last_unstale":"2024-05-10T05:33:14.897159+0000","last_undegraded":"2024-05-10T05:33:14.897159+0000","last_fullsized":"2024-05-10T05:33:14.897159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:16:27.654822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.898953+0000","last_change":"2024-05-10T05:31:51.576483+0000","last_active":"2024-05-10T05:33:14.898953+0000","last_peered":"2024-05-10T05:33:14.898953+0000","last_clean":"2024-05-10T05:33:14.898953+0000","last_became_active":"2024-05-10T05:31:51.574936+0000","last_became_peered":"2024-05-10T05:31:51.574936+0000","last_unstale":"2024-05-10T05:33:14.898953+0000","last_undegraded":"2024-05-10T05:33:14.898953+0000","last_fullsized":"2024-05-10T05:33:14.898953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:56:51.436880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902454+0000","last_change":"2024-05-10T05:31:51.567114+0000","last_active":"2024-05-10T05:33:14.902454+0000","last_peered":"2024-05-10T05:33:14.902454+0000","last_clean":"2024-05-10T05:33:14.902454+0000","last_became_active":"2024-05-10T05:31:51.564552+0000","last_became_peered":"2024-05-10T05:31:51.564552+0000","last_unstale":"2024-05-10T05:33:14.902454+0000","last_undegraded":"2024-05-10T05:33:14.902454+0000","last_fullsized":"2024-05-10T05:33:14.902454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:40.268764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.878967+0000","last_change":"2024-05-10T05:31:51.584471+0000","last_active":"2024-05-10T05:33:14.878967+0000","last_peered":"2024-05-10T05:33:14.878967+0000","last_clean":"2024-05-10T05:33:14.878967+0000","last_became_active":"2024-05-10T05:31:51.583749+0000","last_became_peered":"2024-05-10T05:31:51.583749+0000","last_unstale":"2024-05-10T05:33:14.878967+0000","last_undegraded":"2024-05-10T05:33:14.878967+0000","last_fullsized":"2024-05-10T05:33:14.878967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:30.775286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.894275+0000","last_change":"2024-05-10T05:31:51.577698+0000","last_active":"2024-05-10T05:33:14.894275+0000","last_peered":"2024-05-10T05:33:14.894275+0000","last_clean":"2024-05-10T05:33:14.894275+0000","last_became_active":"2024-05-10T05:31:51.571469+0000","last_became_peered":"2024-05-10T05:31:51.571469+0000","last_unstale":"2024-05-10T05:33:14.894275+0000","last_undegraded":"2024-05-10T05:33:14.894275+0000","last_fullsized":"2024-05-10T05:33:14.894275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:35.161267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.887837+0000","last_change":"2024-05-10T05:31:51.573560+0000","last_active":"2024-05-10T05:33:14.887837+0000","last_peered":"2024-05-10T05:33:14.887837+0000","last_clean":"2024-05-10T05:33:14.887837+0000","last_became_active":"2024-05-10T05:31:51.573227+0000","last_became_peered":"2024-05-10T05:31:51.573227+0000","last_unstale":"2024-05-10T05:33:14.887837+0000","last_undegraded":"2024-05-10T05:33:14.887837+0000","last_fullsized":"2024-05-10T05:33:14.887837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:23.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.891483+0000","last_change":"2024-05-10T05:31:51.573667+0000","last_active":"2024-05-10T05:33:14.891483+0000","last_peered":"2024-05-10T05:33:14.891483+0000","last_clean":"2024-05-10T05:33:14.891483+0000","last_became_active":"2024-05-10T05:31:51.573362+0000","last_became_peered":"2024-05-10T05:31:51.573362+0000","last_unstale":"2024-05-10T05:33:14.891483+0000","last_undegraded":"2024-05-10T05:33:14.891483+0000","last_fullsized":"2024-05-10T05:33:14.891483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:00.300134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.899011+0000","last_change":"2024-05-10T05:31:51.583596+0000","last_active":"2024-05-10T05:33:14.899011+0000","last_peered":"2024-05-10T05:33:14.899011+0000","last_clean":"2024-05-10T05:33:14.899011+0000","last_became_active":"2024-05-10T05:31:51.583235+0000","last_became_peered":"2024-05-10T05:31:51.583235+0000","last_unstale":"2024-05-10T05:33:14.899011+0000","last_undegraded":"2024-05-10T05:33:14.899011+0000","last_fullsized":"2024-05-10T05:33:14.899011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:05.575975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.882166+0000","last_change":"2024-05-10T05:31:51.567627+0000","last_active":"2024-05-10T05:33:14.882166+0000","last_peered":"2024-05-10T05:33:14.882166+0000","last_clean":"2024-05-10T05:33:14.882166+0000","last_became_active":"2024-05-10T05:31:51.567276+0000","last_became_peered":"2024-05-10T05:31:51.567276+0000","last_unstale":"2024-05-10T05:33:14.882166+0000","last_undegraded":"2024-05-10T05:33:14.882166+0000","last_fullsized":"2024-05-10T05:33:14.882166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:16:27.914802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888316+0000","last_change":"2024-05-10T05:31:51.573859+0000","last_active":"2024-05-10T05:33:14.888316+0000","last_peered":"2024-05-10T05:33:14.888316+0000","last_clean":"2024-05-10T05:33:14.888316+0000","last_became_active":"2024-05-10T05:31:51.573493+0000","last_became_peered":"2024-05-10T05:31:51.573493+0000","last_unstale":"2024-05-10T05:33:14.888316+0000","last_undegraded":"2024-05-10T05:33:14.888316+0000","last_fullsized":"2024-05-10T05:33:14.888316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:21.024888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.889191+0000","last_change":"2024-05-10T05:31:51.576056+0000","last_active":"2024-05-10T05:33:14.889191+0000","last_peered":"2024-05-10T05:33:14.889191+0000","last_clean":"2024-05-10T05:33:14.889191+0000","last_became_active":"2024-05-10T05:31:51.575146+0000","last_became_peered":"2024-05-10T05:31:51.575146+0000","last_unstale":"2024-05-10T05:33:14.889191+0000","last_undegraded":"2024-05-10T05:33:14.889191+0000","last_fullsized":"2024-05-10T05:33:14.889191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:41.201052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.885798+0000","last_change":"2024-05-10T05:31:51.584710+0000","last_active":"2024-05-10T05:33:14.885798+0000","last_peered":"2024-05-10T05:33:14.885798+0000","last_clean":"2024-05-10T05:33:14.885798+0000","last_became_active":"2024-05-10T05:31:51.584122+0000","last_became_peered":"2024-05-10T05:31:51.584122+0000","last_unstale":"2024-05-10T05:33:14.885798+0000","last_undegraded":"2024-05-10T05:33:14.885798+0000","last_fullsized":"2024-05-10T05:33:14.885798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:43.622981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.890824+0000","last_change":"2024-05-10T05:32:15.924380+0000","last_active":"2024-05-10T05:33:14.890824+0000","last_peered":"2024-05-10T05:33:14.890824+0000","last_clean":"2024-05-10T05:33:14.890824+0000","last_became_active":"2024-05-10T05:32:15.923952+0000","last_became_peered":"2024-05-10T05:32:15.923952+0000","last_unstale":"2024-05-10T05:33:14.890824+0000","last_undegraded":"2024-05-10T05:33:14.890824+0000","last_fullsized":"2024-05-10T05:33:14.890824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:04:14.087785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897069+0000","last_change":"2024-05-10T05:31:51.574628+0000","last_active":"2024-05-10T05:33:14.897069+0000","last_peered":"2024-05-10T05:33:14.897069+0000","last_clean":"2024-05-10T05:33:14.897069+0000","last_became_active":"2024-05-10T05:31:51.574256+0000","last_became_peered":"2024-05-10T05:31:51.574256+0000","last_unstale":"2024-05-10T05:33:14.897069+0000","last_undegraded":"2024-05-10T05:33:14.897069+0000","last_fullsized":"2024-05-10T05:33:14.897069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:59.174727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892288+0000","last_change":"2024-05-10T05:32:15.920243+0000","last_active":"2024-05-10T05:33:14.892288+0000","last_peered":"2024-05-10T05:33:14.892288+0000","last_clean":"2024-05-10T05:33:14.892288+0000","last_became_active":"2024-05-10T05:32:15.919822+0000","last_became_peered":"2024-05-10T05:32:15.919822+0000","last_unstale":"2024-05-10T05:33:14.892288+0000","last_undegraded":"2024-05-10T05:33:14.892288+0000","last_fullsized":"2024-05-10T05:33:14.892288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:59.392399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.900749+0000","last_change":"2024-05-10T05:31:51.568919+0000","last_active":"2024-05-10T05:33:14.900749+0000","last_peered":"2024-05-10T05:33:14.900749+0000","last_clean":"2024-05-10T05:33:14.900749+0000","last_became_active":"2024-05-10T05:31:51.568582+0000","last_became_peered":"2024-05-10T05:31:51.568582+0000","last_unstale":"2024-05-10T05:33:14.900749+0000","last_undegraded":"2024-05-10T05:33:14.900749+0000","last_fullsized":"2024-05-10T05:33:14.900749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:08.377261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897309+0000","last_change":"2024-05-10T05:31:51.579025+0000","last_active":"2024-05-10T05:33:14.897309+0000","last_peered":"2024-05-10T05:33:14.897309+0000","last_clean":"2024-05-10T05:33:14.897309+0000","last_became_active":"2024-05-10T05:31:51.578704+0000","last_became_peered":"2024-05-10T05:31:51.578704+0000","last_unstale":"2024-05-10T05:33:14.897309+0000","last_undegraded":"2024-05-10T05:33:14.897309+0000","last_fullsized":"2024-05-10T05:33:14.897309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:16.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.887421+0000","last_change":"2024-05-10T05:31:51.573193+0000","last_active":"2024-05-10T05:33:14.887421+0000","last_peered":"2024-05-10T05:33:14.887421+0000","last_clean":"2024-05-10T05:33:14.887421+0000","last_became_active":"2024-05-10T05:31:51.572874+0000","last_became_peered":"2024-05-10T05:31:51.572874+0000","last_unstale":"2024-05-10T05:33:14.887421+0000","last_undegraded":"2024-05-10T05:33:14.887421+0000","last_fullsized":"2024-05-10T05:33:14.887421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:09.867661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.882400+0000","last_change":"2024-05-10T05:31:51.581107+0000","last_active":"2024-05-10T05:33:14.882400+0000","last_peered":"2024-05-10T05:33:14.882400+0000","last_clean":"2024-05-10T05:33:14.882400+0000","last_became_active":"2024-05-10T05:31:51.580765+0000","last_became_peered":"2024-05-10T05:31:51.580765+0000","last_unstale":"2024-05-10T05:33:14.882400+0000","last_undegraded":"2024-05-10T05:33:14.882400+0000","last_fullsized":"2024-05-10T05:33:14.882400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:04.164486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":118,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:26.625187+0000","last_change":"2024-05-10T05:31:50.986534+0000","last_active":"2024-05-10T05:33:26.625187+0000","last_peered":"2024-05-10T05:33:26.625187+0000","last_clean":"2024-05-10T05:33:26.625187+0000","last_became_active":"2024-05-10T05:31:50.983493+0000","last_became_peered":"2024-05-10T05:31:50.983493+0000","last_unstale":"2024-05-10T05:33:26.625187+0000","last_undegraded":"2024-05-10T05:33:26.625187+0000","last_fullsized":"2024-05-10T05:33:26.625187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:18:56.926317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892547+0000","last_change":"2024-05-10T05:31:51.587121+0000","last_active":"2024-05-10T05:33:14.892547+0000","last_peered":"2024-05-10T05:33:14.892547+0000","last_clean":"2024-05-10T05:33:14.892547+0000","last_became_active":"2024-05-10T05:31:51.586641+0000","last_became_peered":"2024-05-10T05:31:51.586641+0000","last_unstale":"2024-05-10T05:33:14.892547+0000","last_undegraded":"2024-05-10T05:33:14.892547+0000","last_fullsized":"2024-05-10T05:33:14.892547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:52.654918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918650+0000","last_change":"2024-05-10T05:31:50.999755+0000","last_active":"2024-05-10T05:32:14.918650+0000","last_peered":"2024-05-10T05:32:14.918650+0000","last_clean":"2024-05-10T05:32:14.918650+0000","last_became_active":"2024-05-10T05:31:50.999336+0000","last_became_peered":"2024-05-10T05:31:50.999336+0000","last_unstale":"2024-05-10T05:32:14.918650+0000","last_undegraded":"2024-05-10T05:32:14.918650+0000","last_fullsized":"2024-05-10T05:32:14.918650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:43.755130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902595+0000","last_change":"2024-05-10T05:31:51.576938+0000","last_active":"2024-05-10T05:33:14.902595+0000","last_peered":"2024-05-10T05:33:14.902595+0000","last_clean":"2024-05-10T05:33:14.902595+0000","last_became_active":"2024-05-10T05:31:51.576517+0000","last_became_peered":"2024-05-10T05:31:51.576517+0000","last_unstale":"2024-05-10T05:33:14.902595+0000","last_undegraded":"2024-05-10T05:33:14.902595+0000","last_fullsized":"2024-05-10T05:33:14.902595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:52:38.709063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916132+0000","last_change":"2024-05-10T05:31:50.992925+0000","last_active":"2024-05-10T05:32:14.916132+0000","last_peered":"2024-05-10T05:32:14.916132+0000","last_clean":"2024-05-10T05:32:14.916132+0000","last_became_active":"2024-05-10T05:31:50.992574+0000","last_became_peered":"2024-05-10T05:31:50.992574+0000","last_unstale":"2024-05-10T05:32:14.916132+0000","last_undegraded":"2024-05-10T05:32:14.916132+0000","last_fullsized":"2024-05-10T05:32:14.916132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:14.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888117+0000","last_change":"2024-05-10T05:31:51.580145+0000","last_active":"2024-05-10T05:33:14.888117+0000","last_peered":"2024-05-10T05:33:14.888117+0000","last_clean":"2024-05-10T05:33:14.888117+0000","last_became_active":"2024-05-10T05:31:51.576701+0000","last_became_peered":"2024-05-10T05:31:51.576701+0000","last_unstale":"2024-05-10T05:33:14.888117+0000","last_undegraded":"2024-05-10T05:33:14.888117+0000","last_fullsized":"2024-05-10T05:33:14.888117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:36:21.075571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915647+0000","last_change":"2024-05-10T05:31:50.987863+0000","last_active":"2024-05-10T05:32:14.915647+0000","last_peered":"2024-05-10T05:32:14.915647+0000","last_clean":"2024-05-10T05:32:14.915647+0000","last_became_active":"2024-05-10T05:31:50.987463+0000","last_became_peered":"2024-05-10T05:31:50.987463+0000","last_unstale":"2024-05-10T05:32:14.915647+0000","last_undegraded":"2024-05-10T05:32:14.915647+0000","last_fullsized":"2024-05-10T05:32:14.915647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:12:49.542288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.895479+0000","last_change":"2024-05-10T05:32:16.105284+0000","last_active":"2024-05-10T05:33:14.895479+0000","last_peered":"2024-05-10T05:33:14.895479+0000","last_clean":"2024-05-10T05:33:14.895479+0000","last_became_active":"2024-05-10T05:32:16.104915+0000","last_became_peered":"2024-05-10T05:32:16.104915+0000","last_unstale":"2024-05-10T05:33:14.895479+0000","last_undegraded":"2024-05-10T05:33:14.895479+0000","last_fullsized":"2024-05-10T05:33:14.895479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:25.628498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":110,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:22.620331+0000","last_change":"2024-05-10T05:31:50.985879+0000","last_active":"2024-05-10T05:33:22.620331+0000","last_peered":"2024-05-10T05:33:22.620331+0000","last_clean":"2024-05-10T05:33:22.620331+0000","last_became_active":"2024-05-10T05:31:50.985476+0000","last_became_peered":"2024-05-10T05:31:50.985476+0000","last_unstale":"2024-05-10T05:33:22.620331+0000","last_undegraded":"2024-05-10T05:33:22.620331+0000","last_fullsized":"2024-05-10T05:33:22.620331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:11:27.737972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1108,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.875759+0000","last_change":"2024-05-10T05:31:51.570985+0000","last_active":"2024-05-10T05:33:14.875759+0000","last_peered":"2024-05-10T05:33:14.875759+0000","last_clean":"2024-05-10T05:33:14.875759+0000","last_became_active":"2024-05-10T05:31:51.570324+0000","last_became_peered":"2024-05-10T05:31:51.570324+0000","last_unstale":"2024-05-10T05:33:14.875759+0000","last_undegraded":"2024-05-10T05:33:14.875759+0000","last_fullsized":"2024-05-10T05:33:14.875759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:26.947574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.911748+0000","last_change":"2024-05-10T05:31:50.988483+0000","last_active":"2024-05-10T05:32:14.911748+0000","last_peered":"2024-05-10T05:32:14.911748+0000","last_clean":"2024-05-10T05:32:14.911748+0000","last_became_active":"2024-05-10T05:31:50.987897+0000","last_became_peered":"2024-05-10T05:31:50.987897+0000","last_unstale":"2024-05-10T05:32:14.911748+0000","last_undegraded":"2024-05-10T05:32:14.911748+0000","last_fullsized":"2024-05-10T05:32:14.911748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:30:23.474637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.875011+0000","last_change":"2024-05-10T05:32:15.922113+0000","last_active":"2024-05-10T05:33:14.875011+0000","last_peered":"2024-05-10T05:33:14.875011+0000","last_clean":"2024-05-10T05:33:14.875011+0000","last_became_active":"2024-05-10T05:32:15.921665+0000","last_became_peered":"2024-05-10T05:32:15.921665+0000","last_unstale":"2024-05-10T05:33:14.875011+0000","last_undegraded":"2024-05-10T05:33:14.875011+0000","last_fullsized":"2024-05-10T05:33:14.875011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:28.155216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6],"acting":[5,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.908880+0000","last_change":"2024-05-10T05:31:50.981877+0000","last_active":"2024-05-10T05:32:14.908880+0000","last_peered":"2024-05-10T05:32:14.908880+0000","last_clean":"2024-05-10T05:32:14.908880+0000","last_became_active":"2024-05-10T05:31:50.981527+0000","last_became_peered":"2024-05-10T05:31:50.981527+0000","last_unstale":"2024-05-10T05:32:14.908880+0000","last_undegraded":"2024-05-10T05:32:14.908880+0000","last_fullsized":"2024-05-10T05:32:14.908880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:40.735304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.898545+0000","last_change":"2024-05-10T05:31:51.581027+0000","last_active":"2024-05-10T05:33:14.898545+0000","last_peered":"2024-05-10T05:33:14.898545+0000","last_clean":"2024-05-10T05:33:14.898545+0000","last_became_active":"2024-05-10T05:31:51.580620+0000","last_became_peered":"2024-05-10T05:31:51.580620+0000","last_unstale":"2024-05-10T05:33:14.898545+0000","last_undegraded":"2024-05-10T05:33:14.898545+0000","last_fullsized":"2024-05-10T05:33:14.898545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:30:15.378179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913340+0000","last_change":"2024-05-10T05:31:50.987633+0000","last_active":"2024-05-10T05:32:14.913340+0000","last_peered":"2024-05-10T05:32:14.913340+0000","last_clean":"2024-05-10T05:32:14.913340+0000","last_became_active":"2024-05-10T05:31:50.986750+0000","last_became_peered":"2024-05-10T05:31:50.986750+0000","last_unstale":"2024-05-10T05:32:14.913340+0000","last_undegraded":"2024-05-10T05:32:14.913340+0000","last_fullsized":"2024-05-10T05:32:14.913340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:17:47.705415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892509+0000","last_change":"2024-05-10T05:31:51.589484+0000","last_active":"2024-05-10T05:33:14.892509+0000","last_peered":"2024-05-10T05:33:14.892509+0000","last_clean":"2024-05-10T05:33:14.892509+0000","last_became_active":"2024-05-10T05:31:51.588992+0000","last_became_peered":"2024-05-10T05:31:51.588992+0000","last_unstale":"2024-05-10T05:33:14.892509+0000","last_undegraded":"2024-05-10T05:33:14.892509+0000","last_fullsized":"2024-05-10T05:33:14.892509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:32.360333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":195,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:25.624293+0000","last_change":"2024-05-10T05:31:50.988945+0000","last_active":"2024-05-10T05:33:25.624293+0000","last_peered":"2024-05-10T05:33:25.624293+0000","last_clean":"2024-05-10T05:33:25.624293+0000","last_became_active":"2024-05-10T05:31:50.988566+0000","last_became_peered":"2024-05-10T05:31:50.988566+0000","last_unstale":"2024-05-10T05:33:25.624293+0000","last_undegraded":"2024-05-10T05:33:25.624293+0000","last_fullsized":"2024-05-10T05:33:25.624293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:06:43.225346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888657+0000","last_change":"2024-05-10T05:31:51.577949+0000","last_active":"2024-05-10T05:33:14.888657+0000","last_peered":"2024-05-10T05:33:14.888657+0000","last_clean":"2024-05-10T05:33:14.888657+0000","last_became_active":"2024-05-10T05:31:51.577011+0000","last_became_peered":"2024-05-10T05:31:51.577011+0000","last_unstale":"2024-05-10T05:33:14.888657+0000","last_undegraded":"2024-05-10T05:33:14.888657+0000","last_fullsized":"2024-05-10T05:33:14.888657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:09.988700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920114+0000","last_change":"2024-05-10T05:31:50.984888+0000","last_active":"2024-05-10T05:32:14.920114+0000","last_peered":"2024-05-10T05:32:14.920114+0000","last_clean":"2024-05-10T05:32:14.920114+0000","last_became_active":"2024-05-10T05:31:50.984267+0000","last_became_peered":"2024-05-10T05:31:50.984267+0000","last_unstale":"2024-05-10T05:32:14.920114+0000","last_undegraded":"2024-05-10T05:32:14.920114+0000","last_fullsized":"2024-05-10T05:32:14.920114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:44.797469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.900759+0000","last_change":"2024-05-10T05:31:51.576863+0000","last_active":"2024-05-10T05:33:14.900759+0000","last_peered":"2024-05-10T05:33:14.900759+0000","last_clean":"2024-05-10T05:33:14.900759+0000","last_became_active":"2024-05-10T05:31:51.576468+0000","last_became_peered":"2024-05-10T05:31:51.576468+0000","last_unstale":"2024-05-10T05:33:14.900759+0000","last_undegraded":"2024-05-10T05:33:14.900759+0000","last_fullsized":"2024-05-10T05:33:14.900759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:20.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913803+0000","last_change":"2024-05-10T05:31:51.001566+0000","last_active":"2024-05-10T05:32:14.913803+0000","last_peered":"2024-05-10T05:32:14.913803+0000","last_clean":"2024-05-10T05:32:14.913803+0000","last_became_active":"2024-05-10T05:31:51.001203+0000","last_became_peered":"2024-05-10T05:31:51.001203+0000","last_unstale":"2024-05-10T05:32:14.913803+0000","last_undegraded":"2024-05-10T05:32:14.913803+0000","last_fullsized":"2024-05-10T05:32:14.913803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:17:45.465039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.883700+0000","last_change":"2024-05-10T05:31:51.579114+0000","last_active":"2024-05-10T05:33:14.883700+0000","last_peered":"2024-05-10T05:33:14.883700+0000","last_clean":"2024-05-10T05:33:14.883700+0000","last_became_active":"2024-05-10T05:31:51.577507+0000","last_became_peered":"2024-05-10T05:31:51.577507+0000","last_unstale":"2024-05-10T05:33:14.883700+0000","last_undegraded":"2024-05-10T05:33:14.883700+0000","last_fullsized":"2024-05-10T05:33:14.883700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:43.777983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'35","reported_seq":235,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:25.624677+0000","last_change":"2024-05-10T05:31:50.983558+0000","last_active":"2024-05-10T05:33:25.624677+0000","last_peered":"2024-05-10T05:33:25.624677+0000","last_clean":"2024-05-10T05:33:25.624677+0000","last_became_active":"2024-05-10T05:31:50.983087+0000","last_became_peered":"2024-05-10T05:31:50.983087+0000","last_unstale":"2024-05-10T05:33:25.624677+0000","last_undegraded":"2024-05-10T05:33:25.624677+0000","last_fullsized":"2024-05-10T05:33:25.624677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:08.753898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65616,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":158,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.889800+0000","last_change":"2024-05-10T05:31:51.574024+0000","last_active":"2024-05-10T05:33:14.889800+0000","last_peered":"2024-05-10T05:33:14.889800+0000","last_clean":"2024-05-10T05:33:14.889800+0000","last_became_active":"2024-05-10T05:31:51.573703+0000","last_became_peered":"2024-05-10T05:31:51.573703+0000","last_unstale":"2024-05-10T05:33:14.889800+0000","last_undegraded":"2024-05-10T05:33:14.889800+0000","last_fullsized":"2024-05-10T05:33:14.889800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:55:23.435991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918608+0000","last_change":"2024-05-10T05:31:50.986795+0000","last_active":"2024-05-10T05:32:14.918608+0000","last_peered":"2024-05-10T05:32:14.918608+0000","last_clean":"2024-05-10T05:32:14.918608+0000","last_became_active":"2024-05-10T05:31:50.986428+0000","last_became_peered":"2024-05-10T05:31:50.986428+0000","last_unstale":"2024-05-10T05:32:14.918608+0000","last_undegraded":"2024-05-10T05:32:14.918608+0000","last_fullsized":"2024-05-10T05:32:14.918608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:11.039632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.894052+0000","last_change":"2024-05-10T05:31:51.585595+0000","last_active":"2024-05-10T05:33:14.894052+0000","last_peered":"2024-05-10T05:33:14.894052+0000","last_clean":"2024-05-10T05:33:14.894052+0000","last_became_active":"2024-05-10T05:31:51.585091+0000","last_became_peered":"2024-05-10T05:31:51.585091+0000","last_unstale":"2024-05-10T05:33:14.894052+0000","last_undegraded":"2024-05-10T05:33:14.894052+0000","last_fullsized":"2024-05-10T05:33:14.894052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:10:10.489430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913631+0000","last_change":"2024-05-10T05:31:50.984408+0000","last_active":"2024-05-10T05:32:14.913631+0000","last_peered":"2024-05-10T05:32:14.913631+0000","last_clean":"2024-05-10T05:32:14.913631+0000","last_became_active":"2024-05-10T05:31:50.983098+0000","last_became_peered":"2024-05-10T05:31:50.983098+0000","last_unstale":"2024-05-10T05:32:14.913631+0000","last_undegraded":"2024-05-10T05:32:14.913631+0000","last_fullsized":"2024-05-10T05:32:14.913631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:31.399649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.893997+0000","last_change":"2024-05-10T05:31:51.576160+0000","last_active":"2024-05-10T05:33:14.893997+0000","last_peered":"2024-05-10T05:33:14.893997+0000","last_clean":"2024-05-10T05:33:14.893997+0000","last_became_active":"2024-05-10T05:31:51.575806+0000","last_became_peered":"2024-05-10T05:31:51.575806+0000","last_unstale":"2024-05-10T05:33:14.893997+0000","last_undegraded":"2024-05-10T05:33:14.893997+0000","last_fullsized":"2024-05-10T05:33:14.893997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:53:25.660988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":114,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:26.625959+0000","last_change":"2024-05-10T05:31:50.982752+0000","last_active":"2024-05-10T05:33:26.625959+0000","last_peered":"2024-05-10T05:33:26.625959+0000","last_clean":"2024-05-10T05:33:26.625959+0000","last_became_active":"2024-05-10T05:31:50.982299+0000","last_became_peered":"2024-05-10T05:31:50.982299+0000","last_unstale":"2024-05-10T05:33:26.625959+0000","last_undegraded":"2024-05-10T05:33:26.625959+0000","last_fullsized":"2024-05-10T05:33:26.625959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:52:40.206967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892923+0000","last_change":"2024-05-10T05:31:51.577081+0000","last_active":"2024-05-10T05:33:14.892923+0000","last_peered":"2024-05-10T05:33:14.892923+0000","last_clean":"2024-05-10T05:33:14.892923+0000","last_became_active":"2024-05-10T05:31:51.576729+0000","last_became_peered":"2024-05-10T05:31:51.576729+0000","last_unstale":"2024-05-10T05:33:14.892923+0000","last_undegraded":"2024-05-10T05:33:14.892923+0000","last_fullsized":"2024-05-10T05:33:14.892923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:14.072427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915219+0000","last_change":"2024-05-10T05:31:50.991868+0000","last_active":"2024-05-10T05:32:14.915219+0000","last_peered":"2024-05-10T05:32:14.915219+0000","last_clean":"2024-05-10T05:32:14.915219+0000","last_became_active":"2024-05-10T05:31:50.991471+0000","last_became_peered":"2024-05-10T05:31:50.991471+0000","last_unstale":"2024-05-10T05:32:14.915219+0000","last_undegraded":"2024-05-10T05:32:14.915219+0000","last_fullsized":"2024-05-10T05:32:14.915219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:22.346376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.873664+0000","last_change":"2024-05-10T05:31:51.584726+0000","last_active":"2024-05-10T05:33:14.873664+0000","last_peered":"2024-05-10T05:33:14.873664+0000","last_clean":"2024-05-10T05:33:14.873664+0000","last_became_active":"2024-05-10T05:31:51.584327+0000","last_became_peered":"2024-05-10T05:31:51.584327+0000","last_unstale":"2024-05-10T05:33:14.873664+0000","last_undegraded":"2024-05-10T05:33:14.873664+0000","last_fullsized":"2024-05-10T05:33:14.873664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:56:43.561717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915510+0000","last_change":"2024-05-10T05:31:50.989071+0000","last_active":"2024-05-10T05:32:14.915510+0000","last_peered":"2024-05-10T05:32:14.915510+0000","last_clean":"2024-05-10T05:32:14.915510+0000","last_became_active":"2024-05-10T05:31:50.988044+0000","last_became_peered":"2024-05-10T05:31:50.988044+0000","last_unstale":"2024-05-10T05:32:14.915510+0000","last_undegraded":"2024-05-10T05:32:14.915510+0000","last_fullsized":"2024-05-10T05:32:14.915510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:24.122564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.884441+0000","last_change":"2024-05-10T05:31:51.576793+0000","last_active":"2024-05-10T05:33:14.884441+0000","last_peered":"2024-05-10T05:33:14.884441+0000","last_clean":"2024-05-10T05:33:14.884441+0000","last_became_active":"2024-05-10T05:31:51.576333+0000","last_became_peered":"2024-05-10T05:31:51.576333+0000","last_unstale":"2024-05-10T05:33:14.884441+0000","last_undegraded":"2024-05-10T05:33:14.884441+0000","last_fullsized":"2024-05-10T05:33:14.884441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:42.418736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913436+0000","last_change":"2024-05-10T05:31:50.997233+0000","last_active":"2024-05-10T05:32:14.913436+0000","last_peered":"2024-05-10T05:32:14.913436+0000","last_clean":"2024-05-10T05:32:14.913436+0000","last_became_active":"2024-05-10T05:31:50.996857+0000","last_became_peered":"2024-05-10T05:31:50.996857+0000","last_unstale":"2024-05-10T05:32:14.913436+0000","last_undegraded":"2024-05-10T05:32:14.913436+0000","last_fullsized":"2024-05-10T05:32:14.913436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:35.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.887621+0000","last_change":"2024-05-10T05:32:15.927682+0000","last_active":"2024-05-10T05:33:14.887621+0000","last_peered":"2024-05-10T05:33:14.887621+0000","last_clean":"2024-05-10T05:33:14.887621+0000","last_became_active":"2024-05-10T05:32:15.924197+0000","last_became_peered":"2024-05-10T05:32:15.924197+0000","last_unstale":"2024-05-10T05:33:14.887621+0000","last_undegraded":"2024-05-10T05:33:14.887621+0000","last_fullsized":"2024-05-10T05:33:14.887621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:14.747603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:24.622894+0000","last_change":"2024-05-10T05:31:50.991113+0000","last_active":"2024-05-10T05:33:24.622894+0000","last_peered":"2024-05-10T05:33:24.622894+0000","last_clean":"2024-05-10T05:33:24.622894+0000","last_became_active":"2024-05-10T05:31:50.990745+0000","last_became_peered":"2024-05-10T05:31:50.990745+0000","last_unstale":"2024-05-10T05:33:24.622894+0000","last_undegraded":"2024-05-10T05:33:24.622894+0000","last_fullsized":"2024-05-10T05:33:24.622894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:49:56.586231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897318+0000","last_change":"2024-05-10T05:31:51.582057+0000","last_active":"2024-05-10T05:33:14.897318+0000","last_peered":"2024-05-10T05:33:14.897318+0000","last_clean":"2024-05-10T05:33:14.897318+0000","last_became_active":"2024-05-10T05:31:51.581672+0000","last_became_peered":"2024-05-10T05:31:51.581672+0000","last_unstale":"2024-05-10T05:33:14.897318+0000","last_undegraded":"2024-05-10T05:33:14.897318+0000","last_fullsized":"2024-05-10T05:33:14.897318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:45:09.122256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.877986+0000","last_change":"2024-05-10T05:31:51.585659+0000","last_active":"2024-05-10T05:33:14.877986+0000","last_peered":"2024-05-10T05:33:14.877986+0000","last_clean":"2024-05-10T05:33:14.877986+0000","last_became_active":"2024-05-10T05:31:51.584540+0000","last_became_peered":"2024-05-10T05:31:51.584540+0000","last_unstale":"2024-05-10T05:33:14.877986+0000","last_undegraded":"2024-05-10T05:33:14.877986+0000","last_fullsized":"2024-05-10T05:33:14.877986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:20.195369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":111,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:26.625190+0000","last_change":"2024-05-10T05:31:50.993547+0000","last_active":"2024-05-10T05:33:26.625190+0000","last_peered":"2024-05-10T05:33:26.625190+0000","last_clean":"2024-05-10T05:33:26.625190+0000","last_became_active":"2024-05-10T05:31:50.993190+0000","last_became_peered":"2024-05-10T05:31:50.993190+0000","last_unstale":"2024-05-10T05:33:26.625190+0000","last_undegraded":"2024-05-10T05:33:26.625190+0000","last_fullsized":"2024-05-10T05:33:26.625190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:06.159572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.890795+0000","last_change":"2024-05-10T05:31:51.587678+0000","last_active":"2024-05-10T05:33:14.890795+0000","last_peered":"2024-05-10T05:33:14.890795+0000","last_clean":"2024-05-10T05:33:14.890795+0000","last_became_active":"2024-05-10T05:31:51.586999+0000","last_became_peered":"2024-05-10T05:31:51.586999+0000","last_unstale":"2024-05-10T05:33:14.890795+0000","last_undegraded":"2024-05-10T05:33:14.890795+0000","last_fullsized":"2024-05-10T05:33:14.890795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:18:00.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914737+0000","last_change":"2024-05-10T05:31:50.979131+0000","last_active":"2024-05-10T05:32:14.914737+0000","last_peered":"2024-05-10T05:32:14.914737+0000","last_clean":"2024-05-10T05:32:14.914737+0000","last_became_active":"2024-05-10T05:31:50.978715+0000","last_became_peered":"2024-05-10T05:31:50.978715+0000","last_unstale":"2024-05-10T05:32:14.914737+0000","last_undegraded":"2024-05-10T05:32:14.914737+0000","last_fullsized":"2024-05-10T05:32:14.914737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:29:11.953963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919607+0000","last_change":"2024-05-10T05:30:59.356688+0000","last_active":"2024-05-10T05:32:14.919607+0000","last_peered":"2024-05-10T05:32:14.919607+0000","last_clean":"2024-05-10T05:32:14.919607+0000","last_became_active":"2024-05-10T05:30:59.356313+0000","last_became_peered":"2024-05-10T05:30:59.356313+0000","last_unstale":"2024-05-10T05:32:14.919607+0000","last_undegraded":"2024-05-10T05:32:14.919607+0000","last_fullsized":"2024-05-10T05:32:14.919607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_clean_scrub_stamp":"2024-05-10T05:28:15.503591+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:43.011495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.879926+0000","last_change":"2024-05-10T05:31:51.580259+0000","last_active":"2024-05-10T05:33:14.879926+0000","last_peered":"2024-05-10T05:33:14.879926+0000","last_clean":"2024-05-10T05:33:14.879926+0000","last_became_active":"2024-05-10T05:31:51.578650+0000","last_became_peered":"2024-05-10T05:31:51.578650+0000","last_unstale":"2024-05-10T05:33:14.879926+0000","last_undegraded":"2024-05-10T05:33:14.879926+0000","last_fullsized":"2024-05-10T05:33:14.879926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:48.630183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919744+0000","last_change":"2024-05-10T05:31:50.987708+0000","last_active":"2024-05-10T05:32:14.919744+0000","last_peered":"2024-05-10T05:32:14.919744+0000","last_clean":"2024-05-10T05:32:14.919744+0000","last_became_active":"2024-05-10T05:31:50.987347+0000","last_became_peered":"2024-05-10T05:31:50.987347+0000","last_unstale":"2024-05-10T05:32:14.919744+0000","last_undegraded":"2024-05-10T05:32:14.919744+0000","last_fullsized":"2024-05-10T05:32:14.919744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:16:29.388690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915952+0000","last_change":"2024-05-10T05:31:50.999338+0000","last_active":"2024-05-10T05:32:14.915952+0000","last_peered":"2024-05-10T05:32:14.915952+0000","last_clean":"2024-05-10T05:32:14.915952+0000","last_became_active":"2024-05-10T05:31:50.999026+0000","last_became_peered":"2024-05-10T05:31:50.999026+0000","last_unstale":"2024-05-10T05:32:14.915952+0000","last_undegraded":"2024-05-10T05:32:14.915952+0000","last_fullsized":"2024-05-10T05:32:14.915952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:31.443409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.880920+0000","last_change":"2024-05-10T05:31:51.574201+0000","last_active":"2024-05-10T05:33:14.880920+0000","last_peered":"2024-05-10T05:33:14.880920+0000","last_clean":"2024-05-10T05:33:14.880920+0000","last_became_active":"2024-05-10T05:31:51.571180+0000","last_became_peered":"2024-05-10T05:31:51.571180+0000","last_unstale":"2024-05-10T05:33:14.880920+0000","last_undegraded":"2024-05-10T05:33:14.880920+0000","last_fullsized":"2024-05-10T05:33:14.880920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:33:16.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:15.336605+0000","last_change":"2024-05-10T05:31:50.985164+0000","last_active":"2024-05-10T05:33:15.336605+0000","last_peered":"2024-05-10T05:33:15.336605+0000","last_clean":"2024-05-10T05:33:15.336605+0000","last_became_active":"2024-05-10T05:31:50.984760+0000","last_became_peered":"2024-05-10T05:31:50.984760+0000","last_unstale":"2024-05-10T05:33:15.336605+0000","last_undegraded":"2024-05-10T05:33:15.336605+0000","last_fullsized":"2024-05-10T05:33:15.336605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:19.827407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.894754+0000","last_change":"2024-05-10T05:31:51.577445+0000","last_active":"2024-05-10T05:33:14.894754+0000","last_peered":"2024-05-10T05:33:14.894754+0000","last_clean":"2024-05-10T05:33:14.894754+0000","last_became_active":"2024-05-10T05:31:51.576671+0000","last_became_peered":"2024-05-10T05:31:51.576671+0000","last_unstale":"2024-05-10T05:33:14.894754+0000","last_undegraded":"2024-05-10T05:33:14.894754+0000","last_fullsized":"2024-05-10T05:33:14.894754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:38.774538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914419+0000","last_change":"2024-05-10T05:31:50.981495+0000","last_active":"2024-05-10T05:32:14.914419+0000","last_peered":"2024-05-10T05:32:14.914419+0000","last_clean":"2024-05-10T05:32:14.914419+0000","last_became_active":"2024-05-10T05:31:50.981127+0000","last_became_peered":"2024-05-10T05:31:50.981127+0000","last_unstale":"2024-05-10T05:32:14.914419+0000","last_undegraded":"2024-05-10T05:32:14.914419+0000","last_fullsized":"2024-05-10T05:32:14.914419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:22:32.257014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902371+0000","last_change":"2024-05-10T05:31:51.585926+0000","last_active":"2024-05-10T05:33:14.902371+0000","last_peered":"2024-05-10T05:33:14.902371+0000","last_clean":"2024-05-10T05:33:14.902371+0000","last_became_active":"2024-05-10T05:31:51.585576+0000","last_became_peered":"2024-05-10T05:31:51.585576+0000","last_unstale":"2024-05-10T05:33:14.902371+0000","last_undegraded":"2024-05-10T05:33:14.902371+0000","last_fullsized":"2024-05-10T05:33:14.902371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:22.344180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.926062+0000","last_change":"2024-05-10T05:32:15.926062+0000","last_active":"2024-05-10T05:32:15.926062+0000","last_peered":"2024-05-10T05:32:15.926062+0000","last_clean":"2024-05-10T05:32:15.926062+0000","last_became_active":"2024-05-10T05:32:15.923026+0000","last_became_peered":"2024-05-10T05:32:15.923026+0000","last_unstale":"2024-05-10T05:32:15.926062+0000","last_undegraded":"2024-05-10T05:32:15.926062+0000","last_fullsized":"2024-05-10T05:32:15.926062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:34.188115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.848777+0000","last_change":"2024-05-10T05:31:51.573279+0000","last_active":"2024-05-10T05:33:14.848777+0000","last_peered":"2024-05-10T05:33:14.848777+0000","last_clean":"2024-05-10T05:33:14.848777+0000","last_became_active":"2024-05-10T05:31:51.572916+0000","last_became_peered":"2024-05-10T05:31:51.572916+0000","last_unstale":"2024-05-10T05:33:14.848777+0000","last_undegraded":"2024-05-10T05:33:14.848777+0000","last_fullsized":"2024-05-10T05:33:14.848777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:32:39.802463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918101+0000","last_change":"2024-05-10T05:31:50.983771+0000","last_active":"2024-05-10T05:32:14.918101+0000","last_peered":"2024-05-10T05:32:14.918101+0000","last_clean":"2024-05-10T05:32:14.918101+0000","last_became_active":"2024-05-10T05:31:50.981849+0000","last_became_peered":"2024-05-10T05:31:50.981849+0000","last_unstale":"2024-05-10T05:32:14.918101+0000","last_undegraded":"2024-05-10T05:32:14.918101+0000","last_fullsized":"2024-05-10T05:32:14.918101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:15.990869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.878726+0000","last_change":"2024-05-10T05:31:51.568005+0000","last_active":"2024-05-10T05:33:14.878726+0000","last_peered":"2024-05-10T05:33:14.878726+0000","last_clean":"2024-05-10T05:33:14.878726+0000","last_became_active":"2024-05-10T05:31:51.567347+0000","last_became_peered":"2024-05-10T05:31:51.567347+0000","last_unstale":"2024-05-10T05:33:14.878726+0000","last_undegraded":"2024-05-10T05:33:14.878726+0000","last_fullsized":"2024-05-10T05:33:14.878726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:29.075816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.926267+0000","last_change":"2024-05-10T05:32:15.925869+0000","last_active":"2024-05-10T05:32:15.926267+0000","last_peered":"2024-05-10T05:32:15.926267+0000","last_clean":"2024-05-10T05:32:15.926267+0000","last_became_active":"2024-05-10T05:32:15.920591+0000","last_became_peered":"2024-05-10T05:32:15.920591+0000","last_unstale":"2024-05-10T05:32:15.926267+0000","last_undegraded":"2024-05-10T05:32:15.926267+0000","last_fullsized":"2024-05-10T05:32:15.926267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:13:20.398637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.901352+0000","last_change":"2024-05-10T05:31:51.588255+0000","last_active":"2024-05-10T05:33:14.901352+0000","last_peered":"2024-05-10T05:33:14.901352+0000","last_clean":"2024-05-10T05:33:14.901352+0000","last_became_active":"2024-05-10T05:31:51.587912+0000","last_became_peered":"2024-05-10T05:31:51.587912+0000","last_unstale":"2024-05-10T05:33:14.901352+0000","last_undegraded":"2024-05-10T05:33:14.901352+0000","last_fullsized":"2024-05-10T05:33:14.901352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:18.515739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916229+0000","last_change":"2024-05-10T05:31:50.992392+0000","last_active":"2024-05-10T05:32:14.916229+0000","last_peered":"2024-05-10T05:32:14.916229+0000","last_clean":"2024-05-10T05:32:14.916229+0000","last_became_active":"2024-05-10T05:31:50.992119+0000","last_became_peered":"2024-05-10T05:31:50.992119+0000","last_unstale":"2024-05-10T05:32:14.916229+0000","last_undegraded":"2024-05-10T05:32:14.916229+0000","last_fullsized":"2024-05-10T05:32:14.916229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:11.515013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.893189+0000","last_change":"2024-05-10T05:31:51.567780+0000","last_active":"2024-05-10T05:33:14.893189+0000","last_peered":"2024-05-10T05:33:14.893189+0000","last_clean":"2024-05-10T05:33:14.893189+0000","last_became_active":"2024-05-10T05:31:51.567455+0000","last_became_peered":"2024-05-10T05:31:51.567455+0000","last_unstale":"2024-05-10T05:33:14.893189+0000","last_undegraded":"2024-05-10T05:33:14.893189+0000","last_fullsized":"2024-05-10T05:33:14.893189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:47.108131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:23.621608+0000","last_change":"2024-05-10T05:31:50.997374+0000","last_active":"2024-05-10T05:33:23.621608+0000","last_peered":"2024-05-10T05:33:23.621608+0000","last_clean":"2024-05-10T05:33:23.621608+0000","last_became_active":"2024-05-10T05:31:50.997069+0000","last_became_peered":"2024-05-10T05:31:50.997069+0000","last_unstale":"2024-05-10T05:33:23.621608+0000","last_undegraded":"2024-05-10T05:33:23.621608+0000","last_fullsized":"2024-05-10T05:33:23.621608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:25:49.140006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.889341+0000","last_change":"2024-05-10T05:31:51.576345+0000","last_active":"2024-05-10T05:33:14.889341+0000","last_peered":"2024-05-10T05:33:14.889341+0000","last_clean":"2024-05-10T05:33:14.889341+0000","last_became_active":"2024-05-10T05:31:51.575704+0000","last_became_peered":"2024-05-10T05:31:51.575704+0000","last_unstale":"2024-05-10T05:33:14.889341+0000","last_undegraded":"2024-05-10T05:33:14.889341+0000","last_fullsized":"2024-05-10T05:33:14.889341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:15.146591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913524+0000","last_change":"2024-05-10T05:31:50.984108+0000","last_active":"2024-05-10T05:32:14.913524+0000","last_peered":"2024-05-10T05:32:14.913524+0000","last_clean":"2024-05-10T05:32:14.913524+0000","last_became_active":"2024-05-10T05:31:50.983728+0000","last_became_peered":"2024-05-10T05:31:50.983728+0000","last_unstale":"2024-05-10T05:32:14.913524+0000","last_undegraded":"2024-05-10T05:32:14.913524+0000","last_fullsized":"2024-05-10T05:32:14.913524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:23:20.096286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.895124+0000","last_change":"2024-05-10T05:31:51.572075+0000","last_active":"2024-05-10T05:33:14.895124+0000","last_peered":"2024-05-10T05:33:14.895124+0000","last_clean":"2024-05-10T05:33:14.895124+0000","last_became_active":"2024-05-10T05:31:51.571711+0000","last_became_peered":"2024-05-10T05:31:51.571711+0000","last_unstale":"2024-05-10T05:33:14.895124+0000","last_undegraded":"2024-05-10T05:33:14.895124+0000","last_fullsized":"2024-05-10T05:33:14.895124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:38:27.813001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.907701+0000","last_change":"2024-05-10T05:31:50.983985+0000","last_active":"2024-05-10T05:32:14.907701+0000","last_peered":"2024-05-10T05:32:14.907701+0000","last_clean":"2024-05-10T05:32:14.907701+0000","last_became_active":"2024-05-10T05:31:50.982931+0000","last_became_peered":"2024-05-10T05:31:50.982931+0000","last_unstale":"2024-05-10T05:32:14.907701+0000","last_undegraded":"2024-05-10T05:32:14.907701+0000","last_fullsized":"2024-05-10T05:32:14.907701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:35:17.158696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888805+0000","last_change":"2024-05-10T05:31:51.587250+0000","last_active":"2024-05-10T05:33:14.888805+0000","last_peered":"2024-05-10T05:33:14.888805+0000","last_clean":"2024-05-10T05:33:14.888805+0000","last_became_active":"2024-05-10T05:31:51.586925+0000","last_became_peered":"2024-05-10T05:31:51.586925+0000","last_unstale":"2024-05-10T05:33:14.888805+0000","last_undegraded":"2024-05-10T05:33:14.888805+0000","last_fullsized":"2024-05-10T05:33:14.888805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:40.585803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.879173+0000","last_change":"2024-05-10T05:31:51.567710+0000","last_active":"2024-05-10T05:33:14.879173+0000","last_peered":"2024-05-10T05:33:14.879173+0000","last_clean":"2024-05-10T05:33:14.879173+0000","last_became_active":"2024-05-10T05:31:51.566860+0000","last_became_peered":"2024-05-10T05:31:51.566860+0000","last_unstale":"2024-05-10T05:33:14.879173+0000","last_undegraded":"2024-05-10T05:33:14.879173+0000","last_fullsized":"2024-05-10T05:33:14.879173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:02:09.253345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.879510+0000","last_change":"2024-05-10T05:31:51.577458+0000","last_active":"2024-05-10T05:33:14.879510+0000","last_peered":"2024-05-10T05:33:14.879510+0000","last_clean":"2024-05-10T05:33:14.879510+0000","last_became_active":"2024-05-10T05:31:51.577123+0000","last_became_peered":"2024-05-10T05:31:51.577123+0000","last_unstale":"2024-05-10T05:33:14.879510+0000","last_undegraded":"2024-05-10T05:33:14.879510+0000","last_fullsized":"2024-05-10T05:33:14.879510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:51.791090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902636+0000","last_change":"2024-05-10T05:31:51.589361+0000","last_active":"2024-05-10T05:33:14.902636+0000","last_peered":"2024-05-10T05:33:14.902636+0000","last_clean":"2024-05-10T05:33:14.902636+0000","last_became_active":"2024-05-10T05:31:51.589049+0000","last_became_peered":"2024-05-10T05:31:51.589049+0000","last_unstale":"2024-05-10T05:33:14.902636+0000","last_undegraded":"2024-05-10T05:33:14.902636+0000","last_fullsized":"2024-05-10T05:33:14.902636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:13.380505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902342+0000","last_change":"2024-05-10T05:32:15.919725+0000","last_active":"2024-05-10T05:33:14.902342+0000","last_peered":"2024-05-10T05:33:14.902342+0000","last_clean":"2024-05-10T05:33:14.902342+0000","last_became_active":"2024-05-10T05:32:15.919342+0000","last_became_peered":"2024-05-10T05:32:15.919342+0000","last_unstale":"2024-05-10T05:33:14.902342+0000","last_undegraded":"2024-05-10T05:33:14.902342+0000","last_fullsized":"2024-05-10T05:33:14.902342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:33.785843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.891254+0000","last_change":"2024-05-10T05:31:51.579791+0000","last_active":"2024-05-10T05:33:14.891254+0000","last_peered":"2024-05-10T05:33:14.891254+0000","last_clean":"2024-05-10T05:33:14.891254+0000","last_became_active":"2024-05-10T05:31:51.577635+0000","last_became_peered":"2024-05-10T05:31:51.577635+0000","last_unstale":"2024-05-10T05:33:14.891254+0000","last_undegraded":"2024-05-10T05:33:14.891254+0000","last_fullsized":"2024-05-10T05:33:14.891254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:54.077805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.873675+0000","last_change":"2024-05-10T05:31:51.584601+0000","last_active":"2024-05-10T05:33:14.873675+0000","last_peered":"2024-05-10T05:33:14.873675+0000","last_clean":"2024-05-10T05:33:14.873675+0000","last_became_active":"2024-05-10T05:31:51.584172+0000","last_became_peered":"2024-05-10T05:31:51.584172+0000","last_unstale":"2024-05-10T05:33:14.873675+0000","last_undegraded":"2024-05-10T05:33:14.873675+0000","last_fullsized":"2024-05-10T05:33:14.873675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:49:44.987379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.884792+0000","last_change":"2024-05-10T05:31:51.573784+0000","last_active":"2024-05-10T05:33:14.884792+0000","last_peered":"2024-05-10T05:33:14.884792+0000","last_clean":"2024-05-10T05:33:14.884792+0000","last_became_active":"2024-05-10T05:31:51.573279+0000","last_became_peered":"2024-05-10T05:31:51.573279+0000","last_unstale":"2024-05-10T05:33:14.884792+0000","last_undegraded":"2024-05-10T05:33:14.884792+0000","last_fullsized":"2024-05-10T05:33:14.884792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:08.375964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.881529+0000","last_change":"2024-05-10T05:31:51.585577+0000","last_active":"2024-05-10T05:33:14.881529+0000","last_peered":"2024-05-10T05:33:14.881529+0000","last_clean":"2024-05-10T05:33:14.881529+0000","last_became_active":"2024-05-10T05:31:51.585222+0000","last_became_peered":"2024-05-10T05:31:51.585222+0000","last_unstale":"2024-05-10T05:33:14.881529+0000","last_undegraded":"2024-05-10T05:33:14.881529+0000","last_fullsized":"2024-05-10T05:33:14.881529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:47:27.069513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892962+0000","last_change":"2024-05-10T05:31:51.571405+0000","last_active":"2024-05-10T05:33:14.892962+0000","last_peered":"2024-05-10T05:33:14.892962+0000","last_clean":"2024-05-10T05:33:14.892962+0000","last_became_active":"2024-05-10T05:31:51.571125+0000","last_became_peered":"2024-05-10T05:31:51.571125+0000","last_unstale":"2024-05-10T05:33:14.892962+0000","last_undegraded":"2024-05-10T05:33:14.892962+0000","last_fullsized":"2024-05-10T05:33:14.892962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:49:29.069121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.895256+0000","last_change":"2024-05-10T05:31:51.578883+0000","last_active":"2024-05-10T05:33:14.895256+0000","last_peered":"2024-05-10T05:33:14.895256+0000","last_clean":"2024-05-10T05:33:14.895256+0000","last_became_active":"2024-05-10T05:31:51.578582+0000","last_became_peered":"2024-05-10T05:31:51.578582+0000","last_unstale":"2024-05-10T05:33:14.895256+0000","last_undegraded":"2024-05-10T05:33:14.895256+0000","last_fullsized":"2024-05-10T05:33:14.895256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:16.402282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.891358+0000","last_change":"2024-05-10T05:31:51.589269+0000","last_active":"2024-05-10T05:33:14.891358+0000","last_peered":"2024-05-10T05:33:14.891358+0000","last_clean":"2024-05-10T05:33:14.891358+0000","last_became_active":"2024-05-10T05:31:51.588875+0000","last_became_peered":"2024-05-10T05:31:51.588875+0000","last_unstale":"2024-05-10T05:33:14.891358+0000","last_undegraded":"2024-05-10T05:33:14.891358+0000","last_fullsized":"2024-05-10T05:33:14.891358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:12.536555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.901574+0000","last_change":"2024-05-10T05:31:51.567789+0000","last_active":"2024-05-10T05:33:14.901574+0000","last_peered":"2024-05-10T05:33:14.901574+0000","last_clean":"2024-05-10T05:33:14.901574+0000","last_became_active":"2024-05-10T05:31:51.566625+0000","last_became_peered":"2024-05-10T05:31:51.566625+0000","last_unstale":"2024-05-10T05:33:14.901574+0000","last_undegraded":"2024-05-10T05:33:14.901574+0000","last_fullsized":"2024-05-10T05:33:14.901574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:43.042557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.876173+0000","last_change":"2024-05-10T05:31:51.585838+0000","last_active":"2024-05-10T05:33:14.876173+0000","last_peered":"2024-05-10T05:33:14.876173+0000","last_clean":"2024-05-10T05:33:14.876173+0000","last_became_active":"2024-05-10T05:31:51.585473+0000","last_became_peered":"2024-05-10T05:31:51.585473+0000","last_unstale":"2024-05-10T05:33:14.876173+0000","last_undegraded":"2024-05-10T05:33:14.876173+0000","last_fullsized":"2024-05-10T05:33:14.876173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:11:11.282076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247530,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72914,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":925,"num_read_kb":546,"num_write":247,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":213369,"data_compressed":5520,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":429949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.52500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.84199999999999997}]},{"osd":2,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.67900000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.36299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70599999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.499},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56799999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97599999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.50600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88300000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.747}]},{"osd":5,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89600000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88300000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.64000000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.03,"5min":1.127,"15min":1.127},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76300000000000001,"15min":0.76800000000000002},"min":{"1min":0.61799999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98499999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.78000000000000003}]},{"osd":9,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84099999999999997,"15min":0.84499999999999997},"min":{"1min":0.68000000000000005,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96799999999999997,"5min":1.514,"15min":1.514},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.82599999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.004,"5min":1.216,"15min":1.216},"last":0.85799999999999998}]},{"osd":10,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.71399999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94199999999999995,"5min":1.254,"15min":1.254},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.86099999999999999,"15min":0.86299999999999999},"min":{"1min":0.69199999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93899999999999995,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.86899999999999999}]}]},{"osd":10,"up_from":68,"seq":292057776161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1087128,"data_compressed":37906,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64500000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.871,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81100000000000005,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.747}]},{"osd":3,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.63500000000000001,"15min":0.64800000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.70399999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.504,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86399999999999999,"5min":1.091,"15min":1.091},"last":0.83299999999999996}]},{"osd":4,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.48299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79400000000000004,"5min":1.036,"15min":1.036},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":1.173,"15min":1.173},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.99099999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.58199999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84499999999999997,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76900000000000002,"15min":0.78900000000000003},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78500000000000003,"5min":1.448,"15min":1.448},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.67700000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.91100000000000003,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":8,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81799999999999995,"15min":0.81999999999999995},"min":{"1min":0.70099999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.65300000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0740000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.88300000000000001,"15min":0.89100000000000001},"min":{"1min":0.67800000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0189999999999999,"5min":1.728,"15min":1.728},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.85099999999999998,"15min":0.86099999999999999},"min":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0289999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.878}]},{"osd":11,"last update":"Fri May 10 05:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.66000000000000003,"5min":0.628,"15min":0.628},"max":{"1min":0.94699999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.85999999999999999},"min":{"1min":0.72199999999999998,"5min":0.625,"15min":0.625},"max":{"1min":1.008,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.97299999999999998}]}]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":921600,"data_stored":430044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.629,"15min":0.64800000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.63}]},{"osd":1,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.02,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.75700000000000001}]},{"osd":2,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.871},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3759999999999999,"5min":1.488,"15min":1.488},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71999999999999997,"15min":0.73599999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86499999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1160000000000001,"5min":1.177,"15min":1.177},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.872,"15min":0.88800000000000001},"min":{"1min":0.53000000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.403,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.874,"15min":0.89400000000000002},"min":{"1min":0.58099999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0580000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.89900000000000002,"15min":0.91400000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1339999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.90500000000000003,"15min":0.92500000000000004},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.115,"5min":1.288,"15min":1.288},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.93500000000000005,"15min":0.95399999999999996},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.135,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4650000000000001,"5min":1.611,"15min":1.611},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.97899999999999998,"15min":1.0009999999999999},"min":{"1min":0.70599999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.196,"5min":1.46,"15min":1.46},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.2450000000000001,"5min":1.337,"15min":1.337},"last":0.91200000000000003}]},{"osd":8,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.998,"15min":1.008},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.427,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.997,"15min":1.0129999999999999},"min":{"1min":0.71599999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2170000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.78900000000000003}]},{"osd":10,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1,"15min":1.016},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.99399999999999999,"15min":1.014},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.032,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.85799999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":430305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.995,"15min":0.995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.65600000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.629,"15min":0.627},"min":{"1min":0.41299999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.626,"15min":0.627},"min":{"1min":0.52800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78700000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84399999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.876,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70299999999999996}]},{"osd":4,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.745},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.54000000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93700000000000006,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.90300000000000002}]},{"osd":5,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89100000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.42199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91900000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.63200000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98599999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.76900000000000002}]},{"osd":7,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.754,"15min":0.753},"min":{"1min":0.54800000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.997,"5min":1.101,"15min":1.101},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.64300000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.96999999999999997,"5min":1.702,"15min":1.702},"last":0.81200000000000006}]},{"osd":9,"last update":"Fri May 10 05:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.76400000000000001,"15min":0.74299999999999999},"min":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.77900000000000003,"15min":0.76600000000000001},"min":{"1min":0.65500000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri May 10 05:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.70199999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.876},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.78400000000000003,"15min":0.76800000000000002},"min":{"1min":0.73299999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.96299999999999997}]},{"osd":11,"last update":"Fri May 10 05:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.98599999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":966656,"data_stored":1023356,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.438,"15min":0.433},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.44500000000000001},"min":{"1min":0.32900000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.39300000000000002}]},{"osd":2,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77900000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.85899999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.49099999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.73399999999999999}]},{"osd":4,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.496,"15min":0.49099999999999999},"min":{"1min":0.374,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.75,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67900000000000005,"15min":0.66600000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96099999999999997,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.53700000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84299999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86499999999999999}]},{"osd":8,"last update":"Fri May 10 05:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.60399999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92100000000000004,"5min":1.002,"15min":1.002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.71199999999999997,"15min":0.70399999999999996},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.748}]},{"osd":9,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.626,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.022,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.65600000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.89200000000000002}]},{"osd":10,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.77800000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.96799999999999997,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.68300000000000005}]},{"osd":11,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.69999999999999996}]}]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":901120,"data_stored":429596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67900000000000005,"15min":0.65100000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.59799999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98299999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.91900000000000004}]},{"osd":2,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72399999999999998,"15min":0.70799999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69399999999999995,"15min":0.69499999999999995},"min":{"1min":0.442,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97499999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.65800000000000003},"min":{"1min":0.41599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.77200000000000002},"min":{"1min":0.56899999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.083,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76700000000000002,"15min":0.75700000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0740000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.83799999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.63}]},{"osd":7,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.82999999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.81200000000000006},"min":{"1min":0.60099999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.86799999999999999}]},{"osd":8,"last update":"Fri May 10 05:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.90400000000000003,"15min":0.94199999999999995},"min":{"1min":0.622,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.99399999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91400000000000003,"15min":0.95499999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.367,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.91700000000000004,"15min":0.92200000000000004},"min":{"1min":0.52500000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.97599999999999998,"15min":0.99099999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2270000000000001,"5min":1.52,"15min":1.52},"last":0.88500000000000001}]},{"osd":10,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.999,"15min":1.0129999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.0740000000000001,"5min":1.458,"15min":1.458},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.93600000000000005,"15min":0.94099999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.81599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":962560,"data_stored":496125,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.628,"15min":0.79000000000000004},"min":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82899999999999996,"5min":6.3810000000000002,"15min":6.3810000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri May 10 05:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.60399999999999998,"15min":0.56799999999999995},"min":{"1min":0.48699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.625,"15min":0.58299999999999996},"min":{"1min":0.61699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri May 10 05:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78700000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.47499999999999998,"15min":0.44},"min":{"1min":0.40000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.501,"15min":0.45400000000000001},"min":{"1min":0.36199999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri May 10 05:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.61299999999999999,"15min":0.55800000000000005},"min":{"1min":0.55400000000000005,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.66700000000000004,"15min":0.61799999999999999},"min":{"1min":0.629,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri May 10 05:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68400000000000005,"15min":0.66500000000000004},"min":{"1min":0.46999999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.67400000000000004},"min":{"1min":0.56100000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.62,"15min":0.58999999999999997},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.46500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81899999999999995}]},{"osd":8,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.73299999999999998,"15min":0.69499999999999995},"min":{"1min":0.66100000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.71799999999999997,"15min":0.67200000000000004},"min":{"1min":0.64300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.04,"5min":1.113,"15min":1.113},"last":0.94499999999999995}]},{"osd":9,"last update":"Fri May 10 05:32:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.79700000000000004,"15min":0.79100000000000004},"min":{"1min":0.59499999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.1379999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.89600000000000002}]},{"osd":10,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.63,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0780000000000001,"5min":1.121,"15min":1.121},"last":0.98799999999999999}]},{"osd":11,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.071}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":921600,"data_stored":430935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 05:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.68600000000000005,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67300000000000004,"15min":0.61899999999999999},"min":{"1min":0.49299999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.849,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69699999999999995,"15min":0.63200000000000001},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96199999999999997,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.63800000000000001},"min":{"1min":0.46800000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.218,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri May 10 05:32:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.177,"5min":3.177,"15min":3.177},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri May 10 05:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72899999999999998,"15min":0.67300000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9450000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.748},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.73899999999999999,"15min":0.67500000000000004},"min":{"1min":0.56999999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":1.026}]},{"osd":5,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73999999999999999,"15min":0.67000000000000004},"min":{"1min":0.53200000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.2789999999999999,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.78700000000000003,"15min":0.71199999999999997},"min":{"1min":0.496,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":6.069,"5min":6.069,"15min":6.069},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri May 10 05:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.80500000000000005,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":4.859,"5min":4.859,"15min":4.859},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.76700000000000002,"15min":0.73999999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.0740000000000001}]},{"osd":7,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90700000000000003}]},{"osd":8,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.86599999999999999,"15min":0.86799999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.123,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":1.123},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.254,"5min":4.6559999999999997,"15min":4.6559999999999997},"last":0.83899999999999997}]},{"osd":9,"last update":"Fri May 10 05:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0349999999999999,"15min":1.0289999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.173,"5min":5.173,"15min":5.173},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.96899999999999997,"15min":0.97599999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"last":1.1659999999999999}]},{"osd":10,"last update":"Fri May 10 05:33:00 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.96199999999999997,"15min":0.93899999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.04,"15min":1.012},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":6.1189999999999998,"5min":6.1189999999999998,"15min":6.1189999999999998},"last":0.93300000000000005}]},{"osd":11,"last update":"Fri May 10 05:33:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0449999999999999,"15min":1.0429999999999999},"min":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":6.0949999999999998,"5min":6.0949999999999998,"15min":6.0949999999999998},"last":1.208},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.99099999999999999,"15min":1.02},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0589999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.98799999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":978944,"data_stored":491458,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70599999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66300000000000003}]},{"osd":1,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.51700000000000002},"min":{"1min":0.41299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81499999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83999999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.439,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.623}]},{"osd":4,"last update":"Fri May 10 05:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.48999999999999999,"15min":0.44400000000000001},"min":{"1min":0.38200000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81499999999999995,"5min":1.026,"15min":1.026},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.47199999999999998},"min":{"1min":0.40100000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69399999999999995,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri May 10 05:32:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.64000000000000001},"min":{"1min":0.53100000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.66000000000000003},"min":{"1min":0.58099999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.745}]},{"osd":8,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.57099999999999995},"min":{"1min":0.46100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.97699999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.60699999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82399999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47199999999999998}]},{"osd":9,"last update":"Fri May 10 05:32:40 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.71399999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.76900000000000002,"15min":0.752},"min":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri May 10 05:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79100000000000004,"15min":0.78900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0189999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.65000000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92800000000000005,"5min":1.121,"15min":1.121},"last":0.747}]},{"osd":11,"last update":"Fri May 10 05:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79400000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":429352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.45400000000000001,"15min":0.41199999999999998},"min":{"1min":0.34899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.46600000000000003,"15min":0.41799999999999998},"min":{"1min":0.38100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67000000000000004,"15min":0.63300000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.113,"5min":1.161,"15min":1.161},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91400000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73499999999999999}]},{"osd":2,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65700000000000003,"15min":0.60499999999999998},"min":{"1min":0.44600000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67800000000000005,"15min":0.63800000000000001},"min":{"1min":0.51900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0780000000000001,"5min":1.206,"15min":1.206},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri May 10 05:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.67900000000000005,"15min":0.64100000000000001},"min":{"1min":0.60299999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70599999999999996,"15min":0.66900000000000004},"min":{"1min":0.498,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99299999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.755}]},{"osd":5,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66900000000000004,"15min":0.64100000000000001},"min":{"1min":0.50600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84599999999999997,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.032,"5min":0.79400000000000004,"15min":0.72699999999999998},"min":{"1min":0.49199999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":7.4160000000000004,"5min":7.4160000000000004,"15min":7.4160000000000004},"last":0.95699999999999996}]},{"osd":6,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71599999999999997,"15min":0.68700000000000006},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri May 10 05:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79100000000000004,"15min":0.77300000000000002},"min":{"1min":0.61399999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.84399999999999997,"15min":0.82799999999999996},"min":{"1min":0.64700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1579999999999999,"5min":1.216,"15min":1.216},"last":0.91700000000000004}]},{"osd":8,"last update":"Fri May 10 05:31:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.753}]},{"osd":9,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.95699999999999996,"15min":0.97399999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1919999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.879,"15min":0.89800000000000002},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.032,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.621}]},{"osd":10,"last update":"Fri May 10 05:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.98999999999999999,"15min":1.0089999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.226,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.88300000000000001}]},{"osd":11,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.82999999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":901120,"data_stored":429991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri May 10 05:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.37},"min":{"1min":0.34499999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.54500000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.441,"15min":0.40300000000000002},"min":{"1min":0.36499999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.629}]},{"osd":2,"last update":"Fri May 10 05:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.45500000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.53100000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82099999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri May 10 05:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.83699999999999997,"15min":0.65900000000000003},"min":{"1min":0.55100000000000005,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97599999999999998,"5min":13.608000000000001,"15min":13.608000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.52000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.83399999999999996,"5min":0.876,"15min":0.876},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri May 10 05:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.65800000000000003,"15min":0.62},"min":{"1min":0.57499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89100000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.61199999999999999},"min":{"1min":0.55300000000000005,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83199999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82799999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.69699999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.754}]},{"osd":7,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61599999999999999,"15min":0.60099999999999998},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76200000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.878},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67900000000000005,"15min":0.66400000000000003},"min":{"1min":0.51600000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86599999999999999,"5min":1.081,"15min":1.081},"last":0.94699999999999995}]},{"osd":8,"last update":"Fri May 10 05:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96599999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78400000000000003,"15min":0.77200000000000002},"min":{"1min":0.61099999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":1.0640000000000001}]},{"osd":9,"last update":"Fri May 10 05:32:41 2024","interfaces":[{"interface":"back","average":{"1min":1.8220000000000001,"5min":1.048,"15min":0.91900000000000004},"min":{"1min":0.65700000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":13.648,"5min":13.648,"15min":13.648},"last":1.042},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.93500000000000005}]},{"osd":10,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.65400000000000003,"5min":0.621,"15min":0.621},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.038},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.874,"15min":0.88},"min":{"1min":0.77200000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":0.92000000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":1.0089999999999999}]},{"osd":11,"last update":"Fri May 10 05:32:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.997},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.0880000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":978944,"data_stored":1022596,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.624,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.751,"5min":4.524,"15min":4.524},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.57999999999999996},"min":{"1min":0.45000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.53400000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.49299999999999999}]},{"osd":2,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.35099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71699999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.68899999999999995}]},{"osd":3,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.66800000000000004,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.3860000000000001,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81100000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61099999999999999,"15min":0.59099999999999997},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86299999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.64100000000000001,"15min":0.62},"min":{"1min":0.435,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90600000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.53000000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91600000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.751},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.53400000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88100000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri May 10 05:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.45800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86199999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70599999999999996,"15min":0.69099999999999995},"min":{"1min":0.51600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97299999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri May 10 05:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.83099999999999996}]},{"osd":9,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.751,"15min":0.751},"min":{"1min":0.57699999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93899999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.875},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.80700000000000005,"15min":0.80900000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.98499999999999999,"5min":1.03,"15min":1.03},"last":0.84399999999999997}]},{"osd":10,"last update":"Fri May 10 05:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.746,"15min":0.73599999999999999},"min":{"1min":0.61799999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93300000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0009999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67320,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62653,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":67838,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T05:33:30.283 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T05:33:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:29 smithi157 ceph-mon[78427]: pgmap v289: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-10T05:33:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:29 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1677322719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:29 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1799367146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:30.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:29 smithi142 ceph-mon[77667]: pgmap v289: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-10T05:33:30.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:29 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1677322719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:30.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:29 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1799367146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:30.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.580+0000 7f05bfe00700 1 -- 172.21.15.131:0/2139712314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b8102d00 msgr2=0x7f05b8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:30.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.580+0000 7f05bfe00700 1 --2- 172.21.15.131:0/2139712314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b8102d00 0x7f05b8103110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009990 tx=0x7f05a4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.580+0000 7f05bfe00700 1 -- 172.21.15.131:0/2139712314 shutdown_connections 2024-05-10T05:33:30.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.580+0000 7f05bfe00700 1 --2- 172.21.15.131:0/2139712314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b81036a0 0x7f05b8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.580+0000 7f05bfe00700 1 --2- 172.21.15.131:0/2139712314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b8101be0 0x7f05b8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.580+0000 7f05bfe00700 1 --2- 172.21.15.131:0/2139712314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b8102d00 0x7f05b8103110 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.580+0000 7f05bfe00700 1 -- 172.21.15.131:0/2139712314 >> 172.21.15.131:0/2139712314 conn(0x7f05b80fd210 msgr2=0x7f05b80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.580+0000 7f05bfe00700 1 -- 172.21.15.131:0/2139712314 shutdown_connections 2024-05-10T05:33:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.580+0000 7f05bfe00700 1 -- 172.21.15.131:0/2139712314 wait complete. 2024-05-10T05:33:30.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.581+0000 7f05bfe00700 1 Processor -- start 2024-05-10T05:33:30.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.581+0000 7f05bfe00700 1 -- start start 2024-05-10T05:33:30.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b8101be0 0x7f05b819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:30.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b8102d00 0x7f05b819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:30.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b81036a0 0x7f05b81a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:30.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bfe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81a1c50 con 0x7f05b81036a0 2024-05-10T05:33:30.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bfe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81a1dc0 con 0x7f05b8101be0 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bfe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81a1f80 con 0x7f05b8102d00 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b81036a0 0x7f05b81a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b81036a0 0x7f05b81a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33396/0 (socket says 172.21.15.131:33396) 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bde00700 1 -- 172.21.15.131:0/242647536 learned_addr learned my addr 172.21.15.131:0/242647536 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bd400700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b8101be0 0x7f05b819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bca00700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b8102d00 0x7f05b819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bde00700 1 -- 172.21.15.131:0/242647536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b8102d00 msgr2=0x7f05b819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bde00700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b8102d00 0x7f05b819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bde00700 1 -- 172.21.15.131:0/242647536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b8101be0 msgr2=0x7f05b819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.582+0000 7f05bde00700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b8101be0 0x7f05b819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.583+0000 7f05bde00700 1 -- 172.21.15.131:0/242647536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a40095f0 con 0x7f05b81036a0 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.583+0000 7f05bde00700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b81036a0 0x7f05b81a1640 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f05b000ef30 tx=0x7f05b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.583+0000 7f05b6000700 1 -- 172.21.15.131:0/242647536 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0018070 con 0x7f05b81036a0 2024-05-10T05:33:30.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.583+0000 7f05b6000700 1 -- 172.21.15.131:0/242647536 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b000f040 con 0x7f05b81036a0 2024-05-10T05:33:30.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.583+0000 7f05bfe00700 1 -- 172.21.15.131:0/242647536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b81a2200 con 0x7f05b81036a0 2024-05-10T05:33:30.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.583+0000 7f05b6000700 1 -- 172.21.15.131:0/242647536 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000cd00 con 0x7f05b81036a0 2024-05-10T05:33:30.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.583+0000 7f05bfe00700 1 -- 172.21.15.131:0/242647536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b81a2710 con 0x7f05b81036a0 2024-05-10T05:33:30.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.584+0000 7f05bfe00700 1 -- 172.21.15.131:0/242647536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b80694d0 con 0x7f05b81036a0 2024-05-10T05:33:30.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.586+0000 7f05b6000700 1 -- 172.21.15.131:0/242647536 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f05b0003a00 con 0x7f05b81036a0 2024-05-10T05:33:30.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.587+0000 7f05b6000700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05a00a11a0 0x7f05a00a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:30.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.588+0000 7f05b6000700 1 -- 172.21.15.131:0/242647536 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f05b00ccd70 con 0x7f05b81036a0 2024-05-10T05:33:30.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.588+0000 7f05bd400700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05a00a11a0 0x7f05a00a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:30.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.588+0000 7f05bd400700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05a00a11a0 0x7f05a00a36b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f05b8102c00 tx=0x7f05ac009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:30.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.590+0000 7f05b6000700 1 -- 172.21.15.131:0/242647536 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f05b0096530 con 0x7f05b81036a0 2024-05-10T05:33:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.796+0000 7f05bfe00700 1 -- 172.21.15.131:0/242647536 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05b8074b60 con 0x7f05a00a11a0 2024-05-10T05:33:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.806+0000 7f05b6000700 1 -- 172.21.15.131:0/242647536 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344599 (secure 0 0 0) 0x7f05b8074b60 con 0x7f05a00a11a0 2024-05-10T05:33:30.812 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.811+0000 7f05bfe00700 1 -- 172.21.15.131:0/242647536 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05a00a11a0 msgr2=0x7f05a00a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.811+0000 7f05bfe00700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05a00a11a0 0x7f05a00a36b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f05b8102c00 tx=0x7f05ac009260 comp rx=0 tx=0).stop 2024-05-10T05:33:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.811+0000 7f05bfe00700 1 -- 172.21.15.131:0/242647536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b81036a0 msgr2=0x7f05b81a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.811+0000 7f05bfe00700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b81036a0 0x7f05b81a1640 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f05b000ef30 tx=0x7f05b000c3a0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.811+0000 7f05bfe00700 1 -- 172.21.15.131:0/242647536 shutdown_connections 2024-05-10T05:33:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.811+0000 7f05bfe00700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05a00a11a0 0x7f05a00a36b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.811+0000 7f05bfe00700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b81036a0 0x7f05b81a1640 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.811+0000 7f05bfe00700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b8101be0 0x7f05b819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.811+0000 7f05bfe00700 1 --2- 172.21.15.131:0/242647536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b8102d00 0x7f05b819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.811+0000 7f05bfe00700 1 -- 172.21.15.131:0/242647536 >> 172.21.15.131:0/242647536 conn(0x7f05b80fd210 msgr2=0x7f05b8073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.812+0000 7f05bfe00700 1 -- 172.21.15.131:0/242647536 shutdown_connections 2024-05-10T05:33:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:30.812+0000 7f05bfe00700 1 -- 172.21.15.131:0/242647536 wait complete. 2024-05-10T05:33:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-10T05:33:30.838 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-05-10T05:33:28.857234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663516,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1051,"num_read_kb":655,"num_write":430,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":854,"ondisk_log_size":854,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380284,"kb_used_data":11036,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124561540,"statfs":{"total":1151940427776,"available":1151551016960,"internally_reserved":0,"allocated":11300864,"data_stored":7130835,"data_compressed":113718,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19128,"internal_metadata":377992520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5965,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":69,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003157"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897159+0000","last_change":"2024-05-10T05:31:51.576054+0000","last_active":"2024-05-10T05:33:14.897159+0000","last_peered":"2024-05-10T05:33:14.897159+0000","last_clean":"2024-05-10T05:33:14.897159+0000","last_became_active":"2024-05-10T05:31:51.575186+0000","last_became_peered":"2024-05-10T05:31:51.575186+0000","last_unstale":"2024-05-10T05:33:14.897159+0000","last_undegraded":"2024-05-10T05:33:14.897159+0000","last_fullsized":"2024-05-10T05:33:14.897159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:16:27.654822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.898953+0000","last_change":"2024-05-10T05:31:51.576483+0000","last_active":"2024-05-10T05:33:14.898953+0000","last_peered":"2024-05-10T05:33:14.898953+0000","last_clean":"2024-05-10T05:33:14.898953+0000","last_became_active":"2024-05-10T05:31:51.574936+0000","last_became_peered":"2024-05-10T05:31:51.574936+0000","last_unstale":"2024-05-10T05:33:14.898953+0000","last_undegraded":"2024-05-10T05:33:14.898953+0000","last_fullsized":"2024-05-10T05:33:14.898953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:56:51.436880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902454+0000","last_change":"2024-05-10T05:31:51.567114+0000","last_active":"2024-05-10T05:33:14.902454+0000","last_peered":"2024-05-10T05:33:14.902454+0000","last_clean":"2024-05-10T05:33:14.902454+0000","last_became_active":"2024-05-10T05:31:51.564552+0000","last_became_peered":"2024-05-10T05:31:51.564552+0000","last_unstale":"2024-05-10T05:33:14.902454+0000","last_undegraded":"2024-05-10T05:33:14.902454+0000","last_fullsized":"2024-05-10T05:33:14.902454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:40.268764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.878967+0000","last_change":"2024-05-10T05:31:51.584471+0000","last_active":"2024-05-10T05:33:14.878967+0000","last_peered":"2024-05-10T05:33:14.878967+0000","last_clean":"2024-05-10T05:33:14.878967+0000","last_became_active":"2024-05-10T05:31:51.583749+0000","last_became_peered":"2024-05-10T05:31:51.583749+0000","last_unstale":"2024-05-10T05:33:14.878967+0000","last_undegraded":"2024-05-10T05:33:14.878967+0000","last_fullsized":"2024-05-10T05:33:14.878967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:30.775286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.894275+0000","last_change":"2024-05-10T05:31:51.577698+0000","last_active":"2024-05-10T05:33:14.894275+0000","last_peered":"2024-05-10T05:33:14.894275+0000","last_clean":"2024-05-10T05:33:14.894275+0000","last_became_active":"2024-05-10T05:31:51.571469+0000","last_became_peered":"2024-05-10T05:31:51.571469+0000","last_unstale":"2024-05-10T05:33:14.894275+0000","last_undegraded":"2024-05-10T05:33:14.894275+0000","last_fullsized":"2024-05-10T05:33:14.894275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:35.161267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.887837+0000","last_change":"2024-05-10T05:31:51.573560+0000","last_active":"2024-05-10T05:33:14.887837+0000","last_peered":"2024-05-10T05:33:14.887837+0000","last_clean":"2024-05-10T05:33:14.887837+0000","last_became_active":"2024-05-10T05:31:51.573227+0000","last_became_peered":"2024-05-10T05:31:51.573227+0000","last_unstale":"2024-05-10T05:33:14.887837+0000","last_undegraded":"2024-05-10T05:33:14.887837+0000","last_fullsized":"2024-05-10T05:33:14.887837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:23.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.891483+0000","last_change":"2024-05-10T05:31:51.573667+0000","last_active":"2024-05-10T05:33:14.891483+0000","last_peered":"2024-05-10T05:33:14.891483+0000","last_clean":"2024-05-10T05:33:14.891483+0000","last_became_active":"2024-05-10T05:31:51.573362+0000","last_became_peered":"2024-05-10T05:31:51.573362+0000","last_unstale":"2024-05-10T05:33:14.891483+0000","last_undegraded":"2024-05-10T05:33:14.891483+0000","last_fullsized":"2024-05-10T05:33:14.891483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:00.300134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.899011+0000","last_change":"2024-05-10T05:31:51.583596+0000","last_active":"2024-05-10T05:33:14.899011+0000","last_peered":"2024-05-10T05:33:14.899011+0000","last_clean":"2024-05-10T05:33:14.899011+0000","last_became_active":"2024-05-10T05:31:51.583235+0000","last_became_peered":"2024-05-10T05:31:51.583235+0000","last_unstale":"2024-05-10T05:33:14.899011+0000","last_undegraded":"2024-05-10T05:33:14.899011+0000","last_fullsized":"2024-05-10T05:33:14.899011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:05.575975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.882166+0000","last_change":"2024-05-10T05:31:51.567627+0000","last_active":"2024-05-10T05:33:14.882166+0000","last_peered":"2024-05-10T05:33:14.882166+0000","last_clean":"2024-05-10T05:33:14.882166+0000","last_became_active":"2024-05-10T05:31:51.567276+0000","last_became_peered":"2024-05-10T05:31:51.567276+0000","last_unstale":"2024-05-10T05:33:14.882166+0000","last_undegraded":"2024-05-10T05:33:14.882166+0000","last_fullsized":"2024-05-10T05:33:14.882166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:16:27.914802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888316+0000","last_change":"2024-05-10T05:31:51.573859+0000","last_active":"2024-05-10T05:33:14.888316+0000","last_peered":"2024-05-10T05:33:14.888316+0000","last_clean":"2024-05-10T05:33:14.888316+0000","last_became_active":"2024-05-10T05:31:51.573493+0000","last_became_peered":"2024-05-10T05:31:51.573493+0000","last_unstale":"2024-05-10T05:33:14.888316+0000","last_undegraded":"2024-05-10T05:33:14.888316+0000","last_fullsized":"2024-05-10T05:33:14.888316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:21.024888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.889191+0000","last_change":"2024-05-10T05:31:51.576056+0000","last_active":"2024-05-10T05:33:14.889191+0000","last_peered":"2024-05-10T05:33:14.889191+0000","last_clean":"2024-05-10T05:33:14.889191+0000","last_became_active":"2024-05-10T05:31:51.575146+0000","last_became_peered":"2024-05-10T05:31:51.575146+0000","last_unstale":"2024-05-10T05:33:14.889191+0000","last_undegraded":"2024-05-10T05:33:14.889191+0000","last_fullsized":"2024-05-10T05:33:14.889191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:41.201052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.885798+0000","last_change":"2024-05-10T05:31:51.584710+0000","last_active":"2024-05-10T05:33:14.885798+0000","last_peered":"2024-05-10T05:33:14.885798+0000","last_clean":"2024-05-10T05:33:14.885798+0000","last_became_active":"2024-05-10T05:31:51.584122+0000","last_became_peered":"2024-05-10T05:31:51.584122+0000","last_unstale":"2024-05-10T05:33:14.885798+0000","last_undegraded":"2024-05-10T05:33:14.885798+0000","last_fullsized":"2024-05-10T05:33:14.885798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:43.622981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.890824+0000","last_change":"2024-05-10T05:32:15.924380+0000","last_active":"2024-05-10T05:33:14.890824+0000","last_peered":"2024-05-10T05:33:14.890824+0000","last_clean":"2024-05-10T05:33:14.890824+0000","last_became_active":"2024-05-10T05:32:15.923952+0000","last_became_peered":"2024-05-10T05:32:15.923952+0000","last_unstale":"2024-05-10T05:33:14.890824+0000","last_undegraded":"2024-05-10T05:33:14.890824+0000","last_fullsized":"2024-05-10T05:33:14.890824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:04:14.087785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897069+0000","last_change":"2024-05-10T05:31:51.574628+0000","last_active":"2024-05-10T05:33:14.897069+0000","last_peered":"2024-05-10T05:33:14.897069+0000","last_clean":"2024-05-10T05:33:14.897069+0000","last_became_active":"2024-05-10T05:31:51.574256+0000","last_became_peered":"2024-05-10T05:31:51.574256+0000","last_unstale":"2024-05-10T05:33:14.897069+0000","last_undegraded":"2024-05-10T05:33:14.897069+0000","last_fullsized":"2024-05-10T05:33:14.897069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:59.174727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892288+0000","last_change":"2024-05-10T05:32:15.920243+0000","last_active":"2024-05-10T05:33:14.892288+0000","last_peered":"2024-05-10T05:33:14.892288+0000","last_clean":"2024-05-10T05:33:14.892288+0000","last_became_active":"2024-05-10T05:32:15.919822+0000","last_became_peered":"2024-05-10T05:32:15.919822+0000","last_unstale":"2024-05-10T05:33:14.892288+0000","last_undegraded":"2024-05-10T05:33:14.892288+0000","last_fullsized":"2024-05-10T05:33:14.892288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:59.392399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.900749+0000","last_change":"2024-05-10T05:31:51.568919+0000","last_active":"2024-05-10T05:33:14.900749+0000","last_peered":"2024-05-10T05:33:14.900749+0000","last_clean":"2024-05-10T05:33:14.900749+0000","last_became_active":"2024-05-10T05:31:51.568582+0000","last_became_peered":"2024-05-10T05:31:51.568582+0000","last_unstale":"2024-05-10T05:33:14.900749+0000","last_undegraded":"2024-05-10T05:33:14.900749+0000","last_fullsized":"2024-05-10T05:33:14.900749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:08.377261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897309+0000","last_change":"2024-05-10T05:31:51.579025+0000","last_active":"2024-05-10T05:33:14.897309+0000","last_peered":"2024-05-10T05:33:14.897309+0000","last_clean":"2024-05-10T05:33:14.897309+0000","last_became_active":"2024-05-10T05:31:51.578704+0000","last_became_peered":"2024-05-10T05:31:51.578704+0000","last_unstale":"2024-05-10T05:33:14.897309+0000","last_undegraded":"2024-05-10T05:33:14.897309+0000","last_fullsized":"2024-05-10T05:33:14.897309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:16.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.887421+0000","last_change":"2024-05-10T05:31:51.573193+0000","last_active":"2024-05-10T05:33:14.887421+0000","last_peered":"2024-05-10T05:33:14.887421+0000","last_clean":"2024-05-10T05:33:14.887421+0000","last_became_active":"2024-05-10T05:31:51.572874+0000","last_became_peered":"2024-05-10T05:31:51.572874+0000","last_unstale":"2024-05-10T05:33:14.887421+0000","last_undegraded":"2024-05-10T05:33:14.887421+0000","last_fullsized":"2024-05-10T05:33:14.887421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:09.867661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.882400+0000","last_change":"2024-05-10T05:31:51.581107+0000","last_active":"2024-05-10T05:33:14.882400+0000","last_peered":"2024-05-10T05:33:14.882400+0000","last_clean":"2024-05-10T05:33:14.882400+0000","last_became_active":"2024-05-10T05:31:51.580765+0000","last_became_peered":"2024-05-10T05:31:51.580765+0000","last_unstale":"2024-05-10T05:33:14.882400+0000","last_undegraded":"2024-05-10T05:33:14.882400+0000","last_fullsized":"2024-05-10T05:33:14.882400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:04.164486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":118,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:26.625187+0000","last_change":"2024-05-10T05:31:50.986534+0000","last_active":"2024-05-10T05:33:26.625187+0000","last_peered":"2024-05-10T05:33:26.625187+0000","last_clean":"2024-05-10T05:33:26.625187+0000","last_became_active":"2024-05-10T05:31:50.983493+0000","last_became_peered":"2024-05-10T05:31:50.983493+0000","last_unstale":"2024-05-10T05:33:26.625187+0000","last_undegraded":"2024-05-10T05:33:26.625187+0000","last_fullsized":"2024-05-10T05:33:26.625187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:18:56.926317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892547+0000","last_change":"2024-05-10T05:31:51.587121+0000","last_active":"2024-05-10T05:33:14.892547+0000","last_peered":"2024-05-10T05:33:14.892547+0000","last_clean":"2024-05-10T05:33:14.892547+0000","last_became_active":"2024-05-10T05:31:51.586641+0000","last_became_peered":"2024-05-10T05:31:51.586641+0000","last_unstale":"2024-05-10T05:33:14.892547+0000","last_undegraded":"2024-05-10T05:33:14.892547+0000","last_fullsized":"2024-05-10T05:33:14.892547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:52.654918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918650+0000","last_change":"2024-05-10T05:31:50.999755+0000","last_active":"2024-05-10T05:32:14.918650+0000","last_peered":"2024-05-10T05:32:14.918650+0000","last_clean":"2024-05-10T05:32:14.918650+0000","last_became_active":"2024-05-10T05:31:50.999336+0000","last_became_peered":"2024-05-10T05:31:50.999336+0000","last_unstale":"2024-05-10T05:32:14.918650+0000","last_undegraded":"2024-05-10T05:32:14.918650+0000","last_fullsized":"2024-05-10T05:32:14.918650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:43.755130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902595+0000","last_change":"2024-05-10T05:31:51.576938+0000","last_active":"2024-05-10T05:33:14.902595+0000","last_peered":"2024-05-10T05:33:14.902595+0000","last_clean":"2024-05-10T05:33:14.902595+0000","last_became_active":"2024-05-10T05:31:51.576517+0000","last_became_peered":"2024-05-10T05:31:51.576517+0000","last_unstale":"2024-05-10T05:33:14.902595+0000","last_undegraded":"2024-05-10T05:33:14.902595+0000","last_fullsized":"2024-05-10T05:33:14.902595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:52:38.709063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916132+0000","last_change":"2024-05-10T05:31:50.992925+0000","last_active":"2024-05-10T05:32:14.916132+0000","last_peered":"2024-05-10T05:32:14.916132+0000","last_clean":"2024-05-10T05:32:14.916132+0000","last_became_active":"2024-05-10T05:31:50.992574+0000","last_became_peered":"2024-05-10T05:31:50.992574+0000","last_unstale":"2024-05-10T05:32:14.916132+0000","last_undegraded":"2024-05-10T05:32:14.916132+0000","last_fullsized":"2024-05-10T05:32:14.916132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:14.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888117+0000","last_change":"2024-05-10T05:31:51.580145+0000","last_active":"2024-05-10T05:33:14.888117+0000","last_peered":"2024-05-10T05:33:14.888117+0000","last_clean":"2024-05-10T05:33:14.888117+0000","last_became_active":"2024-05-10T05:31:51.576701+0000","last_became_peered":"2024-05-10T05:31:51.576701+0000","last_unstale":"2024-05-10T05:33:14.888117+0000","last_undegraded":"2024-05-10T05:33:14.888117+0000","last_fullsized":"2024-05-10T05:33:14.888117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:36:21.075571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915647+0000","last_change":"2024-05-10T05:31:50.987863+0000","last_active":"2024-05-10T05:32:14.915647+0000","last_peered":"2024-05-10T05:32:14.915647+0000","last_clean":"2024-05-10T05:32:14.915647+0000","last_became_active":"2024-05-10T05:31:50.987463+0000","last_became_peered":"2024-05-10T05:31:50.987463+0000","last_unstale":"2024-05-10T05:32:14.915647+0000","last_undegraded":"2024-05-10T05:32:14.915647+0000","last_fullsized":"2024-05-10T05:32:14.915647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:12:49.542288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.895479+0000","last_change":"2024-05-10T05:32:16.105284+0000","last_active":"2024-05-10T05:33:14.895479+0000","last_peered":"2024-05-10T05:33:14.895479+0000","last_clean":"2024-05-10T05:33:14.895479+0000","last_became_active":"2024-05-10T05:32:16.104915+0000","last_became_peered":"2024-05-10T05:32:16.104915+0000","last_unstale":"2024-05-10T05:33:14.895479+0000","last_undegraded":"2024-05-10T05:33:14.895479+0000","last_fullsized":"2024-05-10T05:33:14.895479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:25.628498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":110,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:22.620331+0000","last_change":"2024-05-10T05:31:50.985879+0000","last_active":"2024-05-10T05:33:22.620331+0000","last_peered":"2024-05-10T05:33:22.620331+0000","last_clean":"2024-05-10T05:33:22.620331+0000","last_became_active":"2024-05-10T05:31:50.985476+0000","last_became_peered":"2024-05-10T05:31:50.985476+0000","last_unstale":"2024-05-10T05:33:22.620331+0000","last_undegraded":"2024-05-10T05:33:22.620331+0000","last_fullsized":"2024-05-10T05:33:22.620331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:11:27.737972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1108,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.875759+0000","last_change":"2024-05-10T05:31:51.570985+0000","last_active":"2024-05-10T05:33:14.875759+0000","last_peered":"2024-05-10T05:33:14.875759+0000","last_clean":"2024-05-10T05:33:14.875759+0000","last_became_active":"2024-05-10T05:31:51.570324+0000","last_became_peered":"2024-05-10T05:31:51.570324+0000","last_unstale":"2024-05-10T05:33:14.875759+0000","last_undegraded":"2024-05-10T05:33:14.875759+0000","last_fullsized":"2024-05-10T05:33:14.875759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:26.947574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.911748+0000","last_change":"2024-05-10T05:31:50.988483+0000","last_active":"2024-05-10T05:32:14.911748+0000","last_peered":"2024-05-10T05:32:14.911748+0000","last_clean":"2024-05-10T05:32:14.911748+0000","last_became_active":"2024-05-10T05:31:50.987897+0000","last_became_peered":"2024-05-10T05:31:50.987897+0000","last_unstale":"2024-05-10T05:32:14.911748+0000","last_undegraded":"2024-05-10T05:32:14.911748+0000","last_fullsized":"2024-05-10T05:32:14.911748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:30:23.474637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.875011+0000","last_change":"2024-05-10T05:32:15.922113+0000","last_active":"2024-05-10T05:33:14.875011+0000","last_peered":"2024-05-10T05:33:14.875011+0000","last_clean":"2024-05-10T05:33:14.875011+0000","last_became_active":"2024-05-10T05:32:15.921665+0000","last_became_peered":"2024-05-10T05:32:15.921665+0000","last_unstale":"2024-05-10T05:33:14.875011+0000","last_undegraded":"2024-05-10T05:33:14.875011+0000","last_fullsized":"2024-05-10T05:33:14.875011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:28.155216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6],"acting":[5,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.908880+0000","last_change":"2024-05-10T05:31:50.981877+0000","last_active":"2024-05-10T05:32:14.908880+0000","last_peered":"2024-05-10T05:32:14.908880+0000","last_clean":"2024-05-10T05:32:14.908880+0000","last_became_active":"2024-05-10T05:31:50.981527+0000","last_became_peered":"2024-05-10T05:31:50.981527+0000","last_unstale":"2024-05-10T05:32:14.908880+0000","last_undegraded":"2024-05-10T05:32:14.908880+0000","last_fullsized":"2024-05-10T05:32:14.908880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:40.735304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.898545+0000","last_change":"2024-05-10T05:31:51.581027+0000","last_active":"2024-05-10T05:33:14.898545+0000","last_peered":"2024-05-10T05:33:14.898545+0000","last_clean":"2024-05-10T05:33:14.898545+0000","last_became_active":"2024-05-10T05:31:51.580620+0000","last_became_peered":"2024-05-10T05:31:51.580620+0000","last_unstale":"2024-05-10T05:33:14.898545+0000","last_undegraded":"2024-05-10T05:33:14.898545+0000","last_fullsized":"2024-05-10T05:33:14.898545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:30:15.378179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913340+0000","last_change":"2024-05-10T05:31:50.987633+0000","last_active":"2024-05-10T05:32:14.913340+0000","last_peered":"2024-05-10T05:32:14.913340+0000","last_clean":"2024-05-10T05:32:14.913340+0000","last_became_active":"2024-05-10T05:31:50.986750+0000","last_became_peered":"2024-05-10T05:31:50.986750+0000","last_unstale":"2024-05-10T05:32:14.913340+0000","last_undegraded":"2024-05-10T05:32:14.913340+0000","last_fullsized":"2024-05-10T05:32:14.913340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:17:47.705415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892509+0000","last_change":"2024-05-10T05:31:51.589484+0000","last_active":"2024-05-10T05:33:14.892509+0000","last_peered":"2024-05-10T05:33:14.892509+0000","last_clean":"2024-05-10T05:33:14.892509+0000","last_became_active":"2024-05-10T05:31:51.588992+0000","last_became_peered":"2024-05-10T05:31:51.588992+0000","last_unstale":"2024-05-10T05:33:14.892509+0000","last_undegraded":"2024-05-10T05:33:14.892509+0000","last_fullsized":"2024-05-10T05:33:14.892509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:32.360333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":195,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:25.624293+0000","last_change":"2024-05-10T05:31:50.988945+0000","last_active":"2024-05-10T05:33:25.624293+0000","last_peered":"2024-05-10T05:33:25.624293+0000","last_clean":"2024-05-10T05:33:25.624293+0000","last_became_active":"2024-05-10T05:31:50.988566+0000","last_became_peered":"2024-05-10T05:31:50.988566+0000","last_unstale":"2024-05-10T05:33:25.624293+0000","last_undegraded":"2024-05-10T05:33:25.624293+0000","last_fullsized":"2024-05-10T05:33:25.624293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:06:43.225346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888657+0000","last_change":"2024-05-10T05:31:51.577949+0000","last_active":"2024-05-10T05:33:14.888657+0000","last_peered":"2024-05-10T05:33:14.888657+0000","last_clean":"2024-05-10T05:33:14.888657+0000","last_became_active":"2024-05-10T05:31:51.577011+0000","last_became_peered":"2024-05-10T05:31:51.577011+0000","last_unstale":"2024-05-10T05:33:14.888657+0000","last_undegraded":"2024-05-10T05:33:14.888657+0000","last_fullsized":"2024-05-10T05:33:14.888657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:09.988700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920114+0000","last_change":"2024-05-10T05:31:50.984888+0000","last_active":"2024-05-10T05:32:14.920114+0000","last_peered":"2024-05-10T05:32:14.920114+0000","last_clean":"2024-05-10T05:32:14.920114+0000","last_became_active":"2024-05-10T05:31:50.984267+0000","last_became_peered":"2024-05-10T05:31:50.984267+0000","last_unstale":"2024-05-10T05:32:14.920114+0000","last_undegraded":"2024-05-10T05:32:14.920114+0000","last_fullsized":"2024-05-10T05:32:14.920114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:44.797469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.900759+0000","last_change":"2024-05-10T05:31:51.576863+0000","last_active":"2024-05-10T05:33:14.900759+0000","last_peered":"2024-05-10T05:33:14.900759+0000","last_clean":"2024-05-10T05:33:14.900759+0000","last_became_active":"2024-05-10T05:31:51.576468+0000","last_became_peered":"2024-05-10T05:31:51.576468+0000","last_unstale":"2024-05-10T05:33:14.900759+0000","last_undegraded":"2024-05-10T05:33:14.900759+0000","last_fullsized":"2024-05-10T05:33:14.900759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:20.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913803+0000","last_change":"2024-05-10T05:31:51.001566+0000","last_active":"2024-05-10T05:32:14.913803+0000","last_peered":"2024-05-10T05:32:14.913803+0000","last_clean":"2024-05-10T05:32:14.913803+0000","last_became_active":"2024-05-10T05:31:51.001203+0000","last_became_peered":"2024-05-10T05:31:51.001203+0000","last_unstale":"2024-05-10T05:32:14.913803+0000","last_undegraded":"2024-05-10T05:32:14.913803+0000","last_fullsized":"2024-05-10T05:32:14.913803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:17:45.465039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.883700+0000","last_change":"2024-05-10T05:31:51.579114+0000","last_active":"2024-05-10T05:33:14.883700+0000","last_peered":"2024-05-10T05:33:14.883700+0000","last_clean":"2024-05-10T05:33:14.883700+0000","last_became_active":"2024-05-10T05:31:51.577507+0000","last_became_peered":"2024-05-10T05:31:51.577507+0000","last_unstale":"2024-05-10T05:33:14.883700+0000","last_undegraded":"2024-05-10T05:33:14.883700+0000","last_fullsized":"2024-05-10T05:33:14.883700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:43.777983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'35","reported_seq":235,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:25.624677+0000","last_change":"2024-05-10T05:31:50.983558+0000","last_active":"2024-05-10T05:33:25.624677+0000","last_peered":"2024-05-10T05:33:25.624677+0000","last_clean":"2024-05-10T05:33:25.624677+0000","last_became_active":"2024-05-10T05:31:50.983087+0000","last_became_peered":"2024-05-10T05:31:50.983087+0000","last_unstale":"2024-05-10T05:33:25.624677+0000","last_undegraded":"2024-05-10T05:33:25.624677+0000","last_fullsized":"2024-05-10T05:33:25.624677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:08.753898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65616,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":158,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.889800+0000","last_change":"2024-05-10T05:31:51.574024+0000","last_active":"2024-05-10T05:33:14.889800+0000","last_peered":"2024-05-10T05:33:14.889800+0000","last_clean":"2024-05-10T05:33:14.889800+0000","last_became_active":"2024-05-10T05:31:51.573703+0000","last_became_peered":"2024-05-10T05:31:51.573703+0000","last_unstale":"2024-05-10T05:33:14.889800+0000","last_undegraded":"2024-05-10T05:33:14.889800+0000","last_fullsized":"2024-05-10T05:33:14.889800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:55:23.435991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918608+0000","last_change":"2024-05-10T05:31:50.986795+0000","last_active":"2024-05-10T05:32:14.918608+0000","last_peered":"2024-05-10T05:32:14.918608+0000","last_clean":"2024-05-10T05:32:14.918608+0000","last_became_active":"2024-05-10T05:31:50.986428+0000","last_became_peered":"2024-05-10T05:31:50.986428+0000","last_unstale":"2024-05-10T05:32:14.918608+0000","last_undegraded":"2024-05-10T05:32:14.918608+0000","last_fullsized":"2024-05-10T05:32:14.918608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:11.039632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.894052+0000","last_change":"2024-05-10T05:31:51.585595+0000","last_active":"2024-05-10T05:33:14.894052+0000","last_peered":"2024-05-10T05:33:14.894052+0000","last_clean":"2024-05-10T05:33:14.894052+0000","last_became_active":"2024-05-10T05:31:51.585091+0000","last_became_peered":"2024-05-10T05:31:51.585091+0000","last_unstale":"2024-05-10T05:33:14.894052+0000","last_undegraded":"2024-05-10T05:33:14.894052+0000","last_fullsized":"2024-05-10T05:33:14.894052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:10:10.489430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913631+0000","last_change":"2024-05-10T05:31:50.984408+0000","last_active":"2024-05-10T05:32:14.913631+0000","last_peered":"2024-05-10T05:32:14.913631+0000","last_clean":"2024-05-10T05:32:14.913631+0000","last_became_active":"2024-05-10T05:31:50.983098+0000","last_became_peered":"2024-05-10T05:31:50.983098+0000","last_unstale":"2024-05-10T05:32:14.913631+0000","last_undegraded":"2024-05-10T05:32:14.913631+0000","last_fullsized":"2024-05-10T05:32:14.913631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:31.399649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.893997+0000","last_change":"2024-05-10T05:31:51.576160+0000","last_active":"2024-05-10T05:33:14.893997+0000","last_peered":"2024-05-10T05:33:14.893997+0000","last_clean":"2024-05-10T05:33:14.893997+0000","last_became_active":"2024-05-10T05:31:51.575806+0000","last_became_peered":"2024-05-10T05:31:51.575806+0000","last_unstale":"2024-05-10T05:33:14.893997+0000","last_undegraded":"2024-05-10T05:33:14.893997+0000","last_fullsized":"2024-05-10T05:33:14.893997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:53:25.660988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":114,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:26.625959+0000","last_change":"2024-05-10T05:31:50.982752+0000","last_active":"2024-05-10T05:33:26.625959+0000","last_peered":"2024-05-10T05:33:26.625959+0000","last_clean":"2024-05-10T05:33:26.625959+0000","last_became_active":"2024-05-10T05:31:50.982299+0000","last_became_peered":"2024-05-10T05:31:50.982299+0000","last_unstale":"2024-05-10T05:33:26.625959+0000","last_undegraded":"2024-05-10T05:33:26.625959+0000","last_fullsized":"2024-05-10T05:33:26.625959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:52:40.206967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892923+0000","last_change":"2024-05-10T05:31:51.577081+0000","last_active":"2024-05-10T05:33:14.892923+0000","last_peered":"2024-05-10T05:33:14.892923+0000","last_clean":"2024-05-10T05:33:14.892923+0000","last_became_active":"2024-05-10T05:31:51.576729+0000","last_became_peered":"2024-05-10T05:31:51.576729+0000","last_unstale":"2024-05-10T05:33:14.892923+0000","last_undegraded":"2024-05-10T05:33:14.892923+0000","last_fullsized":"2024-05-10T05:33:14.892923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:14.072427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915219+0000","last_change":"2024-05-10T05:31:50.991868+0000","last_active":"2024-05-10T05:32:14.915219+0000","last_peered":"2024-05-10T05:32:14.915219+0000","last_clean":"2024-05-10T05:32:14.915219+0000","last_became_active":"2024-05-10T05:31:50.991471+0000","last_became_peered":"2024-05-10T05:31:50.991471+0000","last_unstale":"2024-05-10T05:32:14.915219+0000","last_undegraded":"2024-05-10T05:32:14.915219+0000","last_fullsized":"2024-05-10T05:32:14.915219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:22.346376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.873664+0000","last_change":"2024-05-10T05:31:51.584726+0000","last_active":"2024-05-10T05:33:14.873664+0000","last_peered":"2024-05-10T05:33:14.873664+0000","last_clean":"2024-05-10T05:33:14.873664+0000","last_became_active":"2024-05-10T05:31:51.584327+0000","last_became_peered":"2024-05-10T05:31:51.584327+0000","last_unstale":"2024-05-10T05:33:14.873664+0000","last_undegraded":"2024-05-10T05:33:14.873664+0000","last_fullsized":"2024-05-10T05:33:14.873664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:56:43.561717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915510+0000","last_change":"2024-05-10T05:31:50.989071+0000","last_active":"2024-05-10T05:32:14.915510+0000","last_peered":"2024-05-10T05:32:14.915510+0000","last_clean":"2024-05-10T05:32:14.915510+0000","last_became_active":"2024-05-10T05:31:50.988044+0000","last_became_peered":"2024-05-10T05:31:50.988044+0000","last_unstale":"2024-05-10T05:32:14.915510+0000","last_undegraded":"2024-05-10T05:32:14.915510+0000","last_fullsized":"2024-05-10T05:32:14.915510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:24.122564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.884441+0000","last_change":"2024-05-10T05:31:51.576793+0000","last_active":"2024-05-10T05:33:14.884441+0000","last_peered":"2024-05-10T05:33:14.884441+0000","last_clean":"2024-05-10T05:33:14.884441+0000","last_became_active":"2024-05-10T05:31:51.576333+0000","last_became_peered":"2024-05-10T05:31:51.576333+0000","last_unstale":"2024-05-10T05:33:14.884441+0000","last_undegraded":"2024-05-10T05:33:14.884441+0000","last_fullsized":"2024-05-10T05:33:14.884441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:42.418736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913436+0000","last_change":"2024-05-10T05:31:50.997233+0000","last_active":"2024-05-10T05:32:14.913436+0000","last_peered":"2024-05-10T05:32:14.913436+0000","last_clean":"2024-05-10T05:32:14.913436+0000","last_became_active":"2024-05-10T05:31:50.996857+0000","last_became_peered":"2024-05-10T05:31:50.996857+0000","last_unstale":"2024-05-10T05:32:14.913436+0000","last_undegraded":"2024-05-10T05:32:14.913436+0000","last_fullsized":"2024-05-10T05:32:14.913436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:35.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.887621+0000","last_change":"2024-05-10T05:32:15.927682+0000","last_active":"2024-05-10T05:33:14.887621+0000","last_peered":"2024-05-10T05:33:14.887621+0000","last_clean":"2024-05-10T05:33:14.887621+0000","last_became_active":"2024-05-10T05:32:15.924197+0000","last_became_peered":"2024-05-10T05:32:15.924197+0000","last_unstale":"2024-05-10T05:33:14.887621+0000","last_undegraded":"2024-05-10T05:33:14.887621+0000","last_fullsized":"2024-05-10T05:33:14.887621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:14.747603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:24.622894+0000","last_change":"2024-05-10T05:31:50.991113+0000","last_active":"2024-05-10T05:33:24.622894+0000","last_peered":"2024-05-10T05:33:24.622894+0000","last_clean":"2024-05-10T05:33:24.622894+0000","last_became_active":"2024-05-10T05:31:50.990745+0000","last_became_peered":"2024-05-10T05:31:50.990745+0000","last_unstale":"2024-05-10T05:33:24.622894+0000","last_undegraded":"2024-05-10T05:33:24.622894+0000","last_fullsized":"2024-05-10T05:33:24.622894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:49:56.586231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897318+0000","last_change":"2024-05-10T05:31:51.582057+0000","last_active":"2024-05-10T05:33:14.897318+0000","last_peered":"2024-05-10T05:33:14.897318+0000","last_clean":"2024-05-10T05:33:14.897318+0000","last_became_active":"2024-05-10T05:31:51.581672+0000","last_became_peered":"2024-05-10T05:31:51.581672+0000","last_unstale":"2024-05-10T05:33:14.897318+0000","last_undegraded":"2024-05-10T05:33:14.897318+0000","last_fullsized":"2024-05-10T05:33:14.897318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:45:09.122256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.877986+0000","last_change":"2024-05-10T05:31:51.585659+0000","last_active":"2024-05-10T05:33:14.877986+0000","last_peered":"2024-05-10T05:33:14.877986+0000","last_clean":"2024-05-10T05:33:14.877986+0000","last_became_active":"2024-05-10T05:31:51.584540+0000","last_became_peered":"2024-05-10T05:31:51.584540+0000","last_unstale":"2024-05-10T05:33:14.877986+0000","last_undegraded":"2024-05-10T05:33:14.877986+0000","last_fullsized":"2024-05-10T05:33:14.877986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:20.195369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":111,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:26.625190+0000","last_change":"2024-05-10T05:31:50.993547+0000","last_active":"2024-05-10T05:33:26.625190+0000","last_peered":"2024-05-10T05:33:26.625190+0000","last_clean":"2024-05-10T05:33:26.625190+0000","last_became_active":"2024-05-10T05:31:50.993190+0000","last_became_peered":"2024-05-10T05:31:50.993190+0000","last_unstale":"2024-05-10T05:33:26.625190+0000","last_undegraded":"2024-05-10T05:33:26.625190+0000","last_fullsized":"2024-05-10T05:33:26.625190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:06.159572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.890795+0000","last_change":"2024-05-10T05:31:51.587678+0000","last_active":"2024-05-10T05:33:14.890795+0000","last_peered":"2024-05-10T05:33:14.890795+0000","last_clean":"2024-05-10T05:33:14.890795+0000","last_became_active":"2024-05-10T05:31:51.586999+0000","last_became_peered":"2024-05-10T05:31:51.586999+0000","last_unstale":"2024-05-10T05:33:14.890795+0000","last_undegraded":"2024-05-10T05:33:14.890795+0000","last_fullsized":"2024-05-10T05:33:14.890795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:18:00.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914737+0000","last_change":"2024-05-10T05:31:50.979131+0000","last_active":"2024-05-10T05:32:14.914737+0000","last_peered":"2024-05-10T05:32:14.914737+0000","last_clean":"2024-05-10T05:32:14.914737+0000","last_became_active":"2024-05-10T05:31:50.978715+0000","last_became_peered":"2024-05-10T05:31:50.978715+0000","last_unstale":"2024-05-10T05:32:14.914737+0000","last_undegraded":"2024-05-10T05:32:14.914737+0000","last_fullsized":"2024-05-10T05:32:14.914737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:29:11.953963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919607+0000","last_change":"2024-05-10T05:30:59.356688+0000","last_active":"2024-05-10T05:32:14.919607+0000","last_peered":"2024-05-10T05:32:14.919607+0000","last_clean":"2024-05-10T05:32:14.919607+0000","last_became_active":"2024-05-10T05:30:59.356313+0000","last_became_peered":"2024-05-10T05:30:59.356313+0000","last_unstale":"2024-05-10T05:32:14.919607+0000","last_undegraded":"2024-05-10T05:32:14.919607+0000","last_fullsized":"2024-05-10T05:32:14.919607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_clean_scrub_stamp":"2024-05-10T05:28:15.503591+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:43.011495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.879926+0000","last_change":"2024-05-10T05:31:51.580259+0000","last_active":"2024-05-10T05:33:14.879926+0000","last_peered":"2024-05-10T05:33:14.879926+0000","last_clean":"2024-05-10T05:33:14.879926+0000","last_became_active":"2024-05-10T05:31:51.578650+0000","last_became_peered":"2024-05-10T05:31:51.578650+0000","last_unstale":"2024-05-10T05:33:14.879926+0000","last_undegraded":"2024-05-10T05:33:14.879926+0000","last_fullsized":"2024-05-10T05:33:14.879926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:48.630183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919744+0000","last_change":"2024-05-10T05:31:50.987708+0000","last_active":"2024-05-10T05:32:14.919744+0000","last_peered":"2024-05-10T05:32:14.919744+0000","last_clean":"2024-05-10T05:32:14.919744+0000","last_became_active":"2024-05-10T05:31:50.987347+0000","last_became_peered":"2024-05-10T05:31:50.987347+0000","last_unstale":"2024-05-10T05:32:14.919744+0000","last_undegraded":"2024-05-10T05:32:14.919744+0000","last_fullsized":"2024-05-10T05:32:14.919744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:16:29.388690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915952+0000","last_change":"2024-05-10T05:31:50.999338+0000","last_active":"2024-05-10T05:32:14.915952+0000","last_peered":"2024-05-10T05:32:14.915952+0000","last_clean":"2024-05-10T05:32:14.915952+0000","last_became_active":"2024-05-10T05:31:50.999026+0000","last_became_peered":"2024-05-10T05:31:50.999026+0000","last_unstale":"2024-05-10T05:32:14.915952+0000","last_undegraded":"2024-05-10T05:32:14.915952+0000","last_fullsized":"2024-05-10T05:32:14.915952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:31.443409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.880920+0000","last_change":"2024-05-10T05:31:51.574201+0000","last_active":"2024-05-10T05:33:14.880920+0000","last_peered":"2024-05-10T05:33:14.880920+0000","last_clean":"2024-05-10T05:33:14.880920+0000","last_became_active":"2024-05-10T05:31:51.571180+0000","last_became_peered":"2024-05-10T05:31:51.571180+0000","last_unstale":"2024-05-10T05:33:14.880920+0000","last_undegraded":"2024-05-10T05:33:14.880920+0000","last_fullsized":"2024-05-10T05:33:14.880920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:33:16.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:15.336605+0000","last_change":"2024-05-10T05:31:50.985164+0000","last_active":"2024-05-10T05:33:15.336605+0000","last_peered":"2024-05-10T05:33:15.336605+0000","last_clean":"2024-05-10T05:33:15.336605+0000","last_became_active":"2024-05-10T05:31:50.984760+0000","last_became_peered":"2024-05-10T05:31:50.984760+0000","last_unstale":"2024-05-10T05:33:15.336605+0000","last_undegraded":"2024-05-10T05:33:15.336605+0000","last_fullsized":"2024-05-10T05:33:15.336605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:19.827407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.894754+0000","last_change":"2024-05-10T05:31:51.577445+0000","last_active":"2024-05-10T05:33:14.894754+0000","last_peered":"2024-05-10T05:33:14.894754+0000","last_clean":"2024-05-10T05:33:14.894754+0000","last_became_active":"2024-05-10T05:31:51.576671+0000","last_became_peered":"2024-05-10T05:31:51.576671+0000","last_unstale":"2024-05-10T05:33:14.894754+0000","last_undegraded":"2024-05-10T05:33:14.894754+0000","last_fullsized":"2024-05-10T05:33:14.894754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:38.774538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914419+0000","last_change":"2024-05-10T05:31:50.981495+0000","last_active":"2024-05-10T05:32:14.914419+0000","last_peered":"2024-05-10T05:32:14.914419+0000","last_clean":"2024-05-10T05:32:14.914419+0000","last_became_active":"2024-05-10T05:31:50.981127+0000","last_became_peered":"2024-05-10T05:31:50.981127+0000","last_unstale":"2024-05-10T05:32:14.914419+0000","last_undegraded":"2024-05-10T05:32:14.914419+0000","last_fullsized":"2024-05-10T05:32:14.914419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:22:32.257014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902371+0000","last_change":"2024-05-10T05:31:51.585926+0000","last_active":"2024-05-10T05:33:14.902371+0000","last_peered":"2024-05-10T05:33:14.902371+0000","last_clean":"2024-05-10T05:33:14.902371+0000","last_became_active":"2024-05-10T05:31:51.585576+0000","last_became_peered":"2024-05-10T05:31:51.585576+0000","last_unstale":"2024-05-10T05:33:14.902371+0000","last_undegraded":"2024-05-10T05:33:14.902371+0000","last_fullsized":"2024-05-10T05:33:14.902371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:22.344180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.926062+0000","last_change":"2024-05-10T05:32:15.926062+0000","last_active":"2024-05-10T05:32:15.926062+0000","last_peered":"2024-05-10T05:32:15.926062+0000","last_clean":"2024-05-10T05:32:15.926062+0000","last_became_active":"2024-05-10T05:32:15.923026+0000","last_became_peered":"2024-05-10T05:32:15.923026+0000","last_unstale":"2024-05-10T05:32:15.926062+0000","last_undegraded":"2024-05-10T05:32:15.926062+0000","last_fullsized":"2024-05-10T05:32:15.926062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:34.188115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.848777+0000","last_change":"2024-05-10T05:31:51.573279+0000","last_active":"2024-05-10T05:33:14.848777+0000","last_peered":"2024-05-10T05:33:14.848777+0000","last_clean":"2024-05-10T05:33:14.848777+0000","last_became_active":"2024-05-10T05:31:51.572916+0000","last_became_peered":"2024-05-10T05:31:51.572916+0000","last_unstale":"2024-05-10T05:33:14.848777+0000","last_undegraded":"2024-05-10T05:33:14.848777+0000","last_fullsized":"2024-05-10T05:33:14.848777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:32:39.802463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918101+0000","last_change":"2024-05-10T05:31:50.983771+0000","last_active":"2024-05-10T05:32:14.918101+0000","last_peered":"2024-05-10T05:32:14.918101+0000","last_clean":"2024-05-10T05:32:14.918101+0000","last_became_active":"2024-05-10T05:31:50.981849+0000","last_became_peered":"2024-05-10T05:31:50.981849+0000","last_unstale":"2024-05-10T05:32:14.918101+0000","last_undegraded":"2024-05-10T05:32:14.918101+0000","last_fullsized":"2024-05-10T05:32:14.918101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:15.990869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.878726+0000","last_change":"2024-05-10T05:31:51.568005+0000","last_active":"2024-05-10T05:33:14.878726+0000","last_peered":"2024-05-10T05:33:14.878726+0000","last_clean":"2024-05-10T05:33:14.878726+0000","last_became_active":"2024-05-10T05:31:51.567347+0000","last_became_peered":"2024-05-10T05:31:51.567347+0000","last_unstale":"2024-05-10T05:33:14.878726+0000","last_undegraded":"2024-05-10T05:33:14.878726+0000","last_fullsized":"2024-05-10T05:33:14.878726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:29.075816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.926267+0000","last_change":"2024-05-10T05:32:15.925869+0000","last_active":"2024-05-10T05:32:15.926267+0000","last_peered":"2024-05-10T05:32:15.926267+0000","last_clean":"2024-05-10T05:32:15.926267+0000","last_became_active":"2024-05-10T05:32:15.920591+0000","last_became_peered":"2024-05-10T05:32:15.920591+0000","last_unstale":"2024-05-10T05:32:15.926267+0000","last_undegraded":"2024-05-10T05:32:15.926267+0000","last_fullsized":"2024-05-10T05:32:15.926267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:13:20.398637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.901352+0000","last_change":"2024-05-10T05:31:51.588255+0000","last_active":"2024-05-10T05:33:14.901352+0000","last_peered":"2024-05-10T05:33:14.901352+0000","last_clean":"2024-05-10T05:33:14.901352+0000","last_became_active":"2024-05-10T05:31:51.587912+0000","last_became_peered":"2024-05-10T05:31:51.587912+0000","last_unstale":"2024-05-10T05:33:14.901352+0000","last_undegraded":"2024-05-10T05:33:14.901352+0000","last_fullsized":"2024-05-10T05:33:14.901352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:18.515739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916229+0000","last_change":"2024-05-10T05:31:50.992392+0000","last_active":"2024-05-10T05:32:14.916229+0000","last_peered":"2024-05-10T05:32:14.916229+0000","last_clean":"2024-05-10T05:32:14.916229+0000","last_became_active":"2024-05-10T05:31:50.992119+0000","last_became_peered":"2024-05-10T05:31:50.992119+0000","last_unstale":"2024-05-10T05:32:14.916229+0000","last_undegraded":"2024-05-10T05:32:14.916229+0000","last_fullsized":"2024-05-10T05:32:14.916229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:11.515013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.893189+0000","last_change":"2024-05-10T05:31:51.567780+0000","last_active":"2024-05-10T05:33:14.893189+0000","last_peered":"2024-05-10T05:33:14.893189+0000","last_clean":"2024-05-10T05:33:14.893189+0000","last_became_active":"2024-05-10T05:31:51.567455+0000","last_became_peered":"2024-05-10T05:31:51.567455+0000","last_unstale":"2024-05-10T05:33:14.893189+0000","last_undegraded":"2024-05-10T05:33:14.893189+0000","last_fullsized":"2024-05-10T05:33:14.893189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:47.108131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:23.621608+0000","last_change":"2024-05-10T05:31:50.997374+0000","last_active":"2024-05-10T05:33:23.621608+0000","last_peered":"2024-05-10T05:33:23.621608+0000","last_clean":"2024-05-10T05:33:23.621608+0000","last_became_active":"2024-05-10T05:31:50.997069+0000","last_became_peered":"2024-05-10T05:31:50.997069+0000","last_unstale":"2024-05-10T05:33:23.621608+0000","last_undegraded":"2024-05-10T05:33:23.621608+0000","last_fullsized":"2024-05-10T05:33:23.621608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:25:49.140006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.889341+0000","last_change":"2024-05-10T05:31:51.576345+0000","last_active":"2024-05-10T05:33:14.889341+0000","last_peered":"2024-05-10T05:33:14.889341+0000","last_clean":"2024-05-10T05:33:14.889341+0000","last_became_active":"2024-05-10T05:31:51.575704+0000","last_became_peered":"2024-05-10T05:31:51.575704+0000","last_unstale":"2024-05-10T05:33:14.889341+0000","last_undegraded":"2024-05-10T05:33:14.889341+0000","last_fullsized":"2024-05-10T05:33:14.889341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:15.146591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913524+0000","last_change":"2024-05-10T05:31:50.984108+0000","last_active":"2024-05-10T05:32:14.913524+0000","last_peered":"2024-05-10T05:32:14.913524+0000","last_clean":"2024-05-10T05:32:14.913524+0000","last_became_active":"2024-05-10T05:31:50.983728+0000","last_became_peered":"2024-05-10T05:31:50.983728+0000","last_unstale":"2024-05-10T05:32:14.913524+0000","last_undegraded":"2024-05-10T05:32:14.913524+0000","last_fullsized":"2024-05-10T05:32:14.913524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:23:20.096286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.895124+0000","last_change":"2024-05-10T05:31:51.572075+0000","last_active":"2024-05-10T05:33:14.895124+0000","last_peered":"2024-05-10T05:33:14.895124+0000","last_clean":"2024-05-10T05:33:14.895124+0000","last_became_active":"2024-05-10T05:31:51.571711+0000","last_became_peered":"2024-05-10T05:31:51.571711+0000","last_unstale":"2024-05-10T05:33:14.895124+0000","last_undegraded":"2024-05-10T05:33:14.895124+0000","last_fullsized":"2024-05-10T05:33:14.895124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:38:27.813001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.907701+0000","last_change":"2024-05-10T05:31:50.983985+0000","last_active":"2024-05-10T05:32:14.907701+0000","last_peered":"2024-05-10T05:32:14.907701+0000","last_clean":"2024-05-10T05:32:14.907701+0000","last_became_active":"2024-05-10T05:31:50.982931+0000","last_became_peered":"2024-05-10T05:31:50.982931+0000","last_unstale":"2024-05-10T05:32:14.907701+0000","last_undegraded":"2024-05-10T05:32:14.907701+0000","last_fullsized":"2024-05-10T05:32:14.907701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:35:17.158696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888805+0000","last_change":"2024-05-10T05:31:51.587250+0000","last_active":"2024-05-10T05:33:14.888805+0000","last_peered":"2024-05-10T05:33:14.888805+0000","last_clean":"2024-05-10T05:33:14.888805+0000","last_became_active":"2024-05-10T05:31:51.586925+0000","last_became_peered":"2024-05-10T05:31:51.586925+0000","last_unstale":"2024-05-10T05:33:14.888805+0000","last_undegraded":"2024-05-10T05:33:14.888805+0000","last_fullsized":"2024-05-10T05:33:14.888805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:40.585803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.879173+0000","last_change":"2024-05-10T05:31:51.567710+0000","last_active":"2024-05-10T05:33:14.879173+0000","last_peered":"2024-05-10T05:33:14.879173+0000","last_clean":"2024-05-10T05:33:14.879173+0000","last_became_active":"2024-05-10T05:31:51.566860+0000","last_became_peered":"2024-05-10T05:31:51.566860+0000","last_unstale":"2024-05-10T05:33:14.879173+0000","last_undegraded":"2024-05-10T05:33:14.879173+0000","last_fullsized":"2024-05-10T05:33:14.879173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:02:09.253345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.879510+0000","last_change":"2024-05-10T05:31:51.577458+0000","last_active":"2024-05-10T05:33:14.879510+0000","last_peered":"2024-05-10T05:33:14.879510+0000","last_clean":"2024-05-10T05:33:14.879510+0000","last_became_active":"2024-05-10T05:31:51.577123+0000","last_became_peered":"2024-05-10T05:31:51.577123+0000","last_unstale":"2024-05-10T05:33:14.879510+0000","last_undegraded":"2024-05-10T05:33:14.879510+0000","last_fullsized":"2024-05-10T05:33:14.879510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:51.791090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902636+0000","last_change":"2024-05-10T05:31:51.589361+0000","last_active":"2024-05-10T05:33:14.902636+0000","last_peered":"2024-05-10T05:33:14.902636+0000","last_clean":"2024-05-10T05:33:14.902636+0000","last_became_active":"2024-05-10T05:31:51.589049+0000","last_became_peered":"2024-05-10T05:31:51.589049+0000","last_unstale":"2024-05-10T05:33:14.902636+0000","last_undegraded":"2024-05-10T05:33:14.902636+0000","last_fullsized":"2024-05-10T05:33:14.902636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:13.380505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902342+0000","last_change":"2024-05-10T05:32:15.919725+0000","last_active":"2024-05-10T05:33:14.902342+0000","last_peered":"2024-05-10T05:33:14.902342+0000","last_clean":"2024-05-10T05:33:14.902342+0000","last_became_active":"2024-05-10T05:32:15.919342+0000","last_became_peered":"2024-05-10T05:32:15.919342+0000","last_unstale":"2024-05-10T05:33:14.902342+0000","last_undegraded":"2024-05-10T05:33:14.902342+0000","last_fullsized":"2024-05-10T05:33:14.902342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:33.785843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.891254+0000","last_change":"2024-05-10T05:31:51.579791+0000","last_active":"2024-05-10T05:33:14.891254+0000","last_peered":"2024-05-10T05:33:14.891254+0000","last_clean":"2024-05-10T05:33:14.891254+0000","last_became_active":"2024-05-10T05:31:51.577635+0000","last_became_peered":"2024-05-10T05:31:51.577635+0000","last_unstale":"2024-05-10T05:33:14.891254+0000","last_undegraded":"2024-05-10T05:33:14.891254+0000","last_fullsized":"2024-05-10T05:33:14.891254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:54.077805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.873675+0000","last_change":"2024-05-10T05:31:51.584601+0000","last_active":"2024-05-10T05:33:14.873675+0000","last_peered":"2024-05-10T05:33:14.873675+0000","last_clean":"2024-05-10T05:33:14.873675+0000","last_became_active":"2024-05-10T05:31:51.584172+0000","last_became_peered":"2024-05-10T05:31:51.584172+0000","last_unstale":"2024-05-10T05:33:14.873675+0000","last_undegraded":"2024-05-10T05:33:14.873675+0000","last_fullsized":"2024-05-10T05:33:14.873675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:49:44.987379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.884792+0000","last_change":"2024-05-10T05:31:51.573784+0000","last_active":"2024-05-10T05:33:14.884792+0000","last_peered":"2024-05-10T05:33:14.884792+0000","last_clean":"2024-05-10T05:33:14.884792+0000","last_became_active":"2024-05-10T05:31:51.573279+0000","last_became_peered":"2024-05-10T05:31:51.573279+0000","last_unstale":"2024-05-10T05:33:14.884792+0000","last_undegraded":"2024-05-10T05:33:14.884792+0000","last_fullsized":"2024-05-10T05:33:14.884792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:08.375964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.881529+0000","last_change":"2024-05-10T05:31:51.585577+0000","last_active":"2024-05-10T05:33:14.881529+0000","last_peered":"2024-05-10T05:33:14.881529+0000","last_clean":"2024-05-10T05:33:14.881529+0000","last_became_active":"2024-05-10T05:31:51.585222+0000","last_became_peered":"2024-05-10T05:31:51.585222+0000","last_unstale":"2024-05-10T05:33:14.881529+0000","last_undegraded":"2024-05-10T05:33:14.881529+0000","last_fullsized":"2024-05-10T05:33:14.881529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:47:27.069513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892962+0000","last_change":"2024-05-10T05:31:51.571405+0000","last_active":"2024-05-10T05:33:14.892962+0000","last_peered":"2024-05-10T05:33:14.892962+0000","last_clean":"2024-05-10T05:33:14.892962+0000","last_became_active":"2024-05-10T05:31:51.571125+0000","last_became_peered":"2024-05-10T05:31:51.571125+0000","last_unstale":"2024-05-10T05:33:14.892962+0000","last_undegraded":"2024-05-10T05:33:14.892962+0000","last_fullsized":"2024-05-10T05:33:14.892962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:49:29.069121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.895256+0000","last_change":"2024-05-10T05:31:51.578883+0000","last_active":"2024-05-10T05:33:14.895256+0000","last_peered":"2024-05-10T05:33:14.895256+0000","last_clean":"2024-05-10T05:33:14.895256+0000","last_became_active":"2024-05-10T05:31:51.578582+0000","last_became_peered":"2024-05-10T05:31:51.578582+0000","last_unstale":"2024-05-10T05:33:14.895256+0000","last_undegraded":"2024-05-10T05:33:14.895256+0000","last_fullsized":"2024-05-10T05:33:14.895256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:16.402282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.891358+0000","last_change":"2024-05-10T05:31:51.589269+0000","last_active":"2024-05-10T05:33:14.891358+0000","last_peered":"2024-05-10T05:33:14.891358+0000","last_clean":"2024-05-10T05:33:14.891358+0000","last_became_active":"2024-05-10T05:31:51.588875+0000","last_became_peered":"2024-05-10T05:31:51.588875+0000","last_unstale":"2024-05-10T05:33:14.891358+0000","last_undegraded":"2024-05-10T05:33:14.891358+0000","last_fullsized":"2024-05-10T05:33:14.891358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:12.536555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.901574+0000","last_change":"2024-05-10T05:31:51.567789+0000","last_active":"2024-05-10T05:33:14.901574+0000","last_peered":"2024-05-10T05:33:14.901574+0000","last_clean":"2024-05-10T05:33:14.901574+0000","last_became_active":"2024-05-10T05:31:51.566625+0000","last_became_peered":"2024-05-10T05:31:51.566625+0000","last_unstale":"2024-05-10T05:33:14.901574+0000","last_undegraded":"2024-05-10T05:33:14.901574+0000","last_fullsized":"2024-05-10T05:33:14.901574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:43.042557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.876173+0000","last_change":"2024-05-10T05:31:51.585838+0000","last_active":"2024-05-10T05:33:14.876173+0000","last_peered":"2024-05-10T05:33:14.876173+0000","last_clean":"2024-05-10T05:33:14.876173+0000","last_became_active":"2024-05-10T05:31:51.585473+0000","last_became_peered":"2024-05-10T05:31:51.585473+0000","last_unstale":"2024-05-10T05:33:14.876173+0000","last_undegraded":"2024-05-10T05:33:14.876173+0000","last_fullsized":"2024-05-10T05:33:14.876173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:11:11.282076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247530,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72914,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":925,"num_read_kb":546,"num_write":247,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":213369,"data_compressed":5520,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":429949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.52500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.84199999999999997}]},{"osd":2,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.67900000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.36299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70599999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.499},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56799999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97599999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.50600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88300000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.747}]},{"osd":5,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89600000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88300000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.64000000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.03,"5min":1.127,"15min":1.127},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76300000000000001,"15min":0.76800000000000002},"min":{"1min":0.61799999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98499999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.78000000000000003}]},{"osd":9,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84099999999999997,"15min":0.84499999999999997},"min":{"1min":0.68000000000000005,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96799999999999997,"5min":1.514,"15min":1.514},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.82599999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.004,"5min":1.216,"15min":1.216},"last":0.85799999999999998}]},{"osd":10,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.71399999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94199999999999995,"5min":1.254,"15min":1.254},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.86099999999999999,"15min":0.86299999999999999},"min":{"1min":0.69199999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93899999999999995,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.86899999999999999}]}]},{"osd":10,"up_from":68,"seq":292057776161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1087128,"data_compressed":37906,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64500000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.871,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81100000000000005,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.747}]},{"osd":3,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.63500000000000001,"15min":0.64800000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.70399999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.504,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86399999999999999,"5min":1.091,"15min":1.091},"last":0.83299999999999996}]},{"osd":4,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.48299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79400000000000004,"5min":1.036,"15min":1.036},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":1.173,"15min":1.173},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.99099999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.58199999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84499999999999997,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76900000000000002,"15min":0.78900000000000003},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78500000000000003,"5min":1.448,"15min":1.448},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.67700000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.91100000000000003,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":8,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81799999999999995,"15min":0.81999999999999995},"min":{"1min":0.70099999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.65300000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0740000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.88300000000000001,"15min":0.89100000000000001},"min":{"1min":0.67800000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0189999999999999,"5min":1.728,"15min":1.728},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.85099999999999998,"15min":0.86099999999999999},"min":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0289999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.878}]},{"osd":11,"last update":"Fri May 10 05:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.66000000000000003,"5min":0.628,"15min":0.628},"max":{"1min":0.94699999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.85999999999999999},"min":{"1min":0.72199999999999998,"5min":0.625,"15min":0.625},"max":{"1min":1.008,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.97299999999999998}]}]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":921600,"data_stored":430044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.629,"15min":0.64800000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.63}]},{"osd":1,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.02,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.75700000000000001}]},{"osd":2,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.871},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3759999999999999,"5min":1.488,"15min":1.488},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71999999999999997,"15min":0.73599999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86499999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1160000000000001,"5min":1.177,"15min":1.177},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.872,"15min":0.88800000000000001},"min":{"1min":0.53000000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.403,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.874,"15min":0.89400000000000002},"min":{"1min":0.58099999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0580000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.89900000000000002,"15min":0.91400000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1339999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.90500000000000003,"15min":0.92500000000000004},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.115,"5min":1.288,"15min":1.288},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.93500000000000005,"15min":0.95399999999999996},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.135,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4650000000000001,"5min":1.611,"15min":1.611},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.97899999999999998,"15min":1.0009999999999999},"min":{"1min":0.70599999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.196,"5min":1.46,"15min":1.46},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.2450000000000001,"5min":1.337,"15min":1.337},"last":0.91200000000000003}]},{"osd":8,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.998,"15min":1.008},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.427,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.997,"15min":1.0129999999999999},"min":{"1min":0.71599999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2170000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.78900000000000003}]},{"osd":10,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1,"15min":1.016},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.99399999999999999,"15min":1.014},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.032,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.85799999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":430305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.995,"15min":0.995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.65600000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.629,"15min":0.627},"min":{"1min":0.41299999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.626,"15min":0.627},"min":{"1min":0.52800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78700000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84399999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.876,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70299999999999996}]},{"osd":4,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.745},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.54000000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93700000000000006,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.90300000000000002}]},{"osd":5,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89100000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.42199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91900000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.63200000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98599999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.76900000000000002}]},{"osd":7,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.754,"15min":0.753},"min":{"1min":0.54800000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.997,"5min":1.101,"15min":1.101},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.64300000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.96999999999999997,"5min":1.702,"15min":1.702},"last":0.81200000000000006}]},{"osd":9,"last update":"Fri May 10 05:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.76400000000000001,"15min":0.74299999999999999},"min":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.77900000000000003,"15min":0.76600000000000001},"min":{"1min":0.65500000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri May 10 05:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.70199999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.876},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.78400000000000003,"15min":0.76800000000000002},"min":{"1min":0.73299999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.96299999999999997}]},{"osd":11,"last update":"Fri May 10 05:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.98599999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":966656,"data_stored":1023356,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.438,"15min":0.433},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.44500000000000001},"min":{"1min":0.32900000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.39300000000000002}]},{"osd":2,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77900000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.85899999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.49099999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.73399999999999999}]},{"osd":4,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.496,"15min":0.49099999999999999},"min":{"1min":0.374,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.75,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67900000000000005,"15min":0.66600000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96099999999999997,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.53700000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84299999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86499999999999999}]},{"osd":8,"last update":"Fri May 10 05:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.60399999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92100000000000004,"5min":1.002,"15min":1.002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.71199999999999997,"15min":0.70399999999999996},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.748}]},{"osd":9,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.626,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.022,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.65600000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.89200000000000002}]},{"osd":10,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.77800000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.96799999999999997,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.68300000000000005}]},{"osd":11,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.69999999999999996}]}]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":901120,"data_stored":429596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67900000000000005,"15min":0.65100000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.59799999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98299999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.91900000000000004}]},{"osd":2,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72399999999999998,"15min":0.70799999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69399999999999995,"15min":0.69499999999999995},"min":{"1min":0.442,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97499999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.65800000000000003},"min":{"1min":0.41599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.77200000000000002},"min":{"1min":0.56899999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.083,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76700000000000002,"15min":0.75700000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0740000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.83799999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.63}]},{"osd":7,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.82999999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.81200000000000006},"min":{"1min":0.60099999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.86799999999999999}]},{"osd":8,"last update":"Fri May 10 05:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.90400000000000003,"15min":0.94199999999999995},"min":{"1min":0.622,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.99399999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91400000000000003,"15min":0.95499999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.367,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.91700000000000004,"15min":0.92200000000000004},"min":{"1min":0.52500000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.97599999999999998,"15min":0.99099999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2270000000000001,"5min":1.52,"15min":1.52},"last":0.88500000000000001}]},{"osd":10,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.999,"15min":1.0129999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.0740000000000001,"5min":1.458,"15min":1.458},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.93600000000000005,"15min":0.94099999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.81599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":962560,"data_stored":496125,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.628,"15min":0.79000000000000004},"min":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82899999999999996,"5min":6.3810000000000002,"15min":6.3810000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri May 10 05:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.60399999999999998,"15min":0.56799999999999995},"min":{"1min":0.48699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.625,"15min":0.58299999999999996},"min":{"1min":0.61699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri May 10 05:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78700000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.47499999999999998,"15min":0.44},"min":{"1min":0.40000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.501,"15min":0.45400000000000001},"min":{"1min":0.36199999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri May 10 05:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.61299999999999999,"15min":0.55800000000000005},"min":{"1min":0.55400000000000005,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.66700000000000004,"15min":0.61799999999999999},"min":{"1min":0.629,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri May 10 05:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68400000000000005,"15min":0.66500000000000004},"min":{"1min":0.46999999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.67400000000000004},"min":{"1min":0.56100000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.62,"15min":0.58999999999999997},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.46500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81899999999999995}]},{"osd":8,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.73299999999999998,"15min":0.69499999999999995},"min":{"1min":0.66100000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.71799999999999997,"15min":0.67200000000000004},"min":{"1min":0.64300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.04,"5min":1.113,"15min":1.113},"last":0.94499999999999995}]},{"osd":9,"last update":"Fri May 10 05:32:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.79700000000000004,"15min":0.79100000000000004},"min":{"1min":0.59499999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.1379999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.89600000000000002}]},{"osd":10,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.63,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0780000000000001,"5min":1.121,"15min":1.121},"last":0.98799999999999999}]},{"osd":11,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.071}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":921600,"data_stored":430935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 05:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.68600000000000005,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67300000000000004,"15min":0.61899999999999999},"min":{"1min":0.49299999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.849,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69699999999999995,"15min":0.63200000000000001},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96199999999999997,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.63800000000000001},"min":{"1min":0.46800000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.218,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri May 10 05:32:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.177,"5min":3.177,"15min":3.177},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri May 10 05:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72899999999999998,"15min":0.67300000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9450000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.748},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.73899999999999999,"15min":0.67500000000000004},"min":{"1min":0.56999999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":1.026}]},{"osd":5,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73999999999999999,"15min":0.67000000000000004},"min":{"1min":0.53200000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.2789999999999999,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.78700000000000003,"15min":0.71199999999999997},"min":{"1min":0.496,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":6.069,"5min":6.069,"15min":6.069},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri May 10 05:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.80500000000000005,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":4.859,"5min":4.859,"15min":4.859},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.76700000000000002,"15min":0.73999999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.0740000000000001}]},{"osd":7,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90700000000000003}]},{"osd":8,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.86599999999999999,"15min":0.86799999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.123,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":1.123},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.254,"5min":4.6559999999999997,"15min":4.6559999999999997},"last":0.83899999999999997}]},{"osd":9,"last update":"Fri May 10 05:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0349999999999999,"15min":1.0289999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.173,"5min":5.173,"15min":5.173},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.96899999999999997,"15min":0.97599999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"last":1.1659999999999999}]},{"osd":10,"last update":"Fri May 10 05:33:00 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.96199999999999997,"15min":0.93899999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.04,"15min":1.012},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":6.1189999999999998,"5min":6.1189999999999998,"15min":6.1189999999999998},"last":0.93300000000000005}]},{"osd":11,"last update":"Fri May 10 05:33:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0449999999999999,"15min":1.0429999999999999},"min":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":6.0949999999999998,"5min":6.0949999999999998,"15min":6.0949999999999998},"last":1.208},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.99099999999999999,"15min":1.02},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0589999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.98799999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":978944,"data_stored":491458,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70599999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66300000000000003}]},{"osd":1,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.51700000000000002},"min":{"1min":0.41299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81499999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83999999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.439,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.623}]},{"osd":4,"last update":"Fri May 10 05:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.48999999999999999,"15min":0.44400000000000001},"min":{"1min":0.38200000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81499999999999995,"5min":1.026,"15min":1.026},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.47199999999999998},"min":{"1min":0.40100000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69399999999999995,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri May 10 05:32:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.64000000000000001},"min":{"1min":0.53100000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.66000000000000003},"min":{"1min":0.58099999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.745}]},{"osd":8,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.57099999999999995},"min":{"1min":0.46100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.97699999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.60699999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82399999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47199999999999998}]},{"osd":9,"last update":"Fri May 10 05:32:40 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.71399999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.76900000000000002,"15min":0.752},"min":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri May 10 05:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79100000000000004,"15min":0.78900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0189999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.65000000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92800000000000005,"5min":1.121,"15min":1.121},"last":0.747}]},{"osd":11,"last update":"Fri May 10 05:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79400000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":429352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.45400000000000001,"15min":0.41199999999999998},"min":{"1min":0.34899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.46600000000000003,"15min":0.41799999999999998},"min":{"1min":0.38100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67000000000000004,"15min":0.63300000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.113,"5min":1.161,"15min":1.161},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91400000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73499999999999999}]},{"osd":2,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65700000000000003,"15min":0.60499999999999998},"min":{"1min":0.44600000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67800000000000005,"15min":0.63800000000000001},"min":{"1min":0.51900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0780000000000001,"5min":1.206,"15min":1.206},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri May 10 05:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.67900000000000005,"15min":0.64100000000000001},"min":{"1min":0.60299999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70599999999999996,"15min":0.66900000000000004},"min":{"1min":0.498,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99299999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.755}]},{"osd":5,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66900000000000004,"15min":0.64100000000000001},"min":{"1min":0.50600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84599999999999997,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.032,"5min":0.79400000000000004,"15min":0.72699999999999998},"min":{"1min":0.49199999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":7.4160000000000004,"5min":7.4160000000000004,"15min":7.4160000000000004},"last":0.95699999999999996}]},{"osd":6,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71599999999999997,"15min":0.68700000000000006},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri May 10 05:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79100000000000004,"15min":0.77300000000000002},"min":{"1min":0.61399999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.84399999999999997,"15min":0.82799999999999996},"min":{"1min":0.64700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1579999999999999,"5min":1.216,"15min":1.216},"last":0.91700000000000004}]},{"osd":8,"last update":"Fri May 10 05:31:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.753}]},{"osd":9,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.95699999999999996,"15min":0.97399999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1919999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.879,"15min":0.89800000000000002},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.032,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.621}]},{"osd":10,"last update":"Fri May 10 05:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.98999999999999999,"15min":1.0089999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.226,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.88300000000000001}]},{"osd":11,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.82999999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":901120,"data_stored":429991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri May 10 05:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.37},"min":{"1min":0.34499999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.54500000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.441,"15min":0.40300000000000002},"min":{"1min":0.36499999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.629}]},{"osd":2,"last update":"Fri May 10 05:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.45500000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.53100000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82099999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri May 10 05:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.83699999999999997,"15min":0.65900000000000003},"min":{"1min":0.55100000000000005,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97599999999999998,"5min":13.608000000000001,"15min":13.608000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.52000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.83399999999999996,"5min":0.876,"15min":0.876},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri May 10 05:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.65800000000000003,"15min":0.62},"min":{"1min":0.57499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89100000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.61199999999999999},"min":{"1min":0.55300000000000005,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83199999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82799999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.69699999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.754}]},{"osd":7,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61599999999999999,"15min":0.60099999999999998},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76200000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.878},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67900000000000005,"15min":0.66400000000000003},"min":{"1min":0.51600000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86599999999999999,"5min":1.081,"15min":1.081},"last":0.94699999999999995}]},{"osd":8,"last update":"Fri May 10 05:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96599999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78400000000000003,"15min":0.77200000000000002},"min":{"1min":0.61099999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":1.0640000000000001}]},{"osd":9,"last update":"Fri May 10 05:32:41 2024","interfaces":[{"interface":"back","average":{"1min":1.8220000000000001,"5min":1.048,"15min":0.91900000000000004},"min":{"1min":0.65700000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":13.648,"5min":13.648,"15min":13.648},"last":1.042},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.93500000000000005}]},{"osd":10,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.65400000000000003,"5min":0.621,"15min":0.621},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.038},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.874,"15min":0.88},"min":{"1min":0.77200000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":0.92000000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":1.0089999999999999}]},{"osd":11,"last update":"Fri May 10 05:32:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.997},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.0880000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":978944,"data_stored":1022596,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.624,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.751,"5min":4.524,"15min":4.524},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.57999999999999996},"min":{"1min":0.45000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.53400000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.49299999999999999}]},{"osd":2,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.35099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71699999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.68899999999999995}]},{"osd":3,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.66800000000000004,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.3860000000000001,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81100000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61099999999999999,"15min":0.59099999999999997},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86299999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.64100000000000001,"15min":0.62},"min":{"1min":0.435,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90600000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.53000000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91600000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.751},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.53400000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88100000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri May 10 05:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.45800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86199999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70599999999999996,"15min":0.69099999999999995},"min":{"1min":0.51600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97299999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri May 10 05:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.83099999999999996}]},{"osd":9,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.751,"15min":0.751},"min":{"1min":0.57699999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93899999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.875},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.80700000000000005,"15min":0.80900000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.98499999999999999,"5min":1.03,"15min":1.03},"last":0.84399999999999997}]},{"osd":10,"last update":"Fri May 10 05:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.746,"15min":0.73599999999999999},"min":{"1min":0.61799999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93300000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0009999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67320,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62653,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":67838,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T05:33:30.845 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-10T05:33:30.845 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T05:33:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.143+0000 7f963be00700 1 -- 172.21.15.131:0/474636231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f963c1024e0 msgr2=0x7f963c104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.143+0000 7f963be00700 1 --2- 172.21.15.131:0/474636231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f963c1024e0 0x7f963c104900 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f962c009990 tx=0x7f962c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.144+0000 7f963be00700 1 -- 172.21.15.131:0/474636231 shutdown_connections 2024-05-10T05:33:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.144+0000 7f963be00700 1 --2- 172.21.15.131:0/474636231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f963c1077a0 0x7f963c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.144+0000 7f963be00700 1 --2- 172.21.15.131:0/474636231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f963c1024e0 0x7f963c104900 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.144+0000 7f963be00700 1 --2- 172.21.15.131:0/474636231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c104e40 0x7f963c107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.144+0000 7f963be00700 1 -- 172.21.15.131:0/474636231 >> 172.21.15.131:0/474636231 conn(0x7f963c0fbf20 msgr2=0x7f963c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:31.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.144+0000 7f963be00700 1 -- 172.21.15.131:0/474636231 shutdown_connections 2024-05-10T05:33:31.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.144+0000 7f963be00700 1 -- 172.21.15.131:0/474636231 wait complete. 2024-05-10T05:33:31.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963be00700 1 Processor -- start 2024-05-10T05:33:31.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963be00700 1 -- start start 2024-05-10T05:33:31.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f963c1024e0 0x7f963c199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c104e40 0x7f963c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f963c1077a0 0x7f963c1a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963be00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c1a19d0 con 0x7f963c1077a0 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963be00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c1a1b10 con 0x7f963c104e40 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963be00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c1a1ca0 con 0x7f963c1024e0 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f963c1024e0 0x7f963c199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f963c1077a0 0x7f963c1a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f963c1024e0 0x7f963c199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54742/0 (socket says 172.21.15.131:54742) 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.145+0000 7f963aa00700 1 -- 172.21.15.131:0/3638133644 learned_addr learned my addr 172.21.15.131:0/3638133644 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f963a000700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c104e40 0x7f963c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f963aa00700 1 -- 172.21.15.131:0/3638133644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c104e40 msgr2=0x7f963c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f963aa00700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c104e40 0x7f963c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f963aa00700 1 -- 172.21.15.131:0/3638133644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f963c1077a0 msgr2=0x7f963c1a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f963aa00700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f963c1077a0 0x7f963c1a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f963aa00700 1 -- 172.21.15.131:0/3638133644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634009c40 con 0x7f963c1024e0 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f963b400700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f963c1077a0 0x7f963c1a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f963aa00700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f963c1024e0 0x7f963c199f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f962c005740 tx=0x7f962c005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f9633400700 1 -- 172.21.15.131:0/3638133644 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c01d070 con 0x7f963c1024e0 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f963be00700 1 -- 172.21.15.131:0/3638133644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962c0095f0 con 0x7f963c1024e0 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f9633400700 1 -- 172.21.15.131:0/3638133644 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f962c021410 con 0x7f963c1024e0 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.146+0000 7f9633400700 1 -- 172.21.15.131:0/3638133644 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c017650 con 0x7f963c1024e0 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.147+0000 7f963be00700 1 -- 172.21.15.131:0/3638133644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c1a21c0 con 0x7f963c1024e0 2024-05-10T05:33:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.147+0000 7f963be00700 1 -- 172.21.15.131:0/3638133644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c109350 con 0x7f963c1024e0 2024-05-10T05:33:31.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.149+0000 7f9633400700 1 -- 172.21.15.131:0/3638133644 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f962c0217f0 con 0x7f963c1024e0 2024-05-10T05:33:31.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.151+0000 7f9633400700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96200a11c0 0x7f96200a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.151+0000 7f9633400700 1 -- 172.21.15.131:0/3638133644 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f962c0cdc40 con 0x7f963c1024e0 2024-05-10T05:33:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.151+0000 7f963a000700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96200a11c0 0x7f96200a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.152+0000 7f963a000700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96200a11c0 0x7f96200a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f963c19afd0 tx=0x7f9628009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:31.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.155+0000 7f9633400700 1 -- 172.21.15.131:0/3638133644 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f962c097460 con 0x7f963c1024e0 2024-05-10T05:33:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.359+0000 7f963be00700 1 -- 172.21.15.131:0/3638133644 --> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f963c02ce90 con 0x7f96200a11c0 2024-05-10T05:33:31.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.369+0000 7f9633400700 1 -- 172.21.15.131:0/3638133644 <== mgr.14154 v2:172.21.15.131:6800/2109111580 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344670 (secure 0 0 0) 0x7f963c02ce90 con 0x7f96200a11c0 2024-05-10T05:33:31.375 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:31.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 -- 172.21.15.131:0/3638133644 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96200a11c0 msgr2=0x7f96200a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:31.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96200a11c0 0x7f96200a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f963c19afd0 tx=0x7f9628009260 comp rx=0 tx=0).stop 2024-05-10T05:33:31.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 -- 172.21.15.131:0/3638133644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f963c1024e0 msgr2=0x7f963c199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f963c1024e0 0x7f963c199f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f962c005740 tx=0x7f962c005770 comp rx=0 tx=0).stop 2024-05-10T05:33:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 -- 172.21.15.131:0/3638133644 shutdown_connections 2024-05-10T05:33:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f96200a11c0 0x7f96200a36d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f963c1077a0 0x7f963c1a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f963c1024e0 0x7f963c199f30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 --2- 172.21.15.131:0/3638133644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f963c104e40 0x7f963c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 -- 172.21.15.131:0/3638133644 >> 172.21.15.131:0/3638133644 conn(0x7f963c0fbf20 msgr2=0x7f963c1063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 -- 172.21.15.131:0/3638133644 shutdown_connections 2024-05-10T05:33:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.374+0000 7f963be00700 1 -- 172.21.15.131:0/3638133644 wait complete. 2024-05-10T05:33:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-10T05:33:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-05-10T05:33:30.857839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663516,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1061,"num_read_kb":655,"num_write":430,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":854,"ondisk_log_size":854,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380288,"kb_used_data":11040,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124561536,"statfs":{"total":1151940427776,"available":1151551012864,"internally_reserved":0,"allocated":11304960,"data_stored":7136208,"data_compressed":113718,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19128,"internal_metadata":377992520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5824,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":113,"num_read_kb":58,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003278"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897159+0000","last_change":"2024-05-10T05:31:51.576054+0000","last_active":"2024-05-10T05:33:14.897159+0000","last_peered":"2024-05-10T05:33:14.897159+0000","last_clean":"2024-05-10T05:33:14.897159+0000","last_became_active":"2024-05-10T05:31:51.575186+0000","last_became_peered":"2024-05-10T05:31:51.575186+0000","last_unstale":"2024-05-10T05:33:14.897159+0000","last_undegraded":"2024-05-10T05:33:14.897159+0000","last_fullsized":"2024-05-10T05:33:14.897159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:16:27.654822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.898953+0000","last_change":"2024-05-10T05:31:51.576483+0000","last_active":"2024-05-10T05:33:14.898953+0000","last_peered":"2024-05-10T05:33:14.898953+0000","last_clean":"2024-05-10T05:33:14.898953+0000","last_became_active":"2024-05-10T05:31:51.574936+0000","last_became_peered":"2024-05-10T05:31:51.574936+0000","last_unstale":"2024-05-10T05:33:14.898953+0000","last_undegraded":"2024-05-10T05:33:14.898953+0000","last_fullsized":"2024-05-10T05:33:14.898953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:56:51.436880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902454+0000","last_change":"2024-05-10T05:31:51.567114+0000","last_active":"2024-05-10T05:33:14.902454+0000","last_peered":"2024-05-10T05:33:14.902454+0000","last_clean":"2024-05-10T05:33:14.902454+0000","last_became_active":"2024-05-10T05:31:51.564552+0000","last_became_peered":"2024-05-10T05:31:51.564552+0000","last_unstale":"2024-05-10T05:33:14.902454+0000","last_undegraded":"2024-05-10T05:33:14.902454+0000","last_fullsized":"2024-05-10T05:33:14.902454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:40.268764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.878967+0000","last_change":"2024-05-10T05:31:51.584471+0000","last_active":"2024-05-10T05:33:14.878967+0000","last_peered":"2024-05-10T05:33:14.878967+0000","last_clean":"2024-05-10T05:33:14.878967+0000","last_became_active":"2024-05-10T05:31:51.583749+0000","last_became_peered":"2024-05-10T05:31:51.583749+0000","last_unstale":"2024-05-10T05:33:14.878967+0000","last_undegraded":"2024-05-10T05:33:14.878967+0000","last_fullsized":"2024-05-10T05:33:14.878967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:30.775286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.894275+0000","last_change":"2024-05-10T05:31:51.577698+0000","last_active":"2024-05-10T05:33:14.894275+0000","last_peered":"2024-05-10T05:33:14.894275+0000","last_clean":"2024-05-10T05:33:14.894275+0000","last_became_active":"2024-05-10T05:31:51.571469+0000","last_became_peered":"2024-05-10T05:31:51.571469+0000","last_unstale":"2024-05-10T05:33:14.894275+0000","last_undegraded":"2024-05-10T05:33:14.894275+0000","last_fullsized":"2024-05-10T05:33:14.894275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:16:35.161267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.887837+0000","last_change":"2024-05-10T05:31:51.573560+0000","last_active":"2024-05-10T05:33:14.887837+0000","last_peered":"2024-05-10T05:33:14.887837+0000","last_clean":"2024-05-10T05:33:14.887837+0000","last_became_active":"2024-05-10T05:31:51.573227+0000","last_became_peered":"2024-05-10T05:31:51.573227+0000","last_unstale":"2024-05-10T05:33:14.887837+0000","last_undegraded":"2024-05-10T05:33:14.887837+0000","last_fullsized":"2024-05-10T05:33:14.887837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:23.062937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.891483+0000","last_change":"2024-05-10T05:31:51.573667+0000","last_active":"2024-05-10T05:33:14.891483+0000","last_peered":"2024-05-10T05:33:14.891483+0000","last_clean":"2024-05-10T05:33:14.891483+0000","last_became_active":"2024-05-10T05:31:51.573362+0000","last_became_peered":"2024-05-10T05:31:51.573362+0000","last_unstale":"2024-05-10T05:33:14.891483+0000","last_undegraded":"2024-05-10T05:33:14.891483+0000","last_fullsized":"2024-05-10T05:33:14.891483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:16:00.300134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.899011+0000","last_change":"2024-05-10T05:31:51.583596+0000","last_active":"2024-05-10T05:33:14.899011+0000","last_peered":"2024-05-10T05:33:14.899011+0000","last_clean":"2024-05-10T05:33:14.899011+0000","last_became_active":"2024-05-10T05:31:51.583235+0000","last_became_peered":"2024-05-10T05:31:51.583235+0000","last_unstale":"2024-05-10T05:33:14.899011+0000","last_undegraded":"2024-05-10T05:33:14.899011+0000","last_fullsized":"2024-05-10T05:33:14.899011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:05.575975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.882166+0000","last_change":"2024-05-10T05:31:51.567627+0000","last_active":"2024-05-10T05:33:14.882166+0000","last_peered":"2024-05-10T05:33:14.882166+0000","last_clean":"2024-05-10T05:33:14.882166+0000","last_became_active":"2024-05-10T05:31:51.567276+0000","last_became_peered":"2024-05-10T05:31:51.567276+0000","last_unstale":"2024-05-10T05:33:14.882166+0000","last_undegraded":"2024-05-10T05:33:14.882166+0000","last_fullsized":"2024-05-10T05:33:14.882166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:16:27.914802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888316+0000","last_change":"2024-05-10T05:31:51.573859+0000","last_active":"2024-05-10T05:33:14.888316+0000","last_peered":"2024-05-10T05:33:14.888316+0000","last_clean":"2024-05-10T05:33:14.888316+0000","last_became_active":"2024-05-10T05:31:51.573493+0000","last_became_peered":"2024-05-10T05:31:51.573493+0000","last_unstale":"2024-05-10T05:33:14.888316+0000","last_undegraded":"2024-05-10T05:33:14.888316+0000","last_fullsized":"2024-05-10T05:33:14.888316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:21.024888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.889191+0000","last_change":"2024-05-10T05:31:51.576056+0000","last_active":"2024-05-10T05:33:14.889191+0000","last_peered":"2024-05-10T05:33:14.889191+0000","last_clean":"2024-05-10T05:33:14.889191+0000","last_became_active":"2024-05-10T05:31:51.575146+0000","last_became_peered":"2024-05-10T05:31:51.575146+0000","last_unstale":"2024-05-10T05:33:14.889191+0000","last_undegraded":"2024-05-10T05:33:14.889191+0000","last_fullsized":"2024-05-10T05:33:14.889191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:31:41.201052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.885798+0000","last_change":"2024-05-10T05:31:51.584710+0000","last_active":"2024-05-10T05:33:14.885798+0000","last_peered":"2024-05-10T05:33:14.885798+0000","last_clean":"2024-05-10T05:33:14.885798+0000","last_became_active":"2024-05-10T05:31:51.584122+0000","last_became_peered":"2024-05-10T05:31:51.584122+0000","last_unstale":"2024-05-10T05:33:14.885798+0000","last_undegraded":"2024-05-10T05:33:14.885798+0000","last_fullsized":"2024-05-10T05:33:14.885798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:43.622981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.890824+0000","last_change":"2024-05-10T05:32:15.924380+0000","last_active":"2024-05-10T05:33:14.890824+0000","last_peered":"2024-05-10T05:33:14.890824+0000","last_clean":"2024-05-10T05:33:14.890824+0000","last_became_active":"2024-05-10T05:32:15.923952+0000","last_became_peered":"2024-05-10T05:32:15.923952+0000","last_unstale":"2024-05-10T05:33:14.890824+0000","last_undegraded":"2024-05-10T05:33:14.890824+0000","last_fullsized":"2024-05-10T05:33:14.890824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:04:14.087785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897069+0000","last_change":"2024-05-10T05:31:51.574628+0000","last_active":"2024-05-10T05:33:14.897069+0000","last_peered":"2024-05-10T05:33:14.897069+0000","last_clean":"2024-05-10T05:33:14.897069+0000","last_became_active":"2024-05-10T05:31:51.574256+0000","last_became_peered":"2024-05-10T05:31:51.574256+0000","last_unstale":"2024-05-10T05:33:14.897069+0000","last_undegraded":"2024-05-10T05:33:14.897069+0000","last_fullsized":"2024-05-10T05:33:14.897069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:59.174727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892288+0000","last_change":"2024-05-10T05:32:15.920243+0000","last_active":"2024-05-10T05:33:14.892288+0000","last_peered":"2024-05-10T05:33:14.892288+0000","last_clean":"2024-05-10T05:33:14.892288+0000","last_became_active":"2024-05-10T05:32:15.919822+0000","last_became_peered":"2024-05-10T05:32:15.919822+0000","last_unstale":"2024-05-10T05:33:14.892288+0000","last_undegraded":"2024-05-10T05:33:14.892288+0000","last_fullsized":"2024-05-10T05:33:14.892288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:59.392399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.900749+0000","last_change":"2024-05-10T05:31:51.568919+0000","last_active":"2024-05-10T05:33:14.900749+0000","last_peered":"2024-05-10T05:33:14.900749+0000","last_clean":"2024-05-10T05:33:14.900749+0000","last_became_active":"2024-05-10T05:31:51.568582+0000","last_became_peered":"2024-05-10T05:31:51.568582+0000","last_unstale":"2024-05-10T05:33:14.900749+0000","last_undegraded":"2024-05-10T05:33:14.900749+0000","last_fullsized":"2024-05-10T05:33:14.900749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:08.377261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897309+0000","last_change":"2024-05-10T05:31:51.579025+0000","last_active":"2024-05-10T05:33:14.897309+0000","last_peered":"2024-05-10T05:33:14.897309+0000","last_clean":"2024-05-10T05:33:14.897309+0000","last_became_active":"2024-05-10T05:31:51.578704+0000","last_became_peered":"2024-05-10T05:31:51.578704+0000","last_unstale":"2024-05-10T05:33:14.897309+0000","last_undegraded":"2024-05-10T05:33:14.897309+0000","last_fullsized":"2024-05-10T05:33:14.897309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:16.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.887421+0000","last_change":"2024-05-10T05:31:51.573193+0000","last_active":"2024-05-10T05:33:14.887421+0000","last_peered":"2024-05-10T05:33:14.887421+0000","last_clean":"2024-05-10T05:33:14.887421+0000","last_became_active":"2024-05-10T05:31:51.572874+0000","last_became_peered":"2024-05-10T05:31:51.572874+0000","last_unstale":"2024-05-10T05:33:14.887421+0000","last_undegraded":"2024-05-10T05:33:14.887421+0000","last_fullsized":"2024-05-10T05:33:14.887421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:31:09.867661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.882400+0000","last_change":"2024-05-10T05:31:51.581107+0000","last_active":"2024-05-10T05:33:14.882400+0000","last_peered":"2024-05-10T05:33:14.882400+0000","last_clean":"2024-05-10T05:33:14.882400+0000","last_became_active":"2024-05-10T05:31:51.580765+0000","last_became_peered":"2024-05-10T05:31:51.580765+0000","last_unstale":"2024-05-10T05:33:14.882400+0000","last_undegraded":"2024-05-10T05:33:14.882400+0000","last_fullsized":"2024-05-10T05:33:14.882400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:04.164486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":118,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:26.625187+0000","last_change":"2024-05-10T05:31:50.986534+0000","last_active":"2024-05-10T05:33:26.625187+0000","last_peered":"2024-05-10T05:33:26.625187+0000","last_clean":"2024-05-10T05:33:26.625187+0000","last_became_active":"2024-05-10T05:31:50.983493+0000","last_became_peered":"2024-05-10T05:31:50.983493+0000","last_unstale":"2024-05-10T05:33:26.625187+0000","last_undegraded":"2024-05-10T05:33:26.625187+0000","last_fullsized":"2024-05-10T05:33:26.625187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:18:56.926317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892547+0000","last_change":"2024-05-10T05:31:51.587121+0000","last_active":"2024-05-10T05:33:14.892547+0000","last_peered":"2024-05-10T05:33:14.892547+0000","last_clean":"2024-05-10T05:33:14.892547+0000","last_became_active":"2024-05-10T05:31:51.586641+0000","last_became_peered":"2024-05-10T05:31:51.586641+0000","last_unstale":"2024-05-10T05:33:14.892547+0000","last_undegraded":"2024-05-10T05:33:14.892547+0000","last_fullsized":"2024-05-10T05:33:14.892547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:52.654918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918650+0000","last_change":"2024-05-10T05:31:50.999755+0000","last_active":"2024-05-10T05:32:14.918650+0000","last_peered":"2024-05-10T05:32:14.918650+0000","last_clean":"2024-05-10T05:32:14.918650+0000","last_became_active":"2024-05-10T05:31:50.999336+0000","last_became_peered":"2024-05-10T05:31:50.999336+0000","last_unstale":"2024-05-10T05:32:14.918650+0000","last_undegraded":"2024-05-10T05:32:14.918650+0000","last_fullsized":"2024-05-10T05:32:14.918650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:43.755130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902595+0000","last_change":"2024-05-10T05:31:51.576938+0000","last_active":"2024-05-10T05:33:14.902595+0000","last_peered":"2024-05-10T05:33:14.902595+0000","last_clean":"2024-05-10T05:33:14.902595+0000","last_became_active":"2024-05-10T05:31:51.576517+0000","last_became_peered":"2024-05-10T05:31:51.576517+0000","last_unstale":"2024-05-10T05:33:14.902595+0000","last_undegraded":"2024-05-10T05:33:14.902595+0000","last_fullsized":"2024-05-10T05:33:14.902595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:52:38.709063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916132+0000","last_change":"2024-05-10T05:31:50.992925+0000","last_active":"2024-05-10T05:32:14.916132+0000","last_peered":"2024-05-10T05:32:14.916132+0000","last_clean":"2024-05-10T05:32:14.916132+0000","last_became_active":"2024-05-10T05:31:50.992574+0000","last_became_peered":"2024-05-10T05:31:50.992574+0000","last_unstale":"2024-05-10T05:32:14.916132+0000","last_undegraded":"2024-05-10T05:32:14.916132+0000","last_fullsized":"2024-05-10T05:32:14.916132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:02:14.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888117+0000","last_change":"2024-05-10T05:31:51.580145+0000","last_active":"2024-05-10T05:33:14.888117+0000","last_peered":"2024-05-10T05:33:14.888117+0000","last_clean":"2024-05-10T05:33:14.888117+0000","last_became_active":"2024-05-10T05:31:51.576701+0000","last_became_peered":"2024-05-10T05:31:51.576701+0000","last_unstale":"2024-05-10T05:33:14.888117+0000","last_undegraded":"2024-05-10T05:33:14.888117+0000","last_fullsized":"2024-05-10T05:33:14.888117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:36:21.075571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915647+0000","last_change":"2024-05-10T05:31:50.987863+0000","last_active":"2024-05-10T05:32:14.915647+0000","last_peered":"2024-05-10T05:32:14.915647+0000","last_clean":"2024-05-10T05:32:14.915647+0000","last_became_active":"2024-05-10T05:31:50.987463+0000","last_became_peered":"2024-05-10T05:31:50.987463+0000","last_unstale":"2024-05-10T05:32:14.915647+0000","last_undegraded":"2024-05-10T05:32:14.915647+0000","last_fullsized":"2024-05-10T05:32:14.915647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:12:49.542288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.895479+0000","last_change":"2024-05-10T05:32:16.105284+0000","last_active":"2024-05-10T05:33:14.895479+0000","last_peered":"2024-05-10T05:33:14.895479+0000","last_clean":"2024-05-10T05:33:14.895479+0000","last_became_active":"2024-05-10T05:32:16.104915+0000","last_became_peered":"2024-05-10T05:32:16.104915+0000","last_unstale":"2024-05-10T05:33:14.895479+0000","last_undegraded":"2024-05-10T05:33:14.895479+0000","last_fullsized":"2024-05-10T05:33:14.895479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:25.628498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":115,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:27.628328+0000","last_change":"2024-05-10T05:31:50.985879+0000","last_active":"2024-05-10T05:33:27.628328+0000","last_peered":"2024-05-10T05:33:27.628328+0000","last_clean":"2024-05-10T05:33:27.628328+0000","last_became_active":"2024-05-10T05:31:50.985476+0000","last_became_peered":"2024-05-10T05:31:50.985476+0000","last_unstale":"2024-05-10T05:33:27.628328+0000","last_undegraded":"2024-05-10T05:33:27.628328+0000","last_fullsized":"2024-05-10T05:33:27.628328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:11:27.737972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1108,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.875759+0000","last_change":"2024-05-10T05:31:51.570985+0000","last_active":"2024-05-10T05:33:14.875759+0000","last_peered":"2024-05-10T05:33:14.875759+0000","last_clean":"2024-05-10T05:33:14.875759+0000","last_became_active":"2024-05-10T05:31:51.570324+0000","last_became_peered":"2024-05-10T05:31:51.570324+0000","last_unstale":"2024-05-10T05:33:14.875759+0000","last_undegraded":"2024-05-10T05:33:14.875759+0000","last_fullsized":"2024-05-10T05:33:14.875759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:26.947574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.911748+0000","last_change":"2024-05-10T05:31:50.988483+0000","last_active":"2024-05-10T05:32:14.911748+0000","last_peered":"2024-05-10T05:32:14.911748+0000","last_clean":"2024-05-10T05:32:14.911748+0000","last_became_active":"2024-05-10T05:31:50.987897+0000","last_became_peered":"2024-05-10T05:31:50.987897+0000","last_unstale":"2024-05-10T05:32:14.911748+0000","last_undegraded":"2024-05-10T05:32:14.911748+0000","last_fullsized":"2024-05-10T05:32:14.911748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:30:23.474637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.875011+0000","last_change":"2024-05-10T05:32:15.922113+0000","last_active":"2024-05-10T05:33:14.875011+0000","last_peered":"2024-05-10T05:33:14.875011+0000","last_clean":"2024-05-10T05:33:14.875011+0000","last_became_active":"2024-05-10T05:32:15.921665+0000","last_became_peered":"2024-05-10T05:32:15.921665+0000","last_unstale":"2024-05-10T05:33:14.875011+0000","last_undegraded":"2024-05-10T05:33:14.875011+0000","last_fullsized":"2024-05-10T05:33:14.875011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:28.155216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,6],"acting":[5,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.908880+0000","last_change":"2024-05-10T05:31:50.981877+0000","last_active":"2024-05-10T05:32:14.908880+0000","last_peered":"2024-05-10T05:32:14.908880+0000","last_clean":"2024-05-10T05:32:14.908880+0000","last_became_active":"2024-05-10T05:31:50.981527+0000","last_became_peered":"2024-05-10T05:31:50.981527+0000","last_unstale":"2024-05-10T05:32:14.908880+0000","last_undegraded":"2024-05-10T05:32:14.908880+0000","last_fullsized":"2024-05-10T05:32:14.908880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:40.735304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.898545+0000","last_change":"2024-05-10T05:31:51.581027+0000","last_active":"2024-05-10T05:33:14.898545+0000","last_peered":"2024-05-10T05:33:14.898545+0000","last_clean":"2024-05-10T05:33:14.898545+0000","last_became_active":"2024-05-10T05:31:51.580620+0000","last_became_peered":"2024-05-10T05:31:51.580620+0000","last_unstale":"2024-05-10T05:33:14.898545+0000","last_undegraded":"2024-05-10T05:33:14.898545+0000","last_fullsized":"2024-05-10T05:33:14.898545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:30:15.378179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913340+0000","last_change":"2024-05-10T05:31:50.987633+0000","last_active":"2024-05-10T05:32:14.913340+0000","last_peered":"2024-05-10T05:32:14.913340+0000","last_clean":"2024-05-10T05:32:14.913340+0000","last_became_active":"2024-05-10T05:31:50.986750+0000","last_became_peered":"2024-05-10T05:31:50.986750+0000","last_unstale":"2024-05-10T05:32:14.913340+0000","last_undegraded":"2024-05-10T05:32:14.913340+0000","last_fullsized":"2024-05-10T05:32:14.913340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:17:47.705415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892509+0000","last_change":"2024-05-10T05:31:51.589484+0000","last_active":"2024-05-10T05:33:14.892509+0000","last_peered":"2024-05-10T05:33:14.892509+0000","last_clean":"2024-05-10T05:33:14.892509+0000","last_became_active":"2024-05-10T05:31:51.588992+0000","last_became_peered":"2024-05-10T05:31:51.588992+0000","last_unstale":"2024-05-10T05:33:14.892509+0000","last_undegraded":"2024-05-10T05:33:14.892509+0000","last_fullsized":"2024-05-10T05:33:14.892509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:32.360333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":195,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:25.624293+0000","last_change":"2024-05-10T05:31:50.988945+0000","last_active":"2024-05-10T05:33:25.624293+0000","last_peered":"2024-05-10T05:33:25.624293+0000","last_clean":"2024-05-10T05:33:25.624293+0000","last_became_active":"2024-05-10T05:31:50.988566+0000","last_became_peered":"2024-05-10T05:31:50.988566+0000","last_unstale":"2024-05-10T05:33:25.624293+0000","last_undegraded":"2024-05-10T05:33:25.624293+0000","last_fullsized":"2024-05-10T05:33:25.624293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:06:43.225346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888657+0000","last_change":"2024-05-10T05:31:51.577949+0000","last_active":"2024-05-10T05:33:14.888657+0000","last_peered":"2024-05-10T05:33:14.888657+0000","last_clean":"2024-05-10T05:33:14.888657+0000","last_became_active":"2024-05-10T05:31:51.577011+0000","last_became_peered":"2024-05-10T05:31:51.577011+0000","last_unstale":"2024-05-10T05:33:14.888657+0000","last_undegraded":"2024-05-10T05:33:14.888657+0000","last_fullsized":"2024-05-10T05:33:14.888657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:09.988700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.920114+0000","last_change":"2024-05-10T05:31:50.984888+0000","last_active":"2024-05-10T05:32:14.920114+0000","last_peered":"2024-05-10T05:32:14.920114+0000","last_clean":"2024-05-10T05:32:14.920114+0000","last_became_active":"2024-05-10T05:31:50.984267+0000","last_became_peered":"2024-05-10T05:31:50.984267+0000","last_unstale":"2024-05-10T05:32:14.920114+0000","last_undegraded":"2024-05-10T05:32:14.920114+0000","last_fullsized":"2024-05-10T05:32:14.920114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:44.797469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.900759+0000","last_change":"2024-05-10T05:31:51.576863+0000","last_active":"2024-05-10T05:33:14.900759+0000","last_peered":"2024-05-10T05:33:14.900759+0000","last_clean":"2024-05-10T05:33:14.900759+0000","last_became_active":"2024-05-10T05:31:51.576468+0000","last_became_peered":"2024-05-10T05:31:51.576468+0000","last_unstale":"2024-05-10T05:33:14.900759+0000","last_undegraded":"2024-05-10T05:33:14.900759+0000","last_fullsized":"2024-05-10T05:33:14.900759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:20.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913803+0000","last_change":"2024-05-10T05:31:51.001566+0000","last_active":"2024-05-10T05:32:14.913803+0000","last_peered":"2024-05-10T05:32:14.913803+0000","last_clean":"2024-05-10T05:32:14.913803+0000","last_became_active":"2024-05-10T05:31:51.001203+0000","last_became_peered":"2024-05-10T05:31:51.001203+0000","last_unstale":"2024-05-10T05:32:14.913803+0000","last_undegraded":"2024-05-10T05:32:14.913803+0000","last_fullsized":"2024-05-10T05:32:14.913803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:17:45.465039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.883700+0000","last_change":"2024-05-10T05:31:51.579114+0000","last_active":"2024-05-10T05:33:14.883700+0000","last_peered":"2024-05-10T05:33:14.883700+0000","last_clean":"2024-05-10T05:33:14.883700+0000","last_became_active":"2024-05-10T05:31:51.577507+0000","last_became_peered":"2024-05-10T05:31:51.577507+0000","last_unstale":"2024-05-10T05:33:14.883700+0000","last_undegraded":"2024-05-10T05:33:14.883700+0000","last_fullsized":"2024-05-10T05:33:14.883700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:43.777983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'35","reported_seq":235,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:25.624677+0000","last_change":"2024-05-10T05:31:50.983558+0000","last_active":"2024-05-10T05:33:25.624677+0000","last_peered":"2024-05-10T05:33:25.624677+0000","last_clean":"2024-05-10T05:33:25.624677+0000","last_became_active":"2024-05-10T05:31:50.983087+0000","last_became_peered":"2024-05-10T05:31:50.983087+0000","last_unstale":"2024-05-10T05:33:25.624677+0000","last_undegraded":"2024-05-10T05:33:25.624677+0000","last_fullsized":"2024-05-10T05:33:25.624677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:08.753898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65616,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":158,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.889800+0000","last_change":"2024-05-10T05:31:51.574024+0000","last_active":"2024-05-10T05:33:14.889800+0000","last_peered":"2024-05-10T05:33:14.889800+0000","last_clean":"2024-05-10T05:33:14.889800+0000","last_became_active":"2024-05-10T05:31:51.573703+0000","last_became_peered":"2024-05-10T05:31:51.573703+0000","last_unstale":"2024-05-10T05:33:14.889800+0000","last_undegraded":"2024-05-10T05:33:14.889800+0000","last_fullsized":"2024-05-10T05:33:14.889800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:55:23.435991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918608+0000","last_change":"2024-05-10T05:31:50.986795+0000","last_active":"2024-05-10T05:32:14.918608+0000","last_peered":"2024-05-10T05:32:14.918608+0000","last_clean":"2024-05-10T05:32:14.918608+0000","last_became_active":"2024-05-10T05:31:50.986428+0000","last_became_peered":"2024-05-10T05:31:50.986428+0000","last_unstale":"2024-05-10T05:32:14.918608+0000","last_undegraded":"2024-05-10T05:32:14.918608+0000","last_fullsized":"2024-05-10T05:32:14.918608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:11.039632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.894052+0000","last_change":"2024-05-10T05:31:51.585595+0000","last_active":"2024-05-10T05:33:14.894052+0000","last_peered":"2024-05-10T05:33:14.894052+0000","last_clean":"2024-05-10T05:33:14.894052+0000","last_became_active":"2024-05-10T05:31:51.585091+0000","last_became_peered":"2024-05-10T05:31:51.585091+0000","last_unstale":"2024-05-10T05:33:14.894052+0000","last_undegraded":"2024-05-10T05:33:14.894052+0000","last_fullsized":"2024-05-10T05:33:14.894052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:10:10.489430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913631+0000","last_change":"2024-05-10T05:31:50.984408+0000","last_active":"2024-05-10T05:32:14.913631+0000","last_peered":"2024-05-10T05:32:14.913631+0000","last_clean":"2024-05-10T05:32:14.913631+0000","last_became_active":"2024-05-10T05:31:50.983098+0000","last_became_peered":"2024-05-10T05:31:50.983098+0000","last_unstale":"2024-05-10T05:32:14.913631+0000","last_undegraded":"2024-05-10T05:32:14.913631+0000","last_fullsized":"2024-05-10T05:32:14.913631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:31.399649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.893997+0000","last_change":"2024-05-10T05:31:51.576160+0000","last_active":"2024-05-10T05:33:14.893997+0000","last_peered":"2024-05-10T05:33:14.893997+0000","last_clean":"2024-05-10T05:33:14.893997+0000","last_became_active":"2024-05-10T05:31:51.575806+0000","last_became_peered":"2024-05-10T05:31:51.575806+0000","last_unstale":"2024-05-10T05:33:14.893997+0000","last_undegraded":"2024-05-10T05:33:14.893997+0000","last_fullsized":"2024-05-10T05:33:14.893997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:53:25.660988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":114,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:26.625959+0000","last_change":"2024-05-10T05:31:50.982752+0000","last_active":"2024-05-10T05:33:26.625959+0000","last_peered":"2024-05-10T05:33:26.625959+0000","last_clean":"2024-05-10T05:33:26.625959+0000","last_became_active":"2024-05-10T05:31:50.982299+0000","last_became_peered":"2024-05-10T05:31:50.982299+0000","last_unstale":"2024-05-10T05:33:26.625959+0000","last_undegraded":"2024-05-10T05:33:26.625959+0000","last_fullsized":"2024-05-10T05:33:26.625959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:52:40.206967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892923+0000","last_change":"2024-05-10T05:31:51.577081+0000","last_active":"2024-05-10T05:33:14.892923+0000","last_peered":"2024-05-10T05:33:14.892923+0000","last_clean":"2024-05-10T05:33:14.892923+0000","last_became_active":"2024-05-10T05:31:51.576729+0000","last_became_peered":"2024-05-10T05:31:51.576729+0000","last_unstale":"2024-05-10T05:33:14.892923+0000","last_undegraded":"2024-05-10T05:33:14.892923+0000","last_fullsized":"2024-05-10T05:33:14.892923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:14.072427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915219+0000","last_change":"2024-05-10T05:31:50.991868+0000","last_active":"2024-05-10T05:32:14.915219+0000","last_peered":"2024-05-10T05:32:14.915219+0000","last_clean":"2024-05-10T05:32:14.915219+0000","last_became_active":"2024-05-10T05:31:50.991471+0000","last_became_peered":"2024-05-10T05:31:50.991471+0000","last_unstale":"2024-05-10T05:32:14.915219+0000","last_undegraded":"2024-05-10T05:32:14.915219+0000","last_fullsized":"2024-05-10T05:32:14.915219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:21:22.346376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.873664+0000","last_change":"2024-05-10T05:31:51.584726+0000","last_active":"2024-05-10T05:33:14.873664+0000","last_peered":"2024-05-10T05:33:14.873664+0000","last_clean":"2024-05-10T05:33:14.873664+0000","last_became_active":"2024-05-10T05:31:51.584327+0000","last_became_peered":"2024-05-10T05:31:51.584327+0000","last_unstale":"2024-05-10T05:33:14.873664+0000","last_undegraded":"2024-05-10T05:33:14.873664+0000","last_fullsized":"2024-05-10T05:33:14.873664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:56:43.561717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915510+0000","last_change":"2024-05-10T05:31:50.989071+0000","last_active":"2024-05-10T05:32:14.915510+0000","last_peered":"2024-05-10T05:32:14.915510+0000","last_clean":"2024-05-10T05:32:14.915510+0000","last_became_active":"2024-05-10T05:31:50.988044+0000","last_became_peered":"2024-05-10T05:31:50.988044+0000","last_unstale":"2024-05-10T05:32:14.915510+0000","last_undegraded":"2024-05-10T05:32:14.915510+0000","last_fullsized":"2024-05-10T05:32:14.915510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:24.122564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.884441+0000","last_change":"2024-05-10T05:31:51.576793+0000","last_active":"2024-05-10T05:33:14.884441+0000","last_peered":"2024-05-10T05:33:14.884441+0000","last_clean":"2024-05-10T05:33:14.884441+0000","last_became_active":"2024-05-10T05:31:51.576333+0000","last_became_peered":"2024-05-10T05:31:51.576333+0000","last_unstale":"2024-05-10T05:33:14.884441+0000","last_undegraded":"2024-05-10T05:33:14.884441+0000","last_fullsized":"2024-05-10T05:33:14.884441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:42.418736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913436+0000","last_change":"2024-05-10T05:31:50.997233+0000","last_active":"2024-05-10T05:32:14.913436+0000","last_peered":"2024-05-10T05:32:14.913436+0000","last_clean":"2024-05-10T05:32:14.913436+0000","last_became_active":"2024-05-10T05:31:50.996857+0000","last_became_peered":"2024-05-10T05:31:50.996857+0000","last_unstale":"2024-05-10T05:32:14.913436+0000","last_undegraded":"2024-05-10T05:32:14.913436+0000","last_fullsized":"2024-05-10T05:32:14.913436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:35.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.887621+0000","last_change":"2024-05-10T05:32:15.927682+0000","last_active":"2024-05-10T05:33:14.887621+0000","last_peered":"2024-05-10T05:33:14.887621+0000","last_clean":"2024-05-10T05:33:14.887621+0000","last_became_active":"2024-05-10T05:32:15.924197+0000","last_became_peered":"2024-05-10T05:32:15.924197+0000","last_unstale":"2024-05-10T05:33:14.887621+0000","last_undegraded":"2024-05-10T05:33:14.887621+0000","last_fullsized":"2024-05-10T05:33:14.887621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:14.747603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:24.622894+0000","last_change":"2024-05-10T05:31:50.991113+0000","last_active":"2024-05-10T05:33:24.622894+0000","last_peered":"2024-05-10T05:33:24.622894+0000","last_clean":"2024-05-10T05:33:24.622894+0000","last_became_active":"2024-05-10T05:31:50.990745+0000","last_became_peered":"2024-05-10T05:31:50.990745+0000","last_unstale":"2024-05-10T05:33:24.622894+0000","last_undegraded":"2024-05-10T05:33:24.622894+0000","last_fullsized":"2024-05-10T05:33:24.622894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:49:56.586231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.897318+0000","last_change":"2024-05-10T05:31:51.582057+0000","last_active":"2024-05-10T05:33:14.897318+0000","last_peered":"2024-05-10T05:33:14.897318+0000","last_clean":"2024-05-10T05:33:14.897318+0000","last_became_active":"2024-05-10T05:31:51.581672+0000","last_became_peered":"2024-05-10T05:31:51.581672+0000","last_unstale":"2024-05-10T05:33:14.897318+0000","last_undegraded":"2024-05-10T05:33:14.897318+0000","last_fullsized":"2024-05-10T05:33:14.897318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:45:09.122256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.877986+0000","last_change":"2024-05-10T05:31:51.585659+0000","last_active":"2024-05-10T05:33:14.877986+0000","last_peered":"2024-05-10T05:33:14.877986+0000","last_clean":"2024-05-10T05:33:14.877986+0000","last_became_active":"2024-05-10T05:31:51.584540+0000","last_became_peered":"2024-05-10T05:31:51.584540+0000","last_unstale":"2024-05-10T05:33:14.877986+0000","last_undegraded":"2024-05-10T05:33:14.877986+0000","last_fullsized":"2024-05-10T05:33:14.877986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:20.195369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":111,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:26.625190+0000","last_change":"2024-05-10T05:31:50.993547+0000","last_active":"2024-05-10T05:33:26.625190+0000","last_peered":"2024-05-10T05:33:26.625190+0000","last_clean":"2024-05-10T05:33:26.625190+0000","last_became_active":"2024-05-10T05:31:50.993190+0000","last_became_peered":"2024-05-10T05:31:50.993190+0000","last_unstale":"2024-05-10T05:33:26.625190+0000","last_undegraded":"2024-05-10T05:33:26.625190+0000","last_fullsized":"2024-05-10T05:33:26.625190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:01:06.159572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.890795+0000","last_change":"2024-05-10T05:31:51.587678+0000","last_active":"2024-05-10T05:33:14.890795+0000","last_peered":"2024-05-10T05:33:14.890795+0000","last_clean":"2024-05-10T05:33:14.890795+0000","last_became_active":"2024-05-10T05:31:51.586999+0000","last_became_peered":"2024-05-10T05:31:51.586999+0000","last_unstale":"2024-05-10T05:33:14.890795+0000","last_undegraded":"2024-05-10T05:33:14.890795+0000","last_fullsized":"2024-05-10T05:33:14.890795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:18:00.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914737+0000","last_change":"2024-05-10T05:31:50.979131+0000","last_active":"2024-05-10T05:32:14.914737+0000","last_peered":"2024-05-10T05:32:14.914737+0000","last_clean":"2024-05-10T05:32:14.914737+0000","last_became_active":"2024-05-10T05:31:50.978715+0000","last_became_peered":"2024-05-10T05:31:50.978715+0000","last_unstale":"2024-05-10T05:32:14.914737+0000","last_undegraded":"2024-05-10T05:32:14.914737+0000","last_fullsized":"2024-05-10T05:32:14.914737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:29:11.953963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919607+0000","last_change":"2024-05-10T05:30:59.356688+0000","last_active":"2024-05-10T05:32:14.919607+0000","last_peered":"2024-05-10T05:32:14.919607+0000","last_clean":"2024-05-10T05:32:14.919607+0000","last_became_active":"2024-05-10T05:30:59.356313+0000","last_became_peered":"2024-05-10T05:30:59.356313+0000","last_unstale":"2024-05-10T05:32:14.919607+0000","last_undegraded":"2024-05-10T05:32:14.919607+0000","last_fullsized":"2024-05-10T05:32:14.919607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:28:15.503591+0000","last_clean_scrub_stamp":"2024-05-10T05:28:15.503591+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:43.011495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.879926+0000","last_change":"2024-05-10T05:31:51.580259+0000","last_active":"2024-05-10T05:33:14.879926+0000","last_peered":"2024-05-10T05:33:14.879926+0000","last_clean":"2024-05-10T05:33:14.879926+0000","last_became_active":"2024-05-10T05:31:51.578650+0000","last_became_peered":"2024-05-10T05:31:51.578650+0000","last_unstale":"2024-05-10T05:33:14.879926+0000","last_undegraded":"2024-05-10T05:33:14.879926+0000","last_fullsized":"2024-05-10T05:33:14.879926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:41:48.630183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.919744+0000","last_change":"2024-05-10T05:31:50.987708+0000","last_active":"2024-05-10T05:32:14.919744+0000","last_peered":"2024-05-10T05:32:14.919744+0000","last_clean":"2024-05-10T05:32:14.919744+0000","last_became_active":"2024-05-10T05:31:50.987347+0000","last_became_peered":"2024-05-10T05:31:50.987347+0000","last_unstale":"2024-05-10T05:32:14.919744+0000","last_undegraded":"2024-05-10T05:32:14.919744+0000","last_fullsized":"2024-05-10T05:32:14.919744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:16:29.388690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.915952+0000","last_change":"2024-05-10T05:31:50.999338+0000","last_active":"2024-05-10T05:32:14.915952+0000","last_peered":"2024-05-10T05:32:14.915952+0000","last_clean":"2024-05-10T05:32:14.915952+0000","last_became_active":"2024-05-10T05:31:50.999026+0000","last_became_peered":"2024-05-10T05:31:50.999026+0000","last_unstale":"2024-05-10T05:32:14.915952+0000","last_undegraded":"2024-05-10T05:32:14.915952+0000","last_fullsized":"2024-05-10T05:32:14.915952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:31.443409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.880920+0000","last_change":"2024-05-10T05:31:51.574201+0000","last_active":"2024-05-10T05:33:14.880920+0000","last_peered":"2024-05-10T05:33:14.880920+0000","last_clean":"2024-05-10T05:33:14.880920+0000","last_became_active":"2024-05-10T05:31:51.571180+0000","last_became_peered":"2024-05-10T05:31:51.571180+0000","last_unstale":"2024-05-10T05:33:14.880920+0000","last_undegraded":"2024-05-10T05:33:14.880920+0000","last_fullsized":"2024-05-10T05:33:14.880920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:33:16.971822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:15.336605+0000","last_change":"2024-05-10T05:31:50.985164+0000","last_active":"2024-05-10T05:33:15.336605+0000","last_peered":"2024-05-10T05:33:15.336605+0000","last_clean":"2024-05-10T05:33:15.336605+0000","last_became_active":"2024-05-10T05:31:50.984760+0000","last_became_peered":"2024-05-10T05:31:50.984760+0000","last_unstale":"2024-05-10T05:33:15.336605+0000","last_undegraded":"2024-05-10T05:33:15.336605+0000","last_fullsized":"2024-05-10T05:33:15.336605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:19.827407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.894754+0000","last_change":"2024-05-10T05:31:51.577445+0000","last_active":"2024-05-10T05:33:14.894754+0000","last_peered":"2024-05-10T05:33:14.894754+0000","last_clean":"2024-05-10T05:33:14.894754+0000","last_became_active":"2024-05-10T05:31:51.576671+0000","last_became_peered":"2024-05-10T05:31:51.576671+0000","last_unstale":"2024-05-10T05:33:14.894754+0000","last_undegraded":"2024-05-10T05:33:14.894754+0000","last_fullsized":"2024-05-10T05:33:14.894754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:38.774538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.914419+0000","last_change":"2024-05-10T05:31:50.981495+0000","last_active":"2024-05-10T05:32:14.914419+0000","last_peered":"2024-05-10T05:32:14.914419+0000","last_clean":"2024-05-10T05:32:14.914419+0000","last_became_active":"2024-05-10T05:31:50.981127+0000","last_became_peered":"2024-05-10T05:31:50.981127+0000","last_unstale":"2024-05-10T05:32:14.914419+0000","last_undegraded":"2024-05-10T05:32:14.914419+0000","last_fullsized":"2024-05-10T05:32:14.914419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:22:32.257014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902371+0000","last_change":"2024-05-10T05:31:51.585926+0000","last_active":"2024-05-10T05:33:14.902371+0000","last_peered":"2024-05-10T05:33:14.902371+0000","last_clean":"2024-05-10T05:33:14.902371+0000","last_became_active":"2024-05-10T05:31:51.585576+0000","last_became_peered":"2024-05-10T05:31:51.585576+0000","last_unstale":"2024-05-10T05:33:14.902371+0000","last_undegraded":"2024-05-10T05:33:14.902371+0000","last_fullsized":"2024-05-10T05:33:14.902371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:38:22.344180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.926062+0000","last_change":"2024-05-10T05:32:15.926062+0000","last_active":"2024-05-10T05:32:15.926062+0000","last_peered":"2024-05-10T05:32:15.926062+0000","last_clean":"2024-05-10T05:32:15.926062+0000","last_became_active":"2024-05-10T05:32:15.923026+0000","last_became_peered":"2024-05-10T05:32:15.923026+0000","last_unstale":"2024-05-10T05:32:15.926062+0000","last_undegraded":"2024-05-10T05:32:15.926062+0000","last_fullsized":"2024-05-10T05:32:15.926062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:34.188115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.848777+0000","last_change":"2024-05-10T05:31:51.573279+0000","last_active":"2024-05-10T05:33:14.848777+0000","last_peered":"2024-05-10T05:33:14.848777+0000","last_clean":"2024-05-10T05:33:14.848777+0000","last_became_active":"2024-05-10T05:31:51.572916+0000","last_became_peered":"2024-05-10T05:31:51.572916+0000","last_unstale":"2024-05-10T05:33:14.848777+0000","last_undegraded":"2024-05-10T05:33:14.848777+0000","last_fullsized":"2024-05-10T05:33:14.848777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:32:39.802463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.918101+0000","last_change":"2024-05-10T05:31:50.983771+0000","last_active":"2024-05-10T05:32:14.918101+0000","last_peered":"2024-05-10T05:32:14.918101+0000","last_clean":"2024-05-10T05:32:14.918101+0000","last_became_active":"2024-05-10T05:31:50.981849+0000","last_became_peered":"2024-05-10T05:31:50.981849+0000","last_unstale":"2024-05-10T05:32:14.918101+0000","last_undegraded":"2024-05-10T05:32:14.918101+0000","last_fullsized":"2024-05-10T05:32:14.918101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:00:15.990869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.878726+0000","last_change":"2024-05-10T05:31:51.568005+0000","last_active":"2024-05-10T05:33:14.878726+0000","last_peered":"2024-05-10T05:33:14.878726+0000","last_clean":"2024-05-10T05:33:14.878726+0000","last_became_active":"2024-05-10T05:31:51.567347+0000","last_became_peered":"2024-05-10T05:31:51.567347+0000","last_unstale":"2024-05-10T05:33:14.878726+0000","last_undegraded":"2024-05-10T05:33:14.878726+0000","last_fullsized":"2024-05-10T05:33:14.878726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:29.075816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:32:15.926267+0000","last_change":"2024-05-10T05:32:15.925869+0000","last_active":"2024-05-10T05:32:15.926267+0000","last_peered":"2024-05-10T05:32:15.926267+0000","last_clean":"2024-05-10T05:32:15.926267+0000","last_became_active":"2024-05-10T05:32:15.920591+0000","last_became_peered":"2024-05-10T05:32:15.920591+0000","last_unstale":"2024-05-10T05:32:15.926267+0000","last_undegraded":"2024-05-10T05:32:15.926267+0000","last_fullsized":"2024-05-10T05:32:15.926267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:13:20.398637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.901352+0000","last_change":"2024-05-10T05:31:51.588255+0000","last_active":"2024-05-10T05:33:14.901352+0000","last_peered":"2024-05-10T05:33:14.901352+0000","last_clean":"2024-05-10T05:33:14.901352+0000","last_became_active":"2024-05-10T05:31:51.587912+0000","last_became_peered":"2024-05-10T05:31:51.587912+0000","last_unstale":"2024-05-10T05:33:14.901352+0000","last_undegraded":"2024-05-10T05:33:14.901352+0000","last_fullsized":"2024-05-10T05:33:14.901352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:18.515739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.916229+0000","last_change":"2024-05-10T05:31:50.992392+0000","last_active":"2024-05-10T05:32:14.916229+0000","last_peered":"2024-05-10T05:32:14.916229+0000","last_clean":"2024-05-10T05:32:14.916229+0000","last_became_active":"2024-05-10T05:31:50.992119+0000","last_became_peered":"2024-05-10T05:31:50.992119+0000","last_unstale":"2024-05-10T05:32:14.916229+0000","last_undegraded":"2024-05-10T05:32:14.916229+0000","last_fullsized":"2024-05-10T05:32:14.916229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:11.515013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.893189+0000","last_change":"2024-05-10T05:31:51.567780+0000","last_active":"2024-05-10T05:33:14.893189+0000","last_peered":"2024-05-10T05:33:14.893189+0000","last_clean":"2024-05-10T05:33:14.893189+0000","last_became_active":"2024-05-10T05:31:51.567455+0000","last_became_peered":"2024-05-10T05:31:51.567455+0000","last_unstale":"2024-05-10T05:33:14.893189+0000","last_undegraded":"2024-05-10T05:33:14.893189+0000","last_fullsized":"2024-05-10T05:33:14.893189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:47.108131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":113,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:28.629661+0000","last_change":"2024-05-10T05:31:50.997374+0000","last_active":"2024-05-10T05:33:28.629661+0000","last_peered":"2024-05-10T05:33:28.629661+0000","last_clean":"2024-05-10T05:33:28.629661+0000","last_became_active":"2024-05-10T05:31:50.997069+0000","last_became_peered":"2024-05-10T05:31:50.997069+0000","last_unstale":"2024-05-10T05:33:28.629661+0000","last_undegraded":"2024-05-10T05:33:28.629661+0000","last_fullsized":"2024-05-10T05:33:28.629661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:25:49.140006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.889341+0000","last_change":"2024-05-10T05:31:51.576345+0000","last_active":"2024-05-10T05:33:14.889341+0000","last_peered":"2024-05-10T05:33:14.889341+0000","last_clean":"2024-05-10T05:33:14.889341+0000","last_became_active":"2024-05-10T05:31:51.575704+0000","last_became_peered":"2024-05-10T05:31:51.575704+0000","last_unstale":"2024-05-10T05:33:14.889341+0000","last_undegraded":"2024-05-10T05:33:14.889341+0000","last_fullsized":"2024-05-10T05:33:14.889341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:15.146591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.913524+0000","last_change":"2024-05-10T05:31:50.984108+0000","last_active":"2024-05-10T05:32:14.913524+0000","last_peered":"2024-05-10T05:32:14.913524+0000","last_clean":"2024-05-10T05:32:14.913524+0000","last_became_active":"2024-05-10T05:31:50.983728+0000","last_became_peered":"2024-05-10T05:31:50.983728+0000","last_unstale":"2024-05-10T05:32:14.913524+0000","last_undegraded":"2024-05-10T05:32:14.913524+0000","last_fullsized":"2024-05-10T05:32:14.913524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:23:20.096286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.895124+0000","last_change":"2024-05-10T05:31:51.572075+0000","last_active":"2024-05-10T05:33:14.895124+0000","last_peered":"2024-05-10T05:33:14.895124+0000","last_clean":"2024-05-10T05:33:14.895124+0000","last_became_active":"2024-05-10T05:31:51.571711+0000","last_became_peered":"2024-05-10T05:31:51.571711+0000","last_unstale":"2024-05-10T05:33:14.895124+0000","last_undegraded":"2024-05-10T05:33:14.895124+0000","last_fullsized":"2024-05-10T05:33:14.895124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:38:27.813001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-10T05:32:14.907701+0000","last_change":"2024-05-10T05:31:50.983985+0000","last_active":"2024-05-10T05:32:14.907701+0000","last_peered":"2024-05-10T05:32:14.907701+0000","last_clean":"2024-05-10T05:32:14.907701+0000","last_became_active":"2024-05-10T05:31:50.982931+0000","last_became_peered":"2024-05-10T05:31:50.982931+0000","last_unstale":"2024-05-10T05:32:14.907701+0000","last_undegraded":"2024-05-10T05:32:14.907701+0000","last_fullsized":"2024-05-10T05:32:14.907701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:49.914674+0000","last_clean_scrub_stamp":"2024-05-10T05:31:49.914674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:35:17.158696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.888805+0000","last_change":"2024-05-10T05:31:51.587250+0000","last_active":"2024-05-10T05:33:14.888805+0000","last_peered":"2024-05-10T05:33:14.888805+0000","last_clean":"2024-05-10T05:33:14.888805+0000","last_became_active":"2024-05-10T05:31:51.586925+0000","last_became_peered":"2024-05-10T05:31:51.586925+0000","last_unstale":"2024-05-10T05:33:14.888805+0000","last_undegraded":"2024-05-10T05:33:14.888805+0000","last_fullsized":"2024-05-10T05:33:14.888805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:40.585803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.879173+0000","last_change":"2024-05-10T05:31:51.567710+0000","last_active":"2024-05-10T05:33:14.879173+0000","last_peered":"2024-05-10T05:33:14.879173+0000","last_clean":"2024-05-10T05:33:14.879173+0000","last_became_active":"2024-05-10T05:31:51.566860+0000","last_became_peered":"2024-05-10T05:31:51.566860+0000","last_unstale":"2024-05-10T05:33:14.879173+0000","last_undegraded":"2024-05-10T05:33:14.879173+0000","last_fullsized":"2024-05-10T05:33:14.879173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:02:09.253345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.879510+0000","last_change":"2024-05-10T05:31:51.577458+0000","last_active":"2024-05-10T05:33:14.879510+0000","last_peered":"2024-05-10T05:33:14.879510+0000","last_clean":"2024-05-10T05:33:14.879510+0000","last_became_active":"2024-05-10T05:31:51.577123+0000","last_became_peered":"2024-05-10T05:31:51.577123+0000","last_unstale":"2024-05-10T05:33:14.879510+0000","last_undegraded":"2024-05-10T05:33:14.879510+0000","last_fullsized":"2024-05-10T05:33:14.879510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:51.791090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902636+0000","last_change":"2024-05-10T05:31:51.589361+0000","last_active":"2024-05-10T05:33:14.902636+0000","last_peered":"2024-05-10T05:33:14.902636+0000","last_clean":"2024-05-10T05:33:14.902636+0000","last_became_active":"2024-05-10T05:31:51.589049+0000","last_became_peered":"2024-05-10T05:31:51.589049+0000","last_unstale":"2024-05-10T05:33:14.902636+0000","last_undegraded":"2024-05-10T05:33:14.902636+0000","last_fullsized":"2024-05-10T05:33:14.902636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:13.380505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.902342+0000","last_change":"2024-05-10T05:32:15.919725+0000","last_active":"2024-05-10T05:33:14.902342+0000","last_peered":"2024-05-10T05:33:14.902342+0000","last_clean":"2024-05-10T05:33:14.902342+0000","last_became_active":"2024-05-10T05:32:15.919342+0000","last_became_peered":"2024-05-10T05:32:15.919342+0000","last_unstale":"2024-05-10T05:33:14.902342+0000","last_undegraded":"2024-05-10T05:33:14.902342+0000","last_fullsized":"2024-05-10T05:33:14.902342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:33.785843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.891254+0000","last_change":"2024-05-10T05:31:51.579791+0000","last_active":"2024-05-10T05:33:14.891254+0000","last_peered":"2024-05-10T05:33:14.891254+0000","last_clean":"2024-05-10T05:33:14.891254+0000","last_became_active":"2024-05-10T05:31:51.577635+0000","last_became_peered":"2024-05-10T05:31:51.577635+0000","last_unstale":"2024-05-10T05:33:14.891254+0000","last_undegraded":"2024-05-10T05:33:14.891254+0000","last_fullsized":"2024-05-10T05:33:14.891254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:54.077805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.873675+0000","last_change":"2024-05-10T05:31:51.584601+0000","last_active":"2024-05-10T05:33:14.873675+0000","last_peered":"2024-05-10T05:33:14.873675+0000","last_clean":"2024-05-10T05:33:14.873675+0000","last_became_active":"2024-05-10T05:31:51.584172+0000","last_became_peered":"2024-05-10T05:31:51.584172+0000","last_unstale":"2024-05-10T05:33:14.873675+0000","last_undegraded":"2024-05-10T05:33:14.873675+0000","last_fullsized":"2024-05-10T05:33:14.873675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:49:44.987379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.884792+0000","last_change":"2024-05-10T05:31:51.573784+0000","last_active":"2024-05-10T05:33:14.884792+0000","last_peered":"2024-05-10T05:33:14.884792+0000","last_clean":"2024-05-10T05:33:14.884792+0000","last_became_active":"2024-05-10T05:31:51.573279+0000","last_became_peered":"2024-05-10T05:31:51.573279+0000","last_unstale":"2024-05-10T05:33:14.884792+0000","last_undegraded":"2024-05-10T05:33:14.884792+0000","last_fullsized":"2024-05-10T05:33:14.884792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:08.375964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.881529+0000","last_change":"2024-05-10T05:31:51.585577+0000","last_active":"2024-05-10T05:33:14.881529+0000","last_peered":"2024-05-10T05:33:14.881529+0000","last_clean":"2024-05-10T05:33:14.881529+0000","last_became_active":"2024-05-10T05:31:51.585222+0000","last_became_peered":"2024-05-10T05:31:51.585222+0000","last_unstale":"2024-05-10T05:33:14.881529+0000","last_undegraded":"2024-05-10T05:33:14.881529+0000","last_fullsized":"2024-05-10T05:33:14.881529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:47:27.069513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.892962+0000","last_change":"2024-05-10T05:31:51.571405+0000","last_active":"2024-05-10T05:33:14.892962+0000","last_peered":"2024-05-10T05:33:14.892962+0000","last_clean":"2024-05-10T05:33:14.892962+0000","last_became_active":"2024-05-10T05:31:51.571125+0000","last_became_peered":"2024-05-10T05:31:51.571125+0000","last_unstale":"2024-05-10T05:33:14.892962+0000","last_undegraded":"2024-05-10T05:33:14.892962+0000","last_fullsized":"2024-05-10T05:33:14.892962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:49:29.069121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.895256+0000","last_change":"2024-05-10T05:31:51.578883+0000","last_active":"2024-05-10T05:33:14.895256+0000","last_peered":"2024-05-10T05:33:14.895256+0000","last_clean":"2024-05-10T05:33:14.895256+0000","last_became_active":"2024-05-10T05:31:51.578582+0000","last_became_peered":"2024-05-10T05:31:51.578582+0000","last_unstale":"2024-05-10T05:33:14.895256+0000","last_undegraded":"2024-05-10T05:33:14.895256+0000","last_fullsized":"2024-05-10T05:33:14.895256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:16.402282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.891358+0000","last_change":"2024-05-10T05:31:51.589269+0000","last_active":"2024-05-10T05:33:14.891358+0000","last_peered":"2024-05-10T05:33:14.891358+0000","last_clean":"2024-05-10T05:33:14.891358+0000","last_became_active":"2024-05-10T05:31:51.588875+0000","last_became_peered":"2024-05-10T05:31:51.588875+0000","last_unstale":"2024-05-10T05:33:14.891358+0000","last_undegraded":"2024-05-10T05:33:14.891358+0000","last_fullsized":"2024-05-10T05:33:14.891358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:12.536555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.901574+0000","last_change":"2024-05-10T05:31:51.567789+0000","last_active":"2024-05-10T05:33:14.901574+0000","last_peered":"2024-05-10T05:33:14.901574+0000","last_clean":"2024-05-10T05:33:14.901574+0000","last_became_active":"2024-05-10T05:31:51.566625+0000","last_became_peered":"2024-05-10T05:31:51.566625+0000","last_unstale":"2024-05-10T05:33:14.901574+0000","last_undegraded":"2024-05-10T05:33:14.901574+0000","last_fullsized":"2024-05-10T05:33:14.901574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:43.042557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T05:33:14.876173+0000","last_change":"2024-05-10T05:31:51.585838+0000","last_active":"2024-05-10T05:33:14.876173+0000","last_peered":"2024-05-10T05:33:14.876173+0000","last_clean":"2024-05-10T05:33:14.876173+0000","last_became_active":"2024-05-10T05:31:51.585473+0000","last_became_peered":"2024-05-10T05:31:51.585473+0000","last_unstale":"2024-05-10T05:33:14.876173+0000","last_undegraded":"2024-05-10T05:33:14.876173+0000","last_fullsized":"2024-05-10T05:33:14.876173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T05:31:50.960927+0000","last_clean_scrub_stamp":"2024-05-10T05:31:50.960927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:11:11.282076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247530,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72914,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":935,"num_read_kb":546,"num_write":247,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":218742,"data_compressed":5520,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":429949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.52500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63900000000000001}]},{"osd":2,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.67900000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.36299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70599999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.66900000000000004}]},{"osd":3,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.77900000000000003}]},{"osd":4,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56799999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97599999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.50600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88300000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.79800000000000004}]},{"osd":5,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89600000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88300000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.86899999999999999}]},{"osd":6,"last update":"Fri May 10 05:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.64000000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.03,"5min":1.127,"15min":1.127},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76300000000000001,"15min":0.76800000000000002},"min":{"1min":0.61799999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98499999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.92100000000000004}]},{"osd":8,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.90100000000000002}]},{"osd":9,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84099999999999997,"15min":0.84499999999999997},"min":{"1min":0.68000000000000005,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96799999999999997,"5min":1.514,"15min":1.514},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.82599999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.004,"5min":1.216,"15min":1.216},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri May 10 05:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.71399999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94199999999999995,"5min":1.254,"15min":1.254},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.86099999999999999,"15min":0.86299999999999999},"min":{"1min":0.69199999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93899999999999995,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.95599999999999996}]}]},{"osd":10,"up_from":68,"seq":292057776161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1087128,"data_compressed":37906,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64500000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.871,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81100000000000005,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.747}]},{"osd":3,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.63500000000000001,"15min":0.64800000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.70399999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.504,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86399999999999999,"5min":1.091,"15min":1.091},"last":0.83299999999999996}]},{"osd":4,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.48299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79400000000000004,"5min":1.036,"15min":1.036},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":1.173,"15min":1.173},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.99099999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.58199999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84499999999999997,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76900000000000002,"15min":0.78900000000000003},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78500000000000003,"5min":1.448,"15min":1.448},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.67700000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.91100000000000003,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":8,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81799999999999995,"15min":0.81999999999999995},"min":{"1min":0.70099999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.65300000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0740000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri May 10 05:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.88300000000000001,"15min":0.89100000000000001},"min":{"1min":0.67800000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0189999999999999,"5min":1.728,"15min":1.728},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.85099999999999998,"15min":0.86099999999999999},"min":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0289999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.878}]},{"osd":11,"last update":"Fri May 10 05:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.66000000000000003,"5min":0.628,"15min":0.628},"max":{"1min":0.94699999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.85999999999999999},"min":{"1min":0.72199999999999998,"5min":0.625,"15min":0.625},"max":{"1min":1.008,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.97299999999999998}]}]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":921600,"data_stored":430044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.629,"15min":0.64800000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.63}]},{"osd":1,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.02,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.75700000000000001}]},{"osd":2,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.871},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3759999999999999,"5min":1.488,"15min":1.488},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71999999999999997,"15min":0.73599999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86499999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1160000000000001,"5min":1.177,"15min":1.177},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.872,"15min":0.88800000000000001},"min":{"1min":0.53000000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.403,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.874,"15min":0.89400000000000002},"min":{"1min":0.58099999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0580000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.89900000000000002,"15min":0.91400000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1339999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.90500000000000003,"15min":0.92500000000000004},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.115,"5min":1.288,"15min":1.288},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.93500000000000005,"15min":0.95399999999999996},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.135,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4650000000000001,"5min":1.611,"15min":1.611},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.97899999999999998,"15min":1.0009999999999999},"min":{"1min":0.70599999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.196,"5min":1.46,"15min":1.46},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.2450000000000001,"5min":1.337,"15min":1.337},"last":0.91200000000000003}]},{"osd":8,"last update":"Fri May 10 05:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.998,"15min":1.008},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.427,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.997,"15min":1.0129999999999999},"min":{"1min":0.71599999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2170000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.78900000000000003}]},{"osd":10,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1,"15min":1.016},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.99399999999999999,"15min":1.014},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.032,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.85799999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":430305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.995,"15min":0.995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.65600000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.629,"15min":0.627},"min":{"1min":0.41299999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.626,"15min":0.627},"min":{"1min":0.52800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78700000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84399999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.876,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.624},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.54000000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93700000000000006,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89100000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.42199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91900000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.63200000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98599999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.754,"15min":0.753},"min":{"1min":0.54800000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.997,"5min":1.101,"15min":1.101},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.64300000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.96999999999999997,"5min":1.702,"15min":1.702},"last":0.84599999999999997}]},{"osd":9,"last update":"Fri May 10 05:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.76400000000000001,"15min":0.74299999999999999},"min":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.77900000000000003,"15min":0.76600000000000001},"min":{"1min":0.65500000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri May 10 05:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.70199999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.78400000000000003,"15min":0.76800000000000002},"min":{"1min":0.73299999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.79000000000000004}]},{"osd":11,"last update":"Fri May 10 05:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.873},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.81999999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":966656,"data_stored":1023356,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.438,"15min":0.433},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.44500000000000001},"min":{"1min":0.32900000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.39300000000000002}]},{"osd":2,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77900000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.85899999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.49099999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.73399999999999999}]},{"osd":4,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.496,"15min":0.49099999999999999},"min":{"1min":0.374,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.75,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67900000000000005,"15min":0.66600000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96099999999999997,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.53700000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84299999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86499999999999999}]},{"osd":8,"last update":"Fri May 10 05:33:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.60399999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92100000000000004,"5min":1.002,"15min":1.002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.71199999999999997,"15min":0.70399999999999996},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.748}]},{"osd":9,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.626,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.022,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.65600000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.89200000000000002}]},{"osd":10,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.77800000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.96799999999999997,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.68300000000000005}]},{"osd":11,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.69999999999999996}]}]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":901120,"data_stored":429596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67900000000000005,"15min":0.65100000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.59799999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98299999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.91900000000000004}]},{"osd":2,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72399999999999998,"15min":0.70799999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69399999999999995,"15min":0.69499999999999995},"min":{"1min":0.442,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97499999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.65800000000000003},"min":{"1min":0.41599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.77200000000000002},"min":{"1min":0.56899999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.083,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76700000000000002,"15min":0.75700000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0740000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.83799999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.63}]},{"osd":7,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.82999999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.81200000000000006},"min":{"1min":0.60099999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.86799999999999999}]},{"osd":8,"last update":"Fri May 10 05:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.90400000000000003,"15min":0.94199999999999995},"min":{"1min":0.622,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.99399999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91400000000000003,"15min":0.95499999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.367,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri May 10 05:32:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.91700000000000004,"15min":0.92200000000000004},"min":{"1min":0.52500000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.97599999999999998,"15min":0.99099999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2270000000000001,"5min":1.52,"15min":1.52},"last":0.88500000000000001}]},{"osd":10,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.999,"15min":1.0129999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.0740000000000001,"5min":1.458,"15min":1.458},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.93600000000000005,"15min":0.94099999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.81599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":962560,"data_stored":496125,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.628,"15min":0.79000000000000004},"min":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82899999999999996,"5min":6.3810000000000002,"15min":6.3810000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri May 10 05:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.60399999999999998,"15min":0.56799999999999995},"min":{"1min":0.48699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.625,"15min":0.58299999999999996},"min":{"1min":0.61699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri May 10 05:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78700000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.47499999999999998,"15min":0.44},"min":{"1min":0.40000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.501,"15min":0.45400000000000001},"min":{"1min":0.36199999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri May 10 05:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.61299999999999999,"15min":0.55800000000000005},"min":{"1min":0.55400000000000005,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.66700000000000004,"15min":0.61799999999999999},"min":{"1min":0.629,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri May 10 05:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68400000000000005,"15min":0.66500000000000004},"min":{"1min":0.46999999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.67400000000000004},"min":{"1min":0.56100000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.62,"15min":0.58999999999999997},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.46500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81899999999999995}]},{"osd":8,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.73299999999999998,"15min":0.69499999999999995},"min":{"1min":0.66100000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.71799999999999997,"15min":0.67200000000000004},"min":{"1min":0.64300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.04,"5min":1.113,"15min":1.113},"last":0.94499999999999995}]},{"osd":9,"last update":"Fri May 10 05:32:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.79700000000000004,"15min":0.79100000000000004},"min":{"1min":0.59499999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.1379999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.89600000000000002}]},{"osd":10,"last update":"Fri May 10 05:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.63,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0780000000000001,"5min":1.121,"15min":1.121},"last":0.98799999999999999}]},{"osd":11,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.071}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":921600,"data_stored":430935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 05:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.68600000000000005,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67300000000000004,"15min":0.61899999999999999},"min":{"1min":0.49299999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.849,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69699999999999995,"15min":0.63200000000000001},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96199999999999997,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.63800000000000001},"min":{"1min":0.46800000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.218,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri May 10 05:32:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.177,"5min":3.177,"15min":3.177},"last":0.40899999999999997}]},{"osd":4,"last update":"Fri May 10 05:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72899999999999998,"15min":0.67300000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9450000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.73899999999999999,"15min":0.67500000000000004},"min":{"1min":0.56999999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":0.496}]},{"osd":5,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73999999999999999,"15min":0.67000000000000004},"min":{"1min":0.53200000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.2789999999999999,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.78700000000000003,"15min":0.71199999999999997},"min":{"1min":0.496,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":6.069,"5min":6.069,"15min":6.069},"last":0.88400000000000001}]},{"osd":6,"last update":"Fri May 10 05:32:39 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.80500000000000005,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":4.859,"5min":4.859,"15min":4.859},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.76700000000000002,"15min":0.73999999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri May 10 05:30:58 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90700000000000003}]},{"osd":8,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.86599999999999999,"15min":0.86799999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.123,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.254,"5min":4.6559999999999997,"15min":4.6559999999999997},"last":0.68799999999999994}]},{"osd":9,"last update":"Fri May 10 05:32:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0349999999999999,"15min":1.0289999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.173,"5min":5.173,"15min":5.173},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.96899999999999997,"15min":0.97599999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"last":0.63400000000000001}]},{"osd":10,"last update":"Fri May 10 05:33:00 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.96199999999999997,"15min":0.93899999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.21,"5min":1.04,"15min":1.012},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":6.1189999999999998,"5min":6.1189999999999998,"15min":6.1189999999999998},"last":0.82299999999999995}]},{"osd":11,"last update":"Fri May 10 05:33:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0449999999999999,"15min":1.0429999999999999},"min":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":6.0949999999999998,"5min":6.0949999999999998,"15min":6.0949999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.99099999999999999,"15min":1.02},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0589999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.91600000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":983040,"data_stored":496831,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70599999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.51700000000000002},"min":{"1min":0.41299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81499999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83999999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.502},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.439,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri May 10 05:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.48999999999999999,"15min":0.44400000000000001},"min":{"1min":0.38200000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81499999999999995,"5min":1.026,"15min":1.026},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.47199999999999998},"min":{"1min":0.40100000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69399999999999995,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri May 10 05:32:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.64000000000000001},"min":{"1min":0.53100000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.66000000000000003},"min":{"1min":0.58099999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri May 10 05:30:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.745}]},{"osd":8,"last update":"Fri May 10 05:33:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.57099999999999995},"min":{"1min":0.46100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.97699999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.60699999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82399999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri May 10 05:32:40 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.71399999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.753},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.76900000000000002,"15min":0.752},"min":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.78800000000000003}]},{"osd":10,"last update":"Fri May 10 05:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79100000000000004,"15min":0.78900000000000003},"min":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0189999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.65000000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92800000000000005,"5min":1.121,"15min":1.121},"last":0.81100000000000005}]},{"osd":11,"last update":"Fri May 10 05:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.72399999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":429352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.45400000000000001,"15min":0.41199999999999998},"min":{"1min":0.34899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.46600000000000003,"15min":0.41799999999999998},"min":{"1min":0.38100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67000000000000004,"15min":0.63300000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.113,"5min":1.161,"15min":1.161},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91400000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73499999999999999}]},{"osd":2,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65700000000000003,"15min":0.60499999999999998},"min":{"1min":0.44600000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67800000000000005,"15min":0.63800000000000001},"min":{"1min":0.51900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0780000000000001,"5min":1.206,"15min":1.206},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri May 10 05:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.67900000000000005,"15min":0.64100000000000001},"min":{"1min":0.60299999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70599999999999996,"15min":0.66900000000000004},"min":{"1min":0.498,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99299999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.755}]},{"osd":5,"last update":"Fri May 10 05:33:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66900000000000004,"15min":0.64100000000000001},"min":{"1min":0.50600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84599999999999997,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.032,"5min":0.79400000000000004,"15min":0.72699999999999998},"min":{"1min":0.49199999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":7.4160000000000004,"5min":7.4160000000000004,"15min":7.4160000000000004},"last":0.95699999999999996}]},{"osd":6,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71599999999999997,"15min":0.68700000000000006},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri May 10 05:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79100000000000004,"15min":0.77300000000000002},"min":{"1min":0.61399999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.84399999999999997,"15min":0.82799999999999996},"min":{"1min":0.64700000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1579999999999999,"5min":1.216,"15min":1.216},"last":0.91700000000000004}]},{"osd":8,"last update":"Fri May 10 05:31:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.753}]},{"osd":9,"last update":"Fri May 10 05:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.95699999999999996,"15min":0.97399999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1919999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.879,"15min":0.89800000000000002},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.032,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.621}]},{"osd":10,"last update":"Fri May 10 05:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.98999999999999999,"15min":1.0089999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.226,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.88300000000000001}]},{"osd":11,"last update":"Fri May 10 05:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.82999999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":901120,"data_stored":429991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:31:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri May 10 05:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.37},"min":{"1min":0.34499999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.54500000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.441,"15min":0.40300000000000002},"min":{"1min":0.36499999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.629}]},{"osd":2,"last update":"Fri May 10 05:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.45500000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.53100000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82099999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri May 10 05:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.83699999999999997,"15min":0.65900000000000003},"min":{"1min":0.55100000000000005,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97599999999999998,"5min":13.608000000000001,"15min":13.608000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.52000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.83399999999999996,"5min":0.876,"15min":0.876},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri May 10 05:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.65800000000000003,"15min":0.62},"min":{"1min":0.57499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89100000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.61199999999999999},"min":{"1min":0.55300000000000005,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83199999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82799999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.69699999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.754}]},{"osd":7,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61599999999999999,"15min":0.60099999999999998},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76200000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.878},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67900000000000005,"15min":0.66400000000000003},"min":{"1min":0.51600000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86599999999999999,"5min":1.081,"15min":1.081},"last":0.94699999999999995}]},{"osd":8,"last update":"Fri May 10 05:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96599999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78400000000000003,"15min":0.77200000000000002},"min":{"1min":0.61099999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":1.0640000000000001}]},{"osd":9,"last update":"Fri May 10 05:32:41 2024","interfaces":[{"interface":"back","average":{"1min":1.8220000000000001,"5min":1.048,"15min":0.91900000000000004},"min":{"1min":0.65700000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":13.648,"5min":13.648,"15min":13.648},"last":1.042},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.93500000000000005}]},{"osd":10,"last update":"Fri May 10 05:33:01 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.65400000000000003,"5min":0.621,"15min":0.621},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.038},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.874,"15min":0.88},"min":{"1min":0.77200000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":0.92000000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":1.0089999999999999}]},{"osd":11,"last update":"Fri May 10 05:32:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.997},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.0880000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":978944,"data_stored":1022596,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.624,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.751,"5min":4.524,"15min":4.524},"last":0.56200000000000006}]},{"osd":1,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.57999999999999996},"min":{"1min":0.45000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.53400000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.78700000000000003}]},{"osd":2,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.35099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71699999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.69999999999999996}]},{"osd":3,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.66800000000000004,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":6.3860000000000001,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81100000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri May 10 05:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61099999999999999,"15min":0.59099999999999997},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86299999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.753},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.64100000000000001,"15min":0.62},"min":{"1min":0.435,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90600000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76800000000000002}]},{"osd":6,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.53000000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91600000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.53400000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88100000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri May 10 05:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.45800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86199999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70599999999999996,"15min":0.69099999999999995},"min":{"1min":0.51600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97299999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.87}]},{"osd":8,"last update":"Fri May 10 05:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri May 10 05:32:44 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.751,"15min":0.751},"min":{"1min":0.57699999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93899999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.80700000000000005,"15min":0.80900000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.98499999999999999,"5min":1.03,"15min":1.03},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri May 10 05:33:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.746,"15min":0.73599999999999999},"min":{"1min":0.61799999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93300000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0009999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.89500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36066,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67320,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68026,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":67838,"data_compressed":1840,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T05:33:31.409 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:33:31.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.707+0000 7fa715000700 1 -- 172.21.15.131:0/1649569342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7101036a0 msgr2=0x7fa710109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:31.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.707+0000 7fa715000700 1 --2- 172.21.15.131:0/1649569342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7101036a0 0x7fa710109f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa70800b0a0 tx=0x7fa70800b3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.707+0000 7fa715000700 1 -- 172.21.15.131:0/1649569342 shutdown_connections 2024-05-10T05:33:31.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.707+0000 7fa715000700 1 --2- 172.21.15.131:0/1649569342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7101036a0 0x7fa710109f50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.707+0000 7fa715000700 1 --2- 172.21.15.131:0/1649569342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa710101be0 0x7fa710101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.707+0000 7fa715000700 1 --2- 172.21.15.131:0/1649569342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa710102d00 0x7fa710103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.707+0000 7fa715000700 1 -- 172.21.15.131:0/1649569342 >> 172.21.15.131:0/1649569342 conn(0x7fa7100fd1d0 msgr2=0x7fa7100ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:31.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.707+0000 7fa715000700 1 -- 172.21.15.131:0/1649569342 shutdown_connections 2024-05-10T05:33:31.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.707+0000 7fa715000700 1 -- 172.21.15.131:0/1649569342 wait complete. 2024-05-10T05:33:31.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.708+0000 7fa715000700 1 Processor -- start 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa715000700 1 -- start start 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa715000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa710101be0 0x7fa710074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa715000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa710102d00 0x7fa710073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa715000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7101036a0 0x7fa710073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa715000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710074180 con 0x7fa7101036a0 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa715000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100742f0 con 0x7fa710101be0 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa715000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7101a2150 con 0x7fa710102d00 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa70de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa710101be0 0x7fa710074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa70e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7101036a0 0x7fa710073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa707e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa710102d00 0x7fa710073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa707e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa710102d00 0x7fa710073560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54758/0 (socket says 172.21.15.131:54758) 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa707e00700 1 -- 172.21.15.131:0/3888833320 learned_addr learned my addr 172.21.15.131:0/3888833320 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:31.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.709+0000 7fa70e800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7101036a0 0x7fa710073aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33428/0 (socket says 172.21.15.131:33428) 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa707e00700 1 -- 172.21.15.131:0/3888833320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa710101be0 msgr2=0x7fa710074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa707e00700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa710101be0 0x7fa710074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa707e00700 1 -- 172.21.15.131:0/3888833320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7101036a0 msgr2=0x7fa710073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa707e00700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7101036a0 0x7fa710073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa707e00700 1 -- 172.21.15.131:0/3888833320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa708009d00 con 0x7fa710102d00 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa70e800700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7101036a0 0x7fa710073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa70de00700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa710101be0 0x7fa710074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa707e00700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa710102d00 0x7fa710073560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800e9a0 tx=0x7fa6f800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa706a00700 1 -- 172.21.15.131:0/3888833320 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8009790 con 0x7fa710102d00 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa715000700 1 -- 172.21.15.131:0/3888833320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7101a23b0 con 0x7fa710102d00 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa706a00700 1 -- 172.21.15.131:0/3888833320 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6f8010430 con 0x7fa710102d00 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa706a00700 1 -- 172.21.15.131:0/3888833320 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8018680 con 0x7fa710102d00 2024-05-10T05:33:31.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.710+0000 7fa715000700 1 -- 172.21.15.131:0/3888833320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7101a2890 con 0x7fa710102d00 2024-05-10T05:33:31.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.711+0000 7fa715000700 1 -- 172.21.15.131:0/3888833320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa710108df0 con 0x7fa710102d00 2024-05-10T05:33:31.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.714+0000 7fa706a00700 1 -- 172.21.15.131:0/3888833320 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa6f80105a0 con 0x7fa710102d00 2024-05-10T05:33:31.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.715+0000 7fa706a00700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa6f40a1120 0x7fa6f40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:31.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.715+0000 7fa706a00700 1 -- 172.21.15.131:0/3888833320 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa6f8014070 con 0x7fa710102d00 2024-05-10T05:33:31.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.715+0000 7fa70de00700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa6f40a1120 0x7fa6f40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:31.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.716+0000 7fa70de00700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa6f40a1120 0x7fa6f40a3630 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa710102c00 tx=0x7fa700009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.717+0000 7fa706a00700 1 -- 172.21.15.131:0/3888833320 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa6f8096460 con 0x7fa710102d00 2024-05-10T05:33:31.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.928+0000 7fa715000700 1 -- 172.21.15.131:0/3888833320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7100694b0 con 0x7fa710102d00 2024-05-10T05:33:31.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.930+0000 7fa706a00700 1 -- 172.21.15.131:0/3888833320 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7fa6f8026090 con 0x7fa710102d00 2024-05-10T05:33:31.931 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:31.931 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:33:31.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.932+0000 7fa715000700 1 -- 172.21.15.131:0/3888833320 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa6f40a1120 msgr2=0x7fa6f40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:31.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.932+0000 7fa715000700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa6f40a1120 0x7fa6f40a3630 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa710102c00 tx=0x7fa700009260 comp rx=0 tx=0).stop 2024-05-10T05:33:31.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.932+0000 7fa715000700 1 -- 172.21.15.131:0/3888833320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa710102d00 msgr2=0x7fa710073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.932+0000 7fa715000700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa710102d00 0x7fa710073560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800e9a0 tx=0x7fa6f800ecb0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.932+0000 7fa715000700 1 -- 172.21.15.131:0/3888833320 shutdown_connections 2024-05-10T05:33:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.932+0000 7fa715000700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa6f40a1120 0x7fa6f40a3630 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.932+0000 7fa715000700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7101036a0 0x7fa710073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.932+0000 7fa715000700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa710101be0 0x7fa710074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.932+0000 7fa715000700 1 --2- 172.21.15.131:0/3888833320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa710102d00 0x7fa710073560 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.932+0000 7fa715000700 1 -- 172.21.15.131:0/3888833320 >> 172.21.15.131:0/3888833320 conn(0x7fa7100fd1d0 msgr2=0x7fa710105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.933+0000 7fa715000700 1 -- 172.21.15.131:0/3888833320 shutdown_connections 2024-05-10T05:33:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:31.933+0000 7fa715000700 1 -- 172.21.15.131:0/3888833320 wait complete. 2024-05-10T05:33:31.964 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T05:31:49.481667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T05:31:50.960751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:33:31.964 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:33:32.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:31 smithi131 ceph-mon[37264]: from='client.25345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:33:32.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:31 smithi131 ceph-mon[37264]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:33:32.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:31 smithi131 ceph-mon[37264]: pgmap v290: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-10T05:33:32.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:31 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3888833320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:32.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.229+0000 7fe413200700 1 -- 172.21.15.131:0/2609252808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c1034a0 msgr2=0x7fe40c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:32.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.229+0000 7fe413200700 1 --2- 172.21.15.131:0/2609252808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c1034a0 0x7fe40c109d50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe40800b0a0 tx=0x7fe40800b3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.230+0000 7fe413200700 1 -- 172.21.15.131:0/2609252808 shutdown_connections 2024-05-10T05:33:32.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.230+0000 7fe413200700 1 --2- 172.21.15.131:0/2609252808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c1034a0 0x7fe40c109d50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.230+0000 7fe413200700 1 --2- 172.21.15.131:0/2609252808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe40c1019e0 0x7fe40c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.230+0000 7fe413200700 1 --2- 172.21.15.131:0/2609252808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c102b00 0x7fe40c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.230+0000 7fe413200700 1 -- 172.21.15.131:0/2609252808 >> 172.21.15.131:0/2609252808 conn(0x7fe40c0fcfd0 msgr2=0x7fe40c0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:32.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.230+0000 7fe413200700 1 -- 172.21.15.131:0/2609252808 shutdown_connections 2024-05-10T05:33:32.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.230+0000 7fe413200700 1 -- 172.21.15.131:0/2609252808 wait complete. 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.231+0000 7fe413200700 1 Processor -- start 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.231+0000 7fe413200700 1 -- start start 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.231+0000 7fe413200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe40c1019e0 0x7fe40c074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.231+0000 7fe413200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c102b00 0x7fe40c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.231+0000 7fe413200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c1034a0 0x7fe40c073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.231+0000 7fe413200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c074100 con 0x7fe40c102b00 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.231+0000 7fe413200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c074270 con 0x7fe40c1034a0 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.231+0000 7fe413200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c074430 con 0x7fe40c1019e0 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe411400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c102b00 0x7fe40c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe411400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c102b00 0x7fe40c073560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33458/0 (socket says 172.21.15.131:33458) 2024-05-10T05:33:32.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe411400700 1 -- 172.21.15.131:0/3028637433 learned_addr learned my addr 172.21.15.131:0/3028637433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:32.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe411e00700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe40c1019e0 0x7fe40c074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:32.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe412800700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c1034a0 0x7fe40c073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:32.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe411400700 1 -- 172.21.15.131:0/3028637433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe40c1019e0 msgr2=0x7fe40c074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:32.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe411400700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe40c1019e0 0x7fe40c074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe411400700 1 -- 172.21.15.131:0/3028637433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c1034a0 msgr2=0x7fe40c073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:32.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe411400700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c1034a0 0x7fe40c073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe411400700 1 -- 172.21.15.131:0/3028637433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408009d00 con 0x7fe40c102b00 2024-05-10T05:33:32.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe412800700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c1034a0 0x7fe40c073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:32.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe411400700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c102b00 0x7fe40c073560 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00e9a0 tx=0x7fe3fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:32.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.232+0000 7fe406a00700 1 -- 172.21.15.131:0/3028637433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc009790 con 0x7fe40c102b00 2024-05-10T05:33:32.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.233+0000 7fe413200700 1 -- 172.21.15.131:0/3028637433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c1a21a0 con 0x7fe40c102b00 2024-05-10T05:33:32.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.233+0000 7fe406a00700 1 -- 172.21.15.131:0/3028637433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3fc00cbd0 con 0x7fe40c102b00 2024-05-10T05:33:32.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.233+0000 7fe406a00700 1 -- 172.21.15.131:0/3028637433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc018680 con 0x7fe40c102b00 2024-05-10T05:33:32.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.233+0000 7fe413200700 1 -- 172.21.15.131:0/3028637433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c1a26b0 con 0x7fe40c102b00 2024-05-10T05:33:32.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.234+0000 7fe413200700 1 -- 172.21.15.131:0/3028637433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c108bf0 con 0x7fe40c102b00 2024-05-10T05:33:32.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.236+0000 7fe406a00700 1 -- 172.21.15.131:0/3028637433 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe3fc00cd40 con 0x7fe40c102b00 2024-05-10T05:33:32.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.237+0000 7fe406a00700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3f40a1120 0x7fe3f40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:32.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.237+0000 7fe406a00700 1 -- 172.21.15.131:0/3028637433 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe3fc014070 con 0x7fe40c102b00 2024-05-10T05:33:32.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.237+0000 7fe411e00700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3f40a1120 0x7fe3f40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:32.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.238+0000 7fe411e00700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3f40a1120 0x7fe3f40a3630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe40c102a00 tx=0x7fe400009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:32.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.241+0000 7fe406a00700 1 -- 172.21.15.131:0/3028637433 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe3fc052d60 con 0x7fe40c102b00 2024-05-10T05:33:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:31 smithi157 ceph-mon[78427]: from='client.25345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:33:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:31 smithi157 ceph-mon[78427]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:33:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:31 smithi157 ceph-mon[78427]: pgmap v290: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-10T05:33:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:31 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3888833320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:31 smithi142 ceph-mon[77667]: from='client.25345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:33:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:31 smithi142 ceph-mon[77667]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:33:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:31 smithi142 ceph-mon[77667]: pgmap v290: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-10T05:33:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:31 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3888833320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:32.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.449+0000 7fe413200700 1 -- 172.21.15.131:0/3028637433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe40c1085d0 con 0x7fe40c102b00 2024-05-10T05:33:32.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.449+0000 7fe406a00700 1 -- 172.21.15.131:0/3028637433 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3fc0962e0 con 0x7fe40c102b00 2024-05-10T05:33:32.450 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:33:32.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.451+0000 7fe413200700 1 -- 172.21.15.131:0/3028637433 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3f40a1120 msgr2=0x7fe3f40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.451+0000 7fe413200700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3f40a1120 0x7fe3f40a3630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe40c102a00 tx=0x7fe400009260 comp rx=0 tx=0).stop 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.451+0000 7fe413200700 1 -- 172.21.15.131:0/3028637433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c102b00 msgr2=0x7fe40c073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.451+0000 7fe413200700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c102b00 0x7fe40c073560 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00e9a0 tx=0x7fe3fc00ecb0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.452+0000 7fe413200700 1 -- 172.21.15.131:0/3028637433 shutdown_connections 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.452+0000 7fe413200700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3f40a1120 0x7fe3f40a3630 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.452+0000 7fe413200700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c1034a0 0x7fe40c073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.452+0000 7fe413200700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe40c1019e0 0x7fe40c074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.452+0000 7fe413200700 1 --2- 172.21.15.131:0/3028637433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c102b00 0x7fe40c073560 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.452+0000 7fe413200700 1 -- 172.21.15.131:0/3028637433 >> 172.21.15.131:0/3028637433 conn(0x7fe40c0fcfd0 msgr2=0x7fe40c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.452+0000 7fe413200700 1 -- 172.21.15.131:0/3028637433 shutdown_connections 2024-05-10T05:33:32.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.452+0000 7fe413200700 1 -- 172.21.15.131:0/3028637433 wait complete. 2024-05-10T05:33:32.481 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T05:33:32.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.748+0000 7f60a4800700 1 -- 172.21.15.131:0/1771646269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609c102d00 msgr2=0x7f609c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:32.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.748+0000 7f60a4800700 1 --2- 172.21.15.131:0/1771646269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609c102d00 0x7f609c103110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f608c0099c0 tx=0x7f608c009cd0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.749+0000 7f60a4800700 1 -- 172.21.15.131:0/1771646269 shutdown_connections 2024-05-10T05:33:32.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.749+0000 7f60a4800700 1 --2- 172.21.15.131:0/1771646269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f609c1036a0 0x7f609c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.749+0000 7f60a4800700 1 --2- 172.21.15.131:0/1771646269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f609c101be0 0x7f609c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.749+0000 7f60a4800700 1 --2- 172.21.15.131:0/1771646269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609c102d00 0x7f609c103110 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.749+0000 7f60a4800700 1 -- 172.21.15.131:0/1771646269 >> 172.21.15.131:0/1771646269 conn(0x7f609c0fd1d0 msgr2=0x7f609c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:32.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.749+0000 7f60a4800700 1 -- 172.21.15.131:0/1771646269 shutdown_connections 2024-05-10T05:33:32.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.749+0000 7f60a4800700 1 -- 172.21.15.131:0/1771646269 wait complete. 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.750+0000 7f60a4800700 1 Processor -- start 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.750+0000 7f60a4800700 1 -- start start 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a4800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f609c101be0 0x7f609c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a4800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609c102d00 0x7f609c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a4800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f609c1036a0 0x7f609c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a4800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c19fa00 con 0x7f609c1036a0 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a4800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c19fb70 con 0x7f609c101be0 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a4800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c19fd30 con 0x7f609c102d00 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f609c101be0 0x7f609c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f609c1036a0 0x7f609c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f609c101be0 0x7f609c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43474/0 (socket says 172.21.15.131:43474) 2024-05-10T05:33:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a2800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f609c1036a0 0x7f609c19f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33474/0 (socket says 172.21.15.131:33474) 2024-05-10T05:33:32.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a1e00700 1 -- 172.21.15.131:0/451193604 learned_addr learned my addr 172.21.15.131:0/451193604 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:32.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a1400700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609c102d00 0x7f609c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a1e00700 1 -- 172.21.15.131:0/451193604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609c102d00 msgr2=0x7f609c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a1e00700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609c102d00 0x7f609c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.751+0000 7f60a1e00700 1 -- 172.21.15.131:0/451193604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f609c1036a0 msgr2=0x7f609c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.752+0000 7f60a1e00700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f609c1036a0 0x7f609c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.752+0000 7f60a1e00700 1 -- 172.21.15.131:0/451193604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6098009c40 con 0x7f609c101be0 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.752+0000 7f60a2800700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f609c1036a0 0x7f609c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.752+0000 7f60a1e00700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f609c101be0 0x7f609c197f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f609000e940 tx=0x7f609000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.752+0000 7f6096a00700 1 -- 172.21.15.131:0/451193604 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090004220 con 0x7f609c101be0 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.752+0000 7f60a4800700 1 -- 172.21.15.131:0/451193604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c0095f0 con 0x7f609c101be0 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.752+0000 7f6096a00700 1 -- 172.21.15.131:0/451193604 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6090004380 con 0x7f609c101be0 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.752+0000 7f6096a00700 1 -- 172.21.15.131:0/451193604 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090003cb0 con 0x7f609c101be0 2024-05-10T05:33:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.752+0000 7f60a4800700 1 -- 172.21.15.131:0/451193604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609c1a02d0 con 0x7f609c101be0 2024-05-10T05:33:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.753+0000 7f60a4800700 1 -- 172.21.15.131:0/451193604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609c108df0 con 0x7f609c101be0 2024-05-10T05:33:32.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.755+0000 7f6096a00700 1 -- 172.21.15.131:0/451193604 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f609000c7c0 con 0x7f609c101be0 2024-05-10T05:33:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.757+0000 7f6096a00700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f60840a11c0 0x7f60840a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.757+0000 7f6096a00700 1 -- 172.21.15.131:0/451193604 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6090014070 con 0x7f609c101be0 2024-05-10T05:33:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.757+0000 7f60a1400700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f60840a11c0 0x7f60840a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:32.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.757+0000 7f60a1400700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f60840a11c0 0x7f60840a36d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f609c198fc0 tx=0x7f608c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:32.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.759+0000 7f6096a00700 1 -- 172.21.15.131:0/451193604 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6090096390 con 0x7f609c101be0 2024-05-10T05:33:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.967+0000 7f60a4800700 1 -- 172.21.15.131:0/451193604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f609c10d430 con 0x7f609c101be0 2024-05-10T05:33:32.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.968+0000 7f6096a00700 1 -- 172.21.15.131:0/451193604 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f609000fe60 con 0x7f609c101be0 2024-05-10T05:33:32.969 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-10T05:33:32.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 -- 172.21.15.131:0/451193604 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f60840a11c0 msgr2=0x7f60840a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f60840a11c0 0x7f60840a36d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f609c198fc0 tx=0x7f608c005fb0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 -- 172.21.15.131:0/451193604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f609c101be0 msgr2=0x7f609c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f609c101be0 0x7f609c197f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f609000e940 tx=0x7f609000ec50 comp rx=0 tx=0).stop 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 -- 172.21.15.131:0/451193604 shutdown_connections 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f60840a11c0 0x7f60840a36d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f609c1036a0 0x7f609c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f609c101be0 0x7f609c197f20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 --2- 172.21.15.131:0/451193604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f609c102d00 0x7f609c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 -- 172.21.15.131:0/451193604 >> 172.21.15.131:0/451193604 conn(0x7f609c0fd1d0 msgr2=0x7f609c105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 -- 172.21.15.131:0/451193604 shutdown_connections 2024-05-10T05:33:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:32.971+0000 7f60a4800700 1 -- 172.21.15.131:0/451193604 wait complete. 2024-05-10T05:33:32.999 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T05:33:33.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:32 smithi131 ceph-mon[37264]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:33:33.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:32 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3028637433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:33.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.265+0000 7f4fc6c00700 1 -- 172.21.15.131:0/3913746209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fc01019f0 msgr2=0x7f4fc0101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.265+0000 7f4fc6c00700 1 --2- 172.21.15.131:0/3913746209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fc01019f0 0x7f4fc0101e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009990 tx=0x7f4fb4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.265+0000 7f4fc6c00700 1 -- 172.21.15.131:0/3913746209 shutdown_connections 2024-05-10T05:33:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.265+0000 7f4fc6c00700 1 --2- 172.21.15.131:0/3913746209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fc01019f0 0x7f4fc0101e00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.265+0000 7f4fc6c00700 1 --2- 172.21.15.131:0/3913746209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc01034b0 0x7f4fc0109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.265+0000 7f4fc6c00700 1 --2- 172.21.15.131:0/3913746209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc0102b10 0x7f4fc0102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.265+0000 7f4fc6c00700 1 -- 172.21.15.131:0/3913746209 >> 172.21.15.131:0/3913746209 conn(0x7f4fc00fd000 msgr2=0x7f4fc00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.265+0000 7f4fc6c00700 1 -- 172.21.15.131:0/3913746209 shutdown_connections 2024-05-10T05:33:33.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.265+0000 7f4fc6c00700 1 -- 172.21.15.131:0/3913746209 wait complete. 2024-05-10T05:33:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.266+0000 7f4fc6c00700 1 Processor -- start 2024-05-10T05:33:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc6c00700 1 -- start start 2024-05-10T05:33:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc6c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc01019f0 0x7f4fc0074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc6c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc0102b10 0x7f4fc0073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc6c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fc01034b0 0x7f4fc0073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc6c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0074100 con 0x7f4fc01034b0 2024-05-10T05:33:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc6c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0074270 con 0x7f4fc0102b10 2024-05-10T05:33:33.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc6c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0074430 con 0x7f4fc01019f0 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc5800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc01019f0 0x7f4fc0074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc5800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc01019f0 0x7f4fc0074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35682/0 (socket says 172.21.15.131:35682) 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc5800700 1 -- 172.21.15.131:0/325021000 learned_addr learned my addr 172.21.15.131:0/325021000 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.267+0000 7f4fc4e00700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc0102b10 0x7f4fc0073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fc6200700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fc01034b0 0x7f4fc0073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fc5800700 1 -- 172.21.15.131:0/325021000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc0102b10 msgr2=0x7f4fc0073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fc5800700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc0102b10 0x7f4fc0073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fc5800700 1 -- 172.21.15.131:0/325021000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fc01034b0 msgr2=0x7f4fc0073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fc5800700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fc01034b0 0x7f4fc0073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fc5800700 1 -- 172.21.15.131:0/325021000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb40095f0 con 0x7f4fc01019f0 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fc5800700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc01019f0 0x7f4fc0074ec0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4000c00 tx=0x7f4fb4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fbe000700 1 -- 172.21.15.131:0/325021000 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb401d070 con 0x7f4fc01019f0 2024-05-10T05:33:33.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fc6c00700 1 -- 172.21.15.131:0/325021000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc01a2050 con 0x7f4fc01019f0 2024-05-10T05:33:33.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fbe000700 1 -- 172.21.15.131:0/325021000 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb4021410 con 0x7f4fc01019f0 2024-05-10T05:33:33.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.268+0000 7f4fbe000700 1 -- 172.21.15.131:0/325021000 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4017650 con 0x7f4fc01019f0 2024-05-10T05:33:33.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.269+0000 7f4fc6c00700 1 -- 172.21.15.131:0/325021000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc01a2470 con 0x7f4fc01019f0 2024-05-10T05:33:33.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.270+0000 7f4fc6c00700 1 -- 172.21.15.131:0/325021000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc0108c70 con 0x7f4fc01019f0 2024-05-10T05:33:33.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.272+0000 7f4fbe000700 1 -- 172.21.15.131:0/325021000 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4fb4021580 con 0x7f4fc01019f0 2024-05-10T05:33:33.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.273+0000 7f4fbe000700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4fa80a1120 0x7f4fa80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:33.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.273+0000 7f4fbe000700 1 -- 172.21.15.131:0/325021000 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4fb40ce240 con 0x7f4fc01019f0 2024-05-10T05:33:33.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.273+0000 7f4fc4e00700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4fa80a1120 0x7f4fa80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:33.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.274+0000 7f4fc4e00700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4fa80a1120 0x7f4fa80a3630 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0074860 tx=0x7f4fac009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:33.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.275+0000 7f4fbe000700 1 -- 172.21.15.131:0/325021000 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4fb4097a60 con 0x7f4fc01019f0 2024-05-10T05:33:33.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:32 smithi157 ceph-mon[78427]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:33:33.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:32 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3028637433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:33.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:32 smithi142 ceph-mon[77667]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T05:33:33.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:32 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3028637433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:33:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.483+0000 7f4fc6c00700 1 -- 172.21.15.131:0/325021000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fc010d220 con 0x7f4fc01019f0 2024-05-10T05:33:33.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.484+0000 7f4fbe000700 1 -- 172.21.15.131:0/325021000 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fb409b080 con 0x7f4fc01019f0 2024-05-10T05:33:33.484 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-10T05:33:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 -- 172.21.15.131:0/325021000 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4fa80a1120 msgr2=0x7f4fa80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4fa80a1120 0x7f4fa80a3630 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0074860 tx=0x7f4fac009260 comp rx=0 tx=0).stop 2024-05-10T05:33:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 -- 172.21.15.131:0/325021000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc01019f0 msgr2=0x7f4fc0074ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc01019f0 0x7f4fc0074ec0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4000c00 tx=0x7f4fb4004730 comp rx=0 tx=0).stop 2024-05-10T05:33:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 -- 172.21.15.131:0/325021000 shutdown_connections 2024-05-10T05:33:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4fa80a1120 0x7f4fa80a3630 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fc01034b0 0x7f4fc0073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc01019f0 0x7f4fc0074ec0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 --2- 172.21.15.131:0/325021000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc0102b10 0x7f4fc0073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 -- 172.21.15.131:0/325021000 >> 172.21.15.131:0/325021000 conn(0x7f4fc00fd000 msgr2=0x7f4fc0105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:33.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 -- 172.21.15.131:0/325021000 shutdown_connections 2024-05-10T05:33:33.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.486+0000 7f4fc6c00700 1 -- 172.21.15.131:0/325021000 wait complete. 2024-05-10T05:33:33.514 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:33:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.777+0000 7f9143e00700 1 -- 172.21.15.131:0/3533830202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91441034c0 msgr2=0x7f9144109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.777+0000 7f9143e00700 1 --2- 172.21.15.131:0/3533830202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91441034c0 0x7f9144109d70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f913c009f90 tx=0x7f913c00b330 comp rx=0 tx=0).stop 2024-05-10T05:33:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.778+0000 7f9143e00700 1 -- 172.21.15.131:0/3533830202 shutdown_connections 2024-05-10T05:33:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.778+0000 7f9143e00700 1 --2- 172.21.15.131:0/3533830202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91441034c0 0x7f9144109d70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.778+0000 7f9143e00700 1 --2- 172.21.15.131:0/3533830202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9144101a00 0x7f9144101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.778+0000 7f9143e00700 1 --2- 172.21.15.131:0/3533830202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9144102b20 0x7f9144102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.778+0000 7f9143e00700 1 -- 172.21.15.131:0/3533830202 >> 172.21.15.131:0/3533830202 conn(0x7f91440fd010 msgr2=0x7f91440ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.778+0000 7f9143e00700 1 -- 172.21.15.131:0/3533830202 shutdown_connections 2024-05-10T05:33:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.778+0000 7f9143e00700 1 -- 172.21.15.131:0/3533830202 wait complete. 2024-05-10T05:33:33.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9143e00700 1 Processor -- start 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9143e00700 1 -- start start 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9143e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9144101a00 0x7f9144074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9143e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9144102b20 0x7f9144073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9143e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91441034c0 0x7f9144073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9143e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9144074100 con 0x7f91441034c0 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9143e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9144074270 con 0x7f9144101a00 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9143e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9144074430 con 0x7f9144102b20 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9142a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9144101a00 0x7f9144074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9143400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91441034c0 0x7f9144073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9142a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9144101a00 0x7f9144074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:50302/0 (socket says 172.21.15.131:50302) 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9142a00700 1 -- 172.21.15.131:0/2834478221 learned_addr learned my addr 172.21.15.131:0/2834478221 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.779+0000 7f9142000700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9144102b20 0x7f9144073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:33.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f9142a00700 1 -- 172.21.15.131:0/2834478221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9144102b20 msgr2=0x7f9144073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f9142a00700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9144102b20 0x7f9144073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f9142a00700 1 -- 172.21.15.131:0/2834478221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91441034c0 msgr2=0x7f9144073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f9142a00700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91441034c0 0x7f9144073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f9142a00700 1 -- 172.21.15.131:0/2834478221 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340095f0 con 0x7f9144101a00 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f9143400700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91441034c0 0x7f9144073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f9142a00700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9144101a00 0x7f9144074ec0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f913400ebc0 tx=0x7f913400eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f913b400700 1 -- 172.21.15.131:0/2834478221 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913400cc90 con 0x7f9144101a00 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f913b400700 1 -- 172.21.15.131:0/2834478221 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913400cdf0 con 0x7f9144101a00 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f913b400700 1 -- 172.21.15.131:0/2834478221 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134018790 con 0x7f9144101a00 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f9143e00700 1 -- 172.21.15.131:0/2834478221 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c009c40 con 0x7f9144101a00 2024-05-10T05:33:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.780+0000 7f9143e00700 1 -- 172.21.15.131:0/2834478221 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91441a23a0 con 0x7f9144101a00 2024-05-10T05:33:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.782+0000 7f9143e00700 1 -- 172.21.15.131:0/2834478221 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9144108c10 con 0x7f9144101a00 2024-05-10T05:33:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.784+0000 7f913b400700 1 -- 172.21.15.131:0/2834478221 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f91340109f0 con 0x7f9144101a00 2024-05-10T05:33:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.785+0000 7f913b400700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91280a11c0 0x7f91280a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.785+0000 7f913b400700 1 -- 172.21.15.131:0/2834478221 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9134014070 con 0x7f9144101a00 2024-05-10T05:33:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.785+0000 7f9142000700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91280a11c0 0x7f91280a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.786+0000 7f9142000700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91280a11c0 0x7f91280a36d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9144074860 tx=0x7f9130009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:33.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:33.788+0000 7f913b400700 1 -- 172.21.15.131:0/2834478221 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9134096650 con 0x7f9144101a00 2024-05-10T05:33:34.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.046+0000 7f9143e00700 1 -- 172.21.15.131:0/2834478221 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9144068110 con 0x7f9144101a00 2024-05-10T05:33:34.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.047+0000 7f913b400700 1 -- 172.21.15.131:0/2834478221 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f9134099c70 con 0x7f9144101a00 2024-05-10T05:33:34.048 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:33:24.915360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:33:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 -- 172.21.15.131:0/2834478221 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91280a11c0 msgr2=0x7f91280a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91280a11c0 0x7f91280a36d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9144074860 tx=0x7f9130009260 comp rx=0 tx=0).stop 2024-05-10T05:33:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 -- 172.21.15.131:0/2834478221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9144101a00 msgr2=0x7f9144074ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9144101a00 0x7f9144074ec0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f913400ebc0 tx=0x7f913400eed0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 -- 172.21.15.131:0/2834478221 shutdown_connections 2024-05-10T05:33:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91280a11c0 0x7f91280a36d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91441034c0 0x7f9144073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9144101a00 0x7f9144074ec0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 --2- 172.21.15.131:0/2834478221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9144102b20 0x7f9144073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 -- 172.21.15.131:0/2834478221 >> 172.21.15.131:0/2834478221 conn(0x7f91440fd010 msgr2=0x7f9144105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 -- 172.21.15.131:0/2834478221 shutdown_connections 2024-05-10T05:33:34.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.050+0000 7f9143e00700 1 -- 172.21.15.131:0/2834478221 wait complete. 2024-05-10T05:33:34.053 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-05-10T05:33:34.077 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:33:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:33 smithi131 ceph-mon[37264]: pgmap v291: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T05:33:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:33 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/451193604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:34.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:33 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/325021000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:34.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:33 smithi157 ceph-mon[78427]: pgmap v291: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T05:33:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:33 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/451193604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:34.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:33 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/325021000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:34.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.339+0000 7f97ac400700 1 -- 172.21.15.131:0/3542423179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a4101be0 msgr2=0x7f97a4101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.339+0000 7f97ac400700 1 --2- 172.21.15.131:0/3542423179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a4101be0 0x7f97a4101ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9798009990 tx=0x7f9798009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.339+0000 7f97ac400700 1 -- 172.21.15.131:0/3542423179 shutdown_connections 2024-05-10T05:33:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.339+0000 7f97ac400700 1 --2- 172.21.15.131:0/3542423179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a4101be0 0x7f97a4101ff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.339+0000 7f97ac400700 1 --2- 172.21.15.131:0/3542423179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a41036a0 0x7f97a4109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.339+0000 7f97ac400700 1 --2- 172.21.15.131:0/3542423179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a4102d00 0x7f97a4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.339+0000 7f97ac400700 1 -- 172.21.15.131:0/3542423179 >> 172.21.15.131:0/3542423179 conn(0x7f97a40fd210 msgr2=0x7f97a40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.340+0000 7f97ac400700 1 -- 172.21.15.131:0/3542423179 shutdown_connections 2024-05-10T05:33:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.340+0000 7f97ac400700 1 -- 172.21.15.131:0/3542423179 wait complete. 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.341+0000 7f97ac400700 1 Processor -- start 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.341+0000 7f97ac400700 1 -- start start 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97ac400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a4101be0 0x7f97a419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97ac400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4102d00 0x7f97a419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97ac400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41036a0 0x7f97a41a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97ac400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a41a1cd0 con 0x7f97a41036a0 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97ac400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a41a1e40 con 0x7f97a4101be0 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97ac400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a41a2000 con 0x7f97a4102d00 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97aa400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41036a0 0x7f97a41a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97a9a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a4101be0 0x7f97a419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97aa400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41036a0 0x7f97a41a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34262/0 (socket says 172.21.15.131:34262) 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97aa400700 1 -- 172.21.15.131:0/4168210684 learned_addr learned my addr 172.21.15.131:0/4168210684 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97a9000700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4102d00 0x7f97a419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97a9a00700 1 -- 172.21.15.131:0/4168210684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4102d00 msgr2=0x7f97a419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97a9a00700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4102d00 0x7f97a419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97a9a00700 1 -- 172.21.15.131:0/4168210684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41036a0 msgr2=0x7f97a41a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97a9a00700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41036a0 0x7f97a41a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97a9a00700 1 -- 172.21.15.131:0/4168210684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980095f0 con 0x7f97a4101be0 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.342+0000 7f97aa400700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41036a0 0x7f97a41a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.343+0000 7f97a9a00700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a4101be0 0x7f97a419a1f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9798000c00 tx=0x7f9798005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.343+0000 7f97a2000700 1 -- 172.21.15.131:0/4168210684 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979801d070 con 0x7f97a4101be0 2024-05-10T05:33:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.343+0000 7f97ac400700 1 -- 172.21.15.131:0/4168210684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a41a2280 con 0x7f97a4101be0 2024-05-10T05:33:34.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.343+0000 7f97a2000700 1 -- 172.21.15.131:0/4168210684 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9798021410 con 0x7f97a4101be0 2024-05-10T05:33:34.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.343+0000 7f97a2000700 1 -- 172.21.15.131:0/4168210684 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798017650 con 0x7f97a4101be0 2024-05-10T05:33:34.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.343+0000 7f97ac400700 1 -- 172.21.15.131:0/4168210684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a41a2700 con 0x7f97a4101be0 2024-05-10T05:33:34.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.344+0000 7f97ac400700 1 -- 172.21.15.131:0/4168210684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a40694b0 con 0x7f97a4101be0 2024-05-10T05:33:34.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.346+0000 7f97a2000700 1 -- 172.21.15.131:0/4168210684 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f97980217f0 con 0x7f97a4101be0 2024-05-10T05:33:34.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.347+0000 7f97a2000700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f978c0a1120 0x7f978c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.348+0000 7f97a2000700 1 -- 172.21.15.131:0/4168210684 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f97980cdd70 con 0x7f97a4101be0 2024-05-10T05:33:34.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.348+0000 7f97a9000700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f978c0a1120 0x7f978c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.348+0000 7f97a9000700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f978c0a1120 0x7f978c0a3630 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f97a419b290 tx=0x7f9790009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:34.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.352+0000 7f97a2000700 1 -- 172.21.15.131:0/4168210684 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97980975f0 con 0x7f97a4101be0 2024-05-10T05:33:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:33 smithi142 ceph-mon[77667]: pgmap v291: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T05:33:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:33 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/451193604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:33:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:33 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/325021000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:33:34.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.560+0000 7f97ac400700 1 -- 172.21.15.131:0/4168210684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97a41a29b0 con 0x7f97a4101be0 2024-05-10T05:33:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.562+0000 7f97a2000700 1 -- 172.21.15.131:0/4168210684 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7f979809ac10 con 0x7f97a4101be0 2024-05-10T05:33:34.563 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:33:34.564 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:33:34.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 -- 172.21.15.131:0/4168210684 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f978c0a1120 msgr2=0x7f978c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f978c0a1120 0x7f978c0a3630 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f97a419b290 tx=0x7f9790009260 comp rx=0 tx=0).stop 2024-05-10T05:33:34.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 -- 172.21.15.131:0/4168210684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a4101be0 msgr2=0x7f97a419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a4101be0 0x7f97a419a1f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9798000c00 tx=0x7f9798005740 comp rx=0 tx=0).stop 2024-05-10T05:33:34.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 -- 172.21.15.131:0/4168210684 shutdown_connections 2024-05-10T05:33:34.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f978c0a1120 0x7f978c0a3630 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97a41036a0 0x7f97a41a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a4101be0 0x7f97a419a1f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 --2- 172.21.15.131:0/4168210684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97a4102d00 0x7f97a419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 -- 172.21.15.131:0/4168210684 >> 172.21.15.131:0/4168210684 conn(0x7f97a40fd210 msgr2=0x7f97a4073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:34.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 -- 172.21.15.131:0/4168210684 shutdown_connections 2024-05-10T05:33:34.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.565+0000 7f97ac400700 1 -- 172.21.15.131:0/4168210684 wait complete. 2024-05-10T05:33:34.592 DEBUG:tasks.fwd_scrub:Yielding 2024-05-10T05:33:34.592 INFO:teuthology.run_tasks:Running task workunit... 2024-05-10T05:33:34.606 INFO:tasks.workunit:Pulling workunits from ref 50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:33:34.606 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-10T05:33:34.606 INFO:tasks.workunit:timeout=3h 2024-05-10T05:33:34.606 INFO:tasks.workunit:cleanup=True 2024-05-10T05:33:34.607 DEBUG:teuthology.orchestra.run.smithi131:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-10T05:33:34.607 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-10T05:33:34.607 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:33:34.607 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:33:34.651 INFO:teuthology.orchestra.run.smithi131.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-10T05:33:34.651 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-10T05:33:34.651 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 7bh/123d Inode: 1099511628283 Links: 2 2024-05-10T05:33:34.651 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T05:33:34.651 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:cephfs_t:s0 2024-05-10T05:33:34.651 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-10 05:31:59.314244604 +0000 2024-05-10T05:33:34.651 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-10 05:31:59.314244604 +0000 2024-05-10T05:33:34.651 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-10 05:33:23.491087571 +0000 2024-05-10T05:33:34.651 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-10 05:31:59.314244604 +0000 2024-05-10T05:33:34.651 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-10T05:33:34.652 DEBUG:teuthology.orchestra.run.smithi131:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-10T05:33:34.777 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 50174015a7e79f951b988949abe236791f52d4da 2024-05-10T05:33:34.825 INFO:tasks.workunit.client.0.smithi131.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-10T05:33:34.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.897+0000 7f47ce000700 1 -- 172.21.15.131:0/3371122614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c81019f0 msgr2=0x7f47c8101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.897+0000 7f47ce000700 1 --2- 172.21.15.131:0/3371122614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c81019f0 0x7f47c8101e00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009990 tx=0x7f47bc009ca0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.897+0000 7f47ce000700 1 -- 172.21.15.131:0/3371122614 shutdown_connections 2024-05-10T05:33:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.897+0000 7f47ce000700 1 --2- 172.21.15.131:0/3371122614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c81019f0 0x7f47c8101e00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.897+0000 7f47ce000700 1 --2- 172.21.15.131:0/3371122614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81034b0 0x7f47c8109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.897+0000 7f47ce000700 1 --2- 172.21.15.131:0/3371122614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8102b10 0x7f47c8102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.897+0000 7f47ce000700 1 -- 172.21.15.131:0/3371122614 >> 172.21.15.131:0/3371122614 conn(0x7f47c80fd000 msgr2=0x7f47c80ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.897+0000 7f47ce000700 1 -- 172.21.15.131:0/3371122614 shutdown_connections 2024-05-10T05:33:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.897+0000 7f47ce000700 1 -- 172.21.15.131:0/3371122614 wait complete. 2024-05-10T05:33:34.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.898+0000 7f47ce000700 1 Processor -- start 2024-05-10T05:33:34.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47ce000700 1 -- start start 2024-05-10T05:33:34.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47ce000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81019f0 0x7f47c819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47ce000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8102b10 0x7f47c819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47ce000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c81034b0 0x7f47c81a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47ce000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81a1b10 con 0x7f47c81034b0 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47ce000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81a1c50 con 0x7f47c81019f0 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47ce000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81a1de0 con 0x7f47c8102b10 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47cd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c81034b0 0x7f47c81a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47cd600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c81034b0 0x7f47c81a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34280/0 (socket says 172.21.15.131:34280) 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47cd600700 1 -- 172.21.15.131:0/1237708215 learned_addr learned my addr 172.21.15.131:0/1237708215 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47c7e00700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8102b10 0x7f47c819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.899+0000 7f47ccc00700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81019f0 0x7f47c819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47c7e00700 1 -- 172.21.15.131:0/1237708215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81019f0 msgr2=0x7f47c819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47c7e00700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81019f0 0x7f47c819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47c7e00700 1 -- 172.21.15.131:0/1237708215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c81034b0 msgr2=0x7f47c81a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47c7e00700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c81034b0 0x7f47c81a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47c7e00700 1 -- 172.21.15.131:0/1237708215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0095f0 con 0x7f47c8102b10 2024-05-10T05:33:34.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47c7e00700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8102b10 0x7f47c819a600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47b400ea20 tx=0x7f47b400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:34.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47c5600700 1 -- 172.21.15.131:0/1237708215 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b400cbd0 con 0x7f47c8102b10 2024-05-10T05:33:34.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47ce000700 1 -- 172.21.15.131:0/1237708215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c81a20c0 con 0x7f47c8102b10 2024-05-10T05:33:34.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47c5600700 1 -- 172.21.15.131:0/1237708215 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b400ee60 con 0x7f47c8102b10 2024-05-10T05:33:34.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47c5600700 1 -- 172.21.15.131:0/1237708215 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4018680 con 0x7f47c8102b10 2024-05-10T05:33:34.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.900+0000 7f47ce000700 1 -- 172.21.15.131:0/1237708215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c81a2620 con 0x7f47c8102b10 2024-05-10T05:33:34.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.905+0000 7f47c5600700 1 -- 172.21.15.131:0/1237708215 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f47b4010430 con 0x7f47c8102b10 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.906+0000 7f47e75d9b80 1 Processor -- start 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47c5600700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f47b00a1120 0x7f47b00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47e75d9b80 1 -- start start 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47c5600700 1 -- 172.21.15.131:0/1237708215 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f47b4014070 con 0x7f47c8102b10 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47cd600700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f47b00a1120 0x7f47b00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbaaec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 0x5567dbab5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567db9d41d0 con 0x5567dbaad5b0 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567db9d1b90 con 0x5567dbaaf170 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567db9cd480 con 0x5567dbaae820 2024-05-10T05:33:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47ccc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47ccc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34294/0 (socket says 172.21.15.131:34294) 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.907+0000 7f47ccc00700 1 -- 172.21.15.131:0/1665158698 learned_addr learned my addr 172.21.15.131:0/1665158698 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47c7e00700 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbaaec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47cd600700 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 0x5567dbab5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47ccc00700 1 -- 172.21.15.131:0/1665158698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 msgr2=0x5567dbaaec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47ccc00700 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbaaec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47ccc00700 1 -- 172.21.15.131:0/1665158698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 msgr2=0x5567dbab5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47ccc00700 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 0x5567dbab5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47ccc00700 1 -- 172.21.15.131:0/1665158698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567db84e430 con 0x5567dbaad5b0 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47c7e00700 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbaaec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47cd600700 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 0x5567dbab5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47cd600700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f47b00a1120 0x7f47b00a3630 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009c10 tx=0x7f47c0009920 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.908+0000 7f47ccc00700 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaad9e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009990 tx=0x7f47bc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d74c5ed0516d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.909+0000 7f47b9c00700 1 -- 172.21.15.131:0/1665158698 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00ba50 con 0x5567dbaad5b0 2024-05-10T05:33:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.909+0000 7f47b9c00700 1 -- 172.21.15.131:0/1665158698 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47bc01e040 con 0x5567dbaad5b0 2024-05-10T05:33:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.909+0000 7f47b9c00700 1 -- 172.21.15.131:0/1665158698 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00fd40 con 0x5567dbaad5b0 2024-05-10T05:33:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.909+0000 7f47e75d9b80 1 -- 172.21.15.131:0/1665158698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 msgr2=0x5567dbaad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.909+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaad9e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009990 tx=0x7f47bc005fd0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.910+0000 7f47e75d9b80 1 -- 172.21.15.131:0/1665158698 shutdown_connections 2024-05-10T05:33:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.910+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaad9e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.910+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 0x5567dbab5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.910+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/1665158698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbaaec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.910+0000 7f47e75d9b80 1 -- 172.21.15.131:0/1665158698 >> 172.21.15.131:0/1665158698 conn(0x5567dba0fae0 msgr2=0x5567dbaa83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.910+0000 7f47e75d9b80 1 -- 172.21.15.131:0/1665158698 shutdown_connections 2024-05-10T05:33:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.910+0000 7f47e75d9b80 1 -- 172.21.15.131:0/1665158698 wait complete. 2024-05-10T05:33:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.910+0000 7f47e75d9b80 1 -- start start 2024-05-10T05:33:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.910+0000 7f47e75d9b80 1 Processor -- start 2024-05-10T05:33:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaad5b0 0x5567dbab78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbab7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaaf170 0x5567dbabed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567db9695e0 con 0x5567dbaaf170 2024-05-10T05:33:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567db969280 con 0x5567dbaad5b0 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567db850720 con 0x5567dbaae820 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47e75d9b80 10 client.0 _ll_register_callbacks cb 0x5567db972bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47ccc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaad5b0 0x5567dbab78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47ccc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaad5b0 0x5567dbab78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:50374/0 (socket says 172.21.15.131:50374) 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47ccc00700 1 -- 172.21.15.131:0/2044616636 learned_addr learned my addr 172.21.15.131:0/2044616636 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47cd600700 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaaf170 0x5567dbabed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.911+0000 7f47c7e00700 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbab7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.912+0000 7f47ccc00700 1 -- 172.21.15.131:0/2044616636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 msgr2=0x5567dbab7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.912+0000 7f47ccc00700 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbab7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.912+0000 7f47ccc00700 1 -- 172.21.15.131:0/2044616636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaaf170 msgr2=0x5567dbabed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.912+0000 7f47ccc00700 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaaf170 0x5567dbabed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.912+0000 7f47ccc00700 1 -- 172.21.15.131:0/2044616636 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c81a2620 con 0x5567dbaad5b0 2024-05-10T05:33:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.912+0000 7f47ccc00700 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaad5b0 0x5567dbab78d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009ce0 tx=0x7f47bc005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3115ba19ce83cf2a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.912+0000 7f47ae000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.912+0000 7f47ae000700 1 -- 172.21.15.131:0/2044616636 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00bc60 con 0x5567dbaad5b0 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47e75d9b80 1 -- 172.21.15.131:0/2044616636 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567db9262f0 con 0x5567dbaad5b0 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47ae000700 1 -- 172.21.15.131:0/2044616636 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f47bc00bdc0 con 0x5567dbaad5b0 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47ae000700 1 -- 172.21.15.131:0/2044616636 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47bc03b430 con 0x5567dbaad5b0 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47e75d9b80 10 client.25388 fetch_fsmap learned FSMap version 27 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47e75d9b80 1 -- 172.21.15.131:0/2044616636 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f47c810a7f0 con 0x5567dbaad5b0 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47ae000700 1 -- 172.21.15.131:0/2044616636 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567db9262f0 con 0x5567dbaad5b0 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47ae000700 1 -- 172.21.15.131:0/2044616636 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f47bc029030 con 0x5567dbaad5b0 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47e75d9b80 10 client.25388 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47e75d9b80 10 client.25388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47e75d9b80 20 client.25388 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5567dbac3030 0x5567dbac5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47e75d9b80 4 client.25388 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:33:34.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.913+0000 7f47e75d9b80 1 -- 172.21.15.131:0/2044616636 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567dba7c800 con 0x5567dbac3030 2024-05-10T05:33:34.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.914+0000 7f47c7e00700 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5567dbac3030 0x5567dbac5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.914+0000 7f47c7e00700 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5567dbac3030 0x5567dbac5480 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79f8b9be88f3f1ec server_cookie=104fd0870ec7ae68 in_seq=0 out_seq=0 2024-05-10T05:33:34.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.914+0000 7f47ae000700 10 client.25388 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47ae000700 1 -- 172.21.15.131:0/2044616636 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567dba7c800 con 0x5567dbac3030 2024-05-10T05:33:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47ae000700 10 client.25388 handle_command_reply: tid=0 2024-05-10T05:33:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 client.25388 shutdown 2024-05-10T05:33:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 -- 172.21.15.131:0/2044616636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaad5b0 msgr2=0x5567dbab78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaad5b0 0x5567dbab78d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009ce0 tx=0x7f47bc005a40 comp rx=0 tx=0).stop 2024-05-10T05:33:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 -- 172.21.15.131:0/2044616636 shutdown_connections 2024-05-10T05:33:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5567dbac3030 0x5567dbac5480 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaaf170 0x5567dbabed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaad5b0 0x5567dbab78d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/2044616636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbab7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47ae000700 0 client.25388 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 -- 172.21.15.131:0/2044616636 >> 172.21.15.131:0/2044616636 conn(0x5567dba0fae0 msgr2=0x5567dbaac0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 -- 172.21.15.131:0/2044616636 shutdown_connections 2024-05-10T05:33:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.915+0000 7f47e75d9b80 1 -- 172.21.15.131:0/2044616636 wait complete. 2024-05-10T05:33:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.916+0000 7f47e75d9b80 20 client.25388 trim_cache size 0 max 16384 2024-05-10T05:33:34.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.933+0000 7f47e75d9b80 1 Processor -- start 2024-05-10T05:33:34.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.933+0000 7f47e75d9b80 1 -- start start 2024-05-10T05:33:34.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaa7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaae820 0x5567dbabed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaaf170 0x5567dbaa8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567db9262f0 con 0x5567dbaad5b0 2024-05-10T05:33:34.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567dbaa4b60 con 0x5567dbaae820 2024-05-10T05:33:34.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567dbaa9c20 con 0x5567dbaaf170 2024-05-10T05:33:34.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47ccc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaa7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47cd600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaaf170 0x5567dbaa8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47ccc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaa7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34308/0 (socket says 172.21.15.131:34308) 2024-05-10T05:33:34.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47ccc00700 1 -- 172.21.15.131:0/3074403008 learned_addr learned my addr 172.21.15.131:0/3074403008 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:34.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47c7e00700 1 --2- 172.21.15.131:0/3074403008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaae820 0x5567dbabed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47ccc00700 1 -- 172.21.15.131:0/3074403008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaaf170 msgr2=0x5567dbaa8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:34.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47ccc00700 1 --2- 172.21.15.131:0/3074403008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaaf170 0x5567dbaa8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47ccc00700 1 -- 172.21.15.131:0/3074403008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaae820 msgr2=0x5567dbabed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:34.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47ccc00700 1 --2- 172.21.15.131:0/3074403008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaae820 0x5567dbabed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47ccc00700 1 -- 172.21.15.131:0/3074403008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567db7fabe0 con 0x5567dbaad5b0 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.934+0000 7f47c7e00700 1 --2- 172.21.15.131:0/3074403008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaae820 0x5567dbabed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.935+0000 7f47ccc00700 1 --2- 172.21.15.131:0/3074403008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaa7c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f47bc03d040 tx=0x7f47bc00b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d5c4b329753b6c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.935+0000 7f47ba600700 1 -- 172.21.15.131:0/3074403008 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00eb30 con 0x5567dbaad5b0 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.935+0000 7f47ba600700 1 -- 172.21.15.131:0/3074403008 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47bc00ec90 con 0x5567dbaad5b0 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.935+0000 7f47ba600700 1 -- 172.21.15.131:0/3074403008 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00f990 con 0x5567dbaad5b0 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.936+0000 7f47e75d9b80 1 -- 172.21.15.131:0/3074403008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 msgr2=0x5567dbaa7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.936+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/3074403008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaa7c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f47bc03d040 tx=0x7f47bc00b770 comp rx=0 tx=0).stop 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.936+0000 7f47e75d9b80 1 -- 172.21.15.131:0/3074403008 shutdown_connections 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.936+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/3074403008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbaa7c30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.936+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/3074403008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaaf170 0x5567dbaa8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.936+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/3074403008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaae820 0x5567dbabed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.936+0000 7f47e75d9b80 1 -- 172.21.15.131:0/3074403008 >> 172.21.15.131:0/3074403008 conn(0x5567dba0fae0 msgr2=0x5567dbab2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.936+0000 7f47e75d9b80 1 -- 172.21.15.131:0/3074403008 shutdown_connections 2024-05-10T05:33:34.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.936+0000 7f47e75d9b80 1 -- 172.21.15.131:0/3074403008 wait complete. 2024-05-10T05:33:34.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.936+0000 7f47e75d9b80 1 -- start start 2024-05-10T05:33:34.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.937+0000 7f47e75d9b80 1 Processor -- start 2024-05-10T05:33:34.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.937+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbabca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.937+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbabcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.937+0000 7f47e75d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 0x5567dbabdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.937+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567dbabe3f0 con 0x5567dbaad5b0 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.937+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567dbabe560 con 0x5567dbaaf170 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.937+0000 7f47e75d9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567dbabe720 con 0x5567dbaae820 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.937+0000 7f47e75d9b80 10 client.0 _ll_register_callbacks cb 0x5567db972bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.937+0000 7f47ccc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbabca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47c7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbabcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47ccc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbabca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34324/0 (socket says 172.21.15.131:34324) 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47ccc00700 1 -- 172.21.15.131:0/4093122457 learned_addr learned my addr 172.21.15.131:0/4093122457 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47cd600700 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 0x5567dbabdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47c7e00700 1 -- 172.21.15.131:0/4093122457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 msgr2=0x5567dbabdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47c7e00700 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 0x5567dbabdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47c7e00700 1 -- 172.21.15.131:0/4093122457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 msgr2=0x5567dbabca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47c7e00700 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbabca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47c7e00700 1 -- 172.21.15.131:0/4093122457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c810a7f0 con 0x5567dbaae820 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47ccc00700 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbabca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47c7e00700 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbabcf80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f47b400c5b0 tx=0x7f47b400efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69ba6989f3e5d496 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.938+0000 7f47ae000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47ae000700 1 -- 172.21.15.131:0/4093122457 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b409dc40 con 0x5567dbaae820 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47ae000700 1 -- 172.21.15.131:0/4093122457 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f47b4010e40 con 0x5567dbaae820 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47e75d9b80 1 -- 172.21.15.131:0/4093122457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47bc01e040 con 0x5567dbaae820 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47ae000700 1 -- 172.21.15.131:0/4093122457 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b40b81e0 con 0x5567dbaae820 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47e75d9b80 10 client.25411 fetch_fsmap learned FSMap version 27 2024-05-10T05:33:34.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47e75d9b80 1 -- 172.21.15.131:0/4093122457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5567db9935e0 con 0x5567dbaae820 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47ae000700 1 -- 172.21.15.131:0/4093122457 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47bc01e040 con 0x5567dbaae820 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47ae000700 1 -- 172.21.15.131:0/4093122457 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f47b40a1f00 con 0x5567dbaae820 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47e75d9b80 10 client.25411 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47e75d9b80 10 client.25411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47e75d9b80 20 client.25411 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5567dbb1a350 0x5567dbb1c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47e75d9b80 4 client.25411 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.939+0000 7f47e75d9b80 1 -- 172.21.15.131:0/4093122457 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5567db9c5f40 con 0x5567dbb1a350 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.940+0000 7f47ccc00700 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5567dbb1a350 0x5567dbb1c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.940+0000 7f47ae000700 10 client.25411 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.940+0000 7f47ccc00700 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5567dbb1a350 0x5567dbb1c770 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1867f71b4180db23 server_cookie=4738fe0540774c19 in_seq=0 out_seq=0 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.941+0000 7f47ae000700 1 -- 172.21.15.131:0/4093122457 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5567db9c5f40 con 0x5567dbb1a350 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.941+0000 7f47ae000700 10 client.25411 handle_command_reply: tid=0 2024-05-10T05:33:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.941+0000 7f47e75d9b80 1 client.25411 shutdown 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 1 -- 172.21.15.131:0/4093122457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 msgr2=0x5567dbabcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbabcf80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f47b400c5b0 tx=0x7f47b400efd0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 1 -- 172.21.15.131:0/4093122457 shutdown_connections 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5567dbb1a350 0x5567dbb1c770 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567dbaad5b0 0x5567dbabca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567dbaaf170 0x5567dbabdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47ae000700 0 client.25411 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 1 --2- 172.21.15.131:0/4093122457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5567dbaae820 0x5567dbabcf80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 1 -- 172.21.15.131:0/4093122457 >> 172.21.15.131:0/4093122457 conn(0x5567dba0fae0 msgr2=0x5567dbaa7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 1 -- 172.21.15.131:0/4093122457 shutdown_connections 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 1 -- 172.21.15.131:0/4093122457 wait complete. 2024-05-10T05:33:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.942+0000 7f47e75d9b80 20 client.25411 trim_cache size 0 max 16384 2024-05-10T05:33:34.948 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:33:34.948 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:33:34.948 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "1cd57b6e-9d02-404b-9c62-4dc4202961f1", 2024-05-10T05:33:34.948 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:33:34.948 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:33:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 -- 172.21.15.131:0/1237708215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f47b00a1120 msgr2=0x7f47b00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f47b00a1120 0x7f47b00a3630 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009c10 tx=0x7f47c0009920 comp rx=0 tx=0).stop 2024-05-10T05:33:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 -- 172.21.15.131:0/1237708215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8102b10 msgr2=0x7f47c819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8102b10 0x7f47c819a600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47b400ea20 tx=0x7f47b400ed30 comp rx=0 tx=0).stop 2024-05-10T05:33:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 -- 172.21.15.131:0/1237708215 shutdown_connections 2024-05-10T05:33:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f47b00a1120 0x7f47b00a3630 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c81034b0 0x7f47c81a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81019f0 0x7f47c819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 --2- 172.21.15.131:0/1237708215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c8102b10 0x7f47c819a600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 -- 172.21.15.131:0/1237708215 >> 172.21.15.131:0/1237708215 conn(0x7f47c80fd000 msgr2=0x7f47c8073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:34.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 -- 172.21.15.131:0/1237708215 shutdown_connections 2024-05-10T05:33:34.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:34.943+0000 7f47ce000700 1 -- 172.21.15.131:0/1237708215 wait complete. 2024-05-10T05:33:34.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:33:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.251+0000 7fbe98200700 1 -- 172.21.15.131:0/6772293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe901078a0 msgr2=0x7fbe90109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.251+0000 7fbe98200700 1 --2- 172.21.15.131:0/6772293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe901078a0 0x7fbe90109d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800b0a0 tx=0x7fbe8800b3b0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.251+0000 7fbe98200700 1 -- 172.21.15.131:0/6772293 shutdown_connections 2024-05-10T05:33:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.251+0000 7fbe98200700 1 --2- 172.21.15.131:0/6772293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe901078a0 0x7fbe90109d80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.251+0000 7fbe98200700 1 --2- 172.21.15.131:0/6772293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe90069050 0x7fbe90104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.251+0000 7fbe98200700 1 --2- 172.21.15.131:0/6772293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90104d30 0x7fbe90107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.251+0000 7fbe98200700 1 -- 172.21.15.131:0/6772293 >> 172.21.15.131:0/6772293 conn(0x7fbe900fc0d0 msgr2=0x7fbe900fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:35.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.251+0000 7fbe98200700 1 -- 172.21.15.131:0/6772293 shutdown_connections 2024-05-10T05:33:35.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.251+0000 7fbe98200700 1 -- 172.21.15.131:0/6772293 wait complete. 2024-05-10T05:33:35.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.253+0000 7fbe98200700 1 Processor -- start 2024-05-10T05:33:35.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.253+0000 7fbe98200700 1 -- start start 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.253+0000 7fbe98200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90069050 0x7fbe90198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.253+0000 7fbe98200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe90104d30 0x7fbe90198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.253+0000 7fbe98200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe901078a0 0x7fbe9019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.253+0000 7fbe98200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019fb10 con 0x7fbe90069050 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.253+0000 7fbe98200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019fc80 con 0x7fbe901078a0 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.253+0000 7fbe98200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019fe40 con 0x7fbe90104d30 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe96200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe901078a0 0x7fbe9019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe96200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe901078a0 0x7fbe9019f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:50400/0 (socket says 172.21.15.131:50400) 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe96200700 1 -- 172.21.15.131:0/3231712502 learned_addr learned my addr 172.21.15.131:0/3231712502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe94e00700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe90104d30 0x7fbe90198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe95800700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90069050 0x7fbe90198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe96200700 1 -- 172.21.15.131:0/3231712502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe90104d30 msgr2=0x7fbe90198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe96200700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe90104d30 0x7fbe90198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe96200700 1 -- 172.21.15.131:0/3231712502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90069050 msgr2=0x7fbe90198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:35.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe96200700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90069050 0x7fbe90198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe96200700 1 -- 172.21.15.131:0/3231712502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe84009520 con 0x7fbe901078a0 2024-05-10T05:33:35.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.254+0000 7fbe96200700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe901078a0 0x7fbe9019f500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbe88007840 tx=0x7fbe88007870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.255+0000 7fbe8e000700 1 -- 172.21.15.131:0/3231712502 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8800e040 con 0x7fbe901078a0 2024-05-10T05:33:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.255+0000 7fbe8e000700 1 -- 172.21.15.131:0/3231712502 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8801d470 con 0x7fbe901078a0 2024-05-10T05:33:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.255+0000 7fbe98200700 1 -- 172.21.15.131:0/3231712502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe88009d00 con 0x7fbe901078a0 2024-05-10T05:33:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.255+0000 7fbe8e000700 1 -- 172.21.15.131:0/3231712502 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe880052a0 con 0x7fbe901078a0 2024-05-10T05:33:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.255+0000 7fbe98200700 1 -- 172.21.15.131:0/3231712502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe901a0400 con 0x7fbe901078a0 2024-05-10T05:33:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.258+0000 7fbe8e000700 1 -- 172.21.15.131:0/3231712502 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbe880054c0 con 0x7fbe901078a0 2024-05-10T05:33:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.259+0000 7fbeb00d7b80 1 Processor -- start 2024-05-10T05:33:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.259+0000 7fbe8e000700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbe780a1240 0x7fbe780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbeb00d7b80 1 -- start start 2024-05-10T05:33:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbe8e000700 1 -- 172.21.15.131:0/3231712502 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbe880cdbf0 con 0x7fbe901078a0 2024-05-10T05:33:35.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbe94e00700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbe780a1240 0x7fbe780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed74caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74d570 0x5609ed74d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed7547c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed673150 con 0x5609ed74dec0 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed670b10 con 0x5609ed74d570 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed66c400 con 0x5609ed74c6a0 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbe95800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed74caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbe95800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed74caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35794/0 (socket says 172.21.15.131:35794) 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbe95800700 1 -- 172.21.15.131:0/2603106243 learned_addr learned my addr 172.21.15.131:0/2603106243 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.260+0000 7fbe96200700 1 --2- 172.21.15.131:0/2603106243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed7547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe94e00700 1 --2- 172.21.15.131:0/2603106243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74d570 0x5609ed74d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe95800700 1 -- 172.21.15.131:0/2603106243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74d570 msgr2=0x5609ed74d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe95800700 1 --2- 172.21.15.131:0/2603106243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74d570 0x5609ed74d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe95800700 1 -- 172.21.15.131:0/2603106243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 msgr2=0x5609ed7547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe95800700 1 --2- 172.21.15.131:0/2603106243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed7547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe95800700 1 -- 172.21.15.131:0/2603106243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609ed4ed3b0 con 0x5609ed74c6a0 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe96200700 1 --2- 172.21.15.131:0/2603106243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed7547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:33:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe94e00700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbe780a1240 0x7fbe780a3750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbe901990d0 tx=0x7fbe7c011720 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:35.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe95800700 1 --2- 172.21.15.131:0/2603106243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed74caf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009eb0 tx=0x7fbe84011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee4fa359239f894f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:35.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe82a00700 1 -- 172.21.15.131:0/2603106243 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400e670 con 0x5609ed74c6a0 2024-05-10T05:33:35.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe82a00700 1 -- 172.21.15.131:0/2603106243 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8400ec90 con 0x5609ed74c6a0 2024-05-10T05:33:35.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.261+0000 7fbe82a00700 1 -- 172.21.15.131:0/2603106243 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84019d30 con 0x5609ed74c6a0 2024-05-10T05:33:35.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.262+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/2603106243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 msgr2=0x5609ed74caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:35.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.262+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/2603106243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed74caf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009eb0 tx=0x7fbe84011720 comp rx=0 tx=0).stop 2024-05-10T05:33:35.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.262+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/2603106243 shutdown_connections 2024-05-10T05:33:35.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.262+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/2603106243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed7547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.262+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/2603106243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed74caf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.262+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/2603106243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74d570 0x5609ed74d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.262+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/2603106243 >> 172.21.15.131:0/2603106243 conn(0x5609ed6aea60 msgr2=0x5609ed746fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.262+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/2603106243 shutdown_connections 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.262+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/2603106243 wait complete. 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.263+0000 7fbeb00d7b80 1 -- start start 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.263+0000 7fbeb00d7b80 1 Processor -- start 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74c6a0 0x5609ed756710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74d570 0x5609ed756c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed75dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed608560 con 0x5609ed74dec0 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed608200 con 0x5609ed74c6a0 2024-05-10T05:33:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed4ef6a0 con 0x5609ed74d570 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbeb00d7b80 10 client.0 _ll_register_callbacks cb 0x5609ed611b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbe96200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed75dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbe94e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74d570 0x5609ed756c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbe96200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed75dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34372/0 (socket says 172.21.15.131:34372) 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbe96200700 1 -- 172.21.15.131:0/4115191838 learned_addr learned my addr 172.21.15.131:0/4115191838 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbe94e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74d570 0x5609ed756c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35806/0 (socket says 172.21.15.131:35806) 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.264+0000 7fbe95800700 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74c6a0 0x5609ed756710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbe96200700 1 -- 172.21.15.131:0/4115191838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74d570 msgr2=0x5609ed756c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbe96200700 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74d570 0x5609ed756c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbe96200700 1 -- 172.21.15.131:0/4115191838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74c6a0 msgr2=0x5609ed756710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbe96200700 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74c6a0 0x5609ed756710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbe96200700 1 -- 172.21.15.131:0/4115191838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe84009b10 con 0x5609ed74dec0 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbe96200700 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed75dbb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbe880cb690 tx=0x7fbe8809af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ed6a1395e94b43 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbe77400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:33:35.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbe77400700 1 -- 172.21.15.131:0/4115191838 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8809eea0 con 0x5609ed74dec0 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbe77400700 1 -- 172.21.15.131:0/4115191838 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbe880a3460 con 0x5609ed74dec0 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/4115191838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609ed5c5270 con 0x5609ed74dec0 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbe77400700 1 -- 172.21.15.131:0/4115191838 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe880a10b0 con 0x5609ed74dec0 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbeb00d7b80 10 client.15399 fetch_fsmap learned FSMap version 27 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.265+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/4115191838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5609ed499b60 con 0x5609ed74dec0 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.266+0000 7fbe77400700 1 -- 172.21.15.131:0/4115191838 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609ed5c5270 con 0x5609ed74dec0 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.266+0000 7fbe77400700 1 -- 172.21.15.131:0/4115191838 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe8809a560 con 0x5609ed74dec0 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.266+0000 7fbeb00d7b80 10 client.15399 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.266+0000 7fbeb00d7b80 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.266+0000 7fbeb00d7b80 20 client.15399 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.266+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5609ed762130 0x5609ed764550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.266+0000 7fbeb00d7b80 4 client.15399 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:33:35.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.266+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/4115191838 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609ed71b780 con 0x5609ed762130 2024-05-10T05:33:35.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.266+0000 7fbe95800700 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5609ed762130 0x5609ed764550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.267+0000 7fbe95800700 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5609ed762130 0x5609ed764550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c412d916528c5bf8 server_cookie=16cdfa1c7e873ac8 in_seq=0 out_seq=0 2024-05-10T05:33:35.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.267+0000 7fbe77400700 10 client.15399 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbe77400700 1 -- 172.21.15.131:0/4115191838 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609ed71b780 con 0x5609ed762130 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbe77400700 10 client.15399 handle_command_reply: tid=0 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 client.15399 shutdown 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/4115191838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 msgr2=0x5609ed75dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed75dbb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbe880cb690 tx=0x7fbe8809af70 comp rx=0 tx=0).stop 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/4115191838 shutdown_connections 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5609ed762130 0x5609ed764550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74dec0 0x5609ed75dbb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74c6a0 0x5609ed756710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/4115191838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74d570 0x5609ed756c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/4115191838 >> 172.21.15.131:0/4115191838 conn(0x5609ed6aea60 msgr2=0x5609ed746fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbe77400700 0 client.15399 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:35.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/4115191838 shutdown_connections 2024-05-10T05:33:35.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.268+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/4115191838 wait complete. 2024-05-10T05:33:35.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.269+0000 7fbeb00d7b80 20 client.15399 trim_cache size 0 max 16384 2024-05-10T05:33:35.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.286+0000 7fbeb00d7b80 1 Processor -- start 2024-05-10T05:33:35.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.286+0000 7fbeb00d7b80 1 -- start start 2024-05-10T05:33:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.286+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed7513d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74d570 0x5609ed751910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 0x5609ed7566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed74e2d0 con 0x5609ed74d570 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed7481e0 con 0x5609ed74dec0 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed74d230 con 0x5609ed74c6a0 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbe95800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed7513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbe94e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74d570 0x5609ed751910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbe96200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 0x5609ed7566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbe96200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 0x5609ed7566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:50418/0 (socket says 172.21.15.131:50418) 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbe96200700 1 -- 172.21.15.131:0/3391085234 learned_addr learned my addr 172.21.15.131:0/3391085234 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbe96200700 1 -- 172.21.15.131:0/3391085234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 msgr2=0x5609ed7513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbe96200700 1 --2- 172.21.15.131:0/3391085234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed7513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbe96200700 1 -- 172.21.15.131:0/3391085234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74d570 msgr2=0x5609ed751910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbe96200700 1 --2- 172.21.15.131:0/3391085234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74d570 0x5609ed751910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.287+0000 7fbe96200700 1 -- 172.21.15.131:0/3391085234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609ed632560 con 0x5609ed74dec0 2024-05-10T05:33:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.288+0000 7fbe96200700 1 --2- 172.21.15.131:0/3391085234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 0x5609ed7566d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbe8809b000 tx=0x7fbe880973b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37f1230b10a37edc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:35.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.288+0000 7fbe83400700 1 -- 172.21.15.131:0/3391085234 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe880b7070 con 0x5609ed74dec0 2024-05-10T05:33:35.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.288+0000 7fbe83400700 1 -- 172.21.15.131:0/3391085234 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe880a3460 con 0x5609ed74dec0 2024-05-10T05:33:35.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.288+0000 7fbe83400700 1 -- 172.21.15.131:0/3391085234 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8801d8e0 con 0x5609ed74dec0 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3391085234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 msgr2=0x5609ed7566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/3391085234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 0x5609ed7566d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbe8809b000 tx=0x7fbe880973b0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3391085234 shutdown_connections 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/3391085234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 0x5609ed7566d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/3391085234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed7513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/3391085234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74d570 0x5609ed751910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3391085234 >> 172.21.15.131:0/3391085234 conn(0x5609ed6aea60 msgr2=0x5609ed755c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3391085234 shutdown_connections 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3391085234 wait complete. 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 -- start start 2024-05-10T05:33:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.289+0000 7fbeb00d7b80 1 Processor -- start 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed7b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74d570 0x5609ed7b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbeb00d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 0x5609ed7bade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed7bb320 con 0x5609ed74d570 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed7bb460 con 0x5609ed74dec0 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbeb00d7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ed7bb5f0 con 0x5609ed74c6a0 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbeb00d7b80 10 client.0 _ll_register_callbacks cb 0x5609ed611b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbe95800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed7b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbe95800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed7b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35824/0 (socket says 172.21.15.131:35824) 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbe95800700 1 -- 172.21.15.131:0/3876140042 learned_addr learned my addr 172.21.15.131:0/3876140042 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbe94e00700 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74d570 0x5609ed7b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbe96200700 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 0x5609ed7bade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbe95800700 1 -- 172.21.15.131:0/3876140042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 msgr2=0x5609ed7bade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbe95800700 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 0x5609ed7bade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbe95800700 1 -- 172.21.15.131:0/3876140042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74d570 msgr2=0x5609ed7b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbe95800700 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74d570 0x5609ed7b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.290+0000 7fbe95800700 1 -- 172.21.15.131:0/3876140042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609ed499b60 con 0x5609ed74c6a0 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.291+0000 7fbe95800700 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed7b3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbe84011b20 tx=0x7fbe84009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=329cf51c72cd26f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.291+0000 7fbe77400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.291+0000 7fbe77400700 1 -- 172.21.15.131:0/3876140042 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400ea30 con 0x5609ed74c6a0 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.291+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3876140042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe8400ec90 con 0x5609ed74c6a0 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.291+0000 7fbe77400700 1 -- 172.21.15.131:0/3876140042 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbe84005040 con 0x5609ed74c6a0 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.291+0000 7fbe77400700 1 -- 172.21.15.131:0/3876140042 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe84005200 con 0x5609ed74c6a0 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.291+0000 7fbeb00d7b80 10 client.25429 fetch_fsmap learned FSMap version 27 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.291+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3876140042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5609ed4f1cd0 con 0x5609ed74c6a0 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.292+0000 7fbe77400700 1 -- 172.21.15.131:0/3876140042 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8400ec90 con 0x5609ed74c6a0 2024-05-10T05:33:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.292+0000 7fbe77400700 1 -- 172.21.15.131:0/3876140042 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe84019550 con 0x5609ed74c6a0 2024-05-10T05:33:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.292+0000 7fbeb00d7b80 10 client.25429 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:33:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.292+0000 7fbeb00d7b80 10 client.25429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:33:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.292+0000 7fbeb00d7b80 20 client.25429 populate_metadata read hostname 'smithi131' 2024-05-10T05:33:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.292+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5609ed7bcf50 0x5609ed7bf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:33:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.292+0000 7fbeb00d7b80 4 client.25429 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:33:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.292+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3876140042 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5609ed664ec0 con 0x5609ed7bcf50 2024-05-10T05:33:35.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.292+0000 7fbe94e00700 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5609ed7bcf50 0x5609ed7bf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:33:35.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbe77400700 10 client.25429 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:35.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbe94e00700 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5609ed7bcf50 0x5609ed7bf370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5abcf8b56de628a3 server_cookie=349ab3705e8b2947 in_seq=0 out_seq=0 2024-05-10T05:33:35.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbe77400700 1 -- 172.21.15.131:0/3876140042 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5609ed664ec0 con 0x5609ed7bcf50 2024-05-10T05:33:35.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbe77400700 10 client.25429 handle_command_reply: tid=0 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbeb00d7b80 1 client.25429 shutdown 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3876140042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 msgr2=0x5609ed7b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed7b3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbe84011b20 tx=0x7fbe84009420 comp rx=0 tx=0).stop 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3876140042 shutdown_connections 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5609ed7bcf50 0x5609ed7bf370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609ed74dec0 0x5609ed7bade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609ed74c6a0 0x5609ed7b3940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbeb00d7b80 1 --2- 172.21.15.131:0/3876140042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609ed74d570 0x5609ed7b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbe77400700 0 client.25429 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.293+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3876140042 >> 172.21.15.131:0/3876140042 conn(0x5609ed6aea60 msgr2=0x5609ed764140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3876140042 shutdown_connections 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbeb00d7b80 1 -- 172.21.15.131:0/3876140042 wait complete. 2024-05-10T05:33:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbeb00d7b80 20 client.25429 trim_cache size 0 max 16384 2024-05-10T05:33:35.299 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:33:35.299 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:33:35.299 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:33:35.299 INFO:teuthology.orchestra.run.smithi131.stdout: "1cd57b6e-9d02-404b-9c62-4dc4202961f1": { 2024-05-10T05:33:35.299 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:33:35.299 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "1cd57b6e-9d02-404b-9c62-4dc4202961f1", 2024-05-10T05:33:35.299 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:33:35.299 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:33:35.299 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:33:35.299 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:33:35.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbe98200700 1 -- 172.21.15.131:0/3231712502 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbe780a1240 msgr2=0x7fbe780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:35.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbe98200700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbe780a1240 0x7fbe780a3750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbe901990d0 tx=0x7fbe7c011720 comp rx=0 tx=0).stop 2024-05-10T05:33:35.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbe98200700 1 -- 172.21.15.131:0/3231712502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe901078a0 msgr2=0x7fbe9019f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:33:35.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbe98200700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe901078a0 0x7fbe9019f500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbe88007840 tx=0x7fbe88007870 comp rx=0 tx=0).stop 2024-05-10T05:33:35.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbe98200700 1 -- 172.21.15.131:0/3231712502 shutdown_connections 2024-05-10T05:33:35.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbe98200700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbe780a1240 0x7fbe780a3750 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbe98200700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90069050 0x7fbe90198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbe98200700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe901078a0 0x7fbe9019f500 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbe98200700 1 --2- 172.21.15.131:0/3231712502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe90104d30 0x7fbe90198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:33:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.294+0000 7fbe98200700 1 -- 172.21.15.131:0/3231712502 >> 172.21.15.131:0/3231712502 conn(0x7fbe900fc0d0 msgr2=0x7fbe90103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:33:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.295+0000 7fbe98200700 1 -- 172.21.15.131:0/3231712502 shutdown_connections 2024-05-10T05:33:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:33:35.295+0000 7fbe98200700 1 -- 172.21.15.131:0/3231712502 wait complete. 2024-05-10T05:33:35.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:34 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2834478221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:35.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:34 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4168210684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:35.320 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:34 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2834478221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:35.320 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:34 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4168210684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:35.322 INFO:tasks.cephfs.filesystem:scrub status for tag:1cd57b6e-9d02-404b-9c62-4dc4202961f1 - {'path': '/', 'tag': '1cd57b6e-9d02-404b-9c62-4dc4202961f1', 'options': 'recursive,force'} 2024-05-10T05:33:35.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:34 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2834478221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:33:35.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:34 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4168210684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:33:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:35 smithi157 ceph-mon[78427]: pgmap v292: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T05:33:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:35 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:33:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:35 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:33:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:35 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:33:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:35 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:33:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:35 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:33:36.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:35 smithi142 ceph-mon[77667]: pgmap v292: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T05:33:36.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:35 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:33:36.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:35 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:33:36.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:35 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:33:36.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:35 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:33:36.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:35 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:33:36.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:35 smithi131 ceph-mon[37264]: pgmap v292: 97 pgs: 97 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T05:33:36.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:35 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:33:36.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:35 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:33:36.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:35 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:33:36.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:35 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:33:36.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:35 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:33:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:37 smithi157 ceph-mon[78427]: pgmap v293: 97 pgs: 97 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-10T05:33:38.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:37 smithi142 ceph-mon[77667]: pgmap v293: 97 pgs: 97 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-10T05:33:38.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:37 smithi131 ceph-mon[37264]: pgmap v293: 97 pgs: 97 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-10T05:33:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:39 smithi157 ceph-mon[78427]: pgmap v294: 97 pgs: 97 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-10T05:33:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:39 smithi142 ceph-mon[77667]: pgmap v294: 97 pgs: 97 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-10T05:33:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:39 smithi131 ceph-mon[37264]: pgmap v294: 97 pgs: 97 active+clean; 653 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-10T05:33:42.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:41 smithi157 ceph-mon[78427]: pgmap v295: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-10T05:33:42.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:41 smithi142 ceph-mon[77667]: pgmap v295: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-10T05:33:42.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:41 smithi131 ceph-mon[37264]: pgmap v295: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-10T05:33:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:43 smithi157 ceph-mon[78427]: pgmap v296: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-10T05:33:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:43 smithi142 ceph-mon[77667]: pgmap v296: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-10T05:33:44.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:43 smithi131 ceph-mon[37264]: pgmap v296: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-10T05:33:46.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:45 smithi157 ceph-mon[78427]: pgmap v297: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-10T05:33:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:45 smithi142 ceph-mon[77667]: pgmap v297: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-10T05:33:46.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:45 smithi131 ceph-mon[37264]: pgmap v297: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-10T05:33:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:47 smithi157 ceph-mon[78427]: pgmap v298: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-10T05:33:48.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:47 smithi142 ceph-mon[77667]: pgmap v298: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-10T05:33:48.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:47 smithi131 ceph-mon[37264]: pgmap v298: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-10T05:33:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:49 smithi157 ceph-mon[78427]: pgmap v299: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-10T05:33:50.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:49 smithi142 ceph-mon[77667]: pgmap v299: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-10T05:33:50.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:49 smithi131 ceph-mon[37264]: pgmap v299: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-10T05:33:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:51 smithi157 ceph-mon[78427]: pgmap v300: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T05:33:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:51 smithi142 ceph-mon[77667]: pgmap v300: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T05:33:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:51 smithi131 ceph-mon[37264]: pgmap v300: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T05:33:53.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:53 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:33:53.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:53 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:33:53.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:53 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:33:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:54 smithi157 ceph-mon[78427]: pgmap v301: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:54 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:54 smithi142 ceph-mon[77667]: pgmap v301: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:54.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:54 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:54 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:54 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:54 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:54 smithi131 ceph-mon[37264]: pgmap v301: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:54 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:55 smithi131 ceph-mon[37264]: pgmap v302: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:33:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:33:55.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:55 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:56.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:56.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:56.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:55 smithi157 ceph-mon[78427]: pgmap v302: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:56.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:33:56.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:33:56.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:55 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:56.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:56.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:56.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:55 smithi142 ceph-mon[77667]: pgmap v302: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:33:56.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:33:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:33:56.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:55 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:33:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:57 smithi131 ceph-mon[37264]: pgmap v303: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:57 smithi157 ceph-mon[78427]: pgmap v303: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:33:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:57 smithi142 ceph-mon[77667]: pgmap v303: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:00.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:33:59 smithi131 ceph-mon[37264]: pgmap v304: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:00.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:33:59 smithi157 ceph-mon[78427]: pgmap v304: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:33:59 smithi142 ceph-mon[77667]: pgmap v304: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:02.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:01 smithi131 ceph-mon[37264]: pgmap v305: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:02.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:01 smithi157 ceph-mon[78427]: pgmap v305: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:02.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:01 smithi142 ceph-mon[77667]: pgmap v305: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:04.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:03 smithi131 ceph-mon[37264]: pgmap v306: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:04.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:03 smithi157 ceph-mon[78427]: pgmap v306: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:04.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:03 smithi142 ceph-mon[77667]: pgmap v306: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:05.324 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:34:05.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.621+0000 7f2c20400700 1 -- 172.21.15.131:0/639245640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c18075140 msgr2=0x7f2c18073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.621+0000 7f2c20400700 1 --2- 172.21.15.131:0/639245640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c18075140 0x7f2c18073600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009990 tx=0x7f2c0c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.621+0000 7f2c20400700 1 -- 172.21.15.131:0/639245640 shutdown_connections 2024-05-10T05:34:05.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.621+0000 7f2c20400700 1 --2- 172.21.15.131:0/639245640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c18075140 0x7f2c18073600 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.621+0000 7f2c20400700 1 --2- 172.21.15.131:0/639245640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c18109ac0 0x7f2c1810bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.621+0000 7f2c20400700 1 --2- 172.21.15.131:0/639245640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c18073b40 0x7f2c18074010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.621+0000 7f2c20400700 1 -- 172.21.15.131:0/639245640 >> 172.21.15.131:0/639245640 conn(0x7f2c180fd210 msgr2=0x7f2c180ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:05.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.621+0000 7f2c20400700 1 -- 172.21.15.131:0/639245640 shutdown_connections 2024-05-10T05:34:05.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.621+0000 7f2c20400700 1 -- 172.21.15.131:0/639245640 wait complete. 2024-05-10T05:34:05.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.622+0000 7f2c20400700 1 Processor -- start 2024-05-10T05:34:05.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.622+0000 7f2c20400700 1 -- start start 2024-05-10T05:34:05.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c20400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c18073b40 0x7f2c1819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c20400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c18075140 0x7f2c1819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c20400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c18109ac0 0x7f2c181a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c20400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181a1bc0 con 0x7f2c18073b40 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c20400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181a1d30 con 0x7f2c18075140 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c20400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181a1ef0 con 0x7f2c18109ac0 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c1da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c18073b40 0x7f2c1819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c1da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c18073b40 0x7f2c1819a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44368/0 (socket says 172.21.15.131:44368) 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c1da00700 1 -- 172.21.15.131:0/2840909739 learned_addr learned my addr 172.21.15.131:0/2840909739 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c1d000700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c18075140 0x7f2c1819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c1e400700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c18109ac0 0x7f2c181a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c1da00700 1 -- 172.21.15.131:0/2840909739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c18109ac0 msgr2=0x7f2c181a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c1da00700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c18109ac0 0x7f2c181a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c1da00700 1 -- 172.21.15.131:0/2840909739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c18075140 msgr2=0x7f2c1819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c1da00700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c18075140 0x7f2c1819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.623+0000 7f2c1da00700 1 -- 172.21.15.131:0/2840909739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c0095f0 con 0x7f2c18073b40 2024-05-10T05:34:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.624+0000 7f2c1da00700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c18073b40 0x7f2c1819a0e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0098e0 tx=0x7f2c0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.624+0000 7f2c16000700 1 -- 172.21.15.131:0/2840909739 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c01d070 con 0x7f2c18073b40 2024-05-10T05:34:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.624+0000 7f2c16000700 1 -- 172.21.15.131:0/2840909739 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0c021410 con 0x7f2c18073b40 2024-05-10T05:34:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.624+0000 7f2c20400700 1 -- 172.21.15.131:0/2840909739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c181a2170 con 0x7f2c18073b40 2024-05-10T05:34:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.624+0000 7f2c16000700 1 -- 172.21.15.131:0/2840909739 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c017650 con 0x7f2c18073b40 2024-05-10T05:34:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.624+0000 7f2c20400700 1 -- 172.21.15.131:0/2840909739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c181a2590 con 0x7f2c18073b40 2024-05-10T05:34:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.627+0000 7f2c16000700 1 -- 172.21.15.131:0/2840909739 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2c0c0217f0 con 0x7f2c18073b40 2024-05-10T05:34:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.629+0000 7f2c383a0b80 1 Processor -- start 2024-05-10T05:34:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.629+0000 7f2c383a0b80 1 -- start start 2024-05-10T05:34:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.629+0000 7f2c16000700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2c000a11c0 0x7f2c000a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c16000700 1 -- 172.21.15.131:0/2840909739 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2c0c0cdc80 con 0x7f2c18073b40 2024-05-10T05:34:05.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f8276a0 0x55744f827af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c1e400700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2c000a11c0 0x7f2c000a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f828980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f828ec0 0x55744f82f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f74e150 con 0x55744f828ec0 2024-05-10T05:34:05.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f74bb10 con 0x55744f8276a0 2024-05-10T05:34:05.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f747400 con 0x55744f828570 2024-05-10T05:34:05.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c1d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f828980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c1da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f8276a0 0x55744f827af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c1d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f828980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40976/0 (socket says 172.21.15.131:40976) 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c1d000700 1 -- 172.21.15.131:0/3248373737 learned_addr learned my addr 172.21.15.131:0/3248373737 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.630+0000 7f2c1e400700 1 --2- 172.21.15.131:0/3248373737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f828ec0 0x55744f82f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c1d000700 1 -- 172.21.15.131:0/3248373737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f8276a0 msgr2=0x55744f827af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c1d000700 1 --2- 172.21.15.131:0/3248373737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f8276a0 0x55744f827af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c1d000700 1 -- 172.21.15.131:0/3248373737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f828ec0 msgr2=0x55744f82f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c1d000700 1 --2- 172.21.15.131:0/3248373737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f828ec0 0x55744f82f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c1d000700 1 -- 172.21.15.131:0/3248373737 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55744f5c83b0 con 0x55744f828570 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c1da00700 1 --2- 172.21.15.131:0/3248373737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f8276a0 0x55744f827af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c1e400700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2c000a11c0 0x7f2c000a36d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009fd0 tx=0x7f2c1000fba0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:05.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c1d000700 1 --2- 172.21.15.131:0/3248373737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f828980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009d20 tx=0x7f2c04011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f720d19f3ac5063c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:05.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c0aa00700 1 -- 172.21.15.131:0/3248373737 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04004a30 con 0x55744f828570 2024-05-10T05:34:05.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c0aa00700 1 -- 172.21.15.131:0/3248373737 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c040054e0 con 0x55744f828570 2024-05-10T05:34:05.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.631+0000 7f2c0aa00700 1 -- 172.21.15.131:0/3248373737 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04019830 con 0x55744f828570 2024-05-10T05:34:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.632+0000 7f2c383a0b80 1 -- 172.21.15.131:0/3248373737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 msgr2=0x55744f828980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.632+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/3248373737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f828980 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009d20 tx=0x7f2c04011720 comp rx=0 tx=0).stop 2024-05-10T05:34:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.632+0000 7f2c383a0b80 1 -- 172.21.15.131:0/3248373737 shutdown_connections 2024-05-10T05:34:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.632+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/3248373737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f828ec0 0x55744f82f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.632+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/3248373737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f8276a0 0x55744f827af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.632+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/3248373737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f828980 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.632+0000 7f2c383a0b80 1 -- 172.21.15.131:0/3248373737 >> 172.21.15.131:0/3248373737 conn(0x55744f789a60 msgr2=0x55744f821fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.632+0000 7f2c383a0b80 1 -- 172.21.15.131:0/3248373737 shutdown_connections 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.632+0000 7f2c383a0b80 1 -- 172.21.15.131:0/3248373737 wait complete. 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.633+0000 7f2c383a0b80 1 -- start start 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.633+0000 7f2c383a0b80 1 Processor -- start 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.633+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f8316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828570 0x55744f831c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828ec0 0x55744f838b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f6e3560 con 0x55744f8276a0 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f6e3200 con 0x55744f828570 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f5ca6a0 con 0x55744f828ec0 2024-05-10T05:34:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c383a0b80 10 client.0 _ll_register_callbacks cb 0x55744f6ecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c1da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f8316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c1da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f8316c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44382/0 (socket says 172.21.15.131:44382) 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c1da00700 1 -- 172.21.15.131:0/2889294573 learned_addr learned my addr 172.21.15.131:0/2889294573 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c1d000700 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828570 0x55744f831c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c1e400700 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828ec0 0x55744f838b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c1da00700 1 -- 172.21.15.131:0/2889294573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828ec0 msgr2=0x55744f838b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c1da00700 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828ec0 0x55744f838b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c1da00700 1 -- 172.21.15.131:0/2889294573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828570 msgr2=0x55744f831c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c1da00700 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828570 0x55744f831c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.634+0000 7f2c1da00700 1 -- 172.21.15.131:0/2889294573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c181a2590 con 0x55744f8276a0 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.635+0000 7f2c1da00700 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f8316c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0cb660 tx=0x7f2c0c021b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0a38e814158954 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.635+0000 7f2bff400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.635+0000 7f2bff400700 1 -- 172.21.15.131:0/2889294573 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c09b100 con 0x55744f8276a0 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.635+0000 7f2bff400700 1 -- 172.21.15.131:0/2889294573 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2c0c09ef30 con 0x55744f8276a0 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.635+0000 7f2c383a0b80 1 -- 172.21.15.131:0/2889294573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55744f6a0270 con 0x55744f8276a0 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.635+0000 7f2bff400700 1 -- 172.21.15.131:0/2889294573 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c0c0a34f0 con 0x55744f8276a0 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.635+0000 7f2c383a0b80 10 client.15417 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.635+0000 7f2c383a0b80 1 -- 172.21.15.131:0/2889294573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f2c04009980 con 0x55744f8276a0 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.635+0000 7f2bff400700 1 -- 172.21.15.131:0/2889294573 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55744f6a0270 con 0x55744f8276a0 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.635+0000 7f2bff400700 1 -- 172.21.15.131:0/2889294573 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c0c0a3060 con 0x55744f8276a0 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.636+0000 7f2c383a0b80 10 client.15417 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.636+0000 7f2c383a0b80 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.636+0000 7f2c383a0b80 20 client.15417 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.636+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55744f83ce90 0x55744f83f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.636+0000 7f2c383a0b80 4 client.15417 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.636+0000 7f2c383a0b80 1 -- 172.21.15.131:0/2889294573 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55744f7f6780 con 0x55744f83ce90 2024-05-10T05:34:05.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.636+0000 7f2c1d000700 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55744f83ce90 0x55744f83f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.637+0000 7f2bff400700 10 client.15417 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:05.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.637+0000 7f2c1d000700 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55744f83ce90 0x55744f83f2e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f024b4ad33148978 server_cookie=c5945ffa0a04076c in_seq=0 out_seq=0 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.637+0000 7f2bff400700 1 -- 172.21.15.131:0/2889294573 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55744f7f6780 con 0x55744f83ce90 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.637+0000 7f2bff400700 10 client.15417 handle_command_reply: tid=0 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.637+0000 7f2c383a0b80 1 client.15417 shutdown 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.637+0000 7f2c383a0b80 1 -- 172.21.15.131:0/2889294573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 msgr2=0x55744f8316c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.637+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f8316c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0cb660 tx=0x7f2c0c021b90 comp rx=0 tx=0).stop 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.638+0000 7f2c383a0b80 1 -- 172.21.15.131:0/2889294573 shutdown_connections 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.638+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55744f83ce90 0x55744f83f2e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.638+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f8316c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.638+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828ec0 0x55744f838b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.638+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/2889294573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828570 0x55744f831c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.638+0000 7f2c383a0b80 1 -- 172.21.15.131:0/2889294573 >> 172.21.15.131:0/2889294573 conn(0x55744f789a60 msgr2=0x55744f821fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.638+0000 7f2bff400700 0 client.15417 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.638+0000 7f2c383a0b80 1 -- 172.21.15.131:0/2889294573 shutdown_connections 2024-05-10T05:34:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.638+0000 7f2c383a0b80 1 -- 172.21.15.131:0/2889294573 wait complete. 2024-05-10T05:34:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.638+0000 7f2c383a0b80 20 client.15417 trim_cache size 0 max 16384 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c383a0b80 1 Processor -- start 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c383a0b80 1 -- start start 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f789a60 0x55744f836b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f837050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828570 0x55744f82c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f8395c0 con 0x55744f8276a0 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f82f710 con 0x55744f828570 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f83cc30 con 0x55744f789a60 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c1d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f837050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c1d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f837050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44390/0 (socket says 172.21.15.131:44390) 2024-05-10T05:34:05.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c1e400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828570 0x55744f82c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c1d000700 1 -- 172.21.15.131:0/460407657 learned_addr learned my addr 172.21.15.131:0/460407657 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:05.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.656+0000 7f2c1da00700 1 --2- 172.21.15.131:0/460407657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f789a60 0x55744f836b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.657+0000 7f2c1da00700 1 -- 172.21.15.131:0/460407657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828570 msgr2=0x55744f82c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:05.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.657+0000 7f2c1da00700 1 --2- 172.21.15.131:0/460407657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828570 0x55744f82c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.657+0000 7f2c1da00700 1 -- 172.21.15.131:0/460407657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 msgr2=0x55744f837050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:05.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.657+0000 7f2c1da00700 1 --2- 172.21.15.131:0/460407657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f837050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.657+0000 7f2c1da00700 1 -- 172.21.15.131:0/460407657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55744f574b60 con 0x55744f789a60 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.657+0000 7f2c1d000700 1 --2- 172.21.15.131:0/460407657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f837050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.657+0000 7f2c1da00700 1 --2- 172.21.15.131:0/460407657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f789a60 0x55744f836b10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0a2fc0 tx=0x7f2c0c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb4be85fdbde0f3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.657+0000 7f2c0b400700 1 -- 172.21.15.131:0/460407657 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c00fad0 con 0x55744f789a60 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.657+0000 7f2c0b400700 1 -- 172.21.15.131:0/460407657 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0c021e90 con 0x55744f789a60 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.657+0000 7f2c0b400700 1 -- 172.21.15.131:0/460407657 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c09a5f0 con 0x55744f789a60 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.658+0000 7f2c383a0b80 1 -- 172.21.15.131:0/460407657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f789a60 msgr2=0x55744f836b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.658+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/460407657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f789a60 0x55744f836b10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0a2fc0 tx=0x7f2c0c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.658+0000 7f2c383a0b80 1 -- 172.21.15.131:0/460407657 shutdown_connections 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.658+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/460407657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828570 0x55744f82c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.658+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/460407657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f789a60 0x55744f836b10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.658+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/460407657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f837050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.658+0000 7f2c383a0b80 1 -- 172.21.15.131:0/460407657 >> 172.21.15.131:0/460407657 conn(0x55744f81e450 msgr2=0x55744f82b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.658+0000 7f2c383a0b80 1 -- 172.21.15.131:0/460407657 shutdown_connections 2024-05-10T05:34:05.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.658+0000 7f2c383a0b80 1 -- 172.21.15.131:0/460407657 wait complete. 2024-05-10T05:34:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.658+0000 7f2c383a0b80 1 -- start start 2024-05-10T05:34:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c383a0b80 1 Processor -- start 2024-05-10T05:34:05.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f88d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f88f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c383a0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828ec0 0x55744f891db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f892390 con 0x55744f8276a0 2024-05-10T05:34:05.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f892500 con 0x55744f828ec0 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c383a0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744f8926c0 con 0x55744f828570 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c383a0b80 10 client.0 _ll_register_callbacks cb 0x55744f6ecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c1da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f88d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c1d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f88f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c1d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f88f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40996/0 (socket says 172.21.15.131:40996) 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2c1d000700 1 -- 172.21.15.131:0/971719450 learned_addr learned my addr 172.21.15.131:0/971719450 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.659+0000 7f2c1da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f88d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44394/0 (socket says 172.21.15.131:44394) 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2c1e400700 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828ec0 0x55744f891db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2c1d000700 1 -- 172.21.15.131:0/971719450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828ec0 msgr2=0x55744f891db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2c1d000700 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828ec0 0x55744f891db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2c1d000700 1 -- 172.21.15.131:0/971719450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 msgr2=0x55744f88d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2c1d000700 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f88d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2c1d000700 1 -- 172.21.15.131:0/971719450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55744f5c83b0 con 0x55744f828570 2024-05-10T05:34:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2c1d000700 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f88f860 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c0401b040 tx=0x7f2c040058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f808252fe108bf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2bff400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2bff400700 1 -- 172.21.15.131:0/971719450 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04009260 con 0x55744f828570 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2c383a0b80 1 -- 172.21.15.131:0/971719450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c040054e0 con 0x55744f828570 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.660+0000 7f2bff400700 1 -- 172.21.15.131:0/971719450 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2c04005290 con 0x55744f828570 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2bff400700 1 -- 172.21.15.131:0/971719450 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c04003680 con 0x55744f828570 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2c383a0b80 10 client.25447 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2c383a0b80 1 -- 172.21.15.131:0/971719450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f2c04009980 con 0x55744f828570 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2bff400700 1 -- 172.21.15.131:0/971719450 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c040054e0 con 0x55744f828570 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2bff400700 1 -- 172.21.15.131:0/971719450 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c04019800 con 0x55744f828570 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2c383a0b80 10 client.25447 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2c383a0b80 10 client.25447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2c383a0b80 20 client.25447 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55744f894050 0x55744f896470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2c383a0b80 4 client.25447 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.661+0000 7f2c383a0b80 1 -- 172.21.15.131:0/971719450 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55744f73fec0 con 0x55744f894050 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.662+0000 7f2c1da00700 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55744f894050 0x55744f896470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.662+0000 7f2c1da00700 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55744f894050 0x55744f896470 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f71a32d6d59a86c0 server_cookie=ffd8800f3e20cb8d in_seq=0 out_seq=0 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.662+0000 7f2bff400700 10 client.25447 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.662+0000 7f2bff400700 1 -- 172.21.15.131:0/971719450 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55744f73fec0 con 0x55744f894050 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.662+0000 7f2bff400700 10 client.25447 handle_command_reply: tid=0 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.662+0000 7f2c383a0b80 1 client.25447 shutdown 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 1 -- 172.21.15.131:0/971719450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 msgr2=0x55744f88f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f88f860 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c0401b040 tx=0x7f2c040058e0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 1 -- 172.21.15.131:0/971719450 shutdown_connections 2024-05-10T05:34:05.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55744f894050 0x55744f896470 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55744f8276a0 0x55744f88d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55744f828ec0 0x55744f891db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 1 --2- 172.21.15.131:0/971719450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55744f828570 0x55744f88f860 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2bff400700 0 client.25447 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:05.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 1 -- 172.21.15.131:0/971719450 >> 172.21.15.131:0/971719450 conn(0x55744f81e450 msgr2=0x55744f82c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:05.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 1 -- 172.21.15.131:0/971719450 shutdown_connections 2024-05-10T05:34:05.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 1 -- 172.21.15.131:0/971719450 wait complete. 2024-05-10T05:34:05.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.663+0000 7f2c383a0b80 20 client.25447 trim_cache size 0 max 16384 2024-05-10T05:34:05.668 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:34:05.668 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:34:05.669 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:34:05.669 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:34:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 -- 172.21.15.131:0/2840909739 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2c000a11c0 msgr2=0x7f2c000a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2c000a11c0 0x7f2c000a36d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009fd0 tx=0x7f2c1000fba0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 -- 172.21.15.131:0/2840909739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c18073b40 msgr2=0x7f2c1819a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c18073b40 0x7f2c1819a0e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0098e0 tx=0x7f2c0c004730 comp rx=0 tx=0).stop 2024-05-10T05:34:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 -- 172.21.15.131:0/2840909739 shutdown_connections 2024-05-10T05:34:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2c000a11c0 0x7f2c000a36d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c18073b40 0x7f2c1819a0e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c18109ac0 0x7f2c181a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 --2- 172.21.15.131:0/2840909739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c18075140 0x7f2c1819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 -- 172.21.15.131:0/2840909739 >> 172.21.15.131:0/2840909739 conn(0x7f2c180fd210 msgr2=0x7f2c18105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 -- 172.21.15.131:0/2840909739 shutdown_connections 2024-05-10T05:34:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.664+0000 7f2c20400700 1 -- 172.21.15.131:0/2840909739 wait complete. 2024-05-10T05:34:05.694 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:34:05.694 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:34:05.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:05 smithi131 ceph-mon[37264]: pgmap v307: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.961+0000 7f2176400700 1 -- 172.21.15.131:0/1931445822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21701034c0 msgr2=0x7f2170109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.961+0000 7f2176400700 1 --2- 172.21.15.131:0/1931445822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21701034c0 0x7f2170109d70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2168009fc0 tx=0x7f216800b330 comp rx=0 tx=0).stop 2024-05-10T05:34:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.961+0000 7f2176400700 1 -- 172.21.15.131:0/1931445822 shutdown_connections 2024-05-10T05:34:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.961+0000 7f2176400700 1 --2- 172.21.15.131:0/1931445822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21701034c0 0x7f2170109d70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.961+0000 7f2176400700 1 --2- 172.21.15.131:0/1931445822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170101a00 0x7f2170101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.961+0000 7f2176400700 1 --2- 172.21.15.131:0/1931445822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170102b20 0x7f2170102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.961+0000 7f2176400700 1 -- 172.21.15.131:0/1931445822 >> 172.21.15.131:0/1931445822 conn(0x7f21700fd010 msgr2=0x7f21700ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.961+0000 7f2176400700 1 -- 172.21.15.131:0/1931445822 shutdown_connections 2024-05-10T05:34:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.961+0000 7f2176400700 1 -- 172.21.15.131:0/1931445822 wait complete. 2024-05-10T05:34:05.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.962+0000 7f2176400700 1 Processor -- start 2024-05-10T05:34:05.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.962+0000 7f2176400700 1 -- start start 2024-05-10T05:34:05.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f2176400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170101a00 0x7f2170199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f2176400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170102b20 0x7f217019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f2176400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21701034c0 0x7f21701a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f2176400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21701a1a70 con 0x7f2170102b20 2024-05-10T05:34:05.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f2176400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21701a1bb0 con 0x7f21701034c0 2024-05-10T05:34:05.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f2176400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21701a1d40 con 0x7f2170101a00 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f216fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170102b20 0x7f217019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f216fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170102b20 0x7f217019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44420/0 (socket says 172.21.15.131:44420) 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f216fe00700 1 -- 172.21.15.131:0/741576666 learned_addr learned my addr 172.21.15.131:0/741576666 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f2175000700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170101a00 0x7f2170199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f2175a00700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21701034c0 0x7f21701a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f216fe00700 1 -- 172.21.15.131:0/741576666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170101a00 msgr2=0x7f2170199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f216fe00700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170101a00 0x7f2170199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f216fe00700 1 -- 172.21.15.131:0/741576666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21701034c0 msgr2=0x7f21701a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f216fe00700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21701034c0 0x7f21701a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.963+0000 7f216fe00700 1 -- 172.21.15.131:0/741576666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2164009520 con 0x7f2170102b20 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.964+0000 7f216fe00700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170102b20 0x7f217019a510 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f215c00ec40 tx=0x7f215c00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:05.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.964+0000 7f216d600700 1 -- 172.21.15.131:0/741576666 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c009a10 con 0x7f2170102b20 2024-05-10T05:34:05.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.964+0000 7f216d600700 1 -- 172.21.15.131:0/741576666 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f215c0044e0 con 0x7f2170102b20 2024-05-10T05:34:05.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.964+0000 7f2176400700 1 -- 172.21.15.131:0/741576666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c0095f0 con 0x7f2170102b20 2024-05-10T05:34:05.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.964+0000 7f216d600700 1 -- 172.21.15.131:0/741576666 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c010430 con 0x7f2170102b20 2024-05-10T05:34:05.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.964+0000 7f2176400700 1 -- 172.21.15.131:0/741576666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2168009c40 con 0x7f2170102b20 2024-05-10T05:34:05.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.965+0000 7f2176400700 1 -- 172.21.15.131:0/741576666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2170068110 con 0x7f2170102b20 2024-05-10T05:34:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.967+0000 7f216d600700 1 -- 172.21.15.131:0/741576666 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f215c003f40 con 0x7f2170102b20 2024-05-10T05:34:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.968+0000 7f216d600700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f21580a11c0 0x7f21580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.969+0000 7f216d600700 1 -- 172.21.15.131:0/741576666 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f215c014070 con 0x7f2170102b20 2024-05-10T05:34:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.969+0000 7f2175000700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f21580a11c0 0x7f21580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:05.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.969+0000 7f2175000700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f21580a11c0 0x7f21580a36d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2170102a20 tx=0x7f2164009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:05.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:05.973+0000 7f216d600700 1 -- 172.21.15.131:0/741576666 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f215c0965c0 con 0x7f2170102b20 2024-05-10T05:34:06.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.227+0000 7f2176400700 1 -- 172.21.15.131:0/741576666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f217010d430 con 0x7f2170102b20 2024-05-10T05:34:06.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.228+0000 7f216d600700 1 -- 172.21.15.131:0/741576666 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f215c099be0 con 0x7f2170102b20 2024-05-10T05:34:06.228 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:34:06.229 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:33:24.915360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:34:06.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 -- 172.21.15.131:0/741576666 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f21580a11c0 msgr2=0x7f21580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f21580a11c0 0x7f21580a36d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2170102a20 tx=0x7f2164009260 comp rx=0 tx=0).stop 2024-05-10T05:34:06.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 -- 172.21.15.131:0/741576666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170102b20 msgr2=0x7f217019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170102b20 0x7f217019a510 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f215c00ec40 tx=0x7f215c00ef50 comp rx=0 tx=0).stop 2024-05-10T05:34:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 -- 172.21.15.131:0/741576666 shutdown_connections 2024-05-10T05:34:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f21580a11c0 0x7f21580a36d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21701034c0 0x7f21701a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2170101a00 0x7f2170199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 --2- 172.21.15.131:0/741576666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2170102b20 0x7f217019a510 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 -- 172.21.15.131:0/741576666 >> 172.21.15.131:0/741576666 conn(0x7f21700fd010 msgr2=0x7f2170073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 -- 172.21.15.131:0/741576666 shutdown_connections 2024-05-10T05:34:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.230+0000 7f2176400700 1 -- 172.21.15.131:0/741576666 wait complete. 2024-05-10T05:34:06.233 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-05-10T05:34:06.258 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:34:06.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:05 smithi157 ceph-mon[78427]: pgmap v307: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:05 smithi142 ceph-mon[77667]: pgmap v307: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:06.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.524+0000 7f5416e00700 1 -- 172.21.15.131:0/3230818457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54101036a0 msgr2=0x7f5410109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.524+0000 7f5416e00700 1 --2- 172.21.15.131:0/3230818457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54101036a0 0x7f5410109f50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5408009fc0 tx=0x7f540800b330 comp rx=0 tx=0).stop 2024-05-10T05:34:06.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.524+0000 7f5416e00700 1 -- 172.21.15.131:0/3230818457 shutdown_connections 2024-05-10T05:34:06.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.524+0000 7f5416e00700 1 --2- 172.21.15.131:0/3230818457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5410101be0 0x7f5410101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.524+0000 7f5416e00700 1 --2- 172.21.15.131:0/3230818457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54101036a0 0x7f5410109f50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.524+0000 7f5416e00700 1 --2- 172.21.15.131:0/3230818457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5410102d00 0x7f5410103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.524+0000 7f5416e00700 1 -- 172.21.15.131:0/3230818457 >> 172.21.15.131:0/3230818457 conn(0x7f54100fd210 msgr2=0x7f54100ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.524+0000 7f5416e00700 1 -- 172.21.15.131:0/3230818457 shutdown_connections 2024-05-10T05:34:06.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.524+0000 7f5416e00700 1 -- 172.21.15.131:0/3230818457 wait complete. 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.525+0000 7f5416e00700 1 Processor -- start 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.525+0000 7f5416e00700 1 -- start start 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5416e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5410101be0 0x7f541019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5416e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410102d00 0x7f541019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5416e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54101036a0 0x7f54101a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5416e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54101a1d20 con 0x7f5410101be0 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5416e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54101a1e90 con 0x7f54101036a0 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5416e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54101a2050 con 0x7f5410102d00 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f540fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5410101be0 0x7f541019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5414e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54101036a0 0x7f54101a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5414e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54101036a0 0x7f54101a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37736/0 (socket says 172.21.15.131:37736) 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f540fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5410101be0 0x7f541019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44448/0 (socket says 172.21.15.131:44448) 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5414e00700 1 -- 172.21.15.131:0/1685921306 learned_addr learned my addr 172.21.15.131:0/1685921306 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f540f400700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410102d00 0x7f541019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5414e00700 1 -- 172.21.15.131:0/1685921306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410102d00 msgr2=0x7f541019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5414e00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410102d00 0x7f541019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5414e00700 1 -- 172.21.15.131:0/1685921306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5410101be0 msgr2=0x7f541019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5414e00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5410101be0 0x7f541019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f5414e00700 1 -- 172.21.15.131:0/1685921306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54000095f0 con 0x7f54101036a0 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.526+0000 7f540fe00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5410101be0 0x7f541019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.527+0000 7f5414e00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54101036a0 0x7f54101a1710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5408007730 tx=0x7f5408007760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.527+0000 7f540cc00700 1 -- 172.21.15.131:0/1685921306 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540800e040 con 0x7f54101036a0 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.527+0000 7f540cc00700 1 -- 172.21.15.131:0/1685921306 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f540801d470 con 0x7f54101036a0 2024-05-10T05:34:06.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.527+0000 7f5416e00700 1 -- 172.21.15.131:0/1685921306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5408009c40 con 0x7f54101036a0 2024-05-10T05:34:06.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.527+0000 7f540cc00700 1 -- 172.21.15.131:0/1685921306 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540801b6c0 con 0x7f54101036a0 2024-05-10T05:34:06.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.527+0000 7f5416e00700 1 -- 172.21.15.131:0/1685921306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54101a2560 con 0x7f54101036a0 2024-05-10T05:34:06.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.530+0000 7f540cc00700 1 -- 172.21.15.131:0/1685921306 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f540801d5e0 con 0x7f54101036a0 2024-05-10T05:34:06.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.531+0000 7f542ece8b80 1 Processor -- start 2024-05-10T05:34:06.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.531+0000 7f542ece8b80 1 -- start start 2024-05-10T05:34:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 0x563f97d3aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f540cc00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53f40a11c0 0x7f53f40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 0x563f97d3b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d427c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f542ece8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97c61150 con 0x563f97d3bec0 2024-05-10T05:34:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f542ece8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97c5eb10 con 0x563f97d3b570 2024-05-10T05:34:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f542ece8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97c5a400 con 0x563f97d3a6a0 2024-05-10T05:34:06.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f540cc00700 1 -- 172.21.15.131:0/1685921306 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f54080cd560 con 0x7f54101036a0 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f540fe00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53f40a11c0 0x7f53f40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f540f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 0x563f97d3aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f5414e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f540f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 0x563f97d3aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41038/0 (socket says 172.21.15.131:41038) 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f540f400700 1 -- 172.21.15.131:0/2063423665 learned_addr learned my addr 172.21.15.131:0/2063423665 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.532+0000 7f540f400700 1 --2- 172.21.15.131:0/2063423665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 0x563f97d3b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.533+0000 7f5414e00700 1 -- 172.21.15.131:0/2063423665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 msgr2=0x563f97d3aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.533+0000 7f5414e00700 1 --2- 172.21.15.131:0/2063423665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 0x563f97d3aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.533+0000 7f5414e00700 1 -- 172.21.15.131:0/2063423665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 msgr2=0x563f97d3b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.533+0000 7f5414e00700 1 --2- 172.21.15.131:0/2063423665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 0x563f97d3b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.533+0000 7f5414e00700 1 -- 172.21.15.131:0/2063423665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f97adb3b0 con 0x563f97d3bec0 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.533+0000 7f5414e00700 1 --2- 172.21.15.131:0/2063423665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d427c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f540809a6e0 tx=0x7f540800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3e7825f58cfd51 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.533+0000 7f540fe00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53f40a11c0 0x7f53f40a36d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5410102c00 tx=0x7f5400009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.533+0000 7f5405600700 1 -- 172.21.15.131:0/2063423665 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540809a9e0 con 0x563f97d3bec0 2024-05-10T05:34:06.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.533+0000 7f5405600700 1 -- 172.21.15.131:0/2063423665 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f540809e810 con 0x563f97d3bec0 2024-05-10T05:34:06.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.533+0000 7f5405600700 1 -- 172.21.15.131:0/2063423665 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54080cb210 con 0x563f97d3bec0 2024-05-10T05:34:06.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.534+0000 7f542ece8b80 1 -- 172.21.15.131:0/2063423665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 msgr2=0x563f97d427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.534+0000 7f542ece8b80 1 --2- 172.21.15.131:0/2063423665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d427c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f540809a6e0 tx=0x7f540800b330 comp rx=0 tx=0).stop 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.534+0000 7f542ece8b80 1 -- 172.21.15.131:0/2063423665 shutdown_connections 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.534+0000 7f542ece8b80 1 --2- 172.21.15.131:0/2063423665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d427c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.534+0000 7f542ece8b80 1 --2- 172.21.15.131:0/2063423665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 0x563f97d3aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.534+0000 7f542ece8b80 1 --2- 172.21.15.131:0/2063423665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 0x563f97d3b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.534+0000 7f542ece8b80 1 -- 172.21.15.131:0/2063423665 >> 172.21.15.131:0/2063423665 conn(0x563f97c9ca60 msgr2=0x563f97d34fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.534+0000 7f542ece8b80 1 -- 172.21.15.131:0/2063423665 shutdown_connections 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.534+0000 7f542ece8b80 1 -- 172.21.15.131:0/2063423665 wait complete. 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.535+0000 7f542ece8b80 1 -- start start 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.535+0000 7f542ece8b80 1 Processor -- start 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.535+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 0x563f97d44520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.535+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 0x563f97d44a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.535+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.535+0000 7f542ece8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97bf6560 con 0x563f97d3bec0 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.535+0000 7f542ece8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97bf6200 con 0x563f97d3b570 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.535+0000 7f542ece8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97add6a0 con 0x563f97d3a6a0 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.535+0000 7f542ece8b80 10 client.0 _ll_register_callbacks cb 0x563f97bffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f5414e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f540f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 0x563f97d44a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f540fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 0x563f97d44520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f540f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 0x563f97d44a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37762/0 (socket says 172.21.15.131:37762) 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f540f400700 1 -- 172.21.15.131:0/3519096048 learned_addr learned my addr 172.21.15.131:0/3519096048 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f540fe00700 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 0x563f97d44520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41046/0 (socket says 172.21.15.131:41046) 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f5414e00700 1 -- 172.21.15.131:0/3519096048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 msgr2=0x563f97d44520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f5414e00700 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 0x563f97d44520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f5414e00700 1 -- 172.21.15.131:0/3519096048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 msgr2=0x563f97d44a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f5414e00700 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 0x563f97d44a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f5414e00700 1 -- 172.21.15.131:0/3519096048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54101a2560 con 0x563f97d3bec0 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f5414e00700 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4b9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f540809a970 tx=0x7f540801daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c57f2f7b2dc2de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f53fa000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f53fa000700 1 -- 172.21.15.131:0/3519096048 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54080cb1f0 con 0x563f97d3bec0 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f53fa000700 1 -- 172.21.15.131:0/3519096048 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f54080a2dd0 con 0x563f97d3bec0 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.536+0000 7f542ece8b80 1 -- 172.21.15.131:0/3519096048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f97bb3270 con 0x563f97d3bec0 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f53fa000700 1 -- 172.21.15.131:0/3519096048 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f540809a9e0 con 0x563f97d3bec0 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f542ece8b80 10 client.15441 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f542ece8b80 1 -- 172.21.15.131:0/3519096048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563f97a87b60 con 0x563f97d3bec0 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f53fa000700 1 -- 172.21.15.131:0/3519096048 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f97bb3270 con 0x563f97d3bec0 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f53fa000700 1 -- 172.21.15.131:0/3519096048 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f54080cd0d0 con 0x563f97d3bec0 2024-05-10T05:34:06.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f542ece8b80 10 client.15441 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f542ece8b80 10 client.15441 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f542ece8b80 20 client.15441 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x563f97d4fc30 0x563f97d52050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f542ece8b80 4 client.15441 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.537+0000 7f542ece8b80 1 -- 172.21.15.131:0/3519096048 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f97d09780 con 0x563f97d4fc30 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.538+0000 7f540fe00700 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x563f97d4fc30 0x563f97d52050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.538+0000 7f540fe00700 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x563f97d4fc30 0x563f97d52050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2dea5f2e3474bfb1 server_cookie=efd44633734c6d87 in_seq=0 out_seq=0 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.538+0000 7f53fa000700 10 client.15441 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.539+0000 7f53fa000700 1 -- 172.21.15.131:0/3519096048 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563f97d09780 con 0x563f97d4fc30 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.539+0000 7f53fa000700 10 client.15441 handle_command_reply: tid=0 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.539+0000 7f542ece8b80 1 client.15441 shutdown 2024-05-10T05:34:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.539+0000 7f542ece8b80 1 -- 172.21.15.131:0/3519096048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 msgr2=0x563f97d4b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.539+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4b9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f540809a970 tx=0x7f540801daf0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.539+0000 7f542ece8b80 1 -- 172.21.15.131:0/3519096048 shutdown_connections 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.539+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x563f97d4fc30 0x563f97d52050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.540+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4b9c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.540+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3a6a0 0x563f97d44520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.540+0000 7f53fa000700 0 client.15441 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.540+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3519096048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3b570 0x563f97d44a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.540+0000 7f542ece8b80 1 -- 172.21.15.131:0/3519096048 >> 172.21.15.131:0/3519096048 conn(0x563f97c9ca60 msgr2=0x563f97d34fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.540+0000 7f542ece8b80 1 -- 172.21.15.131:0/3519096048 shutdown_connections 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.540+0000 7f542ece8b80 1 -- 172.21.15.131:0/3519096048 wait complete. 2024-05-10T05:34:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.540+0000 7f542ece8b80 20 client.15441 trim_cache size 0 max 16384 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.553+0000 7f542ece8b80 1 Processor -- start 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f542ece8b80 1 -- start start 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 0x563f97d52050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3b570 0x563f97d31180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3bec0 0x563f97d4b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f542ece8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97d35ad0 con 0x563f97d3b570 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f542ece8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97d3af50 con 0x563f97d3a6a0 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f542ece8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97d45120 con 0x563f97d3bec0 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f540f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3b570 0x563f97d31180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f540fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 0x563f97d52050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f540f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3b570 0x563f97d31180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44470/0 (socket says 172.21.15.131:44470) 2024-05-10T05:34:06.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f540fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 0x563f97d52050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37776/0 (socket says 172.21.15.131:37776) 2024-05-10T05:34:06.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f540f400700 1 -- 172.21.15.131:0/3592949473 learned_addr learned my addr 172.21.15.131:0/3592949473 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:06.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.554+0000 7f5414e00700 1 --2- 172.21.15.131:0/3592949473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3bec0 0x563f97d4b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.555+0000 7f540fe00700 1 -- 172.21.15.131:0/3592949473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3bec0 msgr2=0x563f97d4b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.555+0000 7f540fe00700 1 --2- 172.21.15.131:0/3592949473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3bec0 0x563f97d4b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.555+0000 7f540fe00700 1 -- 172.21.15.131:0/3592949473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3b570 msgr2=0x563f97d31180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.555+0000 7f540fe00700 1 --2- 172.21.15.131:0/3592949473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3b570 0x563f97d31180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.555+0000 7f540fe00700 1 -- 172.21.15.131:0/3592949473 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f97c20560 con 0x563f97d3a6a0 2024-05-10T05:34:06.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.555+0000 7f540fe00700 1 --2- 172.21.15.131:0/3592949473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 0x563f97d52050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x563f97d52860 tx=0x7f5400014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd501cfaf800450e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.555+0000 7f5406000700 1 -- 172.21.15.131:0/3592949473 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400022040 con 0x563f97d3a6a0 2024-05-10T05:34:06.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.555+0000 7f5406000700 1 -- 172.21.15.131:0/3592949473 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54000288d0 con 0x563f97d3a6a0 2024-05-10T05:34:06.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.555+0000 7f5406000700 1 -- 172.21.15.131:0/3592949473 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540001dd20 con 0x563f97d3a6a0 2024-05-10T05:34:06.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.556+0000 7f542ece8b80 1 -- 172.21.15.131:0/3592949473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 msgr2=0x563f97d52050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.556+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3592949473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 0x563f97d52050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x563f97d52860 tx=0x7f5400014000 comp rx=0 tx=0).stop 2024-05-10T05:34:06.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.556+0000 7f542ece8b80 1 -- 172.21.15.131:0/3592949473 shutdown_connections 2024-05-10T05:34:06.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.556+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3592949473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3bec0 0x563f97d4b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.556+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3592949473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 0x563f97d52050 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.556+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3592949473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3b570 0x563f97d31180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.556+0000 7f542ece8b80 1 -- 172.21.15.131:0/3592949473 >> 172.21.15.131:0/3592949473 conn(0x563f97c9ca60 msgr2=0x563f97d43b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.556+0000 7f542ece8b80 1 -- 172.21.15.131:0/3592949473 shutdown_connections 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.556+0000 7f542ece8b80 1 -- 172.21.15.131:0/3592949473 wait complete. 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.556+0000 7f542ece8b80 1 -- start start 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.557+0000 7f542ece8b80 1 Processor -- start 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.557+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 0x563f97d48e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.557+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3b570 0x563f97d49370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.557+0000 7f542ece8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.557+0000 7f542ece8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97d4a790 con 0x563f97d3bec0 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.557+0000 7f542ece8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97d4a900 con 0x563f97d3a6a0 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.557+0000 7f542ece8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f97d4aac0 con 0x563f97d3b570 2024-05-10T05:34:06.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.557+0000 7f542ece8b80 10 client.0 _ll_register_callbacks cb 0x563f97bffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.557+0000 7f5414e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.557+0000 7f5414e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44484/0 (socket says 172.21.15.131:44484) 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f5414e00700 1 -- 172.21.15.131:0/3956867481 learned_addr learned my addr 172.21.15.131:0/3956867481 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f540f400700 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3b570 0x563f97d49370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f540fe00700 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 0x563f97d48e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f5414e00700 1 -- 172.21.15.131:0/3956867481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3b570 msgr2=0x563f97d49370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f5414e00700 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3b570 0x563f97d49370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f5414e00700 1 -- 172.21.15.131:0/3956867481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 msgr2=0x563f97d48e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f5414e00700 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 0x563f97d48e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f5414e00700 1 -- 172.21.15.131:0/3956867481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5400017070 con 0x563f97d3bec0 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f5414e00700 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4a250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5408098d30 tx=0x7f54080a2c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a2840dcc8e0d49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f53fa000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:06.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f53fa000700 1 -- 172.21.15.131:0/3956867481 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54080a2e50 con 0x563f97d3bec0 2024-05-10T05:34:06.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f53fa000700 1 -- 172.21.15.131:0/3956867481 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f540809a9e0 con 0x563f97d3bec0 2024-05-10T05:34:06.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.558+0000 7f542ece8b80 1 -- 172.21.15.131:0/3956867481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f540809e810 con 0x563f97d3bec0 2024-05-10T05:34:06.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f53fa000700 1 -- 172.21.15.131:0/3956867481 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54080cb080 con 0x563f97d3bec0 2024-05-10T05:34:06.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f542ece8b80 10 client.15447 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:06.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f542ece8b80 1 -- 172.21.15.131:0/3956867481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563f97adfcd0 con 0x563f97d3bec0 2024-05-10T05:34:06.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f53fa000700 1 -- 172.21.15.131:0/3956867481 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f540809e810 con 0x563f97d3bec0 2024-05-10T05:34:06.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f53fa000700 1 -- 172.21.15.131:0/3956867481 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f540801a390 con 0x563f97d3bec0 2024-05-10T05:34:06.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f542ece8b80 10 client.15447 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:06.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f542ece8b80 10 client.15447 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:34:06.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f542ece8b80 20 client.15447 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:06.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x563f97da6050 0x563f97da8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f542ece8b80 4 client.15447 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:34:06.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f542ece8b80 1 -- 172.21.15.131:0/3956867481 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f97c52ec0 con 0x563f97da6050 2024-05-10T05:34:06.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.559+0000 7f540fe00700 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x563f97da6050 0x563f97da8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.560+0000 7f53fa000700 10 client.15447 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:34:06.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.560+0000 7f540fe00700 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x563f97da6050 0x563f97da8470 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fdbe08581737897f server_cookie=32142c53cc8482f3 in_seq=0 out_seq=0 2024-05-10T05:34:06.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.560+0000 7f53fa000700 1 -- 172.21.15.131:0/3956867481 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f97c52ec0 con 0x563f97da6050 2024-05-10T05:34:06.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.560+0000 7f53fa000700 10 client.15447 handle_command_reply: tid=0 2024-05-10T05:34:06.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.560+0000 7f542ece8b80 1 client.15447 shutdown 2024-05-10T05:34:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.560+0000 7f542ece8b80 1 -- 172.21.15.131:0/3956867481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 msgr2=0x563f97d4a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.560+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4a250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5408098d30 tx=0x7f54080a2c50 comp rx=0 tx=0).stop 2024-05-10T05:34:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f542ece8b80 1 -- 172.21.15.131:0/3956867481 shutdown_connections 2024-05-10T05:34:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x563f97da6050 0x563f97da8470 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f97d3bec0 0x563f97d4a250 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f97d3a6a0 0x563f97d48e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f542ece8b80 1 --2- 172.21.15.131:0/3956867481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f97d3b570 0x563f97d49370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f53fa000700 0 client.15447 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:34:06.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f542ece8b80 1 -- 172.21.15.131:0/3956867481 >> 172.21.15.131:0/3956867481 conn(0x563f97c9ca60 msgr2=0x563f97d3f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f542ece8b80 1 -- 172.21.15.131:0/3956867481 shutdown_connections 2024-05-10T05:34:06.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f542ece8b80 1 -- 172.21.15.131:0/3956867481 wait complete. 2024-05-10T05:34:06.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f542ece8b80 20 client.15447 trim_cache size 0 max 16384 2024-05-10T05:34:06.568 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:34:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f5416e00700 1 -- 172.21.15.131:0/1685921306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53f40a11c0 msgr2=0x7f53f40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f5416e00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53f40a11c0 0x7f53f40a36d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5410102c00 tx=0x7f5400009310 comp rx=0 tx=0).stop 2024-05-10T05:34:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f5416e00700 1 -- 172.21.15.131:0/1685921306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54101036a0 msgr2=0x7f54101a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.561+0000 7f5416e00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54101036a0 0x7f54101a1710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5408007730 tx=0x7f5408007760 comp rx=0 tx=0).stop 2024-05-10T05:34:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.562+0000 7f5416e00700 1 -- 172.21.15.131:0/1685921306 shutdown_connections 2024-05-10T05:34:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.562+0000 7f5416e00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53f40a11c0 0x7f53f40a36d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.562+0000 7f5416e00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5410101be0 0x7f541019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.562+0000 7f5416e00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54101036a0 0x7f54101a1710 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.562+0000 7f5416e00700 1 --2- 172.21.15.131:0/1685921306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5410102d00 0x7f541019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.562+0000 7f5416e00700 1 -- 172.21.15.131:0/1685921306 >> 172.21.15.131:0/1685921306 conn(0x7f54100fd210 msgr2=0x7f5410073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.562+0000 7f5416e00700 1 -- 172.21.15.131:0/1685921306 shutdown_connections 2024-05-10T05:34:06.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.562+0000 7f5416e00700 1 -- 172.21.15.131:0/1685921306 wait complete. 2024-05-10T05:34:06.589 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:34:06.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.854+0000 7fece5200700 1 -- 172.21.15.131:0/4256162721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece01019f0 msgr2=0x7fece0101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.854+0000 7fece5200700 1 --2- 172.21.15.131:0/4256162721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece01019f0 0x7fece0101e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fecd00099c0 tx=0x7fecd0009cd0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.854+0000 7fece5200700 1 -- 172.21.15.131:0/4256162721 shutdown_connections 2024-05-10T05:34:06.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.854+0000 7fece5200700 1 --2- 172.21.15.131:0/4256162721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece01034b0 0x7fece0109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.854+0000 7fece5200700 1 --2- 172.21.15.131:0/4256162721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece01019f0 0x7fece0101e00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.854+0000 7fece5200700 1 --2- 172.21.15.131:0/4256162721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fece0102b10 0x7fece0102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.854+0000 7fece5200700 1 -- 172.21.15.131:0/4256162721 >> 172.21.15.131:0/4256162721 conn(0x7fece00fd000 msgr2=0x7fece00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.854+0000 7fece5200700 1 -- 172.21.15.131:0/4256162721 shutdown_connections 2024-05-10T05:34:06.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.854+0000 7fece5200700 1 -- 172.21.15.131:0/4256162721 wait complete. 2024-05-10T05:34:06.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.855+0000 7fece5200700 1 Processor -- start 2024-05-10T05:34:06.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.855+0000 7fece5200700 1 -- start start 2024-05-10T05:34:06.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fece5200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece01019f0 0x7fece0197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fece5200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fece0102b10 0x7fece01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fece5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece01034b0 0x7fece019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fece5200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece019f900 con 0x7fece0102b10 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fece5200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece019fa40 con 0x7fece01034b0 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fece5200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece019fbd0 con 0x7fece01019f0 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fece0102b10 0x7fece01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fece0102b10 0x7fece01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44492/0 (socket says 172.21.15.131:44492) 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecd7e00700 1 -- 172.21.15.131:0/2203337898 learned_addr learned my addr 172.21.15.131:0/2203337898 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecdf400700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece01019f0 0x7fece0197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecdfe00700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece01034b0 0x7fece019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecd7e00700 1 -- 172.21.15.131:0/2203337898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece01019f0 msgr2=0x7fece0197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecd7e00700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece01019f0 0x7fece0197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecd7e00700 1 -- 172.21.15.131:0/2203337898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece01034b0 msgr2=0x7fece019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecd7e00700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece01034b0 0x7fece019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecd7e00700 1 -- 172.21.15.131:0/2203337898 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd00095f0 con 0x7fece0102b10 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.856+0000 7fecdf400700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece01019f0 0x7fece0197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:06.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.857+0000 7fecd7e00700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fece0102b10 0x7fece01983a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fecc800e9d0 tx=0x7fecc800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.857+0000 7fecdcc00700 1 -- 172.21.15.131:0/2203337898 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8009790 con 0x7fece0102b10 2024-05-10T05:34:06.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.857+0000 7fecdcc00700 1 -- 172.21.15.131:0/2203337898 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecc800cbd0 con 0x7fece0102b10 2024-05-10T05:34:06.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.857+0000 7fece5200700 1 -- 172.21.15.131:0/2203337898 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece019feb0 con 0x7fece0102b10 2024-05-10T05:34:06.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.857+0000 7fecdcc00700 1 -- 172.21.15.131:0/2203337898 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8018680 con 0x7fece0102b10 2024-05-10T05:34:06.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.857+0000 7fece5200700 1 -- 172.21.15.131:0/2203337898 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece01a02e0 con 0x7fece0102b10 2024-05-10T05:34:06.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.862+0000 7fecdcc00700 1 -- 172.21.15.131:0/2203337898 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fecc80187e0 con 0x7fece0102b10 2024-05-10T05:34:06.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.862+0000 7fecfe7dbb80 1 Processor -- start 2024-05-10T05:34:06.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecfe7dbb80 1 -- start start 2024-05-10T05:34:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59982940 0x560f59982d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 0x560f59983fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecdcc00700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fecc40a1120 0x7fecc40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59984500 0x560f5998ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f598a5150 con 0x560f59984500 2024-05-10T05:34:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f598a2b10 con 0x560f59983bb0 2024-05-10T05:34:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5989e400 con 0x560f59982940 2024-05-10T05:34:06.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecdcc00700 1 -- 172.21.15.131:0/2203337898 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fecc8014070 con 0x7fece0102b10 2024-05-10T05:34:06.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59984500 0x560f5998ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecdfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59982940 0x560f59982d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecdfe00700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fecc40a1120 0x7fecc40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59984500 0x560f5998ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44502/0 (socket says 172.21.15.131:44502) 2024-05-10T05:34:06.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecd7e00700 1 -- 172.21.15.131:0/2644055698 learned_addr learned my addr 172.21.15.131:0/2644055698 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:06.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.863+0000 7fecdf400700 1 --2- 172.21.15.131:0/2644055698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 0x560f59983fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecd7e00700 1 -- 172.21.15.131:0/2644055698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59982940 msgr2=0x560f59982d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecd7e00700 1 --2- 172.21.15.131:0/2644055698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59982940 0x560f59982d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecd7e00700 1 -- 172.21.15.131:0/2644055698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 msgr2=0x560f59983fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecd7e00700 1 --2- 172.21.15.131:0/2644055698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 0x560f59983fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecd7e00700 1 -- 172.21.15.131:0/2644055698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f5971f3b0 con 0x560f59984500 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecdf400700 1 --2- 172.21.15.131:0/2644055698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 0x560f59983fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecd7e00700 1 --2- 172.21.15.131:0/2644055698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59984500 0x560f5998ae00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fecc80964a0 tx=0x7fecc8099b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f634b9cda655ac95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecdfe00700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fecc40a1120 0x7fecc40a3630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009550 tx=0x7fecd8009e50 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecd4c00700 1 -- 172.21.15.131:0/2644055698 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc80ac850 con 0x560f59984500 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecd4c00700 1 -- 172.21.15.131:0/2644055698 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecc80ac670 con 0x560f59984500 2024-05-10T05:34:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.864+0000 7fecd4c00700 1 -- 172.21.15.131:0/2644055698 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc809de40 con 0x560f59984500 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.865+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2644055698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59984500 msgr2=0x560f5998ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.865+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/2644055698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59984500 0x560f5998ae00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fecc80964a0 tx=0x7fecc8099b80 comp rx=0 tx=0).stop 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.865+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2644055698 shutdown_connections 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.865+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/2644055698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59984500 0x560f5998ae00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.865+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/2644055698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59982940 0x560f59982d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.865+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/2644055698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 0x560f59983fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.865+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2644055698 >> 172.21.15.131:0/2644055698 conn(0x560f598e0a60 msgr2=0x560f59979330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.865+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2644055698 shutdown_connections 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.865+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2644055698 wait complete. 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.866+0000 7fecfe7dbb80 1 -- start start 2024-05-10T05:34:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.866+0000 7fecfe7dbb80 1 Processor -- start 2024-05-10T05:34:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59982940 0x560f5998cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59983bb0 0x560f5998d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59984500 0x560f59994220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5983a560 con 0x560f59983bb0 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5983a200 con 0x560f59982940 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f597216a0 con 0x560f59984500 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecfe7dbb80 10 client.0 _ll_register_callbacks cb 0x560f59843b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59983bb0 0x560f5998d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecdf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59982940 0x560f5998cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecd7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59983bb0 0x560f5998d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44512/0 (socket says 172.21.15.131:44512) 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecd7e00700 1 -- 172.21.15.131:0/2266178375 learned_addr learned my addr 172.21.15.131:0/2266178375 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecdfe00700 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59984500 0x560f59994220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecd7e00700 1 -- 172.21.15.131:0/2266178375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59984500 msgr2=0x560f59994220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecd7e00700 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59984500 0x560f59994220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecd7e00700 1 -- 172.21.15.131:0/2266178375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59982940 msgr2=0x560f5998cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecd7e00700 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59982940 0x560f5998cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.867+0000 7fecd7e00700 1 -- 172.21.15.131:0/2266178375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece010a7f0 con 0x560f59983bb0 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7fecd7e00700 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59983bb0 0x560f5998d2c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fecc8000c00 tx=0x7fecc8096410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644992c6a361affe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7feccd600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7feccd600700 1 -- 172.21.15.131:0/2266178375 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc809a6a0 con 0x560f59983bb0 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7feccd600700 1 -- 172.21.15.131:0/2266178375 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fecc80a2420 con 0x560f59983bb0 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2266178375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f597f7270 con 0x560f59983bb0 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7feccd600700 1 -- 172.21.15.131:0/2266178375 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecc80ac000 con 0x560f59983bb0 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7fecfe7dbb80 10 client.15459 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2266178375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fece01a02e0 con 0x560f59983bb0 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7feccd600700 1 -- 172.21.15.131:0/2266178375 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f597f7270 con 0x560f59983bb0 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7feccd600700 1 -- 172.21.15.131:0/2266178375 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fecc8099490 con 0x560f59983bb0 2024-05-10T05:34:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.868+0000 7fecfe7dbb80 10 client.15459 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.869+0000 7fecfe7dbb80 10 client.15459 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.869+0000 7fecfe7dbb80 20 client.15459 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.869+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x560f59998750 0x560f5999aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.869+0000 7fecfe7dbb80 4 client.15459 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.869+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2266178375 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f5994d780 con 0x560f59998750 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.869+0000 7fecdf400700 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x560f59998750 0x560f5999aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.869+0000 7fecdf400700 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x560f59998750 0x560f5999aba0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8ec590145dff7c0 server_cookie=b941f8fd4f7e7cff in_seq=0 out_seq=0 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.869+0000 7feccd600700 10 client.15459 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.870+0000 7feccd600700 1 -- 172.21.15.131:0/2266178375 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f5994d780 con 0x560f59998750 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.870+0000 7feccd600700 10 client.15459 handle_command_reply: tid=0 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.870+0000 7fecfe7dbb80 1 client.15459 shutdown 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.870+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2266178375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59983bb0 msgr2=0x560f5998d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.870+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59983bb0 0x560f5998d2c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fecc8000c00 tx=0x7fecc8096410 comp rx=0 tx=0).stop 2024-05-10T05:34:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.870+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2266178375 shutdown_connections 2024-05-10T05:34:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.870+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x560f59998750 0x560f5999aba0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.871+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59984500 0x560f59994220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.871+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59982940 0x560f5998cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.871+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/2266178375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59983bb0 0x560f5998d2c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.871+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2266178375 >> 172.21.15.131:0/2266178375 conn(0x560f598e0a60 msgr2=0x560f59981470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.871+0000 7feccd600700 0 client.15459 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:34:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.871+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2266178375 shutdown_connections 2024-05-10T05:34:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.871+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/2266178375 wait complete. 2024-05-10T05:34:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.871+0000 7fecfe7dbb80 20 client.15459 trim_cache size 0 max 16384 2024-05-10T05:34:06.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.884+0000 7fecfe7dbb80 1 Processor -- start 2024-05-10T05:34:06.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.884+0000 7fecfe7dbb80 1 -- start start 2024-05-10T05:34:06.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59983bb0 0x560f59992710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59984500 0x560f599879c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5997abe0 con 0x560f59982940 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f59973320 con 0x560f59984500 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f59970d60 con 0x560f59983bb0 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecdf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecdf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599921d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44528/0 (socket says 172.21.15.131:44528) 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecdf400700 1 -- 172.21.15.131:0/4137075424 learned_addr learned my addr 172.21.15.131:0/4137075424 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecd7e00700 1 --2- 172.21.15.131:0/4137075424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59983bb0 0x560f59992710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecdfe00700 1 --2- 172.21.15.131:0/4137075424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59984500 0x560f599879c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecdf400700 1 -- 172.21.15.131:0/4137075424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59983bb0 msgr2=0x560f59992710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecdf400700 1 --2- 172.21.15.131:0/4137075424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59983bb0 0x560f59992710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecdf400700 1 -- 172.21.15.131:0/4137075424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59984500 msgr2=0x560f599879c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecdf400700 1 --2- 172.21.15.131:0/4137075424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59984500 0x560f599879c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.885+0000 7fecdf400700 1 -- 172.21.15.131:0/4137075424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f596cbb60 con 0x560f59982940 2024-05-10T05:34:06.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.886+0000 7fecdf400700 1 --2- 172.21.15.131:0/4137075424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599921d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fecd000b580 tx=0x7fecd0005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc78fcc4645a52c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.886+0000 7fecd5600700 1 -- 172.21.15.131:0/4137075424 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd001d070 con 0x560f59982940 2024-05-10T05:34:06.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.886+0000 7fecd5600700 1 -- 172.21.15.131:0/4137075424 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecd001e040 con 0x560f59982940 2024-05-10T05:34:06.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.886+0000 7fecd5600700 1 -- 172.21.15.131:0/4137075424 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00056b0 con 0x560f59982940 2024-05-10T05:34:06.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/4137075424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 msgr2=0x560f599921d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/4137075424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599921d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fecd000b580 tx=0x7fecd0005a90 comp rx=0 tx=0).stop 2024-05-10T05:34:06.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/4137075424 shutdown_connections 2024-05-10T05:34:06.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/4137075424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599921d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/4137075424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59984500 0x560f599879c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/4137075424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59983bb0 0x560f59992710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/4137075424 >> 172.21.15.131:0/4137075424 conn(0x560f598e0a60 msgr2=0x560f5998cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/4137075424 shutdown_connections 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/4137075424 wait complete. 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 -- start start 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.887+0000 7fecfe7dbb80 1 Processor -- start 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 0x560f599eb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecfe7dbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59984500 0x560f599ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f59992af0 con 0x560f59982940 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f599edba0 con 0x560f59983bb0 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecfe7dbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f599edd60 con 0x560f59984500 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecfe7dbb80 10 client.0 _ll_register_callbacks cb 0x560f59843b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecdf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecdf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599e8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44538/0 (socket says 172.21.15.131:44538) 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecdf400700 1 -- 172.21.15.131:0/3022446754 learned_addr learned my addr 172.21.15.131:0/3022446754 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecd7e00700 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 0x560f599eb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.888+0000 7fecdfe00700 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59984500 0x560f599ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7fecdf400700 1 -- 172.21.15.131:0/3022446754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59984500 msgr2=0x560f599ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7fecdf400700 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59984500 0x560f599ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7fecdf400700 1 -- 172.21.15.131:0/3022446754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 msgr2=0x560f599eb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7fecdf400700 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 0x560f599eb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7fecdf400700 1 -- 172.21.15.131:0/3022446754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd000bb50 con 0x560f59982940 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7fecd7e00700 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 0x560f599eb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7fecdf400700 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599e8b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fecd00058e0 tx=0x7fecd0005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7002f27de319068 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7feccd600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7feccd600700 1 -- 172.21.15.131:0/3022446754 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd001d040 con 0x560f59982940 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7feccd600700 1 -- 172.21.15.131:0/3022446754 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fecd0029e40 con 0x560f59982940 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/3022446754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecc80ac670 con 0x560f59982940 2024-05-10T05:34:06.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7feccd600700 1 -- 172.21.15.131:0/3022446754 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecd00037d0 con 0x560f59982940 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7fecfe7dbb80 10 client.15465 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.889+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/3022446754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560f59864560 con 0x560f59982940 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.890+0000 7feccd600700 1 -- 172.21.15.131:0/3022446754 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecc80ac670 con 0x560f59982940 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.890+0000 7feccd600700 1 -- 172.21.15.131:0/3022446754 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fecd0030070 con 0x560f59982940 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.890+0000 7fecfe7dbb80 10 client.15465 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.890+0000 7fecfe7dbb80 10 client.15465 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.890+0000 7fecfe7dbb80 20 client.15465 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.890+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x560f599ef230 0x560f599f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.890+0000 7fecfe7dbb80 4 client.15465 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.890+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/3022446754 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560f59896ec0 con 0x560f599ef230 2024-05-10T05:34:06.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.890+0000 7fecd7e00700 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x560f599ef230 0x560f599f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:06.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7feccd600700 10 client.15465 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:34:06.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecd7e00700 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x560f599ef230 0x560f599f1650 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7925db77646c708 server_cookie=f846e57d0bf11df0 in_seq=0 out_seq=0 2024-05-10T05:34:06.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7feccd600700 1 -- 172.21.15.131:0/3022446754 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560f59896ec0 con 0x560f599ef230 2024-05-10T05:34:06.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7feccd600700 10 client.15465 handle_command_reply: tid=0 2024-05-10T05:34:06.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 client.15465 shutdown 2024-05-10T05:34:06.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/3022446754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 msgr2=0x560f599e8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599e8b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fecd00058e0 tx=0x7fecd0005060 comp rx=0 tx=0).stop 2024-05-10T05:34:06.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/3022446754 shutdown_connections 2024-05-10T05:34:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x560f599ef230 0x560f599f1650 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f59982940 0x560f599e8b90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560f59984500 0x560f599ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7feccd600700 0 client.15465 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:34:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 --2- 172.21.15.131:0/3022446754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f59983bb0 0x560f599eb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/3022446754 >> 172.21.15.131:0/3022446754 conn(0x560f598e0a60 msgr2=0x560f5998cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/3022446754 shutdown_connections 2024-05-10T05:34:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 1 -- 172.21.15.131:0/3022446754 wait complete. 2024-05-10T05:34:06.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.891+0000 7fecfe7dbb80 20 client.15465 trim_cache size 0 max 16384 2024-05-10T05:34:06.897 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:34:06.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 -- 172.21.15.131:0/2203337898 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fecc40a1120 msgr2=0x7fecc40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fecc40a1120 0x7fecc40a3630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009550 tx=0x7fecd8009e50 comp rx=0 tx=0).stop 2024-05-10T05:34:06.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 -- 172.21.15.131:0/2203337898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fece0102b10 msgr2=0x7fece01983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:06.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fece0102b10 0x7fece01983a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fecc800e9d0 tx=0x7fecc800ece0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 -- 172.21.15.131:0/2203337898 shutdown_connections 2024-05-10T05:34:06.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fecc40a1120 0x7fecc40a3630 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece01034b0 0x7fece019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fece01019f0 0x7fece0197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 --2- 172.21.15.131:0/2203337898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fece0102b10 0x7fece01983a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 -- 172.21.15.131:0/2203337898 >> 172.21.15.131:0/2203337898 conn(0x7fece00fd000 msgr2=0x7fece0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 -- 172.21.15.131:0/2203337898 shutdown_connections 2024-05-10T05:34:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:06.892+0000 7fece5200700 1 -- 172.21.15.131:0/2203337898 wait complete. 2024-05-10T05:34:06.918 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:34:07.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.187+0000 7f117ac00700 1 -- 172.21.15.131:0/996469586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1174102b10 msgr2=0x7f1174102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.187+0000 7f117ac00700 1 --2- 172.21.15.131:0/996469586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1174102b10 0x7f1174102f20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1160009990 tx=0x7f1160009ca0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.187+0000 7f117ac00700 1 -- 172.21.15.131:0/996469586 shutdown_connections 2024-05-10T05:34:07.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.187+0000 7f117ac00700 1 --2- 172.21.15.131:0/996469586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11741034b0 0x7f1174109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.187+0000 7f117ac00700 1 --2- 172.21.15.131:0/996469586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11741019f0 0x7f1174101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.187+0000 7f117ac00700 1 --2- 172.21.15.131:0/996469586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1174102b10 0x7f1174102f20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.187+0000 7f117ac00700 1 -- 172.21.15.131:0/996469586 >> 172.21.15.131:0/996469586 conn(0x7f11740fd000 msgr2=0x7f11740ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.187+0000 7f117ac00700 1 -- 172.21.15.131:0/996469586 shutdown_connections 2024-05-10T05:34:07.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.187+0000 7f117ac00700 1 -- 172.21.15.131:0/996469586 wait complete. 2024-05-10T05:34:07.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.188+0000 7f117ac00700 1 Processor -- start 2024-05-10T05:34:07.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.188+0000 7f117ac00700 1 -- start start 2024-05-10T05:34:07.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f117ac00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11741019f0 0x7f117419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f117ac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174102b10 0x7f117419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f117ac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11741034b0 0x7f11741a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f117ac00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741a1b40 con 0x7f11741019f0 2024-05-10T05:34:07.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f117ac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741a1c80 con 0x7f11741034b0 2024-05-10T05:34:07.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f117ac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741a1e10 con 0x7f1174102b10 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f1179800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11741019f0 0x7f117419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f1178e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174102b10 0x7f117419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f1178e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174102b10 0x7f117419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41124/0 (socket says 172.21.15.131:41124) 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f1179800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11741019f0 0x7f117419a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44558/0 (socket says 172.21.15.131:44558) 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f1178e00700 1 -- 172.21.15.131:0/648249190 learned_addr learned my addr 172.21.15.131:0/648249190 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.189+0000 7f117a200700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11741034b0 0x7f11741a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f1179800700 1 -- 172.21.15.131:0/648249190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174102b10 msgr2=0x7f117419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f1179800700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174102b10 0x7f117419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f1179800700 1 -- 172.21.15.131:0/648249190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11741034b0 msgr2=0x7f11741a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f1179800700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11741034b0 0x7f11741a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f1179800700 1 -- 172.21.15.131:0/648249190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11600095f0 con 0x7f11741019f0 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f117a200700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11741034b0 0x7f11741a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f1179800700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11741019f0 0x7f117419a0a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9c0 tx=0x7f116800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f1172000700 1 -- 172.21.15.131:0/648249190 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800cbd0 con 0x7f11741019f0 2024-05-10T05:34:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f1172000700 1 -- 172.21.15.131:0/648249190 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f116800eeb0 con 0x7f11741019f0 2024-05-10T05:34:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f117ac00700 1 -- 172.21.15.131:0/648249190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11741a20f0 con 0x7f11741019f0 2024-05-10T05:34:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f1172000700 1 -- 172.21.15.131:0/648249190 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168003c70 con 0x7f11741019f0 2024-05-10T05:34:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.190+0000 7f117ac00700 1 -- 172.21.15.131:0/648249190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11741a2650 con 0x7f11741019f0 2024-05-10T05:34:07.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.194+0000 7f1172000700 1 -- 172.21.15.131:0/648249190 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1168003dd0 con 0x7f11741019f0 2024-05-10T05:34:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.195+0000 7f1194195b80 1 Processor -- start 2024-05-10T05:34:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.195+0000 7f1194195b80 1 -- start start 2024-05-10T05:34:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.195+0000 7f1172000700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f115c0a11a0 0x7f115c0a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.195+0000 7f1172000700 1 -- 172.21.15.131:0/648249190 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1168014070 con 0x7f11741019f0 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.195+0000 7f1194195b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd75960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.195+0000 7f1194195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd767a0 0x55774fd76bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.195+0000 7f1194195b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd770f0 0x55774fd7d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.195+0000 7f1194195b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fc9c150 con 0x55774fd767a0 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.195+0000 7f1194195b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fc99b10 con 0x55774fd75530 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.195+0000 7f1194195b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fc95400 con 0x55774fd770f0 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f117a200700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f115c0a11a0 0x7f115c0a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f1179800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd75960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f1178e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd767a0 0x55774fd76bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f1179800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd75960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37854/0 (socket says 172.21.15.131:37854) 2024-05-10T05:34:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f1179800700 1 -- 172.21.15.131:0/3843917018 learned_addr learned my addr 172.21.15.131:0/3843917018 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f117a200700 1 --2- 172.21.15.131:0/3843917018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd770f0 0x55774fd7d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f1179800700 1 -- 172.21.15.131:0/3843917018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd770f0 msgr2=0x55774fd7d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f1179800700 1 --2- 172.21.15.131:0/3843917018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd770f0 0x55774fd7d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f1179800700 1 -- 172.21.15.131:0/3843917018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd767a0 msgr2=0x55774fd76bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f1179800700 1 --2- 172.21.15.131:0/3843917018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd767a0 0x55774fd76bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f1179800700 1 -- 172.21.15.131:0/3843917018 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774fb163b0 con 0x55774fd75530 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f117a200700 1 --2- 172.21.15.131:0/3843917018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd770f0 0x55774fd7d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f117a200700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f115c0a11a0 0x7f115c0a36b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f116c0093c0 tx=0x7f116c009920 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.196+0000 7f1179800700 1 --2- 172.21.15.131:0/3843917018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd75960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11680ca550 tx=0x7f1168003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d41b13612ae8a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.197+0000 7f1166a00700 1 -- 172.21.15.131:0/3843917018 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168099f30 con 0x55774fd75530 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.197+0000 7f1166a00700 1 -- 172.21.15.131:0/3843917018 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f116809dd60 con 0x55774fd75530 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.197+0000 7f1166a00700 1 -- 172.21.15.131:0/3843917018 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116809a3f0 con 0x55774fd75530 2024-05-10T05:34:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.198+0000 7f1194195b80 1 -- 172.21.15.131:0/3843917018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 msgr2=0x55774fd75960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.198+0000 7f1194195b80 1 --2- 172.21.15.131:0/3843917018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd75960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11680ca550 tx=0x7f1168003680 comp rx=0 tx=0).stop 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.198+0000 7f1194195b80 1 -- 172.21.15.131:0/3843917018 shutdown_connections 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.198+0000 7f1194195b80 1 --2- 172.21.15.131:0/3843917018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd770f0 0x55774fd7d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.198+0000 7f1194195b80 1 --2- 172.21.15.131:0/3843917018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd75960 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.198+0000 7f1194195b80 1 --2- 172.21.15.131:0/3843917018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd767a0 0x55774fd76bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.198+0000 7f1194195b80 1 -- 172.21.15.131:0/3843917018 >> 172.21.15.131:0/3843917018 conn(0x55774fcd7a60 msgr2=0x55774fd70330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.198+0000 7f1194195b80 1 -- 172.21.15.131:0/3843917018 shutdown_connections 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.198+0000 7f1194195b80 1 -- 172.21.15.131:0/3843917018 wait complete. 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.198+0000 7f1194195b80 1 -- start start 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1194195b80 1 Processor -- start 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1194195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd75530 0x55774fd7f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1194195b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd767a0 0x55774fd7fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1194195b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd770f0 0x55774fd86dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1194195b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fc31560 con 0x55774fd75530 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1194195b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fc31200 con 0x55774fd770f0 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1194195b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fb186a0 con 0x55774fd767a0 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1194195b80 10 client.0 _ll_register_callbacks cb 0x55774fc3ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1179800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd75530 0x55774fd7f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1179800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd75530 0x55774fd7f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44576/0 (socket says 172.21.15.131:44576) 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1179800700 1 -- 172.21.15.131:0/1701049483 learned_addr learned my addr 172.21.15.131:0/1701049483 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.199+0000 7f1178e00700 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd767a0 0x55774fd7fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f117a200700 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd770f0 0x55774fd86dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f1179800700 1 -- 172.21.15.131:0/1701049483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd767a0 msgr2=0x55774fd7fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f1179800700 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd767a0 0x55774fd7fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f1179800700 1 -- 172.21.15.131:0/1701049483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd770f0 msgr2=0x55774fd86dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f1179800700 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd770f0 0x55774fd86dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f1179800700 1 -- 172.21.15.131:0/1701049483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11741a2650 con 0x55774fd75530 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f1179800700 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd75530 0x55774fd7f930 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1168009230 tx=0x7f1168004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65abcaa7cc5dfb73 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f115b400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f115b400700 1 -- 172.21.15.131:0/1701049483 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11680b8180 con 0x55774fd75530 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f115b400700 1 -- 172.21.15.131:0/1701049483 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f11680b82e0 con 0x55774fd75530 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f1194195b80 1 -- 172.21.15.131:0/1701049483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55774fbee270 con 0x55774fd75530 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f115b400700 1 -- 172.21.15.131:0/1701049483 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f116809a190 con 0x55774fd75530 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f1194195b80 10 client.15477 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.200+0000 7f1194195b80 1 -- 172.21.15.131:0/1701049483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55774fac2b60 con 0x55774fd75530 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.201+0000 7f115b400700 1 -- 172.21.15.131:0/1701049483 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55774fbee270 con 0x55774fd75530 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.201+0000 7f115b400700 1 -- 172.21.15.131:0/1701049483 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f116801fbd0 con 0x55774fd75530 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.201+0000 7f1194195b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.201+0000 7f1194195b80 10 client.15477 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.201+0000 7f1194195b80 20 client.15477 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.201+0000 7f1194195b80 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55774fd8b180 0x55774fd8d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.201+0000 7f1194195b80 4 client.15477 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.201+0000 7f1194195b80 1 -- 172.21.15.131:0/1701049483 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55774fd44780 con 0x55774fd8b180 2024-05-10T05:34:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.201+0000 7f1178e00700 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55774fd8b180 0x55774fd8d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.202+0000 7f1178e00700 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55774fd8b180 0x55774fd8d5a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e42c925143c57a99 server_cookie=661ba805aafb132 in_seq=0 out_seq=0 2024-05-10T05:34:07.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.202+0000 7f115b400700 10 client.15477 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:34:07.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.202+0000 7f115b400700 1 -- 172.21.15.131:0/1701049483 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55774fd44780 con 0x55774fd8b180 2024-05-10T05:34:07.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.202+0000 7f115b400700 10 client.15477 handle_command_reply: tid=0 2024-05-10T05:34:07.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.202+0000 7f1194195b80 1 client.15477 shutdown 2024-05-10T05:34:07.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 1 -- 172.21.15.131:0/1701049483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd75530 msgr2=0x55774fd7f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd75530 0x55774fd7f930 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1168009230 tx=0x7f1168004750 comp rx=0 tx=0).stop 2024-05-10T05:34:07.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 1 -- 172.21.15.131:0/1701049483 shutdown_connections 2024-05-10T05:34:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55774fd8b180 0x55774fd8d5a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd75530 0x55774fd7f930 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd770f0 0x55774fd86dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f115b400700 0 client.15477 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:34:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 1 --2- 172.21.15.131:0/1701049483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd767a0 0x55774fd7fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 1 -- 172.21.15.131:0/1701049483 >> 172.21.15.131:0/1701049483 conn(0x55774fcd7a60 msgr2=0x55774fd74060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 1 -- 172.21.15.131:0/1701049483 shutdown_connections 2024-05-10T05:34:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 1 -- 172.21.15.131:0/1701049483 wait complete. 2024-05-10T05:34:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.203+0000 7f1194195b80 20 client.15477 trim_cache size 0 max 16384 2024-05-10T05:34:07.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:06 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/741576666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:34:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.217+0000 7f1194195b80 1 Processor -- start 2024-05-10T05:34:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.217+0000 7f1194195b80 1 -- start start 2024-05-10T05:34:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.217+0000 7f1194195b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd76bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.217+0000 7f1194195b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd767a0 0x55774fd86d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.217+0000 7f1194195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 0x55774fd702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.217+0000 7f1194195b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fd6a900 con 0x55774fd770f0 2024-05-10T05:34:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.217+0000 7f1194195b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fd76040 con 0x55774fd75530 2024-05-10T05:34:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.217+0000 7f1194195b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fd87ed0 con 0x55774fd767a0 2024-05-10T05:34:07.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f117a200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 0x55774fd702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f1179800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd76bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f117a200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 0x55774fd702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44578/0 (socket says 172.21.15.131:44578) 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f1179800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd76bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37872/0 (socket says 172.21.15.131:37872) 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f117a200700 1 -- 172.21.15.131:0/2872967765 learned_addr learned my addr 172.21.15.131:0/2872967765 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f1178e00700 1 --2- 172.21.15.131:0/2872967765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd767a0 0x55774fd86d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f117a200700 1 -- 172.21.15.131:0/2872967765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd767a0 msgr2=0x55774fd86d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f117a200700 1 --2- 172.21.15.131:0/2872967765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd767a0 0x55774fd86d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f117a200700 1 -- 172.21.15.131:0/2872967765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 msgr2=0x55774fd76bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f117a200700 1 --2- 172.21.15.131:0/2872967765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd76bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f117a200700 1 -- 172.21.15.131:0/2872967765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774fc5b560 con 0x55774fd770f0 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f1179800700 1 --2- 172.21.15.131:0/2872967765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd76bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:07.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.218+0000 7f117a200700 1 --2- 172.21.15.131:0/2872967765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 0x55774fd702c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f116c017620 tx=0x7f116c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9353cf80799f6fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.219+0000 7f1167400700 1 -- 172.21.15.131:0/2872967765 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00e050 con 0x55774fd770f0 2024-05-10T05:34:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.219+0000 7f1167400700 1 -- 172.21.15.131:0/2872967765 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f116c00f040 con 0x55774fd770f0 2024-05-10T05:34:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.219+0000 7f1167400700 1 -- 172.21.15.131:0/2872967765 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c018780 con 0x55774fd770f0 2024-05-10T05:34:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.219+0000 7f1194195b80 1 -- 172.21.15.131:0/2872967765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 msgr2=0x55774fd702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.219+0000 7f1194195b80 1 --2- 172.21.15.131:0/2872967765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 0x55774fd702c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f116c017620 tx=0x7f116c018000 comp rx=0 tx=0).stop 2024-05-10T05:34:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.219+0000 7f1194195b80 1 -- 172.21.15.131:0/2872967765 shutdown_connections 2024-05-10T05:34:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.219+0000 7f1194195b80 1 --2- 172.21.15.131:0/2872967765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 0x55774fd702c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.219+0000 7f1194195b80 1 --2- 172.21.15.131:0/2872967765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd75530 0x55774fd76bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.219+0000 7f1194195b80 1 --2- 172.21.15.131:0/2872967765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd767a0 0x55774fd86d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.220+0000 7f1194195b80 1 -- 172.21.15.131:0/2872967765 >> 172.21.15.131:0/2872967765 conn(0x55774fcd7a60 msgr2=0x55774fd6fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.220+0000 7f1194195b80 1 -- 172.21.15.131:0/2872967765 shutdown_connections 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.220+0000 7f1194195b80 1 -- 172.21.15.131:0/2872967765 wait complete. 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.220+0000 7f1194195b80 1 -- start start 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.220+0000 7f1194195b80 1 Processor -- start 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1194195b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd75530 0x55774fd84560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1194195b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd767a0 0x55774fd84aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1194195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 0x55774fd859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1194195b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fd85f10 con 0x55774fd770f0 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1194195b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fd86080 con 0x55774fd767a0 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1194195b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fd86240 con 0x55774fd75530 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1194195b80 10 client.0 _ll_register_callbacks cb 0x55774fc3ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1179800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd75530 0x55774fd84560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1179800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd75530 0x55774fd84560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41172/0 (socket says 172.21.15.131:41172) 2024-05-10T05:34:07.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1179800700 1 -- 172.21.15.131:0/3043222626 learned_addr learned my addr 172.21.15.131:0/3043222626 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f117a200700 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 0x55774fd859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1178e00700 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd767a0 0x55774fd84aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1179800700 1 -- 172.21.15.131:0/3043222626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd767a0 msgr2=0x55774fd84aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1179800700 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd767a0 0x55774fd84aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1179800700 1 -- 172.21.15.131:0/3043222626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 msgr2=0x55774fd859d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1179800700 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 0x55774fd859d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.221+0000 7f1179800700 1 -- 172.21.15.131:0/3043222626 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55774fac2b60 con 0x55774fd75530 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.222+0000 7f1179800700 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd75530 0x55774fd84560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1168099ec0 tx=0x7f11680982d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9128f9da6db5691a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.222+0000 7f115b400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.222+0000 7f115b400700 1 -- 172.21.15.131:0/3043222626 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168098150 con 0x55774fd75530 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.222+0000 7f115b400700 1 -- 172.21.15.131:0/3043222626 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f116807f070 con 0x55774fd75530 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.222+0000 7f1194195b80 1 -- 172.21.15.131:0/3043222626 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f116809dd60 con 0x55774fd75530 2024-05-10T05:34:07.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.222+0000 7f115b400700 1 -- 172.21.15.131:0/3043222626 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11680a1f20 con 0x55774fd75530 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.222+0000 7f1194195b80 10 client.25504 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.222+0000 7f1194195b80 1 -- 172.21.15.131:0/3043222626 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f116c020040 con 0x55774fd75530 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.222+0000 7f115b400700 1 -- 172.21.15.131:0/3043222626 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f116809dd60 con 0x55774fd75530 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.223+0000 7f115b400700 1 -- 172.21.15.131:0/3043222626 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f116809a190 con 0x55774fd75530 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.223+0000 7f1194195b80 10 client.25504 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.223+0000 7f1194195b80 10 client.25504 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.223+0000 7f1194195b80 20 client.25504 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.223+0000 7f1194195b80 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55774fde1a00 0x55774fde3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.223+0000 7f1194195b80 4 client.25504 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.223+0000 7f1194195b80 1 -- 172.21.15.131:0/3043222626 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55774fc8dec0 con 0x55774fde1a00 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.223+0000 7f1178e00700 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55774fde1a00 0x55774fde3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.223+0000 7f1178e00700 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55774fde1a00 0x55774fde3e20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38a32152d935d863 server_cookie=58d36120d7cf5840 in_seq=0 out_seq=0 2024-05-10T05:34:07.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.223+0000 7f115b400700 10 client.25504 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:34:07.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f115b400700 1 -- 172.21.15.131:0/3043222626 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55774fc8dec0 con 0x55774fde1a00 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f115b400700 10 client.25504 handle_command_reply: tid=0 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 client.25504 shutdown 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 -- 172.21.15.131:0/3043222626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd75530 msgr2=0x55774fd84560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd75530 0x55774fd84560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1168099ec0 tx=0x7f11680982d0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 -- 172.21.15.131:0/3043222626 shutdown_connections 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55774fde1a00 0x55774fde3e20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55774fd770f0 0x55774fd859d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f115b400700 0 client.25504 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55774fd75530 0x55774fd84560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 --2- 172.21.15.131:0/3043222626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55774fd767a0 0x55774fd84aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 -- 172.21.15.131:0/3043222626 >> 172.21.15.131:0/3043222626 conn(0x55774fcd7a60 msgr2=0x55774fd692e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 -- 172.21.15.131:0/3043222626 shutdown_connections 2024-05-10T05:34:07.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 1 -- 172.21.15.131:0/3043222626 wait complete. 2024-05-10T05:34:07.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.224+0000 7f1194195b80 20 client.25504 trim_cache size 0 max 16384 2024-05-10T05:34:07.230 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 -- 172.21.15.131:0/648249190 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f115c0a11a0 msgr2=0x7f115c0a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f115c0a11a0 0x7f115c0a36b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f116c0093c0 tx=0x7f116c009920 comp rx=0 tx=0).stop 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 -- 172.21.15.131:0/648249190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11741019f0 msgr2=0x7f117419a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11741019f0 0x7f117419a0a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9c0 tx=0x7f116800ecd0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 -- 172.21.15.131:0/648249190 shutdown_connections 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f115c0a11a0 0x7f115c0a36b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11741019f0 0x7f117419a0a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11741034b0 0x7f11741a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 --2- 172.21.15.131:0/648249190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174102b10 0x7f117419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 -- 172.21.15.131:0/648249190 >> 172.21.15.131:0/648249190 conn(0x7f11740fd000 msgr2=0x7f1174073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 -- 172.21.15.131:0/648249190 shutdown_connections 2024-05-10T05:34:07.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.225+0000 7f117ac00700 1 -- 172.21.15.131:0/648249190 wait complete. 2024-05-10T05:34:07.253 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:34:07.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:06 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/741576666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:34:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:06 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/741576666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:34:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.518+0000 7f8f18a00700 1 -- 172.21.15.131:0/1099008533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14101be0 msgr2=0x7f8f14101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.518+0000 7f8f18a00700 1 --2- 172.21.15.131:0/1099008533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14101be0 0x7f8f14101ff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8f040099c0 tx=0x7f8f04009cd0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.518+0000 7f8f18a00700 1 -- 172.21.15.131:0/1099008533 shutdown_connections 2024-05-10T05:34:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.518+0000 7f8f18a00700 1 --2- 172.21.15.131:0/1099008533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f141036a0 0x7f8f14109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.518+0000 7f8f18a00700 1 --2- 172.21.15.131:0/1099008533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14101be0 0x7f8f14101ff0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.518+0000 7f8f18a00700 1 --2- 172.21.15.131:0/1099008533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f14102d00 0x7f8f14103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.518+0000 7f8f18a00700 1 -- 172.21.15.131:0/1099008533 >> 172.21.15.131:0/1099008533 conn(0x7f8f140fd210 msgr2=0x7f8f140ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.518+0000 7f8f18a00700 1 -- 172.21.15.131:0/1099008533 shutdown_connections 2024-05-10T05:34:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.518+0000 7f8f18a00700 1 -- 172.21.15.131:0/1099008533 wait complete. 2024-05-10T05:34:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.520+0000 7f8f18a00700 1 Processor -- start 2024-05-10T05:34:07.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.520+0000 7f8f18a00700 1 -- start start 2024-05-10T05:34:07.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f18a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f14101be0 0x7f8f1419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f18a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14102d00 0x7f8f1419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f18a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f141036a0 0x7f8f141a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f18a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f141a1be0 con 0x7f8f14101be0 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f18a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f141a1d50 con 0x7f8f141036a0 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f18a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f141a1f10 con 0x7f8f14102d00 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f11e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f14101be0 0x7f8f1419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f11e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f14101be0 0x7f8f1419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44598/0 (socket says 172.21.15.131:44598) 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f11e00700 1 -- 172.21.15.131:0/346486103 learned_addr learned my addr 172.21.15.131:0/346486103 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f12800700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f141036a0 0x7f8f141a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f11400700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14102d00 0x7f8f1419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f11e00700 1 -- 172.21.15.131:0/346486103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14102d00 msgr2=0x7f8f1419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f11e00700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14102d00 0x7f8f1419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f11e00700 1 -- 172.21.15.131:0/346486103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f141036a0 msgr2=0x7f8f141a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f11e00700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f141036a0 0x7f8f141a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f11e00700 1 -- 172.21.15.131:0/346486103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c009c40 con 0x7f8f14101be0 2024-05-10T05:34:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.521+0000 7f8f12800700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f141036a0 0x7f8f141a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.522+0000 7f8f11e00700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f14101be0 0x7f8f1419a100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009fd0 tx=0x7f8f040036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.522+0000 7f8f0aa00700 1 -- 172.21.15.131:0/346486103 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0401d070 con 0x7f8f14101be0 2024-05-10T05:34:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.522+0000 7f8f18a00700 1 -- 172.21.15.131:0/346486103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f040095f0 con 0x7f8f14101be0 2024-05-10T05:34:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.522+0000 7f8f0aa00700 1 -- 172.21.15.131:0/346486103 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f04005030 con 0x7f8f14101be0 2024-05-10T05:34:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.522+0000 7f8f18a00700 1 -- 172.21.15.131:0/346486103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f141a2480 con 0x7f8f14101be0 2024-05-10T05:34:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.522+0000 7f8f0aa00700 1 -- 172.21.15.131:0/346486103 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04004910 con 0x7f8f14101be0 2024-05-10T05:34:07.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.526+0000 7f8f0aa00700 1 -- 172.21.15.131:0/346486103 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8f040054a0 con 0x7f8f14101be0 2024-05-10T05:34:07.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.527+0000 7f8f309a6b80 1 Processor -- start 2024-05-10T05:34:07.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f309a6b80 1 -- start start 2024-05-10T05:34:07.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f0aa00700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8ef80a11c0 0x7f8ef80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f0aa00700 1 -- 172.21.15.131:0/346486103 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8f040cd520 con 0x7f8f14101be0 2024-05-10T05:34:07.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e066a0 0x5574e0e06af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07570 0x5574e0e07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f12800700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8ef80a11c0 0x7f8ef80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07ec0 0x5574e0e0e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0d2d150 con 0x5574e0e07ec0 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0d2ab10 con 0x5574e0e066a0 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0d26400 con 0x5574e0e07570 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.528+0000 7f8f11e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e066a0 0x5574e0e06af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f11e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e066a0 0x5574e0e06af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37916/0 (socket says 172.21.15.131:37916) 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f11e00700 1 -- 172.21.15.131:0/3470419135 learned_addr learned my addr 172.21.15.131:0/3470419135 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f12800700 1 --2- 172.21.15.131:0/3470419135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07ec0 0x5574e0e0e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f11400700 1 --2- 172.21.15.131:0/3470419135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07570 0x5574e0e07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f11e00700 1 -- 172.21.15.131:0/3470419135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07570 msgr2=0x5574e0e07980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f11e00700 1 --2- 172.21.15.131:0/3470419135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07570 0x5574e0e07980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f11e00700 1 -- 172.21.15.131:0/3470419135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07ec0 msgr2=0x5574e0e0e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f11e00700 1 --2- 172.21.15.131:0/3470419135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07ec0 0x5574e0e0e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f11e00700 1 -- 172.21.15.131:0/3470419135 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574e0ba73b0 con 0x5574e0e066a0 2024-05-10T05:34:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f12800700 1 --2- 172.21.15.131:0/3470419135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07ec0 0x5574e0e0e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f12800700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8ef80a11c0 0x7f8ef80a36d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009550 tx=0x7f8f0c007c00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8f11e00700 1 --2- 172.21.15.131:0/3470419135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e066a0 0x5574e0e06af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8f040a2860 tx=0x7f8f0409a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8023fcc174f4307 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.529+0000 7f8eff000700 1 -- 172.21.15.131:0/3470419135 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0409b0a0 con 0x5574e0e066a0 2024-05-10T05:34:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.530+0000 7f8eff000700 1 -- 172.21.15.131:0/3470419135 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f0409a9a0 con 0x5574e0e066a0 2024-05-10T05:34:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.530+0000 7f8eff000700 1 -- 172.21.15.131:0/3470419135 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f040cd0d0 con 0x5574e0e066a0 2024-05-10T05:34:07.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.530+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3470419135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e066a0 msgr2=0x5574e0e06af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.530+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/3470419135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e066a0 0x5574e0e06af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8f040a2860 tx=0x7f8f0409a300 comp rx=0 tx=0).stop 2024-05-10T05:34:07.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.530+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3470419135 shutdown_connections 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.530+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/3470419135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07ec0 0x5574e0e0e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.530+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/3470419135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e066a0 0x5574e0e06af0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.530+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/3470419135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07570 0x5574e0e07980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.530+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3470419135 >> 172.21.15.131:0/3470419135 conn(0x5574e0d68a60 msgr2=0x5574e0e00fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.531+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3470419135 shutdown_connections 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.531+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3470419135 wait complete. 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.531+0000 7f8f309a6b80 1 -- start start 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.532+0000 7f8f309a6b80 1 Processor -- start 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.532+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 0x5574e0e103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.532+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07570 0x5574e0e10900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.532+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07ec0 0x5574e0e17860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.532+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0cc2560 con 0x5574e0e066a0 2024-05-10T05:34:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.532+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0cc2200 con 0x5574e0e07570 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.532+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0ba96a0 con 0x5574e0e07ec0 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.532+0000 7f8f309a6b80 10 client.0 _ll_register_callbacks cb 0x5574e0ccbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 0x5574e0e103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07570 0x5574e0e10900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 0x5574e0e103c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44608/0 (socket says 172.21.15.131:44608) 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11e00700 1 -- 172.21.15.131:0/2881547185 learned_addr learned my addr 172.21.15.131:0/2881547185 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f12800700 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07ec0 0x5574e0e17860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11e00700 1 -- 172.21.15.131:0/2881547185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07ec0 msgr2=0x5574e0e17860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11e00700 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07ec0 0x5574e0e17860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11e00700 1 -- 172.21.15.131:0/2881547185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07570 msgr2=0x5574e0e10900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11e00700 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07570 0x5574e0e10900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11e00700 1 -- 172.21.15.131:0/2881547185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f141a2480 con 0x5574e0e066a0 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11400700 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07570 0x5574e0e10900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:07.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8f11e00700 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 0x5574e0e103c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009990 tx=0x7f8f04009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c512b21c088f2abb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8ef3400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8ef3400700 1 -- 172.21.15.131:0/2881547185 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0409e7d0 con 0x5574e0e066a0 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.533+0000 7f8ef3400700 1 -- 172.21.15.131:0/2881547185 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8f040a2d90 con 0x5574e0e066a0 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2881547185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574e0c7f270 con 0x5574e0e066a0 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8ef3400700 1 -- 172.21.15.131:0/2881547185 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f04099e90 con 0x5574e0e066a0 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8f309a6b80 10 client.15495 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2881547185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5574e0b53b60 con 0x5574e0e066a0 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8ef3400700 1 -- 172.21.15.131:0/2881547185 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5574e0c7f270 con 0x5574e0e066a0 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8ef3400700 1 -- 172.21.15.131:0/2881547185 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f04096790 con 0x5574e0e066a0 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8f309a6b80 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8f309a6b80 10 client.15495 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8f309a6b80 20 client.15495 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:07.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5574e0e1bad0 0x5574e0e1df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8f309a6b80 4 client.15495 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.534+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2881547185 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574e0dd5780 con 0x5574e0e1bad0 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.535+0000 7f8f11400700 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5574e0e1bad0 0x5574e0e1df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.535+0000 7f8f11400700 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5574e0e1bad0 0x5574e0e1df20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc2d9968bab2e829 server_cookie=3dc17a645b8f9132 in_seq=0 out_seq=0 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.535+0000 7f8ef3400700 10 client.15495 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8ef3400700 1 -- 172.21.15.131:0/2881547185 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574e0dd5780 con 0x5574e0e1bad0 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8ef3400700 10 client.15495 handle_command_reply: tid=0 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 client.15495 shutdown 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2881547185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 msgr2=0x5574e0e103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 0x5574e0e103c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009990 tx=0x7f8f04009310 comp rx=0 tx=0).stop 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2881547185 shutdown_connections 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5574e0e1bad0 0x5574e0e1df20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 0x5574e0e103c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07ec0 0x5574e0e17860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8ef3400700 0 client.15495 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:34:07.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/2881547185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07570 0x5574e0e10900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2881547185 >> 172.21.15.131:0/2881547185 conn(0x5574e0d68a60 msgr2=0x5574e0e00fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2881547185 shutdown_connections 2024-05-10T05:34:07.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2881547185 wait complete. 2024-05-10T05:34:07.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.536+0000 7f8f309a6b80 20 client.15495 trim_cache size 0 max 16384 2024-05-10T05:34:07.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.550+0000 7f8f309a6b80 1 Processor -- start 2024-05-10T05:34:07.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.550+0000 7f8f309a6b80 1 -- start start 2024-05-10T05:34:07.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.550+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 0x5574e0e0e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.550+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07570 0x5574e0e17820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.550+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 0x5574e0e0ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.550+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0e07410 con 0x5574e0e066a0 2024-05-10T05:34:07.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.550+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0dfb7c0 con 0x5574e0e07ec0 2024-05-10T05:34:07.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.550+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0c7f270 con 0x5574e0e07570 2024-05-10T05:34:07.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f11e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 0x5574e0e0e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f12800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 0x5574e0e0ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f12800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 0x5574e0e0ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37940/0 (socket says 172.21.15.131:37940) 2024-05-10T05:34:07.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f12800700 1 -- 172.21.15.131:0/2448110222 learned_addr learned my addr 172.21.15.131:0/2448110222 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f11400700 1 --2- 172.21.15.131:0/2448110222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07570 0x5574e0e17820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f12800700 1 -- 172.21.15.131:0/2448110222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07570 msgr2=0x5574e0e17820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f12800700 1 --2- 172.21.15.131:0/2448110222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07570 0x5574e0e17820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f12800700 1 -- 172.21.15.131:0/2448110222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 msgr2=0x5574e0e0e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f12800700 1 --2- 172.21.15.131:0/2448110222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 0x5574e0e0e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f12800700 1 -- 172.21.15.131:0/2448110222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574e0cec560 con 0x5574e0e07ec0 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.551+0000 7f8f12800700 1 --2- 172.21.15.131:0/2448110222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 0x5574e0e0ef20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00eb00 tx=0x7f8f0c01e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936b80559fc13bf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8effa00700 1 -- 172.21.15.131:0/2448110222 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c01ace0 con 0x5574e0e07ec0 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8effa00700 1 -- 172.21.15.131:0/2448110222 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f0c00f040 con 0x5574e0e07ec0 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8effa00700 1 -- 172.21.15.131:0/2448110222 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0186b0 con 0x5574e0e07ec0 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2448110222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 msgr2=0x5574e0e0ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/2448110222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 0x5574e0e0ef20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00eb00 tx=0x7f8f0c01e000 comp rx=0 tx=0).stop 2024-05-10T05:34:07.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2448110222 shutdown_connections 2024-05-10T05:34:07.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/2448110222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e066a0 0x5574e0e0e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/2448110222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 0x5574e0e0ef20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/2448110222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e07570 0x5574e0e17820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2448110222 >> 172.21.15.131:0/2448110222 conn(0x5574e0d68a60 msgr2=0x5574e0e0a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.552+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2448110222 shutdown_connections 2024-05-10T05:34:07.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.553+0000 7f8f309a6b80 1 -- 172.21.15.131:0/2448110222 wait complete. 2024-05-10T05:34:07.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.553+0000 7f8f309a6b80 1 -- start start 2024-05-10T05:34:07.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.553+0000 7f8f309a6b80 1 Processor -- start 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e066a0 0x5574e0e14d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07570 0x5574e0e152d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f309a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 0x5574e0e160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0e16600 con 0x5574e0e07570 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0e16770 con 0x5574e0e07ec0 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f309a6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e0e16930 con 0x5574e0e066a0 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f309a6b80 10 client.0 _ll_register_callbacks cb 0x5574e0ccbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f11400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07570 0x5574e0e152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f11e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e066a0 0x5574e0e14d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f11400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07570 0x5574e0e152d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44632/0 (socket says 172.21.15.131:44632) 2024-05-10T05:34:07.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f11400700 1 -- 172.21.15.131:0/3284860546 learned_addr learned my addr 172.21.15.131:0/3284860546 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f12800700 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 0x5574e0e160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f11400700 1 -- 172.21.15.131:0/3284860546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e066a0 msgr2=0x5574e0e14d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f11400700 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e066a0 0x5574e0e14d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f11400700 1 -- 172.21.15.131:0/3284860546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 msgr2=0x5574e0e160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f11400700 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 0x5574e0e160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.554+0000 7f8f11400700 1 -- 172.21.15.131:0/3284860546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c00e810 con 0x5574e0e07570 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.555+0000 7f8f11400700 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07570 0x5574e0e152d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x5574e0e11430 tx=0x7f8f00012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4bb02c930a0ed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.555+0000 7f8ef3400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.555+0000 7f8ef3400700 1 -- 172.21.15.131:0/3284860546 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00017040 con 0x5574e0e07570 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.555+0000 7f8ef3400700 1 -- 172.21.15.131:0/3284860546 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8f00004220 con 0x5574e0e07570 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.555+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3284860546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f0409a9a0 con 0x5574e0e07570 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.555+0000 7f8ef3400700 1 -- 172.21.15.131:0/3284860546 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f000043e0 con 0x5574e0e07570 2024-05-10T05:34:07.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.555+0000 7f8f309a6b80 10 client.15501 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.555+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3284860546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5574e0babcd0 con 0x5574e0e07570 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.555+0000 7f8ef3400700 1 -- 172.21.15.131:0/3284860546 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f0409a9a0 con 0x5574e0e07570 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.556+0000 7f8ef3400700 1 -- 172.21.15.131:0/3284860546 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f000212e0 con 0x5574e0e07570 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.556+0000 7f8f309a6b80 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.556+0000 7f8f309a6b80 10 client.15501 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.556+0000 7f8f309a6b80 20 client.15501 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.556+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5574e0e72430 0x5574e0e74850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.556+0000 7f8f309a6b80 4 client.15501 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.556+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3284860546 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5574e0d1eec0 con 0x5574e0e72430 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.556+0000 7f8f11e00700 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5574e0e72430 0x5574e0e74850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.556+0000 7f8ef3400700 10 client.15501 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.556+0000 7f8f11e00700 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5574e0e72430 0x5574e0e74850 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a7f83b369414938 server_cookie=d4752bd0df4e07d5 in_seq=0 out_seq=0 2024-05-10T05:34:07.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8ef3400700 1 -- 172.21.15.131:0/3284860546 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5574e0d1eec0 con 0x5574e0e72430 2024-05-10T05:34:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8ef3400700 10 client.15501 handle_command_reply: tid=0 2024-05-10T05:34:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 client.15501 shutdown 2024-05-10T05:34:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3284860546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07570 msgr2=0x5574e0e152d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07570 0x5574e0e152d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x5574e0e11430 tx=0x7f8f00012b70 comp rx=0 tx=0).stop 2024-05-10T05:34:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3284860546 shutdown_connections 2024-05-10T05:34:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5574e0e72430 0x5574e0e74850 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574e0e07ec0 0x5574e0e160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574e0e066a0 0x5574e0e14d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8ef3400700 0 client.15501 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:34:07.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 --2- 172.21.15.131:0/3284860546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5574e0e07570 0x5574e0e152d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3284860546 >> 172.21.15.131:0/3284860546 conn(0x5574e0d68a60 msgr2=0x5574e0e052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3284860546 shutdown_connections 2024-05-10T05:34:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 1 -- 172.21.15.131:0/3284860546 wait complete. 2024-05-10T05:34:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.557+0000 7f8f309a6b80 20 client.15501 trim_cache size 0 max 16384 2024-05-10T05:34:07.563 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 -- 172.21.15.131:0/346486103 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8ef80a11c0 msgr2=0x7f8ef80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8ef80a11c0 0x7f8ef80a36d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009550 tx=0x7f8f0c007c00 comp rx=0 tx=0).stop 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 -- 172.21.15.131:0/346486103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f14101be0 msgr2=0x7f8f1419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f14101be0 0x7f8f1419a100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009fd0 tx=0x7f8f040036f0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 -- 172.21.15.131:0/346486103 shutdown_connections 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8ef80a11c0 0x7f8ef80a36d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f14101be0 0x7f8f1419a100 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f141036a0 0x7f8f141a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 --2- 172.21.15.131:0/346486103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f14102d00 0x7f8f1419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 -- 172.21.15.131:0/346486103 >> 172.21.15.131:0/346486103 conn(0x7f8f140fd210 msgr2=0x7f8f14073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 -- 172.21.15.131:0/346486103 shutdown_connections 2024-05-10T05:34:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.558+0000 7f8f18a00700 1 -- 172.21.15.131:0/346486103 wait complete. 2024-05-10T05:34:07.585 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:34:07.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.850+0000 7fa51e200700 1 -- 172.21.15.131:0/2829392470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa518069090 msgr2=0x7fa518104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.850+0000 7fa51e200700 1 --2- 172.21.15.131:0/2829392470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa518069090 0x7fa518104720 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa508009990 tx=0x7fa508009ca0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.850+0000 7fa51e200700 1 -- 172.21.15.131:0/2829392470 shutdown_connections 2024-05-10T05:34:07.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.850+0000 7fa51e200700 1 --2- 172.21.15.131:0/2829392470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa518069090 0x7fa518104720 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.850+0000 7fa51e200700 1 --2- 172.21.15.131:0/2829392470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa518107880 0x7fa518109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.850+0000 7fa51e200700 1 --2- 172.21.15.131:0/2829392470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518104d10 0x7fa5181071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.850+0000 7fa51e200700 1 -- 172.21.15.131:0/2829392470 >> 172.21.15.131:0/2829392470 conn(0x7fa5180fc0d0 msgr2=0x7fa5180fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.851+0000 7fa51e200700 1 -- 172.21.15.131:0/2829392470 shutdown_connections 2024-05-10T05:34:07.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.851+0000 7fa51e200700 1 -- 172.21.15.131:0/2829392470 wait complete. 2024-05-10T05:34:07.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa51e200700 1 Processor -- start 2024-05-10T05:34:07.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa51e200700 1 -- start start 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa51e200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa518069090 0x7fa51819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa51e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518104d10 0x7fa51819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa51e200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa518107880 0x7fa5181a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa51e200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5181a1c40 con 0x7fa518107880 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa51e200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5181a1db0 con 0x7fa518104d10 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa51e200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5181a1f70 con 0x7fa518069090 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa517e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa518107880 0x7fa5181a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa517400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa518069090 0x7fa51819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa517e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa518107880 0x7fa5181a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44652/0 (socket says 172.21.15.131:44652) 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.852+0000 7fa517e00700 1 -- 172.21.15.131:0/3385445814 learned_addr learned my addr 172.21.15.131:0/3385445814 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa516a00700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518104d10 0x7fa51819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa517e00700 1 -- 172.21.15.131:0/3385445814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa518069090 msgr2=0x7fa51819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa517e00700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa518069090 0x7fa51819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa517e00700 1 -- 172.21.15.131:0/3385445814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518104d10 msgr2=0x7fa51819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa517e00700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518104d10 0x7fa51819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa517e00700 1 -- 172.21.15.131:0/3385445814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5080095f0 con 0x7fa518107880 2024-05-10T05:34:07.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa517e00700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa518107880 0x7fa5181a1630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa51000ef30 tx=0x7fa51000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa50fe00700 1 -- 172.21.15.131:0/3385445814 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510018070 con 0x7fa518107880 2024-05-10T05:34:07.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa51e200700 1 -- 172.21.15.131:0/3385445814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5181a21f0 con 0x7fa518107880 2024-05-10T05:34:07.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa50fe00700 1 -- 172.21.15.131:0/3385445814 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa51000f040 con 0x7fa518107880 2024-05-10T05:34:07.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.853+0000 7fa50fe00700 1 -- 172.21.15.131:0/3385445814 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51000ccc0 con 0x7fa518107880 2024-05-10T05:34:07.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.854+0000 7fa51e200700 1 -- 172.21.15.131:0/3385445814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181a26b0 con 0x7fa518107880 2024-05-10T05:34:07.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.857+0000 7fa50fe00700 1 -- 172.21.15.131:0/3385445814 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa510003a00 con 0x7fa518107880 2024-05-10T05:34:07.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.858+0000 7fa50fe00700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4fc0a11c0 0x7fa4fc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.858+0000 7fa536184b80 1 Processor -- start 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.858+0000 7fa536184b80 1 -- start start 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.858+0000 7fa50fe00700 1 -- 172.21.15.131:0/3385445814 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa5100cc3b0 con 0x7fa518107880 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.858+0000 7fa517400700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4fc0a11c0 0x7fa4fc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa536184b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23324af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa536184b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23325980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa536184b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 0x55ad2332c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa536184b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2324b150 con 0x55ad233246a0 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa536184b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad23248b10 con 0x55ad23325ec0 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa536184b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad23244400 con 0x55ad23325570 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa517400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23324af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa516a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23325980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa516a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23325980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41266/0 (socket says 172.21.15.131:41266) 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa517400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23324af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44658/0 (socket says 172.21.15.131:44658) 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa516a00700 1 -- 172.21.15.131:0/2493124327 learned_addr learned my addr 172.21.15.131:0/2493124327 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa517e00700 1 --2- 172.21.15.131:0/2493124327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 0x55ad2332c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.859+0000 7fa517400700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4fc0a11c0 0x7fa4fc0a36d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa50800b7b0 tx=0x7fa508005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa516a00700 1 -- 172.21.15.131:0/2493124327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 msgr2=0x55ad2332c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa516a00700 1 --2- 172.21.15.131:0/2493124327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 0x55ad2332c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa516a00700 1 -- 172.21.15.131:0/2493124327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 msgr2=0x55ad23324af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa516a00700 1 --2- 172.21.15.131:0/2493124327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23324af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa516a00700 1 -- 172.21.15.131:0/2493124327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad230c53b0 con 0x55ad23325570 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa517e00700 1 --2- 172.21.15.131:0/2493124327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 0x55ad2332c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa517400700 1 --2- 172.21.15.131:0/2493124327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23324af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa516a00700 1 --2- 172.21.15.131:0/2493124327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23325980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa504009d20 tx=0x7fa504011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d14783bf171a395e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa503e00700 1 -- 172.21.15.131:0/2493124327 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504004a30 con 0x55ad23325570 2024-05-10T05:34:07.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa503e00700 1 -- 172.21.15.131:0/2493124327 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5040054e0 con 0x55ad23325570 2024-05-10T05:34:07.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.860+0000 7fa503e00700 1 -- 172.21.15.131:0/2493124327 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504019760 con 0x55ad23325570 2024-05-10T05:34:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.861+0000 7fa536184b80 1 -- 172.21.15.131:0/2493124327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 msgr2=0x55ad23325980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.861+0000 7fa536184b80 1 --2- 172.21.15.131:0/2493124327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23325980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa504009d20 tx=0x7fa504011720 comp rx=0 tx=0).stop 2024-05-10T05:34:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.861+0000 7fa536184b80 1 -- 172.21.15.131:0/2493124327 shutdown_connections 2024-05-10T05:34:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.861+0000 7fa536184b80 1 --2- 172.21.15.131:0/2493124327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23324af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.861+0000 7fa536184b80 1 --2- 172.21.15.131:0/2493124327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 0x55ad2332c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.861+0000 7fa536184b80 1 --2- 172.21.15.131:0/2493124327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23325980 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.861+0000 7fa536184b80 1 -- 172.21.15.131:0/2493124327 >> 172.21.15.131:0/2493124327 conn(0x55ad23286a60 msgr2=0x55ad2331efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.861+0000 7fa536184b80 1 -- 172.21.15.131:0/2493124327 shutdown_connections 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.861+0000 7fa536184b80 1 -- 172.21.15.131:0/2493124327 wait complete. 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.862+0000 7fa536184b80 1 -- start start 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.862+0000 7fa536184b80 1 Processor -- start 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.862+0000 7fa536184b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 0x55ad2332e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.862+0000 7fa536184b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325570 0x55ad2332ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.862+0000 7fa536184b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325ec0 0x55ad23335b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.862+0000 7fa536184b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad231e0560 con 0x55ad23325570 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.862+0000 7fa536184b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad231e0200 con 0x55ad233246a0 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.862+0000 7fa536184b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad230c76a0 con 0x55ad23325ec0 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa536184b80 10 client.0 _ll_register_callbacks cb 0x55ad231e9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa516a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325570 0x55ad2332ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa516a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325570 0x55ad2332ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44666/0 (socket says 172.21.15.131:44666) 2024-05-10T05:34:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa516a00700 1 -- 172.21.15.131:0/2283745436 learned_addr learned my addr 172.21.15.131:0/2283745436 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa517e00700 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325ec0 0x55ad23335b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa517400700 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 0x55ad2332e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa516a00700 1 -- 172.21.15.131:0/2283745436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325ec0 msgr2=0x55ad23335b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa516a00700 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325ec0 0x55ad23335b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa516a00700 1 -- 172.21.15.131:0/2283745436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 msgr2=0x55ad2332e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa516a00700 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 0x55ad2332e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa516a00700 1 -- 172.21.15.131:0/2283745436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181a26b0 con 0x55ad23325570 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa517e00700 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325ec0 0x55ad23335b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa517400700 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 0x55ad2332e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa516a00700 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325570 0x55ad2332ebe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa504009920 tx=0x7fa5040096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99be5919d8a078c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.863+0000 7fa500c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.864+0000 7fa500c00700 1 -- 172.21.15.131:0/2283745436 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504004a20 con 0x55ad23325570 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.864+0000 7fa500c00700 1 -- 172.21.15.131:0/2283745436 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa504004b80 con 0x55ad23325570 2024-05-10T05:34:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.864+0000 7fa536184b80 1 -- 172.21.15.131:0/2283745436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad2319d270 con 0x55ad23325570 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.864+0000 7fa500c00700 1 -- 172.21.15.131:0/2283745436 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5040194a0 con 0x55ad23325570 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.864+0000 7fa536184b80 10 client.15513 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.864+0000 7fa536184b80 1 -- 172.21.15.131:0/2283745436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa504009980 con 0x55ad23325570 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.864+0000 7fa500c00700 1 -- 172.21.15.131:0/2283745436 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad2319d270 con 0x55ad23325570 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.864+0000 7fa500c00700 1 -- 172.21.15.131:0/2283745436 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa504024740 con 0x55ad23325570 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.864+0000 7fa536184b80 10 client.15513 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.865+0000 7fa536184b80 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.865+0000 7fa536184b80 20 client.15513 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.865+0000 7fa536184b80 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad2333a210 0x55ad2333c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.865+0000 7fa536184b80 4 client.15513 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.865+0000 7fa536184b80 1 -- 172.21.15.131:0/2283745436 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad232f3780 con 0x55ad2333a210 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.865+0000 7fa517400700 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad2333a210 0x55ad2333c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.865+0000 7fa517400700 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad2333a210 0x55ad2333c660 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a13fdff04edc32 server_cookie=c4e72edbb7d13869 in_seq=0 out_seq=0 2024-05-10T05:34:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.865+0000 7fa500c00700 10 client.15513 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa500c00700 1 -- 172.21.15.131:0/2283745436 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad232f3780 con 0x55ad2333a210 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa500c00700 10 client.15513 handle_command_reply: tid=0 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa536184b80 1 client.15513 shutdown 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa536184b80 1 -- 172.21.15.131:0/2283745436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325570 msgr2=0x55ad2332ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa536184b80 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325570 0x55ad2332ebe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa504009920 tx=0x7fa5040096d0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa536184b80 1 -- 172.21.15.131:0/2283745436 shutdown_connections 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa536184b80 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad2333a210 0x55ad2333c660 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa536184b80 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325ec0 0x55ad23335b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa536184b80 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 0x55ad2332e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa536184b80 1 --2- 172.21.15.131:0/2283745436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325570 0x55ad2332ebe0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.866+0000 7fa536184b80 1 -- 172.21.15.131:0/2283745436 >> 172.21.15.131:0/2283745436 conn(0x55ad23286a60 msgr2=0x55ad2331efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.867+0000 7fa500c00700 0 client.15513 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:07.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.867+0000 7fa536184b80 1 -- 172.21.15.131:0/2283745436 shutdown_connections 2024-05-10T05:34:07.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.867+0000 7fa536184b80 1 -- 172.21.15.131:0/2283745436 wait complete. 2024-05-10T05:34:07.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.867+0000 7fa536184b80 20 client.15513 trim_cache size 0 max 16384 2024-05-10T05:34:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.880+0000 7fa536184b80 1 Processor -- start 2024-05-10T05:34:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.880+0000 7fa536184b80 1 -- start start 2024-05-10T05:34:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa536184b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 0x55ad2331b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa536184b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23335b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa536184b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325ec0 0x55ad2332dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa536184b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad233197c0 con 0x55ad23325ec0 2024-05-10T05:34:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa536184b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2332c670 con 0x55ad233246a0 2024-05-10T05:34:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa536184b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad23336120 con 0x55ad23325570 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa517e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325ec0 0x55ad2332dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa517e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325ec0 0x55ad2332dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44676/0 (socket says 172.21.15.131:44676) 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa517e00700 1 -- 172.21.15.131:0/1387568217 learned_addr learned my addr 172.21.15.131:0/1387568217 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa517400700 1 --2- 172.21.15.131:0/1387568217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 0x55ad2331b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa516a00700 1 --2- 172.21.15.131:0/1387568217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23335b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa517e00700 1 -- 172.21.15.131:0/1387568217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 msgr2=0x55ad23335b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa517e00700 1 --2- 172.21.15.131:0/1387568217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23335b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa517e00700 1 -- 172.21.15.131:0/1387568217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 msgr2=0x55ad2331b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa517e00700 1 --2- 172.21.15.131:0/1387568217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 0x55ad2331b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa517e00700 1 -- 172.21.15.131:0/1387568217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad23071b60 con 0x55ad23325ec0 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.881+0000 7fa516a00700 1 --2- 172.21.15.131:0/1387568217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23335b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.882+0000 7fa517e00700 1 --2- 172.21.15.131:0/1387568217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325ec0 0x55ad2332dc80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa51000ef00 tx=0x7fa510004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69737573907bc24c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.882+0000 7fa50d200700 1 -- 172.21.15.131:0/1387568217 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51009e580 con 0x55ad23325ec0 2024-05-10T05:34:07.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.882+0000 7fa50d200700 1 -- 172.21.15.131:0/1387568217 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa510004ad0 con 0x55ad23325ec0 2024-05-10T05:34:07.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.882+0000 7fa50d200700 1 -- 172.21.15.131:0/1387568217 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100a1cb0 con 0x55ad23325ec0 2024-05-10T05:34:07.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.883+0000 7fa536184b80 1 -- 172.21.15.131:0/1387568217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325ec0 msgr2=0x55ad2332dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.883+0000 7fa536184b80 1 --2- 172.21.15.131:0/1387568217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325ec0 0x55ad2332dc80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa51000ef00 tx=0x7fa510004dc0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.883+0000 7fa536184b80 1 -- 172.21.15.131:0/1387568217 shutdown_connections 2024-05-10T05:34:07.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.883+0000 7fa536184b80 1 --2- 172.21.15.131:0/1387568217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad23325ec0 0x55ad2332dc80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.883+0000 7fa536184b80 1 --2- 172.21.15.131:0/1387568217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad233246a0 0x55ad2331b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.883+0000 7fa536184b80 1 --2- 172.21.15.131:0/1387568217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23335b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.883+0000 7fa536184b80 1 -- 172.21.15.131:0/1387568217 >> 172.21.15.131:0/1387568217 conn(0x55ad23286a60 msgr2=0x55ad2331ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.883+0000 7fa536184b80 1 -- 172.21.15.131:0/1387568217 shutdown_connections 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.883+0000 7fa536184b80 1 -- 172.21.15.131:0/1387568217 wait complete. 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.884+0000 7fa536184b80 1 -- start start 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.884+0000 7fa536184b80 1 Processor -- start 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.884+0000 7fa536184b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23335ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.884+0000 7fa536184b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23332740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.884+0000 7fa536184b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 0x55ad23332d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.884+0000 7fa536184b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2338e6d0 con 0x55ad233246a0 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.884+0000 7fa536184b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2338e810 con 0x55ad23325ec0 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.884+0000 7fa536184b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2338e950 con 0x55ad23325570 2024-05-10T05:34:07.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.884+0000 7fa536184b80 10 client.0 _ll_register_callbacks cb 0x55ad231e9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23335ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23335ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44690/0 (socket says 172.21.15.131:44690) 2024-05-10T05:34:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517400700 1 -- 172.21.15.131:0/1345366016 learned_addr learned my addr 172.21.15.131:0/1345366016 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa516a00700 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23332740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517e00700 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 0x55ad23332d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517400700 1 -- 172.21.15.131:0/1345366016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 msgr2=0x55ad23332740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517400700 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23332740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517400700 1 -- 172.21.15.131:0/1345366016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 msgr2=0x55ad23332d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517400700 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 0x55ad23332d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517400700 1 -- 172.21.15.131:0/1345366016 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181a21f0 con 0x55ad233246a0 2024-05-10T05:34:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517e00700 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 0x55ad23332d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa517400700 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23335ad0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa508009990 tx=0x7fa508027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95603207e76af314 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.885+0000 7fa500c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa500c00700 1 -- 172.21.15.131:0/1345366016 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5080368c0 con 0x55ad233246a0 2024-05-10T05:34:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa500c00700 1 -- 172.21.15.131:0/1345366016 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa50803f460 con 0x55ad233246a0 2024-05-10T05:34:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa536184b80 1 -- 172.21.15.131:0/1345366016 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5040054e0 con 0x55ad233246a0 2024-05-10T05:34:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa500c00700 1 -- 172.21.15.131:0/1345366016 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa50803f620 con 0x55ad233246a0 2024-05-10T05:34:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa536184b80 10 client.15519 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa536184b80 1 -- 172.21.15.131:0/1345366016 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ad2320a560 con 0x55ad233246a0 2024-05-10T05:34:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa500c00700 1 -- 172.21.15.131:0/1345366016 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5040054e0 con 0x55ad233246a0 2024-05-10T05:34:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa500c00700 1 -- 172.21.15.131:0/1345366016 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa50800f870 con 0x55ad233246a0 2024-05-10T05:34:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa536184b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa536184b80 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa536184b80 20 client.15519 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa536184b80 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad23390310 0x55ad23392730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa536184b80 4 client.15519 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.886+0000 7fa536184b80 1 -- 172.21.15.131:0/1345366016 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ad2323cec0 con 0x55ad23390310 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.887+0000 7fa516a00700 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad23390310 0x55ad23392730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.887+0000 7fa500c00700 10 client.15519 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.887+0000 7fa516a00700 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad23390310 0x55ad23392730 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f39d39ebe2d935 server_cookie=42b901dcf56febec in_seq=0 out_seq=0 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.887+0000 7fa500c00700 1 -- 172.21.15.131:0/1345366016 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ad2323cec0 con 0x55ad23390310 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.887+0000 7fa500c00700 10 client.15519 handle_command_reply: tid=0 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.887+0000 7fa536184b80 1 client.15519 shutdown 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 1 -- 172.21.15.131:0/1345366016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 msgr2=0x55ad23335ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23335ad0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa508009990 tx=0x7fa508027000 comp rx=0 tx=0).stop 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 1 -- 172.21.15.131:0/1345366016 shutdown_connections 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad23390310 0x55ad23392730 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad233246a0 0x55ad23335ad0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad23325ec0 0x55ad23332d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 1 --2- 172.21.15.131:0/1345366016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad23325570 0x55ad23332740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa500c00700 0 client.15519 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 1 -- 172.21.15.131:0/1345366016 >> 172.21.15.131:0/1345366016 conn(0x55ad23286a60 msgr2=0x55ad2333c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 1 -- 172.21.15.131:0/1345366016 shutdown_connections 2024-05-10T05:34:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 1 -- 172.21.15.131:0/1345366016 wait complete. 2024-05-10T05:34:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.888+0000 7fa536184b80 20 client.15519 trim_cache size 0 max 16384 2024-05-10T05:34:07.893 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:34:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 -- 172.21.15.131:0/3385445814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4fc0a11c0 msgr2=0x7fa4fc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4fc0a11c0 0x7fa4fc0a36d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa50800b7b0 tx=0x7fa508005fd0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 -- 172.21.15.131:0/3385445814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa518107880 msgr2=0x7fa5181a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa518107880 0x7fa5181a1630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa51000ef30 tx=0x7fa51000c3a0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 -- 172.21.15.131:0/3385445814 shutdown_connections 2024-05-10T05:34:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4fc0a11c0 0x7fa4fc0a36d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa518107880 0x7fa5181a1630 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa518069090 0x7fa51819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 --2- 172.21.15.131:0/3385445814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518104d10 0x7fa51819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 -- 172.21.15.131:0/3385445814 >> 172.21.15.131:0/3385445814 conn(0x7fa5180fc0d0 msgr2=0x7fa5180fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 -- 172.21.15.131:0/3385445814 shutdown_connections 2024-05-10T05:34:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:07.889+0000 7fa51e200700 1 -- 172.21.15.131:0/3385445814 wait complete. 2024-05-10T05:34:08.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:07 smithi131 ceph-mon[37264]: pgmap v308: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:07 smithi157 ceph-mon[78427]: pgmap v308: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:08.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:07 smithi142 ceph-mon[77667]: pgmap v308: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:08.916 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:34:08.916 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:34:09.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.189+0000 7f1058c00700 1 -- 172.21.15.131:0/4090828974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1054102b10 msgr2=0x7f1054102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.189+0000 7f1058c00700 1 --2- 172.21.15.131:0/4090828974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1054102b10 0x7f1054102f20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f10400098f0 tx=0x7f1040009c00 comp rx=0 tx=0).stop 2024-05-10T05:34:09.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.189+0000 7f1058c00700 1 -- 172.21.15.131:0/4090828974 shutdown_connections 2024-05-10T05:34:09.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.189+0000 7f1058c00700 1 --2- 172.21.15.131:0/4090828974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10541034b0 0x7f1054109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.189+0000 7f1058c00700 1 --2- 172.21.15.131:0/4090828974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10541019f0 0x7f1054101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.189+0000 7f1058c00700 1 --2- 172.21.15.131:0/4090828974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1054102b10 0x7f1054102f20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.189+0000 7f1058c00700 1 -- 172.21.15.131:0/4090828974 >> 172.21.15.131:0/4090828974 conn(0x7f10540fd000 msgr2=0x7f10540ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.189+0000 7f1058c00700 1 -- 172.21.15.131:0/4090828974 shutdown_connections 2024-05-10T05:34:09.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.189+0000 7f1058c00700 1 -- 172.21.15.131:0/4090828974 wait complete. 2024-05-10T05:34:09.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.190+0000 7f1058c00700 1 Processor -- start 2024-05-10T05:34:09.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.190+0000 7f1058c00700 1 -- start start 2024-05-10T05:34:09.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.190+0000 7f1058c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10541019f0 0x7f1054074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.190+0000 7f1058c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1054102b10 0x7f1054073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.190+0000 7f1058c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10541034b0 0x7f1054073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.190+0000 7f1058c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1054074100 con 0x7f10541019f0 2024-05-10T05:34:09.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.190+0000 7f1058c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1054074270 con 0x7f10541034b0 2024-05-10T05:34:09.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.190+0000 7f1058c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1054074430 con 0x7f1054102b10 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1053400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10541019f0 0x7f1054074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1053e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10541034b0 0x7f1054073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1053e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10541034b0 0x7f1054073aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38016/0 (socket says 172.21.15.131:38016) 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1053e00700 1 -- 172.21.15.131:0/1309894962 learned_addr learned my addr 172.21.15.131:0/1309894962 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1052a00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1054102b10 0x7f1054073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1053e00700 1 -- 172.21.15.131:0/1309894962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1054102b10 msgr2=0x7f1054073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1053e00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1054102b10 0x7f1054073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1053e00700 1 -- 172.21.15.131:0/1309894962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10541019f0 msgr2=0x7f1054074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1053e00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10541019f0 0x7f1054074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1053e00700 1 -- 172.21.15.131:0/1309894962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10440095f0 con 0x7f10541034b0 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.191+0000 7f1053e00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10541034b0 0x7f1054073aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f104c00ef30 tx=0x7f104c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.192+0000 7f104be00700 1 -- 172.21.15.131:0/1309894962 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c018070 con 0x7f10541034b0 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.192+0000 7f104be00700 1 -- 172.21.15.131:0/1309894962 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f104c00f040 con 0x7f10541034b0 2024-05-10T05:34:09.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.192+0000 7f1058c00700 1 -- 172.21.15.131:0/1309894962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1040009520 con 0x7f10541034b0 2024-05-10T05:34:09.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.192+0000 7f104be00700 1 -- 172.21.15.131:0/1309894962 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c00cd00 con 0x7f10541034b0 2024-05-10T05:34:09.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.192+0000 7f1058c00700 1 -- 172.21.15.131:0/1309894962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10541a23a0 con 0x7f10541034b0 2024-05-10T05:34:09.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.195+0000 7f104be00700 1 -- 172.21.15.131:0/1309894962 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f104c003a00 con 0x7f10541034b0 2024-05-10T05:34:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.196+0000 7f104be00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10380a1120 0x7f10380a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.196+0000 7f10720afb80 1 Processor -- start 2024-05-10T05:34:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.196+0000 7f104be00700 1 -- 172.21.15.131:0/1309894962 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f104c0cc480 con 0x7f10541034b0 2024-05-10T05:34:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.196+0000 7f1053400700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10380a1120 0x7f10380a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f10720afb80 1 -- start start 2024-05-10T05:34:09.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f10720afb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f6215b0 0x55932f6219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f10720afb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f622c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f10720afb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 0x55932f629a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f10720afb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f5481d0 con 0x55932f622820 2024-05-10T05:34:09.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f10720afb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f545b90 con 0x55932f623170 2024-05-10T05:34:09.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f10720afb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f541480 con 0x55932f6215b0 2024-05-10T05:34:09.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f1052a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f622c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f1053400700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10380a1120 0x7f10380a3630 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1054102a10 tx=0x7f1044009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f1053400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f6215b0 0x55932f6219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f1052a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f622c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44722/0 (socket says 172.21.15.131:44722) 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f1053400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f6215b0 0x55932f6219e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41314/0 (socket says 172.21.15.131:41314) 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f1052a00700 1 -- 172.21.15.131:0/3620851023 learned_addr learned my addr 172.21.15.131:0/3620851023 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.197+0000 7f1053e00700 1 --2- 172.21.15.131:0/3620851023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 0x55932f629a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.198+0000 7f1052a00700 1 -- 172.21.15.131:0/3620851023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f6215b0 msgr2=0x55932f6219e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.198+0000 7f1052a00700 1 --2- 172.21.15.131:0/3620851023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f6215b0 0x55932f6219e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.198+0000 7f1052a00700 1 -- 172.21.15.131:0/3620851023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 msgr2=0x55932f629a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.198+0000 7f1052a00700 1 --2- 172.21.15.131:0/3620851023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 0x55932f629a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.198+0000 7f1052a00700 1 -- 172.21.15.131:0/3620851023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55932f3c2430 con 0x55932f622820 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.198+0000 7f1052a00700 1 --2- 172.21.15.131:0/3620851023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f622c30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1040009230 tx=0x7f1040009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f57de7a665871e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.198+0000 7f103fe00700 1 -- 172.21.15.131:0/3620851023 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104000f660 con 0x55932f622820 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.198+0000 7f103fe00700 1 -- 172.21.15.131:0/3620851023 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f104001d040 con 0x55932f622820 2024-05-10T05:34:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.198+0000 7f103fe00700 1 -- 172.21.15.131:0/3620851023 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040005500 con 0x55932f622820 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.199+0000 7f10720afb80 1 -- 172.21.15.131:0/3620851023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 msgr2=0x55932f622c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.199+0000 7f10720afb80 1 --2- 172.21.15.131:0/3620851023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f622c30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1040009230 tx=0x7f1040009f70 comp rx=0 tx=0).stop 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.199+0000 7f10720afb80 1 -- 172.21.15.131:0/3620851023 shutdown_connections 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.199+0000 7f10720afb80 1 --2- 172.21.15.131:0/3620851023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 0x55932f629a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.199+0000 7f10720afb80 1 --2- 172.21.15.131:0/3620851023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f6215b0 0x55932f6219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.199+0000 7f10720afb80 1 --2- 172.21.15.131:0/3620851023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f622c30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.199+0000 7f10720afb80 1 -- 172.21.15.131:0/3620851023 >> 172.21.15.131:0/3620851023 conn(0x55932f583ae0 msgr2=0x55932f61c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.199+0000 7f10720afb80 1 -- 172.21.15.131:0/3620851023 shutdown_connections 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.199+0000 7f10720afb80 1 -- 172.21.15.131:0/3620851023 wait complete. 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.200+0000 7f10720afb80 1 -- start start 2024-05-10T05:34:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.200+0000 7f10720afb80 1 Processor -- start 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.200+0000 7f10720afb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f6215b0 0x55932f62b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.200+0000 7f10720afb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f62bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.200+0000 7f10720afb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f623170 0x55932f632e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.200+0000 7f10720afb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f4dd5e0 con 0x55932f622820 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.200+0000 7f10720afb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f4dd280 con 0x55932f6215b0 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f10720afb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f3c4720 con 0x55932f623170 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f10720afb80 10 client.0 _ll_register_callbacks cb 0x55932f4e6bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f1052a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f62bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f1052a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f62bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44736/0 (socket says 172.21.15.131:44736) 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f1052a00700 1 -- 172.21.15.131:0/1786082294 learned_addr learned my addr 172.21.15.131:0/1786082294 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f1053400700 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f6215b0 0x55932f62b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f1053e00700 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f623170 0x55932f632e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f1052a00700 1 -- 172.21.15.131:0/1786082294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f623170 msgr2=0x55932f632e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f1052a00700 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f623170 0x55932f632e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f1052a00700 1 -- 172.21.15.131:0/1786082294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f6215b0 msgr2=0x55932f62b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f1052a00700 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f6215b0 0x55932f62b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.201+0000 7f1052a00700 1 -- 172.21.15.131:0/1786082294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105410a7f0 con 0x55932f622820 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.202+0000 7f1052a00700 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f62bf20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1040005e50 tx=0x7f104000bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edba878fab4d63f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.202+0000 7f103cc00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.202+0000 7f103cc00700 1 -- 172.21.15.131:0/1786082294 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104000f460 con 0x55932f622820 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.202+0000 7f10720afb80 1 -- 172.21.15.131:0/1786082294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55932f49a2f0 con 0x55932f622820 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.202+0000 7f103cc00700 1 -- 172.21.15.131:0/1786082294 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f104000f5c0 con 0x55932f622820 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.202+0000 7f103cc00700 1 -- 172.21.15.131:0/1786082294 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f104000f780 con 0x55932f622820 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.202+0000 7f10720afb80 10 client.15531 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.202+0000 7f10720afb80 1 -- 172.21.15.131:0/1786082294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55932f36ebe0 con 0x55932f622820 2024-05-10T05:34:09.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.202+0000 7f103cc00700 1 -- 172.21.15.131:0/1786082294 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1040039020 con 0x55932f622820 2024-05-10T05:34:09.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.202+0000 7f103cc00700 1 -- 172.21.15.131:0/1786082294 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1040015020 con 0x55932f622820 2024-05-10T05:34:09.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.203+0000 7f10720afb80 10 client.15531 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:09.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.203+0000 7f10720afb80 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:09.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.203+0000 7f10720afb80 20 client.15531 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:09.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.203+0000 7f10720afb80 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55932f637180 0x55932f6395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.203+0000 7f10720afb80 4 client.15531 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:09.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.203+0000 7f10720afb80 1 -- 172.21.15.131:0/1786082294 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55932f5f0800 con 0x55932f637180 2024-05-10T05:34:09.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.203+0000 7f1053400700 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55932f637180 0x55932f6395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.204+0000 7f103cc00700 10 client.15531 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.204+0000 7f1053400700 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55932f637180 0x55932f6395d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c5d1e86598c1250 server_cookie=4e7ea0897cc46a4 in_seq=0 out_seq=0 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.204+0000 7f103cc00700 1 -- 172.21.15.131:0/1786082294 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55932f5f0800 con 0x55932f637180 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.204+0000 7f103cc00700 10 client.15531 handle_command_reply: tid=0 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.204+0000 7f10720afb80 1 client.15531 shutdown 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 1 -- 172.21.15.131:0/1786082294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 msgr2=0x55932f62bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f62bf20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1040005e50 tx=0x7f104000bba0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 1 -- 172.21.15.131:0/1786082294 shutdown_connections 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55932f637180 0x55932f6395d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f623170 0x55932f632e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f6215b0 0x55932f62b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f103cc00700 0 client.15531 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 1 --2- 172.21.15.131:0/1786082294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f62bf20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 1 -- 172.21.15.131:0/1786082294 >> 172.21.15.131:0/1786082294 conn(0x55932f583ae0 msgr2=0x55932f6200e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 1 -- 172.21.15.131:0/1786082294 shutdown_connections 2024-05-10T05:34:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 1 -- 172.21.15.131:0/1786082294 wait complete. 2024-05-10T05:34:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.205+0000 7f10720afb80 20 client.15531 trim_cache size 0 max 16384 2024-05-10T05:34:09.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.223+0000 7f10720afb80 1 Processor -- start 2024-05-10T05:34:09.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.223+0000 7f10720afb80 1 -- start start 2024-05-10T05:34:09.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.223+0000 7f10720afb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f6215b0 0x55932f630e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.223+0000 7f10720afb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f622820 0x55932f631370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.223+0000 7f10720afb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 0x55932f61bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.223+0000 7f10720afb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f629df0 con 0x55932f6215b0 2024-05-10T05:34:09.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.223+0000 7f10720afb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f616370 con 0x55932f623170 2024-05-10T05:34:09.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.223+0000 7f10720afb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f616950 con 0x55932f622820 2024-05-10T05:34:09.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f6215b0 0x55932f630e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 0x55932f61bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1052a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f622820 0x55932f631370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 0x55932f61bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38038/0 (socket says 172.21.15.131:38038) 2024-05-10T05:34:09.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053e00700 1 -- 172.21.15.131:0/3942348689 learned_addr learned my addr 172.21.15.131:0/3942348689 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053e00700 1 -- 172.21.15.131:0/3942348689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f622820 msgr2=0x55932f631370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053e00700 1 --2- 172.21.15.131:0/3942348689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f622820 0x55932f631370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053e00700 1 -- 172.21.15.131:0/3942348689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f6215b0 msgr2=0x55932f630e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053e00700 1 --2- 172.21.15.131:0/3942348689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f6215b0 0x55932f630e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053e00700 1 -- 172.21.15.131:0/3942348689 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55932f5075e0 con 0x55932f623170 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053400700 1 --2- 172.21.15.131:0/3942348689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f6215b0 0x55932f630e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.224+0000 7f1053e00700 1 --2- 172.21.15.131:0/3942348689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 0x55932f61bc30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f104c00ef00 tx=0x7f104c0092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab8fef83bf2e43ae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.225+0000 7f1049200700 1 -- 172.21.15.131:0/3942348689 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0b2260 con 0x55932f623170 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.225+0000 7f1049200700 1 -- 172.21.15.131:0/3942348689 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f104c01b320 con 0x55932f623170 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.225+0000 7f1049200700 1 -- 172.21.15.131:0/3942348689 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0a1d80 con 0x55932f623170 2024-05-10T05:34:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.225+0000 7f10720afb80 1 -- 172.21.15.131:0/3942348689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 msgr2=0x55932f61bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.225+0000 7f10720afb80 1 --2- 172.21.15.131:0/3942348689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 0x55932f61bc30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f104c00ef00 tx=0x7f104c0092e0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.225+0000 7f10720afb80 1 -- 172.21.15.131:0/3942348689 shutdown_connections 2024-05-10T05:34:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.225+0000 7f10720afb80 1 --2- 172.21.15.131:0/3942348689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f6215b0 0x55932f630e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.225+0000 7f10720afb80 1 --2- 172.21.15.131:0/3942348689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f623170 0x55932f61bc30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.225+0000 7f10720afb80 1 --2- 172.21.15.131:0/3942348689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f622820 0x55932f631370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.225+0000 7f10720afb80 1 -- 172.21.15.131:0/3942348689 >> 172.21.15.131:0/3942348689 conn(0x55932f583ae0 msgr2=0x55932f626710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.226+0000 7f10720afb80 1 -- 172.21.15.131:0/3942348689 shutdown_connections 2024-05-10T05:34:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.226+0000 7f10720afb80 1 -- 172.21.15.131:0/3942348689 wait complete. 2024-05-10T05:34:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.226+0000 7f10720afb80 1 -- start start 2024-05-10T05:34:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.226+0000 7f10720afb80 1 Processor -- start 2024-05-10T05:34:09.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f10720afb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f6215b0 0x55932f685840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f10720afb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f685d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f10720afb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f623170 0x55932f68cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f10720afb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f68d060 con 0x55932f622820 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f10720afb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f68d1a0 con 0x55932f6215b0 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f10720afb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932f68d330 con 0x55932f623170 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f10720afb80 10 client.0 _ll_register_callbacks cb 0x55932f4e6bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f1052a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f685d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f1052a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f685d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44756/0 (socket says 172.21.15.131:44756) 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f1052a00700 1 -- 172.21.15.131:0/145544830 learned_addr learned my addr 172.21.15.131:0/145544830 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f1053e00700 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f623170 0x55932f68cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f1053400700 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f6215b0 0x55932f685840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f1052a00700 1 -- 172.21.15.131:0/145544830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f623170 msgr2=0x55932f68cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f1052a00700 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f623170 0x55932f68cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f1052a00700 1 -- 172.21.15.131:0/145544830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f6215b0 msgr2=0x55932f685840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f1052a00700 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f6215b0 0x55932f685840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.227+0000 7f1052a00700 1 -- 172.21.15.131:0/145544830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1044017070 con 0x55932f622820 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f1052a00700 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f685d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55932f622680 tx=0x7f1040005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a1d1f7ed3b01cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f103cc00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f103cc00700 1 -- 172.21.15.131:0/145544830 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104002ae90 con 0x55932f622820 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f103cc00700 1 -- 172.21.15.131:0/145544830 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1040000e40 con 0x55932f622820 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f10720afb80 1 -- 172.21.15.131:0/145544830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f104001d040 con 0x55932f622820 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f103cc00700 1 -- 172.21.15.131:0/145544830 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10400050b0 con 0x55932f622820 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f10720afb80 10 client.15537 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:09.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f10720afb80 1 -- 172.21.15.131:0/145544830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f10541a23a0 con 0x55932f622820 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f103cc00700 1 -- 172.21.15.131:0/145544830 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f104001d040 con 0x55932f622820 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f103cc00700 1 -- 172.21.15.131:0/145544830 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1040009920 con 0x55932f622820 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f10720afb80 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f10720afb80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.228+0000 7f10720afb80 20 client.15537 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.229+0000 7f10720afb80 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55932f68e8e0 0x55932f690d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.229+0000 7f10720afb80 4 client.15537 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.229+0000 7f10720afb80 1 -- 172.21.15.131:0/145544830 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55932f539f40 con 0x55932f68e8e0 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.229+0000 7f1053400700 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55932f68e8e0 0x55932f690d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.229+0000 7f103cc00700 10 client.15537 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:09.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.229+0000 7f1053400700 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55932f68e8e0 0x55932f690d00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82ff726e956b053 server_cookie=a9f767533b661286 in_seq=0 out_seq=0 2024-05-10T05:34:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f103cc00700 1 -- 172.21.15.131:0/145544830 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55932f539f40 con 0x55932f68e8e0 2024-05-10T05:34:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f103cc00700 10 client.15537 handle_command_reply: tid=0 2024-05-10T05:34:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 client.15537 shutdown 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 -- 172.21.15.131:0/145544830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 msgr2=0x55932f685d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f685d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55932f622680 tx=0x7f1040005f00 comp rx=0 tx=0).stop 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 -- 172.21.15.131:0/145544830 shutdown_connections 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55932f68e8e0 0x55932f690d00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55932f623170 0x55932f68cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55932f6215b0 0x55932f685840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 --2- 172.21.15.131:0/145544830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55932f622820 0x55932f685d80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 -- 172.21.15.131:0/145544830 >> 172.21.15.131:0/145544830 conn(0x55932f583ae0 msgr2=0x55932f61bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f103cc00700 0 client.15537 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 -- 172.21.15.131:0/145544830 shutdown_connections 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.231+0000 7f10720afb80 1 -- 172.21.15.131:0/145544830 wait complete. 2024-05-10T05:34:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f10720afb80 20 client.15537 trim_cache size 0 max 16384 2024-05-10T05:34:09.235 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:34:09.235 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:34:09.235 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "acf77466-d853-48ec-8b00-2564d7f47264", 2024-05-10T05:34:09.235 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:34:09.235 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f1058c00700 1 -- 172.21.15.131:0/1309894962 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10380a1120 msgr2=0x7f10380a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f1058c00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10380a1120 0x7f10380a3630 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1054102a10 tx=0x7f1044009310 comp rx=0 tx=0).stop 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f1058c00700 1 -- 172.21.15.131:0/1309894962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10541034b0 msgr2=0x7f1054073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f1058c00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10541034b0 0x7f1054073aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f104c00ef30 tx=0x7f104c00c3a0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f1058c00700 1 -- 172.21.15.131:0/1309894962 shutdown_connections 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f1058c00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10380a1120 0x7f10380a3630 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f1058c00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10541019f0 0x7f1054074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f1058c00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10541034b0 0x7f1054073aa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f1058c00700 1 --2- 172.21.15.131:0/1309894962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1054102b10 0x7f1054073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.232+0000 7f1058c00700 1 -- 172.21.15.131:0/1309894962 >> 172.21.15.131:0/1309894962 conn(0x7f10540fd000 msgr2=0x7f1054105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.233+0000 7f1058c00700 1 -- 172.21.15.131:0/1309894962 shutdown_connections 2024-05-10T05:34:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.233+0000 7f1058c00700 1 -- 172.21.15.131:0/1309894962 wait complete. 2024-05-10T05:34:09.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:34:09.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.524+0000 7f6a95a00700 1 -- 172.21.15.131:0/1046888597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a90101be0 msgr2=0x7f6a90101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.524+0000 7f6a95a00700 1 --2- 172.21.15.131:0/1046888597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a90101be0 0x7f6a90101ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009920 tx=0x7f6a80009c30 comp rx=0 tx=0).stop 2024-05-10T05:34:09.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.524+0000 7f6a95a00700 1 -- 172.21.15.131:0/1046888597 shutdown_connections 2024-05-10T05:34:09.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.524+0000 7f6a95a00700 1 --2- 172.21.15.131:0/1046888597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a901036a0 0x7f6a90109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.524+0000 7f6a95a00700 1 --2- 172.21.15.131:0/1046888597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a90101be0 0x7f6a90101ff0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.524+0000 7f6a95a00700 1 --2- 172.21.15.131:0/1046888597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a90102d00 0x7f6a90103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.524+0000 7f6a95a00700 1 -- 172.21.15.131:0/1046888597 >> 172.21.15.131:0/1046888597 conn(0x7f6a900fd210 msgr2=0x7f6a900ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.525+0000 7f6a95a00700 1 -- 172.21.15.131:0/1046888597 shutdown_connections 2024-05-10T05:34:09.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.525+0000 7f6a95a00700 1 -- 172.21.15.131:0/1046888597 wait complete. 2024-05-10T05:34:09.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a95a00700 1 Processor -- start 2024-05-10T05:34:09.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a95a00700 1 -- start start 2024-05-10T05:34:09.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a95a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90101be0 0x7f6a9019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a95a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a90102d00 0x7f6a9019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a95a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a901036a0 0x7f6a901a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a95a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901a1c60 con 0x7f6a901036a0 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a95a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901a1dd0 con 0x7f6a90102d00 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a95a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901a1f90 con 0x7f6a90101be0 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a8ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90101be0 0x7f6a9019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a8ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90101be0 0x7f6a9019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41368/0 (socket says 172.21.15.131:41368) 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a8ea00700 1 -- 172.21.15.131:0/1895983205 learned_addr learned my addr 172.21.15.131:0/1895983205 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.526+0000 7f6a8e000700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a90102d00 0x7f6a9019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a8f400700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a901036a0 0x7f6a901a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a8ea00700 1 -- 172.21.15.131:0/1895983205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a90102d00 msgr2=0x7f6a9019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a8ea00700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a90102d00 0x7f6a9019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a8ea00700 1 -- 172.21.15.131:0/1895983205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a901036a0 msgr2=0x7f6a901a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a8ea00700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a901036a0 0x7f6a901a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a8ea00700 1 -- 172.21.15.131:0/1895983205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a80009520 con 0x7f6a90101be0 2024-05-10T05:34:09.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a8ea00700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90101be0 0x7f6a9019a180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a80000c00 tx=0x7f6a8000f950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a87400700 1 -- 172.21.15.131:0/1895983205 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8001c070 con 0x7f6a90101be0 2024-05-10T05:34:09.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a87400700 1 -- 172.21.15.131:0/1895983205 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a8000bb20 con 0x7f6a90101be0 2024-05-10T05:34:09.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a95a00700 1 -- 172.21.15.131:0/1895983205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a901a2210 con 0x7f6a90101be0 2024-05-10T05:34:09.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a87400700 1 -- 172.21.15.131:0/1895983205 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800038e0 con 0x7f6a90101be0 2024-05-10T05:34:09.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.527+0000 7f6a95a00700 1 -- 172.21.15.131:0/1895983205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a901a2630 con 0x7f6a90101be0 2024-05-10T05:34:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.532+0000 7f6a87400700 1 -- 172.21.15.131:0/1895983205 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6a80020030 con 0x7f6a90101be0 2024-05-10T05:34:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.533+0000 7f6aad989b80 1 Processor -- start 2024-05-10T05:34:09.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.533+0000 7f6aad989b80 1 -- start start 2024-05-10T05:34:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.533+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca81966a0 0x560ca8196af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.533+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca8197570 0x560ca8197980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.533+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 0x560ca819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.533+0000 7f6aad989b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca80bd150 con 0x560ca81966a0 2024-05-10T05:34:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.533+0000 7f6aad989b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca80bab10 con 0x560ca8197ec0 2024-05-10T05:34:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.533+0000 7f6aad989b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca80b6400 con 0x560ca8197570 2024-05-10T05:34:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca81966a0 0x560ca8196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 0x560ca819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca81966a0 0x560ca8196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44786/0 (socket says 172.21.15.131:44786) 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8f400700 1 -- 172.21.15.131:0/435137996 learned_addr learned my addr 172.21.15.131:0/435137996 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8ea00700 1 --2- 172.21.15.131:0/435137996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca8197570 0x560ca8197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a87400700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a740a11c0 0x7f6a740a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8e000700 1 -- 172.21.15.131:0/435137996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca8197570 msgr2=0x560ca8197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8e000700 1 --2- 172.21.15.131:0/435137996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca8197570 0x560ca8197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8e000700 1 -- 172.21.15.131:0/435137996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca81966a0 msgr2=0x560ca8196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8e000700 1 --2- 172.21.15.131:0/435137996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca81966a0 0x560ca8196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8e000700 1 -- 172.21.15.131:0/435137996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ca7f373b0 con 0x560ca8197ec0 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a87400700 1 -- 172.21.15.131:0/1895983205 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6a800cd890 con 0x7f6a90101be0 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.534+0000 7f6a8f400700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a740a11c0 0x7f6a740a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.535+0000 7f6a8e000700 1 --2- 172.21.15.131:0/435137996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 0x560ca819e7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c005fd0 tx=0x7f6a7c011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ec92bb95788fe8a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.535+0000 7f6a7be00700 1 -- 172.21.15.131:0/435137996 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c011cc0 con 0x560ca8197ec0 2024-05-10T05:34:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.535+0000 7f6a7be00700 1 -- 172.21.15.131:0/435137996 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a7c011e20 con 0x560ca8197ec0 2024-05-10T05:34:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.535+0000 7f6a7be00700 1 -- 172.21.15.131:0/435137996 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0193e0 con 0x560ca8197ec0 2024-05-10T05:34:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6aad989b80 1 -- 172.21.15.131:0/435137996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 msgr2=0x560ca819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6a8f400700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a740a11c0 0x7f6a740a36d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x560ca8196380 tx=0x7f6a88009040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6aad989b80 1 --2- 172.21.15.131:0/435137996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 0x560ca819e7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c005fd0 tx=0x7f6a7c011440 comp rx=0 tx=0).stop 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6aad989b80 1 -- 172.21.15.131:0/435137996 shutdown_connections 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6aad989b80 1 --2- 172.21.15.131:0/435137996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca81966a0 0x560ca8196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6aad989b80 1 --2- 172.21.15.131:0/435137996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 0x560ca819e7c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6aad989b80 1 --2- 172.21.15.131:0/435137996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca8197570 0x560ca8197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6aad989b80 1 -- 172.21.15.131:0/435137996 >> 172.21.15.131:0/435137996 conn(0x560ca80f8a60 msgr2=0x560ca8190fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6aad989b80 1 -- 172.21.15.131:0/435137996 shutdown_connections 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6aad989b80 1 -- 172.21.15.131:0/435137996 wait complete. 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.536+0000 7f6aad989b80 1 -- start start 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6aad989b80 1 Processor -- start 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 0x560ca81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197570 0x560ca81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197ec0 0x560ca81a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6aad989b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca8052560 con 0x560ca8197ec0 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6aad989b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca8052200 con 0x560ca8197570 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6aad989b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca7f396a0 con 0x560ca81966a0 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6aad989b80 10 client.0 _ll_register_callbacks cb 0x560ca805bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6a8ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 0x560ca81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6a8ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 0x560ca81a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41384/0 (socket says 172.21.15.131:41384) 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6a8ea00700 1 -- 172.21.15.131:0/3433308928 learned_addr learned my addr 172.21.15.131:0/3433308928 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6a8e000700 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197570 0x560ca81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.537+0000 7f6a8f400700 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197ec0 0x560ca81a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6a8ea00700 1 -- 172.21.15.131:0/3433308928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197570 msgr2=0x560ca81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6a8ea00700 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197570 0x560ca81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6a8ea00700 1 -- 172.21.15.131:0/3433308928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197ec0 msgr2=0x560ca81a7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6a8ea00700 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197ec0 0x560ca81a7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6a8ea00700 1 -- 172.21.15.131:0/3433308928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c009920 con 0x560ca81966a0 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6a8ea00700 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 0x560ca81a0680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000fb70 tx=0x7f6a8000fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a048f7c5abf926c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6a78c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6a78c00700 1 -- 172.21.15.131:0/3433308928 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8009a850 con 0x560ca81966a0 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6a78c00700 1 -- 172.21.15.131:0/3433308928 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6a80021380 con 0x560ca81966a0 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6aad989b80 1 -- 172.21.15.131:0/3433308928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ca800f270 con 0x560ca81966a0 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6a78c00700 1 -- 172.21.15.131:0/3433308928 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8009a670 con 0x560ca81966a0 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6aad989b80 10 client.25570 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.538+0000 7f6aad989b80 1 -- 172.21.15.131:0/3433308928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560ca7ee3b60 con 0x560ca81966a0 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.539+0000 7f6a78c00700 1 -- 172.21.15.131:0/3433308928 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ca800f270 con 0x560ca81966a0 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.539+0000 7f6a78c00700 1 -- 172.21.15.131:0/3433308928 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6a8009bd20 con 0x560ca81966a0 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.539+0000 7f6aad989b80 10 client.25570 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.539+0000 7f6aad989b80 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:09.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.539+0000 7f6aad989b80 20 client.25570 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.539+0000 7f6aad989b80 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560ca81abf90 0x560ca81ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.539+0000 7f6aad989b80 4 client.25570 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.539+0000 7f6aad989b80 1 -- 172.21.15.131:0/3433308928 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ca8165780 con 0x560ca81abf90 2024-05-10T05:34:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.539+0000 7f6a8e000700 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560ca81abf90 0x560ca81ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.540+0000 7f6a78c00700 10 client.25570 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.540+0000 7f6a8e000700 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560ca81abf90 0x560ca81ae3e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=719ba00af9e3a0f5 server_cookie=68b61e73763b9b8a in_seq=0 out_seq=0 2024-05-10T05:34:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6a78c00700 1 -- 172.21.15.131:0/3433308928 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560ca8165780 con 0x560ca81abf90 2024-05-10T05:34:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6a78c00700 10 client.25570 handle_command_reply: tid=0 2024-05-10T05:34:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 client.25570 shutdown 2024-05-10T05:34:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 -- 172.21.15.131:0/3433308928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 msgr2=0x560ca81a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 0x560ca81a0680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000fb70 tx=0x7f6a8000fa40 comp rx=0 tx=0).stop 2024-05-10T05:34:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 -- 172.21.15.131:0/3433308928 shutdown_connections 2024-05-10T05:34:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560ca81abf90 0x560ca81ae3e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197ec0 0x560ca81a7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 0x560ca81a0680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 --2- 172.21.15.131:0/3433308928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197570 0x560ca81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6a78c00700 0 client.25570 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 -- 172.21.15.131:0/3433308928 >> 172.21.15.131:0/3433308928 conn(0x560ca80f8a60 msgr2=0x560ca8190fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 -- 172.21.15.131:0/3433308928 shutdown_connections 2024-05-10T05:34:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 1 -- 172.21.15.131:0/3433308928 wait complete. 2024-05-10T05:34:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.541+0000 7f6aad989b80 20 client.25570 trim_cache size 0 max 16384 2024-05-10T05:34:09.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6aad989b80 1 Processor -- start 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6aad989b80 1 -- start start 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 0x560ca819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197570 0x560ca81a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 0x560ca819e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6aad989b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca818b7c0 con 0x560ca8197570 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6aad989b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca81973c0 con 0x560ca8197ec0 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6aad989b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca81971a0 con 0x560ca81966a0 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6a8e000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197570 0x560ca81a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6a8ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 0x560ca819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6a8e000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197570 0x560ca81a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44810/0 (socket says 172.21.15.131:44810) 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6a8e000700 1 -- 172.21.15.131:0/2583191351 learned_addr learned my addr 172.21.15.131:0/2583191351 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:09.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.559+0000 7f6a8f400700 1 --2- 172.21.15.131:0/2583191351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 0x560ca819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.560+0000 7f6a8e000700 1 -- 172.21.15.131:0/2583191351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 msgr2=0x560ca819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.560+0000 7f6a8e000700 1 --2- 172.21.15.131:0/2583191351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 0x560ca819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.560+0000 7f6a8e000700 1 -- 172.21.15.131:0/2583191351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 msgr2=0x560ca819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.560+0000 7f6a8e000700 1 --2- 172.21.15.131:0/2583191351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 0x560ca819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.560+0000 7f6a8e000700 1 -- 172.21.15.131:0/2583191351 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ca807c560 con 0x560ca8197570 2024-05-10T05:34:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.560+0000 7f6a8f400700 1 --2- 172.21.15.131:0/2583191351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 0x560ca819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.560+0000 7f6a8e000700 1 --2- 172.21.15.131:0/2583191351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197570 0x560ca81a7ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c005fd0 tx=0x7f6a7c013fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b11aa045d1ddd68 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.560+0000 7f6a84c00700 1 -- 172.21.15.131:0/2583191351 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c003e30 con 0x560ca8197570 2024-05-10T05:34:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.560+0000 7f6a84c00700 1 -- 172.21.15.131:0/2583191351 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a7c003b60 con 0x560ca8197570 2024-05-10T05:34:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.560+0000 7f6a84c00700 1 -- 172.21.15.131:0/2583191351 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c01a6a0 con 0x560ca8197570 2024-05-10T05:34:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.561+0000 7f6aad989b80 1 -- 172.21.15.131:0/2583191351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197570 msgr2=0x560ca81a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.561+0000 7f6aad989b80 1 --2- 172.21.15.131:0/2583191351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197570 0x560ca81a7ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c005fd0 tx=0x7f6a7c013fb0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.561+0000 7f6aad989b80 1 -- 172.21.15.131:0/2583191351 shutdown_connections 2024-05-10T05:34:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.561+0000 7f6aad989b80 1 --2- 172.21.15.131:0/2583191351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca8197ec0 0x560ca819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.561+0000 7f6aad989b80 1 --2- 172.21.15.131:0/2583191351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca81966a0 0x560ca819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.561+0000 7f6aad989b80 1 --2- 172.21.15.131:0/2583191351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197570 0x560ca81a7ae0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.561+0000 7f6aad989b80 1 -- 172.21.15.131:0/2583191351 >> 172.21.15.131:0/2583191351 conn(0x560ca80f8a60 msgr2=0x560ca8190fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.561+0000 7f6aad989b80 1 -- 172.21.15.131:0/2583191351 shutdown_connections 2024-05-10T05:34:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.561+0000 7f6aad989b80 1 -- 172.21.15.131:0/2583191351 wait complete. 2024-05-10T05:34:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.561+0000 7f6aad989b80 1 -- start start 2024-05-10T05:34:09.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.562+0000 7f6aad989b80 1 Processor -- start 2024-05-10T05:34:09.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.562+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca81966a0 0x560ca81a5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.562+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca8197570 0x560ca81a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6aad989b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197ec0 0x560ca81a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6aad989b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca81a6e10 con 0x560ca8197ec0 2024-05-10T05:34:09.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6aad989b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca81a6f80 con 0x560ca81966a0 2024-05-10T05:34:09.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6aad989b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca81a7140 con 0x560ca8197570 2024-05-10T05:34:09.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6aad989b80 10 client.0 _ll_register_callbacks cb 0x560ca805bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:09.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca81966a0 0x560ca81a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca81966a0 0x560ca81a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38104/0 (socket says 172.21.15.131:38104) 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8ea00700 1 -- 172.21.15.131:0/2091848821 learned_addr learned my addr 172.21.15.131:0/2091848821 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8f400700 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197ec0 0x560ca81a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8e000700 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca8197570 0x560ca81a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8ea00700 1 -- 172.21.15.131:0/2091848821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca8197570 msgr2=0x560ca81a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8ea00700 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca8197570 0x560ca81a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8ea00700 1 -- 172.21.15.131:0/2091848821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197ec0 msgr2=0x560ca81a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8ea00700 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197ec0 0x560ca81a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8ea00700 1 -- 172.21.15.131:0/2091848821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c01e040 con 0x560ca81966a0 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a8ea00700 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca81966a0 0x560ca81a5460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a80096a90 tx=0x7f6a800cdb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1957195de528cad1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.563+0000 7f6a78c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:34:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6a78c00700 1 -- 172.21.15.131:0/2091848821 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8009a670 con 0x560ca81966a0 2024-05-10T05:34:09.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6aad989b80 1 -- 172.21.15.131:0/2091848821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a7c011e20 con 0x560ca81966a0 2024-05-10T05:34:09.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6a78c00700 1 -- 172.21.15.131:0/2091848821 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6a80021380 con 0x560ca81966a0 2024-05-10T05:34:09.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6a78c00700 1 -- 172.21.15.131:0/2091848821 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8009a850 con 0x560ca81966a0 2024-05-10T05:34:09.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6aad989b80 10 client.25562 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:09.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6aad989b80 1 -- 172.21.15.131:0/2091848821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560ca7f3bcd0 con 0x560ca81966a0 2024-05-10T05:34:09.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6a78c00700 1 -- 172.21.15.131:0/2091848821 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a800b9020 con 0x560ca81966a0 2024-05-10T05:34:09.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6a78c00700 1 -- 172.21.15.131:0/2091848821 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6a8009af70 con 0x560ca81966a0 2024-05-10T05:34:09.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6aad989b80 10 client.25562 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:09.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6aad989b80 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:09.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6aad989b80 20 client.25562 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6aad989b80 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560ca82028d0 0x560ca8204cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6aad989b80 4 client.25562 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.564+0000 7f6aad989b80 1 -- 172.21.15.131:0/2091848821 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ca80aeec0 con 0x560ca82028d0 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.565+0000 7f6a8e000700 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560ca82028d0 0x560ca8204cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.565+0000 7f6a78c00700 10 client.25562 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.565+0000 7f6a8e000700 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560ca82028d0 0x560ca8204cf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da9ad75e6d0ecc8 server_cookie=c0a13deb314e7d02 in_seq=0 out_seq=0 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.565+0000 7f6a78c00700 1 -- 172.21.15.131:0/2091848821 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560ca80aeec0 con 0x560ca82028d0 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.565+0000 7f6a78c00700 10 client.25562 handle_command_reply: tid=0 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.565+0000 7f6aad989b80 1 client.25562 shutdown 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.565+0000 7f6aad989b80 1 -- 172.21.15.131:0/2091848821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca81966a0 msgr2=0x560ca81a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.565+0000 7f6aad989b80 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca81966a0 0x560ca81a5460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a80096a90 tx=0x7f6a800cdb50 comp rx=0 tx=0).stop 2024-05-10T05:34:09.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6aad989b80 1 -- 172.21.15.131:0/2091848821 shutdown_connections 2024-05-10T05:34:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6aad989b80 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560ca82028d0 0x560ca8204cf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6aad989b80 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560ca8197ec0 0x560ca81a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6a78c00700 0 client.25562 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6aad989b80 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ca81966a0 0x560ca81a5460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6aad989b80 1 --2- 172.21.15.131:0/2091848821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ca8197570 0x560ca81a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6aad989b80 1 -- 172.21.15.131:0/2091848821 >> 172.21.15.131:0/2091848821 conn(0x560ca80f8a60 msgr2=0x560ca819e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6aad989b80 1 -- 172.21.15.131:0/2091848821 shutdown_connections 2024-05-10T05:34:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6aad989b80 1 -- 172.21.15.131:0/2091848821 wait complete. 2024-05-10T05:34:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6aad989b80 20 client.25562 trim_cache size 0 max 16384 2024-05-10T05:34:09.571 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:34:09.572 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:34:09.572 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:34:09.572 INFO:teuthology.orchestra.run.smithi131.stdout: "acf77466-d853-48ec-8b00-2564d7f47264": { 2024-05-10T05:34:09.572 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:34:09.572 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "acf77466-d853-48ec-8b00-2564d7f47264", 2024-05-10T05:34:09.572 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:34:09.572 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:34:09.572 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:34:09.572 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:34:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6a95a00700 1 -- 172.21.15.131:0/1895983205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a740a11c0 msgr2=0x7f6a740a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6a95a00700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a740a11c0 0x7f6a740a36d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x560ca8196380 tx=0x7f6a88009040 comp rx=0 tx=0).stop 2024-05-10T05:34:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6a95a00700 1 -- 172.21.15.131:0/1895983205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90101be0 msgr2=0x7f6a9019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.566+0000 7f6a95a00700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90101be0 0x7f6a9019a180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a80000c00 tx=0x7f6a8000f950 comp rx=0 tx=0).stop 2024-05-10T05:34:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.567+0000 7f6a95a00700 1 -- 172.21.15.131:0/1895983205 shutdown_connections 2024-05-10T05:34:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.567+0000 7f6a95a00700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6a740a11c0 0x7f6a740a36d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.567+0000 7f6a95a00700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a901036a0 0x7f6a901a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.567+0000 7f6a95a00700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90101be0 0x7f6a9019a180 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.567+0000 7f6a95a00700 1 --2- 172.21.15.131:0/1895983205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a90102d00 0x7f6a9019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.567+0000 7f6a95a00700 1 -- 172.21.15.131:0/1895983205 >> 172.21.15.131:0/1895983205 conn(0x7f6a900fd210 msgr2=0x7f6a90073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:09.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.567+0000 7f6a95a00700 1 -- 172.21.15.131:0/1895983205 shutdown_connections 2024-05-10T05:34:09.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:09.567+0000 7f6a95a00700 1 -- 172.21.15.131:0/1895983205 wait complete. 2024-05-10T05:34:09.594 INFO:tasks.cephfs.filesystem:scrub status for tag:acf77466-d853-48ec-8b00-2564d7f47264 - {'path': '/', 'tag': 'acf77466-d853-48ec-8b00-2564d7f47264', 'options': 'recursive,force'} 2024-05-10T05:34:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:09 smithi131 ceph-mon[37264]: pgmap v309: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:09 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:34:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:09 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:09 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:34:10.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:09 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:09 smithi157 ceph-mon[78427]: pgmap v309: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:09 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:34:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:09 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:09 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:34:10.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:09 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:09 smithi142 ceph-mon[77667]: pgmap v309: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:09 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:34:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:09 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:10.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:09 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:34:10.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:09 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:11.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:10 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:34:11.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:10 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:34:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:10 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:34:12.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:11 smithi131 ceph-mon[37264]: pgmap v310: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:12.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:11 smithi157 ceph-mon[78427]: pgmap v310: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:12.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:11 smithi142 ceph-mon[77667]: pgmap v310: 97 pgs: 97 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:14.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:13 smithi131 ceph-mon[37264]: pgmap v311: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-10T05:34:14.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:13 smithi157 ceph-mon[78427]: pgmap v311: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-10T05:34:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:13 smithi142 ceph-mon[77667]: pgmap v311: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-10T05:34:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:15 smithi157 ceph-mon[78427]: pgmap v312: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T05:34:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:15 smithi142 ceph-mon[77667]: pgmap v312: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T05:34:16.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:15 smithi131 ceph-mon[37264]: pgmap v312: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T05:34:18.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:17 smithi157 ceph-mon[78427]: pgmap v313: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T05:34:18.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:17 smithi142 ceph-mon[77667]: pgmap v313: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T05:34:18.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:17 smithi131 ceph-mon[37264]: pgmap v313: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T05:34:20.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:19 smithi157 ceph-mon[78427]: pgmap v314: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T05:34:20.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:19 smithi142 ceph-mon[77667]: pgmap v314: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T05:34:20.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:19 smithi131 ceph-mon[37264]: pgmap v314: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T05:34:22.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:21 smithi157 ceph-mon[78427]: pgmap v315: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-10T05:34:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:21 smithi142 ceph-mon[77667]: pgmap v315: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-10T05:34:22.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:21 smithi131 ceph-mon[37264]: pgmap v315: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-10T05:34:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:23 smithi157 ceph-mon[78427]: pgmap v316: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T05:34:24.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:23 smithi142 ceph-mon[77667]: pgmap v316: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T05:34:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:23 smithi131 ceph-mon[37264]: pgmap v316: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T05:34:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:25 smithi157 ceph-mon[78427]: pgmap v317: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T05:34:26.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:25 smithi142 ceph-mon[77667]: pgmap v317: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T05:34:26.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:25 smithi131 ceph-mon[37264]: pgmap v317: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T05:34:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:27 smithi157 ceph-mon[78427]: pgmap v318: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:27 smithi142 ceph-mon[77667]: pgmap v318: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:28.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:27 smithi131 ceph-mon[37264]: pgmap v318: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:30.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:29 smithi157 ceph-mon[78427]: pgmap v319: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:30.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:29 smithi142 ceph-mon[77667]: pgmap v319: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:30.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:29 smithi131 ceph-mon[37264]: pgmap v319: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:31 smithi157 ceph-mon[78427]: pgmap v320: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:31 smithi142 ceph-mon[77667]: pgmap v320: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:32.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:31 smithi131 ceph-mon[37264]: pgmap v320: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:34.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:33 smithi157 ceph-mon[78427]: pgmap v321: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:33 smithi142 ceph-mon[77667]: pgmap v321: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:34.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:33 smithi131 ceph-mon[37264]: pgmap v321: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:36.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:35 smithi157 ceph-mon[78427]: pgmap v322: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:36.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:35 smithi142 ceph-mon[77667]: pgmap v322: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:36.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:35 smithi131 ceph-mon[37264]: pgmap v322: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:37 smithi157 ceph-mon[78427]: pgmap v323: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:38.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:37 smithi142 ceph-mon[77667]: pgmap v323: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:38.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:37 smithi131 ceph-mon[37264]: pgmap v323: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:39.595 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:34:39.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.888+0000 7fd343000700 1 -- 172.21.15.131:0/3858783458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd33c103e00 msgr2=0x7fd33c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:39.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.888+0000 7fd343000700 1 --2- 172.21.15.131:0/3858783458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd33c103e00 0x7fd33c104210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0099c0 tx=0x7fd32c009cd0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.888+0000 7fd343000700 1 -- 172.21.15.131:0/3858783458 shutdown_connections 2024-05-10T05:34:39.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.888+0000 7fd343000700 1 --2- 172.21.15.131:0/3858783458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd33c1058c0 0x7fd33c10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.888+0000 7fd343000700 1 --2- 172.21.15.131:0/3858783458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd33c103e00 0x7fd33c104210 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.888+0000 7fd343000700 1 --2- 172.21.15.131:0/3858783458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd33c104f20 0x7fd33c105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.888+0000 7fd343000700 1 -- 172.21.15.131:0/3858783458 >> 172.21.15.131:0/3858783458 conn(0x7fd33c0ff410 msgr2=0x7fd33c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:39.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.888+0000 7fd343000700 1 -- 172.21.15.131:0/3858783458 shutdown_connections 2024-05-10T05:34:39.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.888+0000 7fd343000700 1 -- 172.21.15.131:0/3858783458 wait complete. 2024-05-10T05:34:39.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.890+0000 7fd343000700 1 Processor -- start 2024-05-10T05:34:39.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.890+0000 7fd343000700 1 -- start start 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd343000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd33c103e00 0x7fd33c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd343000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd33c104f20 0x7fd33c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd343000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd33c1058c0 0x7fd33c1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd343000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c1a1c30 con 0x7fd33c1058c0 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd343000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c1a1da0 con 0x7fd33c104f20 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd343000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c1a1f60 con 0x7fd33c103e00 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd33be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd33c103e00 0x7fd33c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd33be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd33c103e00 0x7fd33c19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56790/0 (socket says 172.21.15.131:56790) 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd33be00700 1 -- 172.21.15.131:0/1661545073 learned_addr learned my addr 172.21.15.131:0/1661545073 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd33b400700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd33c104f20 0x7fd33c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.891+0000 7fd341000700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd33c1058c0 0x7fd33c1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.892+0000 7fd33be00700 1 -- 172.21.15.131:0/1661545073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd33c104f20 msgr2=0x7fd33c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:39.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.892+0000 7fd33be00700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd33c104f20 0x7fd33c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.892+0000 7fd33be00700 1 -- 172.21.15.131:0/1661545073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd33c1058c0 msgr2=0x7fd33c1a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:39.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.892+0000 7fd33be00700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd33c1058c0 0x7fd33c1a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.892+0000 7fd33be00700 1 -- 172.21.15.131:0/1661545073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c0095f0 con 0x7fd33c103e00 2024-05-10T05:34:39.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.892+0000 7fd33be00700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd33c103e00 0x7fd33c19a150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009990 tx=0x7fd32c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:39.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.892+0000 7fd338c00700 1 -- 172.21.15.131:0/1661545073 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c01d070 con 0x7fd33c103e00 2024-05-10T05:34:39.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.892+0000 7fd338c00700 1 -- 172.21.15.131:0/1661545073 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd32c0054f0 con 0x7fd33c103e00 2024-05-10T05:34:39.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.892+0000 7fd338c00700 1 -- 172.21.15.131:0/1661545073 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c004a90 con 0x7fd33c103e00 2024-05-10T05:34:39.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.892+0000 7fd343000700 1 -- 172.21.15.131:0/1661545073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c1a21e0 con 0x7fd33c103e00 2024-05-10T05:34:39.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.893+0000 7fd343000700 1 -- 172.21.15.131:0/1661545073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c1a25b0 con 0x7fd33c103e00 2024-05-10T05:34:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.897+0000 7fd35af3db80 1 Processor -- start 2024-05-10T05:34:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.897+0000 7fd338c00700 1 -- 172.21.15.131:0/1661545073 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd32c00fb80 con 0x7fd33c103e00 2024-05-10T05:34:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.897+0000 7fd35af3db80 1 -- start start 2024-05-10T05:34:39.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.898+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 0x559de79fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.898+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 0x559de79fc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.898+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fcec0 0x559de7a037c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.898+0000 7fd35af3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de7922150 con 0x559de79fcec0 2024-05-10T05:34:39.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.898+0000 7fd35af3db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de791fb10 con 0x559de79fc570 2024-05-10T05:34:39.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.898+0000 7fd35af3db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de791b400 con 0x559de79fb6a0 2024-05-10T05:34:39.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.899+0000 7fd341000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 0x559de79fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.899+0000 7fd341000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 0x559de79fbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56798/0 (socket says 172.21.15.131:56798) 2024-05-10T05:34:39.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.899+0000 7fd341000700 1 -- 172.21.15.131:0/806072391 learned_addr learned my addr 172.21.15.131:0/806072391 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:39.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.899+0000 7fd33b400700 1 --2- 172.21.15.131:0/806072391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fcec0 0x559de7a037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.899+0000 7fd341000700 1 -- 172.21.15.131:0/806072391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 msgr2=0x559de79fc980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:34:39.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.899+0000 7fd341000700 1 --2- 172.21.15.131:0/806072391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 0x559de79fc980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.899+0000 7fd341000700 1 -- 172.21.15.131:0/806072391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fcec0 msgr2=0x559de7a037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:39.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.899+0000 7fd341000700 1 --2- 172.21.15.131:0/806072391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fcec0 0x559de7a037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.899+0000 7fd341000700 1 -- 172.21.15.131:0/806072391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559de779c3b0 con 0x559de79fb6a0 2024-05-10T05:34:39.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.900+0000 7fd341000700 1 --2- 172.21.15.131:0/806072391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 0x559de79fbaf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd334009370 tx=0x7fd334009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67f8b483d2452b67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:39.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.900+0000 7fd338c00700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd3200a12c0 0x7fd3200a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.900+0000 7fd331600700 1 -- 172.21.15.131:0/806072391 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400e030 con 0x559de79fb6a0 2024-05-10T05:34:39.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.900+0000 7fd331600700 1 -- 172.21.15.131:0/806072391 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33400f040 con 0x559de79fb6a0 2024-05-10T05:34:39.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.900+0000 7fd331600700 1 -- 172.21.15.131:0/806072391 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334015cb0 con 0x559de79fb6a0 2024-05-10T05:34:39.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.900+0000 7fd33be00700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd3200a12c0 0x7fd3200a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.900+0000 7fd338c00700 1 -- 172.21.15.131:0/1661545073 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd32c0cd5c0 con 0x7fd33c103e00 2024-05-10T05:34:39.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.901+0000 7fd35af3db80 1 -- 172.21.15.131:0/806072391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 msgr2=0x559de79fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:39.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.901+0000 7fd35af3db80 1 --2- 172.21.15.131:0/806072391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 0x559de79fbaf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd334009370 tx=0x7fd334009920 comp rx=0 tx=0).stop 2024-05-10T05:34:39.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.901+0000 7fd33be00700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd3200a12c0 0x7fd3200a37d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd32c000c00 tx=0x7fd32c0901e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:39.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.901+0000 7fd35af3db80 1 -- 172.21.15.131:0/806072391 shutdown_connections 2024-05-10T05:34:39.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.901+0000 7fd35af3db80 1 --2- 172.21.15.131:0/806072391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fcec0 0x559de7a037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.901+0000 7fd35af3db80 1 --2- 172.21.15.131:0/806072391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 0x559de79fbaf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.901+0000 7fd35af3db80 1 --2- 172.21.15.131:0/806072391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 0x559de79fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.901+0000 7fd35af3db80 1 -- 172.21.15.131:0/806072391 >> 172.21.15.131:0/806072391 conn(0x559de795da60 msgr2=0x559de79f5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:39.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.902+0000 7fd35af3db80 1 -- 172.21.15.131:0/806072391 shutdown_connections 2024-05-10T05:34:39.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.902+0000 7fd35af3db80 1 -- 172.21.15.131:0/806072391 wait complete. 2024-05-10T05:34:39.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.902+0000 7fd35af3db80 1 -- start start 2024-05-10T05:34:39.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.902+0000 7fd35af3db80 1 Processor -- start 2024-05-10T05:34:39.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.903+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 0x559de7a0def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.903+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fc570 0x559de7a0e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.903+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fcec0 0x559de7a15390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.903+0000 7fd35af3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de78b7560 con 0x559de79fb6a0 2024-05-10T05:34:39.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.903+0000 7fd35af3db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de78b7200 con 0x559de79fcec0 2024-05-10T05:34:39.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.903+0000 7fd35af3db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de779e6a0 con 0x559de79fc570 2024-05-10T05:34:39.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.903+0000 7fd35af3db80 10 client.0 _ll_register_callbacks cb 0x559de78c0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:39.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.903+0000 7fd341000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 0x559de7a0def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.903+0000 7fd341000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 0x559de7a0def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59688/0 (socket says 172.21.15.131:59688) 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.903+0000 7fd341000700 1 -- 172.21.15.131:0/2534983907 learned_addr learned my addr 172.21.15.131:0/2534983907 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.904+0000 7fd341000700 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fcec0 0x559de7a15390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.904+0000 7fd33b400700 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fc570 0x559de7a0e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.904+0000 7fd341000700 1 -- 172.21.15.131:0/2534983907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fc570 msgr2=0x559de7a0e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.904+0000 7fd341000700 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fc570 0x559de7a0e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.904+0000 7fd341000700 1 -- 172.21.15.131:0/2534983907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fcec0 msgr2=0x559de7a15390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.904+0000 7fd341000700 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fcec0 0x559de7a15390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.904+0000 7fd341000700 1 -- 172.21.15.131:0/2534983907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd334009050 con 0x559de79fb6a0 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.904+0000 7fd33b400700 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fc570 0x559de7a0e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.904+0000 7fd341000700 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 0x559de7a0def0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd334004f80 tx=0x7fd334015530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50e7ae0df13c103 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:39.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.904+0000 7fd326000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:39.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.905+0000 7fd326000700 1 -- 172.21.15.131:0/2534983907 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400e070 con 0x559de79fb6a0 2024-05-10T05:34:39.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.905+0000 7fd326000700 1 -- 172.21.15.131:0/2534983907 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd33402a030 con 0x559de79fb6a0 2024-05-10T05:34:39.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.905+0000 7fd35af3db80 1 -- 172.21.15.131:0/2534983907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559de7874270 con 0x559de79fb6a0 2024-05-10T05:34:39.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.905+0000 7fd326000700 1 -- 172.21.15.131:0/2534983907 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd334014a00 con 0x559de79fb6a0 2024-05-10T05:34:39.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.905+0000 7fd35af3db80 10 client.15567 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:39.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.905+0000 7fd35af3db80 1 -- 172.21.15.131:0/2534983907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559de7748b60 con 0x559de79fb6a0 2024-05-10T05:34:39.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.905+0000 7fd326000700 1 -- 172.21.15.131:0/2534983907 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559de7874270 con 0x559de79fb6a0 2024-05-10T05:34:39.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.906+0000 7fd326000700 1 -- 172.21.15.131:0/2534983907 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd33402fb10 con 0x559de79fb6a0 2024-05-10T05:34:39.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.906+0000 7fd35af3db80 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:39.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.906+0000 7fd35af3db80 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:39.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.906+0000 7fd35af3db80 20 client.15567 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:39.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.906+0000 7fd35af3db80 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559de7a19a10 0x559de7a1be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.906+0000 7fd35af3db80 4 client.15567 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:39.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.906+0000 7fd35af3db80 1 -- 172.21.15.131:0/2534983907 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559de79ca780 con 0x559de7a19a10 2024-05-10T05:34:39.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.906+0000 7fd33b400700 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559de7a19a10 0x559de7a1be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.907+0000 7fd33b400700 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559de7a19a10 0x559de7a1be60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb23d905506e0dc6 server_cookie=d6b87ea57a26b914 in_seq=0 out_seq=0 2024-05-10T05:34:39.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.907+0000 7fd326000700 10 client.15567 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:39.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd326000700 1 -- 172.21.15.131:0/2534983907 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559de79ca780 con 0x559de7a19a10 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd326000700 10 client.15567 handle_command_reply: tid=0 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd35af3db80 1 client.15567 shutdown 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd35af3db80 1 -- 172.21.15.131:0/2534983907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 msgr2=0x559de7a0def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd35af3db80 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 0x559de7a0def0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd334004f80 tx=0x7fd334015530 comp rx=0 tx=0).stop 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd35af3db80 1 -- 172.21.15.131:0/2534983907 shutdown_connections 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd35af3db80 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559de7a19a10 0x559de7a1be60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd35af3db80 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 0x559de7a0def0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd35af3db80 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fcec0 0x559de7a15390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd35af3db80 1 --2- 172.21.15.131:0/2534983907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fc570 0x559de7a0e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd35af3db80 1 -- 172.21.15.131:0/2534983907 >> 172.21.15.131:0/2534983907 conn(0x559de795da60 msgr2=0x559de79f5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd326000700 0 client.15567 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.908+0000 7fd35af3db80 1 -- 172.21.15.131:0/2534983907 shutdown_connections 2024-05-10T05:34:39.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.909+0000 7fd35af3db80 1 -- 172.21.15.131:0/2534983907 wait complete. 2024-05-10T05:34:39.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.909+0000 7fd35af3db80 20 client.15567 trim_cache size 0 max 16384 2024-05-10T05:34:39.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.929+0000 7fd35af3db80 1 Processor -- start 2024-05-10T05:34:39.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.929+0000 7fd35af3db80 1 -- start start 2024-05-10T05:34:39.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.929+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 0x559de7a0d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.929+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 0x559de7a15350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.929+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fcec0 0x559de79fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.929+0000 7fd35af3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de79fd2d0 con 0x559de79fb6a0 2024-05-10T05:34:39.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.929+0000 7fd35af3db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de7a0ef90 con 0x559de79fc570 2024-05-10T05:34:39.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.929+0000 7fd35af3db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de7a19810 con 0x559de79fcec0 2024-05-10T05:34:39.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.930+0000 7fd341000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 0x559de7a0d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.930+0000 7fd33b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 0x559de7a15350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.930+0000 7fd33b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 0x559de7a15350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46590/0 (socket says 172.21.15.131:46590) 2024-05-10T05:34:39.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.930+0000 7fd33b400700 1 -- 172.21.15.131:0/898497692 learned_addr learned my addr 172.21.15.131:0/898497692 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:39.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.930+0000 7fd33b400700 1 -- 172.21.15.131:0/898497692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fcec0 msgr2=0x559de79fbd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:34:39.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.930+0000 7fd33b400700 1 --2- 172.21.15.131:0/898497692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fcec0 0x559de79fbd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.930+0000 7fd33b400700 1 -- 172.21.15.131:0/898497692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 msgr2=0x559de7a0d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:39.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.930+0000 7fd33b400700 1 --2- 172.21.15.131:0/898497692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 0x559de7a0d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.930+0000 7fd33b400700 1 -- 172.21.15.131:0/898497692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559de78e1560 con 0x559de79fc570 2024-05-10T05:34:39.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.930+0000 7fd33b400700 1 --2- 172.21.15.131:0/898497692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 0x559de7a15350 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x559de7a1c700 tx=0x7fd328009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8ff03ff58eeea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:39.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.931+0000 7fd332000700 1 -- 172.21.15.131:0/898497692 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328016070 con 0x559de79fc570 2024-05-10T05:34:39.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.931+0000 7fd332000700 1 -- 172.21.15.131:0/898497692 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd328017040 con 0x559de79fc570 2024-05-10T05:34:39.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.931+0000 7fd332000700 1 -- 172.21.15.131:0/898497692 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328004e90 con 0x559de79fc570 2024-05-10T05:34:39.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.932+0000 7fd35af3db80 1 -- 172.21.15.131:0/898497692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 msgr2=0x559de7a15350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:39.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.932+0000 7fd35af3db80 1 --2- 172.21.15.131:0/898497692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 0x559de7a15350 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x559de7a1c700 tx=0x7fd328009260 comp rx=0 tx=0).stop 2024-05-10T05:34:39.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.932+0000 7fd35af3db80 1 -- 172.21.15.131:0/898497692 shutdown_connections 2024-05-10T05:34:39.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.932+0000 7fd35af3db80 1 --2- 172.21.15.131:0/898497692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fb6a0 0x559de7a0d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.932+0000 7fd35af3db80 1 --2- 172.21.15.131:0/898497692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fcec0 0x559de79fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.932+0000 7fd35af3db80 1 --2- 172.21.15.131:0/898497692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fc570 0x559de7a15350 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.932+0000 7fd35af3db80 1 -- 172.21.15.131:0/898497692 >> 172.21.15.131:0/898497692 conn(0x559de795da60 msgr2=0x559de79f2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:39.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.932+0000 7fd35af3db80 1 -- 172.21.15.131:0/898497692 shutdown_connections 2024-05-10T05:34:39.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.932+0000 7fd35af3db80 1 -- 172.21.15.131:0/898497692 wait complete. 2024-05-10T05:34:39.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.932+0000 7fd35af3db80 1 -- start start 2024-05-10T05:34:39.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.933+0000 7fd35af3db80 1 Processor -- start 2024-05-10T05:34:39.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.933+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 0x559de7a120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.933+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fc570 0x559de7a12630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.933+0000 7fd35af3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fcec0 0x559de7a6deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd35af3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de7a15570 con 0x559de79fc570 2024-05-10T05:34:39.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd35af3db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de7a156e0 con 0x559de79fcec0 2024-05-10T05:34:39.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd35af3db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de7a6e3f0 con 0x559de79fb6a0 2024-05-10T05:34:39.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd35af3db80 10 client.0 _ll_register_callbacks cb 0x559de78c0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:39.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd33b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fc570 0x559de7a12630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd33b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fc570 0x559de7a12630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59700/0 (socket says 172.21.15.131:59700) 2024-05-10T05:34:39.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd33b400700 1 -- 172.21.15.131:0/546976613 learned_addr learned my addr 172.21.15.131:0/546976613 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:39.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd341000700 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 0x559de7a120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd341000700 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fcec0 0x559de7a6deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd33b400700 1 -- 172.21.15.131:0/546976613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 msgr2=0x559de7a120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:39.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd33b400700 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 0x559de7a120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd33b400700 1 -- 172.21.15.131:0/546976613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fcec0 msgr2=0x559de7a6deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:39.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd33b400700 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fcec0 0x559de7a6deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.934+0000 7fd33b400700 1 -- 172.21.15.131:0/546976613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd328009eb0 con 0x559de79fc570 2024-05-10T05:34:39.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.935+0000 7fd33b400700 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fc570 0x559de7a12630 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd328004e20 tx=0x7fd328004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fe3bef7c2c416a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:39.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.935+0000 7fd326000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:39.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.935+0000 7fd326000700 1 -- 172.21.15.131:0/546976613 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328016040 con 0x559de79fc570 2024-05-10T05:34:39.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.935+0000 7fd326000700 1 -- 172.21.15.131:0/546976613 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd328003840 con 0x559de79fc570 2024-05-10T05:34:39.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.935+0000 7fd35af3db80 1 -- 172.21.15.131:0/546976613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd33400f040 con 0x559de79fc570 2024-05-10T05:34:39.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd326000700 1 -- 172.21.15.131:0/546976613 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd328003a00 con 0x559de79fc570 2024-05-10T05:34:39.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd35af3db80 10 client.15573 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:39.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd35af3db80 1 -- 172.21.15.131:0/546976613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559de77a0cd0 con 0x559de79fc570 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd326000700 1 -- 172.21.15.131:0/546976613 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33400f040 con 0x559de79fc570 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd326000700 1 -- 172.21.15.131:0/546976613 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd328010730 con 0x559de79fc570 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd35af3db80 10 client.15573 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd35af3db80 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd35af3db80 20 client.15573 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd35af3db80 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559de7a6fcf0 0x559de7a72110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd35af3db80 4 client.15573 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.936+0000 7fd35af3db80 1 -- 172.21.15.131:0/546976613 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559de7913ec0 con 0x559de7a6fcf0 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.937+0000 7fd341000700 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559de7a6fcf0 0x559de7a72110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.937+0000 7fd326000700 10 client.15573 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.937+0000 7fd341000700 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559de7a6fcf0 0x559de7a72110 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61466be727cecb5 server_cookie=559de93e0e018457 in_seq=0 out_seq=0 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.937+0000 7fd326000700 1 -- 172.21.15.131:0/546976613 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559de7913ec0 con 0x559de7a6fcf0 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.937+0000 7fd326000700 10 client.15573 handle_command_reply: tid=0 2024-05-10T05:34:39.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.937+0000 7fd35af3db80 1 client.15573 shutdown 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 1 -- 172.21.15.131:0/546976613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fc570 msgr2=0x559de7a12630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fc570 0x559de7a12630 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd328004e20 tx=0x7fd328004e50 comp rx=0 tx=0).stop 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 1 -- 172.21.15.131:0/546976613 shutdown_connections 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559de7a6fcf0 0x559de7a72110 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559de79fcec0 0x559de7a6deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559de79fb6a0 0x559de7a120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd326000700 0 client.15573 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 1 --2- 172.21.15.131:0/546976613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559de79fc570 0x559de7a12630 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 1 -- 172.21.15.131:0/546976613 >> 172.21.15.131:0/546976613 conn(0x559de795da60 msgr2=0x559de79f2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 1 -- 172.21.15.131:0/546976613 shutdown_connections 2024-05-10T05:34:39.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 1 -- 172.21.15.131:0/546976613 wait complete. 2024-05-10T05:34:39.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.938+0000 7fd35af3db80 20 client.15573 trim_cache size 0 max 16384 2024-05-10T05:34:39.941 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:34:39.941 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:34:39.941 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:34:39.941 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:34:39.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 -- 172.21.15.131:0/1661545073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd3200a12c0 msgr2=0x7fd3200a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:39.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd3200a12c0 0x7fd3200a37d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd32c000c00 tx=0x7fd32c0901e0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 -- 172.21.15.131:0/1661545073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd33c103e00 msgr2=0x7fd33c19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:39.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd33c103e00 0x7fd33c19a150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009990 tx=0x7fd32c00f690 comp rx=0 tx=0).stop 2024-05-10T05:34:39.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 -- 172.21.15.131:0/1661545073 shutdown_connections 2024-05-10T05:34:39.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd3200a12c0 0x7fd3200a37d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd33c1058c0 0x7fd33c1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd33c103e00 0x7fd33c19a150 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 --2- 172.21.15.131:0/1661545073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd33c104f20 0x7fd33c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:39.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 -- 172.21.15.131:0/1661545073 >> 172.21.15.131:0/1661545073 conn(0x7fd33c0ff410 msgr2=0x7fd33c108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:39.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.939+0000 7fd343000700 1 -- 172.21.15.131:0/1661545073 shutdown_connections 2024-05-10T05:34:39.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:39.940+0000 7fd343000700 1 -- 172.21.15.131:0/1661545073 wait complete. 2024-05-10T05:34:39.968 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:34:39.968 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:34:40.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.265+0000 7fb79dc00700 1 -- 172.21.15.131:0/2672356691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798073ae0 msgr2=0x7fb798073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.265+0000 7fb79dc00700 1 --2- 172.21.15.131:0/2672356691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798073ae0 0x7fb798073f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb7840098f0 tx=0x7fb784009c00 comp rx=0 tx=0).stop 2024-05-10T05:34:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.265+0000 7fb79dc00700 1 -- 172.21.15.131:0/2672356691 shutdown_connections 2024-05-10T05:34:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.265+0000 7fb79dc00700 1 --2- 172.21.15.131:0/2672356691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7980750e0 0x7fb7980735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.265+0000 7fb79dc00700 1 --2- 172.21.15.131:0/2672356691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798109900 0x7fb79810bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.265+0000 7fb79dc00700 1 --2- 172.21.15.131:0/2672356691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798073ae0 0x7fb798073f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.265+0000 7fb79dc00700 1 -- 172.21.15.131:0/2672356691 >> 172.21.15.131:0/2672356691 conn(0x7fb7980fd010 msgr2=0x7fb7980ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.266+0000 7fb79dc00700 1 -- 172.21.15.131:0/2672356691 shutdown_connections 2024-05-10T05:34:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.266+0000 7fb79dc00700 1 -- 172.21.15.131:0/2672356691 wait complete. 2024-05-10T05:34:40.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.267+0000 7fb79dc00700 1 Processor -- start 2024-05-10T05:34:40.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.267+0000 7fb79dc00700 1 -- start start 2024-05-10T05:34:40.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.267+0000 7fb79dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798073ae0 0x7fb798199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.267+0000 7fb79dc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7980750e0 0x7fb79819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.267+0000 7fb79dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798109900 0x7fb7981a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.267+0000 7fb79dc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7981a1a30 con 0x7fb7980750e0 2024-05-10T05:34:40.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.267+0000 7fb79dc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7981a1b70 con 0x7fb798073ae0 2024-05-10T05:34:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.267+0000 7fb79dc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7981a1d00 con 0x7fb798109900 2024-05-10T05:34:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.268+0000 7fb797400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7980750e0 0x7fb79819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.268+0000 7fb797400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7980750e0 0x7fb79819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59724/0 (socket says 172.21.15.131:59724) 2024-05-10T05:34:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.268+0000 7fb797400700 1 -- 172.21.15.131:0/4272367008 learned_addr learned my addr 172.21.15.131:0/4272367008 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.268+0000 7fb79d200700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798109900 0x7fb7981a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.268+0000 7fb797e00700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798073ae0 0x7fb798199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.268+0000 7fb79d200700 1 -- 172.21.15.131:0/4272367008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798073ae0 msgr2=0x7fb798199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.268+0000 7fb79d200700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798073ae0 0x7fb798199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.268+0000 7fb79d200700 1 -- 172.21.15.131:0/4272367008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7980750e0 msgr2=0x7fb79819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.268+0000 7fb79d200700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7980750e0 0x7fb79819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.268+0000 7fb79d200700 1 -- 172.21.15.131:0/4272367008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb784009520 con 0x7fb798109900 2024-05-10T05:34:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.269+0000 7fb79d200700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798109900 0x7fb7981a1460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb790007f40 tx=0x7fb79000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.269+0000 7fb794c00700 1 -- 172.21.15.131:0/4272367008 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790004db0 con 0x7fb798109900 2024-05-10T05:34:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.269+0000 7fb794c00700 1 -- 172.21.15.131:0/4272367008 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79000f040 con 0x7fb798109900 2024-05-10T05:34:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.269+0000 7fb79dc00700 1 -- 172.21.15.131:0/4272367008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7981a1fe0 con 0x7fb798109900 2024-05-10T05:34:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.269+0000 7fb794c00700 1 -- 172.21.15.131:0/4272367008 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790005a80 con 0x7fb798109900 2024-05-10T05:34:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.269+0000 7fb79dc00700 1 -- 172.21.15.131:0/4272367008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7981a2400 con 0x7fb798109900 2024-05-10T05:34:40.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.271+0000 7fb79dc00700 1 -- 172.21.15.131:0/4272367008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb798108750 con 0x7fb798109900 2024-05-10T05:34:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.272+0000 7fb794c00700 1 -- 172.21.15.131:0/4272367008 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb790005000 con 0x7fb798109900 2024-05-10T05:34:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.275+0000 7fb794c00700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb77c0a11c0 0x7fb77c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.275+0000 7fb794c00700 1 -- 172.21.15.131:0/4272367008 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb7900cc370 con 0x7fb798109900 2024-05-10T05:34:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.275+0000 7fb797e00700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb77c0a11c0 0x7fb77c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.276+0000 7fb797e00700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb77c0a11c0 0x7fb77c0a36d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb798074dd0 tx=0x7fb788009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.279+0000 7fb794c00700 1 -- 172.21.15.131:0/4272367008 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb790095aa0 con 0x7fb798109900 2024-05-10T05:34:40.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:39 smithi157 ceph-mon[78427]: pgmap v324: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:39 smithi142 ceph-mon[77667]: pgmap v324: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:39 smithi131 ceph-mon[37264]: pgmap v324: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:40.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.604+0000 7fb79dc00700 1 -- 172.21.15.131:0/4272367008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb79802ce20 con 0x7fb798109900 2024-05-10T05:34:40.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.605+0000 7fb794c00700 1 -- 172.21.15.131:0/4272367008 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fb7900990c0 con 0x7fb798109900 2024-05-10T05:34:40.606 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:34:40.606 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:33:24.915360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:34:40.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 -- 172.21.15.131:0/4272367008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb77c0a11c0 msgr2=0x7fb77c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb77c0a11c0 0x7fb77c0a36d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb798074dd0 tx=0x7fb788009310 comp rx=0 tx=0).stop 2024-05-10T05:34:40.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 -- 172.21.15.131:0/4272367008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798109900 msgr2=0x7fb7981a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798109900 0x7fb7981a1460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb790007f40 tx=0x7fb79000d3b0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 -- 172.21.15.131:0/4272367008 shutdown_connections 2024-05-10T05:34:40.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb77c0a11c0 0x7fb77c0a36d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb798109900 0x7fb7981a1460 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798073ae0 0x7fb798199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 --2- 172.21.15.131:0/4272367008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7980750e0 0x7fb79819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 -- 172.21.15.131:0/4272367008 >> 172.21.15.131:0/4272367008 conn(0x7fb7980fd010 msgr2=0x7fb798105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:40.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.608+0000 7fb79dc00700 1 -- 172.21.15.131:0/4272367008 shutdown_connections 2024-05-10T05:34:40.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.609+0000 7fb79dc00700 1 -- 172.21.15.131:0/4272367008 wait complete. 2024-05-10T05:34:40.610 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-05-10T05:34:40.638 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:34:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.936+0000 7faf37e00700 1 -- 172.21.15.131:0/50112354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf30069050 msgr2=0x7faf30104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.936+0000 7faf37e00700 1 --2- 172.21.15.131:0/50112354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf30069050 0x7faf30104760 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faf24009990 tx=0x7faf24009ca0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.937+0000 7faf37e00700 1 -- 172.21.15.131:0/50112354 shutdown_connections 2024-05-10T05:34:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.937+0000 7faf37e00700 1 --2- 172.21.15.131:0/50112354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf301078a0 0x7faf30109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.937+0000 7faf37e00700 1 --2- 172.21.15.131:0/50112354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf30069050 0x7faf30104760 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.937+0000 7faf37e00700 1 --2- 172.21.15.131:0/50112354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf30104d30 0x7faf30107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.937+0000 7faf37e00700 1 -- 172.21.15.131:0/50112354 >> 172.21.15.131:0/50112354 conn(0x7faf300fc0d0 msgr2=0x7faf300fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.937+0000 7faf37e00700 1 -- 172.21.15.131:0/50112354 shutdown_connections 2024-05-10T05:34:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.937+0000 7faf37e00700 1 -- 172.21.15.131:0/50112354 wait complete. 2024-05-10T05:34:40.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.938+0000 7faf37e00700 1 Processor -- start 2024-05-10T05:34:40.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.938+0000 7faf37e00700 1 -- start start 2024-05-10T05:34:40.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.938+0000 7faf37e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf30069050 0x7faf3019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.939+0000 7faf37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf30104d30 0x7faf3019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.939+0000 7faf37e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf301078a0 0x7faf301a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.939+0000 7faf37e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf301a1bf0 con 0x7faf30069050 2024-05-10T05:34:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.939+0000 7faf37e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf301a1d60 con 0x7faf30104d30 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.939+0000 7faf37e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf301a1f20 con 0x7faf301078a0 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.939+0000 7faf35400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf30069050 0x7faf3019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.939+0000 7faf35400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf30069050 0x7faf3019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59746/0 (socket says 172.21.15.131:59746) 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.939+0000 7faf35400700 1 -- 172.21.15.131:0/3413798196 learned_addr learned my addr 172.21.15.131:0/3413798196 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.939+0000 7faf34a00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf30104d30 0x7faf3019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.939+0000 7faf35e00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf301078a0 0x7faf301a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.940+0000 7faf35400700 1 -- 172.21.15.131:0/3413798196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf301078a0 msgr2=0x7faf301a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.940+0000 7faf35400700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf301078a0 0x7faf301a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.940+0000 7faf35400700 1 -- 172.21.15.131:0/3413798196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf30104d30 msgr2=0x7faf3019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.940+0000 7faf35400700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf30104d30 0x7faf3019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.940+0000 7faf35400700 1 -- 172.21.15.131:0/3413798196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240095f0 con 0x7faf30069050 2024-05-10T05:34:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.940+0000 7faf35e00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf301078a0 0x7faf301a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.940+0000 7faf34a00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf30104d30 0x7faf3019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.940+0000 7faf35400700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf30069050 0x7faf3019a110 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faf24000c00 tx=0x7faf24005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.941+0000 7faf2e000700 1 -- 172.21.15.131:0/3413798196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2401d070 con 0x7faf30069050 2024-05-10T05:34:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.941+0000 7faf2e000700 1 -- 172.21.15.131:0/3413798196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf24005030 con 0x7faf30069050 2024-05-10T05:34:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.941+0000 7faf2e000700 1 -- 172.21.15.131:0/3413798196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24004ab0 con 0x7faf30069050 2024-05-10T05:34:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.941+0000 7faf37e00700 1 -- 172.21.15.131:0/3413798196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf301a21a0 con 0x7faf30069050 2024-05-10T05:34:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.941+0000 7faf37e00700 1 -- 172.21.15.131:0/3413798196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf301a2510 con 0x7faf30069050 2024-05-10T05:34:40.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.944+0000 7faf2e000700 1 -- 172.21.15.131:0/3413798196 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7faf24004c10 con 0x7faf30069050 2024-05-10T05:34:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.946+0000 7faf2e000700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faf180a1240 0x7faf180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.946+0000 7faf2e000700 1 -- 172.21.15.131:0/3413798196 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7faf240cde10 con 0x7faf30069050 2024-05-10T05:34:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.946+0000 7faf4fd33b80 1 Processor -- start 2024-05-10T05:34:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.947+0000 7faf34a00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faf180a1240 0x7faf180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.947+0000 7faf4fd33b80 1 -- start start 2024-05-10T05:34:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.947+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a1faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.947+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963a20570 0x55b963a20980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.947+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20ec0 0x55b963a277c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.947+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963946150 con 0x55b963a1f6a0 2024-05-10T05:34:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.947+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963943b10 con 0x55b963a20570 2024-05-10T05:34:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.947+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b96393f400 con 0x55b963a20ec0 2024-05-10T05:34:40.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf34a00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faf180a1240 0x7faf180a3750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faf3019b1b0 tx=0x7faf1c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:40.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf35400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a1faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf35400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a1faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59762/0 (socket says 172.21.15.131:59762) 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf35400700 1 -- 172.21.15.131:0/1952992927 learned_addr learned my addr 172.21.15.131:0/1952992927 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf35e00700 1 --2- 172.21.15.131:0/1952992927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20ec0 0x55b963a277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf34a00700 1 --2- 172.21.15.131:0/1952992927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963a20570 0x55b963a20980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf35400700 1 -- 172.21.15.131:0/1952992927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20ec0 msgr2=0x55b963a277c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf35400700 1 --2- 172.21.15.131:0/1952992927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20ec0 0x55b963a277c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf35400700 1 -- 172.21.15.131:0/1952992927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963a20570 msgr2=0x55b963a20980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf35400700 1 --2- 172.21.15.131:0/1952992927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963a20570 0x55b963a20980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.948+0000 7faf35400700 1 -- 172.21.15.131:0/1952992927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9637c03b0 con 0x55b963a1f6a0 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.949+0000 7faf35400700 1 --2- 172.21.15.131:0/1952992927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a1faf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faf2409af90 tx=0x7faf24097700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21961e4e86e793f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.949+0000 7faf22600700 1 -- 172.21.15.131:0/1952992927 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24095ea0 con 0x55b963a1f6a0 2024-05-10T05:34:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.949+0000 7faf22600700 1 -- 172.21.15.131:0/1952992927 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf2409b990 con 0x55b963a1f6a0 2024-05-10T05:34:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.949+0000 7faf22600700 1 -- 172.21.15.131:0/1952992927 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2409b750 con 0x55b963a1f6a0 2024-05-10T05:34:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.950+0000 7faf4fd33b80 1 -- 172.21.15.131:0/1952992927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 msgr2=0x55b963a1faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.950+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/1952992927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a1faf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faf2409af90 tx=0x7faf24097700 comp rx=0 tx=0).stop 2024-05-10T05:34:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.950+0000 7faf4fd33b80 1 -- 172.21.15.131:0/1952992927 shutdown_connections 2024-05-10T05:34:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.950+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/1952992927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a1faf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.950+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/1952992927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20ec0 0x55b963a277c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.950+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/1952992927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963a20570 0x55b963a20980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.950+0000 7faf4fd33b80 1 -- 172.21.15.131:0/1952992927 >> 172.21.15.131:0/1952992927 conn(0x55b963981a60 msgr2=0x55b963a19fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.951+0000 7faf4fd33b80 1 -- 172.21.15.131:0/1952992927 shutdown_connections 2024-05-10T05:34:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.951+0000 7faf4fd33b80 1 -- 172.21.15.131:0/1952992927 wait complete. 2024-05-10T05:34:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.951+0000 7faf4fd33b80 1 -- start start 2024-05-10T05:34:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.952+0000 7faf4fd33b80 1 Processor -- start 2024-05-10T05:34:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.952+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a29390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.952+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.952+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963a20ec0 0x55b963a30830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.952+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9638db560 con 0x55b963a1f6a0 2024-05-10T05:34:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.952+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9638db200 con 0x55b963a20ec0 2024-05-10T05:34:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.952+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9637c26a0 con 0x55b963a20570 2024-05-10T05:34:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.952+0000 7faf4fd33b80 10 client.0 _ll_register_callbacks cb 0x55b9638e4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf34a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf34a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a298d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56882/0 (socket says 172.21.15.131:56882) 2024-05-10T05:34:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf34a00700 1 -- 172.21.15.131:0/2779708817 learned_addr learned my addr 172.21.15.131:0/2779708817 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf35400700 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a29390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf35e00700 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963a20ec0 0x55b963a30830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf34a00700 1 -- 172.21.15.131:0/2779708817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963a20ec0 msgr2=0x55b963a30830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf34a00700 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963a20ec0 0x55b963a30830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf34a00700 1 -- 172.21.15.131:0/2779708817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 msgr2=0x55b963a29390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf34a00700 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a29390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf34a00700 1 -- 172.21.15.131:0/2779708817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf301a2510 con 0x55b963a20570 2024-05-10T05:34:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.953+0000 7faf35400700 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a29390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.954+0000 7faf34a00700 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a298d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0039d0 tx=0x7faf1c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a158d2c519bb03 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.954+0000 7faf16a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:34:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.954+0000 7faf16a00700 1 -- 172.21.15.131:0/2779708817 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c004110 con 0x55b963a20570 2024-05-10T05:34:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.954+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2779708817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b963898270 con 0x55b963a20570 2024-05-10T05:34:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.954+0000 7faf16a00700 1 -- 172.21.15.131:0/2779708817 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7faf1c01e850 con 0x55b963a20570 2024-05-10T05:34:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.954+0000 7faf16a00700 1 -- 172.21.15.131:0/2779708817 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf1c01ea10 con 0x55b963a20570 2024-05-10T05:34:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.954+0000 7faf4fd33b80 10 client.25609 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.954+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2779708817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b96376cb60 con 0x55b963a20570 2024-05-10T05:34:40.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.955+0000 7faf16a00700 1 -- 172.21.15.131:0/2779708817 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b963898270 con 0x55b963a20570 2024-05-10T05:34:40.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.955+0000 7faf16a00700 1 -- 172.21.15.131:0/2779708817 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faf1c028d20 con 0x55b963a20570 2024-05-10T05:34:40.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.955+0000 7faf4fd33b80 10 client.25609 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:40.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.955+0000 7faf4fd33b80 10 client.25609 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:34:40.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.955+0000 7faf4fd33b80 20 client.25609 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:40.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.955+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b963a35060 0x55b963a374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.955+0000 7faf4fd33b80 4 client.25609 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.955+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2779708817 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9639ee780 con 0x55b963a35060 2024-05-10T05:34:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.956+0000 7faf35400700 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b963a35060 0x55b963a374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.956+0000 7faf35400700 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b963a35060 0x55b963a374b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42ed53edca8e1666 server_cookie=b77cadbed6f5bf9d in_seq=0 out_seq=0 2024-05-10T05:34:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.956+0000 7faf16a00700 10 client.25609 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:34:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf16a00700 1 -- 172.21.15.131:0/2779708817 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9639ee780 con 0x55b963a35060 2024-05-10T05:34:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf16a00700 10 client.25609 handle_command_reply: tid=0 2024-05-10T05:34:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 client.25609 shutdown 2024-05-10T05:34:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2779708817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 msgr2=0x55b963a298d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a298d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0039d0 tx=0x7faf1c0047c0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2779708817 shutdown_connections 2024-05-10T05:34:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b963a35060 0x55b963a374b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a29390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963a20ec0 0x55b963a30830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2779708817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a298d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2779708817 >> 172.21.15.131:0/2779708817 conn(0x55b963981a60 msgr2=0x55b963a19fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf16a00700 0 client.25609 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:34:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2779708817 shutdown_connections 2024-05-10T05:34:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.957+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2779708817 wait complete. 2024-05-10T05:34:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.958+0000 7faf4fd33b80 20 client.25609 trim_cache size 0 max 16384 2024-05-10T05:34:40.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.972+0000 7faf4fd33b80 1 Processor -- start 2024-05-10T05:34:40.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.972+0000 7faf4fd33b80 1 -- start start 2024-05-10T05:34:40.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.972+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 0x55b963a2e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.972+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a2ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.973+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a27940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.973+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963a16d30 con 0x55b963a1f6a0 2024-05-10T05:34:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.973+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963a31930 con 0x55b963981a60 2024-05-10T05:34:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.973+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963a31290 con 0x55b963a20570 2024-05-10T05:34:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.973+0000 7faf34a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a2ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.973+0000 7faf34a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a2ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59768/0 (socket says 172.21.15.131:59768) 2024-05-10T05:34:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.973+0000 7faf34a00700 1 -- 172.21.15.131:0/2883328380 learned_addr learned my addr 172.21.15.131:0/2883328380 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.973+0000 7faf35e00700 1 --2- 172.21.15.131:0/2883328380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a27940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.973+0000 7faf35400700 1 --2- 172.21.15.131:0/2883328380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 0x55b963a2e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.974+0000 7faf35e00700 1 -- 172.21.15.131:0/2883328380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 msgr2=0x55b963a2e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.974+0000 7faf35e00700 1 --2- 172.21.15.131:0/2883328380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 0x55b963a2e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.974+0000 7faf35e00700 1 -- 172.21.15.131:0/2883328380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 msgr2=0x55b963a2ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.974+0000 7faf35e00700 1 --2- 172.21.15.131:0/2883328380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a2ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.974+0000 7faf35e00700 1 -- 172.21.15.131:0/2883328380 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b963a1fd80 con 0x55b963a20570 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.974+0000 7faf34a00700 1 --2- 172.21.15.131:0/2883328380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a2ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.974+0000 7faf35e00700 1 --2- 172.21.15.131:0/2883328380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a27940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faf28009fd0 tx=0x7faf28009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316319c6fcd6fe7b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.974+0000 7faf23000700 1 -- 172.21.15.131:0/2883328380 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28004bb0 con 0x55b963a20570 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.974+0000 7faf23000700 1 -- 172.21.15.131:0/2883328380 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf2800f040 con 0x55b963a20570 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.974+0000 7faf23000700 1 -- 172.21.15.131:0/2883328380 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28007900 con 0x55b963a20570 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.975+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2883328380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 msgr2=0x55b963a27940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.975+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2883328380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a27940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faf28009fd0 tx=0x7faf28009920 comp rx=0 tx=0).stop 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.975+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2883328380 shutdown_connections 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.975+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2883328380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a20570 0x55b963a27940 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.975+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2883328380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 0x55b963a2e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.975+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2883328380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a1f6a0 0x55b963a2ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.975+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2883328380 >> 172.21.15.131:0/2883328380 conn(0x55b963a16450 msgr2=0x55b963a24230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.975+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2883328380 shutdown_connections 2024-05-10T05:34:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.976+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2883328380 wait complete. 2024-05-10T05:34:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.976+0000 7faf4fd33b80 1 -- start start 2024-05-10T05:34:40.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.976+0000 7faf4fd33b80 1 Processor -- start 2024-05-10T05:34:40.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.977+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 0x55b963a8dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.977+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a1f6a0 0x55b963a90220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.977+0000 7faf4fd33b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a20570 0x55b963a92770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.977+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963a2f020 con 0x55b963a20570 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.977+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963a92cb0 con 0x55b963981a60 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.977+0000 7faf4fd33b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963a92df0 con 0x55b963a1f6a0 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.977+0000 7faf4fd33b80 10 client.0 _ll_register_callbacks cb 0x55b9638e4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf35e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a20570 0x55b963a92770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf35e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a20570 0x55b963a92770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59772/0 (socket says 172.21.15.131:59772) 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf35e00700 1 -- 172.21.15.131:0/2641197380 learned_addr learned my addr 172.21.15.131:0/2641197380 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf35400700 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 0x55b963a8dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf34a00700 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a1f6a0 0x55b963a90220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf35e00700 1 -- 172.21.15.131:0/2641197380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a1f6a0 msgr2=0x55b963a90220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf35e00700 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a1f6a0 0x55b963a90220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf35e00700 1 -- 172.21.15.131:0/2641197380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 msgr2=0x55b963a8dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf35e00700 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 0x55b963a8dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf35e00700 1 -- 172.21.15.131:0/2641197380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b96376cb60 con 0x55b963a20570 2024-05-10T05:34:40.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.978+0000 7faf35400700 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 0x55b963a8dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:40.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.979+0000 7faf35e00700 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a20570 0x55b963a92770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faf28009cf0 tx=0x7faf28017970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee0c6ef32c7da1f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:40.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.979+0000 7faf16a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.979+0000 7faf16a00700 1 -- 172.21.15.131:0/2641197380 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf280092e0 con 0x55b963a20570 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.979+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2641197380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf2409b990 con 0x55b963a20570 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.979+0000 7faf16a00700 1 -- 172.21.15.131:0/2641197380 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7faf28007600 con 0x55b963a20570 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.979+0000 7faf16a00700 1 -- 172.21.15.131:0/2641197380 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf280077c0 con 0x55b963a20570 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf4fd33b80 10 client.15597 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2641197380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7faf2800e760 con 0x55b963a20570 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf16a00700 1 -- 172.21.15.131:0/2641197380 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf2409b990 con 0x55b963a20570 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf16a00700 1 -- 172.21.15.131:0/2641197380 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faf28027b40 con 0x55b963a20570 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf4fd33b80 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf4fd33b80 10 client.15597 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf4fd33b80 20 client.15597 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b963a20ec0 0x55b963a96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf4fd33b80 4 client.15597 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2641197380 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b963937ec0 con 0x55b963a20ec0 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.980+0000 7faf35400700 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b963a20ec0 0x55b963a96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.981+0000 7faf16a00700 10 client.15597 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.981+0000 7faf35400700 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b963a20ec0 0x55b963a96760 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce7fb4e30a7dd6ec server_cookie=2e781ca3e560e65f in_seq=0 out_seq=0 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.981+0000 7faf16a00700 1 -- 172.21.15.131:0/2641197380 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b963937ec0 con 0x55b963a20ec0 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.981+0000 7faf16a00700 10 client.15597 handle_command_reply: tid=0 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.981+0000 7faf4fd33b80 1 client.15597 shutdown 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2641197380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a20570 msgr2=0x55b963a92770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a20570 0x55b963a92770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faf28009cf0 tx=0x7faf28017970 comp rx=0 tx=0).stop 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2641197380 shutdown_connections 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b963a20ec0 0x55b963a96760 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b963a20570 0x55b963a92770 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b963981a60 0x55b963a8dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 1 --2- 172.21.15.131:0/2641197380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b963a1f6a0 0x55b963a90220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2641197380 >> 172.21.15.131:0/2641197380 conn(0x55b963a16450 msgr2=0x55b963a24230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf16a00700 0 client.15597 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:34:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2641197380 shutdown_connections 2024-05-10T05:34:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 1 -- 172.21.15.131:0/2641197380 wait complete. 2024-05-10T05:34:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.982+0000 7faf4fd33b80 20 client.15597 trim_cache size 0 max 16384 2024-05-10T05:34:40.985 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:34:40.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.983+0000 7faf37e00700 1 -- 172.21.15.131:0/3413798196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faf180a1240 msgr2=0x7faf180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.983+0000 7faf37e00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faf180a1240 0x7faf180a3750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faf3019b1b0 tx=0x7faf1c009260 comp rx=0 tx=0).stop 2024-05-10T05:34:40.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.983+0000 7faf37e00700 1 -- 172.21.15.131:0/3413798196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf30069050 msgr2=0x7faf3019a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:40.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.983+0000 7faf37e00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf30069050 0x7faf3019a110 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faf24000c00 tx=0x7faf24005b00 comp rx=0 tx=0).stop 2024-05-10T05:34:40.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.983+0000 7faf37e00700 1 -- 172.21.15.131:0/3413798196 shutdown_connections 2024-05-10T05:34:40.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.984+0000 7faf37e00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faf180a1240 0x7faf180a3750 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.984+0000 7faf37e00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf30069050 0x7faf3019a110 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.984+0000 7faf37e00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf301078a0 0x7faf301a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.984+0000 7faf37e00700 1 --2- 172.21.15.131:0/3413798196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf30104d30 0x7faf3019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:40.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.984+0000 7faf37e00700 1 -- 172.21.15.131:0/3413798196 >> 172.21.15.131:0/3413798196 conn(0x7faf300fc0d0 msgr2=0x7faf300fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:40.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.984+0000 7faf37e00700 1 -- 172.21.15.131:0/3413798196 shutdown_connections 2024-05-10T05:34:40.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:40.984+0000 7faf37e00700 1 -- 172.21.15.131:0/3413798196 wait complete. 2024-05-10T05:34:41.015 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:34:41.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:41 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4272367008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:34:41.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.306+0000 7f3b93e00700 1 -- 172.21.15.131:0/806916296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94073b40 msgr2=0x7f3b94074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.306+0000 7f3b93e00700 1 --2- 172.21.15.131:0/806916296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94073b40 0x7f3b94074010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009990 tx=0x7f3b80009ca0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.307+0000 7f3b93e00700 1 -- 172.21.15.131:0/806916296 shutdown_connections 2024-05-10T05:34:41.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.307+0000 7f3b93e00700 1 --2- 172.21.15.131:0/806916296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b94075140 0x7f3b94073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.307+0000 7f3b93e00700 1 --2- 172.21.15.131:0/806916296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9410e030 0x7f3b94110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.307+0000 7f3b93e00700 1 --2- 172.21.15.131:0/806916296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94073b40 0x7f3b94074010 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.307+0000 7f3b93e00700 1 -- 172.21.15.131:0/806916296 >> 172.21.15.131:0/806916296 conn(0x7f3b940fd000 msgr2=0x7f3b940ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.307+0000 7f3b93e00700 1 -- 172.21.15.131:0/806916296 shutdown_connections 2024-05-10T05:34:41.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.307+0000 7f3b93e00700 1 -- 172.21.15.131:0/806916296 wait complete. 2024-05-10T05:34:41.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.308+0000 7f3b93e00700 1 Processor -- start 2024-05-10T05:34:41.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.309+0000 7f3b93e00700 1 -- start start 2024-05-10T05:34:41.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.309+0000 7f3b93e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94073b40 0x7f3b9419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.309+0000 7f3b93e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b94075140 0x7f3b9419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.309+0000 7f3b93e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9410e030 0x7f3b941a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.309+0000 7f3b93e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941a5e90 con 0x7f3b9410e030 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.309+0000 7f3b93e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941a5fd0 con 0x7f3b94075140 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.309+0000 7f3b93e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941a6160 con 0x7f3b94073b40 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.309+0000 7f3b92000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b94075140 0x7f3b9419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b92000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b94075140 0x7f3b9419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46668/0 (socket says 172.21.15.131:46668) 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b92000700 1 -- 172.21.15.131:0/1099177187 learned_addr learned my addr 172.21.15.131:0/1099177187 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b93400700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9410e030 0x7f3b941a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b92a00700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94073b40 0x7f3b9419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b92000700 1 -- 172.21.15.131:0/1099177187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94073b40 msgr2=0x7f3b9419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b92000700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94073b40 0x7f3b9419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b92000700 1 -- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9410e030 msgr2=0x7f3b941a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b92000700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9410e030 0x7f3b941a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b92000700 1 -- 172.21.15.131:0/1099177187 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b800095f0 con 0x7f3b94075140 2024-05-10T05:34:41.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b93400700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9410e030 0x7f3b941a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:41.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.310+0000 7f3b92000700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b94075140 0x7f3b9419e930 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3b80000c00 tx=0x7f3b80005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.311+0000 7f3b8b400700 1 -- 172.21.15.131:0/1099177187 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8001d070 con 0x7f3b94075140 2024-05-10T05:34:41.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.311+0000 7f3b8b400700 1 -- 172.21.15.131:0/1099177187 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b800047e0 con 0x7f3b94075140 2024-05-10T05:34:41.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.311+0000 7f3b8b400700 1 -- 172.21.15.131:0/1099177187 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80005590 con 0x7f3b94075140 2024-05-10T05:34:41.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.311+0000 7f3b93e00700 1 -- 172.21.15.131:0/1099177187 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b941a63e0 con 0x7f3b94075140 2024-05-10T05:34:41.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.311+0000 7f3b93e00700 1 -- 172.21.15.131:0/1099177187 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b941a6750 con 0x7f3b94075140 2024-05-10T05:34:41.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.316+0000 7f3b8b400700 1 -- 172.21.15.131:0/1099177187 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3b80004c50 con 0x7f3b94075140 2024-05-10T05:34:41.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.319+0000 7f3bb17f8b80 1 Processor -- start 2024-05-10T05:34:41.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.319+0000 7f3b8b400700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b780a11a0 0x7f3b780a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.319+0000 7f3b8b400700 1 -- 172.21.15.131:0/1099177187 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3b800cdef0 con 0x7f3b94075140 2024-05-10T05:34:41.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.319+0000 7f3b93400700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b780a11a0 0x7f3b780a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.320+0000 7f3bb17f8b80 1 -- start start 2024-05-10T05:34:41.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.320+0000 7f3b93400700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b780a11a0 0x7f3b780a36b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0093a0 tx=0x7f3b8c00b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.321+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f5a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.321+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5b7a0 0x55aef3f5bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.321+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5c0f0 0x55aef3f629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.321+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3e81150 con 0x55aef3f5c0f0 2024-05-10T05:34:41.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.321+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3e7eb10 con 0x55aef3f5a530 2024-05-10T05:34:41.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.321+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3e7a400 con 0x55aef3f5b7a0 2024-05-10T05:34:41.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.321+0000 7f3b92000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5b7a0 0x55aef3f5bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.321+0000 7f3b92000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5b7a0 0x55aef3f5bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56932/0 (socket says 172.21.15.131:56932) 2024-05-10T05:34:41.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.321+0000 7f3b92000700 1 -- 172.21.15.131:0/2346182786 learned_addr learned my addr 172.21.15.131:0/2346182786 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:41.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.321+0000 7f3b93400700 1 --2- 172.21.15.131:0/2346182786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5c0f0 0x55aef3f629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.322+0000 7f3b92a00700 1 --2- 172.21.15.131:0/2346182786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f5a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.322+0000 7f3b92000700 1 -- 172.21.15.131:0/2346182786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 msgr2=0x55aef3f5a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.322+0000 7f3b92000700 1 --2- 172.21.15.131:0/2346182786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f5a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.322+0000 7f3b92000700 1 -- 172.21.15.131:0/2346182786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5c0f0 msgr2=0x55aef3f629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.322+0000 7f3b92000700 1 --2- 172.21.15.131:0/2346182786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5c0f0 0x55aef3f629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.322+0000 7f3b92000700 1 -- 172.21.15.131:0/2346182786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef3cfb3b0 con 0x55aef3f5b7a0 2024-05-10T05:34:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.322+0000 7f3b92a00700 1 --2- 172.21.15.131:0/2346182786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f5a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.322+0000 7f3b92000700 1 --2- 172.21.15.131:0/2346182786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5b7a0 0x55aef3f5bbb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3b80097de0 tx=0x7f3b8009b240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a358b48e1c0b8fca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.323+0000 7f3b7fe00700 1 -- 172.21.15.131:0/2346182786 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80004950 con 0x55aef3f5b7a0 2024-05-10T05:34:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.323+0000 7f3b7fe00700 1 -- 172.21.15.131:0/2346182786 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b8009ba70 con 0x55aef3f5b7a0 2024-05-10T05:34:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.323+0000 7f3b7fe00700 1 -- 172.21.15.131:0/2346182786 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8009aac0 con 0x55aef3f5b7a0 2024-05-10T05:34:41.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.324+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/2346182786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5b7a0 msgr2=0x55aef3f5bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.324+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/2346182786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5b7a0 0x55aef3f5bbb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3b80097de0 tx=0x7f3b8009b240 comp rx=0 tx=0).stop 2024-05-10T05:34:41.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.324+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/2346182786 shutdown_connections 2024-05-10T05:34:41.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.324+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/2346182786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5c0f0 0x55aef3f629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.324+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/2346182786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f5a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.324+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/2346182786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5b7a0 0x55aef3f5bbb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.324+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/2346182786 >> 172.21.15.131:0/2346182786 conn(0x55aef3ebca60 msgr2=0x55aef3f55330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.324+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/2346182786 shutdown_connections 2024-05-10T05:34:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.324+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/2346182786 wait complete. 2024-05-10T05:34:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.325+0000 7f3bb17f8b80 1 -- start start 2024-05-10T05:34:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.326+0000 7f3bb17f8b80 1 Processor -- start 2024-05-10T05:34:41.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.326+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f64610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.326+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 0x55aef3f64b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.326+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 0x55aef3f6bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.326+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3e16560 con 0x55aef3f5b7a0 2024-05-10T05:34:41.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.326+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3e16200 con 0x55aef3f5a530 2024-05-10T05:34:41.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.327+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3cfd6a0 con 0x55aef3f5c0f0 2024-05-10T05:34:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.327+0000 7f3bb17f8b80 10 client.0 _ll_register_callbacks cb 0x55aef3e1fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.327+0000 7f3b92a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f64610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.327+0000 7f3b92a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f64610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46694/0 (socket says 172.21.15.131:46694) 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.327+0000 7f3b92a00700 1 -- 172.21.15.131:0/3028679425 learned_addr learned my addr 172.21.15.131:0/3028679425 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.327+0000 7f3b93400700 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 0x55aef3f6bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.328+0000 7f3b92000700 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 0x55aef3f64b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.328+0000 7f3b92a00700 1 -- 172.21.15.131:0/3028679425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 msgr2=0x55aef3f6bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.328+0000 7f3b92a00700 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 0x55aef3f6bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.328+0000 7f3b92a00700 1 -- 172.21.15.131:0/3028679425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 msgr2=0x55aef3f64b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.328+0000 7f3b92a00700 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 0x55aef3f64b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.328+0000 7f3b92a00700 1 -- 172.21.15.131:0/3028679425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b941a6750 con 0x55aef3f5a530 2024-05-10T05:34:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.328+0000 7f3b92000700 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 0x55aef3f64b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:41.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.328+0000 7f3b93400700 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 0x55aef3f6bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:41.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.328+0000 7f3b92a00700 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f64610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3b84010670 tx=0x7f3b84010980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97dfa443d7b6aefc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.328+0000 7f3b7cc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:34:41.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.329+0000 7f3b7cc00700 1 -- 172.21.15.131:0/3028679425 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b84004a30 con 0x55aef3f5a530 2024-05-10T05:34:41.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.329+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/3028679425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aef3dd3270 con 0x55aef3f5a530 2024-05-10T05:34:41.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.329+0000 7f3b7cc00700 1 -- 172.21.15.131:0/3028679425 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3b84004b90 con 0x55aef3f5a530 2024-05-10T05:34:41.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.329+0000 7f3b7cc00700 1 -- 172.21.15.131:0/3028679425 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b84004d50 con 0x55aef3f5a530 2024-05-10T05:34:41.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.329+0000 7f3bb17f8b80 10 client.25613 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:41.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.329+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/3028679425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aef3ca7b60 con 0x55aef3f5a530 2024-05-10T05:34:41.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.329+0000 7f3b7cc00700 1 -- 172.21.15.131:0/3028679425 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aef3dd3270 con 0x55aef3f5a530 2024-05-10T05:34:41.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.330+0000 7f3b7cc00700 1 -- 172.21.15.131:0/3028679425 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b84005280 con 0x55aef3f5a530 2024-05-10T05:34:41.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.330+0000 7f3bb17f8b80 10 client.25613 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:41.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.330+0000 7f3bb17f8b80 10 client.25613 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:34:41.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.330+0000 7f3bb17f8b80 20 client.25613 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:41.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.331+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55aef3f702c0 0x55aef3f72710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.331+0000 7f3bb17f8b80 4 client.25613 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:41.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.331+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/3028679425 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aef3f29780 con 0x55aef3f702c0 2024-05-10T05:34:41.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.331+0000 7f3b92000700 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55aef3f702c0 0x55aef3f72710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.332+0000 7f3b7cc00700 10 client.25613 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:34:41.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.332+0000 7f3b92000700 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55aef3f702c0 0x55aef3f72710 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f4e1496011b171f server_cookie=d22f2a5b64527909 in_seq=0 out_seq=0 2024-05-10T05:34:41.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.332+0000 7f3b7cc00700 1 -- 172.21.15.131:0/3028679425 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aef3f29780 con 0x55aef3f702c0 2024-05-10T05:34:41.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.332+0000 7f3b7cc00700 10 client.25613 handle_command_reply: tid=0 2024-05-10T05:34:41.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.332+0000 7f3bb17f8b80 1 client.25613 shutdown 2024-05-10T05:34:41.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/3028679425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 msgr2=0x55aef3f64610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f64610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3b84010670 tx=0x7f3b84010980 comp rx=0 tx=0).stop 2024-05-10T05:34:41.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/3028679425 shutdown_connections 2024-05-10T05:34:41.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55aef3f702c0 0x55aef3f72710 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 0x55aef3f6bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5a530 0x55aef3f64610 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/3028679425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 0x55aef3f64b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/3028679425 >> 172.21.15.131:0/3028679425 conn(0x55aef3ebca60 msgr2=0x55aef3f59060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3b7cc00700 0 client.25613 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:34:41.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/3028679425 shutdown_connections 2024-05-10T05:34:41.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/3028679425 wait complete. 2024-05-10T05:34:41.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.333+0000 7f3bb17f8b80 20 client.25613 trim_cache size 0 max 16384 2024-05-10T05:34:41.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.352+0000 7f3bb17f8b80 1 Processor -- start 2024-05-10T05:34:41.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.352+0000 7f3bb17f8b80 1 -- start start 2024-05-10T05:34:41.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5a530 0x55aef3f63b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 0x55aef3f6ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5c0f0 0x55aef3f722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3f4f8d0 con 0x55aef3f5b7a0 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3f778b0 con 0x55aef3f5c0f0 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3f51340 con 0x55aef3f5a530 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3b92a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5a530 0x55aef3f63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3b92a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5a530 0x55aef3f63b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56950/0 (socket says 172.21.15.131:56950) 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3b92a00700 1 -- 172.21.15.131:0/1043198178 learned_addr learned my addr 172.21.15.131:0/1043198178 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3b92000700 1 --2- 172.21.15.131:0/1043198178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 0x55aef3f6ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.353+0000 7f3b93400700 1 --2- 172.21.15.131:0/1043198178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5c0f0 0x55aef3f722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.354+0000 7f3b92a00700 1 -- 172.21.15.131:0/1043198178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5c0f0 msgr2=0x55aef3f722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.354+0000 7f3b92a00700 1 --2- 172.21.15.131:0/1043198178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5c0f0 0x55aef3f722d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.354+0000 7f3b92a00700 1 -- 172.21.15.131:0/1043198178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 msgr2=0x55aef3f6ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.354+0000 7f3b92a00700 1 --2- 172.21.15.131:0/1043198178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 0x55aef3f6ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.354+0000 7f3b92a00700 1 -- 172.21.15.131:0/1043198178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef3e40560 con 0x55aef3f5a530 2024-05-10T05:34:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.354+0000 7f3b92000700 1 --2- 172.21.15.131:0/1043198178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 0x55aef3f6ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.354+0000 7f3b92a00700 1 --2- 172.21.15.131:0/1043198178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5a530 0x55aef3f63b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b84005780 tx=0x7f3b840057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbb06c3a288bd61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.354+0000 7f3b88c00700 1 -- 172.21.15.131:0/1043198178 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8402a070 con 0x55aef3f5a530 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.354+0000 7f3b88c00700 1 -- 172.21.15.131:0/1043198178 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b8401a410 con 0x55aef3f5a530 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.354+0000 7f3b88c00700 1 -- 172.21.15.131:0/1043198178 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8401aa70 con 0x55aef3f5a530 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.355+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/1043198178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5a530 msgr2=0x55aef3f63b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.355+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/1043198178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5a530 0x55aef3f63b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b84005780 tx=0x7f3b840057b0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.355+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/1043198178 shutdown_connections 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.355+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/1043198178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5c0f0 0x55aef3f722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.355+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/1043198178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5a530 0x55aef3f63b80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.355+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/1043198178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f5b7a0 0x55aef3f6ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.355+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/1043198178 >> 172.21.15.131:0/1043198178 conn(0x55aef3ebca60 msgr2=0x55aef3f62bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.356+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/1043198178 shutdown_connections 2024-05-10T05:34:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.356+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/1043198178 wait complete. 2024-05-10T05:34:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.356+0000 7f3bb17f8b80 1 -- start start 2024-05-10T05:34:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.356+0000 7f3bb17f8b80 1 Processor -- start 2024-05-10T05:34:41.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.357+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5b7a0 0x55aef3f68b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.357+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 0x55aef3f69080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.357+0000 7f3bb17f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f69fb0 0x55aef3f6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.357+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3f6a900 con 0x55aef3f69fb0 2024-05-10T05:34:41.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.357+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3f6aa70 con 0x55aef3f5b7a0 2024-05-10T05:34:41.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.357+0000 7f3bb17f8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef3f6ac30 con 0x55aef3f5c0f0 2024-05-10T05:34:41.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.357+0000 7f3bb17f8b80 10 client.0 _ll_register_callbacks cb 0x55aef3e1fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:41.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.357+0000 7f3b92000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 0x55aef3f69080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.357+0000 7f3b92a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5b7a0 0x55aef3f68b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.357+0000 7f3b92a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5b7a0 0x55aef3f68b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46710/0 (socket says 172.21.15.131:46710) 2024-05-10T05:34:41.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.358+0000 7f3b92a00700 1 -- 172.21.15.131:0/646250624 learned_addr learned my addr 172.21.15.131:0/646250624 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:41.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.358+0000 7f3b93400700 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f69fb0 0x55aef3f6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.358+0000 7f3b92a00700 1 -- 172.21.15.131:0/646250624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 msgr2=0x55aef3f69080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.358+0000 7f3b92a00700 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 0x55aef3f69080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.358+0000 7f3b92a00700 1 -- 172.21.15.131:0/646250624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f69fb0 msgr2=0x55aef3f6a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.358+0000 7f3b92a00700 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f69fb0 0x55aef3f6a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.358+0000 7f3b92a00700 1 -- 172.21.15.131:0/646250624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aef3ca7b60 con 0x55aef3f5b7a0 2024-05-10T05:34:41.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.358+0000 7f3b92a00700 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5b7a0 0x55aef3f68b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b840096c0 tx=0x7f3b84012890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f85fa126bb38be6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.358+0000 7f3b7cc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:34:41.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.359+0000 7f3b7cc00700 1 -- 172.21.15.131:0/646250624 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8402a030 con 0x55aef3f5b7a0 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.359+0000 7f3b7cc00700 1 -- 172.21.15.131:0/646250624 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3b8402b070 con 0x55aef3f5b7a0 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.359+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/646250624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b8009ba70 con 0x55aef3f5b7a0 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.359+0000 7f3b7cc00700 1 -- 172.21.15.131:0/646250624 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b84012df0 con 0x55aef3f5b7a0 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.359+0000 7f3bb17f8b80 10 client.25619 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.359+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/646250624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aef3cffcd0 con 0x55aef3f5b7a0 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.360+0000 7f3b7cc00700 1 -- 172.21.15.131:0/646250624 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b8009ba70 con 0x55aef3f5b7a0 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.360+0000 7f3b7cc00700 1 -- 172.21.15.131:0/646250624 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b8403f070 con 0x55aef3f5b7a0 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.360+0000 7f3bb17f8b80 10 client.25619 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.360+0000 7f3bb17f8b80 10 client.25619 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.360+0000 7f3bb17f8b80 20 client.25619 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.360+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55aef3fc6910 0x55aef3fc8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.360+0000 7f3bb17f8b80 4 client.25619 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:34:41.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.360+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/646250624 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aef3e72ec0 con 0x55aef3fc6910 2024-05-10T05:34:41.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.360+0000 7f3b92000700 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55aef3fc6910 0x55aef3fc8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.361+0000 7f3b7cc00700 10 client.25619 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:34:41.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.361+0000 7f3b92000700 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55aef3fc6910 0x55aef3fc8d30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eaea0038a5c4bcee server_cookie=b4403cd2f9b45413 in_seq=0 out_seq=0 2024-05-10T05:34:41.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.361+0000 7f3b7cc00700 1 -- 172.21.15.131:0/646250624 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aef3e72ec0 con 0x55aef3fc6910 2024-05-10T05:34:41.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.361+0000 7f3b7cc00700 10 client.25619 handle_command_reply: tid=0 2024-05-10T05:34:41.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.361+0000 7f3bb17f8b80 1 client.25619 shutdown 2024-05-10T05:34:41.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.361+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/646250624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5b7a0 msgr2=0x55aef3f68b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.361+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5b7a0 0x55aef3f68b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b840096c0 tx=0x7f3b84012890 comp rx=0 tx=0).stop 2024-05-10T05:34:41.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.362+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/646250624 shutdown_connections 2024-05-10T05:34:41.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.362+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55aef3fc6910 0x55aef3fc8d30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.362+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aef3f69fb0 0x55aef3f6a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.362+0000 7f3b7cc00700 0 client.25619 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:34:41.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.362+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aef3f5b7a0 0x55aef3f68b40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.362+0000 7f3bb17f8b80 1 --2- 172.21.15.131:0/646250624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aef3f5c0f0 0x55aef3f69080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.362+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/646250624 >> 172.21.15.131:0/646250624 conn(0x55aef3ebca60 msgr2=0x55aef3f62a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.362+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/646250624 shutdown_connections 2024-05-10T05:34:41.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.362+0000 7f3bb17f8b80 1 -- 172.21.15.131:0/646250624 wait complete. 2024-05-10T05:34:41.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.362+0000 7f3bb17f8b80 20 client.25619 trim_cache size 0 max 16384 2024-05-10T05:34:41.365 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 -- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b780a11a0 msgr2=0x7f3b780a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b780a11a0 0x7f3b780a36b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0093a0 tx=0x7f3b8c00b040 comp rx=0 tx=0).stop 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 -- 172.21.15.131:0/1099177187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b94075140 msgr2=0x7f3b9419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b94075140 0x7f3b9419e930 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3b80000c00 tx=0x7f3b80005ef0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 -- 172.21.15.131:0/1099177187 shutdown_connections 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b780a11a0 0x7f3b780a36b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9410e030 0x7f3b941a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b94073b40 0x7f3b9419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 --2- 172.21.15.131:0/1099177187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b94075140 0x7f3b9419e930 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 -- 172.21.15.131:0/1099177187 >> 172.21.15.131:0/1099177187 conn(0x7f3b940fd000 msgr2=0x7f3b941013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 -- 172.21.15.131:0/1099177187 shutdown_connections 2024-05-10T05:34:41.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.363+0000 7f3b93e00700 1 -- 172.21.15.131:0/1099177187 wait complete. 2024-05-10T05:34:41.388 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:41 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4272367008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:34:41.391 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:34:41.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:41 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4272367008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:34:41.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.666+0000 7fe2d9200700 1 -- 172.21.15.131:0/805478354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d41019e0 msgr2=0x7fe2d4101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.666+0000 7fe2d9200700 1 --2- 172.21.15.131:0/805478354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d41019e0 0x7fe2d4101df0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40099c0 tx=0x7fe2c4009cd0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.666+0000 7fe2d9200700 1 -- 172.21.15.131:0/805478354 shutdown_connections 2024-05-10T05:34:41.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.666+0000 7fe2d9200700 1 --2- 172.21.15.131:0/805478354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d41034a0 0x7fe2d4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.666+0000 7fe2d9200700 1 --2- 172.21.15.131:0/805478354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d41019e0 0x7fe2d4101df0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.666+0000 7fe2d9200700 1 --2- 172.21.15.131:0/805478354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4102b00 0x7fe2d4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.666+0000 7fe2d9200700 1 -- 172.21.15.131:0/805478354 >> 172.21.15.131:0/805478354 conn(0x7fe2d40fcfd0 msgr2=0x7fe2d40ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.666+0000 7fe2d9200700 1 -- 172.21.15.131:0/805478354 shutdown_connections 2024-05-10T05:34:41.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.666+0000 7fe2d9200700 1 -- 172.21.15.131:0/805478354 wait complete. 2024-05-10T05:34:41.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.667+0000 7fe2d9200700 1 Processor -- start 2024-05-10T05:34:41.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.668+0000 7fe2d9200700 1 -- start start 2024-05-10T05:34:41.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.668+0000 7fe2d9200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d41019e0 0x7fe2d419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.668+0000 7fe2d9200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4102b00 0x7fe2d419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.668+0000 7fe2d9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d41034a0 0x7fe2d41a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.668+0000 7fe2d9200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d41a1ba0 con 0x7fe2d4102b00 2024-05-10T05:34:41.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.668+0000 7fe2d9200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d41a1ce0 con 0x7fe2d41019e0 2024-05-10T05:34:41.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.668+0000 7fe2d9200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d41a1e70 con 0x7fe2d41034a0 2024-05-10T05:34:41.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d41019e0 0x7fe2d419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d41019e0 0x7fe2d419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46736/0 (socket says 172.21.15.131:46736) 2024-05-10T05:34:41.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d3400700 1 -- 172.21.15.131:0/803977880 learned_addr learned my addr 172.21.15.131:0/803977880 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:41.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4102b00 0x7fe2d419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d41034a0 0x7fe2d41a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d3400700 1 -- 172.21.15.131:0/803977880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d41034a0 msgr2=0x7fe2d41a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d3400700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d41034a0 0x7fe2d41a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d3400700 1 -- 172.21.15.131:0/803977880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4102b00 msgr2=0x7fe2d419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d3400700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4102b00 0x7fe2d419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d3400700 1 -- 172.21.15.131:0/803977880 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c40095f0 con 0x7fe2d41019e0 2024-05-10T05:34:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.669+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d41034a0 0x7fe2d41a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:34:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.670+0000 7fe2d3400700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d41019e0 0x7fe2d419a100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40098e0 tx=0x7fe2c4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.670+0000 7fe2cbe00700 1 -- 172.21.15.131:0/803977880 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c401d070 con 0x7fe2d41019e0 2024-05-10T05:34:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.670+0000 7fe2cbe00700 1 -- 172.21.15.131:0/803977880 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2c4003f90 con 0x7fe2d41019e0 2024-05-10T05:34:41.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.670+0000 7fe2d9200700 1 -- 172.21.15.131:0/803977880 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d41a20f0 con 0x7fe2d41019e0 2024-05-10T05:34:41.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.670+0000 7fe2cbe00700 1 -- 172.21.15.131:0/803977880 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4022c40 con 0x7fe2d41019e0 2024-05-10T05:34:41.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.670+0000 7fe2d9200700 1 -- 172.21.15.131:0/803977880 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d41a2510 con 0x7fe2d41019e0 2024-05-10T05:34:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.676+0000 7fe2cbe00700 1 -- 172.21.15.131:0/803977880 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe2c4022470 con 0x7fe2d41019e0 2024-05-10T05:34:41.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.677+0000 7fe2f26b7b80 1 Processor -- start 2024-05-10T05:34:41.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.677+0000 7fe2f26b7b80 1 -- start start 2024-05-10T05:34:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.677+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.677+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.677+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.677+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408cfc4150 con 0x56408d0a6fb0 2024-05-10T05:34:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.677+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408cfc1b10 con 0x56408d0a7900 2024-05-10T05:34:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.677+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408cfbd400 con 0x56408d0a5d40 2024-05-10T05:34:41.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.678+0000 7fe2d3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.678+0000 7fe2d3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.678+0000 7fe2d3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0a6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57008/0 (socket says 172.21.15.131:57008) 2024-05-10T05:34:41.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.678+0000 7fe2d3e00700 1 -- 172.21.15.131:0/1929033739 learned_addr learned my addr 172.21.15.131:0/1929033739 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:41.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.678+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/1929033739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.678+0000 7fe2d3e00700 1 -- 172.21.15.131:0/1929033739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 msgr2=0x56408d0ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.678+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/1929033739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.678+0000 7fe2d3e00700 1 -- 172.21.15.131:0/1929033739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 msgr2=0x56408d0a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.678+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/1929033739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0a73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.678+0000 7fe2d3e00700 1 -- 172.21.15.131:0/1929033739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56408ce3e3b0 con 0x56408d0a5d40 2024-05-10T05:34:41.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.679+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/1929033739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0a6170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009ca0 tx=0x7fe2cc007500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657e2bf766f9cabf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.679+0000 7fe2c8c00700 1 -- 172.21.15.131:0/1929033739 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc004db0 con 0x56408d0a5d40 2024-05-10T05:34:41.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.679+0000 7fe2c8c00700 1 -- 172.21.15.131:0/1929033739 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2cc007e40 con 0x56408d0a5d40 2024-05-10T05:34:41.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.679+0000 7fe2c8c00700 1 -- 172.21.15.131:0/1929033739 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc0146f0 con 0x56408d0a5d40 2024-05-10T05:34:41.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.679+0000 7fe2cbe00700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2b80a11a0 0x7fe2b80a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.680+0000 7fe2cbe00700 1 -- 172.21.15.131:0/803977880 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe2c40cdda0 con 0x7fe2d41019e0 2024-05-10T05:34:41.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.681+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1929033739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 msgr2=0x56408d0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.681+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/1929033739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0a6170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009ca0 tx=0x7fe2cc007500 comp rx=0 tx=0).stop 2024-05-10T05:34:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.681+0000 7fe2d3400700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2b80a11a0 0x7fe2b80a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.681+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1929033739 shutdown_connections 2024-05-10T05:34:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.681+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/1929033739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.681+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/1929033739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0a6170 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.682+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/1929033739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.682+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1929033739 >> 172.21.15.131:0/1929033739 conn(0x56408cfffa60 msgr2=0x56408d098330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.682+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1929033739 shutdown_connections 2024-05-10T05:34:41.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.682+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1929033739 wait complete. 2024-05-10T05:34:41.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.682+0000 7fe2d3400700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2b80a11a0 0x7fe2b80a36b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c010 tx=0x7fe2c4009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.683+0000 7fe2f26b7b80 1 -- start start 2024-05-10T05:34:41.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.683+0000 7fe2f26b7b80 1 Processor -- start 2024-05-10T05:34:41.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.684+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.684+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.684+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.684+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408cf59560 con 0x56408d0a6fb0 2024-05-10T05:34:41.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.684+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408cf59200 con 0x56408d0a7900 2024-05-10T05:34:41.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.684+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408ce406a0 con 0x56408d0a5d40 2024-05-10T05:34:41.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.684+0000 7fe2f26b7b80 10 client.0 _ll_register_callbacks cb 0x56408cf62b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:41.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.685+0000 7fe2d3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.685+0000 7fe2d2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.685+0000 7fe2d2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59876/0 (socket says 172.21.15.131:59876) 2024-05-10T05:34:41.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.685+0000 7fe2d2a00700 1 -- 172.21.15.131:0/1447887382 learned_addr learned my addr 172.21.15.131:0/1447887382 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:41.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.685+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.685+0000 7fe2d2a00700 1 -- 172.21.15.131:0/1447887382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 msgr2=0x56408d0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.685+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.685+0000 7fe2d2a00700 1 -- 172.21.15.131:0/1447887382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 msgr2=0x56408d0b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.685+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.686+0000 7fe2d2a00700 1 -- 172.21.15.131:0/1447887382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2cc009900 con 0x56408d0a6fb0 2024-05-10T05:34:41.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.686+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b0640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000baf0 tx=0x7fe2c000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f37edf6c4f1e8ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.686+0000 7fe2bd600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:41.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.686+0000 7fe2bd600700 1 -- 172.21.15.131:0/1447887382 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0004830 con 0x56408d0a6fb0 2024-05-10T05:34:41.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.686+0000 7fe2bd600700 1 -- 172.21.15.131:0/1447887382 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe2c0004990 con 0x56408d0a6fb0 2024-05-10T05:34:41.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.686+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1447887382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56408cf16270 con 0x56408d0a6fb0 2024-05-10T05:34:41.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.687+0000 7fe2bd600700 1 -- 172.21.15.131:0/1447887382 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2c0004b90 con 0x56408d0a6fb0 2024-05-10T05:34:41.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.687+0000 7fe2f26b7b80 10 client.15627 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.687+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1447887382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56408cdeab60 con 0x56408d0a6fb0 2024-05-10T05:34:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.687+0000 7fe2bd600700 1 -- 172.21.15.131:0/1447887382 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56408cf16270 con 0x56408d0a6fb0 2024-05-10T05:34:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.687+0000 7fe2bd600700 1 -- 172.21.15.131:0/1447887382 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe2c0011440 con 0x56408d0a6fb0 2024-05-10T05:34:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.687+0000 7fe2f26b7b80 10 client.15627 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.687+0000 7fe2f26b7b80 10 client.15627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:34:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.688+0000 7fe2f26b7b80 20 client.15627 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.688+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56408d0bbe40 0x56408d0be290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.688+0000 7fe2f26b7b80 4 client.15627 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.688+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1447887382 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56408d06c780 con 0x56408d0bbe40 2024-05-10T05:34:41.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.688+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56408d0bbe40 0x56408d0be290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.689+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56408d0bbe40 0x56408d0be290 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a416dae920a337db server_cookie=e393d9a67120978e in_seq=0 out_seq=0 2024-05-10T05:34:41.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.689+0000 7fe2bd600700 10 client.15627 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:34:41.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.690+0000 7fe2bd600700 1 -- 172.21.15.131:0/1447887382 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56408d06c780 con 0x56408d0bbe40 2024-05-10T05:34:41.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.690+0000 7fe2bd600700 10 client.15627 handle_command_reply: tid=0 2024-05-10T05:34:41.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.690+0000 7fe2f26b7b80 1 client.15627 shutdown 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1447887382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 msgr2=0x56408d0b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b0640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000baf0 tx=0x7fe2c000be00 comp rx=0 tx=0).stop 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1447887382 shutdown_connections 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56408d0bbe40 0x56408d0be290 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/1447887382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b0640 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1447887382 >> 172.21.15.131:0/1447887382 conn(0x56408cfffa60 msgr2=0x56408d0a4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2bd600700 0 client.15627 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1447887382 shutdown_connections 2024-05-10T05:34:41.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.691+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/1447887382 wait complete. 2024-05-10T05:34:41.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.693+0000 7fe2f26b7b80 20 client.15627 trim_cache size 0 max 16384 2024-05-10T05:34:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.718+0000 7fe2f26b7b80 1 Processor -- start 2024-05-10T05:34:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.718+0000 7fe2f26b7b80 1 -- start start 2024-05-10T05:34:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.718+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a5d40 0x56408d0bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.718+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a6fb0 0x56408d0b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.718+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0be390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.718+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408d094340 con 0x56408d0a5d40 2024-05-10T05:34:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.718+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408d0a65d0 con 0x56408d0a7900 2024-05-10T05:34:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.718+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408d0a6a90 con 0x56408d0a6fb0 2024-05-10T05:34:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.719+0000 7fe2d3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a5d40 0x56408d0bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.719+0000 7fe2d3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a5d40 0x56408d0bde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59890/0 (socket says 172.21.15.131:59890) 2024-05-10T05:34:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.719+0000 7fe2d3e00700 1 -- 172.21.15.131:0/2720910120 learned_addr learned my addr 172.21.15.131:0/2720910120 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.719+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/2720910120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a6fb0 0x56408d0b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.719+0000 7fe2d2a00700 1 -- 172.21.15.131:0/2720910120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 msgr2=0x56408d0be390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.719+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/2720910120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0be390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.719+0000 7fe2d2a00700 1 -- 172.21.15.131:0/2720910120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a5d40 msgr2=0x56408d0bde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.719+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/2720910120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a5d40 0x56408d0bde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.719+0000 7fe2d2a00700 1 -- 172.21.15.131:0/2720910120 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56408cf83560 con 0x56408d0a6fb0 2024-05-10T05:34:41.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.719+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/2720910120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a6fb0 0x56408d0b7560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0010020 tx=0x7fe2c000bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8bf9ab0b3b092a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.720+0000 7fe2c9600700 1 -- 172.21.15.131:0/2720910120 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0011440 con 0x56408d0a6fb0 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.720+0000 7fe2c9600700 1 -- 172.21.15.131:0/2720910120 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2c0020030 con 0x56408d0a6fb0 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.720+0000 7fe2c9600700 1 -- 172.21.15.131:0/2720910120 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00048c0 con 0x56408d0a6fb0 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.721+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/2720910120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a6fb0 msgr2=0x56408d0b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.721+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/2720910120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a6fb0 0x56408d0b7560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0010020 tx=0x7fe2c000bf20 comp rx=0 tx=0).stop 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.721+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/2720910120 shutdown_connections 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.721+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/2720910120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a5d40 0x56408d0bde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.721+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/2720910120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d0be390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.721+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/2720910120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a6fb0 0x56408d0b7560 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.721+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/2720910120 >> 172.21.15.131:0/2720910120 conn(0x56408cfffa60 msgr2=0x56408d0aace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.721+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/2720910120 shutdown_connections 2024-05-10T05:34:41.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.721+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/2720910120 wait complete. 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.721+0000 7fe2f26b7b80 1 -- start start 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.722+0000 7fe2f26b7b80 1 Processor -- start 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.722+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.722+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2f26b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d1102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408d0b4dd0 con 0x56408d0a6fb0 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408d0b4f40 con 0x56408d0a7900 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2f26b7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408d110820 con 0x56408d0a5d40 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2f26b7b80 10 client.0 _ll_register_callbacks cb 0x56408cf62b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2d3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2d3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57028/0 (socket says 172.21.15.131:57028) 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2d3e00700 1 -- 172.21.15.131:0/603254214 learned_addr learned my addr 172.21.15.131:0/603254214 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d1102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2d3e00700 1 -- 172.21.15.131:0/603254214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 msgr2=0x56408d1102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d1102e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2d3e00700 1 -- 172.21.15.131:0/603254214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 msgr2=0x56408d0b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:41.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.723+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.724+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.724+0000 7fe2d3e00700 1 -- 172.21.15.131:0/603254214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56408cdeab60 con 0x56408d0a5d40 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.724+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.724+0000 7fe2d3e00700 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0b4350 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x56408d0a6d70 tx=0x7fe2cc007500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9913acc6511bed71 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.724+0000 7fe2bd600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.724+0000 7fe2bd600700 1 -- 172.21.15.131:0/603254214 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc028040 con 0x56408d0a5d40 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.724+0000 7fe2bd600700 1 -- 172.21.15.131:0/603254214 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe2cc02a220 con 0x56408d0a5d40 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.724+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/603254214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2cc007e40 con 0x56408d0a5d40 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2bd600700 1 -- 172.21.15.131:0/603254214 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2cc02a3e0 con 0x56408d0a5d40 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2f26b7b80 10 client.25642 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/603254214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56408ce42cd0 con 0x56408d0a5d40 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2bd600700 1 -- 172.21.15.131:0/603254214 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2cc007e40 con 0x56408d0a5d40 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2bd600700 1 -- 172.21.15.131:0/603254214 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe2cc009cd0 con 0x56408d0a5d40 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2f26b7b80 10 client.25642 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2f26b7b80 10 client.25642 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:34:41.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2f26b7b80 20 client.25642 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:41.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56408d112150 0x56408d114570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:41.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2f26b7b80 4 client.25642 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:34:41.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.725+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/603254214 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56408cfb5ec0 con 0x56408d112150 2024-05-10T05:34:41.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.726+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56408d112150 0x56408d114570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:41.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.726+0000 7fe2d2a00700 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56408d112150 0x56408d114570 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de197b6a3d88a098 server_cookie=1420b0528bad85a5 in_seq=0 out_seq=0 2024-05-10T05:34:41.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.726+0000 7fe2bd600700 10 client.25642 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:34:41.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.728+0000 7fe2bd600700 1 -- 172.21.15.131:0/603254214 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56408cfb5ec0 con 0x56408d112150 2024-05-10T05:34:41.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.728+0000 7fe2bd600700 10 client.25642 handle_command_reply: tid=0 2024-05-10T05:34:41.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.728+0000 7fe2f26b7b80 1 client.25642 shutdown 2024-05-10T05:34:41.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/603254214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 msgr2=0x56408d0b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0b4350 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x56408d0a6d70 tx=0x7fe2cc007500 comp rx=0 tx=0).stop 2024-05-10T05:34:41.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/603254214 shutdown_connections 2024-05-10T05:34:41.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56408d112150 0x56408d114570 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56408d0a7900 0x56408d1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56408d0a5d40 0x56408d0b4350 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2f26b7b80 1 --2- 172.21.15.131:0/603254214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56408d0a6fb0 0x56408d0b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/603254214 >> 172.21.15.131:0/603254214 conn(0x56408cfffa60 msgr2=0x56408d097bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2bd600700 0 client.25642 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:34:41.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/603254214 shutdown_connections 2024-05-10T05:34:41.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.729+0000 7fe2f26b7b80 1 -- 172.21.15.131:0/603254214 wait complete. 2024-05-10T05:34:41.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.730+0000 7fe2f26b7b80 20 client.25642 trim_cache size 0 max 16384 2024-05-10T05:34:41.731 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:34:41.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.730+0000 7fe2d9200700 1 -- 172.21.15.131:0/803977880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2b80a11a0 msgr2=0x7fe2b80a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.730+0000 7fe2d9200700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2b80a11a0 0x7fe2b80a36b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c010 tx=0x7fe2c4009310 comp rx=0 tx=0).stop 2024-05-10T05:34:41.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.730+0000 7fe2d9200700 1 -- 172.21.15.131:0/803977880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d41019e0 msgr2=0x7fe2d419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:41.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.730+0000 7fe2d9200700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d41019e0 0x7fe2d419a100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40098e0 tx=0x7fe2c4009580 comp rx=0 tx=0).stop 2024-05-10T05:34:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.731+0000 7fe2d9200700 1 -- 172.21.15.131:0/803977880 shutdown_connections 2024-05-10T05:34:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.731+0000 7fe2d9200700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2b80a11a0 0x7fe2b80a36b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.731+0000 7fe2d9200700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d41034a0 0x7fe2d41a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.731+0000 7fe2d9200700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2d41019e0 0x7fe2d419a100 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.731+0000 7fe2d9200700 1 --2- 172.21.15.131:0/803977880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4102b00 0x7fe2d419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.731+0000 7fe2d9200700 1 -- 172.21.15.131:0/803977880 >> 172.21.15.131:0/803977880 conn(0x7fe2d40fcfd0 msgr2=0x7fe2d4073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.731+0000 7fe2d9200700 1 -- 172.21.15.131:0/803977880 shutdown_connections 2024-05-10T05:34:41.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:41.731+0000 7fe2d9200700 1 -- 172.21.15.131:0/803977880 wait complete. 2024-05-10T05:34:41.777 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:34:42.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.061+0000 7fbc09400700 1 -- 172.21.15.131:0/2980378197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc041034a0 msgr2=0x7fbc04109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.061+0000 7fbc09400700 1 --2- 172.21.15.131:0/2980378197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc041034a0 0x7fbc04109d50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009fc0 tx=0x7fbbfc00b330 comp rx=0 tx=0).stop 2024-05-10T05:34:42.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.062+0000 7fbc09400700 1 -- 172.21.15.131:0/2980378197 shutdown_connections 2024-05-10T05:34:42.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.062+0000 7fbc09400700 1 --2- 172.21.15.131:0/2980378197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc041034a0 0x7fbc04109d50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.062+0000 7fbc09400700 1 --2- 172.21.15.131:0/2980378197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc041019e0 0x7fbc04101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.062+0000 7fbc09400700 1 --2- 172.21.15.131:0/2980378197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc04102b00 0x7fbc04102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.062+0000 7fbc09400700 1 -- 172.21.15.131:0/2980378197 >> 172.21.15.131:0/2980378197 conn(0x7fbc040fcfd0 msgr2=0x7fbc040ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.062+0000 7fbc09400700 1 -- 172.21.15.131:0/2980378197 shutdown_connections 2024-05-10T05:34:42.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.062+0000 7fbc09400700 1 -- 172.21.15.131:0/2980378197 wait complete. 2024-05-10T05:34:42.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.063+0000 7fbc09400700 1 Processor -- start 2024-05-10T05:34:42.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.063+0000 7fbc09400700 1 -- start start 2024-05-10T05:34:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc09400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041019e0 0x7fbc04197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc09400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102b00 0x7fbc04198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc09400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc041034a0 0x7fbc0419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc09400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0419f9e0 con 0x7fbc041019e0 2024-05-10T05:34:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc09400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0419fb20 con 0x7fbc041034a0 2024-05-10T05:34:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc09400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0419fcb0 con 0x7fbc04102b00 2024-05-10T05:34:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc03e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041019e0 0x7fbc04197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc03e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041019e0 0x7fbc04197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59912/0 (socket says 172.21.15.131:59912) 2024-05-10T05:34:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc03e00700 1 -- 172.21.15.131:0/1112396437 learned_addr learned my addr 172.21.15.131:0/1112396437 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc03400700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102b00 0x7fbc04198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.064+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc041034a0 0x7fbc0419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.065+0000 7fbc03e00700 1 -- 172.21.15.131:0/1112396437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102b00 msgr2=0x7fbc04198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.065+0000 7fbc03e00700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102b00 0x7fbc04198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.065+0000 7fbc03e00700 1 -- 172.21.15.131:0/1112396437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc041034a0 msgr2=0x7fbc0419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.065+0000 7fbc03e00700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc041034a0 0x7fbc0419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.065+0000 7fbc03e00700 1 -- 172.21.15.131:0/1112396437 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc009c40 con 0x7fbc041019e0 2024-05-10T05:34:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.065+0000 7fbc03400700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102b00 0x7fbc04198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.065+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc041034a0 0x7fbc0419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.065+0000 7fbc03e00700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041019e0 0x7fbc04197f40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e970 tx=0x7fbbf400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.065+0000 7fbc00c00700 1 -- 172.21.15.131:0/1112396437 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4004220 con 0x7fbc041019e0 2024-05-10T05:34:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.066+0000 7fbc09400700 1 -- 172.21.15.131:0/1112396437 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0419ff90 con 0x7fbc041019e0 2024-05-10T05:34:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.066+0000 7fbc09400700 1 -- 172.21.15.131:0/1112396437 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc041a0410 con 0x7fbc041019e0 2024-05-10T05:34:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.066+0000 7fbc00c00700 1 -- 172.21.15.131:0/1112396437 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf4004380 con 0x7fbc041019e0 2024-05-10T05:34:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.066+0000 7fbc00c00700 1 -- 172.21.15.131:0/1112396437 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4003c70 con 0x7fbc041019e0 2024-05-10T05:34:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.071+0000 7fbc00c00700 1 -- 172.21.15.131:0/1112396437 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbbf4021440 con 0x7fbc041019e0 2024-05-10T05:34:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.072+0000 7fbc22968b80 1 Processor -- start 2024-05-10T05:34:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.072+0000 7fbc22968b80 1 -- start start 2024-05-10T05:34:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.073+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 0x558e57979960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.073+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 0x558e5797abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.073+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797b0f0 0x558e579819f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.073+0000 7fbc22968b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e578a0150 con 0x558e5797b0f0 2024-05-10T05:34:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.073+0000 7fbc22968b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5789db10 con 0x558e5797a7a0 2024-05-10T05:34:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.073+0000 7fbc22968b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e57899400 con 0x558e57979530 2024-05-10T05:34:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.073+0000 7fbc03e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 0x558e5797abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.073+0000 7fbc08a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 0x558e57979960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.073+0000 7fbc08a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 0x558e57979960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57060/0 (socket says 172.21.15.131:57060) 2024-05-10T05:34:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.073+0000 7fbc08a00700 1 -- 172.21.15.131:0/223833466 learned_addr learned my addr 172.21.15.131:0/223833466 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.074+0000 7fbc08a00700 1 -- 172.21.15.131:0/223833466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 msgr2=0x558e5797abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.074+0000 7fbc08a00700 1 --2- 172.21.15.131:0/223833466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 0x558e5797abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.074+0000 7fbc08a00700 1 -- 172.21.15.131:0/223833466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797b0f0 msgr2=0x558e579819f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.074+0000 7fbc08a00700 1 --2- 172.21.15.131:0/223833466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797b0f0 0x558e579819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.074+0000 7fbc08a00700 1 -- 172.21.15.131:0/223833466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5771a3b0 con 0x558e57979530 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.074+0000 7fbc00c00700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80a11a0 0x7fbbe80a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.074+0000 7fbc08a00700 1 --2- 172.21.15.131:0/223833466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 0x558e57979960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc007960 tx=0x7fbbfc009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e861861b13d4554e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.074+0000 7fbbf9200700 1 -- 172.21.15.131:0/223833466 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc00e040 con 0x558e57979530 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.075+0000 7fbbf9200700 1 -- 172.21.15.131:0/223833466 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbfc003a00 con 0x558e57979530 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.075+0000 7fbbf9200700 1 -- 172.21.15.131:0/223833466 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc00bdb0 con 0x558e57979530 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.075+0000 7fbc03e00700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80a11a0 0x7fbbe80a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.075+0000 7fbc00c00700 1 -- 172.21.15.131:0/1112396437 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbbf4014070 con 0x7fbc041019e0 2024-05-10T05:34:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.076+0000 7fbc03e00700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80a11a0 0x7fbbe80a36b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40972e0 tx=0x7fbbf40d2530 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.076+0000 7fbc22968b80 1 -- 172.21.15.131:0/223833466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 msgr2=0x558e57979960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.076+0000 7fbc22968b80 1 --2- 172.21.15.131:0/223833466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 0x558e57979960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc007960 tx=0x7fbbfc009ac0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.076+0000 7fbc22968b80 1 -- 172.21.15.131:0/223833466 shutdown_connections 2024-05-10T05:34:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.076+0000 7fbc22968b80 1 --2- 172.21.15.131:0/223833466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797b0f0 0x558e579819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.076+0000 7fbc22968b80 1 --2- 172.21.15.131:0/223833466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 0x558e57979960 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.076+0000 7fbc22968b80 1 --2- 172.21.15.131:0/223833466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 0x558e5797abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.076+0000 7fbc22968b80 1 -- 172.21.15.131:0/223833466 >> 172.21.15.131:0/223833466 conn(0x558e578dba60 msgr2=0x558e57974330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.078+0000 7fbc22968b80 1 -- 172.21.15.131:0/223833466 shutdown_connections 2024-05-10T05:34:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.078+0000 7fbc22968b80 1 -- 172.21.15.131:0/223833466 wait complete. 2024-05-10T05:34:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.078+0000 7fbc22968b80 1 -- start start 2024-05-10T05:34:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.079+0000 7fbc22968b80 1 Processor -- start 2024-05-10T05:34:42.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.080+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 0x558e57983850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.080+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 0x558e57983d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.080+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797b0f0 0x558e5798acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.080+0000 7fbc22968b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e57835560 con 0x558e5797b0f0 2024-05-10T05:34:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.080+0000 7fbc22968b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e57835200 con 0x558e5797a7a0 2024-05-10T05:34:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.080+0000 7fbc22968b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5771c6a0 con 0x558e57979530 2024-05-10T05:34:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.080+0000 7fbc22968b80 10 client.0 _ll_register_callbacks cb 0x558e5783eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc03400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 0x558e57983d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc03400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 0x558e57983d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46796/0 (socket says 172.21.15.131:46796) 2024-05-10T05:34:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc03400700 1 -- 172.21.15.131:0/1420193391 learned_addr learned my addr 172.21.15.131:0/1420193391 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 0x558e57983850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797b0f0 0x558e5798acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc03400700 1 -- 172.21.15.131:0/1420193391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 msgr2=0x558e57983850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc03400700 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 0x558e57983850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc03400700 1 -- 172.21.15.131:0/1420193391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797b0f0 msgr2=0x558e5798acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc03400700 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797b0f0 0x558e5798acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc03400700 1 -- 172.21.15.131:0/1420193391 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0410a7e0 con 0x558e5797a7a0 2024-05-10T05:34:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.081+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797b0f0 0x558e5798acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.082+0000 7fbc03400700 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 0x558e57983d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009720 tx=0x7fbbf0009d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e9ffb5c98070e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.082+0000 7fbbed600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:34:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.082+0000 7fbbed600700 1 -- 172.21.15.131:0/1420193391 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000fb00 con 0x558e5797a7a0 2024-05-10T05:34:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.082+0000 7fbc22968b80 1 -- 172.21.15.131:0/1420193391 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e577f2270 con 0x558e5797a7a0 2024-05-10T05:34:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.082+0000 7fbbed600700 1 -- 172.21.15.131:0/1420193391 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbbf000fc60 con 0x558e5797a7a0 2024-05-10T05:34:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.083+0000 7fbbed600700 1 -- 172.21.15.131:0/1420193391 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbf000fe20 con 0x558e5797a7a0 2024-05-10T05:34:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.083+0000 7fbc22968b80 10 client.25646 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.083+0000 7fbc22968b80 1 -- 172.21.15.131:0/1420193391 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558e576c6b60 con 0x558e5797a7a0 2024-05-10T05:34:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.083+0000 7fbbed600700 1 -- 172.21.15.131:0/1420193391 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e577f2270 con 0x558e5797a7a0 2024-05-10T05:34:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.084+0000 7fbbed600700 1 -- 172.21.15.131:0/1420193391 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbbf00223d0 con 0x558e5797a7a0 2024-05-10T05:34:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.084+0000 7fbc22968b80 10 client.25646 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.084+0000 7fbc22968b80 10 client.25646 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:34:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.084+0000 7fbc22968b80 20 client.25646 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.084+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558e5798f480 0x558e579918d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.084+0000 7fbc22968b80 4 client.25646 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.084+0000 7fbc22968b80 1 -- 172.21.15.131:0/1420193391 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e57948780 con 0x558e5798f480 2024-05-10T05:34:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.084+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558e5798f480 0x558e579918d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.085+0000 7fbbed600700 10 client.25646 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:34:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.085+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558e5798f480 0x558e579918d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e0d3d7d364bc1ba server_cookie=c748afabcceae6fa in_seq=0 out_seq=0 2024-05-10T05:34:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbbed600700 1 -- 172.21.15.131:0/1420193391 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e57948780 con 0x558e5798f480 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbbed600700 10 client.25646 handle_command_reply: tid=0 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbc22968b80 1 client.25646 shutdown 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbc22968b80 1 -- 172.21.15.131:0/1420193391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 msgr2=0x558e57983d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 0x558e57983d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009720 tx=0x7fbbf0009d50 comp rx=0 tx=0).stop 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbc22968b80 1 -- 172.21.15.131:0/1420193391 shutdown_connections 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558e5798f480 0x558e579918d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797b0f0 0x558e5798acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e57979530 0x558e57983850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1420193391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797a7a0 0x558e57983d90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbc22968b80 1 -- 172.21.15.131:0/1420193391 >> 172.21.15.131:0/1420193391 conn(0x558e578dba60 msgr2=0x558e57978060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.086+0000 7fbbed600700 0 client.25646 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.087+0000 7fbc22968b80 1 -- 172.21.15.131:0/1420193391 shutdown_connections 2024-05-10T05:34:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.087+0000 7fbc22968b80 1 -- 172.21.15.131:0/1420193391 wait complete. 2024-05-10T05:34:42.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.087+0000 7fbc22968b80 20 client.25646 trim_cache size 0 max 16384 2024-05-10T05:34:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.113+0000 7fbc22968b80 1 Processor -- start 2024-05-10T05:34:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.113+0000 7fbc22968b80 1 -- start start 2024-05-10T05:34:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e57979530 0x558e57973bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797a7a0 0x558e5798acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797b0f0 0x558e579740f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc22968b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5796e8d0 con 0x558e5797a7a0 2024-05-10T05:34:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc22968b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e57996a70 con 0x558e57979530 2024-05-10T05:34:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc22968b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e57979b70 con 0x558e5797b0f0 2024-05-10T05:34:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc08a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797b0f0 0x558e579740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc08a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797b0f0 0x558e579740f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57080/0 (socket says 172.21.15.131:57080) 2024-05-10T05:34:42.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc08a00700 1 -- 172.21.15.131:0/589701505 learned_addr learned my addr 172.21.15.131:0/589701505 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:42.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc03400700 1 --2- 172.21.15.131:0/589701505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797a7a0 0x558e5798acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.114+0000 7fbc08a00700 1 --2- 172.21.15.131:0/589701505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e57979530 0x558e57973bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.115+0000 7fbc08a00700 1 -- 172.21.15.131:0/589701505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797b0f0 msgr2=0x558e579740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.115+0000 7fbc08a00700 1 --2- 172.21.15.131:0/589701505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797b0f0 0x558e579740f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.115+0000 7fbc08a00700 1 -- 172.21.15.131:0/589701505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797a7a0 msgr2=0x558e5798acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.115+0000 7fbc08a00700 1 --2- 172.21.15.131:0/589701505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797a7a0 0x558e5798acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.115+0000 7fbc08a00700 1 -- 172.21.15.131:0/589701505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5785f560 con 0x558e57979530 2024-05-10T05:34:42.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.115+0000 7fbc08a00700 1 --2- 172.21.15.131:0/589701505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e57979530 0x558e57973bb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc007a30 tx=0x7fbbfc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8aa4e64875a9f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.115+0000 7fbbf9c00700 1 -- 172.21.15.131:0/589701505 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc00e040 con 0x558e57979530 2024-05-10T05:34:42.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.115+0000 7fbbf9c00700 1 -- 172.21.15.131:0/589701505 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbfc02b040 con 0x558e57979530 2024-05-10T05:34:42.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.116+0000 7fbbf9c00700 1 -- 172.21.15.131:0/589701505 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc025460 con 0x558e57979530 2024-05-10T05:34:42.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.117+0000 7fbc22968b80 1 -- 172.21.15.131:0/589701505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e57979530 msgr2=0x558e57973bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.117+0000 7fbc22968b80 1 --2- 172.21.15.131:0/589701505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e57979530 0x558e57973bb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc007a30 tx=0x7fbbfc00b330 comp rx=0 tx=0).stop 2024-05-10T05:34:42.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.117+0000 7fbc22968b80 1 -- 172.21.15.131:0/589701505 shutdown_connections 2024-05-10T05:34:42.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.117+0000 7fbc22968b80 1 --2- 172.21.15.131:0/589701505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797b0f0 0x558e579740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.117+0000 7fbc22968b80 1 --2- 172.21.15.131:0/589701505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e57979530 0x558e57973bb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.117+0000 7fbc22968b80 1 --2- 172.21.15.131:0/589701505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e5797a7a0 0x558e5798acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.117+0000 7fbc22968b80 1 -- 172.21.15.131:0/589701505 >> 172.21.15.131:0/589701505 conn(0x558e578dba60 msgr2=0x558e5797e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.117+0000 7fbc22968b80 1 -- 172.21.15.131:0/589701505 shutdown_connections 2024-05-10T05:34:42.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.117+0000 7fbc22968b80 1 -- 172.21.15.131:0/589701505 wait complete. 2024-05-10T05:34:42.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.118+0000 7fbc22968b80 1 -- start start 2024-05-10T05:34:42.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.118+0000 7fbc22968b80 1 Processor -- start 2024-05-10T05:34:42.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e57979530 0x558e57986450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797a7a0 0x558e57986990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc22968b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797b0f0 0x558e579898d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc22968b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e57989e10 con 0x558e57979530 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc22968b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e57989f80 con 0x558e5797b0f0 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc22968b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5798a140 con 0x558e5797a7a0 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc22968b80 10 client.0 _ll_register_callbacks cb 0x558e5783eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc03400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797a7a0 0x558e57986990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc08a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e57979530 0x558e57986450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc03400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797a7a0 0x558e57986990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57094/0 (socket says 172.21.15.131:57094) 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.119+0000 7fbc03400700 1 -- 172.21.15.131:0/1979428802 learned_addr learned my addr 172.21.15.131:0/1979428802 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.120+0000 7fbc03400700 1 -- 172.21.15.131:0/1979428802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797b0f0 msgr2=0x558e579898d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.120+0000 7fbc03400700 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797b0f0 0x558e579898d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.120+0000 7fbc03400700 1 -- 172.21.15.131:0/1979428802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e57979530 msgr2=0x558e57986450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.120+0000 7fbc03400700 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e57979530 0x558e57986450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.120+0000 7fbc03400700 1 -- 172.21.15.131:0/1979428802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc02d040 con 0x558e5797a7a0 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.120+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e57979530 0x558e57986450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.121+0000 7fbc03400700 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797a7a0 0x558e57986990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0022ff0 tx=0x7fbbf000fbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e26d9f0e0c17de71 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.121+0000 7fbbed600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.121+0000 7fbbed600700 1 -- 172.21.15.131:0/1979428802 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000fe20 con 0x558e5797a7a0 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.121+0000 7fbc22968b80 1 -- 172.21.15.131:0/1979428802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbfc003a00 con 0x558e5797a7a0 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.121+0000 7fbbed600700 1 -- 172.21.15.131:0/1979428802 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbbf0000e40 con 0x558e5797a7a0 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.121+0000 7fbbed600700 1 -- 172.21.15.131:0/1979428802 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbf00050e0 con 0x558e5797a7a0 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.121+0000 7fbc22968b80 10 client.25660 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.121+0000 7fbc22968b80 1 -- 172.21.15.131:0/1979428802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558e5771ecd0 con 0x558e5797a7a0 2024-05-10T05:34:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.122+0000 7fbbed600700 1 -- 172.21.15.131:0/1979428802 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbfc003a00 con 0x558e5797a7a0 2024-05-10T05:34:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.122+0000 7fbbed600700 1 -- 172.21.15.131:0/1979428802 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbbf0003cc0 con 0x558e5797a7a0 2024-05-10T05:34:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.122+0000 7fbc22968b80 10 client.25660 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.122+0000 7fbc22968b80 10 client.25660 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:34:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.122+0000 7fbc22968b80 20 client.25660 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.122+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558e579ea530 0x558e579ec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.122+0000 7fbc22968b80 4 client.25660 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:34:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.122+0000 7fbc22968b80 1 -- 172.21.15.131:0/1979428802 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e57891ec0 con 0x558e579ea530 2024-05-10T05:34:42.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.123+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558e579ea530 0x558e579ec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.124+0000 7fbc08a00700 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558e579ea530 0x558e579ec950 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ceb0dcd0403fec server_cookie=60677d65d316daf8 in_seq=0 out_seq=0 2024-05-10T05:34:42.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.124+0000 7fbbed600700 10 client.25660 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:34:42.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.124+0000 7fbbed600700 1 -- 172.21.15.131:0/1979428802 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558e57891ec0 con 0x558e579ea530 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.124+0000 7fbbed600700 10 client.25660 handle_command_reply: tid=0 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.124+0000 7fbc22968b80 1 client.25660 shutdown 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 1 -- 172.21.15.131:0/1979428802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797a7a0 msgr2=0x558e57986990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797a7a0 0x558e57986990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0022ff0 tx=0x7fbbf000fbb0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 1 -- 172.21.15.131:0/1979428802 shutdown_connections 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558e579ea530 0x558e579ec950 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e57979530 0x558e57986450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e5797b0f0 0x558e579898d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 1 --2- 172.21.15.131:0/1979428802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e5797a7a0 0x558e57986990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 1 -- 172.21.15.131:0/1979428802 >> 172.21.15.131:0/1979428802 conn(0x558e578dba60 msgr2=0x558e5796d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbbed600700 0 client.25660 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:34:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 1 -- 172.21.15.131:0/1979428802 shutdown_connections 2024-05-10T05:34:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 1 -- 172.21.15.131:0/1979428802 wait complete. 2024-05-10T05:34:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.125+0000 7fbc22968b80 20 client.25660 trim_cache size 0 max 16384 2024-05-10T05:34:42.127 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:34:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.126+0000 7fbc09400700 1 -- 172.21.15.131:0/1112396437 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80a11a0 msgr2=0x7fbbe80a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.126+0000 7fbc09400700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80a11a0 0x7fbbe80a36b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40972e0 tx=0x7fbbf40d2530 comp rx=0 tx=0).stop 2024-05-10T05:34:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.126+0000 7fbc09400700 1 -- 172.21.15.131:0/1112396437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041019e0 msgr2=0x7fbc04197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.126+0000 7fbc09400700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041019e0 0x7fbc04197f40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e970 tx=0x7fbbf400ec80 comp rx=0 tx=0).stop 2024-05-10T05:34:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.126+0000 7fbc09400700 1 -- 172.21.15.131:0/1112396437 shutdown_connections 2024-05-10T05:34:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.126+0000 7fbc09400700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbbe80a11a0 0x7fbbe80a36b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.126+0000 7fbc09400700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc041019e0 0x7fbc04197f40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.126+0000 7fbc09400700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc041034a0 0x7fbc0419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.126+0000 7fbc09400700 1 --2- 172.21.15.131:0/1112396437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc04102b00 0x7fbc04198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.126+0000 7fbc09400700 1 -- 172.21.15.131:0/1112396437 >> 172.21.15.131:0/1112396437 conn(0x7fbc040fcfd0 msgr2=0x7fbc04105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.127+0000 7fbc09400700 1 -- 172.21.15.131:0/1112396437 shutdown_connections 2024-05-10T05:34:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.127+0000 7fbc09400700 1 -- 172.21.15.131:0/1112396437 wait complete. 2024-05-10T05:34:42.166 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:42 smithi131 ceph-mon[37264]: pgmap v325: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:42.169 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:34:42.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:42 smithi157 ceph-mon[78427]: pgmap v325: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:42.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:42 smithi142 ceph-mon[77667]: pgmap v325: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:34:42.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.450+0000 7fa489a00700 1 -- 172.21.15.131:0/510246869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4841019e0 msgr2=0x7fa484101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.450+0000 7fa489a00700 1 --2- 172.21.15.131:0/510246869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4841019e0 0x7fa484101df0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa4740099c0 tx=0x7fa474009cd0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.450+0000 7fa489a00700 1 -- 172.21.15.131:0/510246869 shutdown_connections 2024-05-10T05:34:42.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.450+0000 7fa489a00700 1 --2- 172.21.15.131:0/510246869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4841034a0 0x7fa484109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.450+0000 7fa489a00700 1 --2- 172.21.15.131:0/510246869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4841019e0 0x7fa484101df0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.450+0000 7fa489a00700 1 --2- 172.21.15.131:0/510246869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa484102b00 0x7fa484102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.450+0000 7fa489a00700 1 -- 172.21.15.131:0/510246869 >> 172.21.15.131:0/510246869 conn(0x7fa4840fcfd0 msgr2=0x7fa4840ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.450+0000 7fa489a00700 1 -- 172.21.15.131:0/510246869 shutdown_connections 2024-05-10T05:34:42.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.450+0000 7fa489a00700 1 -- 172.21.15.131:0/510246869 wait complete. 2024-05-10T05:34:42.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.451+0000 7fa489a00700 1 Processor -- start 2024-05-10T05:34:42.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa489a00700 1 -- start start 2024-05-10T05:34:42.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa489a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4841019e0 0x7fa484074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa489a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa484102b00 0x7fa484073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa489a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4841034a0 0x7fa484073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa489a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48419bc50 con 0x7fa4841034a0 2024-05-10T05:34:42.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa489a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48419bdc0 con 0x7fa4841019e0 2024-05-10T05:34:42.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa489a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48419bf80 con 0x7fa484102b00 2024-05-10T05:34:42.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa489000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4841034a0 0x7fa484073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa489000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4841034a0 0x7fa484073aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59978/0 (socket says 172.21.15.131:59978) 2024-05-10T05:34:42.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa489000700 1 -- 172.21.15.131:0/630713142 learned_addr learned my addr 172.21.15.131:0/630713142 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:42.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.452+0000 7fa483400700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa484102b00 0x7fa484073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.453+0000 7fa489000700 1 -- 172.21.15.131:0/630713142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa484102b00 msgr2=0x7fa484073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.453+0000 7fa489000700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa484102b00 0x7fa484073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.453+0000 7fa489000700 1 -- 172.21.15.131:0/630713142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4841019e0 msgr2=0x7fa484074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.453+0000 7fa483e00700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4841019e0 0x7fa484074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.453+0000 7fa489000700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4841019e0 0x7fa484074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.453+0000 7fa489000700 1 -- 172.21.15.131:0/630713142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740095f0 con 0x7fa4841034a0 2024-05-10T05:34:42.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.453+0000 7fa483400700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa484102b00 0x7fa484073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:42.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.453+0000 7fa489000700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4841034a0 0x7fa484073aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00ef30 tx=0x7fa47c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.453+0000 7fa480c00700 1 -- 172.21.15.131:0/630713142 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c018070 con 0x7fa4841034a0 2024-05-10T05:34:42.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.454+0000 7fa480c00700 1 -- 172.21.15.131:0/630713142 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa47c00f040 con 0x7fa4841034a0 2024-05-10T05:34:42.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.454+0000 7fa480c00700 1 -- 172.21.15.131:0/630713142 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c003dc0 con 0x7fa4841034a0 2024-05-10T05:34:42.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.454+0000 7fa489a00700 1 -- 172.21.15.131:0/630713142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48419c200 con 0x7fa4841034a0 2024-05-10T05:34:42.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.454+0000 7fa489a00700 1 -- 172.21.15.131:0/630713142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48419c6c0 con 0x7fa4841034a0 2024-05-10T05:34:42.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.459+0000 7fa480c00700 1 -- 172.21.15.131:0/630713142 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa47c00caa0 con 0x7fa4841034a0 2024-05-10T05:34:42.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.460+0000 7fa4a2ea0b80 1 Processor -- start 2024-05-10T05:34:42.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.460+0000 7fa4a2ea0b80 1 -- start start 2024-05-10T05:34:42.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.461+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f32fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.461+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f32fcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.461+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f33039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.461+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f3222150 con 0x5590f32fc7a0 2024-05-10T05:34:42.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.461+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f321fb10 con 0x5590f32fb530 2024-05-10T05:34:42.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.461+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f321b400 con 0x5590f32fd0f0 2024-05-10T05:34:42.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.461+0000 7fa483400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f33039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.461+0000 7fa483400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f33039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57124/0 (socket says 172.21.15.131:57124) 2024-05-10T05:34:42.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.461+0000 7fa483400700 1 -- 172.21.15.131:0/455723256 learned_addr learned my addr 172.21.15.131:0/455723256 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:42.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.461+0000 7fa483400700 1 --2- 172.21.15.131:0/455723256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f32fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.462+0000 7fa483e00700 1 --2- 172.21.15.131:0/455723256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f32fcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.462+0000 7fa483400700 1 -- 172.21.15.131:0/455723256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 msgr2=0x5590f32fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.462+0000 7fa483400700 1 --2- 172.21.15.131:0/455723256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f32fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.462+0000 7fa483400700 1 -- 172.21.15.131:0/455723256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 msgr2=0x5590f32fcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.462+0000 7fa483400700 1 --2- 172.21.15.131:0/455723256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f32fcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.462+0000 7fa483400700 1 -- 172.21.15.131:0/455723256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590f309c3b0 con 0x5590f32fd0f0 2024-05-10T05:34:42.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.462+0000 7fa483e00700 1 --2- 172.21.15.131:0/455723256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f32fcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:34:42.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.462+0000 7fa480c00700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4680a1220 0x7fa4680a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.462+0000 7fa483400700 1 --2- 172.21.15.131:0/455723256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f33039f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa470014a50 tx=0x7fa470007590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd4e7bb962038616 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.462+0000 7fa480c00700 1 -- 172.21.15.131:0/630713142 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa47c0cc560 con 0x7fa4841034a0 2024-05-10T05:34:42.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.463+0000 7fa483e00700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4680a1220 0x7fa4680a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.463+0000 7fa479600700 1 -- 172.21.15.131:0/455723256 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4700142d0 con 0x5590f32fd0f0 2024-05-10T05:34:42.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.463+0000 7fa479600700 1 -- 172.21.15.131:0/455723256 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa47001a030 con 0x5590f32fd0f0 2024-05-10T05:34:42.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.463+0000 7fa479600700 1 -- 172.21.15.131:0/455723256 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470004d30 con 0x5590f32fd0f0 2024-05-10T05:34:42.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.464+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/455723256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 msgr2=0x5590f33039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.464+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/455723256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f33039f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa470014a50 tx=0x7fa470007590 comp rx=0 tx=0).stop 2024-05-10T05:34:42.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.464+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/455723256 shutdown_connections 2024-05-10T05:34:42.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.464+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/455723256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f33039f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.464+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/455723256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f32fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.464+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/455723256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f32fcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.464+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/455723256 >> 172.21.15.131:0/455723256 conn(0x5590f325da60 msgr2=0x5590f32f6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.464+0000 7fa483e00700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4680a1220 0x7fa4680a3730 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa474004800 tx=0x7fa474019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.464+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/455723256 shutdown_connections 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.464+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/455723256 wait complete. 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.465+0000 7fa4a2ea0b80 1 -- start start 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.465+0000 7fa4a2ea0b80 1 Processor -- start 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f33058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f3305e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f330cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f31b7560 con 0x5590f32fc7a0 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f31b7200 con 0x5590f32fb530 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f309e6a0 con 0x5590f32fd0f0 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa4a2ea0b80 10 client.0 _ll_register_callbacks cb 0x5590f31c0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:42.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa489000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f330cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa489000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f330cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57126/0 (socket says 172.21.15.131:57126) 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa489000700 1 -- 172.21.15.131:0/2465349201 learned_addr learned my addr 172.21.15.131:0/2465349201 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa483400700 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f3305e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa483e00700 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f33058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.466+0000 7fa489000700 1 -- 172.21.15.131:0/2465349201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 msgr2=0x5590f33058c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa489000700 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f33058c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa489000700 1 -- 172.21.15.131:0/2465349201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 msgr2=0x5590f3305e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa489000700 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f3305e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa489000700 1 -- 172.21.15.131:0/2465349201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa470014730 con 0x5590f32fd0f0 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa489000700 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f330cd60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0090a0 tx=0x7fa47c01b600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f9cfdbe796bf68 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa46e000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa483e00700 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f33058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa46e000700 1 -- 172.21.15.131:0/2465349201 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0999e0 con 0x5590f32fd0f0 2024-05-10T05:34:42.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa46e000700 1 -- 172.21.15.131:0/2465349201 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa47c09d810 con 0x5590f32fd0f0 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2465349201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590f3174270 con 0x5590f32fd0f0 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.467+0000 7fa46e000700 1 -- 172.21.15.131:0/2465349201 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa47c0a1dd0 con 0x5590f32fd0f0 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.468+0000 7fa4a2ea0b80 10 client.25672 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.468+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2465349201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5590f3048b60 con 0x5590f32fd0f0 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.468+0000 7fa46e000700 1 -- 172.21.15.131:0/2465349201 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa47c0b6080 con 0x5590f32fd0f0 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.468+0000 7fa46e000700 1 -- 172.21.15.131:0/2465349201 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa47c0cc0f0 con 0x5590f32fd0f0 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.468+0000 7fa4a2ea0b80 10 client.25672 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.468+0000 7fa4a2ea0b80 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.468+0000 7fa4a2ea0b80 20 client.25672 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.468+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590f3311510 0x5590f3313960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.469+0000 7fa4a2ea0b80 4 client.25672 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.469+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2465349201 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590f32ca780 con 0x5590f3311510 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.469+0000 7fa483e00700 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590f3311510 0x5590f3313960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.470+0000 7fa483e00700 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590f3311510 0x5590f3313960 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f79c456b8ab5e3b2 server_cookie=e9030189e4abd10a in_seq=0 out_seq=0 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.470+0000 7fa46e000700 10 client.25672 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.470+0000 7fa46e000700 1 -- 172.21.15.131:0/2465349201 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590f32ca780 con 0x5590f3311510 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.470+0000 7fa46e000700 10 client.25672 handle_command_reply: tid=0 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.470+0000 7fa4a2ea0b80 1 client.25672 shutdown 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.471+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2465349201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 msgr2=0x5590f330cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.471+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f330cd60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0090a0 tx=0x7fa47c01b600 comp rx=0 tx=0).stop 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.471+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2465349201 shutdown_connections 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.471+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590f3311510 0x5590f3313960 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.471+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f330cd60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.471+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f33058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.471+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/2465349201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f3305e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.471+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2465349201 >> 172.21.15.131:0/2465349201 conn(0x5590f325da60 msgr2=0x5590f32fa060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.471+0000 7fa46e000700 0 client.25672 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:42.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.472+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2465349201 shutdown_connections 2024-05-10T05:34:42.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.472+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2465349201 wait complete. 2024-05-10T05:34:42.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.473+0000 7fa4a2ea0b80 20 client.25672 trim_cache size 0 max 16384 2024-05-10T05:34:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.499+0000 7fa4a2ea0b80 1 Processor -- start 2024-05-10T05:34:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.499+0000 7fa4a2ea0b80 1 -- start start 2024-05-10T05:34:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.499+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f330d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.499+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f330cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.499+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f3304380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.499+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f32f2340 con 0x5590f32fc7a0 2024-05-10T05:34:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.499+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f32fbdc0 con 0x5590f32fb530 2024-05-10T05:34:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.499+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f33048c0 con 0x5590f32fd0f0 2024-05-10T05:34:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.500+0000 7fa483400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f330cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.500+0000 7fa489000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f3304380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.500+0000 7fa483400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f330cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60012/0 (socket says 172.21.15.131:60012) 2024-05-10T05:34:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.500+0000 7fa483400700 1 -- 172.21.15.131:0/2170661230 learned_addr learned my addr 172.21.15.131:0/2170661230 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.500+0000 7fa483e00700 1 --2- 172.21.15.131:0/2170661230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f330d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.500+0000 7fa489000700 1 -- 172.21.15.131:0/2170661230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 msgr2=0x5590f330d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.500+0000 7fa489000700 1 --2- 172.21.15.131:0/2170661230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f330d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.500+0000 7fa489000700 1 -- 172.21.15.131:0/2170661230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 msgr2=0x5590f330cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.500+0000 7fa489000700 1 --2- 172.21.15.131:0/2170661230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f330cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.500+0000 7fa489000700 1 -- 172.21.15.131:0/2170661230 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590f31e1560 con 0x5590f32fd0f0 2024-05-10T05:34:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.501+0000 7fa489000700 1 --2- 172.21.15.131:0/2170661230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f3304380 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00c850 tx=0x7fa47c097f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddeb9d18b782a85a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.501+0000 7fa47a000700 1 -- 172.21.15.131:0/2170661230 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c09d810 con 0x5590f32fd0f0 2024-05-10T05:34:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.501+0000 7fa47a000700 1 -- 172.21.15.131:0/2170661230 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590f3174270 con 0x5590f32fd0f0 2024-05-10T05:34:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.501+0000 7fa47a000700 1 -- 172.21.15.131:0/2170661230 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0cc0f0 con 0x5590f32fd0f0 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.502+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2170661230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 msgr2=0x5590f3304380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.502+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/2170661230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f3304380 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00c850 tx=0x7fa47c097f90 comp rx=0 tx=0).stop 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.502+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2170661230 shutdown_connections 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.502+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/2170661230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fd0f0 0x5590f3304380 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.502+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/2170661230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fb530 0x5590f330d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.502+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/2170661230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fc7a0 0x5590f330cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.502+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2170661230 >> 172.21.15.131:0/2170661230 conn(0x5590f325da60 msgr2=0x5590f3303c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.502+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2170661230 shutdown_connections 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.502+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/2170661230 wait complete. 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.503+0000 7fa4a2ea0b80 1 -- start start 2024-05-10T05:34:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.503+0000 7fa4a2ea0b80 1 Processor -- start 2024-05-10T05:34:42.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fb530 0x5590f330a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fc7a0 0x5590f330ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa4a2ea0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fd0f0 0x5590f330bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f330c060 con 0x5590f32fd0f0 2024-05-10T05:34:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f330c1d0 con 0x5590f32fc7a0 2024-05-10T05:34:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa4a2ea0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f330c390 con 0x5590f32fb530 2024-05-10T05:34:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa4a2ea0b80 10 client.0 _ll_register_callbacks cb 0x5590f31c0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa483e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fb530 0x5590f330a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa483e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fb530 0x5590f330a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:57150/0 (socket says 172.21.15.131:57150) 2024-05-10T05:34:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa483e00700 1 -- 172.21.15.131:0/1677571002 learned_addr learned my addr 172.21.15.131:0/1677571002 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.505+0000 7fa483400700 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fc7a0 0x5590f330ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.506+0000 7fa483e00700 1 -- 172.21.15.131:0/1677571002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fc7a0 msgr2=0x5590f330ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.506+0000 7fa483e00700 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fc7a0 0x5590f330ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.506+0000 7fa483e00700 1 -- 172.21.15.131:0/1677571002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fd0f0 msgr2=0x5590f330bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.506+0000 7fa489000700 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fd0f0 0x5590f330bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.506+0000 7fa483e00700 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fd0f0 0x5590f330bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.506+0000 7fa483e00700 1 -- 172.21.15.131:0/1677571002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590f3048b60 con 0x5590f32fb530 2024-05-10T05:34:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.506+0000 7fa483400700 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fc7a0 0x5590f330ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.506+0000 7fa489000700 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fd0f0 0x5590f330bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:34:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.507+0000 7fa483e00700 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fb530 0x5590f330a700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa4740099c0 tx=0x7fa474009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45afdbb7224575fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.507+0000 7fa46e000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:34:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.507+0000 7fa46e000700 1 -- 172.21.15.131:0/1677571002 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474035070 con 0x5590f32fb530 2024-05-10T05:34:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.507+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/1677571002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa47001a030 con 0x5590f32fb530 2024-05-10T05:34:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.507+0000 7fa46e000700 1 -- 172.21.15.131:0/1677571002 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa474030730 con 0x5590f32fb530 2024-05-10T05:34:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.507+0000 7fa46e000700 1 -- 172.21.15.131:0/1677571002 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa474027040 con 0x5590f32fb530 2024-05-10T05:34:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.507+0000 7fa4a2ea0b80 10 client.25678 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.507+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/1677571002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5590f30a0cd0 con 0x5590f32fb530 2024-05-10T05:34:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.508+0000 7fa46e000700 1 -- 172.21.15.131:0/1677571002 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa47001a030 con 0x5590f32fb530 2024-05-10T05:34:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.509+0000 7fa46e000700 1 -- 172.21.15.131:0/1677571002 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa47403a470 con 0x5590f32fb530 2024-05-10T05:34:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.509+0000 7fa4a2ea0b80 10 client.25678 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.509+0000 7fa4a2ea0b80 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.509+0000 7fa4a2ea0b80 20 client.25678 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.509+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590f3368470 0x5590f336a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.509+0000 7fa4a2ea0b80 4 client.25678 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:34:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.509+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/1677571002 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5590f3213ec0 con 0x5590f3368470 2024-05-10T05:34:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.509+0000 7fa483400700 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590f3368470 0x5590f336a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.510+0000 7fa46e000700 10 client.25678 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.510+0000 7fa483400700 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590f3368470 0x5590f336a890 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72262da40865dabd server_cookie=e63ddabcb4d36e4b in_seq=0 out_seq=0 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.510+0000 7fa46e000700 1 -- 172.21.15.131:0/1677571002 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5590f3213ec0 con 0x5590f3368470 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.510+0000 7fa46e000700 10 client.25678 handle_command_reply: tid=0 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.510+0000 7fa4a2ea0b80 1 client.25678 shutdown 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.510+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/1677571002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fb530 msgr2=0x5590f330a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.510+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fb530 0x5590f330a700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa4740099c0 tx=0x7fa474009310 comp rx=0 tx=0).stop 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.511+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/1677571002 shutdown_connections 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.511+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590f3368470 0x5590f336a890 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.511+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590f32fd0f0 0x5590f330bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.511+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f32fb530 0x5590f330a700 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.511+0000 7fa46e000700 0 client.25678 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.511+0000 7fa4a2ea0b80 1 --2- 172.21.15.131:0/1677571002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f32fc7a0 0x5590f330ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.511+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/1677571002 >> 172.21.15.131:0/1677571002 conn(0x5590f325da60 msgr2=0x5590f32f5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.511+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/1677571002 shutdown_connections 2024-05-10T05:34:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.511+0000 7fa4a2ea0b80 1 -- 172.21.15.131:0/1677571002 wait complete. 2024-05-10T05:34:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.511+0000 7fa4a2ea0b80 20 client.25678 trim_cache size 0 max 16384 2024-05-10T05:34:42.513 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:34:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 -- 172.21.15.131:0/630713142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4680a1220 msgr2=0x7fa4680a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4680a1220 0x7fa4680a3730 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa474004800 tx=0x7fa474019040 comp rx=0 tx=0).stop 2024-05-10T05:34:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 -- 172.21.15.131:0/630713142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4841034a0 msgr2=0x7fa484073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4841034a0 0x7fa484073aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00ef30 tx=0x7fa47c00c450 comp rx=0 tx=0).stop 2024-05-10T05:34:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 -- 172.21.15.131:0/630713142 shutdown_connections 2024-05-10T05:34:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4680a1220 0x7fa4680a3730 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4841034a0 0x7fa484073aa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4841019e0 0x7fa484074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 --2- 172.21.15.131:0/630713142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa484102b00 0x7fa484073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 -- 172.21.15.131:0/630713142 >> 172.21.15.131:0/630713142 conn(0x7fa4840fcfd0 msgr2=0x7fa484105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:42.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 -- 172.21.15.131:0/630713142 shutdown_connections 2024-05-10T05:34:42.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:42.512+0000 7fa489a00700 1 -- 172.21.15.131:0/630713142 wait complete. 2024-05-10T05:34:43.540 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:34:43.541 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:34:43.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.832+0000 7f9f49c00700 1 -- 172.21.15.131:0/3753143981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f44102d00 msgr2=0x7f9f44103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:43.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.832+0000 7f9f49c00700 1 --2- 172.21.15.131:0/3753143981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f44102d00 0x7f9f44103110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009920 tx=0x7f9f30009c30 comp rx=0 tx=0).stop 2024-05-10T05:34:43.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.832+0000 7f9f49c00700 1 -- 172.21.15.131:0/3753143981 shutdown_connections 2024-05-10T05:34:43.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.832+0000 7f9f49c00700 1 --2- 172.21.15.131:0/3753143981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f441036a0 0x7f9f44109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.832+0000 7f9f49c00700 1 --2- 172.21.15.131:0/3753143981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f44101be0 0x7f9f44101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.832+0000 7f9f49c00700 1 --2- 172.21.15.131:0/3753143981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f44102d00 0x7f9f44103110 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.832+0000 7f9f49c00700 1 -- 172.21.15.131:0/3753143981 >> 172.21.15.131:0/3753143981 conn(0x7f9f440fd1d0 msgr2=0x7f9f440ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:43.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.832+0000 7f9f49c00700 1 -- 172.21.15.131:0/3753143981 shutdown_connections 2024-05-10T05:34:43.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.833+0000 7f9f49c00700 1 -- 172.21.15.131:0/3753143981 wait complete. 2024-05-10T05:34:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.834+0000 7f9f49c00700 1 Processor -- start 2024-05-10T05:34:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.834+0000 7f9f49c00700 1 -- start start 2024-05-10T05:34:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.834+0000 7f9f49c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f44101be0 0x7f9f44197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.834+0000 7f9f49c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f44102d00 0x7f9f44198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.834+0000 7f9f49c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f441036a0 0x7f9f4419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.834+0000 7f9f49c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4419fa00 con 0x7f9f44102d00 2024-05-10T05:34:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.834+0000 7f9f49c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4419fb70 con 0x7f9f44101be0 2024-05-10T05:34:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.834+0000 7f9f49c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4419fd30 con 0x7f9f441036a0 2024-05-10T05:34:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.835+0000 7f9f42000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f44102d00 0x7f9f44198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.835+0000 7f9f42000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f44102d00 0x7f9f44198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41980/0 (socket says 172.21.15.131:41980) 2024-05-10T05:34:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.835+0000 7f9f42000700 1 -- 172.21.15.131:0/3117719863 learned_addr learned my addr 172.21.15.131:0/3117719863 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.835+0000 7f9f43400700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f441036a0 0x7f9f4419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.835+0000 7f9f42000700 1 -- 172.21.15.131:0/3117719863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f441036a0 msgr2=0x7f9f4419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.835+0000 7f9f42000700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f441036a0 0x7f9f4419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.835+0000 7f9f42000700 1 -- 172.21.15.131:0/3117719863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f44101be0 msgr2=0x7f9f44197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.835+0000 7f9f42000700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f44101be0 0x7f9f44197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.835+0000 7f9f42000700 1 -- 172.21.15.131:0/3117719863 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f30009520 con 0x7f9f44102d00 2024-05-10T05:34:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.835+0000 7f9f43400700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f441036a0 0x7f9f4419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.836+0000 7f9f42000700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f44102d00 0x7f9f44198460 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f300098f0 tx=0x7f9f3000bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.836+0000 7f9f3b400700 1 -- 172.21.15.131:0/3117719863 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30005450 con 0x7f9f44102d00 2024-05-10T05:34:43.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.836+0000 7f9f49c00700 1 -- 172.21.15.131:0/3117719863 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4419ffb0 con 0x7f9f44102d00 2024-05-10T05:34:43.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.836+0000 7f9f3b400700 1 -- 172.21.15.131:0/3117719863 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f300055b0 con 0x7f9f44102d00 2024-05-10T05:34:43.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.836+0000 7f9f3b400700 1 -- 172.21.15.131:0/3117719863 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30017820 con 0x7f9f44102d00 2024-05-10T05:34:43.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.836+0000 7f9f49c00700 1 -- 172.21.15.131:0/3117719863 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f441a0380 con 0x7f9f44102d00 2024-05-10T05:34:43.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.842+0000 7f9f61bdeb80 1 Processor -- start 2024-05-10T05:34:43.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.843+0000 7f9f61bdeb80 1 -- start start 2024-05-10T05:34:43.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.843+0000 7f9f3b400700 1 -- 172.21.15.131:0/3117719863 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9f30003680 con 0x7f9f44102d00 2024-05-10T05:34:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.843+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff651f0 0x562e4ff65640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.843+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5b380 0x562e4ff5b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.843+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 0x562e4ff6c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.843+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4fe821d0 con 0x562e4ff5bcd0 2024-05-10T05:34:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.843+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4fe7fb90 con 0x562e4ff651f0 2024-05-10T05:34:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.843+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4fe7b480 con 0x562e4ff5b380 2024-05-10T05:34:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.844+0000 7f9f42000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 0x562e4ff6c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.844+0000 7f9f42a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5b380 0x562e4ff5b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.844+0000 7f9f42a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5b380 0x562e4ff5b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42958/0 (socket says 172.21.15.131:42958) 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.844+0000 7f9f42a00700 1 -- 172.21.15.131:0/1876179848 learned_addr learned my addr 172.21.15.131:0/1876179848 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.844+0000 7f9f43400700 1 --2- 172.21.15.131:0/1876179848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff651f0 0x562e4ff65640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.844+0000 7f9f42a00700 1 -- 172.21.15.131:0/1876179848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff651f0 msgr2=0x562e4ff65640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.844+0000 7f9f42a00700 1 --2- 172.21.15.131:0/1876179848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff651f0 0x562e4ff65640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.845+0000 7f9f42a00700 1 -- 172.21.15.131:0/1876179848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 msgr2=0x562e4ff6c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.845+0000 7f9f42a00700 1 --2- 172.21.15.131:0/1876179848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 0x562e4ff6c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.845+0000 7f9f42a00700 1 -- 172.21.15.131:0/1876179848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e4fcfc430 con 0x562e4ff5b380 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.845+0000 7f9f42000700 1 --2- 172.21.15.131:0/1876179848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 0x562e4ff6c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.845+0000 7f9f42a00700 1 --2- 172.21.15.131:0/1876179848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5b380 0x562e4ff5b790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009b80 tx=0x7f9f34009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec52884e0b9f1517 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.845+0000 7f9f2fe00700 1 -- 172.21.15.131:0/1876179848 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34009700 con 0x562e4ff5b380 2024-05-10T05:34:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.845+0000 7f9f2fe00700 1 -- 172.21.15.131:0/1876179848 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f34010430 con 0x562e4ff5b380 2024-05-10T05:34:43.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.845+0000 7f9f2fe00700 1 -- 172.21.15.131:0/1876179848 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34018660 con 0x562e4ff5b380 2024-05-10T05:34:43.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.846+0000 7f9f3b400700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9f280a1240 0x7f9f280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.846+0000 7f9f3b400700 1 -- 172.21.15.131:0/3117719863 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9f300cdff0 con 0x7f9f44102d00 2024-05-10T05:34:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.847+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/1876179848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5b380 msgr2=0x562e4ff5b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.847+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/1876179848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5b380 0x562e4ff5b790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009b80 tx=0x7f9f34009e90 comp rx=0 tx=0).stop 2024-05-10T05:34:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.847+0000 7f9f43400700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9f280a1240 0x7f9f280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.847+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/1876179848 shutdown_connections 2024-05-10T05:34:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.847+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/1876179848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 0x562e4ff6c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.847+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/1876179848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff651f0 0x562e4ff65640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.847+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/1876179848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5b380 0x562e4ff5b790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.847+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/1876179848 >> 172.21.15.131:0/1876179848 conn(0x562e4febdae0 msgr2=0x562e4ff56060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:43.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.847+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/1876179848 shutdown_connections 2024-05-10T05:34:43.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.847+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/1876179848 wait complete. 2024-05-10T05:34:43.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.848+0000 7f9f43400700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9f280a1240 0x7f9f280a3750 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x562e4ff5c6b0 tx=0x7f9f3c00b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:43.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.848+0000 7f9f61bdeb80 1 -- start start 2024-05-10T05:34:43.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.849+0000 7f9f61bdeb80 1 Processor -- start 2024-05-10T05:34:43.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.849+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5b380 0x562e4ff6df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.849+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5bcd0 0x562e4ff6e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.849+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff651f0 0x562e4ff75410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.849+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4fe175e0 con 0x562e4ff5b380 2024-05-10T05:34:43.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.849+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4fe17280 con 0x562e4ff651f0 2024-05-10T05:34:43.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.849+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4fcfe720 con 0x562e4ff5bcd0 2024-05-10T05:34:43.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.849+0000 7f9f61bdeb80 10 client.0 _ll_register_callbacks cb 0x562e4fe20bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:43.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f42a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5b380 0x562e4ff6df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f42a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5b380 0x562e4ff6df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41992/0 (socket says 172.21.15.131:41992) 2024-05-10T05:34:43.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f42a00700 1 -- 172.21.15.131:0/2073167736 learned_addr learned my addr 172.21.15.131:0/2073167736 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:43.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f43400700 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff651f0 0x562e4ff75410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f42000700 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5bcd0 0x562e4ff6e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f42a00700 1 -- 172.21.15.131:0/2073167736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5bcd0 msgr2=0x562e4ff6e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:43.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f42a00700 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5bcd0 0x562e4ff6e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f42a00700 1 -- 172.21.15.131:0/2073167736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff651f0 msgr2=0x562e4ff75410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:43.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f42a00700 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff651f0 0x562e4ff75410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f42a00700 1 -- 172.21.15.131:0/2073167736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f441a0380 con 0x562e4ff5b380 2024-05-10T05:34:43.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.850+0000 7f9f42000700 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5bcd0 0x562e4ff6e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:34:43.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.851+0000 7f9f42a00700 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5b380 0x562e4ff6df70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009350 tx=0x7f9f3400ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ce040aa79fd39b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:43.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.851+0000 7f9f2cc00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:34:43.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.851+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2073167736 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34009700 con 0x562e4ff5b380 2024-05-10T05:34:43.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.851+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2073167736 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9f34014070 con 0x562e4ff5b380 2024-05-10T05:34:43.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.851+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2073167736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e4fdd42f0 con 0x562e4ff5b380 2024-05-10T05:34:43.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2073167736 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f34010ba0 con 0x562e4ff5b380 2024-05-10T05:34:43.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f61bdeb80 10 client.15666 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:43.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2073167736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9f34009860 con 0x562e4ff5b380 2024-05-10T05:34:43.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2073167736 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e4fdd42f0 con 0x562e4ff5b380 2024-05-10T05:34:43.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2073167736 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9f34020040 con 0x562e4ff5b380 2024-05-10T05:34:43.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f61bdeb80 10 client.15666 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:43.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f61bdeb80 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:43.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f61bdeb80 20 client.15666 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:43.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562e4ff79a50 0x562e4ff7bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f61bdeb80 4 client.15666 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:43.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.852+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2073167736 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e4ff2a800 con 0x562e4ff79a50 2024-05-10T05:34:43.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.854+0000 7f9f42000700 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562e4ff79a50 0x562e4ff7bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.854+0000 7f9f42000700 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562e4ff79a50 0x562e4ff7bea0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b77432477dd670 server_cookie=880f8dd8ca172130 in_seq=0 out_seq=0 2024-05-10T05:34:43.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.854+0000 7f9f2cc00700 10 client.15666 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:43.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.855+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2073167736 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e4ff2a800 con 0x562e4ff79a50 2024-05-10T05:34:43.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.855+0000 7f9f2cc00700 10 client.15666 handle_command_reply: tid=0 2024-05-10T05:34:43.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.855+0000 7f9f61bdeb80 1 client.15666 shutdown 2024-05-10T05:34:43.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2073167736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5b380 msgr2=0x562e4ff6df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:43.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5b380 0x562e4ff6df70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009350 tx=0x7f9f3400ebe0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2073167736 shutdown_connections 2024-05-10T05:34:43.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562e4ff79a50 0x562e4ff7bea0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5b380 0x562e4ff6df70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff651f0 0x562e4ff75410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2073167736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff5bcd0 0x562e4ff6e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2073167736 >> 172.21.15.131:0/2073167736 conn(0x562e4febdae0 msgr2=0x562e4ff56030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:43.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f2cc00700 0 client.15666 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:43.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2073167736 shutdown_connections 2024-05-10T05:34:43.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2073167736 wait complete. 2024-05-10T05:34:43.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.856+0000 7f9f61bdeb80 20 client.15666 trim_cache size 0 max 16384 2024-05-10T05:34:43.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.892+0000 7f9f61bdeb80 1 Processor -- start 2024-05-10T05:34:43.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.892+0000 7f9f61bdeb80 1 -- start start 2024-05-10T05:34:43.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.893+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4febdae0 0x562e4ff68d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.893+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff5bcd0 0x562e4ff753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.893+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 0x562e4ff69250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.893+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4ff687f0 con 0x562e4febdae0 2024-05-10T05:34:43.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.893+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4ff6c420 con 0x562e4ff5bcd0 2024-05-10T05:34:43.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.893+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4ff52200 con 0x562e4ff651f0 2024-05-10T05:34:43.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.893+0000 7f9f42000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff5bcd0 0x562e4ff753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.893+0000 7f9f42a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4febdae0 0x562e4ff68d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.894+0000 7f9f42000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff5bcd0 0x562e4ff753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45432/0 (socket says 172.21.15.131:45432) 2024-05-10T05:34:43.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.894+0000 7f9f42000700 1 -- 172.21.15.131:0/3989620075 learned_addr learned my addr 172.21.15.131:0/3989620075 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:43.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.894+0000 7f9f43400700 1 --2- 172.21.15.131:0/3989620075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 0x562e4ff69250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.894+0000 7f9f42000700 1 -- 172.21.15.131:0/3989620075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 msgr2=0x562e4ff69250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:43.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.894+0000 7f9f42000700 1 --2- 172.21.15.131:0/3989620075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 0x562e4ff69250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.894+0000 7f9f42000700 1 -- 172.21.15.131:0/3989620075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4febdae0 msgr2=0x562e4ff68d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:43.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.894+0000 7f9f42000700 1 --2- 172.21.15.131:0/3989620075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4febdae0 0x562e4ff68d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.894+0000 7f9f42000700 1 -- 172.21.15.131:0/3989620075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e4fca8be0 con 0x562e4ff5bcd0 2024-05-10T05:34:43.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.894+0000 7f9f42a00700 1 --2- 172.21.15.131:0/3989620075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4febdae0 0x562e4ff68d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:43.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.894+0000 7f9f42000700 1 --2- 172.21.15.131:0/3989620075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff5bcd0 0x562e4ff753d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f3009b170 tx=0x7f9f300d19f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ba5c4d851db236 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:43.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.895+0000 7f9f38c00700 1 -- 172.21.15.131:0/3989620075 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f300977b0 con 0x562e4ff5bcd0 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.895+0000 7f9f38c00700 1 -- 172.21.15.131:0/3989620075 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f3009b240 con 0x562e4ff5bcd0 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.895+0000 7f9f38c00700 1 -- 172.21.15.131:0/3989620075 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3009b500 con 0x562e4ff5bcd0 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.896+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/3989620075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff5bcd0 msgr2=0x562e4ff753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.896+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/3989620075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff5bcd0 0x562e4ff753d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f3009b170 tx=0x7f9f300d19f0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.896+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/3989620075 shutdown_connections 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.896+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/3989620075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4febdae0 0x562e4ff68d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.896+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/3989620075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 0x562e4ff69250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.896+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/3989620075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4ff5bcd0 0x562e4ff753d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.896+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/3989620075 >> 172.21.15.131:0/3989620075 conn(0x562e4ff524d0 msgr2=0x562e4ff6c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.896+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/3989620075 shutdown_connections 2024-05-10T05:34:43.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.896+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/3989620075 wait complete. 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.897+0000 7f9f61bdeb80 1 -- start start 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.897+0000 7f9f61bdeb80 1 Processor -- start 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.898+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4febdae0 0x562e4ff753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.898+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 0x562e4ff73a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.898+0000 7f9f61bdeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 0x562e4ff740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.898+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4ff74720 con 0x562e4ff5bcd0 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.898+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4ff74890 con 0x562e4febdae0 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.898+0000 7f9f61bdeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4ffcfeb0 con 0x562e4ff651f0 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.898+0000 7f9f61bdeb80 10 client.0 _ll_register_callbacks cb 0x562e4fe20bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.898+0000 7f9f42a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4febdae0 0x562e4ff753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.898+0000 7f9f42a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4febdae0 0x562e4ff753a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45444/0 (socket says 172.21.15.131:45444) 2024-05-10T05:34:43.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.898+0000 7f9f42a00700 1 -- 172.21.15.131:0/2017199389 learned_addr learned my addr 172.21.15.131:0/2017199389 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:43.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.899+0000 7f9f43400700 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 0x562e4ff740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.899+0000 7f9f42000700 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 0x562e4ff73a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.899+0000 7f9f43400700 1 -- 172.21.15.131:0/2017199389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4febdae0 msgr2=0x562e4ff753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:43.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.899+0000 7f9f43400700 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4febdae0 0x562e4ff753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.899+0000 7f9f43400700 1 -- 172.21.15.131:0/2017199389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 msgr2=0x562e4ff73a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:43.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.899+0000 7f9f43400700 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 0x562e4ff73a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.899+0000 7f9f43400700 1 -- 172.21.15.131:0/2017199389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f34009860 con 0x562e4ff651f0 2024-05-10T05:34:43.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.899+0000 7f9f43400700 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 0x562e4ff740b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009fd0 tx=0x7f9f3c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd50e4fcbe92c994 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:43.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.899+0000 7f9f2cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:34:43.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.900+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2017199389 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c004db0 con 0x562e4ff651f0 2024-05-10T05:34:43.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.900+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2017199389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f34010430 con 0x562e4ff651f0 2024-05-10T05:34:43.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.900+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2017199389 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9f3c00e700 con 0x562e4ff651f0 2024-05-10T05:34:43.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.901+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2017199389 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f3c00e8c0 con 0x562e4ff651f0 2024-05-10T05:34:43.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.901+0000 7f9f61bdeb80 10 client.25693 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:43.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.901+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2017199389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9f4419ffb0 con 0x562e4ff651f0 2024-05-10T05:34:43.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.901+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2017199389 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f34010430 con 0x562e4ff651f0 2024-05-10T05:34:43.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.901+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2017199389 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9f3c016b70 con 0x562e4ff651f0 2024-05-10T05:34:43.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.902+0000 7f9f61bdeb80 10 client.25693 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:43.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.902+0000 7f9f61bdeb80 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:43.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.902+0000 7f9f61bdeb80 20 client.25693 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:43.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.902+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562e4ffd18d0 0x562e4ffd3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:43.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.902+0000 7f9f61bdeb80 4 client.25693 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:34:43.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.902+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2017199389 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562e4fe73f40 con 0x562e4ffd18d0 2024-05-10T05:34:43.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.902+0000 7f9f42a00700 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562e4ffd18d0 0x562e4ffd3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:43.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.903+0000 7f9f2cc00700 10 client.25693 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:43.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.903+0000 7f9f42a00700 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562e4ffd18d0 0x562e4ffd3cf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e1580c593543f7 server_cookie=1791f7d982064b33 in_seq=0 out_seq=0 2024-05-10T05:34:43.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.904+0000 7f9f2cc00700 1 -- 172.21.15.131:0/2017199389 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562e4fe73f40 con 0x562e4ffd18d0 2024-05-10T05:34:43.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.904+0000 7f9f2cc00700 10 client.25693 handle_command_reply: tid=0 2024-05-10T05:34:43.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.904+0000 7f9f61bdeb80 1 client.25693 shutdown 2024-05-10T05:34:43.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2017199389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 msgr2=0x562e4ff740b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:43.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 0x562e4ff740b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009fd0 tx=0x7f9f3c004b40 comp rx=0 tx=0).stop 2024-05-10T05:34:43.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2017199389 shutdown_connections 2024-05-10T05:34:43.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562e4ffd18d0 0x562e4ffd3cf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f2cc00700 0 client.25693 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:43.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e4ff651f0 0x562e4ff740b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e4febdae0 0x562e4ff753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f61bdeb80 1 --2- 172.21.15.131:0/2017199389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e4ff5bcd0 0x562e4ff73a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2017199389 >> 172.21.15.131:0/2017199389 conn(0x562e4ff524d0 msgr2=0x562e4ff68d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:43.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2017199389 shutdown_connections 2024-05-10T05:34:43.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.905+0000 7f9f61bdeb80 1 -- 172.21.15.131:0/2017199389 wait complete. 2024-05-10T05:34:43.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.906+0000 7f9f61bdeb80 20 client.25693 trim_cache size 0 max 16384 2024-05-10T05:34:43.908 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:34:43.909 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:34:43.909 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "5fbed165-21ed-4307-b2ed-e4e4a012ec81", 2024-05-10T05:34:43.909 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:34:43.909 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:34:43.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.907+0000 7f9f49c00700 1 -- 172.21.15.131:0/3117719863 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9f280a1240 msgr2=0x7f9f280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.907+0000 7f9f49c00700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9f280a1240 0x7f9f280a3750 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x562e4ff5c6b0 tx=0x7f9f3c00b040 comp rx=0 tx=0).stop 2024-05-10T05:34:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.907+0000 7f9f49c00700 1 -- 172.21.15.131:0/3117719863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f44102d00 msgr2=0x7f9f44198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.907+0000 7f9f49c00700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f44102d00 0x7f9f44198460 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f300098f0 tx=0x7f9f3000bb80 comp rx=0 tx=0).stop 2024-05-10T05:34:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.907+0000 7f9f49c00700 1 -- 172.21.15.131:0/3117719863 shutdown_connections 2024-05-10T05:34:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.907+0000 7f9f49c00700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f9f280a1240 0x7f9f280a3750 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.908+0000 7f9f49c00700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f441036a0 0x7f9f4419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.908+0000 7f9f49c00700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f44101be0 0x7f9f44197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.908+0000 7f9f49c00700 1 --2- 172.21.15.131:0/3117719863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f44102d00 0x7f9f44198460 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.908+0000 7f9f49c00700 1 -- 172.21.15.131:0/3117719863 >> 172.21.15.131:0/3117719863 conn(0x7f9f440fd1d0 msgr2=0x7f9f44105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.908+0000 7f9f49c00700 1 -- 172.21.15.131:0/3117719863 shutdown_connections 2024-05-10T05:34:43.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:43.908+0000 7f9f49c00700 1 -- 172.21.15.131:0/3117719863 wait complete. 2024-05-10T05:34:43.955 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:34:44.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:44 smithi131 ceph-mon[37264]: pgmap v326: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:44.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.281+0000 7f59fca00700 1 -- 172.21.15.131:0/2646980828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f81024b0 msgr2=0x7f59f81048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:44.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.281+0000 7f59fca00700 1 --2- 172.21.15.131:0/2646980828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f81024b0 0x7f59f81048d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59e80098f0 tx=0x7f59e8009c00 comp rx=0 tx=0).stop 2024-05-10T05:34:44.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.281+0000 7f59fca00700 1 -- 172.21.15.131:0/2646980828 shutdown_connections 2024-05-10T05:34:44.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.281+0000 7f59fca00700 1 --2- 172.21.15.131:0/2646980828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f8107770 0x7f59f8109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.281+0000 7f59fca00700 1 --2- 172.21.15.131:0/2646980828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f81024b0 0x7f59f81048d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.281+0000 7f59fca00700 1 --2- 172.21.15.131:0/2646980828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59f8104e10 0x7f59f8107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.281+0000 7f59fca00700 1 -- 172.21.15.131:0/2646980828 >> 172.21.15.131:0/2646980828 conn(0x7f59f80fbef0 msgr2=0x7f59f80fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:44.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.282+0000 7f59fca00700 1 -- 172.21.15.131:0/2646980828 shutdown_connections 2024-05-10T05:34:44.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.282+0000 7f59fca00700 1 -- 172.21.15.131:0/2646980828 wait complete. 2024-05-10T05:34:44.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.283+0000 7f59fca00700 1 Processor -- start 2024-05-10T05:34:44.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.283+0000 7f59fca00700 1 -- start start 2024-05-10T05:34:44.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59fca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f81024b0 0x7f59f8197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59fca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59f8104e10 0x7f59f8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59fca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f8107770 0x7f59f819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59fca00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f819f890 con 0x7f59f8104e10 2024-05-10T05:34:44.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59fca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f819f9d0 con 0x7f59f81024b0 2024-05-10T05:34:44.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59fca00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f819fb60 con 0x7f59f8107770 2024-05-10T05:34:44.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59f7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f81024b0 0x7f59f8197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59f6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59f8104e10 0x7f59f8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59f7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f8107770 0x7f59f819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59f6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59f8104e10 0x7f59f8198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42038/0 (socket says 172.21.15.131:42038) 2024-05-10T05:34:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.284+0000 7f59f6a00700 1 -- 172.21.15.131:0/109384225 learned_addr learned my addr 172.21.15.131:0/109384225 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.285+0000 7f59f7e00700 1 -- 172.21.15.131:0/109384225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f81024b0 msgr2=0x7f59f8197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.285+0000 7f59f7e00700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f81024b0 0x7f59f8197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.285+0000 7f59f7e00700 1 -- 172.21.15.131:0/109384225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59f8104e10 msgr2=0x7f59f8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.285+0000 7f59f7e00700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59f8104e10 0x7f59f8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.285+0000 7f59f7e00700 1 -- 172.21.15.131:0/109384225 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e8009520 con 0x7f59f8107770 2024-05-10T05:34:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.285+0000 7f59f7e00700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f8107770 0x7f59f819f2c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f59f000efc0 tx=0x7f59f000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:44.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.286+0000 7f59efe00700 1 -- 172.21.15.131:0/109384225 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0004bb0 con 0x7f59f8107770 2024-05-10T05:34:44.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.286+0000 7f59efe00700 1 -- 172.21.15.131:0/109384225 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f000f040 con 0x7f59f8107770 2024-05-10T05:34:44.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.286+0000 7f59fca00700 1 -- 172.21.15.131:0/109384225 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f8100300 con 0x7f59f8107770 2024-05-10T05:34:44.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.286+0000 7f59efe00700 1 -- 172.21.15.131:0/109384225 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0004020 con 0x7f59f8107770 2024-05-10T05:34:44.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.286+0000 7f59fca00700 1 -- 172.21.15.131:0/109384225 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f8100810 con 0x7f59f8107770 2024-05-10T05:34:44.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.290+0000 7f5a15fecb80 1 Processor -- start 2024-05-10T05:34:44.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.291+0000 7f59efe00700 1 -- 172.21.15.131:0/109384225 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f59f0004180 con 0x7f59f8107770 2024-05-10T05:34:44.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.291+0000 7f5a15fecb80 1 -- start start 2024-05-10T05:34:44.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.292+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6ad4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.292+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 0x5632a6ad5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.292+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6adc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.292+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a69fb150 con 0x5632a6ad60f0 2024-05-10T05:34:44.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.292+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a69f8b10 con 0x5632a6ad57a0 2024-05-10T05:34:44.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.292+0000 7f59f6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6ad4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.293+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a69f4400 con 0x5632a6ad4530 2024-05-10T05:34:44.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.293+0000 7f59f6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6ad4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42994/0 (socket says 172.21.15.131:42994) 2024-05-10T05:34:44.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.293+0000 7f59f6a00700 1 -- 172.21.15.131:0/1922328238 learned_addr learned my addr 172.21.15.131:0/1922328238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:44.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.293+0000 7f59f6a00700 1 --2- 172.21.15.131:0/1922328238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6adc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.293+0000 7f59f7400700 1 --2- 172.21.15.131:0/1922328238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 0x5632a6ad5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.293+0000 7f59f6a00700 1 -- 172.21.15.131:0/1922328238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 msgr2=0x5632a6ad5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:44.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.293+0000 7f59f6a00700 1 --2- 172.21.15.131:0/1922328238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 0x5632a6ad5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.294+0000 7f59f6a00700 1 -- 172.21.15.131:0/1922328238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 msgr2=0x5632a6adc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:44.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.294+0000 7f59f6a00700 1 --2- 172.21.15.131:0/1922328238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6adc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.294+0000 7f59f6a00700 1 -- 172.21.15.131:0/1922328238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632a68753b0 con 0x5632a6ad4530 2024-05-10T05:34:44.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.294+0000 7f59efe00700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f59dc0a11a0 0x7f59dc0a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.294+0000 7f59f7400700 1 --2- 172.21.15.131:0/1922328238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 0x5632a6ad5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:34:44.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.294+0000 7f59efe00700 1 -- 172.21.15.131:0/109384225 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f59f00d00c0 con 0x7f59f8107770 2024-05-10T05:34:44.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.294+0000 7f59f7400700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f59dc0a11a0 0x7f59dc0a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.295+0000 7f59f6a00700 1 --2- 172.21.15.131:0/1922328238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6ad4960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f59e4011940 tx=0x7f59e4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=314d412048a92ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:44.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.295+0000 7f59e3e00700 1 -- 172.21.15.131:0/1922328238 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4004030 con 0x5632a6ad4530 2024-05-10T05:34:44.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.295+0000 7f59e3e00700 1 -- 172.21.15.131:0/1922328238 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59e4003680 con 0x5632a6ad4530 2024-05-10T05:34:44.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.295+0000 7f59e3e00700 1 -- 172.21.15.131:0/1922328238 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e400e610 con 0x5632a6ad4530 2024-05-10T05:34:44.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.296+0000 7f5a15fecb80 1 -- 172.21.15.131:0/1922328238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 msgr2=0x5632a6ad4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:44.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.296+0000 7f59f7400700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f59dc0a11a0 0x7f59dc0a36b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f59e80098c0 tx=0x7f59e8009f70 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:44.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.296+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/1922328238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6ad4960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f59e4011940 tx=0x7f59e4009920 comp rx=0 tx=0).stop 2024-05-10T05:34:44.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.296+0000 7f5a15fecb80 1 -- 172.21.15.131:0/1922328238 shutdown_connections 2024-05-10T05:34:44.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.296+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/1922328238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6adc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.296+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/1922328238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6ad4960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.296+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/1922328238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 0x5632a6ad5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.296+0000 7f5a15fecb80 1 -- 172.21.15.131:0/1922328238 >> 172.21.15.131:0/1922328238 conn(0x5632a6a36a60 msgr2=0x5632a6acf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:44.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.296+0000 7f5a15fecb80 1 -- 172.21.15.131:0/1922328238 shutdown_connections 2024-05-10T05:34:44.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.297+0000 7f5a15fecb80 1 -- 172.21.15.131:0/1922328238 wait complete. 2024-05-10T05:34:44.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.297+0000 7f5a15fecb80 1 -- start start 2024-05-10T05:34:44.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.298+0000 7f5a15fecb80 1 Processor -- start 2024-05-10T05:34:44.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.298+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad4530 0x5632a6ade940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.298+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6adee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.298+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6ae5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a6990560 con 0x5632a6ad60f0 2024-05-10T05:34:44.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a6990200 con 0x5632a6ad4530 2024-05-10T05:34:44.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a68776a0 con 0x5632a6ad57a0 2024-05-10T05:34:44.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f5a15fecb80 10 client.0 _ll_register_callbacks cb 0x5632a6999b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:44.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f59f6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6adee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f59f6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6adee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43004/0 (socket says 172.21.15.131:43004) 2024-05-10T05:34:44.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f59f6a00700 1 -- 172.21.15.131:0/216659872 learned_addr learned my addr 172.21.15.131:0/216659872 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:44.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f59f7400700 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad4530 0x5632a6ade940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f59f7e00700 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6ae5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f59f6a00700 1 -- 172.21.15.131:0/216659872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad4530 msgr2=0x5632a6ade940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:44.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f59f6a00700 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad4530 0x5632a6ade940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.299+0000 7f59f6a00700 1 -- 172.21.15.131:0/216659872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 msgr2=0x5632a6ae5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:44.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.300+0000 7f59f6a00700 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6ae5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.300+0000 7f59f6a00700 1 -- 172.21.15.131:0/216659872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e4019040 con 0x5632a6ad57a0 2024-05-10T05:34:44.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.300+0000 7f59f7400700 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad4530 0x5632a6ade940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:34:44.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.300+0000 7f59f7e00700 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6ae5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.300+0000 7f59f6a00700 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6adee80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59e4003d70 tx=0x7f59e4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7606c8ef96cd78e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.300+0000 7f59e0c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.300+0000 7f59e0c00700 1 -- 172.21.15.131:0/216659872 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4025070 con 0x5632a6ad57a0 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.300+0000 7f5a15fecb80 1 -- 172.21.15.131:0/216659872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632a694d270 con 0x5632a6ad57a0 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.300+0000 7f59e0c00700 1 -- 172.21.15.131:0/216659872 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f59e40041f0 con 0x5632a6ad57a0 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f59e0c00700 1 -- 172.21.15.131:0/216659872 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59e40043f0 con 0x5632a6ad57a0 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f5a15fecb80 10 client.25705 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f5a15fecb80 1 -- 172.21.15.131:0/216659872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5632a6821b60 con 0x5632a6ad57a0 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f59e0c00700 1 -- 172.21.15.131:0/216659872 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632a694d270 con 0x5632a6ad57a0 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f59e0c00700 1 -- 172.21.15.131:0/216659872 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f59e40049d0 con 0x5632a6ad57a0 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f5a15fecb80 10 client.25705 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f5a15fecb80 10 client.25705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f5a15fecb80 20 client.25705 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5632a6aea3d0 0x5632a6aec820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f5a15fecb80 4 client.25705 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:34:44.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.301+0000 7f5a15fecb80 1 -- 172.21.15.131:0/216659872 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632a6aa3780 con 0x5632a6aea3d0 2024-05-10T05:34:44.304 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:44 smithi157 ceph-mon[78427]: pgmap v326: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:44.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.302+0000 7f59f7400700 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5632a6aea3d0 0x5632a6aec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.302+0000 7f59e0c00700 10 client.25705 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:44.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.302+0000 7f59f7400700 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5632a6aea3d0 0x5632a6aec820 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bdfdc2f7c644f40 server_cookie=4be6dc2f936948af in_seq=0 out_seq=0 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.303+0000 7f59e0c00700 1 -- 172.21.15.131:0/216659872 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632a6aa3780 con 0x5632a6aea3d0 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.303+0000 7f59e0c00700 10 client.25705 handle_command_reply: tid=0 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.303+0000 7f5a15fecb80 1 client.25705 shutdown 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 1 -- 172.21.15.131:0/216659872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 msgr2=0x5632a6adee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6adee80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59e4003d70 tx=0x7f59e4009310 comp rx=0 tx=0).stop 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 1 -- 172.21.15.131:0/216659872 shutdown_connections 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5632a6aea3d0 0x5632a6aec820 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6ae5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad4530 0x5632a6ade940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/216659872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6adee80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 1 -- 172.21.15.131:0/216659872 >> 172.21.15.131:0/216659872 conn(0x5632a6a36a60 msgr2=0x5632a6ad3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f59e0c00700 0 client.25705 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 1 -- 172.21.15.131:0/216659872 shutdown_connections 2024-05-10T05:34:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 1 -- 172.21.15.131:0/216659872 wait complete. 2024-05-10T05:34:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.304+0000 7f5a15fecb80 20 client.25705 trim_cache size 0 max 16384 2024-05-10T05:34:44.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.337+0000 7f5a15fecb80 1 Processor -- start 2024-05-10T05:34:44.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.337+0000 7f5a15fecb80 1 -- start start 2024-05-10T05:34:44.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6addeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 0x5632a6ae5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6aec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a6ad0ba0 con 0x5632a6ad60f0 2024-05-10T05:34:44.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a6ac98d0 con 0x5632a6ad57a0 2024-05-10T05:34:44.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a6adf800 con 0x5632a6ad4530 2024-05-10T05:34:44.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f59f7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6addeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f59f7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6addeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43016/0 (socket says 172.21.15.131:43016) 2024-05-10T05:34:44.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f59f7400700 1 -- 172.21.15.131:0/1837521712 learned_addr learned my addr 172.21.15.131:0/1837521712 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:44.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f59f6a00700 1 --2- 172.21.15.131:0/1837521712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 0x5632a6ae5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.338+0000 7f59f7e00700 1 --2- 172.21.15.131:0/1837521712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6aec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.339+0000 7f59f7400700 1 -- 172.21.15.131:0/1837521712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 msgr2=0x5632a6ae5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:44.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.339+0000 7f59f7400700 1 --2- 172.21.15.131:0/1837521712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 0x5632a6ae5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.339+0000 7f59f7400700 1 -- 172.21.15.131:0/1837521712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 msgr2=0x5632a6aec3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:44.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.339+0000 7f59f7400700 1 --2- 172.21.15.131:0/1837521712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6aec3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.339+0000 7f59f7400700 1 -- 172.21.15.131:0/1837521712 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632a69ba560 con 0x5632a6ad4530 2024-05-10T05:34:44.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.339+0000 7f59f7400700 1 --2- 172.21.15.131:0/1837521712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6addeb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f59e80098f0 tx=0x7f59e8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=804b414098272d49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:44.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.339+0000 7f59ed200700 1 -- 172.21.15.131:0/1837521712 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8027070 con 0x5632a6ad4530 2024-05-10T05:34:44.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.340+0000 7f59ed200700 1 -- 172.21.15.131:0/1837521712 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59e8028040 con 0x5632a6ad4530 2024-05-10T05:34:44.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.340+0000 7f59ed200700 1 -- 172.21.15.131:0/1837521712 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e80038e0 con 0x5632a6ad4530 2024-05-10T05:34:44.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.341+0000 7f5a15fecb80 1 -- 172.21.15.131:0/1837521712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 msgr2=0x5632a6addeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:44.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.341+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/1837521712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6addeb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f59e80098f0 tx=0x7f59e8019000 comp rx=0 tx=0).stop 2024-05-10T05:34:44.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.341+0000 7f5a15fecb80 1 -- 172.21.15.131:0/1837521712 shutdown_connections 2024-05-10T05:34:44.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.341+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/1837521712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad60f0 0x5632a6aec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.341+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/1837521712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad4530 0x5632a6addeb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.341+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/1837521712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad57a0 0x5632a6ae5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.341+0000 7f5a15fecb80 1 -- 172.21.15.131:0/1837521712 >> 172.21.15.131:0/1837521712 conn(0x5632a6a36a60 msgr2=0x5632a6ad95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:44.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.341+0000 7f5a15fecb80 1 -- 172.21.15.131:0/1837521712 shutdown_connections 2024-05-10T05:34:44.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.341+0000 7f5a15fecb80 1 -- 172.21.15.131:0/1837521712 wait complete. 2024-05-10T05:34:44.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.342+0000 7f5a15fecb80 1 -- start start 2024-05-10T05:34:44.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.342+0000 7f5a15fecb80 1 Processor -- start 2024-05-10T05:34:44.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad4530 0x5632a6ae3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6ae37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f5a15fecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad60f0 0x5632a6ae46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a6ae4bf0 con 0x5632a6ad4530 2024-05-10T05:34:44.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a6ae4d60 con 0x5632a6ad60f0 2024-05-10T05:34:44.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f5a15fecb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a6ae4f20 con 0x5632a6ad57a0 2024-05-10T05:34:44.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f5a15fecb80 10 client.0 _ll_register_callbacks cb 0x5632a6999b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:34:44.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f59f6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6ae37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f59f6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6ae37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43020/0 (socket says 172.21.15.131:43020) 2024-05-10T05:34:44.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f59f6a00700 1 -- 172.21.15.131:0/3411960517 learned_addr learned my addr 172.21.15.131:0/3411960517 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:34:44.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.343+0000 7f59f7e00700 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad60f0 0x5632a6ae46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59f7400700 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad4530 0x5632a6ae3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59f7e00700 1 -- 172.21.15.131:0/3411960517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 msgr2=0x5632a6ae37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59f7e00700 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6ae37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59f7e00700 1 -- 172.21.15.131:0/3411960517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad4530 msgr2=0x5632a6ae3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59f7e00700 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad4530 0x5632a6ae3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59f7e00700 1 -- 172.21.15.131:0/3411960517 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f810a7b0 con 0x5632a6ad60f0 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59f7400700 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad4530 0x5632a6ae3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59f7e00700 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad60f0 0x5632a6ae46b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f59f00cdc10 tx=0x7f59f000c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b1712c32346db0c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59e0c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59e0c00700 1 -- 172.21.15.131:0/3411960517 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f009cc00 con 0x5632a6ad60f0 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f5a15fecb80 1 -- 172.21.15.131:0/3411960517 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59e4003680 con 0x5632a6ad60f0 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.344+0000 7f59e0c00700 1 -- 172.21.15.131:0/3411960517 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f59f009dc10 con 0x5632a6ad60f0 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.345+0000 7f59e0c00700 1 -- 172.21.15.131:0/3411960517 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59f00a1150 con 0x5632a6ad60f0 2024-05-10T05:34:44.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.345+0000 7f5a15fecb80 10 client.25691 fetch_fsmap learned FSMap version 27 2024-05-10T05:34:44.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.345+0000 7f5a15fecb80 1 -- 172.21.15.131:0/3411960517 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5632a6879cd0 con 0x5632a6ad60f0 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.345+0000 7f59e0c00700 1 -- 172.21.15.131:0/3411960517 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f0080080 con 0x5632a6ad60f0 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.346+0000 7f59e0c00700 1 -- 172.21.15.131:0/3411960517 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f59f0099300 con 0x5632a6ad60f0 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.346+0000 7f5a15fecb80 10 client.25691 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.346+0000 7f5a15fecb80 10 client.25691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.346+0000 7f5a15fecb80 20 client.25691 populate_metadata read hostname 'smithi131' 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.346+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5632a6b40e30 0x5632a6b43250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.346+0000 7f5a15fecb80 4 client.25691 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.346+0000 7f5a15fecb80 1 -- 172.21.15.131:0/3411960517 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5632a69ecec0 con 0x5632a6b40e30 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.347+0000 7f59f7400700 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5632a6b40e30 0x5632a6b43250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.347+0000 7f59f7400700 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5632a6b40e30 0x5632a6b43250 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=843c8fed372a34b1 server_cookie=468cd3cc75182d94 in_seq=0 out_seq=0 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.347+0000 7f59e0c00700 10 client.25691 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.347+0000 7f59e0c00700 1 -- 172.21.15.131:0/3411960517 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5632a69ecec0 con 0x5632a6b40e30 2024-05-10T05:34:44.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.347+0000 7f59e0c00700 10 client.25691 handle_command_reply: tid=0 2024-05-10T05:34:44.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.347+0000 7f5a15fecb80 1 client.25691 shutdown 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 1 -- 172.21.15.131:0/3411960517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad60f0 msgr2=0x5632a6ae46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad60f0 0x5632a6ae46b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f59f00cdc10 tx=0x7f59f000c940 comp rx=0 tx=0).stop 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 1 -- 172.21.15.131:0/3411960517 shutdown_connections 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5632a6b40e30 0x5632a6b43250 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632a6ad4530 0x5632a6ae3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5632a6ad60f0 0x5632a6ae46b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 1 --2- 172.21.15.131:0/3411960517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5632a6ad57a0 0x5632a6ae37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 1 -- 172.21.15.131:0/3411960517 >> 172.21.15.131:0/3411960517 conn(0x5632a6a36a60 msgr2=0x5632a6aec3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f59e0c00700 0 client.25691 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 1 -- 172.21.15.131:0/3411960517 shutdown_connections 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 1 -- 172.21.15.131:0/3411960517 wait complete. 2024-05-10T05:34:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.348+0000 7f5a15fecb80 20 client.25691 trim_cache size 0 max 16384 2024-05-10T05:34:44.351 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:34:44.351 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:34:44.351 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:34:44.351 INFO:teuthology.orchestra.run.smithi131.stdout: "5fbed165-21ed-4307-b2ed-e4e4a012ec81": { 2024-05-10T05:34:44.351 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:34:44.351 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "5fbed165-21ed-4307-b2ed-e4e4a012ec81", 2024-05-10T05:34:44.351 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:34:44.351 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:34:44.351 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:34:44.351 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:34:44.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.349+0000 7f59fca00700 1 -- 172.21.15.131:0/109384225 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f59dc0a11a0 msgr2=0x7f59dc0a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:44.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.349+0000 7f59fca00700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f59dc0a11a0 0x7f59dc0a36b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f59e80098c0 tx=0x7f59e8009f70 comp rx=0 tx=0).stop 2024-05-10T05:34:44.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.349+0000 7f59fca00700 1 -- 172.21.15.131:0/109384225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f8107770 msgr2=0x7f59f819f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:34:44.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.349+0000 7f59fca00700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f8107770 0x7f59f819f2c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f59f000efc0 tx=0x7f59f000c3a0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.349+0000 7f59fca00700 1 -- 172.21.15.131:0/109384225 shutdown_connections 2024-05-10T05:34:44.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.349+0000 7f59fca00700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f59dc0a11a0 0x7f59dc0a36b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.349+0000 7f59fca00700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59f8107770 0x7f59f819f2c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.349+0000 7f59fca00700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f81024b0 0x7f59f8197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.349+0000 7f59fca00700 1 --2- 172.21.15.131:0/109384225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59f8104e10 0x7f59f8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:34:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.349+0000 7f59fca00700 1 -- 172.21.15.131:0/109384225 >> 172.21.15.131:0/109384225 conn(0x7f59f80fbef0 msgr2=0x7f59f81044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:34:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.350+0000 7f59fca00700 1 -- 172.21.15.131:0/109384225 shutdown_connections 2024-05-10T05:34:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:34:44.350+0000 7f59fca00700 1 -- 172.21.15.131:0/109384225 wait complete. 2024-05-10T05:34:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:44 smithi142 ceph-mon[77667]: pgmap v326: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:34:44.396 INFO:tasks.cephfs.filesystem:scrub status for tag:5fbed165-21ed-4307-b2ed-e4e4a012ec81 - {'path': '/', 'tag': '5fbed165-21ed-4307-b2ed-e4e4a012ec81', 'options': 'recursive,force'} 2024-05-10T05:34:45.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:45 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:34:45.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:45 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:45.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:45 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:34:45.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:45 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:45.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:45 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:34:45.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:45 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:45.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:45 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:34:45.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:45 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:45.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:45 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:34:45.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:45 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:45.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:45 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:34:45.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:45 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:34:46.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:46 smithi157 ceph-mon[78427]: pgmap v327: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:46.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:46 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:34:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:46 smithi142 ceph-mon[77667]: pgmap v327: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:46.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:46 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:34:46.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:46 smithi131 ceph-mon[37264]: pgmap v327: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:34:46.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:46 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:34:48.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:48 smithi157 ceph-mon[78427]: pgmap v328: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-10T05:34:48.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:48 smithi142 ceph-mon[77667]: pgmap v328: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-10T05:34:48.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:48 smithi131 ceph-mon[37264]: pgmap v328: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-10T05:34:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:50 smithi157 ceph-mon[78427]: pgmap v329: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-10T05:34:50.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:50 smithi142 ceph-mon[77667]: pgmap v329: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-10T05:34:50.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:50 smithi131 ceph-mon[37264]: pgmap v329: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-10T05:34:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:52 smithi157 ceph-mon[78427]: pgmap v330: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-10T05:34:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:52 smithi142 ceph-mon[77667]: pgmap v330: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-10T05:34:52.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:52 smithi131 ceph-mon[37264]: pgmap v330: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-10T05:34:54.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:54 smithi157 ceph-mon[78427]: pgmap v331: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-10T05:34:54.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:54 smithi142 ceph-mon[77667]: pgmap v331: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-10T05:34:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:54 smithi131 ceph-mon[37264]: pgmap v331: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-10T05:34:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:56 smithi157 ceph-mon[78427]: pgmap v332: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-10T05:34:56.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:56 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:34:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:56 smithi142 ceph-mon[77667]: pgmap v332: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-10T05:34:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:56 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:34:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:56 smithi131 ceph-mon[37264]: pgmap v332: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-10T05:34:56.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:56 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:34:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:57 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:34:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:57 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:34:57.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:57 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:34:57.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:34:57.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:34:57.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:34:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:57 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:34:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:57 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:34:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:57 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:34:58.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:34:58 smithi157 ceph-mon[78427]: pgmap v333: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-10T05:34:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:34:58 smithi142 ceph-mon[77667]: pgmap v333: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-10T05:34:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:34:58 smithi131 ceph-mon[37264]: pgmap v333: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-10T05:35:00.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:00 smithi157 ceph-mon[78427]: pgmap v334: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-10T05:35:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:00 smithi142 ceph-mon[77667]: pgmap v334: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-10T05:35:00.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:00 smithi131 ceph-mon[37264]: pgmap v334: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-10T05:35:02.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:02 smithi157 ceph-mon[78427]: pgmap v335: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:35:02.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:02 smithi142 ceph-mon[77667]: pgmap v335: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:35:02.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:02 smithi131 ceph-mon[37264]: pgmap v335: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T05:35:03.163 INFO:tasks.workunit.client.0.smithi131.stderr:Updating files: 63% (7833/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-10T05:35:03.976 INFO:tasks.workunit.client.0.smithi131.stderr:Note: switching to '50174015a7e79f951b988949abe236791f52d4da'. 2024-05-10T05:35:03.976 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-10T05:35:03.976 INFO:tasks.workunit.client.0.smithi131.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-10T05:35:03.976 INFO:tasks.workunit.client.0.smithi131.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-10T05:35:03.976 INFO:tasks.workunit.client.0.smithi131.stderr:state without impacting any branches by switching back to a branch. 2024-05-10T05:35:03.976 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-10T05:35:03.976 INFO:tasks.workunit.client.0.smithi131.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-10T05:35:03.976 INFO:tasks.workunit.client.0.smithi131.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-10T05:35:03.976 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-10T05:35:03.977 INFO:tasks.workunit.client.0.smithi131.stderr: git switch -c 2024-05-10T05:35:03.977 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-10T05:35:03.977 INFO:tasks.workunit.client.0.smithi131.stderr:Or undo this operation with: 2024-05-10T05:35:03.977 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-10T05:35:03.977 INFO:tasks.workunit.client.0.smithi131.stderr: git switch - 2024-05-10T05:35:03.977 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-10T05:35:03.977 INFO:tasks.workunit.client.0.smithi131.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-10T05:35:03.977 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-10T05:35:03.977 INFO:tasks.workunit.client.0.smithi131.stderr:HEAD is now at 50174015a7e Merge PR #57343 into wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T05:35:03.995 DEBUG:teuthology.orchestra.run.smithi131:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-10T05:35:04.045 INFO:tasks.workunit.client.0.smithi131.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-10T05:35:04.050 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T05:35:04.050 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-10T05:35:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:04 smithi157 ceph-mon[78427]: pgmap v336: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:35:04.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:04 smithi142 ceph-mon[77667]: pgmap v336: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:35:04.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:04 smithi131 ceph-mon[37264]: pgmap v336: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-10T05:35:04.803 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-10T05:35:04.889 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-10T05:35:04.952 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T05:35:04.956 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T05:35:04.956 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-10T05:35:05.020 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T05:35:05.027 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:35:05.027 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-10T05:35:05.070 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-05-10T05:35:05.071 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-05-10T05:35:05.071 DEBUG:teuthology.orchestra.run.smithi131:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=50174015a7e79f951b988949abe236791f52d4da TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-05-10T05:35:05.166 INFO:tasks.workunit.client.0.smithi131.stdout:getting iogen 2024-05-10T05:35:05.214 INFO:tasks.workunit.client.0.smithi131.stderr:--2024-05-10 05:35:05-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-05-10T05:35:05.215 INFO:tasks.workunit.client.0.smithi131.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-10T05:35:05.237 INFO:tasks.workunit.client.0.smithi131.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-10T05:35:05.258 INFO:tasks.workunit.client.0.smithi131.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-10T05:35:05.258 INFO:tasks.workunit.client.0.smithi131.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-05-10T05:35:05.259 INFO:tasks.workunit.client.0.smithi131.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-05-10T05:35:05.264 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-10T05:35:05.264 INFO:tasks.workunit.client.0.smithi131.stderr: 0K ....... 100% 453M=0s 2024-05-10T05:35:05.264 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-10T05:35:05.264 INFO:tasks.workunit.client.0.smithi131.stderr:2024-05-10 05:35:05 (453 MB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-05-10T05:35:05.264 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-05-10T05:35:05.297 INFO:tasks.workunit.client.0.smithi131.stdout:iogen_3.1p0 2024-05-10T05:35:05.298 INFO:tasks.workunit.client.0.smithi131.stdout:iogen_3.1p0/src 2024-05-10T05:35:05.302 INFO:tasks.workunit.client.0.smithi131.stdout:iogen_3.1p0/src/iogen.8 2024-05-10T05:35:05.302 INFO:tasks.workunit.client.0.smithi131.stdout:iogen_3.1p0/src/iogen.c 2024-05-10T05:35:05.302 INFO:tasks.workunit.client.0.smithi131.stdout:iogen_3.1p0/README 2024-05-10T05:35:05.303 INFO:tasks.workunit.client.0.smithi131.stdout:iogen_3.1p0/Makefile.bsd 2024-05-10T05:35:05.303 INFO:tasks.workunit.client.0.smithi131.stdout:iogen_3.1p0/Makefile 2024-05-10T05:35:05.304 INFO:tasks.workunit.client.0.smithi131.stdout:making iogen 2024-05-10T05:35:05.316 INFO:tasks.workunit.client.0.smithi131.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-05-10T05:35:05.588 INFO:tasks.workunit.client.0.smithi131.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-05-10T05:35:05.698 INFO:tasks.workunit.client.0.smithi131.stdout:running iogen 2024-05-10T05:35:05.700 INFO:tasks.workunit.client.0.smithi131.stderr:child spawned 104797 2024-05-10T05:35:05.700 INFO:tasks.workunit.client.0.smithi131.stderr:child spawned 104798 2024-05-10T05:35:05.700 INFO:tasks.workunit.client.0.smithi131.stderr:child spawned 104799 2024-05-10T05:35:05.701 INFO:tasks.workunit.client.0.smithi131.stderr:child spawned 104800 2024-05-10T05:35:05.701 INFO:tasks.workunit.client.0.smithi131.stderr:child spawned 104801 2024-05-10T05:35:05.701 INFO:tasks.workunit.client.0.smithi131.stdout:sleep for 10 min 2024-05-10T05:35:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:06 smithi142 ceph-mon[77667]: pgmap v337: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:35:06.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:06 smithi131 ceph-mon[37264]: pgmap v337: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:35:06.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:06 smithi157 ceph-mon[78427]: pgmap v337: 97 pgs: 97 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T05:35:08.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:08 smithi142 ceph-mon[77667]: pgmap v338: 97 pgs: 97 active+clean; 807 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-05-10T05:35:08.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:08 smithi131 ceph-mon[37264]: pgmap v338: 97 pgs: 97 active+clean; 807 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-05-10T05:35:08.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:08 smithi157 ceph-mon[78427]: pgmap v338: 97 pgs: 97 active+clean; 807 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-05-10T05:35:09.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:09 smithi131 ceph-mon[37264]: pgmap v339: 97 pgs: 97 active+clean; 53 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 14 op/s 2024-05-10T05:35:09.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:09 smithi157 ceph-mon[78427]: pgmap v339: 97 pgs: 97 active+clean; 53 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 14 op/s 2024-05-10T05:35:09.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:09 smithi142 ceph-mon[77667]: pgmap v339: 97 pgs: 97 active+clean; 53 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 14 op/s 2024-05-10T05:35:12.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:11 smithi131 ceph-mon[37264]: pgmap v340: 97 pgs: 97 active+clean; 329 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-05-10T05:35:12.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:11 smithi157 ceph-mon[78427]: pgmap v340: 97 pgs: 97 active+clean; 329 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-05-10T05:35:12.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:11 smithi142 ceph-mon[77667]: pgmap v340: 97 pgs: 97 active+clean; 329 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-05-10T05:35:14.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:13 smithi157 ceph-mon[78427]: pgmap v341: 97 pgs: 97 active+clean; 645 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 54 MiB/s wr, 33 op/s 2024-05-10T05:35:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:13 smithi142 ceph-mon[77667]: pgmap v341: 97 pgs: 97 active+clean; 645 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 54 MiB/s wr, 33 op/s 2024-05-10T05:35:14.397 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:35:14.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:13 smithi131 ceph-mon[37264]: pgmap v341: 97 pgs: 97 active+clean; 645 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 54 MiB/s wr, 33 op/s 2024-05-10T05:35:15.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.004+0000 7fade9800700 1 -- 172.21.15.131:0/1580356276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fade4075140 msgr2=0x7fade4073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:15.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.004+0000 7fade9800700 1 --2- 172.21.15.131:0/1580356276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fade4075140 0x7fade4073600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009f90 tx=0x7fadd400b330 comp rx=0 tx=0).stop 2024-05-10T05:35:15.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.005+0000 7fade9800700 1 -- 172.21.15.131:0/1580356276 shutdown_connections 2024-05-10T05:35:15.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.005+0000 7fade9800700 1 --2- 172.21.15.131:0/1580356276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade410e220 0x7fade4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.007+0000 7fade9800700 1 --2- 172.21.15.131:0/1580356276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fade4075140 0x7fade4073600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.007+0000 7fade9800700 1 --2- 172.21.15.131:0/1580356276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade4073b40 0x7fade4074010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.007+0000 7fade9800700 1 -- 172.21.15.131:0/1580356276 >> 172.21.15.131:0/1580356276 conn(0x7fade40fd210 msgr2=0x7fade40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:15.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.008+0000 7fade9800700 1 -- 172.21.15.131:0/1580356276 shutdown_connections 2024-05-10T05:35:15.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.009+0000 7fade9800700 1 -- 172.21.15.131:0/1580356276 wait complete. 2024-05-10T05:35:15.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.011+0000 7fade9800700 1 Processor -- start 2024-05-10T05:35:15.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.011+0000 7fade9800700 1 -- start start 2024-05-10T05:35:15.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.011+0000 7fade9800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade4073b40 0x7fade419e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.011+0000 7fade9800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade4075140 0x7fade419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.012+0000 7fade9800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fade410e220 0x7fade41a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.012+0000 7fade9800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41a5fa0 con 0x7fade4073b40 2024-05-10T05:35:15.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.012+0000 7fade9800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade406ac60 con 0x7fade410e220 2024-05-10T05:35:15.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.012+0000 7fade9800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade406ae20 con 0x7fade4075140 2024-05-10T05:35:15.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.015+0000 7fadda000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade4075140 0x7fade419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.015+0000 7fadda000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade4075140 0x7fade419ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51704/0 (socket says 172.21.15.131:51704) 2024-05-10T05:35:15.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.015+0000 7fadda000700 1 -- 172.21.15.131:0/3336909633 learned_addr learned my addr 172.21.15.131:0/3336909633 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:15.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.015+0000 7fade2a00700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade4073b40 0x7fade419e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.016+0000 7fade2a00700 1 -- 172.21.15.131:0/3336909633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade4075140 msgr2=0x7fade419ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:15.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.016+0000 7fade2a00700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade4075140 0x7fade419ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.016+0000 7fade2a00700 1 -- 172.21.15.131:0/3336909633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fade410e220 msgr2=0x7fade41a5990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:35:15.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.016+0000 7fade2a00700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fade410e220 0x7fade41a5990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.016+0000 7fade2a00700 1 -- 172.21.15.131:0/3336909633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc009520 con 0x7fade4073b40 2024-05-10T05:35:15.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.017+0000 7fade2a00700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade4073b40 0x7fade419e4c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fade4074e30 tx=0x7fadd400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:15.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.018+0000 7faddbe00700 1 -- 172.21.15.131:0/3336909633 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400e040 con 0x7fade4073b40 2024-05-10T05:35:15.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.018+0000 7faddbe00700 1 -- 172.21.15.131:0/3336909633 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadd4007e20 con 0x7fade4073b40 2024-05-10T05:35:15.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.018+0000 7faddbe00700 1 -- 172.21.15.131:0/3336909633 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd401b7a0 con 0x7fade4073b40 2024-05-10T05:35:15.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.018+0000 7fade9800700 1 -- 172.21.15.131:0/3336909633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd4009c40 con 0x7fade4073b40 2024-05-10T05:35:15.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.018+0000 7fade9800700 1 -- 172.21.15.131:0/3336909633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade406b340 con 0x7fade4073b40 2024-05-10T05:35:15.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.026+0000 7fae017dfb80 1 Processor -- start 2024-05-10T05:35:15.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.028+0000 7faddbe00700 1 -- 172.21.15.131:0/3336909633 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fadd4005040 con 0x7fade4073b40 2024-05-10T05:35:15.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.029+0000 7fae017dfb80 1 -- start start 2024-05-10T05:35:15.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.029+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5d6a0 0x55e523a5daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.029+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a5e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.029+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5eec0 0x55e523a657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.029+0000 7fae017dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523984150 con 0x55e523a5eec0 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.029+0000 7fae017dfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523981b10 con 0x55e523a5e570 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.029+0000 7fae017dfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52397d400 con 0x55e523a5d6a0 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.030+0000 7fade3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5d6a0 0x55e523a5daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.030+0000 7fade3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5d6a0 0x55e523a5daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51708/0 (socket says 172.21.15.131:51708) 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.030+0000 7fade3400700 1 -- 172.21.15.131:0/519380614 learned_addr learned my addr 172.21.15.131:0/519380614 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.030+0000 7fade3400700 1 -- 172.21.15.131:0/519380614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 msgr2=0x55e523a5e980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.030+0000 7fade3400700 1 --2- 172.21.15.131:0/519380614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a5e980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.030+0000 7fade3400700 1 -- 172.21.15.131:0/519380614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5eec0 msgr2=0x55e523a657c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:15.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.030+0000 7fade3400700 1 --2- 172.21.15.131:0/519380614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5eec0 0x55e523a657c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.030+0000 7fade3400700 1 -- 172.21.15.131:0/519380614 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5237fe3b0 con 0x55e523a5d6a0 2024-05-10T05:35:15.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.031+0000 7fade3400700 1 --2- 172.21.15.131:0/519380614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5d6a0 0x55e523a5daf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faddc009ab0 tx=0x7faddc009dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a84aa54c582c65c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:15.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.031+0000 7fadd3e00700 1 -- 172.21.15.131:0/519380614 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc004a30 con 0x55e523a5d6a0 2024-05-10T05:35:15.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.031+0000 7fadd3e00700 1 -- 172.21.15.131:0/519380614 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc0054e0 con 0x55e523a5d6a0 2024-05-10T05:35:15.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.031+0000 7fadd3e00700 1 -- 172.21.15.131:0/519380614 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc00ed20 con 0x55e523a5d6a0 2024-05-10T05:35:15.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.032+0000 7fae017dfb80 1 -- 172.21.15.131:0/519380614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5d6a0 msgr2=0x55e523a5daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:15.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.032+0000 7fae017dfb80 1 --2- 172.21.15.131:0/519380614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5d6a0 0x55e523a5daf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faddc009ab0 tx=0x7faddc009dc0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.033+0000 7faddbe00700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fadc80a1240 0x7fadc80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.033+0000 7faddbe00700 1 -- 172.21.15.131:0/3336909633 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fadd40cdd60 con 0x7fade4073b40 2024-05-10T05:35:15.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.034+0000 7fade3400700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fadc80a1240 0x7fadc80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.035+0000 7fae017dfb80 1 -- 172.21.15.131:0/519380614 shutdown_connections 2024-05-10T05:35:15.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.035+0000 7fae017dfb80 1 --2- 172.21.15.131:0/519380614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5eec0 0x55e523a657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.035+0000 7fae017dfb80 1 --2- 172.21.15.131:0/519380614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5d6a0 0x55e523a5daf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.035+0000 7fae017dfb80 1 --2- 172.21.15.131:0/519380614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a5e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.035+0000 7fae017dfb80 1 -- 172.21.15.131:0/519380614 >> 172.21.15.131:0/519380614 conn(0x55e5239bfa60 msgr2=0x55e523a57fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:15.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.035+0000 7fae017dfb80 1 -- 172.21.15.131:0/519380614 shutdown_connections 2024-05-10T05:35:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.035+0000 7fae017dfb80 1 -- 172.21.15.131:0/519380614 wait complete. 2024-05-10T05:35:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.036+0000 7fade3400700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fadc80a1240 0x7fadc80a3750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faddc009a80 tx=0x7faddc00ee00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.036+0000 7fae017dfb80 1 -- start start 2024-05-10T05:35:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.036+0000 7fae017dfb80 1 Processor -- start 2024-05-10T05:35:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 0x55e523a786c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a78c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 0x55e523a7fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fae017dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523919560 con 0x55e523a5d6a0 2024-05-10T05:35:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fae017dfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523919200 con 0x55e523a5e570 2024-05-10T05:35:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fae017dfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5238006a0 con 0x55e523a5eec0 2024-05-10T05:35:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fae017dfb80 10 client.0 _ll_register_callbacks cb 0x55e523922b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fadda000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a78c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fadda000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a78c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46392/0 (socket says 172.21.15.131:46392) 2024-05-10T05:35:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fadda000700 1 -- 172.21.15.131:0/822220576 learned_addr learned my addr 172.21.15.131:0/822220576 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.037+0000 7fade3400700 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 0x55e523a7fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.040+0000 7fade2a00700 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 0x55e523a786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.041+0000 7fade3400700 1 -- 172.21.15.131:0/822220576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 msgr2=0x55e523a78c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:15.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.041+0000 7fade3400700 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a78c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.041+0000 7fade3400700 1 -- 172.21.15.131:0/822220576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 msgr2=0x55e523a786c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.041+0000 7fade3400700 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 0x55e523a786c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.041+0000 7fade3400700 1 -- 172.21.15.131:0/822220576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faddc009760 con 0x55e523a5eec0 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.041+0000 7fade2a00700 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 0x55e523a786c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.041+0000 7fade3400700 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 0x55e523a7fb60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faddc00e800 tx=0x7faddc009dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd09dccd30edcba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.041+0000 7fadd0c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.042+0000 7fadd0c00700 1 -- 172.21.15.131:0/822220576 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc023ad0 con 0x55e523a5eec0 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.042+0000 7fadd0c00700 1 -- 172.21.15.131:0/822220576 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7faddc014070 con 0x55e523a5eec0 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.042+0000 7fae017dfb80 1 -- 172.21.15.131:0/822220576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5238d6270 con 0x55e523a5eec0 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.042+0000 7fadd0c00700 1 -- 172.21.15.131:0/822220576 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faddc023c30 con 0x55e523a5eec0 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.042+0000 7fae017dfb80 10 client.25723 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.042+0000 7fae017dfb80 1 -- 172.21.15.131:0/822220576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e5237aab60 con 0x55e523a5eec0 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.042+0000 7fadd0c00700 1 -- 172.21.15.131:0/822220576 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc03d020 con 0x55e523a5eec0 2024-05-10T05:35:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.043+0000 7fadd0c00700 1 -- 172.21.15.131:0/822220576 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faddc02e440 con 0x55e523a5eec0 2024-05-10T05:35:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.043+0000 7fae017dfb80 10 client.25723 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.043+0000 7fae017dfb80 10 client.25723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:35:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.043+0000 7fae017dfb80 20 client.25723 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.043+0000 7fae017dfb80 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e523a84250 0x55e523a866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.043+0000 7fae017dfb80 4 client.25723 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.043+0000 7fae017dfb80 1 -- 172.21.15.131:0/822220576 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5238d6270 con 0x55e523a84250 2024-05-10T05:35:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.044+0000 7fade2a00700 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e523a84250 0x55e523a866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.045+0000 7fadd0c00700 10 client.25723 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.045+0000 7fade2a00700 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e523a84250 0x55e523a866b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc28435d3e1fc3d8 server_cookie=799aa37dc85e8167 in_seq=0 out_seq=0 2024-05-10T05:35:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fadd0c00700 1 -- 172.21.15.131:0/822220576 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5238d6270 con 0x55e523a84250 2024-05-10T05:35:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fadd0c00700 10 client.25723 handle_command_reply: tid=0 2024-05-10T05:35:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fae017dfb80 1 client.25723 shutdown 2024-05-10T05:35:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fae017dfb80 1 -- 172.21.15.131:0/822220576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 msgr2=0x55e523a7fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fae017dfb80 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 0x55e523a7fb60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faddc00e800 tx=0x7faddc009dc0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fae017dfb80 1 -- 172.21.15.131:0/822220576 shutdown_connections 2024-05-10T05:35:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fae017dfb80 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e523a84250 0x55e523a866b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fae017dfb80 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 0x55e523a786c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fae017dfb80 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 0x55e523a7fb60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fadd0c00700 0 client.25723 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:15.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fae017dfb80 1 --2- 172.21.15.131:0/822220576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a78c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fae017dfb80 1 -- 172.21.15.131:0/822220576 >> 172.21.15.131:0/822220576 conn(0x55e5239bfa60 msgr2=0x55e523a57fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:15.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.046+0000 7fae017dfb80 1 -- 172.21.15.131:0/822220576 shutdown_connections 2024-05-10T05:35:15.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.047+0000 7fae017dfb80 1 -- 172.21.15.131:0/822220576 wait complete. 2024-05-10T05:35:15.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.047+0000 7fae017dfb80 20 client.25723 trim_cache size 0 max 16384 2024-05-10T05:35:15.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.076+0000 7fae017dfb80 1 Processor -- start 2024-05-10T05:35:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.076+0000 7fae017dfb80 1 -- start start 2024-05-10T05:35:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.076+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5d6a0 0x55e523a86670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.076+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5e570 0x55e523a7fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.078+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5eec0 0x55e523a617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.078+0000 7fadda000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5e570 0x55e523a7fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.078+0000 7fadda000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5e570 0x55e523a7fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51720/0 (socket says 172.21.15.131:51720) 2024-05-10T05:35:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.078+0000 7fadda000700 1 -- 172.21.15.131:0/1428916928 learned_addr learned my addr 172.21.15.131:0/1428916928 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.078+0000 7fae017dfb80 1 -- 172.21.15.131:0/1428916928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523a54d60 con 0x55e523a5eec0 2024-05-10T05:35:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.078+0000 7fae017dfb80 1 -- 172.21.15.131:0/1428916928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523a80c90 con 0x55e523a5d6a0 2024-05-10T05:35:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.078+0000 7fae017dfb80 1 -- 172.21.15.131:0/1428916928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523a8b930 con 0x55e523a5e570 2024-05-10T05:35:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.079+0000 7fade2a00700 1 --2- 172.21.15.131:0/1428916928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5d6a0 0x55e523a86670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.079+0000 7fadda000700 1 -- 172.21.15.131:0/1428916928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5d6a0 msgr2=0x55e523a86670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.079+0000 7fadda000700 1 --2- 172.21.15.131:0/1428916928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5d6a0 0x55e523a86670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.079+0000 7fadda000700 1 -- 172.21.15.131:0/1428916928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5eec0 msgr2=0x55e523a617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.079+0000 7fadda000700 1 --2- 172.21.15.131:0/1428916928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5eec0 0x55e523a617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.079+0000 7fadda000700 1 -- 172.21.15.131:0/1428916928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e523943560 con 0x55e523a5e570 2024-05-10T05:35:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.079+0000 7fadda000700 1 --2- 172.21.15.131:0/1428916928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5e570 0x55e523a7fb20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009fb0 tx=0x7fadcc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2ff2d80d20eef16 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.080+0000 7fadd8c00700 1 -- 172.21.15.131:0/1428916928 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc016070 con 0x55e523a5e570 2024-05-10T05:35:15.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.080+0000 7fadd8c00700 1 -- 172.21.15.131:0/1428916928 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadcc01a8d0 con 0x55e523a5e570 2024-05-10T05:35:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.080+0000 7fadd8c00700 1 -- 172.21.15.131:0/1428916928 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc0109b0 con 0x55e523a5e570 2024-05-10T05:35:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.085+0000 7fae017dfb80 1 -- 172.21.15.131:0/1428916928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5e570 msgr2=0x55e523a7fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.085+0000 7fae017dfb80 1 --2- 172.21.15.131:0/1428916928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5e570 0x55e523a7fb20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009fb0 tx=0x7fadcc011720 comp rx=0 tx=0).stop 2024-05-10T05:35:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.085+0000 7fae017dfb80 1 -- 172.21.15.131:0/1428916928 shutdown_connections 2024-05-10T05:35:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.085+0000 7fae017dfb80 1 --2- 172.21.15.131:0/1428916928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5eec0 0x55e523a617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.085+0000 7fae017dfb80 1 --2- 172.21.15.131:0/1428916928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5d6a0 0x55e523a86670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.085+0000 7fae017dfb80 1 --2- 172.21.15.131:0/1428916928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5e570 0x55e523a7fb20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.085+0000 7fae017dfb80 1 -- 172.21.15.131:0/1428916928 >> 172.21.15.131:0/1428916928 conn(0x55e5239bfa60 msgr2=0x55e523a54180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:15.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.086+0000 7fae017dfb80 1 -- 172.21.15.131:0/1428916928 shutdown_connections 2024-05-10T05:35:15.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.086+0000 7fae017dfb80 1 -- 172.21.15.131:0/1428916928 wait complete. 2024-05-10T05:35:15.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.087+0000 7fae017dfb80 1 -- start start 2024-05-10T05:35:15.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.088+0000 7fae017dfb80 1 Processor -- start 2024-05-10T05:35:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.097+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 0x55e523a7d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.097+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a7ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fae017dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 0x55e523a7ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fadda000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a7ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fadda000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a7ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46404/0 (socket says 172.21.15.131:46404) 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fadda000700 1 -- 172.21.15.131:0/510553863 learned_addr learned my addr 172.21.15.131:0/510553863 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523a7f340 con 0x55e523a5d6a0 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523a7f4b0 con 0x55e523a5e570 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523a7f670 con 0x55e523a5eec0 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fade3400700 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 0x55e523a7ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fae017dfb80 10 client.0 _ll_register_callbacks cb 0x55e523922b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fade3400700 1 -- 172.21.15.131:0/510553863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 msgr2=0x55e523a7ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fade3400700 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a7ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.098+0000 7fade3400700 1 -- 172.21.15.131:0/510553863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 msgr2=0x55e523a7d990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.102+0000 7fade2a00700 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 0x55e523a7d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.102+0000 7fade3400700 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 0x55e523a7d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.102+0000 7fade3400700 1 -- 172.21.15.131:0/510553863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc009c60 con 0x55e523a5eec0 2024-05-10T05:35:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.102+0000 7fade2a00700 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 0x55e523a7d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:15.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.103+0000 7fade3400700 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 0x55e523a7ee00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faddc00f010 tx=0x7faddc005410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b1786f5dee8ccc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:15.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.103+0000 7fadd0c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:35:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.108+0000 7fadd0c00700 1 -- 172.21.15.131:0/510553863 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc023950 con 0x55e523a5eec0 2024-05-10T05:35:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.109+0000 7fadd0c00700 1 -- 172.21.15.131:0/510553863 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7faddc014040 con 0x55e523a5eec0 2024-05-10T05:35:15.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.109+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faddc0054e0 con 0x55e523a5eec0 2024-05-10T05:35:15.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.111+0000 7fadd0c00700 1 -- 172.21.15.131:0/510553863 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faddc023ab0 con 0x55e523a5eec0 2024-05-10T05:35:15.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.111+0000 7fae017dfb80 10 client.25729 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:15.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.111+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e523802cd0 con 0x55e523a5eec0 2024-05-10T05:35:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.112+0000 7fadd0c00700 1 -- 172.21.15.131:0/510553863 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc0054e0 con 0x55e523a5eec0 2024-05-10T05:35:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.112+0000 7fadd0c00700 1 -- 172.21.15.131:0/510553863 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faddc049070 con 0x55e523a5eec0 2024-05-10T05:35:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.113+0000 7fae017dfb80 10 client.25729 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.113+0000 7fae017dfb80 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:35:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.113+0000 7fae017dfb80 20 client.25729 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.113+0000 7fae017dfb80 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e523adb350 0x55e523add770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.117+0000 7fae017dfb80 4 client.25729 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:35:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.117+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e523a2c780 con 0x55e523adb350 2024-05-10T05:35:15.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.121+0000 7fade2a00700 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e523adb350 0x55e523add770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.123+0000 7fadd0c00700 10 client.25729 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:15.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.123+0000 7fade2a00700 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e523adb350 0x55e523add770 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de2d0ba145fd3a9 server_cookie=26ace7d8289f3768 in_seq=0 out_seq=0 2024-05-10T05:35:15.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.124+0000 7fadd0c00700 1 -- 172.21.15.131:0/510553863 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e523a2c780 con 0x55e523adb350 2024-05-10T05:35:15.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.124+0000 7fadd0c00700 10 client.25729 handle_command_reply: tid=0 2024-05-10T05:35:15.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.124+0000 7fae017dfb80 1 client.25729 shutdown 2024-05-10T05:35:15.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.128+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 msgr2=0x55e523a7ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:15.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.128+0000 7fae017dfb80 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 0x55e523a7ee00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faddc00f010 tx=0x7faddc005410 comp rx=0 tx=0).stop 2024-05-10T05:35:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.131+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 shutdown_connections 2024-05-10T05:35:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.131+0000 7fae017dfb80 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e523adb350 0x55e523add770 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.131+0000 7fae017dfb80 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e523a5d6a0 0x55e523a7d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.131+0000 7fae017dfb80 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e523a5eec0 0x55e523a7ee00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.131+0000 7fae017dfb80 1 --2- 172.21.15.131:0/510553863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e523a5e570 0x55e523a7ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.131+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 >> 172.21.15.131:0/510553863 conn(0x55e5239bfa60 msgr2=0x55e523a54180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:15.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.131+0000 7fadd0c00700 0 client.25729 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.133+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 shutdown_connections 2024-05-10T05:35:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.133+0000 7fae017dfb80 1 -- 172.21.15.131:0/510553863 wait complete. 2024-05-10T05:35:15.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.146+0000 7fae017dfb80 20 client.25729 trim_cache size 0 max 16384 2024-05-10T05:35:15.149 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:35:15.149 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:35:15.149 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:35:15.150 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:35:15.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.153+0000 7fade9800700 1 -- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fadc80a1240 msgr2=0x7fadc80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:15.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.153+0000 7fade9800700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fadc80a1240 0x7fadc80a3750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faddc009a80 tx=0x7faddc00ee00 comp rx=0 tx=0).stop 2024-05-10T05:35:15.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.153+0000 7fade9800700 1 -- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade4073b40 msgr2=0x7fade419e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:15.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.153+0000 7fade9800700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade4073b40 0x7fade419e4c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fade4074e30 tx=0x7fadd400bd50 comp rx=0 tx=0).stop 2024-05-10T05:35:15.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.156+0000 7fade9800700 1 -- 172.21.15.131:0/3336909633 shutdown_connections 2024-05-10T05:35:15.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.156+0000 7fade9800700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fadc80a1240 0x7fadc80a3750 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.156+0000 7fade9800700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade4073b40 0x7fade419e4c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.157+0000 7fade9800700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fade410e220 0x7fade41a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.157+0000 7fade9800700 1 --2- 172.21.15.131:0/3336909633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade4075140 0x7fade419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.157+0000 7fade9800700 1 -- 172.21.15.131:0/3336909633 >> 172.21.15.131:0/3336909633 conn(0x7fade40fd210 msgr2=0x7fade40ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:15.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.157+0000 7fade9800700 1 -- 172.21.15.131:0/3336909633 shutdown_connections 2024-05-10T05:35:15.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.157+0000 7fade9800700 1 -- 172.21.15.131:0/3336909633 wait complete. 2024-05-10T05:35:15.246 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:35:15.247 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:35:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.928+0000 7f7385000700 1 -- 172.21.15.131:0/2146294290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7380109c50 msgr2=0x7f738010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.928+0000 7f7385000700 1 --2- 172.21.15.131:0/2146294290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7380109c50 0x7f738010e560 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f73700098f0 tx=0x7f7370009c00 comp rx=0 tx=0).stop 2024-05-10T05:35:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.929+0000 7f7385000700 1 -- 172.21.15.131:0/2146294290 shutdown_connections 2024-05-10T05:35:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.929+0000 7f7385000700 1 --2- 172.21.15.131:0/2146294290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7380109c50 0x7f738010e560 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.929+0000 7f7385000700 1 --2- 172.21.15.131:0/2146294290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7380108630 0x7f7380108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.929+0000 7f7385000700 1 --2- 172.21.15.131:0/2146294290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7380109300 0x7f7380109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.929+0000 7f7385000700 1 -- 172.21.15.131:0/2146294290 >> 172.21.15.131:0/2146294290 conn(0x7f738006ea00 msgr2=0x7f7380070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.929+0000 7f7385000700 1 -- 172.21.15.131:0/2146294290 shutdown_connections 2024-05-10T05:35:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.929+0000 7f7385000700 1 -- 172.21.15.131:0/2146294290 wait complete. 2024-05-10T05:35:15.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.931+0000 7f7385000700 1 Processor -- start 2024-05-10T05:35:15.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.931+0000 7f7385000700 1 -- start start 2024-05-10T05:35:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.932+0000 7f7385000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7380108630 0x7f73801a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.932+0000 7f7385000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7380109300 0x7f73801a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.932+0000 7f7385000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7380109c50 0x7f73801ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.932+0000 7f7385000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73801ae860 con 0x7f7380109300 2024-05-10T05:35:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.932+0000 7f7385000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73801ae9d0 con 0x7f7380109c50 2024-05-10T05:35:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.932+0000 7f7385000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73801aeb90 con 0x7f7380108630 2024-05-10T05:35:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.932+0000 7f737d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7380109300 0x7f73801a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.933+0000 7f737d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7380109300 0x7f73801a72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38222/0 (socket says 172.21.15.131:38222) 2024-05-10T05:35:15.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.933+0000 7f737d400700 1 -- 172.21.15.131:0/545448211 learned_addr learned my addr 172.21.15.131:0/545448211 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:15.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.933+0000 7f737e800700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7380109c50 0x7f73801ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.933+0000 7f737d400700 1 -- 172.21.15.131:0/545448211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7380108630 msgr2=0x7f73801a6d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:35:15.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.933+0000 7f737d400700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7380108630 0x7f73801a6d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.933+0000 7f737d400700 1 -- 172.21.15.131:0/545448211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7380109c50 msgr2=0x7f73801ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:15.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.933+0000 7f737d400700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7380109c50 0x7f73801ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:15.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.933+0000 7f737d400700 1 -- 172.21.15.131:0/545448211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7370009520 con 0x7f7380109300 2024-05-10T05:35:15.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.934+0000 7f737d400700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7380109300 0x7f73801a72c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f736c00d950 tx=0x7f736c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:15.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.934+0000 7f7376a00700 1 -- 172.21.15.131:0/545448211 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c009b40 con 0x7f7380109300 2024-05-10T05:35:15.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.935+0000 7f7376a00700 1 -- 172.21.15.131:0/545448211 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f736c009ca0 con 0x7f7380109300 2024-05-10T05:35:15.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.935+0000 7f7376a00700 1 -- 172.21.15.131:0/545448211 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c00f620 con 0x7f7380109300 2024-05-10T05:35:15.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.935+0000 7f7385000700 1 -- 172.21.15.131:0/545448211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73801aee70 con 0x7f7380109300 2024-05-10T05:35:15.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.936+0000 7f7385000700 1 -- 172.21.15.131:0/545448211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73801af2a0 con 0x7f7380109300 2024-05-10T05:35:15.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.938+0000 7f7385000700 1 -- 172.21.15.131:0/545448211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738010dcd0 con 0x7f7380109300 2024-05-10T05:35:15.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.944+0000 7f7376a00700 1 -- 172.21.15.131:0/545448211 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f736c010460 con 0x7f7380109300 2024-05-10T05:35:15.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.946+0000 7f7376a00700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73640a1220 0x7f73640a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:15.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.946+0000 7f7376a00700 1 -- 172.21.15.131:0/545448211 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f736c0cc650 con 0x7f7380109300 2024-05-10T05:35:15.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.947+0000 7f737de00700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73640a1220 0x7f73640a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.948+0000 7f737de00700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73640a1220 0x7f73640a3730 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f73780060b0 tx=0x7f7378006040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:15.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:15.953+0000 7f7376a00700 1 -- 172.21.15.131:0/545448211 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f736c095d80 con 0x7f7380109300 2024-05-10T05:35:16.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:16 smithi157 ceph-mon[78427]: pgmap v342: 97 pgs: 97 active+clean; 1.3 GiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 107 MiB/s wr, 45 op/s 2024-05-10T05:35:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:16 smithi142 ceph-mon[77667]: pgmap v342: 97 pgs: 97 active+clean; 1.3 GiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 107 MiB/s wr, 45 op/s 2024-05-10T05:35:16.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:16 smithi131 ceph-mon[37264]: pgmap v342: 97 pgs: 97 active+clean; 1.3 GiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 107 MiB/s wr, 45 op/s 2024-05-10T05:35:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.573+0000 7f7385000700 1 -- 172.21.15.131:0/545448211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f738002ce20 con 0x7f7380109300 2024-05-10T05:35:16.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.576+0000 7f7376a00700 1 -- 172.21.15.131:0/545448211 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f736c0993a0 con 0x7f7380109300 2024-05-10T05:35:16.578 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:35:16.579 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:33:24.915360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:35:16.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.591+0000 7f7385000700 1 -- 172.21.15.131:0/545448211 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73640a1220 msgr2=0x7f73640a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:16.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.591+0000 7f7385000700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73640a1220 0x7f73640a3730 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f73780060b0 tx=0x7f7378006040 comp rx=0 tx=0).stop 2024-05-10T05:35:16.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.591+0000 7f7385000700 1 -- 172.21.15.131:0/545448211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7380109300 msgr2=0x7f73801a72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:16.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.591+0000 7f7385000700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7380109300 0x7f73801a72c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f736c00d950 tx=0x7f736c00dd10 comp rx=0 tx=0).stop 2024-05-10T05:35:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.593+0000 7f7385000700 1 -- 172.21.15.131:0/545448211 shutdown_connections 2024-05-10T05:35:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.593+0000 7f7385000700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73640a1220 0x7f73640a3730 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.593+0000 7f7385000700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7380109c50 0x7f73801ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.593+0000 7f7385000700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7380108630 0x7f73801a6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.593+0000 7f7385000700 1 --2- 172.21.15.131:0/545448211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7380109300 0x7f73801a72c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.594+0000 7f7385000700 1 -- 172.21.15.131:0/545448211 >> 172.21.15.131:0/545448211 conn(0x7f738006ea00 msgr2=0x7f738006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.594+0000 7f7385000700 1 -- 172.21.15.131:0/545448211 shutdown_connections 2024-05-10T05:35:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:16.595+0000 7f7385000700 1 -- 172.21.15.131:0/545448211 wait complete. 2024-05-10T05:35:16.600 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-05-10T05:35:16.638 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:35:17.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.272+0000 7f5c36000700 1 -- 172.21.15.131:0/2193326653 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28004230 con 0x7f5c38109c50 2024-05-10T05:35:17.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.273+0000 7f5c3e600700 1 -- 172.21.15.131:0/2193326653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38109c50 msgr2=0x7f5c3810a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:17.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.273+0000 7f5c3e600700 1 --2- 172.21.15.131:0/2193326653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38109c50 0x7f5c3810a540 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5c280099e0 tx=0x7f5c28009cf0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.274+0000 7f5c3e600700 1 -- 172.21.15.131:0/2193326653 shutdown_connections 2024-05-10T05:35:17.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.275+0000 7f5c3e600700 1 --2- 172.21.15.131:0/2193326653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38109c50 0x7f5c3810a540 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.275+0000 7f5c3e600700 1 --2- 172.21.15.131:0/2193326653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c38108630 0x7f5c38108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.275+0000 7f5c3e600700 1 --2- 172.21.15.131:0/2193326653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38109300 0x7f5c38109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.275+0000 7f5c3e600700 1 -- 172.21.15.131:0/2193326653 >> 172.21.15.131:0/2193326653 conn(0x7f5c3806ea00 msgr2=0x7f5c38070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:17.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.276+0000 7f5c3e600700 1 -- 172.21.15.131:0/2193326653 shutdown_connections 2024-05-10T05:35:17.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.276+0000 7f5c3e600700 1 -- 172.21.15.131:0/2193326653 wait complete. 2024-05-10T05:35:17.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.278+0000 7f5c3e600700 1 Processor -- start 2024-05-10T05:35:17.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.278+0000 7f5c3e600700 1 -- start start 2024-05-10T05:35:17.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.279+0000 7f5c3e600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38108630 0x7f5c381a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.279+0000 7f5c3e600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c38109300 0x7f5c381a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.280+0000 7f5c3e600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38109c50 0x7f5c381a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.280+0000 7f5c36a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c38109300 0x7f5c381a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.280+0000 7f5c36a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c38109300 0x7f5c381a2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51756/0 (socket says 172.21.15.131:51756) 2024-05-10T05:35:17.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.280+0000 7f5c37400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38108630 0x7f5c381a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.280+0000 7f5c37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38109c50 0x7f5c381a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.280+0000 7f5c37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38109c50 0x7f5c381a9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46434/0 (socket says 172.21.15.131:46434) 2024-05-10T05:35:17.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.280+0000 7f5c37400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38108630 0x7f5c381a2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38248/0 (socket says 172.21.15.131:38248) 2024-05-10T05:35:17.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.281+0000 7f5c36a00700 1 -- 172.21.15.131:0/4262624065 learned_addr learned my addr 172.21.15.131:0/4262624065 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:17.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.281+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381aa130 con 0x7f5c38108630 2024-05-10T05:35:17.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.281+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381aa270 con 0x7f5c38109c50 2024-05-10T05:35:17.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.281+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381aa400 con 0x7f5c38109300 2024-05-10T05:35:17.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.282+0000 7f5c37e00700 1 -- 172.21.15.131:0/4262624065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c38109300 msgr2=0x7f5c381a2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:17.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.282+0000 7f5c37e00700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c38109300 0x7f5c381a2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.282+0000 7f5c37e00700 1 -- 172.21.15.131:0/4262624065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38108630 msgr2=0x7f5c381a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:17.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.282+0000 7f5c37e00700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38108630 0x7f5c381a2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.282+0000 7f5c37e00700 1 -- 172.21.15.131:0/4262624065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c280095f0 con 0x7f5c38109c50 2024-05-10T05:35:17.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.282+0000 7f5c36a00700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c38109300 0x7f5c381a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:17.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.282+0000 7f5c37e00700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38109c50 0x7f5c381a9b60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5c280099e0 tx=0x7f5c2800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:17.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.283+0000 7f5c2fe00700 1 -- 172.21.15.131:0/4262624065 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2801d070 con 0x7f5c38109c50 2024-05-10T05:35:17.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.283+0000 7f5c2fe00700 1 -- 172.21.15.131:0/4262624065 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c28005510 con 0x7f5c38109c50 2024-05-10T05:35:17.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.283+0000 7f5c2fe00700 1 -- 172.21.15.131:0/4262624065 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28004d80 con 0x7f5c38109c50 2024-05-10T05:35:17.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.283+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c381aa680 con 0x7f5c38109c50 2024-05-10T05:35:17.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.283+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c381aaa50 con 0x7f5c38109c50 2024-05-10T05:35:17.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.291+0000 7f5c2fe00700 1 -- 172.21.15.131:0/4262624065 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5c2802b440 con 0x7f5c38109c50 2024-05-10T05:35:17.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.293+0000 7f5c2fe00700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c1c0a1240 0x7f5c1c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.293+0000 7f5c2fe00700 1 -- 172.21.15.131:0/4262624065 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5c280cd990 con 0x7f5c38109c50 2024-05-10T05:35:17.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.295+0000 7f5c565b0b80 1 Processor -- start 2024-05-10T05:35:17.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.296+0000 7f5c565b0b80 1 -- start start 2024-05-10T05:35:17.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.297+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafeaeb0 0x55afdafeb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.297+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafebd80 0x55afdafec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.297+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 0x55afdaff2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.297+0000 7f5c565b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdaf09150 con 0x55afdafeaeb0 2024-05-10T05:35:17.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.297+0000 7f5c565b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdaf06b10 con 0x55afdafec6d0 2024-05-10T05:35:17.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.297+0000 7f5c565b0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdaf02400 con 0x55afdafebd80 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.298+0000 7f5c37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 0x55afdaff2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.298+0000 7f5c36a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafebd80 0x55afdafec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.298+0000 7f5c36a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafebd80 0x55afdafec190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51760/0 (socket says 172.21.15.131:51760) 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.298+0000 7f5c36a00700 1 -- 172.21.15.131:0/1967401452 learned_addr learned my addr 172.21.15.131:0/1967401452 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.298+0000 7f5c36a00700 1 -- 172.21.15.131:0/1967401452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 msgr2=0x55afdaff2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.299+0000 7f5c36a00700 1 --2- 172.21.15.131:0/1967401452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 0x55afdaff2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.299+0000 7f5c37400700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c1c0a1240 0x7f5c1c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.299+0000 7f5c36a00700 1 -- 172.21.15.131:0/1967401452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafeaeb0 msgr2=0x55afdafeb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.299+0000 7f5c36a00700 1 --2- 172.21.15.131:0/1967401452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafeaeb0 0x55afdafeb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.299+0000 7f5c36a00700 1 -- 172.21.15.131:0/1967401452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afdad833b0 con 0x55afdafebd80 2024-05-10T05:35:17.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.299+0000 7f5c36a00700 1 --2- 172.21.15.131:0/1967401452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafebd80 0x55afdafec190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400bf60 tx=0x7f5c2400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cb939cad61e5744 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:17.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.299+0000 7f5c2cc00700 1 -- 172.21.15.131:0/1967401452 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2401a460 con 0x55afdafebd80 2024-05-10T05:35:17.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.300+0000 7f5c2cc00700 1 -- 172.21.15.131:0/1967401452 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c2401aa80 con 0x55afdafebd80 2024-05-10T05:35:17.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.300+0000 7f5c2cc00700 1 -- 172.21.15.131:0/1967401452 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24011a50 con 0x55afdafebd80 2024-05-10T05:35:17.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.301+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1967401452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafebd80 msgr2=0x55afdafec190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:17.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.301+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/1967401452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafebd80 0x55afdafec190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400bf60 tx=0x7f5c2400bf90 comp rx=0 tx=0).stop 2024-05-10T05:35:17.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.301+0000 7f5c37e00700 1 --2- 172.21.15.131:0/1967401452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 0x55afdaff2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:17.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:17 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/545448211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:35:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.302+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1967401452 shutdown_connections 2024-05-10T05:35:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.302+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/1967401452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafeaeb0 0x55afdafeb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.302+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/1967401452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 0x55afdaff2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.302+0000 7f5c37400700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c1c0a1240 0x7f5c1c0a3750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55afdafeab90 tx=0x7f5c30007850 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.302+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/1967401452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafebd80 0x55afdafec190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.302+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1967401452 >> 172.21.15.131:0/1967401452 conn(0x55afdaf44a60 msgr2=0x55afdafdcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.305+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1967401452 shutdown_connections 2024-05-10T05:35:17.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.305+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1967401452 wait complete. 2024-05-10T05:35:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.305+0000 7f5c565b0b80 1 -- start start 2024-05-10T05:35:17.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.306+0000 7f5c565b0b80 1 Processor -- start 2024-05-10T05:35:17.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.307+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafeaeb0 0x55afdaff4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.311+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafebd80 0x55afdaff5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.311+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafec6d0 0x55afdaffc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.311+0000 7f5c565b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdae9e560 con 0x55afdafebd80 2024-05-10T05:35:17.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.311+0000 7f5c565b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdae9e200 con 0x55afdafeaeb0 2024-05-10T05:35:17.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.311+0000 7f5c565b0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdad856a0 con 0x55afdafec6d0 2024-05-10T05:35:17.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.311+0000 7f5c565b0b80 10 client.0 _ll_register_callbacks cb 0x55afdaea7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:17.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.312+0000 7f5c37400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafeaeb0 0x55afdaff4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.312+0000 7f5c37400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafeaeb0 0x55afdaff4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46454/0 (socket says 172.21.15.131:46454) 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.312+0000 7f5c37400700 1 -- 172.21.15.131:0/1574878148 learned_addr learned my addr 172.21.15.131:0/1574878148 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.312+0000 7f5c37e00700 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafec6d0 0x55afdaffc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.312+0000 7f5c37400700 1 -- 172.21.15.131:0/1574878148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafec6d0 msgr2=0x55afdaffc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.312+0000 7f5c37400700 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafec6d0 0x55afdaffc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.312+0000 7f5c37400700 1 -- 172.21.15.131:0/1574878148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafebd80 msgr2=0x55afdaff5490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.312+0000 7f5c37400700 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafebd80 0x55afdaff5490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.312+0000 7f5c37400700 1 -- 172.21.15.131:0/1574878148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2400bb90 con 0x55afdafeaeb0 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.313+0000 7f5c37400700 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafeaeb0 0x55afdaff4f50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005b90 tx=0x7f5c300146d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0050139b1162cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.313+0000 7f5c21600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.313+0000 7f5c21600700 1 -- 172.21.15.131:0/1574878148 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3000ee60 con 0x55afdafeaeb0 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.314+0000 7f5c21600700 1 -- 172.21.15.131:0/1574878148 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5c3001f440 con 0x55afdafeaeb0 2024-05-10T05:35:17.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.314+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1574878148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afdae5b270 con 0x55afdafeaeb0 2024-05-10T05:35:17.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.315+0000 7f5c21600700 1 -- 172.21.15.131:0/1574878148 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c30014e90 con 0x55afdafeaeb0 2024-05-10T05:35:17.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.315+0000 7f5c565b0b80 10 client.25718 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:17.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.315+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1574878148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55afdad2fb60 con 0x55afdafeaeb0 2024-05-10T05:35:17.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.320+0000 7f5c21600700 1 -- 172.21.15.131:0/1574878148 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c30018070 con 0x55afdafeaeb0 2024-05-10T05:35:17.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.320+0000 7f5c21600700 1 -- 172.21.15.131:0/1574878148 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5c30028780 con 0x55afdafeaeb0 2024-05-10T05:35:17.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.321+0000 7f5c565b0b80 10 client.25718 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:17.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.321+0000 7f5c565b0b80 10 client.25718 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:35:17.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.321+0000 7f5c565b0b80 20 client.25718 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:17.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.322+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55afdb000ba0 0x55afdb003070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.322+0000 7f5c565b0b80 4 client.25718 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:17.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.322+0000 7f5c36a00700 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55afdb000ba0 0x55afdb003070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.323+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1574878148 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afdae5b270 con 0x55afdb000ba0 2024-05-10T05:35:17.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.323+0000 7f5c21600700 10 client.25718 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:35:17.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.324+0000 7f5c36a00700 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55afdb000ba0 0x55afdb003070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ec62a404ce89ff5 server_cookie=44a45719f014c886 in_seq=0 out_seq=0 2024-05-10T05:35:17.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.324+0000 7f5c21600700 1 -- 172.21.15.131:0/1574878148 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55afdae5b270 con 0x55afdb000ba0 2024-05-10T05:35:17.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.325+0000 7f5c21600700 10 client.25718 handle_command_reply: tid=0 2024-05-10T05:35:17.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.325+0000 7f5c565b0b80 1 client.25718 shutdown 2024-05-10T05:35:17.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.325+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1574878148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafeaeb0 msgr2=0x55afdaff4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:17.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.325+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafeaeb0 0x55afdaff4f50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005b90 tx=0x7f5c300146d0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.326+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1574878148 shutdown_connections 2024-05-10T05:35:17.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.326+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55afdb000ba0 0x55afdb003070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.326+0000 7f5c21600700 0 client.25718 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:35:17.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.327+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafec6d0 0x55afdaffc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.327+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafeaeb0 0x55afdaff4f50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.327+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/1574878148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafebd80 0x55afdaff5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.327+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1574878148 >> 172.21.15.131:0/1574878148 conn(0x55afdaf44a60 msgr2=0x55afdafdcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:17.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.327+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1574878148 shutdown_connections 2024-05-10T05:35:17.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.327+0000 7f5c565b0b80 1 -- 172.21.15.131:0/1574878148 wait complete. 2024-05-10T05:35:17.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.328+0000 7f5c565b0b80 20 client.25718 trim_cache size 0 max 16384 2024-05-10T05:35:17.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.357+0000 7f5c565b0b80 1 Processor -- start 2024-05-10T05:35:17.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.357+0000 7f5c565b0b80 1 -- start start 2024-05-10T05:35:17.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.359+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdafefb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.359+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafebd80 0x55afdaffc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.359+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafec6d0 0x55afdaff0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.359+0000 7f5c37400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdafefb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.359+0000 7f5c36a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafebd80 0x55afdaffc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.359+0000 7f5c37400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdafefb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51784/0 (socket says 172.21.15.131:51784) 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.359+0000 7f5c36a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafebd80 0x55afdaffc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46470/0 (socket says 172.21.15.131:46470) 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.360+0000 7f5c565b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdafdeb80 con 0x55afdafec6d0 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.360+0000 7f5c565b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdafd77c0 con 0x55afdafebd80 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.360+0000 7f5c565b0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdafef630 con 0x55afdafeaeb0 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.360+0000 7f5c37400700 1 -- 172.21.15.131:0/4136583044 learned_addr learned my addr 172.21.15.131:0/4136583044 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.360+0000 7f5c37e00700 1 --2- 172.21.15.131:0/4136583044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafec6d0 0x55afdaff0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.361+0000 7f5c37400700 1 -- 172.21.15.131:0/4136583044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafebd80 msgr2=0x55afdaffc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.361+0000 7f5c37400700 1 --2- 172.21.15.131:0/4136583044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafebd80 0x55afdaffc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.361+0000 7f5c37400700 1 -- 172.21.15.131:0/4136583044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafec6d0 msgr2=0x55afdaff0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.361+0000 7f5c37400700 1 --2- 172.21.15.131:0/4136583044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafec6d0 0x55afdaff0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.361+0000 7f5c37400700 1 -- 172.21.15.131:0/4136583044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afdaec8560 con 0x55afdafeaeb0 2024-05-10T05:35:17.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.369+0000 7f5c37400700 1 --2- 172.21.15.131:0/4136583044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdafefb00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c30014c50 tx=0x7f5c30007980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b228487c1cdaddd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:17.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.369+0000 7f5c2d600700 1 -- 172.21.15.131:0/4136583044 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30028400 con 0x55afdafeaeb0 2024-05-10T05:35:17.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.370+0000 7f5c2d600700 1 -- 172.21.15.131:0/4136583044 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c30018040 con 0x55afdafeaeb0 2024-05-10T05:35:17.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.370+0000 7f5c2d600700 1 -- 172.21.15.131:0/4136583044 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3001fe30 con 0x55afdafeaeb0 2024-05-10T05:35:17.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.372+0000 7f5c565b0b80 1 -- 172.21.15.131:0/4136583044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 msgr2=0x55afdafefb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:17.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.372+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/4136583044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdafefb00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c30014c50 tx=0x7f5c30007980 comp rx=0 tx=0).stop 2024-05-10T05:35:17.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.373+0000 7f5c565b0b80 1 -- 172.21.15.131:0/4136583044 shutdown_connections 2024-05-10T05:35:17.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.373+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/4136583044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafec6d0 0x55afdaff0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.373+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/4136583044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdafefb00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.373+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/4136583044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafebd80 0x55afdaffc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.373+0000 7f5c565b0b80 1 -- 172.21.15.131:0/4136583044 >> 172.21.15.131:0/4136583044 conn(0x55afdaf44a60 msgr2=0x55afdb002ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:17.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.374+0000 7f5c565b0b80 1 -- 172.21.15.131:0/4136583044 shutdown_connections 2024-05-10T05:35:17.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.374+0000 7f5c565b0b80 1 -- 172.21.15.131:0/4136583044 wait complete. 2024-05-10T05:35:17.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.374+0000 7f5c565b0b80 1 -- start start 2024-05-10T05:35:17.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.375+0000 7f5c565b0b80 1 Processor -- start 2024-05-10T05:35:17.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.376+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdaff9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.376+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafebd80 0x55afdaffa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.377+0000 7f5c565b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 0x55afdaffb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.377+0000 7f5c37400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdaff9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.377+0000 7f5c37400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdaff9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51800/0 (socket says 172.21.15.131:51800) 2024-05-10T05:35:17.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.377+0000 7f5c565b0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdaffb790 con 0x55afdafebd80 2024-05-10T05:35:17.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.377+0000 7f5c565b0b80 1 -- 172.21.15.131:0/889168787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdaffb900 con 0x55afdafec6d0 2024-05-10T05:35:17.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.377+0000 7f5c565b0b80 1 -- 172.21.15.131:0/889168787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afdaffbac0 con 0x55afdafeaeb0 2024-05-10T05:35:17.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.377+0000 7f5c565b0b80 10 client.0 _ll_register_callbacks cb 0x55afdaea7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:17.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.377+0000 7f5c37e00700 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 0x55afdaffb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.377+0000 7f5c36a00700 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafebd80 0x55afdaffa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.377+0000 7f5c37400700 1 -- 172.21.15.131:0/889168787 learned_addr learned my addr 172.21.15.131:0/889168787 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:17.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.379+0000 7f5c37e00700 1 -- 172.21.15.131:0/889168787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 msgr2=0x55afdaff9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:17.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.379+0000 7f5c37e00700 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdaff9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.379+0000 7f5c37e00700 1 -- 172.21.15.131:0/889168787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafebd80 msgr2=0x55afdaffa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:17.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.379+0000 7f5c37e00700 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafebd80 0x55afdaffa320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.379+0000 7f5c37e00700 1 -- 172.21.15.131:0/889168787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afdad2fb60 con 0x55afdafec6d0 2024-05-10T05:35:17.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.381+0000 7f5c21600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:35:17.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.381+0000 7f5c37e00700 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 0x55afdaffb250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c280cb4e0 tx=0x7f5c2809b570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f6a065d760f5e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:17.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.381+0000 7f5c21600700 1 -- 172.21.15.131:0/889168787 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2809a560 con 0x55afdafec6d0 2024-05-10T05:35:17.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.382+0000 7f5c21600700 1 -- 172.21.15.131:0/889168787 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5c2809eca0 con 0x55afdafec6d0 2024-05-10T05:35:17.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.382+0000 7f5c565b0b80 1 -- 172.21.15.131:0/889168787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c2401aa80 con 0x55afdafec6d0 2024-05-10T05:35:17.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.382+0000 7f5c21600700 1 -- 172.21.15.131:0/889168787 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c280a3260 con 0x55afdafec6d0 2024-05-10T05:35:17.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.382+0000 7f5c565b0b80 10 client.25724 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:17.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.383+0000 7f5c565b0b80 1 -- 172.21.15.131:0/889168787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55afdad87cd0 con 0x55afdafec6d0 2024-05-10T05:35:17.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.383+0000 7f5c21600700 1 -- 172.21.15.131:0/889168787 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c2401aa80 con 0x55afdafec6d0 2024-05-10T05:35:17.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.385+0000 7f5c21600700 1 -- 172.21.15.131:0/889168787 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5c28099dd0 con 0x55afdafec6d0 2024-05-10T05:35:17.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.385+0000 7f5c565b0b80 10 client.25724 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:17.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.385+0000 7f5c565b0b80 10 client.25724 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:35:17.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.385+0000 7f5c565b0b80 20 client.25724 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:17.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.386+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55afdb000ba0 0x55afdb059880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:17.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.386+0000 7f5c565b0b80 4 client.25724 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:35:17.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.386+0000 7f5c565b0b80 1 -- 172.21.15.131:0/889168787 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55afdafb1780 con 0x55afdb000ba0 2024-05-10T05:35:17.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.386+0000 7f5c37400700 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55afdb000ba0 0x55afdb059880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:17.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.387+0000 7f5c21600700 10 client.25724 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:35:17.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.387+0000 7f5c37400700 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55afdb000ba0 0x55afdb059880 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b40aa2427c1db497 server_cookie=16a737d09de62ca8 in_seq=0 out_seq=0 2024-05-10T05:35:17.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.388+0000 7f5c21600700 1 -- 172.21.15.131:0/889168787 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55afdafb1780 con 0x55afdb000ba0 2024-05-10T05:35:17.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.388+0000 7f5c21600700 10 client.25724 handle_command_reply: tid=0 2024-05-10T05:35:17.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.388+0000 7f5c565b0b80 1 client.25724 shutdown 2024-05-10T05:35:17.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.388+0000 7f5c565b0b80 1 -- 172.21.15.131:0/889168787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 msgr2=0x55afdaffb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:17.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.388+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 0x55afdaffb250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c280cb4e0 tx=0x7f5c2809b570 comp rx=0 tx=0).stop 2024-05-10T05:35:17.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.388+0000 7f5c565b0b80 1 -- 172.21.15.131:0/889168787 shutdown_connections 2024-05-10T05:35:17.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.388+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55afdb000ba0 0x55afdb059880 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.389+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdafec6d0 0x55afdaffb250 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.389+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55afdafeaeb0 0x55afdaff9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.389+0000 7f5c565b0b80 1 --2- 172.21.15.131:0/889168787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55afdafebd80 0x55afdaffa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.389+0000 7f5c565b0b80 1 -- 172.21.15.131:0/889168787 >> 172.21.15.131:0/889168787 conn(0x55afdaf44a60 msgr2=0x55afdafefb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:17.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.389+0000 7f5c21600700 0 client.25724 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:35:17.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.389+0000 7f5c565b0b80 1 -- 172.21.15.131:0/889168787 shutdown_connections 2024-05-10T05:35:17.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.389+0000 7f5c565b0b80 1 -- 172.21.15.131:0/889168787 wait complete. 2024-05-10T05:35:17.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.390+0000 7f5c565b0b80 20 client.25724 trim_cache size 0 max 16384 2024-05-10T05:35:17.391 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:35:17.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.391+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c1c0a1240 msgr2=0x7f5c1c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:17.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.391+0000 7f5c3e600700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c1c0a1240 0x7f5c1c0a3750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55afdafeab90 tx=0x7f5c30007850 comp rx=0 tx=0).stop 2024-05-10T05:35:17.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.391+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38109c50 msgr2=0x7f5c381a9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:17.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.391+0000 7f5c3e600700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38109c50 0x7f5c381a9b60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5c280099e0 tx=0x7f5c2800fa00 comp rx=0 tx=0).stop 2024-05-10T05:35:17.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.391+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 shutdown_connections 2024-05-10T05:35:17.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.391+0000 7f5c3e600700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5c1c0a1240 0x7f5c1c0a3750 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.391+0000 7f5c3e600700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38108630 0x7f5c381a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.391+0000 7f5c3e600700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c38109c50 0x7f5c381a9b60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.391+0000 7f5c3e600700 1 --2- 172.21.15.131:0/4262624065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c38109300 0x7f5c381a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:17.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.391+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 >> 172.21.15.131:0/4262624065 conn(0x7f5c3806ea00 msgr2=0x7f5c38070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:17.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.392+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 shutdown_connections 2024-05-10T05:35:17.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:17.392+0000 7f5c3e600700 1 -- 172.21.15.131:0/4262624065 wait complete. 2024-05-10T05:35:17.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:17 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/545448211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:35:17.425 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:17 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/545448211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:35:17.427 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:35:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.019+0000 7f4d4d800700 1 -- 172.21.15.131:0/219809461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d48100ac0 msgr2=0x7f4d48100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.019+0000 7f4d4d800700 1 --2- 172.21.15.131:0/219809461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d48100ac0 0x7f4d48100ed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000b0d0 tx=0x7f4d4000b3e0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.022+0000 7f4d4d800700 1 -- 172.21.15.131:0/219809461 shutdown_connections 2024-05-10T05:35:18.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.022+0000 7f4d4d800700 1 --2- 172.21.15.131:0/219809461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d48102580 0x7f4d48108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.022+0000 7f4d4d800700 1 --2- 172.21.15.131:0/219809461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d48100ac0 0x7f4d48100ed0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.022+0000 7f4d4d800700 1 --2- 172.21.15.131:0/219809461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d48101be0 0x7f4d48101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.022+0000 7f4d4d800700 1 -- 172.21.15.131:0/219809461 >> 172.21.15.131:0/219809461 conn(0x7f4d480fc0d0 msgr2=0x7f4d480fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.022+0000 7f4d4d800700 1 -- 172.21.15.131:0/219809461 shutdown_connections 2024-05-10T05:35:18.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.022+0000 7f4d4d800700 1 -- 172.21.15.131:0/219809461 wait complete. 2024-05-10T05:35:18.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.026+0000 7f4d4d800700 1 Processor -- start 2024-05-10T05:35:18.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.027+0000 7f4d4d800700 1 -- start start 2024-05-10T05:35:18.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.028+0000 7f4d4d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d48100ac0 0x7f4d48195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.028+0000 7f4d4d800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d48101be0 0x7f4d48196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.028+0000 7f4d4d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d48102580 0x7f4d481973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.028+0000 7f4d4d800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d481979c0 con 0x7f4d48101be0 2024-05-10T05:35:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.028+0000 7f4d4d800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d48197b30 con 0x7f4d48100ac0 2024-05-10T05:35:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.028+0000 7f4d4d800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d48197cf0 con 0x7f4d48102580 2024-05-10T05:35:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.029+0000 7f4d46a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d48100ac0 0x7f4d48195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.029+0000 7f4d46a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d48100ac0 0x7f4d48195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46504/0 (socket says 172.21.15.131:46504) 2024-05-10T05:35:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.029+0000 7f4d46a00700 1 -- 172.21.15.131:0/968064933 learned_addr learned my addr 172.21.15.131:0/968064933 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.029+0000 7f4d46000700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d48101be0 0x7f4d48196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.029+0000 7f4d47400700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d48102580 0x7f4d481973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.029+0000 7f4d46a00700 1 -- 172.21.15.131:0/968064933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d48102580 msgr2=0x7f4d481973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.029+0000 7f4d46a00700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d48102580 0x7f4d481973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.029+0000 7f4d46a00700 1 -- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d48101be0 msgr2=0x7f4d48196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.030+0000 7f4d46a00700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d48101be0 0x7f4d48196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.030+0000 7f4d46a00700 1 -- 172.21.15.131:0/968064933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40009d00 con 0x7f4d48100ac0 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.030+0000 7f4d46000700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d48101be0 0x7f4d48196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.030+0000 7f4d47400700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d48102580 0x7f4d481973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.030+0000 7f4d46a00700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d48100ac0 0x7f4d48195f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000b770 tx=0x7f4d40008e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.031+0000 7f4d3f400700 1 -- 172.21.15.131:0/968064933 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000e040 con 0x7f4d48100ac0 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.031+0000 7f4d3f400700 1 -- 172.21.15.131:0/968064933 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d40005110 con 0x7f4d48100ac0 2024-05-10T05:35:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.031+0000 7f4d3f400700 1 -- 172.21.15.131:0/968064933 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4001d6d0 con 0x7f4d48100ac0 2024-05-10T05:35:18.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.033+0000 7f4d4d800700 1 -- 172.21.15.131:0/968064933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d48197f70 con 0x7f4d48100ac0 2024-05-10T05:35:18.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.033+0000 7f4d4d800700 1 -- 172.21.15.131:0/968064933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d48198400 con 0x7f4d48100ac0 2024-05-10T05:35:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.043+0000 7f4d657dab80 1 Processor -- start 2024-05-10T05:35:18.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.047+0000 7f4d3f400700 1 -- 172.21.15.131:0/968064933 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4d40005280 con 0x7f4d48100ac0 2024-05-10T05:35:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.049+0000 7f4d3f400700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4d2c0a1400 0x7f4d2c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.049+0000 7f4d657dab80 1 -- start start 2024-05-10T05:35:18.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.050+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dcfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.050+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60dd0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.050+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 0x564c60dd77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.050+0000 7f4d657dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60cf6150 con 0x564c60dd0ec0 2024-05-10T05:35:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.050+0000 7f4d657dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60cf3b10 con 0x564c60dcf6a0 2024-05-10T05:35:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.050+0000 7f4d657dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60cef400 con 0x564c60dd0570 2024-05-10T05:35:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.050+0000 7f4d46000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60dd0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.051+0000 7f4d46000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60dd0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51822/0 (socket says 172.21.15.131:51822) 2024-05-10T05:35:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.051+0000 7f4d46000700 1 -- 172.21.15.131:0/2717357001 learned_addr learned my addr 172.21.15.131:0/2717357001 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.051+0000 7f4d46a00700 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dcfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.052+0000 7f4d47400700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4d2c0a1400 0x7f4d2c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.053+0000 7f4d47400700 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 0x564c60dd77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.053+0000 7f4d46000700 1 -- 172.21.15.131:0/2717357001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 msgr2=0x564c60dcfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.053+0000 7f4d3f400700 1 -- 172.21.15.131:0/968064933 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4d400cd080 con 0x7f4d48100ac0 2024-05-10T05:35:18.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.054+0000 7f4d46000700 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dcfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.054+0000 7f4d46000700 1 -- 172.21.15.131:0/2717357001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 msgr2=0x564c60dd77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.054+0000 7f4d46000700 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 0x564c60dd77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.054+0000 7f4d46000700 1 -- 172.21.15.131:0/2717357001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c60b703b0 con 0x564c60dd0570 2024-05-10T05:35:18.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.055+0000 7f4d47400700 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 0x564c60dd77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:18.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.055+0000 7f4d46a00700 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dcfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:18.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.055+0000 7f4d47400700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4d2c0a1400 0x7f4d2c0a3910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800d440 tx=0x7f4d3800f040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.055+0000 7f4d46000700 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60dd0980 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4d340115f0 tx=0x7f4d34011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba91649a5484f0b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.056+0000 7f4d33e00700 1 -- 172.21.15.131:0/2717357001 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34004030 con 0x564c60dd0570 2024-05-10T05:35:18.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.056+0000 7f4d33e00700 1 -- 172.21.15.131:0/2717357001 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d34003680 con 0x564c60dd0570 2024-05-10T05:35:18.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.057+0000 7f4d657dab80 1 -- 172.21.15.131:0/2717357001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 msgr2=0x564c60dd0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.058+0000 7f4d33e00700 1 -- 172.21.15.131:0/2717357001 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34004030 con 0x564c60dd0570 2024-05-10T05:35:18.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.058+0000 7f4d657dab80 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60dd0980 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4d340115f0 tx=0x7f4d34011900 comp rx=0 tx=0).stop 2024-05-10T05:35:18.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.058+0000 7f4d657dab80 1 -- 172.21.15.131:0/2717357001 shutdown_connections 2024-05-10T05:35:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.059+0000 7f4d657dab80 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 0x564c60dd77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.059+0000 7f4d657dab80 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dcfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.059+0000 7f4d657dab80 1 --2- 172.21.15.131:0/2717357001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60dd0980 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.059+0000 7f4d657dab80 1 -- 172.21.15.131:0/2717357001 >> 172.21.15.131:0/2717357001 conn(0x564c60d31a60 msgr2=0x564c60dc9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.060+0000 7f4d657dab80 1 -- 172.21.15.131:0/2717357001 shutdown_connections 2024-05-10T05:35:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.060+0000 7f4d657dab80 1 -- 172.21.15.131:0/2717357001 wait complete. 2024-05-10T05:35:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.063+0000 7f4d657dab80 1 -- start start 2024-05-10T05:35:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.063+0000 7f4d657dab80 1 Processor -- start 2024-05-10T05:35:18.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.064+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.064+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60deaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.064+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 0x564c60df1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.064+0000 7f4d657dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60c8b560 con 0x564c60dd0ec0 2024-05-10T05:35:18.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.064+0000 7f4d657dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60c8b200 con 0x564c60dcf6a0 2024-05-10T05:35:18.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.064+0000 7f4d657dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60b726a0 con 0x564c60dd0570 2024-05-10T05:35:18.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.064+0000 7f4d657dab80 10 client.0 _ll_register_callbacks cb 0x564c60c94b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.070+0000 7f4d46a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.071+0000 7f4d46000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60deaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.071+0000 7f4d47400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 0x564c60df1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.071+0000 7f4d46a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dea4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46520/0 (socket says 172.21.15.131:46520) 2024-05-10T05:35:18.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.071+0000 7f4d46a00700 1 -- 172.21.15.131:0/901626765 learned_addr learned my addr 172.21.15.131:0/901626765 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:18.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.072+0000 7f4d46000700 1 -- 172.21.15.131:0/901626765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 msgr2=0x564c60dea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.072+0000 7f4d46000700 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.072+0000 7f4d46000700 1 -- 172.21.15.131:0/901626765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 msgr2=0x564c60df1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.072+0000 7f4d46000700 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 0x564c60df1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.072+0000 7f4d46000700 1 -- 172.21.15.131:0/901626765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d34009d50 con 0x564c60dd0570 2024-05-10T05:35:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.073+0000 7f4d46000700 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60deaa30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4d340046e0 tx=0x7f4d34004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd660835f5ef48f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.073+0000 7f4d30c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:35:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.074+0000 7f4d46a00700 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.074+0000 7f4d30c00700 1 -- 172.21.15.131:0/901626765 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34005020 con 0x564c60dd0570 2024-05-10T05:35:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.074+0000 7f4d30c00700 1 -- 172.21.15.131:0/901626765 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4d34005180 con 0x564c60dd0570 2024-05-10T05:35:18.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.075+0000 7f4d657dab80 1 -- 172.21.15.131:0/901626765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c60c48270 con 0x564c60dd0570 2024-05-10T05:35:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.077+0000 7f4d30c00700 1 -- 172.21.15.131:0/901626765 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d34005340 con 0x564c60dd0570 2024-05-10T05:35:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.077+0000 7f4d657dab80 10 client.25759 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.077+0000 7f4d657dab80 1 -- 172.21.15.131:0/901626765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564c60b1cb60 con 0x564c60dd0570 2024-05-10T05:35:18.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.079+0000 7f4d30c00700 1 -- 172.21.15.131:0/901626765 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d3402f030 con 0x564c60dd0570 2024-05-10T05:35:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.079+0000 7f4d30c00700 1 -- 172.21.15.131:0/901626765 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4d34019920 con 0x564c60dd0570 2024-05-10T05:35:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.083+0000 7f4d657dab80 10 client.25759 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.083+0000 7f4d657dab80 10 client.25759 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:35:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.083+0000 7f4d657dab80 20 client.25759 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.083+0000 7f4d657dab80 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564c60df5e40 0x564c60df8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.083+0000 7f4d657dab80 4 client.25759 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:18.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.083+0000 7f4d657dab80 1 -- 172.21.15.131:0/901626765 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c60d9e780 con 0x564c60df5e40 2024-05-10T05:35:18.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.084+0000 7f4d46a00700 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564c60df5e40 0x564c60df8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.084+0000 7f4d46a00700 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564c60df5e40 0x564c60df8290 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77bd93231f19342f server_cookie=9937db1b5f46d128 in_seq=0 out_seq=0 2024-05-10T05:35:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.084+0000 7f4d30c00700 10 client.25759 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:35:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.094+0000 7f4d30c00700 1 -- 172.21.15.131:0/901626765 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c60d9e780 con 0x564c60df5e40 2024-05-10T05:35:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.094+0000 7f4d30c00700 10 client.25759 handle_command_reply: tid=0 2024-05-10T05:35:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.095+0000 7f4d657dab80 1 client.25759 shutdown 2024-05-10T05:35:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.095+0000 7f4d657dab80 1 -- 172.21.15.131:0/901626765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 msgr2=0x564c60deaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.095+0000 7f4d657dab80 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60deaa30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4d340046e0 tx=0x7f4d34004510 comp rx=0 tx=0).stop 2024-05-10T05:35:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.096+0000 7f4d657dab80 1 -- 172.21.15.131:0/901626765 shutdown_connections 2024-05-10T05:35:18.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.096+0000 7f4d657dab80 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564c60df5e40 0x564c60df8290 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.096+0000 7f4d657dab80 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0ec0 0x564c60df1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.096+0000 7f4d657dab80 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dcf6a0 0x564c60dea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.096+0000 7f4d657dab80 1 --2- 172.21.15.131:0/901626765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60deaa30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.096+0000 7f4d657dab80 1 -- 172.21.15.131:0/901626765 >> 172.21.15.131:0/901626765 conn(0x564c60d31a60 msgr2=0x564c60dc9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.096+0000 7f4d30c00700 0 client.25759 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:35:18.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.097+0000 7f4d657dab80 1 -- 172.21.15.131:0/901626765 shutdown_connections 2024-05-10T05:35:18.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.097+0000 7f4d657dab80 1 -- 172.21.15.131:0/901626765 wait complete. 2024-05-10T05:35:18.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.100+0000 7f4d657dab80 20 client.25759 trim_cache size 0 max 16384 2024-05-10T05:35:18.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.134+0000 7f4d657dab80 1 Processor -- start 2024-05-10T05:35:18.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.134+0000 7f4d657dab80 1 -- start start 2024-05-10T05:35:18.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.135+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dcf6a0 0x564c60dcae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.135+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0570 0x564c60dcfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.136+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60dd37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.136+0000 7f4d46a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dcf6a0 0x564c60dcae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.136+0000 7f4d46a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dcf6a0 0x564c60dcae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51844/0 (socket says 172.21.15.131:51844) 2024-05-10T05:35:18.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.136+0000 7f4d657dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60dd0400 con 0x564c60dd0570 2024-05-10T05:35:18.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.136+0000 7f4d657dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60dd01a0 con 0x564c60dd0ec0 2024-05-10T05:35:18.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.136+0000 7f4d657dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60dc47f0 con 0x564c60dcf6a0 2024-05-10T05:35:18.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.136+0000 7f4d47400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60dd37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.136+0000 7f4d47400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60dd37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46524/0 (socket says 172.21.15.131:46524) 2024-05-10T05:35:18.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.136+0000 7f4d47400700 1 -- 172.21.15.131:0/3651552499 learned_addr learned my addr 172.21.15.131:0/3651552499 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.138+0000 7f4d47400700 1 -- 172.21.15.131:0/3651552499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dcf6a0 msgr2=0x564c60dcae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.138+0000 7f4d47400700 1 --2- 172.21.15.131:0/3651552499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dcf6a0 0x564c60dcae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.138+0000 7f4d47400700 1 -- 172.21.15.131:0/3651552499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0570 msgr2=0x564c60dcfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.138+0000 7f4d47400700 1 --2- 172.21.15.131:0/3651552499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0570 0x564c60dcfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.138+0000 7f4d47400700 1 -- 172.21.15.131:0/3651552499 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c60cb5560 con 0x564c60dd0ec0 2024-05-10T05:35:18.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.139+0000 7f4d47400700 1 --2- 172.21.15.131:0/3651552499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60dd37a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009ed0 tx=0x7f4d38017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca433d41dfcbdb0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.139+0000 7f4d3cc00700 1 -- 172.21.15.131:0/3651552499 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38019830 con 0x564c60dd0ec0 2024-05-10T05:35:18.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.140+0000 7f4d3cc00700 1 -- 172.21.15.131:0/3651552499 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d38019e50 con 0x564c60dd0ec0 2024-05-10T05:35:18.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.140+0000 7f4d3cc00700 1 -- 172.21.15.131:0/3651552499 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38025c60 con 0x564c60dd0ec0 2024-05-10T05:35:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.141+0000 7f4d657dab80 1 -- 172.21.15.131:0/3651552499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 msgr2=0x564c60dd37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.141+0000 7f4d657dab80 1 --2- 172.21.15.131:0/3651552499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60dd37a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009ed0 tx=0x7f4d38017000 comp rx=0 tx=0).stop 2024-05-10T05:35:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.142+0000 7f4d657dab80 1 -- 172.21.15.131:0/3651552499 shutdown_connections 2024-05-10T05:35:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.142+0000 7f4d657dab80 1 --2- 172.21.15.131:0/3651552499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60dd37a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.142+0000 7f4d657dab80 1 --2- 172.21.15.131:0/3651552499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dcf6a0 0x564c60dcae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.142+0000 7f4d657dab80 1 --2- 172.21.15.131:0/3651552499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dd0570 0x564c60dcfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.142+0000 7f4d657dab80 1 -- 172.21.15.131:0/3651552499 >> 172.21.15.131:0/3651552499 conn(0x564c60d31a60 msgr2=0x564c60dc9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.142+0000 7f4d657dab80 1 -- 172.21.15.131:0/3651552499 shutdown_connections 2024-05-10T05:35:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.142+0000 7f4d657dab80 1 -- 172.21.15.131:0/3651552499 wait complete. 2024-05-10T05:35:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.143+0000 7f4d657dab80 1 -- start start 2024-05-10T05:35:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.143+0000 7f4d657dab80 1 Processor -- start 2024-05-10T05:35:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.144+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dcf6a0 0x564c60e462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.144+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60e48830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.144+0000 7f4d657dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60e4aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.144+0000 7f4d657dab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60e4b550 con 0x564c60dcf6a0 2024-05-10T05:35:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.144+0000 7f4d657dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60e4b6c0 con 0x564c60dd0ec0 2024-05-10T05:35:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.144+0000 7f4d657dab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60e4b880 con 0x564c60dd0570 2024-05-10T05:35:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.144+0000 7f4d657dab80 10 client.0 _ll_register_callbacks cb 0x564c60c94b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.145+0000 7f4d47400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60e4aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.145+0000 7f4d47400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60e4aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46540/0 (socket says 172.21.15.131:46540) 2024-05-10T05:35:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.145+0000 7f4d47400700 1 -- 172.21.15.131:0/356963859 learned_addr learned my addr 172.21.15.131:0/356963859 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.145+0000 7f4d46a00700 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dcf6a0 0x564c60e462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.145+0000 7f4d46000700 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60e48830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.146+0000 7f4d47400700 1 -- 172.21.15.131:0/356963859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 msgr2=0x564c60e48830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.146+0000 7f4d47400700 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60e48830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.146+0000 7f4d47400700 1 -- 172.21.15.131:0/356963859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dcf6a0 msgr2=0x564c60e462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.146+0000 7f4d47400700 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dcf6a0 0x564c60e462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.146+0000 7f4d47400700 1 -- 172.21.15.131:0/356963859 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d38021040 con 0x564c60dd0ec0 2024-05-10T05:35:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.146+0000 7f4d47400700 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60e4aee0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d38019700 tx=0x7f4d38009cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a9dfbc1a988b4d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.146+0000 7f4d30c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:35:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.147+0000 7f4d30c00700 1 -- 172.21.15.131:0/356963859 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38019830 con 0x564c60dd0ec0 2024-05-10T05:35:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.147+0000 7f4d30c00700 1 -- 172.21.15.131:0/356963859 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4d38019990 con 0x564c60dd0ec0 2024-05-10T05:35:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.147+0000 7f4d657dab80 1 -- 172.21.15.131:0/356963859 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d34003680 con 0x564c60dd0ec0 2024-05-10T05:35:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.147+0000 7f4d30c00700 1 -- 172.21.15.131:0/356963859 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d38019b50 con 0x564c60dd0ec0 2024-05-10T05:35:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.147+0000 7f4d657dab80 10 client.25736 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.147+0000 7f4d657dab80 1 -- 172.21.15.131:0/356963859 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564c60b74cd0 con 0x564c60dd0ec0 2024-05-10T05:35:18.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.147+0000 7f4d30c00700 1 -- 172.21.15.131:0/356963859 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d38019cb0 con 0x564c60dd0ec0 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.149+0000 7f4d30c00700 1 -- 172.21.15.131:0/356963859 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4d38004a90 con 0x564c60dd0ec0 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.149+0000 7f4d657dab80 10 client.25736 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.149+0000 7f4d657dab80 10 client.25736 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.149+0000 7f4d657dab80 20 client.25736 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.149+0000 7f4d657dab80 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564c60e4d160 0x564c60e4f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.149+0000 7f4d657dab80 4 client.25736 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.149+0000 7f4d657dab80 1 -- 172.21.15.131:0/356963859 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f4d34003680 con 0x564c60e4d160 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.149+0000 7f4d46a00700 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564c60e4d160 0x564c60e4f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d30c00700 10 client.25736 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d46a00700 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564c60e4d160 0x564c60e4f580 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e281c60261d9fd0d server_cookie=5e80525e7b3f9259 in_seq=0 out_seq=0 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d30c00700 1 -- 172.21.15.131:0/356963859 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f4d34003680 con 0x564c60e4d160 2024-05-10T05:35:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d30c00700 10 client.25736 handle_command_reply: tid=0 2024-05-10T05:35:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d657dab80 1 client.25736 shutdown 2024-05-10T05:35:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d657dab80 1 -- 172.21.15.131:0/356963859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 msgr2=0x564c60e4aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d657dab80 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60e4aee0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d38019700 tx=0x7f4d38009cf0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.152 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:35:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d657dab80 1 -- 172.21.15.131:0/356963859 shutdown_connections 2024-05-10T05:35:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d657dab80 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564c60e4d160 0x564c60e4f580 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d657dab80 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c60dcf6a0 0x564c60e462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d657dab80 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c60dd0ec0 0x564c60e4aee0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.150+0000 7f4d657dab80 1 --2- 172.21.15.131:0/356963859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c60dd0570 0x564c60e48830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.151+0000 7f4d657dab80 1 -- 172.21.15.131:0/356963859 >> 172.21.15.131:0/356963859 conn(0x564c60d31a60 msgr2=0x564c60def9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.151+0000 7f4d30c00700 0 client.25736 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:35:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.151+0000 7f4d657dab80 1 -- 172.21.15.131:0/356963859 shutdown_connections 2024-05-10T05:35:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.151+0000 7f4d657dab80 1 -- 172.21.15.131:0/356963859 wait complete. 2024-05-10T05:35:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.151+0000 7f4d657dab80 20 client.25736 trim_cache size 0 max 16384 2024-05-10T05:35:18.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.154+0000 7f4d4d800700 1 -- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4d2c0a1400 msgr2=0x7f4d2c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.154+0000 7f4d4d800700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4d2c0a1400 0x7f4d2c0a3910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800d440 tx=0x7f4d3800f040 comp rx=0 tx=0).stop 2024-05-10T05:35:18.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.154+0000 7f4d4d800700 1 -- 172.21.15.131:0/968064933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d48100ac0 msgr2=0x7f4d48195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.154+0000 7f4d4d800700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d48100ac0 0x7f4d48195f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000b770 tx=0x7f4d40008e40 comp rx=0 tx=0).stop 2024-05-10T05:35:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.156+0000 7f4d4d800700 1 -- 172.21.15.131:0/968064933 shutdown_connections 2024-05-10T05:35:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.156+0000 7f4d4d800700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4d2c0a1400 0x7f4d2c0a3910 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.156+0000 7f4d4d800700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d48102580 0x7f4d481973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.156+0000 7f4d4d800700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d48100ac0 0x7f4d48195f10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.156+0000 7f4d4d800700 1 --2- 172.21.15.131:0/968064933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d48101be0 0x7f4d48196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.156+0000 7f4d4d800700 1 -- 172.21.15.131:0/968064933 >> 172.21.15.131:0/968064933 conn(0x7f4d480fc0d0 msgr2=0x7f4d48104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.156+0000 7f4d4d800700 1 -- 172.21.15.131:0/968064933 shutdown_connections 2024-05-10T05:35:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.156+0000 7f4d4d800700 1 -- 172.21.15.131:0/968064933 wait complete. 2024-05-10T05:35:18.206 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:35:18.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:18 smithi157 ceph-mon[78427]: pgmap v343: 97 pgs: 97 active+clean; 1.7 GiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 149 MiB/s wr, 58 op/s 2024-05-10T05:35:18.311 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:18 smithi131 ceph-mon[37264]: pgmap v343: 97 pgs: 97 active+clean; 1.7 GiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 149 MiB/s wr, 58 op/s 2024-05-10T05:35:18.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:18 smithi142 ceph-mon[77667]: pgmap v343: 97 pgs: 97 active+clean; 1.7 GiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 149 MiB/s wr, 58 op/s 2024-05-10T05:35:18.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.853+0000 7fc4d6a00700 1 -- 172.21.15.131:0/2793315491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d0075140 msgr2=0x7fc4d0073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.853+0000 7fc4d6a00700 1 --2- 172.21.15.131:0/2793315491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d0075140 0x7fc4d0073600 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009990 tx=0x7fc4c0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.853+0000 7fc4d6a00700 1 -- 172.21.15.131:0/2793315491 shutdown_connections 2024-05-10T05:35:18.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.853+0000 7fc4d6a00700 1 --2- 172.21.15.131:0/2793315491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d0075140 0x7fc4d0073600 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.853+0000 7fc4d6a00700 1 --2- 172.21.15.131:0/2793315491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d010e220 0x7fc4d0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.853+0000 7fc4d6a00700 1 --2- 172.21.15.131:0/2793315491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d0073b40 0x7fc4d0074010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.853+0000 7fc4d6a00700 1 -- 172.21.15.131:0/2793315491 >> 172.21.15.131:0/2793315491 conn(0x7fc4d00fd1d0 msgr2=0x7fc4d00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.854+0000 7fc4d6a00700 1 -- 172.21.15.131:0/2793315491 shutdown_connections 2024-05-10T05:35:18.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.854+0000 7fc4d6a00700 1 -- 172.21.15.131:0/2793315491 wait complete. 2024-05-10T05:35:18.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.855+0000 7fc4d6a00700 1 Processor -- start 2024-05-10T05:35:18.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.856+0000 7fc4d6a00700 1 -- start start 2024-05-10T05:35:18.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.856+0000 7fc4d6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d0073b40 0x7fc4d019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.856+0000 7fc4d6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0075140 0x7fc4d019ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.856+0000 7fc4d6a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d010e220 0x7fc4d01a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.856+0000 7fc4d6a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d01a5fc0 con 0x7fc4d010e220 2024-05-10T05:35:18.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.856+0000 7fc4d6a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d01a6130 con 0x7fc4d0075140 2024-05-10T05:35:18.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.856+0000 7fc4d6a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d01a62f0 con 0x7fc4d0073b40 2024-05-10T05:35:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.857+0000 7fc4cf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0075140 0x7fc4d019ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.857+0000 7fc4cf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0075140 0x7fc4d019ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46558/0 (socket says 172.21.15.131:46558) 2024-05-10T05:35:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.857+0000 7fc4cf400700 1 -- 172.21.15.131:0/553565229 learned_addr learned my addr 172.21.15.131:0/553565229 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.857+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d0073b40 0x7fc4d019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.857+0000 7fc4cfe00700 1 -- 172.21.15.131:0/553565229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0075140 msgr2=0x7fc4d019ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.857+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0075140 0x7fc4d019ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.857+0000 7fc4cfe00700 1 -- 172.21.15.131:0/553565229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d010e220 msgr2=0x7fc4d01a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.857+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d010e220 0x7fc4d01a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.857+0000 7fc4cfe00700 1 -- 172.21.15.131:0/553565229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c00095f0 con 0x7fc4d0073b40 2024-05-10T05:35:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.858+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d0073b40 0x7fc4d019e4e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00098e0 tx=0x7fc4c0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.858+0000 7fc4ccc00700 1 -- 172.21.15.131:0/553565229 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c001d070 con 0x7fc4d0073b40 2024-05-10T05:35:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.858+0000 7fc4ccc00700 1 -- 172.21.15.131:0/553565229 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4c000fbd0 con 0x7fc4d0073b40 2024-05-10T05:35:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.858+0000 7fc4d6a00700 1 -- 172.21.15.131:0/553565229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d01a6570 con 0x7fc4d0073b40 2024-05-10T05:35:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.859+0000 7fc4ccc00700 1 -- 172.21.15.131:0/553565229 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0017650 con 0x7fc4d0073b40 2024-05-10T05:35:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.859+0000 7fc4d6a00700 1 -- 172.21.15.131:0/553565229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d01a69f0 con 0x7fc4d0073b40 2024-05-10T05:35:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.867+0000 7fc4eea13b80 1 Processor -- start 2024-05-10T05:35:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.867+0000 7fc4ccc00700 1 -- 172.21.15.131:0/553565229 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc4c000fd40 con 0x7fc4d0073b40 2024-05-10T05:35:18.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.870+0000 7fc4ccc00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4b40a12c0 0x7fc4b40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.871+0000 7fc4ccc00700 1 -- 172.21.15.131:0/553565229 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc4c00cdff0 con 0x7fc4d0073b40 2024-05-10T05:35:18.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.871+0000 7fc4d4a00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4b40a12c0 0x7fc4b40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.872+0000 7fc4d4a00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4b40a12c0 0x7fc4b40a37d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80060b0 tx=0x7fc4c8006040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.872+0000 7fc4eea13b80 1 -- start start 2024-05-10T05:35:18.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 0x5609a3ea1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2570 0x5609a3ea2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2ec0 0x5609a3ea97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3dc8150 con 0x5609a3ea16a0 2024-05-10T05:35:18.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3dc5b10 con 0x5609a3ea2570 2024-05-10T05:35:18.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3dc1400 con 0x5609a3ea2ec0 2024-05-10T05:35:18.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4cf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2570 0x5609a3ea2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4cf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2570 0x5609a3ea2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46568/0 (socket says 172.21.15.131:46568) 2024-05-10T05:35:18.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4cf400700 1 -- 172.21.15.131:0/781872240 learned_addr learned my addr 172.21.15.131:0/781872240 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:18.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/781872240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 0x5609a3ea1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4d4a00700 1 --2- 172.21.15.131:0/781872240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2ec0 0x5609a3ea97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4cf400700 1 -- 172.21.15.131:0/781872240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2ec0 msgr2=0x5609a3ea97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4cf400700 1 --2- 172.21.15.131:0/781872240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2ec0 0x5609a3ea97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4cf400700 1 -- 172.21.15.131:0/781872240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 msgr2=0x5609a3ea1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4cf400700 1 --2- 172.21.15.131:0/781872240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 0x5609a3ea1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.873+0000 7fc4cf400700 1 -- 172.21.15.131:0/781872240 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a3c423b0 con 0x5609a3ea2570 2024-05-10T05:35:18.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.874+0000 7fc4cf400700 1 --2- 172.21.15.131:0/781872240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2570 0x5609a3ea2980 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009d60 tx=0x7fc4bc0116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33920992a31fc2f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.874+0000 7fc4c5200700 1 -- 172.21.15.131:0/781872240 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc011ed0 con 0x5609a3ea2570 2024-05-10T05:35:18.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.876+0000 7fc4c5200700 1 -- 172.21.15.131:0/781872240 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4bc012910 con 0x5609a3ea2570 2024-05-10T05:35:18.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.876+0000 7fc4c5200700 1 -- 172.21.15.131:0/781872240 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00f860 con 0x5609a3ea2570 2024-05-10T05:35:18.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.877+0000 7fc4eea13b80 1 -- 172.21.15.131:0/781872240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2570 msgr2=0x5609a3ea2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.877+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/781872240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2570 0x5609a3ea2980 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009d60 tx=0x7fc4bc0116c0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.877+0000 7fc4eea13b80 1 -- 172.21.15.131:0/781872240 shutdown_connections 2024-05-10T05:35:18.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.877+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/781872240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 0x5609a3ea1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.877+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/781872240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2ec0 0x5609a3ea97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.877+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/781872240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2570 0x5609a3ea2980 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.877+0000 7fc4eea13b80 1 -- 172.21.15.131:0/781872240 >> 172.21.15.131:0/781872240 conn(0x5609a3e03a60 msgr2=0x5609a3e9bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.880+0000 7fc4eea13b80 1 -- 172.21.15.131:0/781872240 shutdown_connections 2024-05-10T05:35:18.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.880+0000 7fc4eea13b80 1 -- 172.21.15.131:0/781872240 wait complete. 2024-05-10T05:35:18.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.884+0000 7fc4eea13b80 1 -- start start 2024-05-10T05:35:18.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.885+0000 7fc4eea13b80 1 Processor -- start 2024-05-10T05:35:18.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.890+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea16a0 0x5609a3ebc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.890+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 0x5609a3ebcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.890+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 0x5609a3ec3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.890+0000 7fc4cfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea16a0 0x5609a3ebc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.890+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3d5d560 con 0x5609a3ea2570 2024-05-10T05:35:18.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.890+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3d5d200 con 0x5609a3ea2ec0 2024-05-10T05:35:18.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.890+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3c446a0 con 0x5609a3ea16a0 2024-05-10T05:35:18.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.890+0000 7fc4cfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea16a0 0x5609a3ebc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51904/0 (socket says 172.21.15.131:51904) 2024-05-10T05:35:18.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.890+0000 7fc4cfe00700 1 -- 172.21.15.131:0/2142508268 learned_addr learned my addr 172.21.15.131:0/2142508268 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:18.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.890+0000 7fc4eea13b80 10 client.0 _ll_register_callbacks cb 0x5609a3d66b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:18.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.891+0000 7fc4d4a00700 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 0x5609a3ec3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.891+0000 7fc4cf400700 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 0x5609a3ebcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.891+0000 7fc4d4a00700 1 -- 172.21.15.131:0/2142508268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea16a0 msgr2=0x5609a3ebc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.891+0000 7fc4d4a00700 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea16a0 0x5609a3ebc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.891+0000 7fc4d4a00700 1 -- 172.21.15.131:0/2142508268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 msgr2=0x5609a3ebcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.891+0000 7fc4d4a00700 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 0x5609a3ebcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.891+0000 7fc4d4a00700 1 -- 172.21.15.131:0/2142508268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc009990 con 0x5609a3ea2ec0 2024-05-10T05:35:18.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.892+0000 7fc4d4a00700 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 0x5609a3ec3b70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009fd0 tx=0x7fc4c8009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6f57a26807da44 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.892+0000 7fc4b9600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:35:18.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.892+0000 7fc4b9600700 1 -- 172.21.15.131:0/2142508268 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c80147b0 con 0x5609a3ea2ec0 2024-05-10T05:35:18.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.892+0000 7fc4b9600700 1 -- 172.21.15.131:0/2142508268 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc4c8014910 con 0x5609a3ea2ec0 2024-05-10T05:35:18.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.892+0000 7fc4eea13b80 1 -- 172.21.15.131:0/2142508268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609a3d1a270 con 0x5609a3ea2ec0 2024-05-10T05:35:18.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.892+0000 7fc4b9600700 1 -- 172.21.15.131:0/2142508268 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c8014ad0 con 0x5609a3ea2ec0 2024-05-10T05:35:18.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.893+0000 7fc4eea13b80 10 client.25748 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:18.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.893+0000 7fc4eea13b80 1 -- 172.21.15.131:0/2142508268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5609a3beeb60 con 0x5609a3ea2ec0 2024-05-10T05:35:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.894+0000 7fc4b9600700 1 -- 172.21.15.131:0/2142508268 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609a3d1a270 con 0x5609a3ea2ec0 2024-05-10T05:35:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.894+0000 7fc4b9600700 1 -- 172.21.15.131:0/2142508268 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc4c8029070 con 0x5609a3ea2ec0 2024-05-10T05:35:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.894+0000 7fc4eea13b80 10 client.25748 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.894+0000 7fc4eea13b80 10 client.25748 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:35:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.894+0000 7fc4eea13b80 20 client.25748 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.894+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5609a3ec82c0 0x5609a3eca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.894+0000 7fc4eea13b80 4 client.25748 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:18.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.894+0000 7fc4eea13b80 1 -- 172.21.15.131:0/2142508268 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609a3e70780 con 0x5609a3ec82c0 2024-05-10T05:35:18.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.895+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5609a3ec82c0 0x5609a3eca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.895+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5609a3ec82c0 0x5609a3eca710 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5c4569183104232 server_cookie=c0b7a07d2c37b14d in_seq=0 out_seq=0 2024-05-10T05:35:18.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.895+0000 7fc4b9600700 10 client.25748 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:35:18.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.898+0000 7fc4b9600700 1 -- 172.21.15.131:0/2142508268 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609a3e70780 con 0x5609a3ec82c0 2024-05-10T05:35:18.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.898+0000 7fc4b9600700 10 client.25748 handle_command_reply: tid=0 2024-05-10T05:35:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.898+0000 7fc4eea13b80 1 client.25748 shutdown 2024-05-10T05:35:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.898+0000 7fc4eea13b80 1 -- 172.21.15.131:0/2142508268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 msgr2=0x5609a3ec3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.898+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 0x5609a3ec3b70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009fd0 tx=0x7fc4c8009620 comp rx=0 tx=0).stop 2024-05-10T05:35:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.899+0000 7fc4eea13b80 1 -- 172.21.15.131:0/2142508268 shutdown_connections 2024-05-10T05:35:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.899+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5609a3ec82c0 0x5609a3eca710 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.899+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 0x5609a3ec3b70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.899+0000 7fc4b9600700 0 client.25748 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:35:18.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.899+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea16a0 0x5609a3ebc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.899+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/2142508268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 0x5609a3ebcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.899+0000 7fc4eea13b80 1 -- 172.21.15.131:0/2142508268 >> 172.21.15.131:0/2142508268 conn(0x5609a3e03a60 msgr2=0x5609a3e9bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.899+0000 7fc4eea13b80 1 -- 172.21.15.131:0/2142508268 shutdown_connections 2024-05-10T05:35:18.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.899+0000 7fc4eea13b80 1 -- 172.21.15.131:0/2142508268 wait complete. 2024-05-10T05:35:18.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.899+0000 7fc4eea13b80 20 client.25748 trim_cache size 0 max 16384 2024-05-10T05:35:18.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.927+0000 7fc4eea13b80 1 Processor -- start 2024-05-10T05:35:18.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.928+0000 7fc4eea13b80 1 -- start start 2024-05-10T05:35:18.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.929+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea16a0 0x5609a3e9cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.929+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 0x5609a3ec3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.929+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2ec0 0x5609a3ebbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.929+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3ec4cb0 con 0x5609a3ea2570 2024-05-10T05:35:18.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.929+0000 7fc4cfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea16a0 0x5609a3e9cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.929+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3e9d770 con 0x5609a3ea16a0 2024-05-10T05:35:18.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.929+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3ecad90 con 0x5609a3ea2ec0 2024-05-10T05:35:18.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.929+0000 7fc4cfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea16a0 0x5609a3e9cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46596/0 (socket says 172.21.15.131:46596) 2024-05-10T05:35:18.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.929+0000 7fc4cfe00700 1 -- 172.21.15.131:0/3514253820 learned_addr learned my addr 172.21.15.131:0/3514253820 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:18.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4cf400700 1 --2- 172.21.15.131:0/3514253820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 0x5609a3ec3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4cfe00700 1 -- 172.21.15.131:0/3514253820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2ec0 msgr2=0x5609a3ebbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/3514253820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2ec0 0x5609a3ebbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4cfe00700 1 -- 172.21.15.131:0/3514253820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 msgr2=0x5609a3ec3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/3514253820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 0x5609a3ec3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4cfe00700 1 -- 172.21.15.131:0/3514253820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a3d87560 con 0x5609a3ea16a0 2024-05-10T05:35:18.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4cf400700 1 --2- 172.21.15.131:0/3514253820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 0x5609a3ec3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:18.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/3514253820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea16a0 0x5609a3e9cad0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0097ee0 tx=0x7fc4c0086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6ca29b8e9e9cf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4c5c00700 1 -- 172.21.15.131:0/3514253820 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b7070 con 0x5609a3ea16a0 2024-05-10T05:35:18.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4c5c00700 1 -- 172.21.15.131:0/3514253820 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4c0021e90 con 0x5609a3ea16a0 2024-05-10T05:35:18.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.930+0000 7fc4c5c00700 1 -- 172.21.15.131:0/3514253820 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0099810 con 0x5609a3ea16a0 2024-05-10T05:35:18.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.932+0000 7fc4eea13b80 1 -- 172.21.15.131:0/3514253820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea16a0 msgr2=0x5609a3e9cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.932+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/3514253820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea16a0 0x5609a3e9cad0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0097ee0 tx=0x7fc4c0086040 comp rx=0 tx=0).stop 2024-05-10T05:35:18.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.932+0000 7fc4eea13b80 1 -- 172.21.15.131:0/3514253820 shutdown_connections 2024-05-10T05:35:18.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.932+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/3514253820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2ec0 0x5609a3ebbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.932+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/3514253820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea16a0 0x5609a3e9cad0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.932+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/3514253820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea2570 0x5609a3ec3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.932+0000 7fc4eea13b80 1 -- 172.21.15.131:0/3514253820 >> 172.21.15.131:0/3514253820 conn(0x5609a3e03a60 msgr2=0x5609a3e9be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.932+0000 7fc4eea13b80 1 -- 172.21.15.131:0/3514253820 shutdown_connections 2024-05-10T05:35:18.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.932+0000 7fc4eea13b80 1 -- 172.21.15.131:0/3514253820 wait complete. 2024-05-10T05:35:18.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.933+0000 7fc4eea13b80 1 -- start start 2024-05-10T05:35:18.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.934+0000 7fc4eea13b80 1 Processor -- start 2024-05-10T05:35:18.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.935+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 0x5609a3ec0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.935+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2570 0x5609a3ec1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.935+0000 7fc4eea13b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 0x5609a3ec23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.935+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3ec28f0 con 0x5609a3ea16a0 2024-05-10T05:35:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.935+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3ec2a60 con 0x5609a3ea2ec0 2024-05-10T05:35:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.935+0000 7fc4eea13b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a3ec2c20 con 0x5609a3ea2570 2024-05-10T05:35:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.935+0000 7fc4eea13b80 10 client.0 _ll_register_callbacks cb 0x5609a3d66b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.936+0000 7fc4cfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 0x5609a3ec0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.936+0000 7fc4cfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 0x5609a3ec0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38366/0 (socket says 172.21.15.131:38366) 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.936+0000 7fc4cfe00700 1 -- 172.21.15.131:0/1341249056 learned_addr learned my addr 172.21.15.131:0/1341249056 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.937+0000 7fc4cfe00700 1 -- 172.21.15.131:0/1341249056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2570 msgr2=0x5609a3ec1480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.937+0000 7fc4cf400700 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2570 0x5609a3ec1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.937+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2570 0x5609a3ec1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.937+0000 7fc4cfe00700 1 -- 172.21.15.131:0/1341249056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 msgr2=0x5609a3ec23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.937+0000 7fc4d4a00700 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 0x5609a3ec23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.937+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 0x5609a3ec23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.937+0000 7fc4cfe00700 1 -- 172.21.15.131:0/1341249056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d01a69f0 con 0x5609a3ea16a0 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.937+0000 7fc4cf400700 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2570 0x5609a3ec1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.938+0000 7fc4cfe00700 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 0x5609a3ec0f40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0086010 tx=0x7fc4c0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784f72266e627648 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.938+0000 7fc4b9600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.938+0000 7fc4b9600700 1 -- 172.21.15.131:0/1341249056 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b7070 con 0x5609a3ea16a0 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.938+0000 7fc4eea13b80 1 -- 172.21.15.131:0/1341249056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4bc012910 con 0x5609a3ea16a0 2024-05-10T05:35:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.938+0000 7fc4b9600700 1 -- 172.21.15.131:0/1341249056 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc4c009b470 con 0x5609a3ea16a0 2024-05-10T05:35:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.939+0000 7fc4b9600700 1 -- 172.21.15.131:0/1341249056 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c00cbba0 con 0x5609a3ea16a0 2024-05-10T05:35:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.939+0000 7fc4eea13b80 10 client.15738 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.940+0000 7fc4eea13b80 1 -- 172.21.15.131:0/1341249056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5609a3c46cd0 con 0x5609a3ea16a0 2024-05-10T05:35:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.940+0000 7fc4b9600700 1 -- 172.21.15.131:0/1341249056 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4bc012910 con 0x5609a3ea16a0 2024-05-10T05:35:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.940+0000 7fc4b9600700 1 -- 172.21.15.131:0/1341249056 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc4c0082070 con 0x5609a3ea16a0 2024-05-10T05:35:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.943+0000 7fc4eea13b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:18.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.943+0000 7fc4eea13b80 10 client.15738 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:35:18.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.943+0000 7fc4eea13b80 20 client.15738 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:18.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.943+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5609a3ec82c0 0x5609a3f207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:18.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.943+0000 7fc4eea13b80 4 client.15738 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:35:18.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.943+0000 7fc4eea13b80 1 -- 172.21.15.131:0/1341249056 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609a3db9ec0 con 0x5609a3ec82c0 2024-05-10T05:35:18.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.943+0000 7fc4cf400700 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5609a3ec82c0 0x5609a3f207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:18.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.944+0000 7fc4cf400700 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5609a3ec82c0 0x5609a3f207e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b15364236e5bdfd8 server_cookie=ca0aa952dbbdcaba in_seq=0 out_seq=0 2024-05-10T05:35:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4b9600700 10 client.15738 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:35:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4b9600700 1 -- 172.21.15.131:0/1341249056 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609a3db9ec0 con 0x5609a3ec82c0 2024-05-10T05:35:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4b9600700 10 client.15738 handle_command_reply: tid=0 2024-05-10T05:35:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4eea13b80 1 client.15738 shutdown 2024-05-10T05:35:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4eea13b80 1 -- 172.21.15.131:0/1341249056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 msgr2=0x5609a3ec0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 0x5609a3ec0f40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0086010 tx=0x7fc4c0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4eea13b80 1 -- 172.21.15.131:0/1341249056 shutdown_connections 2024-05-10T05:35:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5609a3ec82c0 0x5609a3f207e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609a3ea16a0 0x5609a3ec0f40 secure :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0086010 tx=0x7fc4c0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a3ea2ec0 0x5609a3ec23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4b9600700 0 client.15738 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:35:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4eea13b80 1 --2- 172.21.15.131:0/1341249056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609a3ea2570 0x5609a3ec1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.947+0000 7fc4eea13b80 1 -- 172.21.15.131:0/1341249056 >> 172.21.15.131:0/1341249056 conn(0x5609a3e03a60 msgr2=0x5609a3ea6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.948+0000 7fc4eea13b80 1 -- 172.21.15.131:0/1341249056 shutdown_connections 2024-05-10T05:35:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.948+0000 7fc4eea13b80 1 -- 172.21.15.131:0/1341249056 wait complete. 2024-05-10T05:35:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.949+0000 7fc4eea13b80 20 client.15738 trim_cache size 0 max 16384 2024-05-10T05:35:18.950 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:35:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.950+0000 7fc4d6a00700 1 -- 172.21.15.131:0/553565229 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4b40a12c0 msgr2=0x7fc4b40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.950+0000 7fc4d6a00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4b40a12c0 0x7fc4b40a37d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80060b0 tx=0x7fc4c8006040 comp rx=0 tx=0).stop 2024-05-10T05:35:18.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.951+0000 7fc4d6a00700 1 -- 172.21.15.131:0/553565229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d0073b40 msgr2=0x7fc4d019e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:18.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.951+0000 7fc4d6a00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d0073b40 0x7fc4d019e4e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00098e0 tx=0x7fc4c0009580 comp rx=0 tx=0).stop 2024-05-10T05:35:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.951+0000 7fc4d6a00700 1 -- 172.21.15.131:0/553565229 shutdown_connections 2024-05-10T05:35:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.951+0000 7fc4d6a00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4b40a12c0 0x7fc4b40a37d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.951+0000 7fc4d6a00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d010e220 0x7fc4d01a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.951+0000 7fc4d6a00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d0073b40 0x7fc4d019e4e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.951+0000 7fc4d6a00700 1 --2- 172.21.15.131:0/553565229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0075140 0x7fc4d019ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.952+0000 7fc4d6a00700 1 -- 172.21.15.131:0/553565229 >> 172.21.15.131:0/553565229 conn(0x7fc4d00fd1d0 msgr2=0x7fc4d01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:18.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.952+0000 7fc4d6a00700 1 -- 172.21.15.131:0/553565229 shutdown_connections 2024-05-10T05:35:18.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:18.952+0000 7fc4d6a00700 1 -- 172.21.15.131:0/553565229 wait complete. 2024-05-10T05:35:19.018 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:35:19.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.795+0000 7efe0aa00700 1 -- 172.21.15.131:0/3698139260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe04109730 msgr2=0x7efe04108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:19.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.795+0000 7efe02a00700 1 -- 172.21.15.131:0/3698139260 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400b810 con 0x7efe04109730 2024-05-10T05:35:19.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.795+0000 7efe0aa00700 1 --2- 172.21.15.131:0/3698139260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe04109730 0x7efe04108210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efdf40098f0 tx=0x7efdf4009c00 comp rx=0 tx=0).stop 2024-05-10T05:35:19.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.796+0000 7efe0aa00700 1 -- 172.21.15.131:0/3698139260 shutdown_connections 2024-05-10T05:35:19.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.796+0000 7efe0aa00700 1 --2- 172.21.15.131:0/3698139260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe04109730 0x7efe04108210 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.796+0000 7efe0aa00700 1 --2- 172.21.15.131:0/3698139260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe0410fef0 0x7efe041123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.796+0000 7efe0aa00700 1 --2- 172.21.15.131:0/3698139260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe041087e0 0x7efe04108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.796+0000 7efe0aa00700 1 -- 172.21.15.131:0/3698139260 >> 172.21.15.131:0/3698139260 conn(0x7efe0406e830 msgr2=0x7efe04070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:19.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.797+0000 7efe0aa00700 1 -- 172.21.15.131:0/3698139260 shutdown_connections 2024-05-10T05:35:19.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.797+0000 7efe0aa00700 1 -- 172.21.15.131:0/3698139260 wait complete. 2024-05-10T05:35:19.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.800+0000 7efe0aa00700 1 Processor -- start 2024-05-10T05:35:19.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.801+0000 7efe0aa00700 1 -- start start 2024-05-10T05:35:19.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.801+0000 7efe0aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe041087e0 0x7efe041a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.802+0000 7efe0aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe04109730 0x7efe041a7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.802+0000 7efe03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe041087e0 0x7efe041a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.802+0000 7efe0aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0410fef0 0x7efe041ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.802+0000 7efe03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe041087e0 0x7efe041a6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51954/0 (socket says 172.21.15.131:51954) 2024-05-10T05:35:19.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.802+0000 7efe03e00700 1 -- 172.21.15.131:0/3308778010 learned_addr learned my addr 172.21.15.131:0/3308778010 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:19.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.802+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041ae5d0 con 0x7efe0410fef0 2024-05-10T05:35:19.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.802+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041ae710 con 0x7efe04109730 2024-05-10T05:35:19.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.802+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041ae8a0 con 0x7efe041087e0 2024-05-10T05:35:19.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.803+0000 7efe03e00700 1 -- 172.21.15.131:0/3308778010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe04109730 msgr2=0x7efe041a7070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:35:19.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.803+0000 7efe03e00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe04109730 0x7efe041a7070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.803+0000 7efe03e00700 1 -- 172.21.15.131:0/3308778010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0410fef0 msgr2=0x7efe041ae000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:35:19.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.803+0000 7efe03e00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0410fef0 0x7efe041ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.803+0000 7efe03e00700 1 -- 172.21.15.131:0/3308778010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009520 con 0x7efe041087e0 2024-05-10T05:35:19.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.804+0000 7efe03e00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe041087e0 0x7efe041a6b30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c010 tx=0x7efdf400b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:19.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.806+0000 7efe00c00700 1 -- 172.21.15.131:0/3308778010 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4005450 con 0x7efe041087e0 2024-05-10T05:35:19.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.806+0000 7efe00c00700 1 -- 172.21.15.131:0/3308778010 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdf40055b0 con 0x7efe041087e0 2024-05-10T05:35:19.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.806+0000 7efe00c00700 1 -- 172.21.15.131:0/3308778010 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4017860 con 0x7efe041087e0 2024-05-10T05:35:19.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.810+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe041aeb20 con 0x7efe041087e0 2024-05-10T05:35:19.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.810+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe041aef70 con 0x7efe041087e0 2024-05-10T05:35:19.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.819+0000 7efe00c00700 1 -- 172.21.15.131:0/3308778010 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7efdf40179c0 con 0x7efe041087e0 2024-05-10T05:35:19.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.821+0000 7efe228d3b80 1 Processor -- start 2024-05-10T05:35:19.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.822+0000 7efe00c00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efde80a12a0 0x7efde80a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.822+0000 7efe00c00700 1 -- 172.21.15.131:0/3308778010 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7efdf40ce2a0 con 0x7efe041087e0 2024-05-10T05:35:19.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.824+0000 7efe08a00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efde80a12a0 0x7efde80a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.824+0000 7efe228d3b80 1 -- start start 2024-05-10T05:35:19.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.826+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef9545530 0x563ef9545960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.826+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95467a0 0x563ef9546bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.827+0000 7efe03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef9545530 0x563ef9545960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.828+0000 7efe03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef9545530 0x563ef9545960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51960/0 (socket says 172.21.15.131:51960) 2024-05-10T05:35:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.828+0000 7efe03e00700 1 -- 172.21.15.131:0/215557822 learned_addr learned my addr 172.21.15.131:0/215557822 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.828+0000 7efe08a00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efde80a12a0 0x7efde80a37b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00ab10 tx=0x7efdfc009250 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.829+0000 7efe228d3b80 1 --2- 172.21.15.131:0/215557822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef95470f0 0x563ef954d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.829+0000 7efe228d3b80 1 -- 172.21.15.131:0/215557822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef946c150 con 0x563ef95467a0 2024-05-10T05:35:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.829+0000 7efe228d3b80 1 -- 172.21.15.131:0/215557822 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9469b10 con 0x563ef95470f0 2024-05-10T05:35:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.829+0000 7efe228d3b80 1 -- 172.21.15.131:0/215557822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9465400 con 0x563ef9545530 2024-05-10T05:35:19.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.830+0000 7efe08a00700 1 --2- 172.21.15.131:0/215557822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef95470f0 0x563ef954d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.831+0000 7efe03e00700 1 -- 172.21.15.131:0/215557822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef95470f0 msgr2=0x563ef954d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:19.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.831+0000 7efe03e00700 1 --2- 172.21.15.131:0/215557822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef95470f0 0x563ef954d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.831+0000 7efe03e00700 1 -- 172.21.15.131:0/215557822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95467a0 msgr2=0x563ef9546bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:19.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.831+0000 7efe03e00700 1 --2- 172.21.15.131:0/215557822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95467a0 0x563ef9546bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.831+0000 7efe03e00700 1 -- 172.21.15.131:0/215557822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef92e63b0 con 0x563ef9545530 2024-05-10T05:35:19.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.833+0000 7efe03e00700 1 --2- 172.21.15.131:0/215557822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef9545530 0x563ef9545960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efdf4098b40 tx=0x7efdf400bd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b850e66eb00c48 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:19.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.835+0000 7efdf9200700 1 -- 172.21.15.131:0/215557822 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf40b41c0 con 0x563ef9545530 2024-05-10T05:35:19.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.836+0000 7efdf9200700 1 -- 172.21.15.131:0/215557822 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdf409c640 con 0x563ef9545530 2024-05-10T05:35:19.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.837+0000 7efdf9200700 1 -- 172.21.15.131:0/215557822 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4098d50 con 0x563ef9545530 2024-05-10T05:35:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.845+0000 7efe228d3b80 1 -- 172.21.15.131:0/215557822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef9545530 msgr2=0x563ef9545960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.845+0000 7efe228d3b80 1 --2- 172.21.15.131:0/215557822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef9545530 0x563ef9545960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efdf4098b40 tx=0x7efdf400bd40 comp rx=0 tx=0).stop 2024-05-10T05:35:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.845+0000 7efe228d3b80 1 -- 172.21.15.131:0/215557822 shutdown_connections 2024-05-10T05:35:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.846+0000 7efe228d3b80 1 --2- 172.21.15.131:0/215557822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef95470f0 0x563ef954d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.846+0000 7efe228d3b80 1 --2- 172.21.15.131:0/215557822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef9545530 0x563ef9545960 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efdf4098b40 tx=0x7efdf400bd40 comp rx=0 tx=0).stop 2024-05-10T05:35:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.846+0000 7efe228d3b80 1 --2- 172.21.15.131:0/215557822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95467a0 0x563ef9546bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.846+0000 7efe228d3b80 1 -- 172.21.15.131:0/215557822 >> 172.21.15.131:0/215557822 conn(0x563ef94a7a60 msgr2=0x563ef9540330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.850+0000 7efe228d3b80 1 -- 172.21.15.131:0/215557822 shutdown_connections 2024-05-10T05:35:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.850+0000 7efe228d3b80 1 -- 172.21.15.131:0/215557822 wait complete. 2024-05-10T05:35:19.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.851+0000 7efe228d3b80 1 -- start start 2024-05-10T05:35:19.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.851+0000 7efe228d3b80 1 Processor -- start 2024-05-10T05:35:19.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.854+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 0x563ef954f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.854+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef95470f0 0x563ef954fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.854+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef9554ba0 0x563ef9556fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.854+0000 7efe228d3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9401560 con 0x563ef9554ba0 2024-05-10T05:35:19.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.854+0000 7efe228d3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9401200 con 0x563ef95470f0 2024-05-10T05:35:19.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.854+0000 7efe228d3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef92e86a0 con 0x563ef95467a0 2024-05-10T05:35:19.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.854+0000 7efe228d3b80 10 client.0 _ll_register_callbacks cb 0x563ef940ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:19.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.855+0000 7efe03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 0x563ef954f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.855+0000 7efe03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 0x563ef954f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51970/0 (socket says 172.21.15.131:51970) 2024-05-10T05:35:19.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.855+0000 7efe03e00700 1 -- 172.21.15.131:0/1692999447 learned_addr learned my addr 172.21.15.131:0/1692999447 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:19.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.855+0000 7efe08a00700 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef9554ba0 0x563ef9556fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.856+0000 7efe08a00700 1 -- 172.21.15.131:0/1692999447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 msgr2=0x563ef954f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:19.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.856+0000 7efe08a00700 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 0x563ef954f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.856+0000 7efe08a00700 1 -- 172.21.15.131:0/1692999447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef95470f0 msgr2=0x563ef954fd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:19.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.856+0000 7efe08a00700 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef95470f0 0x563ef954fd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.856+0000 7efe08a00700 1 -- 172.21.15.131:0/1692999447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe041aef70 con 0x563ef9554ba0 2024-05-10T05:35:19.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.857+0000 7efe08a00700 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef9554ba0 0x563ef9556fc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00c690 tx=0x7efdfc004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e34db08c75f596d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:19.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.857+0000 7efded600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:35:19.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.858+0000 7efded600700 1 -- 172.21.15.131:0/1692999447 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc01aa50 con 0x563ef9554ba0 2024-05-10T05:35:19.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.858+0000 7efded600700 1 -- 172.21.15.131:0/1692999447 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7efdfc013070 con 0x563ef9554ba0 2024-05-10T05:35:19.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.858+0000 7efe228d3b80 1 -- 172.21.15.131:0/1692999447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ef93be270 con 0x563ef9554ba0 2024-05-10T05:35:19.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.859+0000 7efded600700 1 -- 172.21.15.131:0/1692999447 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdfc01aa50 con 0x563ef9554ba0 2024-05-10T05:35:19.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.859+0000 7efe228d3b80 10 client.15744 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:19.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.859+0000 7efe228d3b80 1 -- 172.21.15.131:0/1692999447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563ef9292b60 con 0x563ef9554ba0 2024-05-10T05:35:19.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.859+0000 7efded600700 1 -- 172.21.15.131:0/1692999447 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdfc01f070 con 0x563ef9554ba0 2024-05-10T05:35:19.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.861+0000 7efded600700 1 -- 172.21.15.131:0/1692999447 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efdfc02e470 con 0x563ef9554ba0 2024-05-10T05:35:19.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.861+0000 7efe228d3b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:19.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.861+0000 7efe228d3b80 10 client.15744 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:35:19.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.861+0000 7efe228d3b80 20 client.15744 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:19.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.862+0000 7efe228d3b80 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563ef955b820 0x563ef955dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.862+0000 7efe228d3b80 4 client.15744 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:19.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.862+0000 7efe228d3b80 1 -- 172.21.15.131:0/1692999447 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ef93be270 con 0x563ef955b820 2024-05-10T05:35:19.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.862+0000 7efe03e00700 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563ef955b820 0x563ef955dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.863+0000 7efe03e00700 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563ef955b820 0x563ef955dcd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6151e4790999f0f server_cookie=6e2b2156ea88e166 in_seq=0 out_seq=0 2024-05-10T05:35:19.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.863+0000 7efded600700 10 client.15744 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:35:19.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.864+0000 7efded600700 1 -- 172.21.15.131:0/1692999447 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ef93be270 con 0x563ef955b820 2024-05-10T05:35:19.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.864+0000 7efded600700 10 client.15744 handle_command_reply: tid=0 2024-05-10T05:35:19.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.864+0000 7efe228d3b80 1 client.15744 shutdown 2024-05-10T05:35:19.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.865+0000 7efe228d3b80 1 -- 172.21.15.131:0/1692999447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef9554ba0 msgr2=0x563ef9556fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:19.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.865+0000 7efe228d3b80 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef9554ba0 0x563ef9556fc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00c690 tx=0x7efdfc004c50 comp rx=0 tx=0).stop 2024-05-10T05:35:19.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.866+0000 7efe228d3b80 1 -- 172.21.15.131:0/1692999447 shutdown_connections 2024-05-10T05:35:19.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.866+0000 7efe228d3b80 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563ef955b820 0x563ef955dcd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.866+0000 7efe228d3b80 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef9554ba0 0x563ef9556fc0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.866+0000 7efe228d3b80 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 0x563ef954f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.866+0000 7efe228d3b80 1 --2- 172.21.15.131:0/1692999447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef95470f0 0x563ef954fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.866+0000 7efe228d3b80 1 -- 172.21.15.131:0/1692999447 >> 172.21.15.131:0/1692999447 conn(0x563ef94a7a60 msgr2=0x563ef9544060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:19.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.866+0000 7efded600700 0 client.15744 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:35:19.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.867+0000 7efe228d3b80 1 -- 172.21.15.131:0/1692999447 shutdown_connections 2024-05-10T05:35:19.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.867+0000 7efe228d3b80 1 -- 172.21.15.131:0/1692999447 wait complete. 2024-05-10T05:35:19.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.867+0000 7efe228d3b80 20 client.15744 trim_cache size 0 max 16384 2024-05-10T05:35:19.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.897+0000 7efe228d3b80 1 Processor -- start 2024-05-10T05:35:19.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.897+0000 7efe228d3b80 1 -- start start 2024-05-10T05:35:19.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.898+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 0x563ef953fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.898+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95470f0 0x563ef9557670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.900+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 0x563ef9545940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.900+0000 7efe03400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95470f0 0x563ef9557670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.900+0000 7efe03400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95470f0 0x563ef9557670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38396/0 (socket says 172.21.15.131:38396) 2024-05-10T05:35:19.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.901+0000 7efe03e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 0x563ef953fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.901+0000 7efe03400700 1 -- 172.21.15.131:0/199321162 learned_addr learned my addr 172.21.15.131:0/199321162 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:19.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.901+0000 7efe228d3b80 1 -- 172.21.15.131:0/199321162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef93be270 con 0x563ef95470f0 2024-05-10T05:35:19.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.901+0000 7efe228d3b80 1 -- 172.21.15.131:0/199321162 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef953a2f0 con 0x563ef9554ba0 2024-05-10T05:35:19.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.901+0000 7efe228d3b80 1 -- 172.21.15.131:0/199321162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef953a8d0 con 0x563ef95467a0 2024-05-10T05:35:19.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.902+0000 7efe08a00700 1 --2- 172.21.15.131:0/199321162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 0x563ef9545940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.902+0000 7efe03400700 1 -- 172.21.15.131:0/199321162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 msgr2=0x563ef953fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:19.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.902+0000 7efe03400700 1 --2- 172.21.15.131:0/199321162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 0x563ef953fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.902+0000 7efe03400700 1 -- 172.21.15.131:0/199321162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 msgr2=0x563ef9545940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:19.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.902+0000 7efe03400700 1 --2- 172.21.15.131:0/199321162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 0x563ef9545940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.902+0000 7efe03400700 1 -- 172.21.15.131:0/199321162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef942b560 con 0x563ef95470f0 2024-05-10T05:35:19.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.902+0000 7efe03e00700 1 --2- 172.21.15.131:0/199321162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 0x563ef953fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:19.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.902+0000 7efe08a00700 1 --2- 172.21.15.131:0/199321162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 0x563ef9545940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:19.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.903+0000 7efe03400700 1 --2- 172.21.15.131:0/199321162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95470f0 0x563ef9557670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b940 tx=0x7efdf000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36069b60ff6c47c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:19.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.904+0000 7efdf9c00700 1 -- 172.21.15.131:0/199321162 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000d950 con 0x563ef95470f0 2024-05-10T05:35:19.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.904+0000 7efdf9c00700 1 -- 172.21.15.131:0/199321162 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdf000be90 con 0x563ef95470f0 2024-05-10T05:35:19.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.905+0000 7efe228d3b80 1 -- 172.21.15.131:0/199321162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95470f0 msgr2=0x563ef9557670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:19.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.905+0000 7efe228d3b80 1 --2- 172.21.15.131:0/199321162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95470f0 0x563ef9557670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b940 tx=0x7efdf000bc50 comp rx=0 tx=0).stop 2024-05-10T05:35:19.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.904+0000 7efdf9c00700 1 -- 172.21.15.131:0/199321162 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000d950 con 0x563ef95470f0 2024-05-10T05:35:19.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.906+0000 7efe228d3b80 1 -- 172.21.15.131:0/199321162 shutdown_connections 2024-05-10T05:35:19.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.906+0000 7efe228d3b80 1 --2- 172.21.15.131:0/199321162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 0x563ef9545940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.906+0000 7efe228d3b80 1 --2- 172.21.15.131:0/199321162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95467a0 0x563ef953fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.906+0000 7efe228d3b80 1 --2- 172.21.15.131:0/199321162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95470f0 0x563ef9557670 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.906+0000 7efe228d3b80 1 -- 172.21.15.131:0/199321162 >> 172.21.15.131:0/199321162 conn(0x563ef94a7a60 msgr2=0x563ef9540190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.906+0000 7efe228d3b80 1 -- 172.21.15.131:0/199321162 shutdown_connections 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.906+0000 7efe228d3b80 1 -- 172.21.15.131:0/199321162 wait complete. 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.907+0000 7efe228d3b80 1 -- start start 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.907+0000 7efe228d3b80 1 Processor -- start 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.908+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95467a0 0x563ef956b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.909+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95470f0 0x563ef956b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.909+0000 7efe228d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 0x563ef95b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.910+0000 7efe03400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95470f0 0x563ef956b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.910+0000 7efe03400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95470f0 0x563ef956b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:51998/0 (socket says 172.21.15.131:51998) 2024-05-10T05:35:19.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.910+0000 7efe03400700 1 -- 172.21.15.131:0/2266178946 learned_addr learned my addr 172.21.15.131:0/2266178946 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:19.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.910+0000 7efe08a00700 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 0x563ef95b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.910+0000 7efe03e00700 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95467a0 0x563ef956b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.910+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef956bd00 con 0x563ef95467a0 2024-05-10T05:35:19.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.910+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef95b0e00 con 0x563ef9554ba0 2024-05-10T05:35:19.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.910+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef95b0fc0 con 0x563ef95470f0 2024-05-10T05:35:19.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.911+0000 7efe228d3b80 10 client.0 _ll_register_callbacks cb 0x563ef940ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:19.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.911+0000 7efe08a00700 1 -- 172.21.15.131:0/2266178946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95470f0 msgr2=0x563ef956b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:19.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.911+0000 7efe08a00700 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95470f0 0x563ef956b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.911+0000 7efe08a00700 1 -- 172.21.15.131:0/2266178946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95467a0 msgr2=0x563ef956b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:19.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.911+0000 7efe08a00700 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95467a0 0x563ef956b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.911+0000 7efe08a00700 1 -- 172.21.15.131:0/2266178946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf000b5f0 con 0x563ef9554ba0 2024-05-10T05:35:19.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.912+0000 7efe03e00700 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95467a0 0x563ef956b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:19.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.912+0000 7efded600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:35:19.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.912+0000 7efe08a00700 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 0x563ef95b08c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efdfc02e7a0 tx=0x7efdfc01a7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec2cf8ad487f6a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:19.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.913+0000 7efded600700 1 -- 172.21.15.131:0/2266178946 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc02ebd0 con 0x563ef9554ba0 2024-05-10T05:35:19.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.913+0000 7efded600700 1 -- 172.21.15.131:0/2266178946 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7efdfc013070 con 0x563ef9554ba0 2024-05-10T05:35:19.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.913+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdf409c640 con 0x563ef9554ba0 2024-05-10T05:35:19.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.914+0000 7efded600700 1 -- 172.21.15.131:0/2266178946 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdfc01ad40 con 0x563ef9554ba0 2024-05-10T05:35:19.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.914+0000 7efe228d3b80 10 client.25763 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:19.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.914+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563ef92eacd0 con 0x563ef9554ba0 2024-05-10T05:35:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.914+0000 7efded600700 1 -- 172.21.15.131:0/2266178946 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdf409c640 con 0x563ef9554ba0 2024-05-10T05:35:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.914+0000 7efded600700 1 -- 172.21.15.131:0/2266178946 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efdfc01b430 con 0x563ef9554ba0 2024-05-10T05:35:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.914+0000 7efe228d3b80 10 client.25763 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.914+0000 7efe228d3b80 10 client.25763 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:35:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.914+0000 7efe228d3b80 20 client.25763 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.914+0000 7efe228d3b80 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563ef95b27c0 0x563ef95b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.915+0000 7efe228d3b80 4 client.25763 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:35:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.915+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563ef9514780 con 0x563ef95b27c0 2024-05-10T05:35:19.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.915+0000 7efe03e00700 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563ef95b27c0 0x563ef95b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:19.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.916+0000 7efded600700 10 client.25763 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:35:19.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.917+0000 7efe03e00700 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563ef95b27c0 0x563ef95b4be0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=424f9c23e7ea9760 server_cookie=1730564ef023b946 in_seq=0 out_seq=0 2024-05-10T05:35:19.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.917+0000 7efded600700 1 -- 172.21.15.131:0/2266178946 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563ef9514780 con 0x563ef95b27c0 2024-05-10T05:35:19.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.917+0000 7efded600700 10 client.25763 handle_command_reply: tid=0 2024-05-10T05:35:19.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.918+0000 7efe228d3b80 1 client.25763 shutdown 2024-05-10T05:35:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.918+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 msgr2=0x563ef95b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.918+0000 7efe228d3b80 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 0x563ef95b08c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efdfc02e7a0 tx=0x7efdfc01a7e0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.918+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 shutdown_connections 2024-05-10T05:35:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.918+0000 7efe228d3b80 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563ef95b27c0 0x563ef95b4be0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.918+0000 7efe228d3b80 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563ef95467a0 0x563ef956b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.918+0000 7efe228d3b80 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef9554ba0 0x563ef95b08c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.918+0000 7efe228d3b80 1 --2- 172.21.15.131:0/2266178946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ef95470f0 0x563ef956b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.918+0000 7efded600700 0 client.25763 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:35:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.918+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 >> 172.21.15.131:0/2266178946 conn(0x563ef94a7a60 msgr2=0x563ef954a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:19.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.919+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 shutdown_connections 2024-05-10T05:35:19.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.919+0000 7efe228d3b80 1 -- 172.21.15.131:0/2266178946 wait complete. 2024-05-10T05:35:19.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.920+0000 7efe228d3b80 20 client.25763 trim_cache size 0 max 16384 2024-05-10T05:35:19.921 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:35:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efde80a12a0 msgr2=0x7efde80a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efde80a12a0 0x7efde80a37b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00ab10 tx=0x7efdfc009250 comp rx=0 tx=0).stop 2024-05-10T05:35:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe041087e0 msgr2=0x7efe041a6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe041087e0 0x7efe041a6b30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c010 tx=0x7efdf400b9e0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 shutdown_connections 2024-05-10T05:35:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efde80a12a0 0x7efde80a37b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe0410fef0 0x7efe041ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe041087e0 0x7efe041a6b30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 --2- 172.21.15.131:0/3308778010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe04109730 0x7efe041a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 >> 172.21.15.131:0/3308778010 conn(0x7efe0406e830 msgr2=0x7efe04070c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 shutdown_connections 2024-05-10T05:35:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:19.921+0000 7efe0aa00700 1 -- 172.21.15.131:0/3308778010 wait complete. 2024-05-10T05:35:19.993 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:35:20.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:20 smithi157 ceph-mon[78427]: pgmap v344: 97 pgs: 97 active+clean; 2.1 GiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 183 MiB/s wr, 61 op/s 2024-05-10T05:35:20.310 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:20 smithi131 ceph-mon[37264]: pgmap v344: 97 pgs: 97 active+clean; 2.1 GiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 183 MiB/s wr, 61 op/s 2024-05-10T05:35:20.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:20 smithi142 ceph-mon[77667]: pgmap v344: 97 pgs: 97 active+clean; 2.1 GiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 183 MiB/s wr, 61 op/s 2024-05-10T05:35:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.773+0000 7f6d7a200700 1 -- 172.21.15.131:0/329878289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d74101be0 msgr2=0x7f6d74101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.773+0000 7f6d7a200700 1 --2- 172.21.15.131:0/329878289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d74101be0 0x7f6d74101ff0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6d600099c0 tx=0x7f6d60009cd0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.773+0000 7f6d7a200700 1 -- 172.21.15.131:0/329878289 shutdown_connections 2024-05-10T05:35:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.773+0000 7f6d7a200700 1 --2- 172.21.15.131:0/329878289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d74100ac0 0x7f6d74100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.773+0000 7f6d7a200700 1 --2- 172.21.15.131:0/329878289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d74102580 0x7f6d74108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.773+0000 7f6d7a200700 1 --2- 172.21.15.131:0/329878289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d74101be0 0x7f6d74101ff0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.773+0000 7f6d7a200700 1 -- 172.21.15.131:0/329878289 >> 172.21.15.131:0/329878289 conn(0x7f6d740fc0d0 msgr2=0x7f6d740fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.773+0000 7f6d7a200700 1 -- 172.21.15.131:0/329878289 shutdown_connections 2024-05-10T05:35:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.773+0000 7f6d7a200700 1 -- 172.21.15.131:0/329878289 wait complete. 2024-05-10T05:35:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.775+0000 7f6d7a200700 1 Processor -- start 2024-05-10T05:35:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.775+0000 7f6d7a200700 1 -- start start 2024-05-10T05:35:20.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.775+0000 7f6d7a200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d74100ac0 0x7f6d7419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d7a200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d74101be0 0x7f6d7419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d7a200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d74102580 0x7f6d741a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d7a200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d741a1b70 con 0x7f6d74102580 2024-05-10T05:35:20.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d7a200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d741a1ce0 con 0x7f6d74101be0 2024-05-10T05:35:20.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d7a200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d741a1ea0 con 0x7f6d74100ac0 2024-05-10T05:35:20.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d73400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d74100ac0 0x7f6d7419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d73400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d74100ac0 0x7f6d7419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52018/0 (socket says 172.21.15.131:52018) 2024-05-10T05:35:20.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d73400700 1 -- 172.21.15.131:0/1830393857 learned_addr learned my addr 172.21.15.131:0/1830393857 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:20.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d72a00700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d74101be0 0x7f6d7419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d73e00700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d74102580 0x7f6d741a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d73400700 1 -- 172.21.15.131:0/1830393857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d74101be0 msgr2=0x7f6d7419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:20.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d73400700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d74101be0 0x7f6d7419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d73400700 1 -- 172.21.15.131:0/1830393857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d74102580 msgr2=0x7f6d741a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:20.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.776+0000 7f6d73400700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d74102580 0x7f6d741a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.777+0000 7f6d73400700 1 -- 172.21.15.131:0/1830393857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d600095f0 con 0x7f6d74100ac0 2024-05-10T05:35:20.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.777+0000 7f6d73400700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d74100ac0 0x7f6d7419a090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6d640097b0 tx=0x7f6d64009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:20.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.777+0000 7f6d6be00700 1 -- 172.21.15.131:0/1830393857 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64018070 con 0x7f6d74100ac0 2024-05-10T05:35:20.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.777+0000 7f6d6be00700 1 -- 172.21.15.131:0/1830393857 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d640044e0 con 0x7f6d74100ac0 2024-05-10T05:35:20.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.777+0000 7f6d7a200700 1 -- 172.21.15.131:0/1830393857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d741a2180 con 0x7f6d74100ac0 2024-05-10T05:35:20.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.777+0000 7f6d6be00700 1 -- 172.21.15.131:0/1830393857 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64010040 con 0x7f6d74100ac0 2024-05-10T05:35:20.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.777+0000 7f6d7a200700 1 -- 172.21.15.131:0/1830393857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d741a26e0 con 0x7f6d74100ac0 2024-05-10T05:35:20.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.781+0000 7f6d6be00700 1 -- 172.21.15.131:0/1830393857 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6d640137d0 con 0x7f6d74100ac0 2024-05-10T05:35:20.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.783+0000 7f6d6be00700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6d580a1240 0x7f6d580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.783+0000 7f6d6be00700 1 -- 172.21.15.131:0/1830393857 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6d640cceb0 con 0x7f6d74100ac0 2024-05-10T05:35:20.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.789+0000 7f6d9203cb80 1 Processor -- start 2024-05-10T05:35:20.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.790+0000 7f6d9203cb80 1 -- start start 2024-05-10T05:35:20.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.792+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1c6a0 0x55dc49b1caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.792+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b1d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.792+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1dec0 0x55dc49b247c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.792+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49a43150 con 0x55dc49b1d570 2024-05-10T05:35:20.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.792+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49a40b10 con 0x55dc49b1c6a0 2024-05-10T05:35:20.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.792+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49a3c400 con 0x55dc49b1dec0 2024-05-10T05:35:20.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.793+0000 7f6d73e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1dec0 0x55dc49b247c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.793+0000 7f6d73e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1dec0 0x55dc49b247c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52026/0 (socket says 172.21.15.131:52026) 2024-05-10T05:35:20.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.793+0000 7f6d73e00700 1 -- 172.21.15.131:0/3281136718 learned_addr learned my addr 172.21.15.131:0/3281136718 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:20.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.793+0000 7f6d73400700 1 --2- 172.21.15.131:0/3281136718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1c6a0 0x55dc49b1caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.793+0000 7f6d72a00700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6d580a1240 0x7f6d580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.793+0000 7f6d73e00700 1 -- 172.21.15.131:0/3281136718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1c6a0 msgr2=0x55dc49b1caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:20.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.793+0000 7f6d73e00700 1 --2- 172.21.15.131:0/3281136718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1c6a0 0x55dc49b1caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.793+0000 7f6d73e00700 1 -- 172.21.15.131:0/3281136718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 msgr2=0x55dc49b1d980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.793+0000 7f6d73e00700 1 --2- 172.21.15.131:0/3281136718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b1d980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.793+0000 7f6d73e00700 1 -- 172.21.15.131:0/3281136718 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc498bd3b0 con 0x55dc49b1dec0 2024-05-10T05:35:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.794+0000 7f6d73e00700 1 --2- 172.21.15.131:0/3281136718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1dec0 0x55dc49b247c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009370 tx=0x7f6d6c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=505b9464176969ee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.794+0000 7f6d5fe00700 1 -- 172.21.15.131:0/3281136718 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00e030 con 0x55dc49b1dec0 2024-05-10T05:35:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.794+0000 7f6d5fe00700 1 -- 172.21.15.131:0/3281136718 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6c00f040 con 0x55dc49b1dec0 2024-05-10T05:35:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.794+0000 7f6d5fe00700 1 -- 172.21.15.131:0/3281136718 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c015c00 con 0x55dc49b1dec0 2024-05-10T05:35:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.796+0000 7f6d9203cb80 1 -- 172.21.15.131:0/3281136718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1dec0 msgr2=0x55dc49b247c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.796+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/3281136718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1dec0 0x55dc49b247c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009370 tx=0x7f6d6c009920 comp rx=0 tx=0).stop 2024-05-10T05:35:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.799+0000 7f6d72a00700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6d580a1240 0x7f6d580a3750 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6d7419b130 tx=0x7f6d60009530 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.799+0000 7f6d9203cb80 1 -- 172.21.15.131:0/3281136718 shutdown_connections 2024-05-10T05:35:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.799+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/3281136718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1dec0 0x55dc49b247c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.799+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/3281136718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1c6a0 0x55dc49b1caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.799+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/3281136718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b1d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.799+0000 7f6d9203cb80 1 -- 172.21.15.131:0/3281136718 >> 172.21.15.131:0/3281136718 conn(0x55dc49a7ea60 msgr2=0x55dc49b16fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:20.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.803+0000 7f6d9203cb80 1 -- 172.21.15.131:0/3281136718 shutdown_connections 2024-05-10T05:35:20.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.805+0000 7f6d9203cb80 1 -- 172.21.15.131:0/3281136718 wait complete. 2024-05-10T05:35:20.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.807+0000 7f6d9203cb80 1 -- start start 2024-05-10T05:35:20.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.808+0000 7f6d9203cb80 1 Processor -- start 2024-05-10T05:35:20.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.809+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b26710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.809+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b26c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.809+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 0x55dc49b2dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.809+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc499d8560 con 0x55dc49b1d570 2024-05-10T05:35:20.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.809+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc499d8200 con 0x55dc49b1dec0 2024-05-10T05:35:20.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.809+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc498bf6a0 con 0x55dc49b1c6a0 2024-05-10T05:35:20.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.809+0000 7f6d9203cb80 10 client.0 _ll_register_callbacks cb 0x55dc499e1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:20.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.811+0000 7f6d73e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 0x55dc49b2dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.811+0000 7f6d73e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 0x55dc49b2dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46662/0 (socket says 172.21.15.131:46662) 2024-05-10T05:35:20.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.811+0000 7f6d73e00700 1 -- 172.21.15.131:0/776853087 learned_addr learned my addr 172.21.15.131:0/776853087 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:20.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.812+0000 7f6d72a00700 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b26c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.814+0000 7f6d73400700 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b26710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.815+0000 7f6d72a00700 1 -- 172.21.15.131:0/776853087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 msgr2=0x55dc49b26710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:20.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.815+0000 7f6d72a00700 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b26710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.815+0000 7f6d72a00700 1 -- 172.21.15.131:0/776853087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 msgr2=0x55dc49b2dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.815+0000 7f6d72a00700 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 0x55dc49b2dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.815+0000 7f6d72a00700 1 -- 172.21.15.131:0/776853087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c009050 con 0x55dc49b1d570 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.815+0000 7f6d72a00700 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b26c50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6d600058e0 tx=0x7f6d60009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b189471494d0a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.815+0000 7f6d5cc00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.816+0000 7f6d5cc00700 1 -- 172.21.15.131:0/776853087 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6001d070 con 0x55dc49b1d570 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.816+0000 7f6d5cc00700 1 -- 172.21.15.131:0/776853087 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6d600039e0 con 0x55dc49b1d570 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.816+0000 7f6d9203cb80 1 -- 172.21.15.131:0/776853087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc49995270 con 0x55dc49b1d570 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.817+0000 7f6d5cc00700 1 -- 172.21.15.131:0/776853087 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d60003ba0 con 0x55dc49b1d570 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.817+0000 7f6d9203cb80 10 client.15756 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.817+0000 7f6d9203cb80 1 -- 172.21.15.131:0/776853087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc49869b60 con 0x55dc49b1d570 2024-05-10T05:35:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.818+0000 7f6d5cc00700 1 -- 172.21.15.131:0/776853087 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d60035030 con 0x55dc49b1d570 2024-05-10T05:35:20.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.819+0000 7f6d5cc00700 1 -- 172.21.15.131:0/776853087 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d6000f460 con 0x55dc49b1d570 2024-05-10T05:35:20.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.819+0000 7f6d9203cb80 10 client.15756 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:20.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.819+0000 7f6d9203cb80 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:35:20.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.819+0000 7f6d9203cb80 20 client.15756 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:20.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.819+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dc49b4bae0 0x55dc49b4df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.819+0000 7f6d9203cb80 4 client.15756 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:20.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.819+0000 7f6d9203cb80 1 -- 172.21.15.131:0/776853087 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc49aeb780 con 0x55dc49b4bae0 2024-05-10T05:35:20.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.822+0000 7f6d73400700 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dc49b4bae0 0x55dc49b4df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.823+0000 7f6d5cc00700 10 client.15756 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:20.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.823+0000 7f6d73400700 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dc49b4bae0 0x55dc49b4df50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad2fdad30cd1f8d4 server_cookie=96adbbfbf56cb0f2 in_seq=0 out_seq=0 2024-05-10T05:35:20.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.824+0000 7f6d5cc00700 1 -- 172.21.15.131:0/776853087 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc49aeb780 con 0x55dc49b4bae0 2024-05-10T05:35:20.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.824+0000 7f6d5cc00700 10 client.15756 handle_command_reply: tid=0 2024-05-10T05:35:20.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.824+0000 7f6d9203cb80 1 client.15756 shutdown 2024-05-10T05:35:20.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.827+0000 7f6d9203cb80 1 -- 172.21.15.131:0/776853087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 msgr2=0x55dc49b26c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:20.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.827+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b26c50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6d600058e0 tx=0x7f6d60009310 comp rx=0 tx=0).stop 2024-05-10T05:35:20.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.828+0000 7f6d9203cb80 1 -- 172.21.15.131:0/776853087 shutdown_connections 2024-05-10T05:35:20.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.828+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dc49b4bae0 0x55dc49b4df50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.829+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 0x55dc49b2dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.829+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b26710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.829+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/776853087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b26c50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.829+0000 7f6d9203cb80 1 -- 172.21.15.131:0/776853087 >> 172.21.15.131:0/776853087 conn(0x55dc49a7ea60 msgr2=0x55dc49b16fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:20.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.829+0000 7f6d5cc00700 0 client.15756 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:20.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.830+0000 7f6d9203cb80 1 -- 172.21.15.131:0/776853087 shutdown_connections 2024-05-10T05:35:20.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.830+0000 7f6d9203cb80 1 -- 172.21.15.131:0/776853087 wait complete. 2024-05-10T05:35:20.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.831+0000 7f6d9203cb80 20 client.15756 trim_cache size 0 max 16384 2024-05-10T05:35:20.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.891+0000 7f6d9203cb80 1 Processor -- start 2024-05-10T05:35:20.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.891+0000 7f6d9203cb80 1 -- start start 2024-05-10T05:35:20.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.893+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b1d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.895+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b2db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.895+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 0x55dc49b3f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.895+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49b3fcf0 con 0x55dc49b1d570 2024-05-10T05:35:20.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.895+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49b26e50 con 0x55dc49b1dec0 2024-05-10T05:35:20.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.895+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49b4b830 con 0x55dc49b1c6a0 2024-05-10T05:35:20.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.895+0000 7f6d73400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b1d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.896+0000 7f6d72a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b2db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.896+0000 7f6d72a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b2db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38442/0 (socket says 172.21.15.131:38442) 2024-05-10T05:35:20.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.896+0000 7f6d73400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b1d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52050/0 (socket says 172.21.15.131:52050) 2024-05-10T05:35:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.896+0000 7f6d73e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 0x55dc49b3f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.896+0000 7f6d73400700 1 -- 172.21.15.131:0/1802817089 learned_addr learned my addr 172.21.15.131:0/1802817089 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.897+0000 7f6d73e00700 1 -- 172.21.15.131:0/1802817089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 msgr2=0x55dc49b1d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.897+0000 7f6d73e00700 1 --2- 172.21.15.131:0/1802817089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b1d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.897+0000 7f6d73e00700 1 -- 172.21.15.131:0/1802817089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 msgr2=0x55dc49b2db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.897+0000 7f6d73e00700 1 --2- 172.21.15.131:0/1802817089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b2db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.897+0000 7f6d73e00700 1 -- 172.21.15.131:0/1802817089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc49a02560 con 0x55dc49b1dec0 2024-05-10T05:35:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.897+0000 7f6d72a00700 1 --2- 172.21.15.131:0/1802817089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b2db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.898+0000 7f6d73400700 1 --2- 172.21.15.131:0/1802817089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b1d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.898+0000 7f6d73e00700 1 --2- 172.21.15.131:0/1802817089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 0x55dc49b3f150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009d90 tx=0x7f6d6c0060e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9ec3e702279db3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.902+0000 7f6d69200700 1 -- 172.21.15.131:0/1802817089 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00e040 con 0x55dc49b1dec0 2024-05-10T05:35:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.902+0000 7f6d69200700 1 -- 172.21.15.131:0/1802817089 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6c004c90 con 0x55dc49b1dec0 2024-05-10T05:35:20.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.902+0000 7f6d69200700 1 -- 172.21.15.131:0/1802817089 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c015e90 con 0x55dc49b1dec0 2024-05-10T05:35:20.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.903+0000 7f6d9203cb80 1 -- 172.21.15.131:0/1802817089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 msgr2=0x55dc49b3f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.903+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/1802817089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 0x55dc49b3f150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009d90 tx=0x7f6d6c0060e0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.903+0000 7f6d9203cb80 1 -- 172.21.15.131:0/1802817089 shutdown_connections 2024-05-10T05:35:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.904+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/1802817089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1dec0 0x55dc49b3f150 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.904+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/1802817089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b1d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.904+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/1802817089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1d570 0x55dc49b2db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.904+0000 7f6d9203cb80 1 -- 172.21.15.131:0/1802817089 >> 172.21.15.131:0/1802817089 conn(0x55dc49a7ea60 msgr2=0x55dc49b26710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:20.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.904+0000 7f6d9203cb80 1 -- 172.21.15.131:0/1802817089 shutdown_connections 2024-05-10T05:35:20.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.905+0000 7f6d9203cb80 1 -- 172.21.15.131:0/1802817089 wait complete. 2024-05-10T05:35:20.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.908+0000 7f6d9203cb80 1 -- start start 2024-05-10T05:35:20.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.908+0000 7f6d9203cb80 1 Processor -- start 2024-05-10T05:35:20.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.921+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b2b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.921+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1d570 0x55dc49b2bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.921+0000 7f6d9203cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1dec0 0x55dc49b2cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.921+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49b2d170 con 0x55dc49b1dec0 2024-05-10T05:35:20.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.921+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49b2d2e0 con 0x55dc49b1d570 2024-05-10T05:35:20.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.921+0000 7f6d9203cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49b2d4a0 con 0x55dc49b1c6a0 2024-05-10T05:35:20.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.921+0000 7f6d9203cb80 10 client.0 _ll_register_callbacks cb 0x55dc499e1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:20.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.922+0000 7f6d73e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1dec0 0x55dc49b2cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.923+0000 7f6d73e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1dec0 0x55dc49b2cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38450/0 (socket says 172.21.15.131:38450) 2024-05-10T05:35:20.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.923+0000 7f6d73e00700 1 -- 172.21.15.131:0/4040160306 learned_addr learned my addr 172.21.15.131:0/4040160306 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:20.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.924+0000 7f6d72a00700 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1d570 0x55dc49b2bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.925+0000 7f6d73e00700 1 -- 172.21.15.131:0/4040160306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 msgr2=0x55dc49b2b8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:35:20.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.925+0000 7f6d73e00700 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b2b8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.925+0000 7f6d73e00700 1 -- 172.21.15.131:0/4040160306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1d570 msgr2=0x55dc49b2bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:20.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.925+0000 7f6d73e00700 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1d570 0x55dc49b2bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.925+0000 7f6d73e00700 1 -- 172.21.15.131:0/4040160306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d640cae70 con 0x55dc49b1dec0 2024-05-10T05:35:20.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.928+0000 7f6d73e00700 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1dec0 0x55dc49b2cc30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009cf0 tx=0x7f6d6c016f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca3a5832503a5bc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:20.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.928+0000 7f6d5cc00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:35:20.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.930+0000 7f6d5cc00700 1 -- 172.21.15.131:0/4040160306 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00e040 con 0x55dc49b1dec0 2024-05-10T05:35:20.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.930+0000 7f6d5cc00700 1 -- 172.21.15.131:0/4040160306 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6d6c0147f0 con 0x55dc49b1dec0 2024-05-10T05:35:20.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.930+0000 7f6d9203cb80 1 -- 172.21.15.131:0/4040160306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d6c00f040 con 0x55dc49b1dec0 2024-05-10T05:35:20.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.936+0000 7f6d5cc00700 1 -- 172.21.15.131:0/4040160306 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d6c0149b0 con 0x55dc49b1dec0 2024-05-10T05:35:20.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.936+0000 7f6d9203cb80 10 client.15762 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:20.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.937+0000 7f6d9203cb80 1 -- 172.21.15.131:0/4040160306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc49869b60 con 0x55dc49b1dec0 2024-05-10T05:35:20.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.948+0000 7f6d5cc00700 1 -- 172.21.15.131:0/4040160306 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6c00f040 con 0x55dc49b1dec0 2024-05-10T05:35:20.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.948+0000 7f6d5cc00700 1 -- 172.21.15.131:0/4040160306 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d6c027070 con 0x55dc49b1dec0 2024-05-10T05:35:20.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.956+0000 7f6d9203cb80 10 client.15762 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:20.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.956+0000 7f6d9203cb80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:35:20.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.956+0000 7f6d9203cb80 20 client.15762 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:20.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.956+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dc49ba2880 0x55dc49ba4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:20.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.956+0000 7f6d9203cb80 4 client.15762 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:35:20.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.956+0000 7f6d9203cb80 1 -- 172.21.15.131:0/4040160306 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc49a34ec0 con 0x55dc49ba2880 2024-05-10T05:35:20.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.959+0000 7f6d73400700 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dc49ba2880 0x55dc49ba4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:20.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.963+0000 7f6d73400700 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dc49ba2880 0x55dc49ba4ca0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b3cc18a56a2538 server_cookie=d9130a68fdd82c81 in_seq=0 out_seq=0 2024-05-10T05:35:20.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.964+0000 7f6d5cc00700 10 client.15762 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:20.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.964+0000 7f6d5cc00700 1 -- 172.21.15.131:0/4040160306 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc49a34ec0 con 0x55dc49ba2880 2024-05-10T05:35:20.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.964+0000 7f6d5cc00700 10 client.15762 handle_command_reply: tid=0 2024-05-10T05:35:20.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.964+0000 7f6d9203cb80 1 client.15762 shutdown 2024-05-10T05:35:20.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.964+0000 7f6d9203cb80 1 -- 172.21.15.131:0/4040160306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1dec0 msgr2=0x55dc49b2cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:20.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.964+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1dec0 0x55dc49b2cc30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009cf0 tx=0x7f6d6c016f00 comp rx=0 tx=0).stop 2024-05-10T05:35:20.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.967+0000 7f6d9203cb80 1 -- 172.21.15.131:0/4040160306 shutdown_connections 2024-05-10T05:35:20.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.967+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dc49ba2880 0x55dc49ba4ca0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.967+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc49b1dec0 0x55dc49b2cc30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.967+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc49b1c6a0 0x55dc49b2b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.967+0000 7f6d9203cb80 1 --2- 172.21.15.131:0/4040160306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc49b1d570 0x55dc49b2bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.967+0000 7f6d5cc00700 0 client.15762 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:20.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.967+0000 7f6d9203cb80 1 -- 172.21.15.131:0/4040160306 >> 172.21.15.131:0/4040160306 conn(0x55dc49a7ea60 msgr2=0x55dc49b213d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:20.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.967+0000 7f6d9203cb80 1 -- 172.21.15.131:0/4040160306 shutdown_connections 2024-05-10T05:35:20.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.968+0000 7f6d9203cb80 1 -- 172.21.15.131:0/4040160306 wait complete. 2024-05-10T05:35:20.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.968+0000 7f6d9203cb80 20 client.15762 trim_cache size 0 max 16384 2024-05-10T05:35:20.969 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:35:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.969+0000 7f6d7a200700 1 -- 172.21.15.131:0/1830393857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6d580a1240 msgr2=0x7f6d580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.969+0000 7f6d7a200700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6d580a1240 0x7f6d580a3750 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6d7419b130 tx=0x7f6d60009530 comp rx=0 tx=0).stop 2024-05-10T05:35:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.969+0000 7f6d7a200700 1 -- 172.21.15.131:0/1830393857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d74100ac0 msgr2=0x7f6d7419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.969+0000 7f6d7a200700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d74100ac0 0x7f6d7419a090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6d640097b0 tx=0x7f6d64009b70 comp rx=0 tx=0).stop 2024-05-10T05:35:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.970+0000 7f6d7a200700 1 -- 172.21.15.131:0/1830393857 shutdown_connections 2024-05-10T05:35:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.970+0000 7f6d7a200700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6d580a1240 0x7f6d580a3750 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.970+0000 7f6d7a200700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d74102580 0x7f6d741a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.970+0000 7f6d7a200700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d74100ac0 0x7f6d7419a090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.970+0000 7f6d7a200700 1 --2- 172.21.15.131:0/1830393857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d74101be0 0x7f6d7419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:20.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.970+0000 7f6d7a200700 1 -- 172.21.15.131:0/1830393857 >> 172.21.15.131:0/1830393857 conn(0x7f6d740fc0d0 msgr2=0x7f6d74104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.974+0000 7f6d7a200700 1 -- 172.21.15.131:0/1830393857 shutdown_connections 2024-05-10T05:35:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:20.974+0000 7f6d7a200700 1 -- 172.21.15.131:0/1830393857 wait complete. 2024-05-10T05:35:22.055 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:35:22.055 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:35:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:22 smithi142 ceph-mon[77667]: pgmap v345: 97 pgs: 97 active+clean; 2.6 GiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 214 MiB/s wr, 68 op/s 2024-05-10T05:35:22.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:22 smithi131 ceph-mon[37264]: pgmap v345: 97 pgs: 97 active+clean; 2.6 GiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 214 MiB/s wr, 68 op/s 2024-05-10T05:35:22.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:22 smithi157 ceph-mon[78427]: pgmap v345: 97 pgs: 97 active+clean; 2.6 GiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 214 MiB/s wr, 68 op/s 2024-05-10T05:35:22.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.589+0000 7f4107600700 1 -- 172.21.15.131:0/3741579859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100109c50 msgr2=0x7f410010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:22.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.589+0000 7f4107600700 1 --2- 172.21.15.131:0/3741579859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100109c50 0x7f410010e560 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f40f801c0d0 tx=0x7f40f801c3e0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.590+0000 7f4107600700 1 -- 172.21.15.131:0/3741579859 shutdown_connections 2024-05-10T05:35:22.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.590+0000 7f4107600700 1 --2- 172.21.15.131:0/3741579859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4100108630 0x7f4100108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.590+0000 7f4107600700 1 --2- 172.21.15.131:0/3741579859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100109c50 0x7f410010e560 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.590+0000 7f4107600700 1 --2- 172.21.15.131:0/3741579859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4100109300 0x7f4100109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.590+0000 7f4107600700 1 -- 172.21.15.131:0/3741579859 >> 172.21.15.131:0/3741579859 conn(0x7f410006ea00 msgr2=0x7f4100070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:22.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.590+0000 7f4107600700 1 -- 172.21.15.131:0/3741579859 shutdown_connections 2024-05-10T05:35:22.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.590+0000 7f4107600700 1 -- 172.21.15.131:0/3741579859 wait complete. 2024-05-10T05:35:22.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.592+0000 7f4107600700 1 Processor -- start 2024-05-10T05:35:22.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.592+0000 7f4107600700 1 -- start start 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f4107600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100108630 0x7f410019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f4107600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4100109300 0x7f410019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f4107600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4100109c50 0x7f41001a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f4107600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41001a5f30 con 0x7f4100109300 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f4107600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41001a60a0 con 0x7f4100109c50 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f4107600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41001a6260 con 0x7f4100108630 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f4104c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100108630 0x7f410019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f4104c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100108630 0x7f410019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52076/0 (socket says 172.21.15.131:52076) 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f40ffe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4100109300 0x7f410019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f4104c00700 1 -- 172.21.15.131:0/3528824142 learned_addr learned my addr 172.21.15.131:0/3528824142 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:22.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.593+0000 7f4105600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4100109c50 0x7f41001a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.594+0000 7f4105600700 1 -- 172.21.15.131:0/3528824142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100108630 msgr2=0x7f410019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:22.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.594+0000 7f4105600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100108630 0x7f410019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.594+0000 7f4105600700 1 -- 172.21.15.131:0/3528824142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4100109300 msgr2=0x7f410019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:22.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.594+0000 7f4105600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4100109300 0x7f410019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.594+0000 7f4105600700 1 -- 172.21.15.131:0/3528824142 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f801ad10 con 0x7f4100109c50 2024-05-10T05:35:22.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.594+0000 7f4104c00700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100108630 0x7f410019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:22.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.594+0000 7f4105600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4100109c50 0x7f41001a5920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f40f801c7b0 tx=0x7f40f801ddd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:22.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.595+0000 7f40fd600700 1 -- 172.21.15.131:0/3528824142 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f801f050 con 0x7f4100109c50 2024-05-10T05:35:22.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.595+0000 7f4107600700 1 -- 172.21.15.131:0/3528824142 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41001a64e0 con 0x7f4100109c50 2024-05-10T05:35:22.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.595+0000 7f40fd600700 1 -- 172.21.15.131:0/3528824142 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40f8006160 con 0x7f4100109c50 2024-05-10T05:35:22.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.595+0000 7f40fd600700 1 -- 172.21.15.131:0/3528824142 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f802c8b0 con 0x7f4100109c50 2024-05-10T05:35:22.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.596+0000 7f4107600700 1 -- 172.21.15.131:0/3528824142 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41001a6960 con 0x7f4100109c50 2024-05-10T05:35:22.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.601+0000 7f40fd600700 1 -- 172.21.15.131:0/3528824142 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f40f8005040 con 0x7f4100109c50 2024-05-10T05:35:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.604+0000 7f411f57cb80 1 Processor -- start 2024-05-10T05:35:22.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.604+0000 7f40fd600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40e80a1240 0x7f40e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.604+0000 7f411f57cb80 1 -- start start 2024-05-10T05:35:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.605+0000 7f40fd600700 1 -- 172.21.15.131:0/3528824142 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f40f80dece0 con 0x7f4100109c50 2024-05-10T05:35:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.605+0000 7f40ffe00700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40e80a1240 0x7f40e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.605+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bb2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.605+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb35f0 0x5648d8bb3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.605+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.605+0000 7f411f57cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8ad91d0 con 0x5648d8bb35f0 2024-05-10T05:35:22.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.605+0000 7f411f57cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8ad6b90 con 0x5648d8bb2720 2024-05-10T05:35:22.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.605+0000 7f411f57cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8ad2480 con 0x5648d8bb3f40 2024-05-10T05:35:22.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.606+0000 7f4104c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bb2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.606+0000 7f4104c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bb2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46708/0 (socket says 172.21.15.131:46708) 2024-05-10T05:35:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.606+0000 7f4104c00700 1 -- 172.21.15.131:0/725765598 learned_addr learned my addr 172.21.15.131:0/725765598 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.606+0000 7f4105600700 1 --2- 172.21.15.131:0/725765598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.606+0000 7f4104c00700 1 -- 172.21.15.131:0/725765598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 msgr2=0x5648d8bba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.606+0000 7f4104c00700 1 --2- 172.21.15.131:0/725765598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.606+0000 7f4104c00700 1 -- 172.21.15.131:0/725765598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb35f0 msgr2=0x5648d8bb3a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.606+0000 7f4104c00700 1 --2- 172.21.15.131:0/725765598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb35f0 0x5648d8bb3a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.606+0000 7f4104c00700 1 -- 172.21.15.131:0/725765598 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648d8953430 con 0x5648d8bb2720 2024-05-10T05:35:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.606+0000 7f4105600700 1 --2- 172.21.15.131:0/725765598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.607+0000 7f40ffe00700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40e80a1240 0x7f40e80a3750 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005fd0 tx=0x7f40ec005f60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:22.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.607+0000 7f4104c00700 1 --2- 172.21.15.131:0/725765598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bb2b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009e30 tx=0x7f40f40116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba00c70aaa16f62a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.608+0000 7f40f1c00700 1 -- 172.21.15.131:0/725765598 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f4011ed0 con 0x5648d8bb2720 2024-05-10T05:35:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.608+0000 7f40f1c00700 1 -- 172.21.15.131:0/725765598 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40f4012910 con 0x5648d8bb2720 2024-05-10T05:35:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.609+0000 7f411f57cb80 1 -- 172.21.15.131:0/725765598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 msgr2=0x5648d8bb2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.609+0000 7f411f57cb80 1 --2- 172.21.15.131:0/725765598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bb2b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009e30 tx=0x7f40f40116c0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.610+0000 7f411f57cb80 1 -- 172.21.15.131:0/725765598 shutdown_connections 2024-05-10T05:35:22.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.610+0000 7f411f57cb80 1 --2- 172.21.15.131:0/725765598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.610+0000 7f411f57cb80 1 --2- 172.21.15.131:0/725765598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bb2b70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.610+0000 7f411f57cb80 1 --2- 172.21.15.131:0/725765598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb35f0 0x5648d8bb3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.610+0000 7f411f57cb80 1 -- 172.21.15.131:0/725765598 >> 172.21.15.131:0/725765598 conn(0x5648d8b14ae0 msgr2=0x5648d8bad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.610+0000 7f411f57cb80 1 -- 172.21.15.131:0/725765598 shutdown_connections 2024-05-10T05:35:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.610+0000 7f411f57cb80 1 -- 172.21.15.131:0/725765598 wait complete. 2024-05-10T05:35:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.612+0000 7f411f57cb80 1 -- start start 2024-05-10T05:35:22.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.614+0000 7f411f57cb80 1 Processor -- start 2024-05-10T05:35:22.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.622+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 0x5648d8bbc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.622+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb35f0 0x5648d8bbcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.622+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bc3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.622+0000 7f411f57cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8a6e5e0 con 0x5648d8bb2720 2024-05-10T05:35:22.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.622+0000 7f411f57cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8a6e280 con 0x5648d8bb35f0 2024-05-10T05:35:22.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.622+0000 7f411f57cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8955720 con 0x5648d8bb3f40 2024-05-10T05:35:22.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.623+0000 7f411f57cb80 10 client.0 _ll_register_callbacks cb 0x5648d8a77bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:22.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.625+0000 7f4105600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bc3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.626+0000 7f4105600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bc3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52094/0 (socket says 172.21.15.131:52094) 2024-05-10T05:35:22.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.626+0000 7f4105600700 1 -- 172.21.15.131:0/2107173306 learned_addr learned my addr 172.21.15.131:0/2107173306 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:22.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.626+0000 7f40ffe00700 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb35f0 0x5648d8bbcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.626+0000 7f4105600700 1 -- 172.21.15.131:0/2107173306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb35f0 msgr2=0x5648d8bbcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.626+0000 7f4105600700 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb35f0 0x5648d8bbcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.626+0000 7f4105600700 1 -- 172.21.15.131:0/2107173306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 msgr2=0x5648d8bbc750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.626+0000 7f4105600700 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 0x5648d8bbc750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.627+0000 7f4105600700 1 -- 172.21.15.131:0/2107173306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f4009a60 con 0x5648d8bb3f40 2024-05-10T05:35:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.627+0000 7f40ffe00700 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb35f0 0x5648d8bbcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.627+0000 7f40e6000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:35:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.628+0000 7f4105600700 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bc3bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f40f80b4080 tx=0x7f40f80abe30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a4f1270b14c9b4f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:22.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.628+0000 7f40e6000700 1 -- 172.21.15.131:0/2107173306 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80ac1c0 con 0x5648d8bb3f40 2024-05-10T05:35:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.628+0000 7f411f57cb80 1 -- 172.21.15.131:0/2107173306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648d8a2b2f0 con 0x5648d8bb3f40 2024-05-10T05:35:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.628+0000 7f40e6000700 1 -- 172.21.15.131:0/2107173306 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f40f80afff0 con 0x5648d8bb3f40 2024-05-10T05:35:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.629+0000 7f40e6000700 1 -- 172.21.15.131:0/2107173306 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40f80ac1c0 con 0x5648d8bb3f40 2024-05-10T05:35:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.629+0000 7f411f57cb80 10 client.25810 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.629+0000 7f411f57cb80 1 -- 172.21.15.131:0/2107173306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648d88ffbe0 con 0x5648d8bb3f40 2024-05-10T05:35:22.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.631+0000 7f40e6000700 1 -- 172.21.15.131:0/2107173306 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648d8a2b2f0 con 0x5648d8bb3f40 2024-05-10T05:35:22.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.631+0000 7f40e6000700 1 -- 172.21.15.131:0/2107173306 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f40f80b20b0 con 0x5648d8bb3f40 2024-05-10T05:35:22.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.631+0000 7f411f57cb80 10 client.25810 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.632+0000 7f411f57cb80 10 client.25810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:35:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.632+0000 7f411f57cb80 20 client.25810 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.632+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648d8bc7ff0 0x5648d8bca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.632+0000 7f411f57cb80 4 client.25810 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.632+0000 7f411f57cb80 1 -- 172.21.15.131:0/2107173306 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648d8b81800 con 0x5648d8bc7ff0 2024-05-10T05:35:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.632+0000 7f4104c00700 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648d8bc7ff0 0x5648d8bca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.633+0000 7f40e6000700 10 client.25810 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.634+0000 7f4104c00700 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648d8bc7ff0 0x5648d8bca440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba8775373c2f921 server_cookie=a4cd85674eaefb02 in_seq=0 out_seq=0 2024-05-10T05:35:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.634+0000 7f40e6000700 1 -- 172.21.15.131:0/2107173306 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648d8b81800 con 0x5648d8bc7ff0 2024-05-10T05:35:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.634+0000 7f40e6000700 10 client.25810 handle_command_reply: tid=0 2024-05-10T05:35:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.635+0000 7f411f57cb80 1 client.25810 shutdown 2024-05-10T05:35:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.635+0000 7f411f57cb80 1 -- 172.21.15.131:0/2107173306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 msgr2=0x5648d8bc3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.635+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bc3bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f40f80b4080 tx=0x7f40f80abe30 comp rx=0 tx=0).stop 2024-05-10T05:35:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.635+0000 7f411f57cb80 1 -- 172.21.15.131:0/2107173306 shutdown_connections 2024-05-10T05:35:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.635+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648d8bc7ff0 0x5648d8bca440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.635+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 0x5648d8bbc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.635+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb3f40 0x5648d8bc3bf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.635+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2107173306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb35f0 0x5648d8bbcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.635+0000 7f411f57cb80 1 -- 172.21.15.131:0/2107173306 >> 172.21.15.131:0/2107173306 conn(0x5648d8b14ae0 msgr2=0x5648d8bad030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.635+0000 7f40e6000700 0 client.25810 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.636+0000 7f411f57cb80 1 -- 172.21.15.131:0/2107173306 shutdown_connections 2024-05-10T05:35:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.636+0000 7f411f57cb80 1 -- 172.21.15.131:0/2107173306 wait complete. 2024-05-10T05:35:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.637+0000 7f411f57cb80 20 client.25810 trim_cache size 0 max 16384 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.678+0000 7f411f57cb80 1 Processor -- start 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.678+0000 7f411f57cb80 1 -- start start 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.679+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8b14ae0 0x5648d8bb7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.679+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 0x5648d8bc3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.679+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 0x5648d8bb7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.679+0000 7f411f57cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8bb34b0 con 0x5648d8bb2720 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.679+0000 7f411f57cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8bba670 con 0x5648d8b14ae0 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.679+0000 7f411f57cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8bb6a70 con 0x5648d8bb35f0 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.680+0000 7f40ffe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 0x5648d8bc3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.680+0000 7f40ffe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 0x5648d8bc3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38476/0 (socket says 172.21.15.131:38476) 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.680+0000 7f40ffe00700 1 -- 172.21.15.131:0/1050645213 learned_addr learned my addr 172.21.15.131:0/1050645213 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.680+0000 7f4104c00700 1 --2- 172.21.15.131:0/1050645213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8b14ae0 0x5648d8bb7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.680+0000 7f4105600700 1 --2- 172.21.15.131:0/1050645213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 0x5648d8bb7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.681+0000 7f40ffe00700 1 -- 172.21.15.131:0/1050645213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 msgr2=0x5648d8bb7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.681+0000 7f40ffe00700 1 --2- 172.21.15.131:0/1050645213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 0x5648d8bb7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.681+0000 7f40ffe00700 1 -- 172.21.15.131:0/1050645213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8b14ae0 msgr2=0x5648d8bb7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.681+0000 7f40ffe00700 1 --2- 172.21.15.131:0/1050645213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8b14ae0 0x5648d8bb7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.681+0000 7f40ffe00700 1 -- 172.21.15.131:0/1050645213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648d8a985e0 con 0x5648d8bb2720 2024-05-10T05:35:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.681+0000 7f40ffe00700 1 --2- 172.21.15.131:0/1050645213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 0x5648d8bc3bb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00fcc0 tx=0x7f40ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b1c0875ce7527f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.683+0000 7f40f2600700 1 -- 172.21.15.131:0/1050645213 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec011b10 con 0x5648d8bb2720 2024-05-10T05:35:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.683+0000 7f40f2600700 1 -- 172.21.15.131:0/1050645213 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ec004cf0 con 0x5648d8bb2720 2024-05-10T05:35:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.683+0000 7f40f2600700 1 -- 172.21.15.131:0/1050645213 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec011b10 con 0x5648d8bb2720 2024-05-10T05:35:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.686+0000 7f411f57cb80 1 -- 172.21.15.131:0/1050645213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 msgr2=0x5648d8bc3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.686+0000 7f411f57cb80 1 --2- 172.21.15.131:0/1050645213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 0x5648d8bc3bb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00fcc0 tx=0x7f40ec012000 comp rx=0 tx=0).stop 2024-05-10T05:35:22.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.687+0000 7f411f57cb80 1 -- 172.21.15.131:0/1050645213 shutdown_connections 2024-05-10T05:35:22.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.687+0000 7f411f57cb80 1 --2- 172.21.15.131:0/1050645213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 0x5648d8bb7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.687+0000 7f411f57cb80 1 --2- 172.21.15.131:0/1050645213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8b14ae0 0x5648d8bb7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.687+0000 7f411f57cb80 1 --2- 172.21.15.131:0/1050645213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8bb2720 0x5648d8bc3bb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.687+0000 7f411f57cb80 1 -- 172.21.15.131:0/1050645213 >> 172.21.15.131:0/1050645213 conn(0x5648d8ba94d0 msgr2=0x5648d8bbc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:22.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.688+0000 7f411f57cb80 1 -- 172.21.15.131:0/1050645213 shutdown_connections 2024-05-10T05:35:22.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.688+0000 7f411f57cb80 1 -- 172.21.15.131:0/1050645213 wait complete. 2024-05-10T05:35:22.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.689+0000 7f411f57cb80 1 -- start start 2024-05-10T05:35:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.689+0000 7f411f57cb80 1 Processor -- start 2024-05-10T05:35:22.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.692+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8b14ae0 0x5648d8bc2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.694+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bc2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.694+0000 7f4104c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8b14ae0 0x5648d8bc2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.694+0000 7f411f57cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 0x5648d8bc3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.694+0000 7f411f57cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8c1e570 con 0x5648d8b14ae0 2024-05-10T05:35:22.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.694+0000 7f40ffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bc2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.694+0000 7f40ffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bc2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:46744/0 (socket says 172.21.15.131:46744) 2024-05-10T05:35:22.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.694+0000 7f40ffe00700 1 -- 172.21.15.131:0/2416952244 learned_addr learned my addr 172.21.15.131:0/2416952244 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:22.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.695+0000 7f4105600700 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 0x5648d8bc3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.695+0000 7f411f57cb80 1 -- 172.21.15.131:0/2416952244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8c1e6e0 con 0x5648d8bb2720 2024-05-10T05:35:22.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.695+0000 7f411f57cb80 1 -- 172.21.15.131:0/2416952244 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8c1e8a0 con 0x5648d8bb35f0 2024-05-10T05:35:22.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.695+0000 7f411f57cb80 10 client.0 _ll_register_callbacks cb 0x5648d8a77bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:22.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.697+0000 7f40ffe00700 1 -- 172.21.15.131:0/2416952244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 msgr2=0x5648d8bc3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:22.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.697+0000 7f40ffe00700 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 0x5648d8bc3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.697+0000 7f40ffe00700 1 -- 172.21.15.131:0/2416952244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8b14ae0 msgr2=0x5648d8bc2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:22.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.697+0000 7f40ffe00700 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8b14ae0 0x5648d8bc2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.697+0000 7f40ffe00700 1 -- 172.21.15.131:0/2416952244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec00f970 con 0x5648d8bb2720 2024-05-10T05:35:22.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.698+0000 7f4105600700 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 0x5648d8bc3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:22.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.698+0000 7f4104c00700 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8b14ae0 0x5648d8bc2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:22.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.700+0000 7f40e6000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:35:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.700+0000 7f40ffe00700 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bc2ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005100 tx=0x7f40ec0055e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=601a1ca3a4359c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.701+0000 7f40e6000700 1 -- 172.21.15.131:0/2416952244 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec011910 con 0x5648d8bb2720 2024-05-10T05:35:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.702+0000 7f411f57cb80 1 -- 172.21.15.131:0/2416952244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40f4012910 con 0x5648d8bb2720 2024-05-10T05:35:22.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.702+0000 7f40e6000700 1 -- 172.21.15.131:0/2416952244 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f40ec011a70 con 0x5648d8bb2720 2024-05-10T05:35:22.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.704+0000 7f40e6000700 1 -- 172.21.15.131:0/2416952244 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40ec011910 con 0x5648d8bb2720 2024-05-10T05:35:22.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.705+0000 7f411f57cb80 10 client.25793 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:22.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.705+0000 7f411f57cb80 1 -- 172.21.15.131:0/2416952244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648d8957d50 con 0x5648d8bb2720 2024-05-10T05:35:22.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.706+0000 7f40e6000700 1 -- 172.21.15.131:0/2416952244 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40f4012910 con 0x5648d8bb2720 2024-05-10T05:35:22.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.706+0000 7f40e6000700 1 -- 172.21.15.131:0/2416952244 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f40ec046030 con 0x5648d8bb2720 2024-05-10T05:35:22.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.706+0000 7f411f57cb80 10 client.25793 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:22.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.706+0000 7f411f57cb80 10 client.25793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:35:22.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.706+0000 7f411f57cb80 20 client.25793 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:22.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.706+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648d8bb3f40 0x5648d8c22230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:22.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.709+0000 7f411f57cb80 4 client.25793 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:35:22.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.709+0000 7f411f57cb80 1 -- 172.21.15.131:0/2416952244 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5648d8acaf40 con 0x5648d8bb3f40 2024-05-10T05:35:22.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.710+0000 7f4104c00700 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648d8bb3f40 0x5648d8c22230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.714+0000 7f4104c00700 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648d8bb3f40 0x5648d8c22230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8463af7147c2a32 server_cookie=3dd9c2404f5d406 in_seq=0 out_seq=0 2024-05-10T05:35:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.714+0000 7f40e6000700 10 client.25793 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.717+0000 7f40e6000700 1 -- 172.21.15.131:0/2416952244 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5648d8acaf40 con 0x5648d8bb3f40 2024-05-10T05:35:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.717+0000 7f40e6000700 10 client.25793 handle_command_reply: tid=0 2024-05-10T05:35:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.720+0000 7f411f57cb80 1 client.25793 shutdown 2024-05-10T05:35:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.720+0000 7f411f57cb80 1 -- 172.21.15.131:0/2416952244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 msgr2=0x5648d8bc2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.720+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bc2ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005100 tx=0x7f40ec0055e0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.721+0000 7f411f57cb80 1 -- 172.21.15.131:0/2416952244 shutdown_connections 2024-05-10T05:35:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.721+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648d8bb3f40 0x5648d8c22230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.722+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648d8b14ae0 0x5648d8bc2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.722+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648d8bb35f0 0x5648d8bc3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.722+0000 7f411f57cb80 1 --2- 172.21.15.131:0/2416952244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648d8bb2720 0x5648d8bc2ad0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.722+0000 7f411f57cb80 1 -- 172.21.15.131:0/2416952244 >> 172.21.15.131:0/2416952244 conn(0x5648d8ba94d0 msgr2=0x5648d8bb7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.722+0000 7f40e6000700 0 client.25793 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.724+0000 7f411f57cb80 1 -- 172.21.15.131:0/2416952244 shutdown_connections 2024-05-10T05:35:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.725+0000 7f411f57cb80 1 -- 172.21.15.131:0/2416952244 wait complete. 2024-05-10T05:35:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.726+0000 7f411f57cb80 20 client.25793 trim_cache size 0 max 16384 2024-05-10T05:35:22.727 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:35:22.727 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:35:22.727 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "2f3bce68-8ca4-4899-ad2e-739e39958edc", 2024-05-10T05:35:22.727 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:35:22.727 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:35:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.733+0000 7f4107600700 1 -- 172.21.15.131:0/3528824142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40e80a1240 msgr2=0x7f40e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.733+0000 7f4107600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40e80a1240 0x7f40e80a3750 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005fd0 tx=0x7f40ec005f60 comp rx=0 tx=0).stop 2024-05-10T05:35:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.733+0000 7f4107600700 1 -- 172.21.15.131:0/3528824142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4100109c50 msgr2=0x7f41001a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.733+0000 7f4107600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4100109c50 0x7f41001a5920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f40f801c7b0 tx=0x7f40f801ddd0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.743+0000 7f4107600700 1 -- 172.21.15.131:0/3528824142 shutdown_connections 2024-05-10T05:35:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.743+0000 7f4107600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40e80a1240 0x7f40e80a3750 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.743+0000 7f4107600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4100109c50 0x7f41001a5920 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.743+0000 7f4107600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100108630 0x7f410019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.743+0000 7f4107600700 1 --2- 172.21.15.131:0/3528824142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4100109300 0x7f410019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:22.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.743+0000 7f4107600700 1 -- 172.21.15.131:0/3528824142 >> 172.21.15.131:0/3528824142 conn(0x7f410006ea00 msgr2=0x7f410010c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:22.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.743+0000 7f4107600700 1 -- 172.21.15.131:0/3528824142 shutdown_connections 2024-05-10T05:35:22.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:22.748+0000 7f4107600700 1 -- 172.21.15.131:0/3528824142 wait complete. 2024-05-10T05:35:22.833 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:35:23.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.551+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2482134962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3e40a6550 msgr2=0x7fc3e40a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:23.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.551+0000 7fc3ebe00700 1 --2- 172.21.15.131:0/2482134962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3e40a6550 0x7fc3e40a6960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80098f0 tx=0x7fc3d8009c00 comp rx=0 tx=0).stop 2024-05-10T05:35:23.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.555+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2482134962 shutdown_connections 2024-05-10T05:35:23.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.555+0000 7fc3ebe00700 1 --2- 172.21.15.131:0/2482134962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3e40a6ea0 0x7fc3e40ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.555+0000 7fc3ebe00700 1 --2- 172.21.15.131:0/2482134962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e40a5570 0x7fc3e40a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.555+0000 7fc3ebe00700 1 --2- 172.21.15.131:0/2482134962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3e40a6550 0x7fc3e40a6960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.555+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2482134962 >> 172.21.15.131:0/2482134962 conn(0x7fc3e40a0a00 msgr2=0x7fc3e40a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:23.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.557+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2482134962 shutdown_connections 2024-05-10T05:35:23.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.558+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2482134962 wait complete. 2024-05-10T05:35:23.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.559+0000 7fc3ebe00700 1 Processor -- start 2024-05-10T05:35:23.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.560+0000 7fc3ebe00700 1 -- start start 2024-05-10T05:35:23.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.560+0000 7fc3ebe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3e40a5570 0x7fc3e40b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.560+0000 7fc3ebe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e40a6550 0x7fc3e40b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.560+0000 7fc3ebe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3e40a6ea0 0x7fc3e4164dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.561+0000 7fc3ebe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4165390 con 0x7fc3e40a5570 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.561+0000 7fc3ebe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e41654d0 con 0x7fc3e40a6550 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.561+0000 7fc3ebe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4165660 con 0x7fc3e40a6ea0 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.561+0000 7fc3ea000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e40a6550 0x7fc3e40b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.561+0000 7fc3eb400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3e40a6ea0 0x7fc3e4164dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.561+0000 7fc3eb400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3e40a6ea0 0x7fc3e4164dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54552/0 (socket says 172.21.15.131:54552) 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.561+0000 7fc3eb400700 1 -- 172.21.15.131:0/2394641943 learned_addr learned my addr 172.21.15.131:0/2394641943 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.561+0000 7fc3eb400700 1 -- 172.21.15.131:0/2394641943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e40a6550 msgr2=0x7fc3e40b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.561+0000 7fc3eb400700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e40a6550 0x7fc3e40b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.562+0000 7fc3eb400700 1 -- 172.21.15.131:0/2394641943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3e40a5570 msgr2=0x7fc3e40b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.562+0000 7fc3eb400700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3e40a5570 0x7fc3e40b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.562+0000 7fc3eb400700 1 -- 172.21.15.131:0/2394641943 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8009520 con 0x7fc3e40a6ea0 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.568+0000 7fc3eb400700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3e40a6ea0 0x7fc3e4164dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec064650 tx=0x7fc3ec072bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:23.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.568+0000 7fc3f0a00700 1 -- 172.21.15.131:0/2394641943 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0709e0 con 0x7fc3e40a6ea0 2024-05-10T05:35:23.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.568+0000 7fc3f0a00700 1 -- 172.21.15.131:0/2394641943 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3ec074430 con 0x7fc3e40a6ea0 2024-05-10T05:35:23.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.568+0000 7fc3f0a00700 1 -- 172.21.15.131:0/2394641943 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0709e0 con 0x7fc3e40a6ea0 2024-05-10T05:35:23.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.570+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2394641943 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e4165940 con 0x7fc3e40a6ea0 2024-05-10T05:35:23.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.571+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2394641943 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e4165d70 con 0x7fc3e40a6ea0 2024-05-10T05:35:23.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.587+0000 7fc3f0a00700 1 -- 172.21.15.131:0/2394641943 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc3ec074830 con 0x7fc3e40a6ea0 2024-05-10T05:35:23.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.589+0000 7fc3f0a00700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc3d00a1280 0x7fc3d00a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.590+0000 7fc3f0a00700 1 -- 172.21.15.131:0/2394641943 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc3ec12e9c0 con 0x7fc3e40a6ea0 2024-05-10T05:35:23.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.593+0000 7fc409e0cb80 1 Processor -- start 2024-05-10T05:35:23.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.594+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc3d00a1280 0x7fc3d00a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.594+0000 7fc409e0cb80 1 -- start start 2024-05-10T05:35:23.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.596+0000 7fc409e0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94db960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.596+0000 7fc409e0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dc7a0 0x5638c94dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.596+0000 7fc409e0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dd0f0 0x5638c94e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.596+0000 7fc409e0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c9402150 con 0x5638c94db530 2024-05-10T05:35:23.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.596+0000 7fc409e0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c93ffb10 con 0x5638c94dd0f0 2024-05-10T05:35:23.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.596+0000 7fc409e0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c93fb400 con 0x5638c94dc7a0 2024-05-10T05:35:23.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.597+0000 7fc3eb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dd0f0 0x5638c94e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.598+0000 7fc3eb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dd0f0 0x5638c94e39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43038/0 (socket says 172.21.15.131:43038) 2024-05-10T05:35:23.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.598+0000 7fc3eb400700 1 -- 172.21.15.131:0/544460194 learned_addr learned my addr 172.21.15.131:0/544460194 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:23.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.599+0000 7fc3ea000700 1 --2- 172.21.15.131:0/544460194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dc7a0 0x5638c94dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.599+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/544460194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94db960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.600+0000 7fc3eb400700 1 -- 172.21.15.131:0/544460194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dc7a0 msgr2=0x5638c94dcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:23.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.600+0000 7fc3eb400700 1 --2- 172.21.15.131:0/544460194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dc7a0 0x5638c94dcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.600+0000 7fc3eb400700 1 -- 172.21.15.131:0/544460194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 msgr2=0x5638c94db960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:23.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.600+0000 7fc3eb400700 1 --2- 172.21.15.131:0/544460194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94db960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.600+0000 7fc3eb400700 1 -- 172.21.15.131:0/544460194 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638c927c3b0 con 0x5638c94dd0f0 2024-05-10T05:35:23.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.601+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/544460194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94db960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:23.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.601+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc3d00a1280 0x7fc3d00a3790 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc005fd0 tx=0x7fc3dc010650 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:23.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.602+0000 7fc3eb400700 1 --2- 172.21.15.131:0/544460194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dd0f0 0x5638c94e39f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0f8820 tx=0x7fc3ec072fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bb3fac51b5af25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:23.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.602+0000 7fc3d7e00700 1 -- 172.21.15.131:0/544460194 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0fc4b0 con 0x5638c94dd0f0 2024-05-10T05:35:23.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.603+0000 7fc3d7e00700 1 -- 172.21.15.131:0/544460194 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3ec0fbdb0 con 0x5638c94dd0f0 2024-05-10T05:35:23.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.603+0000 7fc3d7e00700 1 -- 172.21.15.131:0/544460194 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0fc210 con 0x5638c94dd0f0 2024-05-10T05:35:23.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.604+0000 7fc409e0cb80 1 -- 172.21.15.131:0/544460194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dd0f0 msgr2=0x5638c94e39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:23.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.604+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/544460194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dd0f0 0x5638c94e39f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0f8820 tx=0x7fc3ec072fd0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.605+0000 7fc409e0cb80 1 -- 172.21.15.131:0/544460194 shutdown_connections 2024-05-10T05:35:23.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.605+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/544460194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94db960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.605+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/544460194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dd0f0 0x5638c94e39f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.605+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/544460194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dc7a0 0x5638c94dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.605+0000 7fc409e0cb80 1 -- 172.21.15.131:0/544460194 >> 172.21.15.131:0/544460194 conn(0x5638c943da60 msgr2=0x5638c94d6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:23.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.605+0000 7fc409e0cb80 1 -- 172.21.15.131:0/544460194 shutdown_connections 2024-05-10T05:35:23.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.606+0000 7fc409e0cb80 1 -- 172.21.15.131:0/544460194 wait complete. 2024-05-10T05:35:23.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.606+0000 7fc409e0cb80 1 -- start start 2024-05-10T05:35:23.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.607+0000 7fc409e0cb80 1 Processor -- start 2024-05-10T05:35:23.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.608+0000 7fc409e0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94db530 0x5638c94e5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.609+0000 7fc409e0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 0x5638c94e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.610+0000 7fc409e0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dd0f0 0x5638c94ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.610+0000 7fc409e0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c9397560 con 0x5638c94dd0f0 2024-05-10T05:35:23.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.610+0000 7fc409e0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c9397200 con 0x5638c94dc7a0 2024-05-10T05:35:23.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.610+0000 7fc409e0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c927e6a0 con 0x5638c94db530 2024-05-10T05:35:23.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.610+0000 7fc409e0cb80 10 client.0 _ll_register_callbacks cb 0x5638c93a0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:23.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.611+0000 7fc3ea000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 0x5638c94e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.609+0000 7fc3eaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94db530 0x5638c94e5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.611+0000 7fc3eaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94db530 0x5638c94e5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54564/0 (socket says 172.21.15.131:54564) 2024-05-10T05:35:23.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.611+0000 7fc3eaa00700 1 -- 172.21.15.131:0/1944560349 learned_addr learned my addr 172.21.15.131:0/1944560349 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:23.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.611+0000 7fc3eaa00700 1 -- 172.21.15.131:0/1944560349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 msgr2=0x5638c94e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:23.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.611+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 0x5638c94e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.611+0000 7fc3eaa00700 1 -- 172.21.15.131:0/1944560349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dd0f0 msgr2=0x5638c94ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:23.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.611+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dd0f0 0x5638c94ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.611+0000 7fc3eaa00700 1 -- 172.21.15.131:0/1944560349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e4165d70 con 0x5638c94db530 2024-05-10T05:35:23.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.613+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94db530 0x5638c94e5640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc012aa0 tx=0x7fc3dc01a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b72773342179642 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:23.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.613+0000 7fc3d4c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:35:23.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.614+0000 7fc3d4c00700 1 -- 172.21.15.131:0/1944560349 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc00ec00 con 0x5638c94db530 2024-05-10T05:35:23.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.614+0000 7fc3d4c00700 1 -- 172.21.15.131:0/1944560349 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc3dc00ed60 con 0x5638c94db530 2024-05-10T05:35:23.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.614+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1944560349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638c9354270 con 0x5638c94db530 2024-05-10T05:35:23.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.614+0000 7fc3d4c00700 1 -- 172.21.15.131:0/1944560349 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3dc012e60 con 0x5638c94db530 2024-05-10T05:35:23.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.614+0000 7fc409e0cb80 10 client.25825 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:23.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.614+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1944560349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5638c9228b60 con 0x5638c94db530 2024-05-10T05:35:23.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.615+0000 7fc3d4c00700 1 -- 172.21.15.131:0/1944560349 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638c9354270 con 0x5638c94db530 2024-05-10T05:35:23.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.615+0000 7fc3d4c00700 1 -- 172.21.15.131:0/1944560349 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc3dc01e470 con 0x5638c94db530 2024-05-10T05:35:23.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.617+0000 7fc409e0cb80 10 client.25825 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:23.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.617+0000 7fc409e0cb80 10 client.25825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:35:23.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.617+0000 7fc409e0cb80 20 client.25825 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:23.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.617+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5638c94f12b0 0x5638c94f3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.617+0000 7fc409e0cb80 4 client.25825 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:23.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.617+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1944560349 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638c94aa780 con 0x5638c94f12b0 2024-05-10T05:35:23.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.618+0000 7fc3ea000700 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5638c94f12b0 0x5638c94f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.619+0000 7fc3ea000700 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5638c94f12b0 0x5638c94f3700 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f9cf8c596cf876 server_cookie=fcc52211f16ee49e in_seq=0 out_seq=0 2024-05-10T05:35:23.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.619+0000 7fc3d4c00700 10 client.25825 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:23.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.620+0000 7fc3d4c00700 1 -- 172.21.15.131:0/1944560349 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638c94aa780 con 0x5638c94f12b0 2024-05-10T05:35:23.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.620+0000 7fc3d4c00700 10 client.25825 handle_command_reply: tid=0 2024-05-10T05:35:23.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.621+0000 7fc409e0cb80 1 client.25825 shutdown 2024-05-10T05:35:23.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.621+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1944560349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94db530 msgr2=0x5638c94e5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:23.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.621+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94db530 0x5638c94e5640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc012aa0 tx=0x7fc3dc01a320 comp rx=0 tx=0).stop 2024-05-10T05:35:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.622+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1944560349 shutdown_connections 2024-05-10T05:35:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.622+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5638c94f12b0 0x5638c94f3700 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.622+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dd0f0 0x5638c94ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.622+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94db530 0x5638c94e5640 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.622+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1944560349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 0x5638c94e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.622+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1944560349 >> 172.21.15.131:0/1944560349 conn(0x5638c943da60 msgr2=0x5638c94da060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:23.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.622+0000 7fc3d4c00700 0 client.25825 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:23.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.623+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1944560349 shutdown_connections 2024-05-10T05:35:23.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.626+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1944560349 wait complete. 2024-05-10T05:35:23.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.629+0000 7fc409e0cb80 20 client.25825 trim_cache size 0 max 16384 2024-05-10T05:35:23.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.707+0000 7fc409e0cb80 1 Processor -- start 2024-05-10T05:35:23.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.708+0000 7fc409e0cb80 1 -- start start 2024-05-10T05:35:23.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.709+0000 7fc409e0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94d62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.710+0000 7fc3eaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94d62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.710+0000 7fc3eaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94d62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36742/0 (socket says 172.21.15.131:36742) 2024-05-10T05:35:23.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.710+0000 7fc3eaa00700 1 -- 172.21.15.131:0/1647152797 learned_addr learned my addr 172.21.15.131:0/1647152797 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:23.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.710+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 0x5638c94ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.711+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 0x5638c94da060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.711+0000 7fc3ea000700 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 0x5638c94ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.712+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1647152797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c94d7ba0 con 0x5638c94db530 2024-05-10T05:35:23.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.712+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1647152797 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c94d02f0 con 0x5638c94dc7a0 2024-05-10T05:35:23.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.712+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1647152797 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c94edbe0 con 0x5638c94dd0f0 2024-05-10T05:35:23.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.712+0000 7fc3eb400700 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 0x5638c94da060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.713+0000 7fc3eaa00700 1 -- 172.21.15.131:0/1647152797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 msgr2=0x5638c94da060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:23.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.713+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 0x5638c94da060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.713+0000 7fc3eaa00700 1 -- 172.21.15.131:0/1647152797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 msgr2=0x5638c94ecaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:23.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.713+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 0x5638c94ecaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.713+0000 7fc3eaa00700 1 -- 172.21.15.131:0/1647152797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638c93c1560 con 0x5638c94db530 2024-05-10T05:35:23.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.713+0000 7fc3ea000700 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 0x5638c94ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:23.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.713+0000 7fc3eb400700 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 0x5638c94da060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:23.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.714+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94d62c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc01e9d0 tx=0x7fc3dc010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdefa18726f33d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:23.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.714+0000 7fc3e1200700 1 -- 172.21.15.131:0/1647152797 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc012e60 con 0x5638c94db530 2024-05-10T05:35:23.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.715+0000 7fc3e1200700 1 -- 172.21.15.131:0/1647152797 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3dc019040 con 0x5638c94db530 2024-05-10T05:35:23.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.716+0000 7fc3e1200700 1 -- 172.21.15.131:0/1647152797 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc012e60 con 0x5638c94db530 2024-05-10T05:35:23.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.716+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1647152797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 msgr2=0x5638c94d62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:23.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.716+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94d62c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc01e9d0 tx=0x7fc3dc010fd0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.717+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1647152797 shutdown_connections 2024-05-10T05:35:23.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.717+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94db530 0x5638c94d62c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.717+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 0x5638c94da060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.717+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/1647152797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94dc7a0 0x5638c94ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.717+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1647152797 >> 172.21.15.131:0/1647152797 conn(0x5638c943da60 msgr2=0x5638c94d5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:23.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.718+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1647152797 shutdown_connections 2024-05-10T05:35:23.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.719+0000 7fc409e0cb80 1 -- 172.21.15.131:0/1647152797 wait complete. 2024-05-10T05:35:23.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.720+0000 7fc409e0cb80 1 -- start start 2024-05-10T05:35:23.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.720+0000 7fc409e0cb80 1 Processor -- start 2024-05-10T05:35:23.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.721+0000 7fc409e0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94db530 0x5638c94ec750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.721+0000 7fc409e0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dc7a0 0x5638c94eae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.721+0000 7fc409e0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 0x5638c94eb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.721+0000 7fc409e0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c94eb890 con 0x5638c94dc7a0 2024-05-10T05:35:23.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.721+0000 7fc409e0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c94eba00 con 0x5638c94db530 2024-05-10T05:35:23.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.721+0000 7fc409e0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c94ebbc0 con 0x5638c94dd0f0 2024-05-10T05:35:23.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.721+0000 7fc409e0cb80 10 client.0 _ll_register_callbacks cb 0x5638c93a0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:23.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.722+0000 7fc3ea000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dc7a0 0x5638c94eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.722+0000 7fc3ea000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dc7a0 0x5638c94eae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36744/0 (socket says 172.21.15.131:36744) 2024-05-10T05:35:23.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.722+0000 7fc3ea000700 1 -- 172.21.15.131:0/2418130665 learned_addr learned my addr 172.21.15.131:0/2418130665 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:23.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.723+0000 7fc3eb400700 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 0x5638c94eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.724+0000 7fc3ea000700 1 -- 172.21.15.131:0/2418130665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 msgr2=0x5638c94eb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:23.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.724+0000 7fc3ea000700 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 0x5638c94eb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.724+0000 7fc3ea000700 1 -- 172.21.15.131:0/2418130665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94db530 msgr2=0x5638c94ec750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:35:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.724+0000 7fc3ea000700 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94db530 0x5638c94ec750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.724+0000 7fc3ea000700 1 -- 172.21.15.131:0/2418130665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e40ac210 con 0x5638c94dc7a0 2024-05-10T05:35:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.724+0000 7fc3ea000700 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dc7a0 0x5638c94eae10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80098c0 tx=0x7fc3d80264c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f928e67a73c7209f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.724+0000 7fc3d4c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:35:23.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.725+0000 7fc3d4c00700 1 -- 172.21.15.131:0/2418130665 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d801d040 con 0x5638c94dc7a0 2024-05-10T05:35:23.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.725+0000 7fc3d4c00700 1 -- 172.21.15.131:0/2418130665 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc3d800b760 con 0x5638c94dc7a0 2024-05-10T05:35:23.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.725+0000 7fc409e0cb80 1 -- 172.21.15.131:0/2418130665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3ec0fbdb0 con 0x5638c94dc7a0 2024-05-10T05:35:23.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.726+0000 7fc3d4c00700 1 -- 172.21.15.131:0/2418130665 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3d800b920 con 0x5638c94dc7a0 2024-05-10T05:35:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.726+0000 7fc409e0cb80 10 client.15783 fetch_fsmap learned FSMap version 27 2024-05-10T05:35:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.726+0000 7fc409e0cb80 1 -- 172.21.15.131:0/2418130665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5638c9228b60 con 0x5638c94dc7a0 2024-05-10T05:35:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.726+0000 7fc3d4c00700 1 -- 172.21.15.131:0/2418130665 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3ec0fbdb0 con 0x5638c94dc7a0 2024-05-10T05:35:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.727+0000 7fc3d4c00700 1 -- 172.21.15.131:0/2418130665 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc3d8015020 con 0x5638c94dc7a0 2024-05-10T05:35:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.727+0000 7fc409e0cb80 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T05:35:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.727+0000 7fc409e0cb80 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:35:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.727+0000 7fc409e0cb80 20 client.15783 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:23.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.727+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5638c9548d00 0x5638c954b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:23.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.727+0000 7fc409e0cb80 4 client.15783 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:35:23.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.727+0000 7fc409e0cb80 1 -- 172.21.15.131:0/2418130665 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5638c93f3ec0 con 0x5638c9548d00 2024-05-10T05:35:23.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.727+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5638c9548d00 0x5638c954b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:23.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.728+0000 7fc3eaa00700 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5638c9548d00 0x5638c954b120 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ba7487db8854d0 server_cookie=52a48463934e8aa1 in_seq=0 out_seq=0 2024-05-10T05:35:23.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.728+0000 7fc3d4c00700 10 client.15783 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:23.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.729+0000 7fc3d4c00700 1 -- 172.21.15.131:0/2418130665 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5638c93f3ec0 con 0x5638c9548d00 2024-05-10T05:35:23.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.729+0000 7fc3d4c00700 10 client.15783 handle_command_reply: tid=0 2024-05-10T05:35:23.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.729+0000 7fc409e0cb80 1 client.15783 shutdown 2024-05-10T05:35:23.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.729+0000 7fc409e0cb80 1 -- 172.21.15.131:0/2418130665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dc7a0 msgr2=0x5638c94eae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:23.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.729+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dc7a0 0x5638c94eae10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80098c0 tx=0x7fc3d80264c0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.730+0000 7fc409e0cb80 1 -- 172.21.15.131:0/2418130665 shutdown_connections 2024-05-10T05:35:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.730+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5638c9548d00 0x5638c954b120 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.730+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638c94dd0f0 0x5638c94eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.730+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638c94db530 0x5638c94ec750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.730+0000 7fc409e0cb80 1 --2- 172.21.15.131:0/2418130665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638c94dc7a0 0x5638c94eae10 secure :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80098c0 tx=0x7fc3d80264c0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.730+0000 7fc409e0cb80 1 -- 172.21.15.131:0/2418130665 >> 172.21.15.131:0/2418130665 conn(0x5638c943da60 msgr2=0x5638c94cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.730+0000 7fc3d4c00700 0 client.15783 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.730+0000 7fc409e0cb80 1 -- 172.21.15.131:0/2418130665 shutdown_connections 2024-05-10T05:35:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.731+0000 7fc409e0cb80 1 -- 172.21.15.131:0/2418130665 wait complete. 2024-05-10T05:35:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.732+0000 7fc409e0cb80 20 client.15783 trim_cache size 0 max 16384 2024-05-10T05:35:23.733 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:35:23.733 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:35:23.733 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:35:23.733 INFO:teuthology.orchestra.run.smithi131.stdout: "2f3bce68-8ca4-4899-ad2e-739e39958edc": { 2024-05-10T05:35:23.733 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:35:23.733 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "2f3bce68-8ca4-4899-ad2e-739e39958edc", 2024-05-10T05:35:23.733 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:35:23.733 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:35:23.733 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:35:23.733 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:35:23.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.734+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2394641943 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc3d00a1280 msgr2=0x7fc3d00a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:23.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.734+0000 7fc3ebe00700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc3d00a1280 0x7fc3d00a3790 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc005fd0 tx=0x7fc3dc010650 comp rx=0 tx=0).stop 2024-05-10T05:35:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.734+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2394641943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3e40a6ea0 msgr2=0x7fc3e4164dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.734+0000 7fc3ebe00700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3e40a6ea0 0x7fc3e4164dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec064650 tx=0x7fc3ec072bd0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.734+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2394641943 shutdown_connections 2024-05-10T05:35:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.734+0000 7fc3ebe00700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc3d00a1280 0x7fc3d00a3790 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.734+0000 7fc3ebe00700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3e40a5570 0x7fc3e40b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.734+0000 7fc3ebe00700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3e40a6ea0 0x7fc3e4164dc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.734+0000 7fc3ebe00700 1 --2- 172.21.15.131:0/2394641943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3e40a6550 0x7fc3e40b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:23.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.734+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2394641943 >> 172.21.15.131:0/2394641943 conn(0x7fc3e40a0a00 msgr2=0x7fc3e40a97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:23.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.735+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2394641943 shutdown_connections 2024-05-10T05:35:23.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:23.735+0000 7fc3ebe00700 1 -- 172.21.15.131:0/2394641943 wait complete. 2024-05-10T05:35:23.795 INFO:tasks.cephfs.filesystem:scrub status for tag:2f3bce68-8ca4-4899-ad2e-739e39958edc - {'path': '/', 'tag': '2f3bce68-8ca4-4899-ad2e-739e39958edc', 'options': 'recursive,force'} 2024-05-10T05:35:24.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:24 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:35:24.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:24 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:35:24.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:24 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:35:24.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:24 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:35:24.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:24 smithi142 ceph-mon[77667]: pgmap v346: 97 pgs: 97 active+clean; 2.9 GiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 220 MiB/s wr, 72 op/s 2024-05-10T05:35:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:24 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:35:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:24 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:35:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:24 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:35:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:24 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:35:24.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:24 smithi131 ceph-mon[37264]: pgmap v346: 97 pgs: 97 active+clean; 2.9 GiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 220 MiB/s wr, 72 op/s 2024-05-10T05:35:24.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:24 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:35:24.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:24 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:35:24.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:24 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:35:24.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:24 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:35:24.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:24 smithi157 ceph-mon[78427]: pgmap v346: 97 pgs: 97 active+clean; 2.9 GiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 220 MiB/s wr, 72 op/s 2024-05-10T05:35:26.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:26 smithi142 ceph-mon[77667]: pgmap v347: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 244 MiB/s wr, 73 op/s 2024-05-10T05:35:26.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:26 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:35:26.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:26 smithi131 ceph-mon[37264]: pgmap v347: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 244 MiB/s wr, 73 op/s 2024-05-10T05:35:26.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:26 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:35:26.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:26 smithi157 ceph-mon[78427]: pgmap v347: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 244 MiB/s wr, 73 op/s 2024-05-10T05:35:26.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:26 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:35:27.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:27 smithi142 ceph-mon[77667]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:35:27.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:27 smithi142 ceph-mon[77667]: pgmap v348: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 MiB/s wr, 70 op/s 2024-05-10T05:35:27.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:27 smithi131 ceph-mon[37264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:35:27.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:27 smithi131 ceph-mon[37264]: pgmap v348: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 MiB/s wr, 70 op/s 2024-05-10T05:35:27.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:27 smithi157 ceph-mon[78427]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:35:27.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:27 smithi157 ceph-mon[78427]: pgmap v348: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 MiB/s wr, 70 op/s 2024-05-10T05:35:30.306 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:29 smithi157 ceph-mon[78427]: pgmap v349: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 227 MiB/s wr, 67 op/s 2024-05-10T05:35:30.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:29 smithi142 ceph-mon[77667]: pgmap v349: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 227 MiB/s wr, 67 op/s 2024-05-10T05:35:30.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:29 smithi131 ceph-mon[37264]: pgmap v349: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 227 MiB/s wr, 67 op/s 2024-05-10T05:35:32.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:31 smithi157 ceph-mon[78427]: pgmap v350: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 69 op/s 2024-05-10T05:35:32.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:31 smithi142 ceph-mon[77667]: pgmap v350: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 69 op/s 2024-05-10T05:35:32.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:31 smithi131 ceph-mon[37264]: pgmap v350: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 69 op/s 2024-05-10T05:35:34.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:34 smithi157 ceph-mon[78427]: pgmap v351: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:35:34.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:34 smithi142 ceph-mon[77667]: pgmap v351: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:35:34.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:34 smithi131 ceph-mon[37264]: pgmap v351: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:35:36.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:36 smithi157 ceph-mon[78427]: pgmap v352: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 269 MiB/s wr, 78 op/s 2024-05-10T05:35:36.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:36 smithi142 ceph-mon[77667]: pgmap v352: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 269 MiB/s wr, 78 op/s 2024-05-10T05:35:36.473 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:36 smithi131 ceph-mon[37264]: pgmap v352: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 269 MiB/s wr, 78 op/s 2024-05-10T05:35:38.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:38 smithi157 ceph-mon[78427]: pgmap v353: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 255 MiB/s wr, 76 op/s 2024-05-10T05:35:38.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:38 smithi142 ceph-mon[77667]: pgmap v353: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 255 MiB/s wr, 76 op/s 2024-05-10T05:35:38.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:38 smithi131 ceph-mon[37264]: pgmap v353: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 255 MiB/s wr, 76 op/s 2024-05-10T05:35:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:40 smithi142 ceph-mon[77667]: pgmap v354: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 271 MiB/s wr, 79 op/s 2024-05-10T05:35:40.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:40 smithi131 ceph-mon[37264]: pgmap v354: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 271 MiB/s wr, 79 op/s 2024-05-10T05:35:40.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:40 smithi157 ceph-mon[78427]: pgmap v354: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 271 MiB/s wr, 79 op/s 2024-05-10T05:35:42.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:42 smithi142 ceph-mon[77667]: pgmap v355: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 266 MiB/s wr, 77 op/s 2024-05-10T05:35:42.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:42 smithi131 ceph-mon[37264]: pgmap v355: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 266 MiB/s wr, 77 op/s 2024-05-10T05:35:42.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:42 smithi157 ceph-mon[78427]: pgmap v355: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 266 MiB/s wr, 77 op/s 2024-05-10T05:35:44.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:44 smithi131 ceph-mon[37264]: pgmap v356: 97 pgs: 97 active+clean; 7.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 257 MiB/s wr, 76 op/s 2024-05-10T05:35:44.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:44 smithi157 ceph-mon[78427]: pgmap v356: 97 pgs: 97 active+clean; 7.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 257 MiB/s wr, 76 op/s 2024-05-10T05:35:44.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:44 smithi142 ceph-mon[77667]: pgmap v356: 97 pgs: 97 active+clean; 7.9 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 257 MiB/s wr, 76 op/s 2024-05-10T05:35:45.472 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:45 smithi131 ceph-mon[37264]: pgmap v357: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 280 MiB/s wr, 80 op/s 2024-05-10T05:35:45.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:45 smithi157 ceph-mon[78427]: pgmap v357: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 280 MiB/s wr, 80 op/s 2024-05-10T05:35:45.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:45 smithi142 ceph-mon[77667]: pgmap v357: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 280 MiB/s wr, 80 op/s 2024-05-10T05:35:48.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:48 smithi157 ceph-mon[78427]: pgmap v358: 97 pgs: 97 active+clean; 9.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 73 op/s 2024-05-10T05:35:48.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:48 smithi142 ceph-mon[77667]: pgmap v358: 97 pgs: 97 active+clean; 9.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 73 op/s 2024-05-10T05:35:48.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:48 smithi131 ceph-mon[37264]: pgmap v358: 97 pgs: 97 active+clean; 9.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 73 op/s 2024-05-10T05:35:50.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:50 smithi157 ceph-mon[78427]: pgmap v359: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 255 MiB/s wr, 74 op/s 2024-05-10T05:35:50.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:50 smithi142 ceph-mon[77667]: pgmap v359: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 255 MiB/s wr, 74 op/s 2024-05-10T05:35:50.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:50 smithi131 ceph-mon[37264]: pgmap v359: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 255 MiB/s wr, 74 op/s 2024-05-10T05:35:52.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:52 smithi157 ceph-mon[78427]: pgmap v360: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 237 MiB/s wr, 69 op/s 2024-05-10T05:35:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:52 smithi142 ceph-mon[77667]: pgmap v360: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 237 MiB/s wr, 69 op/s 2024-05-10T05:35:52.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:52 smithi131 ceph-mon[37264]: pgmap v360: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 237 MiB/s wr, 69 op/s 2024-05-10T05:35:53.796 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:35:54.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:54 smithi142 ceph-mon[77667]: pgmap v361: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 236 MiB/s wr, 71 op/s 2024-05-10T05:35:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:54 smithi131 ceph-mon[37264]: pgmap v361: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 236 MiB/s wr, 71 op/s 2024-05-10T05:35:54.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.492+0000 7f2d8b800700 1 -- 172.21.15.131:0/3251381268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d841092c0 msgr2=0x7f2d841096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:54.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.492+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3251381268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d841092c0 0x7f2d841096d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009990 tx=0x7f2d70009ca0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.493+0000 7f2d8b800700 1 -- 172.21.15.131:0/3251381268 shutdown_connections 2024-05-10T05:35:54.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.494+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3251381268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d84109c10 0x7f2d8410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.494+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3251381268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d84108500 0x7f2d84108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.494+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3251381268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d841092c0 0x7f2d841096d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.494+0000 7f2d8b800700 1 -- 172.21.15.131:0/3251381268 >> 172.21.15.131:0/3251381268 conn(0x7f2d8406e970 msgr2=0x7f2d84070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:54.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.494+0000 7f2d8b800700 1 -- 172.21.15.131:0/3251381268 shutdown_connections 2024-05-10T05:35:54.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.494+0000 7f2d8b800700 1 -- 172.21.15.131:0/3251381268 wait complete. 2024-05-10T05:35:54.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.497+0000 7f2d8b800700 1 Processor -- start 2024-05-10T05:35:54.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.497+0000 7f2d8b800700 1 -- start start 2024-05-10T05:35:54.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.504+0000 7f2d8b800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d84108500 0x7f2d841a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.505+0000 7f2d88e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d84108500 0x7f2d841a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.505+0000 7f2d88e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d84108500 0x7f2d841a6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41438/0 (socket says 172.21.15.131:41438) 2024-05-10T05:35:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.505+0000 7f2d88e00700 1 -- 172.21.15.131:0/3213388292 learned_addr learned my addr 172.21.15.131:0/3213388292 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.505+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d841092c0 0x7f2d841a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.505+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d84109c10 0x7f2d841ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.505+0000 7f2d83e00700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d841092c0 0x7f2d841a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.505+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841ae700 con 0x7f2d84108500 2024-05-10T05:35:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.506+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841ae840 con 0x7f2d841092c0 2024-05-10T05:35:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.506+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841aea00 con 0x7f2d84109c10 2024-05-10T05:35:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.506+0000 7f2d89800700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d84109c10 0x7f2d841ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.507+0000 7f2d83e00700 1 -- 172.21.15.131:0/3213388292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d84109c10 msgr2=0x7f2d841ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.507+0000 7f2d83e00700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d84109c10 0x7f2d841ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.507+0000 7f2d83e00700 1 -- 172.21.15.131:0/3213388292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d84108500 msgr2=0x7f2d841a6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.507+0000 7f2d83e00700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d84108500 0x7f2d841a6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.507+0000 7f2d83e00700 1 -- 172.21.15.131:0/3213388292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700095f0 con 0x7f2d841092c0 2024-05-10T05:35:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.507+0000 7f2d88e00700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d84108500 0x7f2d841a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.508+0000 7f2d83e00700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d841092c0 0x7f2d841a7240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000c00 tx=0x7f2d7000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.508+0000 7f2d81600700 1 -- 172.21.15.131:0/3213388292 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7001d070 con 0x7f2d841092c0 2024-05-10T05:35:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.509+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d841aec80 con 0x7f2d841092c0 2024-05-10T05:35:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.509+0000 7f2d81600700 1 -- 172.21.15.131:0/3213388292 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d700047d0 con 0x7f2d841092c0 2024-05-10T05:35:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.510+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d841af100 con 0x7f2d841092c0 2024-05-10T05:35:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.510+0000 7f2d81600700 1 -- 172.21.15.131:0/3213388292 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7001d070 con 0x7f2d841092c0 2024-05-10T05:35:54.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.520+0000 7f2d81600700 1 -- 172.21.15.131:0/3213388292 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2d700050b0 con 0x7f2d841092c0 2024-05-10T05:35:54.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.522+0000 7f2d81600700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2d680a12b0 0x7f2d680a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.523+0000 7f2d81600700 1 -- 172.21.15.131:0/3213388292 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2d700cd4b0 con 0x7f2d841092c0 2024-05-10T05:35:54.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.526+0000 7f2d88e00700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2d680a12b0 0x7f2d680a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.529+0000 7f2da37bab80 1 Processor -- start 2024-05-10T05:35:54.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.529+0000 7f2d88e00700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2d680a12b0 0x7f2d680a37c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2d841a7b90 tx=0x7f2d7400a400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:54.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.529+0000 7f2da37bab80 1 -- start start 2024-05-10T05:35:54.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.531+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e567946a0 0x558e56794af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.531+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 0x558e56795980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.531+0000 7f2d88e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e567946a0 0x558e56794af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.531+0000 7f2d88e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e567946a0 0x558e56794af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40940/0 (socket says 172.21.15.131:40940) 2024-05-10T05:35:54.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.531+0000 7f2d88e00700 1 -- 172.21.15.131:0/2973745429 learned_addr learned my addr 172.21.15.131:0/2973745429 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:54.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.531+0000 7f2da37bab80 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795ec0 0x558e5679c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.531+0000 7f2da37bab80 1 -- 172.21.15.131:0/2973745429 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e566bb150 con 0x558e56795570 2024-05-10T05:35:54.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.532+0000 7f2d89800700 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795ec0 0x558e5679c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.531+0000 7f2da37bab80 1 -- 172.21.15.131:0/2973745429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e566b8b10 con 0x558e56795ec0 2024-05-10T05:35:54.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.532+0000 7f2da37bab80 1 -- 172.21.15.131:0/2973745429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e566b4400 con 0x558e567946a0 2024-05-10T05:35:54.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.533+0000 7f2d83e00700 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 0x558e56795980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.534+0000 7f2d89800700 1 -- 172.21.15.131:0/2973745429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e567946a0 msgr2=0x558e56794af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:54.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.534+0000 7f2d89800700 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e567946a0 0x558e56794af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.534+0000 7f2d89800700 1 -- 172.21.15.131:0/2973745429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 msgr2=0x558e56795980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:54.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.534+0000 7f2d89800700 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 0x558e56795980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.534+0000 7f2d89800700 1 -- 172.21.15.131:0/2973745429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e565353b0 con 0x558e56795ec0 2024-05-10T05:35:54.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.534+0000 7f2d83e00700 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 0x558e56795980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:54.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.535+0000 7f2d89800700 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795ec0 0x558e5679c7c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c003ef0 tx=0x7f2d7c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14c3ffa34c78e6b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:54.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.535+0000 7f2d7a000700 1 -- 172.21.15.131:0/2973745429 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c019070 con 0x558e56795ec0 2024-05-10T05:35:54.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.535+0000 7f2d7a000700 1 -- 172.21.15.131:0/2973745429 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d7c00f040 con 0x558e56795ec0 2024-05-10T05:35:54.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.535+0000 7f2d7a000700 1 -- 172.21.15.131:0/2973745429 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c007a50 con 0x558e56795ec0 2024-05-10T05:35:54.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.536+0000 7f2da37bab80 1 -- 172.21.15.131:0/2973745429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795ec0 msgr2=0x558e5679c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:54.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.536+0000 7f2da37bab80 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795ec0 0x558e5679c7c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c003ef0 tx=0x7f2d7c009ba0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.537+0000 7f2da37bab80 1 -- 172.21.15.131:0/2973745429 shutdown_connections 2024-05-10T05:35:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.537+0000 7f2da37bab80 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795ec0 0x558e5679c7c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.537+0000 7f2da37bab80 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e567946a0 0x558e56794af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.537+0000 7f2da37bab80 1 --2- 172.21.15.131:0/2973745429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 0x558e56795980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.537+0000 7f2da37bab80 1 -- 172.21.15.131:0/2973745429 >> 172.21.15.131:0/2973745429 conn(0x558e566f6a60 msgr2=0x558e5678efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.537+0000 7f2da37bab80 1 -- 172.21.15.131:0/2973745429 shutdown_connections 2024-05-10T05:35:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.537+0000 7f2da37bab80 1 -- 172.21.15.131:0/2973745429 wait complete. 2024-05-10T05:35:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.538+0000 7f2da37bab80 1 -- start start 2024-05-10T05:35:54.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.539+0000 7f2da37bab80 1 Processor -- start 2024-05-10T05:35:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.542+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e5679e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.542+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e56795570 0x558e5679ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.542+0000 7f2d88e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e5679e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.542+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795ec0 0x558e567a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.543+0000 7f2d88e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e5679e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45076/0 (socket says 172.21.15.131:45076) 2024-05-10T05:35:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.545+0000 7f2d83e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e56795570 0x558e5679ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.545+0000 7f2da37bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e56650560 con 0x558e56795ec0 2024-05-10T05:35:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.545+0000 7f2da37bab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e56650200 con 0x558e567946a0 2024-05-10T05:35:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.545+0000 7f2da37bab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e565376a0 con 0x558e56795570 2024-05-10T05:35:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.545+0000 7f2da37bab80 10 client.0 _ll_register_callbacks cb 0x558e56659b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.545+0000 7f2d88e00700 1 -- 172.21.15.131:0/209540917 learned_addr learned my addr 172.21.15.131:0/209540917 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:54.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.546+0000 7f2d89800700 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795ec0 0x558e567a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.547+0000 7f2d89800700 1 -- 172.21.15.131:0/209540917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e56795570 msgr2=0x558e5679ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:54.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.547+0000 7f2d89800700 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e56795570 0x558e5679ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.547+0000 7f2d89800700 1 -- 172.21.15.131:0/209540917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 msgr2=0x558e5679e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:54.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.547+0000 7f2d89800700 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e5679e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.547+0000 7f2d89800700 1 -- 172.21.15.131:0/209540917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c003b50 con 0x558e56795ec0 2024-05-10T05:35:54.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.548+0000 7f2d89800700 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795ec0 0x558e567a5ba0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c003af0 tx=0x7f2d7c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3355cb35be00f86 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:54.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.548+0000 7f2d6ea00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:35:54.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.549+0000 7f2d6ea00700 1 -- 172.21.15.131:0/209540917 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c019040 con 0x558e56795ec0 2024-05-10T05:35:54.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.549+0000 7f2da37bab80 1 -- 172.21.15.131:0/209540917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e5660d270 con 0x558e56795ec0 2024-05-10T05:35:54.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.549+0000 7f2d6ea00700 1 -- 172.21.15.131:0/209540917 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2d7c007d50 con 0x558e56795ec0 2024-05-10T05:35:54.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.550+0000 7f2d6ea00700 1 -- 172.21.15.131:0/209540917 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d7c015480 con 0x558e56795ec0 2024-05-10T05:35:54.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.552+0000 7f2da37bab80 10 client.15795 fetch_fsmap learned FSMap version 28 2024-05-10T05:35:54.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.552+0000 7f2da37bab80 1 -- 172.21.15.131:0/209540917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558e564e1b60 con 0x558e56795ec0 2024-05-10T05:35:54.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.552+0000 7f2d6ea00700 1 -- 172.21.15.131:0/209540917 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e5660d270 con 0x558e56795ec0 2024-05-10T05:35:54.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:54 smithi157 ceph-mon[78427]: pgmap v361: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 236 MiB/s wr, 71 op/s 2024-05-10T05:35:54.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.554+0000 7f2d6ea00700 1 -- 172.21.15.131:0/209540917 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2d7c0155e0 con 0x558e56795ec0 2024-05-10T05:35:54.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.554+0000 7f2da37bab80 10 client.15795 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:35:54.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.554+0000 7f2da37bab80 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:35:54.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.554+0000 7f2da37bab80 20 client.15795 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:54.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.554+0000 7f2da37bab80 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558e567aa0b0 0x558e567ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.555+0000 7f2da37bab80 4 client.15795 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:54.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.555+0000 7f2da37bab80 1 -- 172.21.15.131:0/209540917 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e56763780 con 0x558e567aa0b0 2024-05-10T05:35:54.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.555+0000 7f2d88e00700 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558e567aa0b0 0x558e567ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.558+0000 7f2d6ea00700 10 client.15795 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:54.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.558+0000 7f2d88e00700 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558e567aa0b0 0x558e567ac500 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204cfb0bbab81841 server_cookie=8968e61fe6173dbe in_seq=0 out_seq=0 2024-05-10T05:35:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.559+0000 7f2d6ea00700 1 -- 172.21.15.131:0/209540917 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e56763780 con 0x558e567aa0b0 2024-05-10T05:35:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.559+0000 7f2d6ea00700 10 client.15795 handle_command_reply: tid=0 2024-05-10T05:35:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.559+0000 7f2da37bab80 1 client.15795 shutdown 2024-05-10T05:35:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.560+0000 7f2da37bab80 1 -- 172.21.15.131:0/209540917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795ec0 msgr2=0x558e567a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.560+0000 7f2da37bab80 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795ec0 0x558e567a5ba0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c003af0 tx=0x7f2d7c007750 comp rx=0 tx=0).stop 2024-05-10T05:35:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.561+0000 7f2da37bab80 1 -- 172.21.15.131:0/209540917 shutdown_connections 2024-05-10T05:35:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.561+0000 7f2da37bab80 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558e567aa0b0 0x558e567ac500 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.561+0000 7f2da37bab80 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795ec0 0x558e567a5ba0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.561+0000 7f2d6ea00700 0 client.15795 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:54.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.561+0000 7f2da37bab80 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e5679e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.561+0000 7f2da37bab80 1 --2- 172.21.15.131:0/209540917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e56795570 0x558e5679ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.561+0000 7f2da37bab80 1 -- 172.21.15.131:0/209540917 >> 172.21.15.131:0/209540917 conn(0x558e566f6a60 msgr2=0x558e5678efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:54.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.561+0000 7f2da37bab80 1 -- 172.21.15.131:0/209540917 shutdown_connections 2024-05-10T05:35:54.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.561+0000 7f2da37bab80 1 -- 172.21.15.131:0/209540917 wait complete. 2024-05-10T05:35:54.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.562+0000 7f2da37bab80 20 client.15795 trim_cache size 0 max 16384 2024-05-10T05:35:54.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.604+0000 7f2da37bab80 1 Processor -- start 2024-05-10T05:35:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.604+0000 7f2da37bab80 1 -- start start 2024-05-10T05:35:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.605+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e566f6a60 0x558e56799350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.605+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e567946a0 0x558e567a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.605+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795570 0x558e56799890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.605+0000 7f2da37bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e567897c0 con 0x558e567946a0 2024-05-10T05:35:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.605+0000 7f2da37bab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e567901e0 con 0x558e56795570 2024-05-10T05:35:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.605+0000 7f2da37bab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e56794f50 con 0x558e566f6a60 2024-05-10T05:35:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.606+0000 7f2d89800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795570 0x558e56799890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.606+0000 7f2d89800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795570 0x558e56799890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45086/0 (socket says 172.21.15.131:45086) 2024-05-10T05:35:54.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.606+0000 7f2d89800700 1 -- 172.21.15.131:0/1178815118 learned_addr learned my addr 172.21.15.131:0/1178815118 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:54.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.608+0000 7f2d89800700 1 -- 172.21.15.131:0/1178815118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e566f6a60 msgr2=0x558e56799350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:54.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.608+0000 7f2d89800700 1 --2- 172.21.15.131:0/1178815118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e566f6a60 0x558e56799350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.608+0000 7f2d89800700 1 -- 172.21.15.131:0/1178815118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e567946a0 msgr2=0x558e567a5b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:54.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.608+0000 7f2d89800700 1 --2- 172.21.15.131:0/1178815118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e567946a0 0x558e567a5b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.608+0000 7f2d89800700 1 -- 172.21.15.131:0/1178815118 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5667a560 con 0x558e56795570 2024-05-10T05:35:54.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.610+0000 7f2d89800700 1 --2- 172.21.15.131:0/1178815118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795570 0x558e56799890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c000f60 tx=0x7f2d7c01f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b3788ea891f59eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:54.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.612+0000 7f2d7aa00700 1 -- 172.21.15.131:0/1178815118 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c019070 con 0x558e56795570 2024-05-10T05:35:54.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.612+0000 7f2d7aa00700 1 -- 172.21.15.131:0/1178815118 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e5660d270 con 0x558e56795570 2024-05-10T05:35:54.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.612+0000 7f2d7aa00700 1 -- 172.21.15.131:0/1178815118 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c019070 con 0x558e56795570 2024-05-10T05:35:54.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.614+0000 7f2da37bab80 1 -- 172.21.15.131:0/1178815118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795570 msgr2=0x558e56799890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:54.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.614+0000 7f2da37bab80 1 --2- 172.21.15.131:0/1178815118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795570 0x558e56799890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c000f60 tx=0x7f2d7c01f4c0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.615+0000 7f2da37bab80 1 -- 172.21.15.131:0/1178815118 shutdown_connections 2024-05-10T05:35:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.615+0000 7f2da37bab80 1 --2- 172.21.15.131:0/1178815118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e56795570 0x558e56799890 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.615+0000 7f2da37bab80 1 --2- 172.21.15.131:0/1178815118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e566f6a60 0x558e56799350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.615+0000 7f2da37bab80 1 --2- 172.21.15.131:0/1178815118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e567946a0 0x558e567a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.615+0000 7f2da37bab80 1 -- 172.21.15.131:0/1178815118 >> 172.21.15.131:0/1178815118 conn(0x558e5678b450 msgr2=0x558e5678fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.616+0000 7f2da37bab80 1 -- 172.21.15.131:0/1178815118 shutdown_connections 2024-05-10T05:35:54.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.617+0000 7f2da37bab80 1 -- 172.21.15.131:0/1178815118 wait complete. 2024-05-10T05:35:54.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.620+0000 7f2da37bab80 1 -- start start 2024-05-10T05:35:54.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.620+0000 7f2da37bab80 1 Processor -- start 2024-05-10T05:35:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.624+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e566f6a60 0x558e567a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.624+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e567a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.624+0000 7f2da37bab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 0x558e567a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.624+0000 7f2da37bab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e567a50f0 con 0x558e56795570 2024-05-10T05:35:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.624+0000 7f2da37bab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e567a5260 con 0x558e567946a0 2024-05-10T05:35:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.624+0000 7f2da37bab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e567a5420 con 0x558e566f6a60 2024-05-10T05:35:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.625+0000 7f2da37bab80 10 client.0 _ll_register_callbacks cb 0x558e56659b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.625+0000 7f2d83e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e567a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.625+0000 7f2d89800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 0x558e567a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.625+0000 7f2d83e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e567a3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45092/0 (socket says 172.21.15.131:45092) 2024-05-10T05:35:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.625+0000 7f2d83e00700 1 -- 172.21.15.131:0/3148837243 learned_addr learned my addr 172.21.15.131:0/3148837243 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:54.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.626+0000 7f2d83e00700 1 -- 172.21.15.131:0/3148837243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e566f6a60 msgr2=0x558e567a3790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:54.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.629+0000 7f2d83e00700 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e566f6a60 0x558e567a3790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.629+0000 7f2d83e00700 1 -- 172.21.15.131:0/3148837243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 msgr2=0x558e567a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:54.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.629+0000 7f2d83e00700 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 0x558e567a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.630+0000 7f2d83e00700 1 -- 172.21.15.131:0/3148837243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e564e1b60 con 0x558e567946a0 2024-05-10T05:35:54.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.630+0000 7f2d89800700 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 0x558e567a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.633+0000 7f2d83e00700 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e567a3cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2d70021040 tx=0x7f2d700cd330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b45aeeb6aa96cfa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.633+0000 7f2d6ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:35:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.636+0000 7f2d6ea00700 1 -- 172.21.15.131:0/3148837243 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7009a020 con 0x558e567946a0 2024-05-10T05:35:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.636+0000 7f2d6ea00700 1 -- 172.21.15.131:0/3148837243 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2d7009e760 con 0x558e567946a0 2024-05-10T05:35:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.636+0000 7f2da37bab80 1 -- 172.21.15.131:0/3148837243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d7c00f040 con 0x558e567946a0 2024-05-10T05:35:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.637+0000 7f2d6ea00700 1 -- 172.21.15.131:0/3148837243 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d700a2d20 con 0x558e567946a0 2024-05-10T05:35:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.637+0000 7f2da37bab80 10 client.25823 fetch_fsmap learned FSMap version 28 2024-05-10T05:35:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.637+0000 7f2da37bab80 1 -- 172.21.15.131:0/3148837243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558e56539cd0 con 0x558e567946a0 2024-05-10T05:35:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.637+0000 7f2d6ea00700 1 -- 172.21.15.131:0/3148837243 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d700b7080 con 0x558e567946a0 2024-05-10T05:35:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.643+0000 7f2d6ea00700 1 -- 172.21.15.131:0/3148837243 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2d700ba140 con 0x558e567946a0 2024-05-10T05:35:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.643+0000 7f2da37bab80 10 client.25823 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:35:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.643+0000 7f2da37bab80 10 client.25823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:35:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.643+0000 7f2da37bab80 20 client.25823 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.643+0000 7f2da37bab80 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558e56795ec0 0x558e56802f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.643+0000 7f2da37bab80 4 client.25823 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:35:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.643+0000 7f2da37bab80 1 -- 172.21.15.131:0/3148837243 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558e566acec0 con 0x558e56795ec0 2024-05-10T05:35:54.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.643+0000 7f2d88e00700 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558e56795ec0 0x558e56802f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:54.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.644+0000 7f2d6ea00700 10 client.25823 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:54.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.644+0000 7f2d88e00700 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558e56795ec0 0x558e56802f80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1844c98e3706893d server_cookie=2fbf4838af407e1b in_seq=0 out_seq=0 2024-05-10T05:35:54.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2d6ea00700 1 -- 172.21.15.131:0/3148837243 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558e566acec0 con 0x558e56795ec0 2024-05-10T05:35:54.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2d6ea00700 10 client.25823 handle_command_reply: tid=0 2024-05-10T05:35:54.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2da37bab80 1 client.25823 shutdown 2024-05-10T05:35:54.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2da37bab80 1 -- 172.21.15.131:0/3148837243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 msgr2=0x558e567a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2da37bab80 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e567a3cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2d70021040 tx=0x7f2d700cd330 comp rx=0 tx=0).stop 2024-05-10T05:35:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2da37bab80 1 -- 172.21.15.131:0/3148837243 shutdown_connections 2024-05-10T05:35:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2da37bab80 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558e56795ec0 0x558e56802f80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2da37bab80 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e56795570 0x558e567a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2da37bab80 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e566f6a60 0x558e567a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2da37bab80 1 --2- 172.21.15.131:0/3148837243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e567946a0 0x558e567a3cd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2da37bab80 1 -- 172.21.15.131:0/3148837243 >> 172.21.15.131:0/3148837243 conn(0x558e5678b450 msgr2=0x558e56799350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.645+0000 7f2d6ea00700 0 client.25823 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:35:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.646+0000 7f2da37bab80 1 -- 172.21.15.131:0/3148837243 shutdown_connections 2024-05-10T05:35:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.646+0000 7f2da37bab80 1 -- 172.21.15.131:0/3148837243 wait complete. 2024-05-10T05:35:54.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.646+0000 7f2da37bab80 20 client.25823 trim_cache size 0 max 16384 2024-05-10T05:35:54.647 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:35:54.647 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:35:54.647 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:35:54.647 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:35:54.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.647+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2d680a12b0 msgr2=0x7f2d680a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:54.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.647+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2d680a12b0 0x7f2d680a37c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2d841a7b90 tx=0x7f2d7400a400 comp rx=0 tx=0).stop 2024-05-10T05:35:54.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.647+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d841092c0 msgr2=0x7f2d841a7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:54.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.647+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d841092c0 0x7f2d841a7240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000c00 tx=0x7f2d7000f690 comp rx=0 tx=0).stop 2024-05-10T05:35:54.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.648+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 shutdown_connections 2024-05-10T05:35:54.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.648+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2d680a12b0 0x7f2d680a37c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.648+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d84108500 0x7f2d841a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.648+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d84109c10 0x7f2d841ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.648+0000 7f2d8b800700 1 --2- 172.21.15.131:0/3213388292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d841092c0 0x7f2d841a7240 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:54.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.648+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 >> 172.21.15.131:0/3213388292 conn(0x7f2d8406e970 msgr2=0x7f2d8406f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:54.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.648+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 shutdown_connections 2024-05-10T05:35:54.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:54.648+0000 7f2d8b800700 1 -- 172.21.15.131:0/3213388292 wait complete. 2024-05-10T05:35:54.726 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:35:54.726 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:35:55.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.353+0000 7fc107e00700 1 -- 172.21.15.131:0/2575573272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc108109b80 msgr2=0x7fc108077340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:55.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.353+0000 7fc107e00700 1 --2- 172.21.15.131:0/2575573272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc108109b80 0x7fc108077340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80098f0 tx=0x7fc0f8009c00 comp rx=0 tx=0).stop 2024-05-10T05:35:55.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.355+0000 7fc107e00700 1 -- 172.21.15.131:0/2575573272 shutdown_connections 2024-05-10T05:35:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.355+0000 7fc107e00700 1 --2- 172.21.15.131:0/2575573272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc108077880 0x7fc108079ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.355+0000 7fc107e00700 1 --2- 172.21.15.131:0/2575573272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108108aa0 0x7fc108109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.355+0000 7fc107e00700 1 --2- 172.21.15.131:0/2575573272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc108109b80 0x7fc108077340 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.355+0000 7fc107e00700 1 -- 172.21.15.131:0/2575573272 >> 172.21.15.131:0/2575573272 conn(0x7fc10806e820 msgr2=0x7fc108070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.359+0000 7fc107e00700 1 -- 172.21.15.131:0/2575573272 shutdown_connections 2024-05-10T05:35:55.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.359+0000 7fc107e00700 1 -- 172.21.15.131:0/2575573272 wait complete. 2024-05-10T05:35:55.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.368+0000 7fc107e00700 1 Processor -- start 2024-05-10T05:35:55.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.368+0000 7fc107e00700 1 -- start start 2024-05-10T05:35:55.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.369+0000 7fc107e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108077880 0x7fc1081b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:55.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.370+0000 7fc106a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108077880 0x7fc1081b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:55.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.371+0000 7fc106a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108077880 0x7fc1081b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45106/0 (socket says 172.21.15.131:45106) 2024-05-10T05:35:55.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.371+0000 7fc107e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc108108aa0 0x7fc1081b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:55.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.371+0000 7fc106a00700 1 -- 172.21.15.131:0/1486909356 learned_addr learned my addr 172.21.15.131:0/1486909356 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:55.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.372+0000 7fc107e00700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc108109b80 0x7fc1081b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:55.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.372+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081b91c0 con 0x7fc108109b80 2024-05-10T05:35:55.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.372+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081b9300 con 0x7fc108077880 2024-05-10T05:35:55.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.373+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081b9490 con 0x7fc108108aa0 2024-05-10T05:35:55.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.374+0000 7fc107400700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc108109b80 0x7fc1081b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:55.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.374+0000 7fc106000700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc108108aa0 0x7fc1081b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:55.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.375+0000 7fc106a00700 1 -- 172.21.15.131:0/1486909356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc108108aa0 msgr2=0x7fc1081b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:55.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.375+0000 7fc106a00700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc108108aa0 0x7fc1081b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.375+0000 7fc106a00700 1 -- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc108109b80 msgr2=0x7fc1081b8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:55.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.376+0000 7fc106a00700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc108109b80 0x7fc1081b8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.376+0000 7fc106a00700 1 -- 172.21.15.131:0/1486909356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f8009520 con 0x7fc108077880 2024-05-10T05:35:55.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.376+0000 7fc106000700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc108108aa0 0x7fc1081b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:55.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.376+0000 7fc107400700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc108109b80 0x7fc1081b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:55.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.377+0000 7fc106a00700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108077880 0x7fc1081b1720 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400dbe0 tx=0x7fc0f400def0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:55.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.378+0000 7fc0ff400700 1 -- 172.21.15.131:0/1486909356 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f400acc0 con 0x7fc108077880 2024-05-10T05:35:55.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.378+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1081b9770 con 0x7fc108077880 2024-05-10T05:35:55.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.379+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1081b9d50 con 0x7fc108077880 2024-05-10T05:35:55.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.381+0000 7fc0ff400700 1 -- 172.21.15.131:0/1486909356 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f400ae20 con 0x7fc108077880 2024-05-10T05:35:55.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.381+0000 7fc0ff400700 1 -- 172.21.15.131:0/1486909356 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f400acc0 con 0x7fc108077880 2024-05-10T05:35:55.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.387+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e8005320 con 0x7fc108077880 2024-05-10T05:35:55.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.387+0000 7fc0ff400700 1 -- 172.21.15.131:0/1486909356 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc0f400e450 con 0x7fc108077880 2024-05-10T05:35:55.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.390+0000 7fc0ff400700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc0ec0a1300 0x7fc0ec0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:55.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.394+0000 7fc0ff400700 1 -- 172.21.15.131:0/1486909356 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc0f4099da0 con 0x7fc108077880 2024-05-10T05:35:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.395+0000 7fc106000700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc0ec0a1300 0x7fc0ec0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.398+0000 7fc106000700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc0ec0a1300 0x7fc0ec0a3810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc1081b27c0 tx=0x7fc0f800bfa0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.402+0000 7fc0ff400700 1 -- 172.21.15.131:0/1486909356 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc0f4096170 con 0x7fc108077880 2024-05-10T05:35:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.975+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0e8005190 con 0x7fc108077880 2024-05-10T05:35:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.977+0000 7fc0ff400700 1 -- 172.21.15.131:0/1486909356 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10931 (secure 0 0 0) 0x7fc0f401b090 con 0x7fc108077880 2024-05-10T05:35:55.977 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:35:55.977 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:35:26.081961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:35:55.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.980+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc0ec0a1300 msgr2=0x7fc0ec0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:55.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.980+0000 7fc107e00700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc0ec0a1300 0x7fc0ec0a3810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc1081b27c0 tx=0x7fc0f800bfa0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.980+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108077880 msgr2=0x7fc1081b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.980+0000 7fc107e00700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108077880 0x7fc1081b1720 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400dbe0 tx=0x7fc0f400def0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.980+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 shutdown_connections 2024-05-10T05:35:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.980+0000 7fc107e00700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc0ec0a1300 0x7fc0ec0a3810 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.980+0000 7fc107e00700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc108109b80 0x7fc1081b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.980+0000 7fc107e00700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108077880 0x7fc1081b1720 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.980+0000 7fc107e00700 1 --2- 172.21.15.131:0/1486909356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc108108aa0 0x7fc1081b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.980+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 >> 172.21.15.131:0/1486909356 conn(0x7fc10806e820 msgr2=0x7fc1080754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.981+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 shutdown_connections 2024-05-10T05:35:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:55.981+0000 7fc107e00700 1 -- 172.21.15.131:0/1486909356 wait complete. 2024-05-10T05:35:55.982 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 28 2024-05-10T05:35:56.013 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:35:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:56 smithi142 ceph-mon[77667]: pgmap v362: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 80 op/s 2024-05-10T05:35:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:56 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1486909356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:35:56.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:56 smithi131 ceph-mon[37264]: pgmap v362: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 80 op/s 2024-05-10T05:35:56.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:56 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1486909356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:35:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.501+0000 7f3354800700 1 -- 172.21.15.131:0/1103082211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f334c108630 msgr2=0x7f334c108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.501+0000 7f3354800700 1 --2- 172.21.15.131:0/1103082211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f334c108630 0x7f334c108a40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f333c009990 tx=0x7f333c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.503+0000 7f3354800700 1 -- 172.21.15.131:0/1103082211 shutdown_connections 2024-05-10T05:35:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.503+0000 7f3354800700 1 --2- 172.21.15.131:0/1103082211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c109c50 0x7f334c10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.503+0000 7f3354800700 1 --2- 172.21.15.131:0/1103082211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f334c108630 0x7f334c108a40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.503+0000 7f3354800700 1 --2- 172.21.15.131:0/1103082211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c109300 0x7f334c109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.503+0000 7f3354800700 1 -- 172.21.15.131:0/1103082211 >> 172.21.15.131:0/1103082211 conn(0x7f334c06ea00 msgr2=0x7f334c070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.504+0000 7f3354800700 1 -- 172.21.15.131:0/1103082211 shutdown_connections 2024-05-10T05:35:56.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.504+0000 7f3354800700 1 -- 172.21.15.131:0/1103082211 wait complete. 2024-05-10T05:35:56.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.506+0000 7f3354800700 1 Processor -- start 2024-05-10T05:35:56.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.516+0000 7f3354800700 1 -- start start 2024-05-10T05:35:56.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.517+0000 7f3354800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f334c109300 0x7f334c1a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.517+0000 7f3354800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c109c50 0x7f334c1a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.517+0000 7f3354800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c1abfe0 0x7f334c1ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.517+0000 7f3354800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1aeb70 con 0x7f334c109c50 2024-05-10T05:35:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.517+0000 7f3354800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1aece0 con 0x7f334c1abfe0 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.517+0000 7f3354800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1aeea0 con 0x7f334c109300 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.518+0000 7f3352800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c1abfe0 0x7f334c1ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.518+0000 7f3352800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c1abfe0 0x7f334c1ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45120/0 (socket says 172.21.15.131:45120) 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.518+0000 7f3352800700 1 -- 172.21.15.131:0/4115937368 learned_addr learned my addr 172.21.15.131:0/4115937368 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.518+0000 7f3351400700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c109c50 0x7f334c1a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.518+0000 7f3352800700 1 -- 172.21.15.131:0/4115937368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f334c109300 msgr2=0x7f334c1a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.518+0000 7f3352800700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f334c109300 0x7f334c1a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.518+0000 7f3352800700 1 -- 172.21.15.131:0/4115937368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c109c50 msgr2=0x7f334c1a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.518+0000 7f3352800700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c109c50 0x7f334c1a71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.518+0000 7f3352800700 1 -- 172.21.15.131:0/4115937368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0095f0 con 0x7f334c1abfe0 2024-05-10T05:35:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.519+0000 7f3352800700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c1abfe0 0x7f334c1ae440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f334800ef30 tx=0x7f334800c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.519+0000 7f3346a00700 1 -- 172.21.15.131:0/4115937368 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348018070 con 0x7f334c1abfe0 2024-05-10T05:35:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.519+0000 7f3346a00700 1 -- 172.21.15.131:0/4115937368 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334800f040 con 0x7f334c1abfe0 2024-05-10T05:35:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.519+0000 7f3346a00700 1 -- 172.21.15.131:0/4115937368 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348003dc0 con 0x7f334c1abfe0 2024-05-10T05:35:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.519+0000 7f3354800700 1 -- 172.21.15.131:0/4115937368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c1af120 con 0x7f334c1abfe0 2024-05-10T05:35:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.520+0000 7f3354800700 1 -- 172.21.15.131:0/4115937368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c1af5e0 con 0x7f334c1abfe0 2024-05-10T05:35:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.523+0000 7f3346a00700 1 -- 172.21.15.131:0/4115937368 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f334800caa0 con 0x7f334c1abfe0 2024-05-10T05:35:56.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.524+0000 7f3346a00700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33340a1360 0x7f33340a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.525+0000 7f3346a00700 1 -- 172.21.15.131:0/4115937368 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f33480cc550 con 0x7f334c1abfe0 2024-05-10T05:35:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.525+0000 7f3351e00700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33340a1360 0x7f33340a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.526+0000 7f3351e00700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33340a1360 0x7f33340a3870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f333c00b2e0 tx=0x7f333c009400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:56.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.527+0000 7f336c7fdb80 1 Processor -- start 2024-05-10T05:35:56.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.528+0000 7f336c7fdb80 1 -- start start 2024-05-10T05:35:56.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.528+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 0x5626abf8baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.528+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8c570 0x5626abf8c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.528+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626abf937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.528+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626abeb2150 con 0x5626abf8c570 2024-05-10T05:35:56.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.528+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626abeafb10 con 0x5626abf8cec0 2024-05-10T05:35:56.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.528+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626abeab400 con 0x5626abf8b6a0 2024-05-10T05:35:56.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.529+0000 7f3351e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 0x5626abf8baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.530+0000 7f3351e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 0x5626abf8baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40990/0 (socket says 172.21.15.131:40990) 2024-05-10T05:35:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.530+0000 7f3351e00700 1 -- 172.21.15.131:0/3556881949 learned_addr learned my addr 172.21.15.131:0/3556881949 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.530+0000 7f3352800700 1 --2- 172.21.15.131:0/3556881949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626abf937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.530+0000 7f3351400700 1 --2- 172.21.15.131:0/3556881949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8c570 0x5626abf8c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.530+0000 7f3351e00700 1 -- 172.21.15.131:0/3556881949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 msgr2=0x5626abf937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.530+0000 7f3351e00700 1 --2- 172.21.15.131:0/3556881949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626abf937c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.530+0000 7f3351e00700 1 -- 172.21.15.131:0/3556881949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8c570 msgr2=0x5626abf8c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.530+0000 7f3351e00700 1 --2- 172.21.15.131:0/3556881949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8c570 0x5626abf8c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.530+0000 7f3351e00700 1 -- 172.21.15.131:0/3556881949 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626abd2c3b0 con 0x5626abf8b6a0 2024-05-10T05:35:56.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.531+0000 7f3351e00700 1 --2- 172.21.15.131:0/3556881949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 0x5626abf8baf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f333c009990 tx=0x7f333c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6caa82792a0f59af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:56.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.531+0000 7f333b000700 1 -- 172.21.15.131:0/3556881949 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c003680 con 0x5626abf8b6a0 2024-05-10T05:35:56.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.531+0000 7f333b000700 1 -- 172.21.15.131:0/3556881949 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f333c003ca0 con 0x5626abf8b6a0 2024-05-10T05:35:56.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.531+0000 7f333b000700 1 -- 172.21.15.131:0/3556881949 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c00fae0 con 0x5626abf8b6a0 2024-05-10T05:35:56.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.533+0000 7f336c7fdb80 1 -- 172.21.15.131:0/3556881949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 msgr2=0x5626abf8baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:56.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.533+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/3556881949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 0x5626abf8baf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f333c009990 tx=0x7f333c01a000 comp rx=0 tx=0).stop 2024-05-10T05:35:56.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.533+0000 7f336c7fdb80 1 -- 172.21.15.131:0/3556881949 shutdown_connections 2024-05-10T05:35:56.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.533+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/3556881949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626abf937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.533+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/3556881949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 0x5626abf8baf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.533+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/3556881949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8c570 0x5626abf8c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.533+0000 7f336c7fdb80 1 -- 172.21.15.131:0/3556881949 >> 172.21.15.131:0/3556881949 conn(0x5626abeeda60 msgr2=0x5626abf85fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:56.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.533+0000 7f336c7fdb80 1 -- 172.21.15.131:0/3556881949 shutdown_connections 2024-05-10T05:35:56.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.533+0000 7f336c7fdb80 1 -- 172.21.15.131:0/3556881949 wait complete. 2024-05-10T05:35:56.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.535+0000 7f336c7fdb80 1 -- start start 2024-05-10T05:35:56.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.536+0000 7f336c7fdb80 1 Processor -- start 2024-05-10T05:35:56.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.538+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf956b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.538+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 0x5626abf95bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.538+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626abf9cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.539+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626abe47560 con 0x5626abf8b6a0 2024-05-10T05:35:56.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.539+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626abe47200 con 0x5626abf8cec0 2024-05-10T05:35:56.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.539+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626abd2e6a0 con 0x5626abf8c570 2024-05-10T05:35:56.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.539+0000 7f336c7fdb80 10 client.0 _ll_register_callbacks cb 0x5626abe50b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:56.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.541+0000 7f3352800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626abf9cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.541+0000 7f3351400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 0x5626abf95bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.541+0000 7f3351e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.541+0000 7f3352800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626abf9cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45136/0 (socket says 172.21.15.131:45136) 2024-05-10T05:35:56.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.541+0000 7f3352800700 1 -- 172.21.15.131:0/1658252954 learned_addr learned my addr 172.21.15.131:0/1658252954 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:56.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.542+0000 7f3351400700 1 -- 172.21.15.131:0/1658252954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 msgr2=0x5626abf9cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:56.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.542+0000 7f3351400700 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626abf9cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.542+0000 7f3351400700 1 -- 172.21.15.131:0/1658252954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 msgr2=0x5626abf956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:56.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.542+0000 7f3351400700 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf956b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.542+0000 7f3351400700 1 -- 172.21.15.131:0/1658252954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c10efc0 con 0x5626abf8c570 2024-05-10T05:35:56.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.542+0000 7f3352800700 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626abf9cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:56.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.542+0000 7f3351e00700 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:56.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.548+0000 7f3351400700 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 0x5626abf95bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3340012700 tx=0x7f3340012a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49b869e92d822a91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:56.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.548+0000 7f332f400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:35:56.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.548+0000 7f332f400700 1 -- 172.21.15.131:0/1658252954 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340009690 con 0x5626abf8c570 2024-05-10T05:35:56.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.548+0000 7f332f400700 1 -- 172.21.15.131:0/1658252954 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3340004460 con 0x5626abf8c570 2024-05-10T05:35:56.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.548+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1658252954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626abe04270 con 0x5626abf8c570 2024-05-10T05:35:56.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.549+0000 7f332f400700 1 -- 172.21.15.131:0/1658252954 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3340003680 con 0x5626abf8c570 2024-05-10T05:35:56.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.549+0000 7f336c7fdb80 10 client.25852 fetch_fsmap learned FSMap version 28 2024-05-10T05:35:56.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.549+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1658252954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5626abcd8b60 con 0x5626abf8c570 2024-05-10T05:35:56.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.549+0000 7f332f400700 1 -- 172.21.15.131:0/1658252954 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626abe04270 con 0x5626abf8c570 2024-05-10T05:35:56.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.551+0000 7f332f400700 1 -- 172.21.15.131:0/1658252954 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3340025a00 con 0x5626abf8c570 2024-05-10T05:35:56.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.551+0000 7f336c7fdb80 10 client.25852 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:35:56.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.551+0000 7f336c7fdb80 10 client.25852 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:35:56.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.551+0000 7f336c7fdb80 20 client.25852 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:56.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.551+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5626abfb2340 0x5626abfb4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.551+0000 7f336c7fdb80 4 client.25852 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:56.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.551+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1658252954 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626abf5a780 con 0x5626abfb2340 2024-05-10T05:35:56.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.552+0000 7f3351e00700 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5626abfb2340 0x5626abfb4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:56 smithi157 ceph-mon[78427]: pgmap v362: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 80 op/s 2024-05-10T05:35:56.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:56 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1486909356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:35:56.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.553+0000 7f3351e00700 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5626abfb2340 0x5626abfb4790 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=410339bf717cb77e server_cookie=e4c3a7a7bf5ce9b5 in_seq=0 out_seq=0 2024-05-10T05:35:56.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.553+0000 7f332f400700 10 client.25852 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:35:56.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.554+0000 7f332f400700 1 -- 172.21.15.131:0/1658252954 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626abf5a780 con 0x5626abfb2340 2024-05-10T05:35:56.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.554+0000 7f332f400700 10 client.25852 handle_command_reply: tid=0 2024-05-10T05:35:56.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.554+0000 7f336c7fdb80 1 client.25852 shutdown 2024-05-10T05:35:56.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.559+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1658252954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 msgr2=0x5626abf95bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.559+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 0x5626abf95bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3340012700 tx=0x7f3340012a10 comp rx=0 tx=0).stop 2024-05-10T05:35:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.559+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1658252954 shutdown_connections 2024-05-10T05:35:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.559+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5626abfb2340 0x5626abfb4790 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.559+0000 7f332f400700 0 client.25852 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:35:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.559+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.559+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626abf9cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.559+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1658252954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 0x5626abf95bf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.559+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1658252954 >> 172.21.15.131:0/1658252954 conn(0x5626abeeda60 msgr2=0x5626abf85fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.560+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1658252954 shutdown_connections 2024-05-10T05:35:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.560+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1658252954 wait complete. 2024-05-10T05:35:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.561+0000 7f336c7fdb80 20 client.25852 trim_cache size 0 max 16384 2024-05-10T05:35:56.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.589+0000 7f336c7fdb80 1 Processor -- start 2024-05-10T05:35:56.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.590+0000 7f336c7fdb80 1 -- start start 2024-05-10T05:35:56.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.590+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 0x5626abf8c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.590+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8c570 0x5626abf9cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.590+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8cec0 0x5626abf94c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.590+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626abfaf870 con 0x5626abf8cec0 2024-05-10T05:35:56.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.590+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626abf96570 con 0x5626abf8c570 2024-05-10T05:35:56.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.590+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626abf96780 con 0x5626abf8b6a0 2024-05-10T05:35:56.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.591+0000 7f3351400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8c570 0x5626abf9cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.591+0000 7f3352800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8cec0 0x5626abf94c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.592+0000 7f3352800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8cec0 0x5626abf94c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41540/0 (socket says 172.21.15.131:41540) 2024-05-10T05:35:56.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.592+0000 7f3352800700 1 -- 172.21.15.131:0/20532324 learned_addr learned my addr 172.21.15.131:0/20532324 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:56.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.593+0000 7f3352800700 1 -- 172.21.15.131:0/20532324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 msgr2=0x5626abf8c980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:35:56.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.593+0000 7f3352800700 1 --2- 172.21.15.131:0/20532324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 0x5626abf8c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.593+0000 7f3352800700 1 -- 172.21.15.131:0/20532324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8c570 msgr2=0x5626abf9cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:56.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.593+0000 7f3352800700 1 --2- 172.21.15.131:0/20532324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8c570 0x5626abf9cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.593+0000 7f3352800700 1 -- 172.21.15.131:0/20532324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626abe71560 con 0x5626abf8cec0 2024-05-10T05:35:56.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.594+0000 7f3351400700 1 --2- 172.21.15.131:0/20532324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8c570 0x5626abf9cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:56.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.594+0000 7f3352800700 1 --2- 172.21.15.131:0/20532324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8cec0 0x5626abf94c20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f33480996d0 tx=0x7f3348095d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b728534194455e7f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:56.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.595+0000 7f333ba00700 1 -- 172.21.15.131:0/20532324 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33480999d0 con 0x5626abf8cec0 2024-05-10T05:35:56.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.595+0000 7f333ba00700 1 -- 172.21.15.131:0/20532324 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334809d800 con 0x5626abf8cec0 2024-05-10T05:35:56.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.596+0000 7f333ba00700 1 -- 172.21.15.131:0/20532324 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33480999d0 con 0x5626abf8cec0 2024-05-10T05:35:56.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.597+0000 7f336c7fdb80 1 -- 172.21.15.131:0/20532324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8cec0 msgr2=0x5626abf94c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:56.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.597+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/20532324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8cec0 0x5626abf94c20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f33480996d0 tx=0x7f3348095d10 comp rx=0 tx=0).stop 2024-05-10T05:35:56.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.598+0000 7f336c7fdb80 1 -- 172.21.15.131:0/20532324 shutdown_connections 2024-05-10T05:35:56.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.598+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/20532324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8cec0 0x5626abf94c20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.598+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/20532324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8b6a0 0x5626abf8c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.598+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/20532324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8c570 0x5626abf9cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.598+0000 7f336c7fdb80 1 -- 172.21.15.131:0/20532324 >> 172.21.15.131:0/20532324 conn(0x5626abeeda60 msgr2=0x5626abf82180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:56.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.599+0000 7f336c7fdb80 1 -- 172.21.15.131:0/20532324 shutdown_connections 2024-05-10T05:35:56.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.599+0000 7f336c7fdb80 1 -- 172.21.15.131:0/20532324 wait complete. 2024-05-10T05:35:56.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.600+0000 7f336c7fdb80 1 -- start start 2024-05-10T05:35:56.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.601+0000 7f336c7fdb80 1 Processor -- start 2024-05-10T05:35:56.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.602+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf99aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.602+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 0x5626abf99fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.602+0000 7f336c7fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626ac0067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.602+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626abf9a630 con 0x5626abf8b6a0 2024-05-10T05:35:56.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.602+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ac006d20 con 0x5626abf8cec0 2024-05-10T05:35:56.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.602+0000 7f336c7fdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ac006e60 con 0x5626abf8c570 2024-05-10T05:35:56.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.605+0000 7f336c7fdb80 10 client.0 _ll_register_callbacks cb 0x5626abe50b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:56.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.606+0000 7f3351e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf99aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.607+0000 7f3351400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 0x5626abf99fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.607+0000 7f3351e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf99aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41554/0 (socket says 172.21.15.131:41554) 2024-05-10T05:35:56.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.607+0000 7f3351e00700 1 -- 172.21.15.131:0/1308155767 learned_addr learned my addr 172.21.15.131:0/1308155767 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:56.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.607+0000 7f3352800700 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626ac0067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.608+0000 7f3351e00700 1 -- 172.21.15.131:0/1308155767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 msgr2=0x5626abf99fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:56.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.608+0000 7f3351e00700 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 0x5626abf99fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.608+0000 7f3351e00700 1 -- 172.21.15.131:0/1308155767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 msgr2=0x5626ac0067e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:56.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.608+0000 7f3351e00700 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626ac0067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.608+0000 7f3351e00700 1 -- 172.21.15.131:0/1308155767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c1af5e0 con 0x5626abf8b6a0 2024-05-10T05:35:56.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.608+0000 7f332f400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:35:56.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.608+0000 7f3351e00700 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf99aa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c010 tx=0x7f333c00b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75a8af6354e1b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:56.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.609+0000 7f332f400700 1 -- 172.21.15.131:0/1308155767 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c031040 con 0x5626abf8b6a0 2024-05-10T05:35:56.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.610+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1308155767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f333c003ca0 con 0x5626abf8b6a0 2024-05-10T05:35:56.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.610+0000 7f332f400700 1 -- 172.21.15.131:0/1308155767 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f333c003680 con 0x5626abf8b6a0 2024-05-10T05:35:56.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.611+0000 7f332f400700 1 -- 172.21.15.131:0/1308155767 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f333c031040 con 0x5626abf8b6a0 2024-05-10T05:35:56.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.611+0000 7f336c7fdb80 10 client.15819 fetch_fsmap learned FSMap version 28 2024-05-10T05:35:56.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.611+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1308155767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5626abd30cd0 con 0x5626abf8b6a0 2024-05-10T05:35:56.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.612+0000 7f332f400700 1 -- 172.21.15.131:0/1308155767 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f333c003ca0 con 0x5626abf8b6a0 2024-05-10T05:35:56.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.614+0000 7f332f400700 1 -- 172.21.15.131:0/1308155767 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f333c034070 con 0x5626abf8b6a0 2024-05-10T05:35:56.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.614+0000 7f336c7fdb80 10 client.15819 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:35:56.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.614+0000 7f336c7fdb80 10 client.15819 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:35:56.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.614+0000 7f336c7fdb80 20 client.15819 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:56.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.614+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5626ac0083f0 0x5626ac00a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:56.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.614+0000 7f336c7fdb80 4 client.15819 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:35:56.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.614+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1308155767 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626abea3ec0 con 0x5626ac0083f0 2024-05-10T05:35:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.615+0000 7f3351400700 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5626ac0083f0 0x5626ac00a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.615+0000 7f332f400700 10 client.15819 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:35:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.615+0000 7f3351400700 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5626ac0083f0 0x5626ac00a810 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f32cc4d033ab6158 server_cookie=807600c1848315b in_seq=0 out_seq=0 2024-05-10T05:35:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.616+0000 7f332f400700 1 -- 172.21.15.131:0/1308155767 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626abea3ec0 con 0x5626ac0083f0 2024-05-10T05:35:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.616+0000 7f332f400700 10 client.15819 handle_command_reply: tid=0 2024-05-10T05:35:56.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.617+0000 7f336c7fdb80 1 client.15819 shutdown 2024-05-10T05:35:56.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.617+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1308155767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 msgr2=0x5626abf99aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:56.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.617+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf99aa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c010 tx=0x7f333c00b3c0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.617+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1308155767 shutdown_connections 2024-05-10T05:35:56.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.617+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5626ac0083f0 0x5626ac00a810 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.617+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626abf8b6a0 0x5626abf99aa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.617+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626abf8cec0 0x5626ac0067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.617+0000 7f332f400700 0 client.15819 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:35:56.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.617+0000 7f336c7fdb80 1 --2- 172.21.15.131:0/1308155767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626abf8c570 0x5626abf99fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.617+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1308155767 >> 172.21.15.131:0/1308155767 conn(0x5626abeeda60 msgr2=0x5626abf82180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:56.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.618+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1308155767 shutdown_connections 2024-05-10T05:35:56.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.618+0000 7f336c7fdb80 1 -- 172.21.15.131:0/1308155767 wait complete. 2024-05-10T05:35:56.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.619+0000 7f336c7fdb80 20 client.15819 trim_cache size 0 max 16384 2024-05-10T05:35:56.620 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:35:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 -- 172.21.15.131:0/4115937368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33340a1360 msgr2=0x7f33340a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:56.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33340a1360 0x7f33340a3870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f333c00b2e0 tx=0x7f333c009400 comp rx=0 tx=0).stop 2024-05-10T05:35:56.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 -- 172.21.15.131:0/4115937368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c1abfe0 msgr2=0x7f334c1ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:56.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c1abfe0 0x7f334c1ae440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f334800ef30 tx=0x7f334800c450 comp rx=0 tx=0).stop 2024-05-10T05:35:56.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 -- 172.21.15.131:0/4115937368 shutdown_connections 2024-05-10T05:35:56.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33340a1360 0x7f33340a3870 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c1abfe0 0x7f334c1ae440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f334c109300 0x7f334c1a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 --2- 172.21.15.131:0/4115937368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c109c50 0x7f334c1a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:56.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 -- 172.21.15.131:0/4115937368 >> 172.21.15.131:0/4115937368 conn(0x7f334c06ea00 msgr2=0x7f334c10a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:56.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.622+0000 7f3354800700 1 -- 172.21.15.131:0/4115937368 shutdown_connections 2024-05-10T05:35:56.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:56.623+0000 7f3354800700 1 -- 172.21.15.131:0/4115937368 wait complete. 2024-05-10T05:35:56.706 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:35:57.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:57 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:35:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:57 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:35:57.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:57 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:35:57.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.555+0000 7f061d400700 1 -- 172.21.15.131:0/2784599321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0618110320 msgr2=0x7f0618112740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:57.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.555+0000 7f061d400700 1 --2- 172.21.15.131:0/2784599321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0618110320 0x7f0618112740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0610014a30 tx=0x7f0610014d40 comp rx=0 tx=0).stop 2024-05-10T05:35:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.556+0000 7f061d400700 1 -- 172.21.15.131:0/2784599321 shutdown_connections 2024-05-10T05:35:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.556+0000 7f061d400700 1 --2- 172.21.15.131:0/2784599321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0618110320 0x7f0618112740 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.556+0000 7f061d400700 1 --2- 172.21.15.131:0/2784599321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0618108210 0x7f0618108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.556+0000 7f061d400700 1 --2- 172.21.15.131:0/2784599321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0618108b60 0x7f061810fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.556+0000 7f061d400700 1 -- 172.21.15.131:0/2784599321 >> 172.21.15.131:0/2784599321 conn(0x7f061806ea00 msgr2=0x7f0618070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.556+0000 7f061d400700 1 -- 172.21.15.131:0/2784599321 shutdown_connections 2024-05-10T05:35:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.556+0000 7f061d400700 1 -- 172.21.15.131:0/2784599321 wait complete. 2024-05-10T05:35:57.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.558+0000 7f061d400700 1 Processor -- start 2024-05-10T05:35:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.558+0000 7f061d400700 1 -- start start 2024-05-10T05:35:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.559+0000 7f061d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0618108210 0x7f061819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.559+0000 7f061d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0618108b60 0x7f061819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.559+0000 7f061d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0618110320 0x7f06181a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.559+0000 7f0617200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0618110320 0x7f06181a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.559+0000 7f0617200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0618110320 0x7f06181a5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41060/0 (socket says 172.21.15.131:41060) 2024-05-10T05:35:57.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.559+0000 7f0617200700 1 -- 172.21.15.131:0/1180689184 learned_addr learned my addr 172.21.15.131:0/1180689184 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:57.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.559+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06181a6110 con 0x7f0618108210 2024-05-10T05:35:57.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.559+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06181a6280 con 0x7f0618108b60 2024-05-10T05:35:57.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.559+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06181a6440 con 0x7f0618110320 2024-05-10T05:35:57.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.560+0000 7f0617200700 1 -- 172.21.15.131:0/1180689184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0618108b60 msgr2=0x7f061819eb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:35:57.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.560+0000 7f0617200700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0618108b60 0x7f061819eb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.560+0000 7f0617200700 1 -- 172.21.15.131:0/1180689184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0618108210 msgr2=0x7f061819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:57.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.560+0000 7f0617200700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0618108210 0x7f061819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.560+0000 7f0617200700 1 -- 172.21.15.131:0/1180689184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610014660 con 0x7f0618110320 2024-05-10T05:35:57.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.560+0000 7f0617200700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0618110320 0x7f06181a5b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f061001cbb0 tx=0x7f061001c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.561+0000 7f060f400700 1 -- 172.21.15.131:0/1180689184 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061001a310 con 0x7f0618110320 2024-05-10T05:35:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.561+0000 7f060f400700 1 -- 172.21.15.131:0/1180689184 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0610012040 con 0x7f0618110320 2024-05-10T05:35:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.561+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06181a66c0 con 0x7f0618110320 2024-05-10T05:35:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.562+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06181a6bc0 con 0x7f0618110320 2024-05-10T05:35:57.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.563+0000 7f060f400700 1 -- 172.21.15.131:0/1180689184 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610004390 con 0x7f0618110320 2024-05-10T05:35:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.585+0000 7f063539ab80 1 Processor -- start 2024-05-10T05:35:57.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.591+0000 7f060f400700 1 -- 172.21.15.131:0/1180689184 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0610003a00 con 0x7f0618110320 2024-05-10T05:35:57.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.595+0000 7f063539ab80 1 -- start start 2024-05-10T05:35:57.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.596+0000 7f063539ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858193f6a0 0x55858193faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.596+0000 7f063539ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 0x558581940980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.596+0000 7f063539ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940ec0 0x5585819477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.596+0000 7f063539ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558581866150 con 0x55858193f6a0 2024-05-10T05:35:57.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.596+0000 7f063539ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558581863b10 con 0x558581940570 2024-05-10T05:35:57.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.596+0000 7f063539ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858185f400 con 0x558581940ec0 2024-05-10T05:35:57.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.601+0000 7f0615e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940ec0 0x5585819477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.601+0000 7f060f400700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05fc0a12c0 0x7f05fc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.601+0000 7f060f400700 1 -- 172.21.15.131:0/1180689184 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f06100d6300 con 0x7f0618110320 2024-05-10T05:35:57.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.602+0000 7f0615e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940ec0 0x5585819477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41076/0 (socket says 172.21.15.131:41076) 2024-05-10T05:35:57.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.602+0000 7f0615e00700 1 -- 172.21.15.131:0/3654946613 learned_addr learned my addr 172.21.15.131:0/3654946613 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:57.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.602+0000 7f0616800700 1 --2- 172.21.15.131:0/3654946613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 0x558581940980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.606+0000 7f0617200700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05fc0a12c0 0x7f05fc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.607+0000 7f0615e00700 1 --2- 172.21.15.131:0/3654946613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858193f6a0 0x55858193faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.607+0000 7f0615e00700 1 -- 172.21.15.131:0/3654946613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 msgr2=0x558581940980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.607+0000 7f0615e00700 1 --2- 172.21.15.131:0/3654946613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 0x558581940980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.607+0000 7f0615e00700 1 -- 172.21.15.131:0/3654946613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858193f6a0 msgr2=0x55858193faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.607+0000 7f0615e00700 1 --2- 172.21.15.131:0/3654946613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858193f6a0 0x55858193faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.607+0000 7f0615e00700 1 -- 172.21.15.131:0/3654946613 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585816e03b0 con 0x558581940ec0 2024-05-10T05:35:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.607+0000 7f0617200700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05fc0a12c0 0x7f05fc0a37d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f061001f040 tx=0x7f06100143a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.609+0000 7f0615e00700 1 --2- 172.21.15.131:0/3654946613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940ec0 0x5585819477c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06040121e0 tx=0x7f060400b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f417146e5f9379 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.612+0000 7f0603400700 1 -- 172.21.15.131:0/3654946613 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604012c80 con 0x558581940ec0 2024-05-10T05:35:57.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.612+0000 7f0603400700 1 -- 172.21.15.131:0/3654946613 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0604011070 con 0x558581940ec0 2024-05-10T05:35:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.616+0000 7f0603400700 1 -- 172.21.15.131:0/3654946613 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06040066e0 con 0x558581940ec0 2024-05-10T05:35:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.616+0000 7f063539ab80 1 -- 172.21.15.131:0/3654946613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940ec0 msgr2=0x5585819477c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.616+0000 7f063539ab80 1 --2- 172.21.15.131:0/3654946613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940ec0 0x5585819477c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06040121e0 tx=0x7f060400b6e0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.616+0000 7f063539ab80 1 -- 172.21.15.131:0/3654946613 shutdown_connections 2024-05-10T05:35:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.616+0000 7f063539ab80 1 --2- 172.21.15.131:0/3654946613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858193f6a0 0x55858193faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.616+0000 7f063539ab80 1 --2- 172.21.15.131:0/3654946613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940ec0 0x5585819477c0 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06040121e0 tx=0x7f060400b6e0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.616+0000 7f063539ab80 1 --2- 172.21.15.131:0/3654946613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 0x558581940980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.616+0000 7f063539ab80 1 -- 172.21.15.131:0/3654946613 >> 172.21.15.131:0/3654946613 conn(0x5585818a1a60 msgr2=0x558581939fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:57.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.619+0000 7f063539ab80 1 -- 172.21.15.131:0/3654946613 shutdown_connections 2024-05-10T05:35:57.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.623+0000 7f063539ab80 1 -- 172.21.15.131:0/3654946613 wait complete. 2024-05-10T05:35:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.627+0000 7f063539ab80 1 -- start start 2024-05-10T05:35:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.627+0000 7f063539ab80 1 Processor -- start 2024-05-10T05:35:57.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.628+0000 7f063539ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858193f6a0 0x55858195a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.628+0000 7f063539ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 0x55858195ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.628+0000 7f063539ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858195fcb0 0x5585819620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.628+0000 7f063539ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585817fb560 con 0x55858195fcb0 2024-05-10T05:35:57.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.628+0000 7f063539ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585817fb200 con 0x558581940570 2024-05-10T05:35:57.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.628+0000 7f063539ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585816e26a0 con 0x55858193f6a0 2024-05-10T05:35:57.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.628+0000 7f063539ab80 10 client.0 _ll_register_callbacks cb 0x558581804b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:57.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.629+0000 7f0615e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858193f6a0 0x55858195a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.629+0000 7f0615e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858193f6a0 0x55858195a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41078/0 (socket says 172.21.15.131:41078) 2024-05-10T05:35:57.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.629+0000 7f0615e00700 1 -- 172.21.15.131:0/3886627317 learned_addr learned my addr 172.21.15.131:0/3886627317 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:57.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.629+0000 7f0616800700 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 0x55858195ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.630+0000 7f0616800700 1 -- 172.21.15.131:0/3886627317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858193f6a0 msgr2=0x55858195a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:57.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.630+0000 7f0616800700 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858193f6a0 0x55858195a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.630+0000 7f0616800700 1 -- 172.21.15.131:0/3886627317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858195fcb0 msgr2=0x5585819620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:57.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.630+0000 7f0616800700 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858195fcb0 0x5585819620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.630+0000 7f0616800700 1 -- 172.21.15.131:0/3886627317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0604013040 con 0x558581940570 2024-05-10T05:35:57.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.633+0000 7f0616800700 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 0x55858195ae20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0618072dd0 tx=0x7f060800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c06bdd5d5177e9bf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:57.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.640+0000 7f05f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:35:57.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.640+0000 7f05f7e00700 1 -- 172.21.15.131:0/3886627317 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060800dd90 con 0x558581940570 2024-05-10T05:35:57.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.641+0000 7f05f7e00700 1 -- 172.21.15.131:0/3886627317 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0608012420 con 0x558581940570 2024-05-10T05:35:57.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.641+0000 7f063539ab80 1 -- 172.21.15.131:0/3886627317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585817b8270 con 0x558581940570 2024-05-10T05:35:57.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.641+0000 7f05f7e00700 1 -- 172.21.15.131:0/3886627317 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06080125e0 con 0x558581940570 2024-05-10T05:35:57.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.641+0000 7f063539ab80 10 client.25850 fetch_fsmap learned FSMap version 28 2024-05-10T05:35:57.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.641+0000 7f063539ab80 1 -- 172.21.15.131:0/3886627317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55858168cb60 con 0x558581940570 2024-05-10T05:35:57.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.642+0000 7f05f7e00700 1 -- 172.21.15.131:0/3886627317 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5585817b8270 con 0x558581940570 2024-05-10T05:35:57.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.643+0000 7f05f7e00700 1 -- 172.21.15.131:0/3886627317 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f06080148e0 con 0x558581940570 2024-05-10T05:35:57.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.643+0000 7f063539ab80 10 client.25850 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:35:57.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.643+0000 7f063539ab80 10 client.25850 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:35:57.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.644+0000 7f063539ab80 20 client.25850 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:57.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.644+0000 7f063539ab80 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558581966a10 0x558581968e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.644+0000 7f063539ab80 4 client.25850 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:57.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.644+0000 7f063539ab80 1 -- 172.21.15.131:0/3886627317 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55858190e780 con 0x558581966a10 2024-05-10T05:35:57.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.645+0000 7f0615e00700 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558581966a10 0x558581968e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.646+0000 7f0615e00700 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558581966a10 0x558581968e60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=701cc8c62311167a server_cookie=1ab8d3744da3a43 in_seq=0 out_seq=0 2024-05-10T05:35:57.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.646+0000 7f05f7e00700 10 client.25850 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:35:57.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.647+0000 7f05f7e00700 1 -- 172.21.15.131:0/3886627317 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55858190e780 con 0x558581966a10 2024-05-10T05:35:57.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.647+0000 7f05f7e00700 10 client.25850 handle_command_reply: tid=0 2024-05-10T05:35:57.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.648+0000 7f063539ab80 1 client.25850 shutdown 2024-05-10T05:35:57.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.649+0000 7f063539ab80 1 -- 172.21.15.131:0/3886627317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 msgr2=0x55858195ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:57.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.649+0000 7f063539ab80 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 0x55858195ae20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0618072dd0 tx=0x7f060800bf90 comp rx=0 tx=0).stop 2024-05-10T05:35:57.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.654+0000 7f063539ab80 1 -- 172.21.15.131:0/3886627317 shutdown_connections 2024-05-10T05:35:57.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.655+0000 7f063539ab80 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558581966a10 0x558581968e60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.656+0000 7f063539ab80 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858195fcb0 0x5585819620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.656+0000 7f063539ab80 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858193f6a0 0x55858195a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.656+0000 7f05f7e00700 0 client.25850 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:35:57.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.656+0000 7f063539ab80 1 --2- 172.21.15.131:0/3886627317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558581940570 0x55858195ae20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.656+0000 7f063539ab80 1 -- 172.21.15.131:0/3886627317 >> 172.21.15.131:0/3886627317 conn(0x5585818a1a60 msgr2=0x558581939fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:57.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.658+0000 7f063539ab80 1 -- 172.21.15.131:0/3886627317 shutdown_connections 2024-05-10T05:35:57.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.658+0000 7f063539ab80 1 -- 172.21.15.131:0/3886627317 wait complete. 2024-05-10T05:35:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.664+0000 7f063539ab80 20 client.25850 trim_cache size 0 max 16384 2024-05-10T05:35:57.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.723+0000 7f063539ab80 1 Processor -- start 2024-05-10T05:35:57.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.723+0000 7f063539ab80 1 -- start start 2024-05-10T05:35:57.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.723+0000 7f063539ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 0x558581962780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.723+0000 7f063539ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940570 0x55858193ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.723+0000 7f063539ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858195fcb0 0x5585819442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.724+0000 7f063539ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858193b770 con 0x55858195fcb0 2024-05-10T05:35:57.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.724+0000 7f063539ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858196e0a0 con 0x55858193f6a0 2024-05-10T05:35:57.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.724+0000 7f063539ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585819347c0 con 0x558581940570 2024-05-10T05:35:57.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.724+0000 7f0616800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940570 0x55858193ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.724+0000 7f0616800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940570 0x55858193ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41080/0 (socket says 172.21.15.131:41080) 2024-05-10T05:35:57.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.724+0000 7f0616800700 1 -- 172.21.15.131:0/99941052 learned_addr learned my addr 172.21.15.131:0/99941052 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:57.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.725+0000 7f0616800700 1 -- 172.21.15.131:0/99941052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 msgr2=0x558581962780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:57.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.725+0000 7f0616800700 1 --2- 172.21.15.131:0/99941052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 0x558581962780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.725+0000 7f0616800700 1 -- 172.21.15.131:0/99941052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858195fcb0 msgr2=0x5585819442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:57.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.727+0000 7f0615e00700 1 --2- 172.21.15.131:0/99941052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858195fcb0 0x5585819442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.727+0000 7f0616800700 1 --2- 172.21.15.131:0/99941052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858195fcb0 0x5585819442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.727+0000 7f0616800700 1 -- 172.21.15.131:0/99941052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558581825560 con 0x558581940570 2024-05-10T05:35:57.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.728+0000 7f0616800700 1 --2- 172.21.15.131:0/99941052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940570 0x55858193ad80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0608014e90 tx=0x7f060800d900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81384eec2aabe44f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:57.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.729+0000 7f0603e00700 1 -- 172.21.15.131:0/99941052 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060800a030 con 0x558581940570 2024-05-10T05:35:57.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.729+0000 7f0603e00700 1 -- 172.21.15.131:0/99941052 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f060800dd90 con 0x558581940570 2024-05-10T05:35:57.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.729+0000 7f0603e00700 1 -- 172.21.15.131:0/99941052 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060800a030 con 0x558581940570 2024-05-10T05:35:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.732+0000 7f063539ab80 1 -- 172.21.15.131:0/99941052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940570 msgr2=0x55858193ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.732+0000 7f063539ab80 1 --2- 172.21.15.131:0/99941052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940570 0x55858193ad80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0608014e90 tx=0x7f060800d900 comp rx=0 tx=0).stop 2024-05-10T05:35:57.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.734+0000 7f063539ab80 1 -- 172.21.15.131:0/99941052 shutdown_connections 2024-05-10T05:35:57.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.734+0000 7f063539ab80 1 --2- 172.21.15.131:0/99941052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55858195fcb0 0x5585819442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.735+0000 7f063539ab80 1 --2- 172.21.15.131:0/99941052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 0x558581962780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.735+0000 7f063539ab80 1 --2- 172.21.15.131:0/99941052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558581940570 0x55858193ad80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.735+0000 7f063539ab80 1 -- 172.21.15.131:0/99941052 >> 172.21.15.131:0/99941052 conn(0x5585818a1a60 msgr2=0x5585819437a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:57.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.742+0000 7f063539ab80 1 -- 172.21.15.131:0/99941052 shutdown_connections 2024-05-10T05:35:57.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.742+0000 7f063539ab80 1 -- 172.21.15.131:0/99941052 wait complete. 2024-05-10T05:35:57.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.744+0000 7f063539ab80 1 -- start start 2024-05-10T05:35:57.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.745+0000 7f063539ab80 1 Processor -- start 2024-05-10T05:35:57.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.748+0000 7f063539ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 0x5585819762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.748+0000 7f063539ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558581940570 0x5585819767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.749+0000 7f0615e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 0x5585819762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.749+0000 7f0615e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 0x5585819762b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45188/0 (socket says 172.21.15.131:45188) 2024-05-10T05:35:57.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.750+0000 7f0615e00700 1 -- 172.21.15.131:0/565515425 learned_addr learned my addr 172.21.15.131:0/565515425 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:57.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.750+0000 7f063539ab80 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858195fcb0 0x5585819bb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.750+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558581976ec0 con 0x558581940570 2024-05-10T05:35:57.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.750+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585819bbf30 con 0x55858193f6a0 2024-05-10T05:35:57.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.750+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585819bc0f0 con 0x55858195fcb0 2024-05-10T05:35:57.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.750+0000 7f063539ab80 10 client.0 _ll_register_callbacks cb 0x558581804b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:57.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.750+0000 7f0615e00700 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858195fcb0 0x5585819bb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.751+0000 7f0615e00700 1 -- 172.21.15.131:0/565515425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858195fcb0 msgr2=0x5585819bb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:57.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.751+0000 7f0615e00700 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858195fcb0 0x5585819bb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.751+0000 7f0615e00700 1 -- 172.21.15.131:0/565515425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558581940570 msgr2=0x5585819767f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:57.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.751+0000 7f0615e00700 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558581940570 0x5585819767f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.751+0000 7f0615e00700 1 -- 172.21.15.131:0/565515425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55858168cb60 con 0x55858193f6a0 2024-05-10T05:35:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.752+0000 7f0615e00700 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 0x5585819762b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f060400b8f0 tx=0x7f060400b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a887f19a47f05e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.752+0000 7f05f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:35:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.752+0000 7f05f7e00700 1 -- 172.21.15.131:0/565515425 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06040250d0 con 0x55858193f6a0 2024-05-10T05:35:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.753+0000 7f05f7e00700 1 -- 172.21.15.131:0/565515425 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f060402c060 con 0x55858193f6a0 2024-05-10T05:35:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.753+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0604011070 con 0x55858193f6a0 2024-05-10T05:35:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.753+0000 7f05f7e00700 1 -- 172.21.15.131:0/565515425 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0604025740 con 0x55858193f6a0 2024-05-10T05:35:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.754+0000 7f063539ab80 10 client.25853 fetch_fsmap learned FSMap version 28 2024-05-10T05:35:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.754+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5585816e4cd0 con 0x55858193f6a0 2024-05-10T05:35:57.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.754+0000 7f05f7e00700 1 -- 172.21.15.131:0/565515425 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0604011070 con 0x55858193f6a0 2024-05-10T05:35:57.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.754+0000 7f05f7e00700 1 -- 172.21.15.131:0/565515425 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f06040039e0 con 0x55858193f6a0 2024-05-10T05:35:57.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.755+0000 7f063539ab80 10 client.25853 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:35:57.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.755+0000 7f063539ab80 10 client.25853 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:35:57.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.755+0000 7f063539ab80 20 client.25853 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:57.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.755+0000 7f063539ab80 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558581966a10 0x5585819bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:57.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.755+0000 7f063539ab80 4 client.25853 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:35:57.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.756+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558581857ec0 con 0x558581966a10 2024-05-10T05:35:57.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.756+0000 7f0616800700 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558581966a10 0x5585819bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:57.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.756+0000 7f05f7e00700 10 client.25853 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:35:57.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.757+0000 7f0616800700 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558581966a10 0x5585819bf9c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc9bfb2fd6a40120 server_cookie=203d8d0250f36cf2 in_seq=0 out_seq=0 2024-05-10T05:35:57.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.757+0000 7f05f7e00700 1 -- 172.21.15.131:0/565515425 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558581857ec0 con 0x558581966a10 2024-05-10T05:35:57.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.757+0000 7f05f7e00700 10 client.25853 handle_command_reply: tid=0 2024-05-10T05:35:57.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.757+0000 7f063539ab80 1 client.25853 shutdown 2024-05-10T05:35:57.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.758+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 msgr2=0x5585819762b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:57.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.758+0000 7f063539ab80 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 0x5585819762b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f060400b8f0 tx=0x7f060400b880 comp rx=0 tx=0).stop 2024-05-10T05:35:57.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.758+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 shutdown_connections 2024-05-10T05:35:57.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.758+0000 7f063539ab80 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558581966a10 0x5585819bf9c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.758+0000 7f063539ab80 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858195fcb0 0x5585819bb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.758+0000 7f063539ab80 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858193f6a0 0x5585819762b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.758+0000 7f063539ab80 1 --2- 172.21.15.131:0/565515425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558581940570 0x5585819767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.758+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 >> 172.21.15.131:0/565515425 conn(0x5585818a1a60 msgr2=0x5585819437a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.759+0000 7f05f7e00700 0 client.25853 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:35:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.759+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 shutdown_connections 2024-05-10T05:35:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.759+0000 7f063539ab80 1 -- 172.21.15.131:0/565515425 wait complete. 2024-05-10T05:35:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.759+0000 7f063539ab80 20 client.25853 trim_cache size 0 max 16384 2024-05-10T05:35:57.761 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:35:57.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.760+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05fc0a12c0 msgr2=0x7f05fc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:57.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.760+0000 7f061d400700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05fc0a12c0 0x7f05fc0a37d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f061001f040 tx=0x7f06100143a0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.761+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0618110320 msgr2=0x7f06181a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:57.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.761+0000 7f061d400700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0618110320 0x7f06181a5b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f061001cbb0 tx=0x7f061001c6f0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.761+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 shutdown_connections 2024-05-10T05:35:57.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.761+0000 7f061d400700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05fc0a12c0 0x7f05fc0a37d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.761+0000 7f061d400700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0618108210 0x7f061819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.761+0000 7f061d400700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0618110320 0x7f06181a5b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.761+0000 7f061d400700 1 --2- 172.21.15.131:0/1180689184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0618108b60 0x7f061819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:57.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.761+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 >> 172.21.15.131:0/1180689184 conn(0x7f061806ea00 msgr2=0x7f061806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:57.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.761+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 shutdown_connections 2024-05-10T05:35:57.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:57.761+0000 7f061d400700 1 -- 172.21.15.131:0/1180689184 wait complete. 2024-05-10T05:35:57.838 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:35:58.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:58 smithi131 ceph-mon[37264]: pgmap v363: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 238 MiB/s wr, 73 op/s 2024-05-10T05:35:58.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:58 smithi142 ceph-mon[77667]: pgmap v363: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 238 MiB/s wr, 73 op/s 2024-05-10T05:35:58.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.478+0000 7f9561800700 1 -- 172.21.15.131:0/2754707623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95540a6560 msgr2=0x7f95540a6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:58.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.478+0000 7f9561800700 1 --2- 172.21.15.131:0/2754707623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95540a6560 0x7f95540a6970 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f95480098f0 tx=0x7f9548009c00 comp rx=0 tx=0).stop 2024-05-10T05:35:58.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.479+0000 7f9561800700 1 -- 172.21.15.131:0/2754707623 shutdown_connections 2024-05-10T05:35:58.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.479+0000 7f9561800700 1 --2- 172.21.15.131:0/2754707623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95540a6eb0 0x7f95540ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.479+0000 7f9561800700 1 --2- 172.21.15.131:0/2754707623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95540a5580 0x7f95540a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.479+0000 7f9561800700 1 --2- 172.21.15.131:0/2754707623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95540a6560 0x7f95540a6970 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.479+0000 7f9561800700 1 -- 172.21.15.131:0/2754707623 >> 172.21.15.131:0/2754707623 conn(0x7f95540a0a10 msgr2=0x7f95540a2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:58.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.479+0000 7f9561800700 1 -- 172.21.15.131:0/2754707623 shutdown_connections 2024-05-10T05:35:58.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.479+0000 7f9561800700 1 -- 172.21.15.131:0/2754707623 wait complete. 2024-05-10T05:35:58.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.482+0000 7f9561800700 1 Processor -- start 2024-05-10T05:35:58.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.488+0000 7f9561800700 1 -- start start 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.489+0000 7f9561800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95540a5580 0x7f9554144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.489+0000 7f9561800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95540a6560 0x7f9554144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.489+0000 7f9561800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95540a6eb0 0x7f955414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.490+0000 7f9561800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955414bbe0 con 0x7f95540a6eb0 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.490+0000 7f9561800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540ad170 con 0x7f95540a5580 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.490+0000 7f9561800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540ad330 con 0x7f95540a6560 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.490+0000 7f955be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95540a5580 0x7f9554144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.490+0000 7f955be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95540a5580 0x7f9554144140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45208/0 (socket says 172.21.15.131:45208) 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.490+0000 7f955be00700 1 -- 172.21.15.131:0/3738322077 learned_addr learned my addr 172.21.15.131:0/3738322077 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.490+0000 7f9560e00700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95540a6eb0 0x7f955414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.490+0000 7f955b400700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95540a6560 0x7f9554144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.490+0000 7f955be00700 1 -- 172.21.15.131:0/3738322077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95540a6560 msgr2=0x7f9554144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:58.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.491+0000 7f955be00700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95540a6560 0x7f9554144680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.491+0000 7f955be00700 1 -- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95540a6eb0 msgr2=0x7f955414b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.491+0000 7f955be00700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95540a6eb0 0x7f955414b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.491+0000 7f955be00700 1 -- 172.21.15.131:0/3738322077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9548009520 con 0x7f95540a5580 2024-05-10T05:35:58.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.491+0000 7f9560e00700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95540a6eb0 0x7f955414b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:58.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.491+0000 7f955be00700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95540a5580 0x7f9554144140 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f955000b520 tx=0x7f955000b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:58.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.491+0000 7f955b400700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95540a6560 0x7f9554144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:58.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.492+0000 7f9558c00700 1 -- 172.21.15.131:0/3738322077 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550011830 con 0x7f95540a5580 2024-05-10T05:35:58.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.492+0000 7f9558c00700 1 -- 172.21.15.131:0/3738322077 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9550011e50 con 0x7f95540a5580 2024-05-10T05:35:58.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.492+0000 7f9558c00700 1 -- 172.21.15.131:0/3738322077 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955000f2e0 con 0x7f95540a5580 2024-05-10T05:35:58.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.493+0000 7f9561800700 1 -- 172.21.15.131:0/3738322077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95540ad610 con 0x7f95540a5580 2024-05-10T05:35:58.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.493+0000 7f9561800700 1 -- 172.21.15.131:0/3738322077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95540adb20 con 0x7f95540a5580 2024-05-10T05:35:58.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.504+0000 7f9558c00700 1 -- 172.21.15.131:0/3738322077 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f95500105f0 con 0x7f95540a5580 2024-05-10T05:35:58.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.507+0000 7f9558c00700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f95440a1270 0x7f95440a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.507+0000 7f9558c00700 1 -- 172.21.15.131:0/3738322077 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f95500cc310 con 0x7f95540a5580 2024-05-10T05:35:58.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.508+0000 7f957acd5b80 1 Processor -- start 2024-05-10T05:35:58.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.508+0000 7f957acd5b80 1 -- start start 2024-05-10T05:35:58.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.509+0000 7f955b400700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f95440a1270 0x7f95440a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.509+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a92530 0x558d04a92960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.509+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a937a0 0x558d04a93bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.509+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 0x558d04a9a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.509+0000 7f955be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a92530 0x558d04a92960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.510+0000 7f955be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a92530 0x558d04a92960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41640/0 (socket says 172.21.15.131:41640) 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.510+0000 7f955be00700 1 -- 172.21.15.131:0/1028441834 learned_addr learned my addr 172.21.15.131:0/1028441834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.510+0000 7f957acd5b80 1 -- 172.21.15.131:0/1028441834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d049b9150 con 0x558d04a92530 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.510+0000 7f957acd5b80 1 -- 172.21.15.131:0/1028441834 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d049b6b10 con 0x558d04a937a0 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.510+0000 7f957acd5b80 1 -- 172.21.15.131:0/1028441834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d049b2400 con 0x558d04a940f0 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.510+0000 7f9560e00700 1 --2- 172.21.15.131:0/1028441834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 0x558d04a9a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.510+0000 7f955b400700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f95440a1270 0x7f95440a3780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f954800fae0 tx=0x7f954800fa70 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.510+0000 7f9560e00700 1 -- 172.21.15.131:0/1028441834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a937a0 msgr2=0x558d04a93bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.510+0000 7f9560e00700 1 --2- 172.21.15.131:0/1028441834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a937a0 0x558d04a93bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.511+0000 7f9560e00700 1 -- 172.21.15.131:0/1028441834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a92530 msgr2=0x558d04a92960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.511+0000 7f9560e00700 1 --2- 172.21.15.131:0/1028441834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a92530 0x558d04a92960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.511+0000 7f9560e00700 1 -- 172.21.15.131:0/1028441834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d048333b0 con 0x558d04a940f0 2024-05-10T05:35:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.511+0000 7f955be00700 1 --2- 172.21.15.131:0/1028441834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a92530 0x558d04a92960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:35:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.511+0000 7f9560e00700 1 --2- 172.21.15.131:0/1028441834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 0x558d04a9a9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f955c04f420 tx=0x7f955c0757d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50314b9ae7f184a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.514+0000 7f954d600700 1 -- 172.21.15.131:0/1028441834 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c0726a0 con 0x558d04a940f0 2024-05-10T05:35:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.514+0000 7f954d600700 1 -- 172.21.15.131:0/1028441834 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f955c072cc0 con 0x558d04a940f0 2024-05-10T05:35:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.514+0000 7f954d600700 1 -- 172.21.15.131:0/1028441834 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c07ed40 con 0x558d04a940f0 2024-05-10T05:35:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.517+0000 7f957acd5b80 1 -- 172.21.15.131:0/1028441834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 msgr2=0x558d04a9a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.517+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1028441834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 0x558d04a9a9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f955c04f420 tx=0x7f955c0757d0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.517+0000 7f957acd5b80 1 -- 172.21.15.131:0/1028441834 shutdown_connections 2024-05-10T05:35:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.517+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1028441834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a92530 0x558d04a92960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.517+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1028441834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 0x558d04a9a9f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.517+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1028441834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a937a0 0x558d04a93bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.517+0000 7f957acd5b80 1 -- 172.21.15.131:0/1028441834 >> 172.21.15.131:0/1028441834 conn(0x558d049f4a60 msgr2=0x558d04a8d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.518+0000 7f957acd5b80 1 -- 172.21.15.131:0/1028441834 shutdown_connections 2024-05-10T05:35:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.518+0000 7f957acd5b80 1 -- 172.21.15.131:0/1028441834 wait complete. 2024-05-10T05:35:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.518+0000 7f957acd5b80 1 -- start start 2024-05-10T05:35:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.520+0000 7f957acd5b80 1 Processor -- start 2024-05-10T05:35:58.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.521+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a92530 0x558d04ae0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.521+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a937a0 0x558d04ae0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.521+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 0x558d04ae7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.523+0000 7f955be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a92530 0x558d04ae0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.523+0000 7f955be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a92530 0x558d04ae0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45228/0 (socket says 172.21.15.131:45228) 2024-05-10T05:35:58.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.524+0000 7f957acd5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0494e560 con 0x558d04a937a0 2024-05-10T05:35:58.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.524+0000 7f957acd5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0494e200 con 0x558d04a92530 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.524+0000 7f957acd5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d048356a0 con 0x558d04a940f0 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.524+0000 7f957acd5b80 10 client.0 _ll_register_callbacks cb 0x558d04957b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.524+0000 7f955be00700 1 -- 172.21.15.131:0/1728271108 learned_addr learned my addr 172.21.15.131:0/1728271108 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.524+0000 7f9560e00700 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 0x558d04ae7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.525+0000 7f955be00700 1 -- 172.21.15.131:0/1728271108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 msgr2=0x558d04ae7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.525+0000 7f955be00700 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 0x558d04ae7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.525+0000 7f955be00700 1 -- 172.21.15.131:0/1728271108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a937a0 msgr2=0x558d04ae0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.525+0000 7f955be00700 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a937a0 0x558d04ae0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.525+0000 7f955be00700 1 -- 172.21.15.131:0/1728271108 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c068dc0 con 0x558d04a92530 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.525+0000 7f955be00700 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a92530 0x558d04ae0780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9550096220 tx=0x7f9550099370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a1e880bac73e11a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.525+0000 7f9542000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.525+0000 7f9542000700 1 -- 172.21.15.131:0/1728271108 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550010380 con 0x558d04a92530 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.525+0000 7f9542000700 1 -- 172.21.15.131:0/1728271108 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f95500997b0 con 0x558d04a92530 2024-05-10T05:35:58.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.525+0000 7f957acd5b80 1 -- 172.21.15.131:0/1728271108 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d0490b270 con 0x558d04a92530 2024-05-10T05:35:58.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.526+0000 7f9542000700 1 -- 172.21.15.131:0/1728271108 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f955009d530 con 0x558d04a92530 2024-05-10T05:35:58.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.527+0000 7f957acd5b80 10 client.25862 fetch_fsmap learned FSMap version 28 2024-05-10T05:35:58.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.530+0000 7f957acd5b80 1 -- 172.21.15.131:0/1728271108 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558d047dfb60 con 0x558d04a92530 2024-05-10T05:35:58.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.532+0000 7f9542000700 1 -- 172.21.15.131:0/1728271108 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d0490b270 con 0x558d04a92530 2024-05-10T05:35:58.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.532+0000 7f9542000700 1 -- 172.21.15.131:0/1728271108 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f95500b3020 con 0x558d04a92530 2024-05-10T05:35:58.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.532+0000 7f957acd5b80 10 client.25862 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:35:58.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.532+0000 7f957acd5b80 10 client.25862 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:35:58.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.532+0000 7f957acd5b80 20 client.25862 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:58.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.532+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x558d04aec390 0x558d04aee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.532+0000 7f957acd5b80 4 client.25862 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:58.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.532+0000 7f957acd5b80 1 -- 172.21.15.131:0/1728271108 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d04a61780 con 0x558d04aec390 2024-05-10T05:35:58.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.533+0000 7f955b400700 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x558d04aec390 0x558d04aee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.538+0000 7f9542000700 10 client.25862 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:35:58.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.538+0000 7f955b400700 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x558d04aec390 0x558d04aee7e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6c035bcaf130ee0 server_cookie=136296ad8d9aac30 in_seq=0 out_seq=0 2024-05-10T05:35:58.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.539+0000 7f9542000700 1 -- 172.21.15.131:0/1728271108 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d04a61780 con 0x558d04aec390 2024-05-10T05:35:58.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.540+0000 7f9542000700 10 client.25862 handle_command_reply: tid=0 2024-05-10T05:35:58.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.540+0000 7f957acd5b80 1 client.25862 shutdown 2024-05-10T05:35:58.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.541+0000 7f957acd5b80 1 -- 172.21.15.131:0/1728271108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a92530 msgr2=0x558d04ae0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:58.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.541+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a92530 0x558d04ae0780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9550096220 tx=0x7f9550099370 comp rx=0 tx=0).stop 2024-05-10T05:35:58.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.541+0000 7f957acd5b80 1 -- 172.21.15.131:0/1728271108 shutdown_connections 2024-05-10T05:35:58.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.541+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x558d04aec390 0x558d04aee7e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.541+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a940f0 0x558d04ae7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.541+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a92530 0x558d04ae0780 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.541+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1728271108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a937a0 0x558d04ae0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.541+0000 7f957acd5b80 1 -- 172.21.15.131:0/1728271108 >> 172.21.15.131:0/1728271108 conn(0x558d049f4a60 msgr2=0x558d04a91060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.545+0000 7f9542000700 0 client.25862 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:35:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.546+0000 7f957acd5b80 1 -- 172.21.15.131:0/1728271108 shutdown_connections 2024-05-10T05:35:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.546+0000 7f957acd5b80 1 -- 172.21.15.131:0/1728271108 wait complete. 2024-05-10T05:35:58.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:58 smithi157 ceph-mon[78427]: pgmap v363: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 238 MiB/s wr, 73 op/s 2024-05-10T05:35:58.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.550+0000 7f957acd5b80 20 client.25862 trim_cache size 0 max 16384 2024-05-10T05:35:58.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.596+0000 7f957acd5b80 1 Processor -- start 2024-05-10T05:35:58.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.596+0000 7f957acd5b80 1 -- start start 2024-05-10T05:35:58.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.599+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04abd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.599+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a937a0 0x558d04ae5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.600+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a940f0 0x558d04ae6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.600+0000 7f957acd5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d04ae8d20 con 0x558d04a940f0 2024-05-10T05:35:58.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.600+0000 7f957acd5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d04af3980 con 0x558d04a937a0 2024-05-10T05:35:58.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.600+0000 7f957acd5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d04a97dc0 con 0x558d04a92530 2024-05-10T05:35:58.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.600+0000 7f955be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04abd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.601+0000 7f955be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04abd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41134/0 (socket says 172.21.15.131:41134) 2024-05-10T05:35:58.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.601+0000 7f955be00700 1 -- 172.21.15.131:0/1813880327 learned_addr learned my addr 172.21.15.131:0/1813880327 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:58.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.601+0000 7f955b400700 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a937a0 0x558d04ae5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.601+0000 7f9560e00700 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a940f0 0x558d04ae6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.602+0000 7f955be00700 1 -- 172.21.15.131:0/1813880327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a937a0 msgr2=0x558d04ae5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:58.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.604+0000 7f955be00700 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a937a0 0x558d04ae5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.604+0000 7f955be00700 1 -- 172.21.15.131:0/1813880327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a940f0 msgr2=0x558d04ae6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:58.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.604+0000 7f955be00700 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a940f0 0x558d04ae6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.604+0000 7f955be00700 1 -- 172.21.15.131:0/1813880327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d04978560 con 0x558d04a92530 2024-05-10T05:35:58.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.604+0000 7f9560e00700 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a940f0 0x558d04ae6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:58.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.604+0000 7f955b400700 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a937a0 0x558d04ae5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:58.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.606+0000 7f955be00700 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04abd650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f955000bc30 tx=0x7f95500c9d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bd36db75c445464 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:58.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.615+0000 7f954e000700 1 -- 172.21.15.131:0/1813880327 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955009aea0 con 0x558d04a92530 2024-05-10T05:35:58.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.616+0000 7f954e000700 1 -- 172.21.15.131:0/1813880327 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9550010380 con 0x558d04a92530 2024-05-10T05:35:58.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.616+0000 7f954e000700 1 -- 172.21.15.131:0/1813880327 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95500a1b80 con 0x558d04a92530 2024-05-10T05:35:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.622+0000 7f957acd5b80 1 -- 172.21.15.131:0/1813880327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 msgr2=0x558d04abd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.622+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04abd650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f955000bc30 tx=0x7f95500c9d70 comp rx=0 tx=0).stop 2024-05-10T05:35:58.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.622+0000 7f957acd5b80 1 -- 172.21.15.131:0/1813880327 shutdown_connections 2024-05-10T05:35:58.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.622+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a940f0 0x558d04ae6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.622+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04abd650 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.622+0000 7f957acd5b80 1 --2- 172.21.15.131:0/1813880327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a937a0 0x558d04ae5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.622+0000 7f957acd5b80 1 -- 172.21.15.131:0/1813880327 >> 172.21.15.131:0/1813880327 conn(0x558d049f4a60 msgr2=0x558d04abcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.624+0000 7f957acd5b80 1 -- 172.21.15.131:0/1813880327 shutdown_connections 2024-05-10T05:35:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.625+0000 7f957acd5b80 1 -- 172.21.15.131:0/1813880327 wait complete. 2024-05-10T05:35:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.626+0000 7f957acd5b80 1 -- start start 2024-05-10T05:35:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.626+0000 7f957acd5b80 1 Processor -- start 2024-05-10T05:35:58.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.631+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04b3c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.632+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a937a0 0x558d04b3ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.632+0000 7f957acd5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a940f0 0x558d04b412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.634+0000 7f9560e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a940f0 0x558d04b412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.634+0000 7f9560e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a940f0 0x558d04b412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45250/0 (socket says 172.21.15.131:45250) 2024-05-10T05:35:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.635+0000 7f9560e00700 1 -- 172.21.15.131:0/2947282367 learned_addr learned my addr 172.21.15.131:0/2947282367 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.638+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d04ae6480 con 0x558d04a937a0 2024-05-10T05:35:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.638+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d04b417f0 con 0x558d04a940f0 2024-05-10T05:35:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.638+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d04b41960 con 0x558d04a92530 2024-05-10T05:35:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.638+0000 7f957acd5b80 10 client.0 _ll_register_callbacks cb 0x558d04957b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.639+0000 7f955be00700 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04b3c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.639+0000 7f955b400700 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a937a0 0x558d04b3ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.640+0000 7f955be00700 1 -- 172.21.15.131:0/2947282367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a940f0 msgr2=0x558d04b412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.640+0000 7f955be00700 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a940f0 0x558d04b412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.640+0000 7f955be00700 1 -- 172.21.15.131:0/2947282367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a937a0 msgr2=0x558d04b3ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:58.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.643+0000 7f955be00700 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a937a0 0x558d04b3ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.643+0000 7f955be00700 1 -- 172.21.15.131:0/2947282367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95540ac220 con 0x558d04a92530 2024-05-10T05:35:58.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.646+0000 7f9542000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:35:58.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.646+0000 7f955be00700 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04b3c810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x558d04a878f0 tx=0x7f95500109f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a962385e394233f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:58.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.647+0000 7f9542000700 1 -- 172.21.15.131:0/2947282367 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955009aea0 con 0x558d04a92530 2024-05-10T05:35:58.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.648+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f955c072cc0 con 0x558d04a92530 2024-05-10T05:35:58.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.649+0000 7f9542000700 1 -- 172.21.15.131:0/2947282367 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f95500958f0 con 0x558d04a92530 2024-05-10T05:35:58.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.649+0000 7f9542000700 1 -- 172.21.15.131:0/2947282367 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f955009aea0 con 0x558d04a92530 2024-05-10T05:35:58.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.649+0000 7f957acd5b80 10 client.25882 fetch_fsmap learned FSMap version 28 2024-05-10T05:35:58.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.650+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558d048333b0 con 0x558d04a92530 2024-05-10T05:35:58.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.651+0000 7f9542000700 1 -- 172.21.15.131:0/2947282367 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f955c072cc0 con 0x558d04a92530 2024-05-10T05:35:58.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.653+0000 7f9542000700 1 -- 172.21.15.131:0/2947282367 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f95500b3020 con 0x558d04a92530 2024-05-10T05:35:58.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.653+0000 7f957acd5b80 10 client.25882 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:35:58.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.653+0000 7f957acd5b80 10 client.25882 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:35:58.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.653+0000 7f957acd5b80 20 client.25882 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:58.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.653+0000 7f957acd5b80 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x558d04b42d00 0x558d04b45120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:58.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.654+0000 7f957acd5b80 4 client.25882 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:35:58.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.654+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558d049aaec0 con 0x558d04b42d00 2024-05-10T05:35:58.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.662+0000 7f955b400700 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x558d04b42d00 0x558d04b45120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:58.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.670+0000 7f955b400700 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x558d04b42d00 0x558d04b45120 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f287e37324af6327 server_cookie=f0d66399d2f461b1 in_seq=0 out_seq=0 2024-05-10T05:35:58.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.670+0000 7f9542000700 10 client.25882 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:35:58.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.673+0000 7f9542000700 1 -- 172.21.15.131:0/2947282367 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558d049aaec0 con 0x558d04b42d00 2024-05-10T05:35:58.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.673+0000 7f9542000700 10 client.25882 handle_command_reply: tid=0 2024-05-10T05:35:58.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.673+0000 7f957acd5b80 1 client.25882 shutdown 2024-05-10T05:35:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.678+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 msgr2=0x558d04b3c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.678+0000 7f957acd5b80 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04b3c810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x558d04a878f0 tx=0x7f95500109f0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.678+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 shutdown_connections 2024-05-10T05:35:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.678+0000 7f957acd5b80 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x558d04b42d00 0x558d04b45120 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.681+0000 7f9542000700 0 client.25882 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:35:58.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.681+0000 7f957acd5b80 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d04a940f0 0x558d04b412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.681+0000 7f957acd5b80 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d04a92530 0x558d04b3c810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.681+0000 7f957acd5b80 1 --2- 172.21.15.131:0/2947282367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558d04a937a0 0x558d04b3ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.681+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 >> 172.21.15.131:0/2947282367 conn(0x558d049f4a60 msgr2=0x558d04a8cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.682+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 shutdown_connections 2024-05-10T05:35:58.686 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:35:58.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.682+0000 7f957acd5b80 1 -- 172.21.15.131:0/2947282367 wait complete. 2024-05-10T05:35:58.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.684+0000 7f957acd5b80 20 client.25882 trim_cache size 0 max 16384 2024-05-10T05:35:58.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.686+0000 7f9561800700 1 -- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f95440a1270 msgr2=0x7f95440a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:58.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.686+0000 7f9561800700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f95440a1270 0x7f95440a3780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f954800fae0 tx=0x7f954800fa70 comp rx=0 tx=0).stop 2024-05-10T05:35:58.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.686+0000 7f9561800700 1 -- 172.21.15.131:0/3738322077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95540a5580 msgr2=0x7f9554144140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:58.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.686+0000 7f9561800700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95540a5580 0x7f9554144140 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f955000b520 tx=0x7f955000b830 comp rx=0 tx=0).stop 2024-05-10T05:35:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.691+0000 7f9561800700 1 -- 172.21.15.131:0/3738322077 shutdown_connections 2024-05-10T05:35:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.691+0000 7f9561800700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f95440a1270 0x7f95440a3780 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.691+0000 7f9561800700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95540a6eb0 0x7f955414b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.691+0000 7f9561800700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95540a5580 0x7f9554144140 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.691+0000 7f9561800700 1 --2- 172.21.15.131:0/3738322077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95540a6560 0x7f9554144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:58.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.691+0000 7f9561800700 1 -- 172.21.15.131:0/3738322077 >> 172.21.15.131:0/3738322077 conn(0x7f95540a0a10 msgr2=0x7f95540a1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:58.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.695+0000 7f9561800700 1 -- 172.21.15.131:0/3738322077 shutdown_connections 2024-05-10T05:35:58.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:58.699+0000 7f9561800700 1 -- 172.21.15.131:0/3738322077 wait complete. 2024-05-10T05:35:58.773 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:35:59.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:35:59 smithi131 ceph-mon[37264]: pgmap v364: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 75 op/s 2024-05-10T05:35:59.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.467+0000 7f8e0e200700 1 -- 172.21.15.131:0/2943159270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e081097a0 msgr2=0x7f8e08109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:59.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.467+0000 7f8e0e200700 1 --2- 172.21.15.131:0/2943159270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e081097a0 0x7f8e08109bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000dd20 tx=0x7f8e0000f330 comp rx=0 tx=0).stop 2024-05-10T05:35:59.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.494+0000 7f8e0e200700 1 -- 172.21.15.131:0/2943159270 shutdown_connections 2024-05-10T05:35:59.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.494+0000 7f8e0e200700 1 --2- 172.21.15.131:0/2943159270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e08108c90 0x7f8e081107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.494+0000 7f8e0e200700 1 --2- 172.21.15.131:0/2943159270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e081097a0 0x7f8e08109bf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.494+0000 7f8e0e200700 1 --2- 172.21.15.131:0/2943159270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e08108340 0x7f8e08108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.494+0000 7f8e0e200700 1 -- 172.21.15.131:0/2943159270 >> 172.21.15.131:0/2943159270 conn(0x7f8e0806ec70 msgr2=0x7f8e08071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:59.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.495+0000 7f8e0e200700 1 -- 172.21.15.131:0/2943159270 shutdown_connections 2024-05-10T05:35:59.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.495+0000 7f8e0e200700 1 -- 172.21.15.131:0/2943159270 wait complete. 2024-05-10T05:35:59.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.513+0000 7f8e0e200700 1 Processor -- start 2024-05-10T05:35:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.516+0000 7f8e0e200700 1 -- start start 2024-05-10T05:35:59.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.522+0000 7f8e0e200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e08108340 0x7f8e0819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.523+0000 7f8e0e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e08108c90 0x7f8e0819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.523+0000 7f8e0ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e08108340 0x7f8e0819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.523+0000 7f8e0ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e08108340 0x7f8e0819e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41684/0 (socket says 172.21.15.131:41684) 2024-05-10T05:35:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.523+0000 7f8e0e200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e081097a0 0x7f8e081a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.524+0000 7f8e0e200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e081a5e40 con 0x7f8e08108340 2024-05-10T05:35:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.524+0000 7f8e0e200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e081a5f80 con 0x7f8e08108c90 2024-05-10T05:35:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.524+0000 7f8e0e200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e081a6110 con 0x7f8e081097a0 2024-05-10T05:35:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.524+0000 7f8e0d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e081097a0 0x7f8e081a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.524+0000 7f8e0d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e081097a0 0x7f8e081a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41164/0 (socket says 172.21.15.131:41164) 2024-05-10T05:35:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.524+0000 7f8e0d800700 1 -- 172.21.15.131:0/4227188740 learned_addr learned my addr 172.21.15.131:0/4227188740 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.525+0000 7f8e07e00700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e08108c90 0x7f8e0819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.525+0000 7f8e0d800700 1 -- 172.21.15.131:0/4227188740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e08108c90 msgr2=0x7f8e0819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.525+0000 7f8e0d800700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e08108c90 0x7f8e0819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.525+0000 7f8e0d800700 1 -- 172.21.15.131:0/4227188740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e08108340 msgr2=0x7f8e0819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.525+0000 7f8e0d800700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e08108340 0x7f8e0819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.525+0000 7f8e0d800700 1 -- 172.21.15.131:0/4227188740 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0000da00 con 0x7f8e081097a0 2024-05-10T05:35:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.527+0000 7f8e0d800700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e081097a0 0x7f8e081a5900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8df800ef60 tx=0x7f8df800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:59.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.531+0000 7f8e05600700 1 -- 172.21.15.131:0/4227188740 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8018070 con 0x7f8e081097a0 2024-05-10T05:35:59.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.532+0000 7f8e05600700 1 -- 172.21.15.131:0/4227188740 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df800f040 con 0x7f8e081097a0 2024-05-10T05:35:59.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.532+0000 7f8e05600700 1 -- 172.21.15.131:0/4227188740 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8004020 con 0x7f8e081097a0 2024-05-10T05:35:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.534+0000 7f8e0e200700 1 -- 172.21.15.131:0/4227188740 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e081a6390 con 0x7f8e081097a0 2024-05-10T05:35:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.535+0000 7f8e0e200700 1 -- 172.21.15.131:0/4227188740 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e081a6820 con 0x7f8e081097a0 2024-05-10T05:35:59.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:35:59 smithi157 ceph-mon[78427]: pgmap v364: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 75 op/s 2024-05-10T05:35:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.561+0000 7f8e277a2b80 1 Processor -- start 2024-05-10T05:35:59.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.570+0000 7f8e05600700 1 -- 172.21.15.131:0/4227188740 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8df8004ad0 con 0x7f8e081097a0 2024-05-10T05:35:59.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.570+0000 7f8e277a2b80 1 -- start start 2024-05-10T05:35:59.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.571+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5aca530 0x55d7d5aca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.571+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 0x55d7d5acbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.571+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.571+0000 7f8e277a2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d59f1150 con 0x55d7d5aca530 2024-05-10T05:35:59.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.571+0000 7f8e277a2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d59eeb10 con 0x55d7d5acc0f0 2024-05-10T05:35:59.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.571+0000 7f8e277a2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d59ea400 con 0x55d7d5acb7a0 2024-05-10T05:35:59.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.572+0000 7f8e07e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.572+0000 7f8e07e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45278/0 (socket says 172.21.15.131:45278) 2024-05-10T05:35:59.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.572+0000 7f8e07e00700 1 -- 172.21.15.131:0/138641543 learned_addr learned my addr 172.21.15.131:0/138641543 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:59.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.572+0000 7f8e0ce00700 1 --2- 172.21.15.131:0/138641543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 0x55d7d5acbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.573+0000 7f8e07e00700 1 -- 172.21.15.131:0/138641543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 msgr2=0x55d7d5acbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:59.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.573+0000 7f8e07e00700 1 --2- 172.21.15.131:0/138641543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 0x55d7d5acbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.573+0000 7f8e07e00700 1 -- 172.21.15.131:0/138641543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5aca530 msgr2=0x55d7d5aca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.573+0000 7f8e07e00700 1 --2- 172.21.15.131:0/138641543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5aca530 0x55d7d5aca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.573+0000 7f8e07e00700 1 -- 172.21.15.131:0/138641543 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7d586b3b0 con 0x55d7d5acc0f0 2024-05-10T05:35:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.574+0000 7f8e07e00700 1 --2- 172.21.15.131:0/138641543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad29f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8df400bd50 tx=0x7f8df40106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de56a52c9bafe60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.575+0000 7f8dfe000700 1 -- 172.21.15.131:0/138641543 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df400bea0 con 0x55d7d5acc0f0 2024-05-10T05:35:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.575+0000 7f8dfe000700 1 -- 172.21.15.131:0/138641543 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df4010e60 con 0x55d7d5acc0f0 2024-05-10T05:35:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.575+0000 7f8dfe000700 1 -- 172.21.15.131:0/138641543 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df400bea0 con 0x55d7d5acc0f0 2024-05-10T05:35:59.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.577+0000 7f8e277a2b80 1 -- 172.21.15.131:0/138641543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5acc0f0 msgr2=0x55d7d5ad29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:59.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.578+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/138641543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad29f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8df400bd50 tx=0x7f8df40106b0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.580+0000 7f8e277a2b80 1 -- 172.21.15.131:0/138641543 shutdown_connections 2024-05-10T05:35:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.580+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/138641543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5aca530 0x55d7d5aca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.580+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/138641543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad29f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.580+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/138641543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 0x55d7d5acbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.581+0000 7f8e277a2b80 1 -- 172.21.15.131:0/138641543 >> 172.21.15.131:0/138641543 conn(0x55d7d5a2ca60 msgr2=0x55d7d5ac5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.581+0000 7f8e277a2b80 1 -- 172.21.15.131:0/138641543 shutdown_connections 2024-05-10T05:35:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.581+0000 7f8e277a2b80 1 -- 172.21.15.131:0/138641543 wait complete. 2024-05-10T05:35:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.583+0000 7f8e277a2b80 1 -- start start 2024-05-10T05:35:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.584+0000 7f8e277a2b80 1 Processor -- start 2024-05-10T05:35:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.585+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5aca530 0x55d7d5ad4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.585+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 0x55d7d5ad4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.585+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5adbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.585+0000 7f8e277a2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d5986560 con 0x55d7d5acc0f0 2024-05-10T05:35:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.585+0000 7f8e277a2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d5986200 con 0x55d7d5aca530 2024-05-10T05:35:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.585+0000 7f8e277a2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d586d6a0 con 0x55d7d5acb7a0 2024-05-10T05:35:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.586+0000 7f8e07e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5aca530 0x55d7d5ad4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.586+0000 7f8e07e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5aca530 0x55d7d5ad4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45294/0 (socket says 172.21.15.131:45294) 2024-05-10T05:35:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.586+0000 7f8e07e00700 1 -- 172.21.15.131:0/2078084833 learned_addr learned my addr 172.21.15.131:0/2078084833 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.586+0000 7f8e05600700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8dec0a1180 0x7f8dec0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.587+0000 7f8e0ce00700 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 0x55d7d5ad4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.588+0000 7f8e07e00700 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5adbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.588+0000 7f8e0d800700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8dec0a1180 0x7f8dec0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.589+0000 7f8e277a2b80 10 client.0 _ll_register_callbacks cb 0x55d7d598fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.589+0000 7f8e0ce00700 1 -- 172.21.15.131:0/2078084833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5aca530 msgr2=0x55d7d5ad4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.589+0000 7f8e0ce00700 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5aca530 0x55d7d5ad4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.589+0000 7f8e0ce00700 1 -- 172.21.15.131:0/2078084833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 msgr2=0x55d7d5adbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.589+0000 7f8e0ce00700 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5adbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.590+0000 7f8e0ce00700 1 -- 172.21.15.131:0/2078084833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df400ba00 con 0x55d7d5acb7a0 2024-05-10T05:35:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.590+0000 7f8e07e00700 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5aca530 0x55d7d5ad4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:35:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.590+0000 7f8df2a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:35:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.591+0000 7f8e0ce00700 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 0x55d7d5ad4c40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8e00000c00 tx=0x7f8e00005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bd7dd6ce2b7237f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.592+0000 7f8df2a00700 1 -- 172.21.15.131:0/2078084833 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00012040 con 0x55d7d5acb7a0 2024-05-10T05:35:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.593+0000 7f8e277a2b80 1 -- 172.21.15.131:0/2078084833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7d5943270 con 0x55d7d5acb7a0 2024-05-10T05:35:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.593+0000 7f8df2a00700 1 -- 172.21.15.131:0/2078084833 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8e00003e40 con 0x55d7d5acb7a0 2024-05-10T05:35:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.593+0000 7f8e0d800700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8dec0a1180 0x7f8dec0a3690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8df8095a90 tx=0x7f8df8094d00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.593+0000 7f8e05600700 1 -- 172.21.15.131:0/4227188740 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8df80cbbd0 con 0x7f8e081097a0 2024-05-10T05:35:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.594+0000 7f8df2a00700 1 -- 172.21.15.131:0/2078084833 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e00012040 con 0x55d7d5acb7a0 2024-05-10T05:35:59.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.594+0000 7f8e277a2b80 10 client.25894 fetch_fsmap learned FSMap version 28 2024-05-10T05:35:59.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.594+0000 7f8e277a2b80 1 -- 172.21.15.131:0/2078084833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d7d5817b60 con 0x55d7d5acb7a0 2024-05-10T05:35:59.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.595+0000 7f8df2a00700 1 -- 172.21.15.131:0/2078084833 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7d5943270 con 0x55d7d5acb7a0 2024-05-10T05:35:59.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.595+0000 7f8df2a00700 1 -- 172.21.15.131:0/2078084833 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8e00016bb0 con 0x55d7d5acb7a0 2024-05-10T05:35:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.598+0000 7f8e277a2b80 10 client.25894 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:35:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.598+0000 7f8e277a2b80 10 client.25894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:35:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.598+0000 7f8e277a2b80 20 client.25894 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.598+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55d7d5ae8c90 0x55d7d5aeb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.600+0000 7f8e277a2b80 4 client.25894 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:35:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.600+0000 7f8e277a2b80 1 -- 172.21.15.131:0/2078084833 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7d5a99780 con 0x55d7d5ae8c90 2024-05-10T05:35:59.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.601+0000 7f8e07e00700 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55d7d5ae8c90 0x55d7d5aeb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.603+0000 7f8df2a00700 10 client.25894 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:35:59.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.603+0000 7f8e07e00700 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55d7d5ae8c90 0x55d7d5aeb0e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b59cefdfce27f30 server_cookie=f6509a837261c34e in_seq=0 out_seq=0 2024-05-10T05:35:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.605+0000 7f8df2a00700 1 -- 172.21.15.131:0/2078084833 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7d5a99780 con 0x55d7d5ae8c90 2024-05-10T05:35:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.606+0000 7f8df2a00700 10 client.25894 handle_command_reply: tid=0 2024-05-10T05:35:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.606+0000 7f8e277a2b80 1 client.25894 shutdown 2024-05-10T05:35:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.613+0000 7f8e277a2b80 1 -- 172.21.15.131:0/2078084833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 msgr2=0x55d7d5ad4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.613+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 0x55d7d5ad4c40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8e00000c00 tx=0x7f8e00005ee0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.613+0000 7f8e277a2b80 1 -- 172.21.15.131:0/2078084833 shutdown_connections 2024-05-10T05:35:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.613+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55d7d5ae8c90 0x55d7d5aeb0e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.613+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5adbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.613+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5aca530 0x55d7d5ad4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.613+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/2078084833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5acb7a0 0x55d7d5ad4c40 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8e00000c00 tx=0x7f8e00005ee0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.613+0000 7f8e277a2b80 1 -- 172.21.15.131:0/2078084833 >> 172.21.15.131:0/2078084833 conn(0x55d7d5a2ca60 msgr2=0x55d7d5ac9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.613+0000 7f8df2a00700 0 client.25894 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:35:59.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.616+0000 7f8e277a2b80 1 -- 172.21.15.131:0/2078084833 shutdown_connections 2024-05-10T05:35:59.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.616+0000 7f8e277a2b80 1 -- 172.21.15.131:0/2078084833 wait complete. 2024-05-10T05:35:59.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.618+0000 7f8e277a2b80 20 client.25894 trim_cache size 0 max 16384 2024-05-10T05:35:59.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:35:59 smithi142 ceph-mon[77667]: pgmap v364: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 75 op/s 2024-05-10T05:35:59.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.663+0000 7f8e277a2b80 1 Processor -- start 2024-05-10T05:35:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.663+0000 7f8e277a2b80 1 -- start start 2024-05-10T05:35:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.664+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ace9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.664+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5adbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.664+0000 7f8e07e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ace9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.664+0000 7f8e07e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ace9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41198/0 (socket says 172.21.15.131:41198) 2024-05-10T05:35:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.665+0000 7f8e07e00700 1 -- 172.21.15.131:0/3169510363 learned_addr learned my addr 172.21.15.131:0/3169510363 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.665+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/3169510363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5ad4e10 0x55d7d5aca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.665+0000 7f8e277a2b80 1 -- 172.21.15.131:0/3169510363 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d5ac1340 con 0x55d7d5acc0f0 2024-05-10T05:35:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.665+0000 7f8e277a2b80 1 -- 172.21.15.131:0/3169510363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d5ad55c0 con 0x55d7d5ad4e10 2024-05-10T05:35:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.665+0000 7f8e277a2b80 1 -- 172.21.15.131:0/3169510363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d5ad2660 con 0x55d7d5aca530 2024-05-10T05:35:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.665+0000 7f8e0ce00700 1 --2- 172.21.15.131:0/3169510363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5adbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.665+0000 7f8e07e00700 1 -- 172.21.15.131:0/3169510363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5ad4e10 msgr2=0x55d7d5aca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.665+0000 7f8e07e00700 1 --2- 172.21.15.131:0/3169510363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5ad4e10 0x55d7d5aca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.665+0000 7f8e07e00700 1 -- 172.21.15.131:0/3169510363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 msgr2=0x55d7d5adbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:59.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.665+0000 7f8e07e00700 1 --2- 172.21.15.131:0/3169510363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5adbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.666+0000 7f8e07e00700 1 -- 172.21.15.131:0/3169510363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7d59b0560 con 0x55d7d5aca530 2024-05-10T05:35:59.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.667+0000 7f8e07e00700 1 --2- 172.21.15.131:0/3169510363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ace9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8df4005eb0 tx=0x7f8df4001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe6d4c7579f0e53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.668+0000 7f8dfea00700 1 -- 172.21.15.131:0/3169510363 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004460 con 0x55d7d5aca530 2024-05-10T05:35:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.668+0000 7f8dfea00700 1 -- 172.21.15.131:0/3169510363 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df40054f0 con 0x55d7d5aca530 2024-05-10T05:35:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.669+0000 7f8dfea00700 1 -- 172.21.15.131:0/3169510363 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004460 con 0x55d7d5aca530 2024-05-10T05:35:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.670+0000 7f8e277a2b80 1 -- 172.21.15.131:0/3169510363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 msgr2=0x55d7d5ace9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.670+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/3169510363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ace9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8df4005eb0 tx=0x7f8df4001e50 comp rx=0 tx=0).stop 2024-05-10T05:35:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.670+0000 7f8e277a2b80 1 -- 172.21.15.131:0/3169510363 shutdown_connections 2024-05-10T05:35:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.670+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/3169510363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5ad4e10 0x55d7d5aca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.670+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/3169510363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ace9d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.670+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/3169510363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5adbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.670+0000 7f8e277a2b80 1 -- 172.21.15.131:0/3169510363 >> 172.21.15.131:0/3169510363 conn(0x55d7d5a2ca60 msgr2=0x55d7d5ad4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.670+0000 7f8e277a2b80 1 -- 172.21.15.131:0/3169510363 shutdown_connections 2024-05-10T05:35:59.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.670+0000 7f8e277a2b80 1 -- 172.21.15.131:0/3169510363 wait complete. 2024-05-10T05:35:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.671+0000 7f8e277a2b80 1 -- start start 2024-05-10T05:35:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.672+0000 7f8e277a2b80 1 Processor -- start 2024-05-10T05:35:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.672+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ad8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.672+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.672+0000 7f8e277a2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5ad4e10 0x55d7d5ada220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.672+0000 7f8e277a2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d5ada760 con 0x55d7d5acc0f0 2024-05-10T05:35:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.672+0000 7f8e277a2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d5ada8d0 con 0x55d7d5ad4e10 2024-05-10T05:35:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.672+0000 7f8e277a2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d5adaa90 con 0x55d7d5aca530 2024-05-10T05:35:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.672+0000 7f8e277a2b80 10 client.0 _ll_register_callbacks cb 0x55d7d598fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:35:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.673+0000 7f8e07e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ad8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.673+0000 7f8e07e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ad8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41204/0 (socket says 172.21.15.131:41204) 2024-05-10T05:35:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.673+0000 7f8e07e00700 1 -- 172.21.15.131:0/677275509 learned_addr learned my addr 172.21.15.131:0/677275509 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:35:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.673+0000 7f8e07e00700 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5ad4e10 0x55d7d5ada220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.674+0000 7f8e0ce00700 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.675+0000 7f8e07e00700 1 -- 172.21.15.131:0/677275509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5ad4e10 msgr2=0x55d7d5ada220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.675+0000 7f8e07e00700 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5ad4e10 0x55d7d5ada220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.675+0000 7f8e07e00700 1 -- 172.21.15.131:0/677275509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 msgr2=0x55d7d5ad9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:35:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.675+0000 7f8e07e00700 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.675+0000 7f8e07e00700 1 -- 172.21.15.131:0/677275509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df4022040 con 0x55d7d5aca530 2024-05-10T05:35:59.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.675+0000 7f8e0ce00700 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:35:59.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.676+0000 7f8e07e00700 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ad8e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8df4006010 tx=0x7f8df400b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea708ecd5da03284 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:35:59.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.676+0000 7f8df2a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:35:59.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.676+0000 7f8df2a00700 1 -- 172.21.15.131:0/677275509 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4005700 con 0x55d7d5aca530 2024-05-10T05:35:59.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.676+0000 7f8df2a00700 1 -- 172.21.15.131:0/677275509 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8df4003bc0 con 0x55d7d5aca530 2024-05-10T05:35:59.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.676+0000 7f8e277a2b80 1 -- 172.21.15.131:0/677275509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8df4010e60 con 0x55d7d5aca530 2024-05-10T05:35:59.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.677+0000 7f8df2a00700 1 -- 172.21.15.131:0/677275509 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8df4003d80 con 0x55d7d5aca530 2024-05-10T05:35:59.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.677+0000 7f8e277a2b80 10 client.25900 fetch_fsmap learned FSMap version 28 2024-05-10T05:35:59.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.677+0000 7f8e277a2b80 1 -- 172.21.15.131:0/677275509 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d7d586fcd0 con 0x55d7d5aca530 2024-05-10T05:35:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.677+0000 7f8df2a00700 1 -- 172.21.15.131:0/677275509 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df4010e60 con 0x55d7d5aca530 2024-05-10T05:35:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.678+0000 7f8df2a00700 1 -- 172.21.15.131:0/677275509 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8df4035070 con 0x55d7d5aca530 2024-05-10T05:35:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.678+0000 7f8e277a2b80 10 client.25900 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:35:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.678+0000 7f8e277a2b80 10 client.25900 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:35:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.678+0000 7f8e277a2b80 20 client.25900 populate_metadata read hostname 'smithi131' 2024-05-10T05:35:59.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.678+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55d7d5b3f160 0x55d7d5b41580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:35:59.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.678+0000 7f8e277a2b80 4 client.25900 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:35:59.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.678+0000 7f8e277a2b80 1 -- 172.21.15.131:0/677275509 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d7d59e2ec0 con 0x55d7d5b3f160 2024-05-10T05:35:59.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.679+0000 7f8e0ce00700 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55d7d5b3f160 0x55d7d5b41580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:35:59.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.680+0000 7f8df2a00700 10 client.25900 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:35:59.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.680+0000 7f8e0ce00700 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55d7d5b3f160 0x55d7d5b41580 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37d85be5db91ff11 server_cookie=433f1e3cbcb9c39 in_seq=0 out_seq=0 2024-05-10T05:35:59.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.680+0000 7f8df2a00700 1 -- 172.21.15.131:0/677275509 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d7d59e2ec0 con 0x55d7d5b3f160 2024-05-10T05:35:59.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.680+0000 7f8df2a00700 10 client.25900 handle_command_reply: tid=0 2024-05-10T05:35:59.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.681+0000 7f8e277a2b80 1 client.25900 shutdown 2024-05-10T05:35:59.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.681+0000 7f8e277a2b80 1 -- 172.21.15.131:0/677275509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 msgr2=0x55d7d5ad8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:59.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.681+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ad8e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8df4006010 tx=0x7f8df400b6e0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.681+0000 7f8e277a2b80 1 -- 172.21.15.131:0/677275509 shutdown_connections 2024-05-10T05:35:59.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.682+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55d7d5b3f160 0x55d7d5b41580 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.682+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7d5ad4e10 0x55d7d5ada220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.682+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7d5aca530 0x55d7d5ad8e00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.682+0000 7f8e277a2b80 1 --2- 172.21.15.131:0/677275509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7d5acc0f0 0x55d7d5ad9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.682+0000 7f8e277a2b80 1 -- 172.21.15.131:0/677275509 >> 172.21.15.131:0/677275509 conn(0x55d7d5a2ca60 msgr2=0x55d7d5acf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.682+0000 7f8df2a00700 0 client.25900 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:35:59.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.682+0000 7f8e277a2b80 1 -- 172.21.15.131:0/677275509 shutdown_connections 2024-05-10T05:35:59.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.682+0000 7f8e277a2b80 1 -- 172.21.15.131:0/677275509 wait complete. 2024-05-10T05:35:59.684 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:35:59.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.683+0000 7f8e277a2b80 20 client.25900 trim_cache size 0 max 16384 2024-05-10T05:35:59.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.684+0000 7f8e0e200700 1 -- 172.21.15.131:0/4227188740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8dec0a1180 msgr2=0x7f8dec0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:59.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.684+0000 7f8e0e200700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8dec0a1180 0x7f8dec0a3690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8df8095a90 tx=0x7f8df8094d00 comp rx=0 tx=0).stop 2024-05-10T05:35:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.684+0000 7f8e0e200700 1 -- 172.21.15.131:0/4227188740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e081097a0 msgr2=0x7f8e081a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:35:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.684+0000 7f8e0e200700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e081097a0 0x7f8e081a5900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8df800ef60 tx=0x7f8df800ef90 comp rx=0 tx=0).stop 2024-05-10T05:35:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.684+0000 7f8e0e200700 1 -- 172.21.15.131:0/4227188740 shutdown_connections 2024-05-10T05:35:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.684+0000 7f8e0e200700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8dec0a1180 0x7f8dec0a3690 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.684+0000 7f8e0e200700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e08108340 0x7f8e0819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.684+0000 7f8e0e200700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e081097a0 0x7f8e081a5900 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.684+0000 7f8e0e200700 1 --2- 172.21.15.131:0/4227188740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e08108c90 0x7f8e0819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:35:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.684+0000 7f8e0e200700 1 -- 172.21.15.131:0/4227188740 >> 172.21.15.131:0/4227188740 conn(0x7f8e0806ec70 msgr2=0x7f8e0810c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:35:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.685+0000 7f8e0e200700 1 -- 172.21.15.131:0/4227188740 shutdown_connections 2024-05-10T05:35:59.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:35:59.685+0000 7f8e0e200700 1 -- 172.21.15.131:0/4227188740 wait complete. 2024-05-10T05:35:59.739 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:36:00.150 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:00 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:36:00.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:00 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:36:00.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:00 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:36:00.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:00 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:36:00.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:00 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:36:00.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:00 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:36:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.592+0000 7f4827800700 1 -- 172.21.15.131:0/2993707519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4820109cd0 msgr2=0x7f4820110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.596+0000 7f4827800700 1 --2- 172.21.15.131:0/2993707519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4820109cd0 0x7f4820110580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f48100099c0 tx=0x7f4810009cd0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.598+0000 7f4827800700 1 -- 172.21.15.131:0/2993707519 shutdown_connections 2024-05-10T05:36:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.598+0000 7f4827800700 1 --2- 172.21.15.131:0/2993707519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820108210 0x7f4820108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.598+0000 7f4827800700 1 --2- 172.21.15.131:0/2993707519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4820109cd0 0x7f4820110580 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.598+0000 7f4827800700 1 --2- 172.21.15.131:0/2993707519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4820109330 0x7f4820109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.598+0000 7f4827800700 1 -- 172.21.15.131:0/2993707519 >> 172.21.15.131:0/2993707519 conn(0x7f4820077240 msgr2=0x7f4820106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:00.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.614+0000 7f4827800700 1 -- 172.21.15.131:0/2993707519 shutdown_connections 2024-05-10T05:36:00.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.617+0000 7f4827800700 1 -- 172.21.15.131:0/2993707519 wait complete. 2024-05-10T05:36:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:00 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:36:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:00 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:36:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:00 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:36:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.643+0000 7f4827800700 1 Processor -- start 2024-05-10T05:36:00.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.646+0000 7f4827800700 1 -- start start 2024-05-10T05:36:00.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.651+0000 7f4827800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820108210 0x7f482019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.651+0000 7f4827800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4820109330 0x7f482019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.651+0000 7f4827800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4820109cd0 0x7f48201a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.651+0000 7f4827800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48201a5fd0 con 0x7f4820108210 2024-05-10T05:36:00.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.652+0000 7f481fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4820109330 0x7f482019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.652+0000 7f4827800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48201a6110 con 0x7f4820109330 2024-05-10T05:36:00.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.652+0000 7f481fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4820109330 0x7f482019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45310/0 (socket says 172.21.15.131:45310) 2024-05-10T05:36:00.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.652+0000 7f481fe00700 1 -- 172.21.15.131:0/3069328087 learned_addr learned my addr 172.21.15.131:0/3069328087 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:00.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.652+0000 7f4827800700 1 -- 172.21.15.131:0/3069328087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48201a62a0 con 0x7f4820109cd0 2024-05-10T05:36:00.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.653+0000 7f481fe00700 1 -- 172.21.15.131:0/3069328087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4820109cd0 msgr2=0x7f48201a5a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:36:00.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.653+0000 7f481fe00700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4820109cd0 0x7f48201a5a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.653+0000 7f481fe00700 1 -- 172.21.15.131:0/3069328087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820108210 msgr2=0x7f482019e530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:36:00.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.653+0000 7f481fe00700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820108210 0x7f482019e530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.653+0000 7f481fe00700 1 -- 172.21.15.131:0/3069328087 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100095f0 con 0x7f4820109330 2024-05-10T05:36:00.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.654+0000 7f481fe00700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4820109330 0x7f482019ea70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c9b0 tx=0x7f480c00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:00.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.657+0000 7f481d600700 1 -- 172.21.15.131:0/3069328087 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c007b00 con 0x7f4820109330 2024-05-10T05:36:00.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.657+0000 7f481d600700 1 -- 172.21.15.131:0/3069328087 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f480c007c60 con 0x7f4820109330 2024-05-10T05:36:00.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.657+0000 7f481d600700 1 -- 172.21.15.131:0/3069328087 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c018660 con 0x7f4820109330 2024-05-10T05:36:00.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.665+0000 7f4827800700 1 -- 172.21.15.131:0/3069328087 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48201a6580 con 0x7f4820109330 2024-05-10T05:36:00.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.669+0000 7f4827800700 1 -- 172.21.15.131:0/3069328087 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48201a6ac0 con 0x7f4820109330 2024-05-10T05:36:00.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.678+0000 7f481d600700 1 -- 172.21.15.131:0/3069328087 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f480c00f450 con 0x7f4820109330 2024-05-10T05:36:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.680+0000 7f483f754b80 1 Processor -- start 2024-05-10T05:36:00.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.680+0000 7f483f754b80 1 -- start start 2024-05-10T05:36:00.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.681+0000 7f483f754b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 0x561b8e808af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.681+0000 7f483f754b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809570 0x561b8e809980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.681+0000 7f483f754b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809ec0 0x561b8e8107c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.681+0000 7f483f754b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e72f150 con 0x561b8e809570 2024-05-10T05:36:00.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.681+0000 7f483f754b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e72cb10 con 0x561b8e8086a0 2024-05-10T05:36:00.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.681+0000 7f483f754b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e728400 con 0x561b8e809ec0 2024-05-10T05:36:00.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.682+0000 7f4824e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809570 0x561b8e809980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.683+0000 7f4824e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809570 0x561b8e809980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41722/0 (socket says 172.21.15.131:41722) 2024-05-10T05:36:00.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.683+0000 7f4824e00700 1 -- 172.21.15.131:0/4264268207 learned_addr learned my addr 172.21.15.131:0/4264268207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:00.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.684+0000 7f481d600700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48040a1400 0x7f48040a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.684+0000 7f4824e00700 1 -- 172.21.15.131:0/4264268207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809ec0 msgr2=0x561b8e8107c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.684+0000 7f481fe00700 1 --2- 172.21.15.131:0/4264268207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809ec0 0x561b8e8107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.684+0000 7f481d600700 1 -- 172.21.15.131:0/3069328087 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f480c0cbb20 con 0x7f4820109330 2024-05-10T05:36:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.684+0000 7f4824e00700 1 --2- 172.21.15.131:0/4264268207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809ec0 0x561b8e8107c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.684+0000 7f4824e00700 1 -- 172.21.15.131:0/4264268207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 msgr2=0x561b8e808af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.684+0000 7f4825800700 1 --2- 172.21.15.131:0/4264268207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 0x561b8e808af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.684+0000 7f4824e00700 1 --2- 172.21.15.131:0/4264268207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 0x561b8e808af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.684+0000 7f4824e00700 1 -- 172.21.15.131:0/4264268207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b8e5a93b0 con 0x561b8e809570 2024-05-10T05:36:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.685+0000 7f481fe00700 1 --2- 172.21.15.131:0/4264268207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809ec0 0x561b8e8107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.685+0000 7f4824e00700 1 --2- 172.21.15.131:0/4264268207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809570 0x561b8e809980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f481800b450 tx=0x7f481800b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df42b03da983e01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.686+0000 7f4816000700 1 -- 172.21.15.131:0/4264268207 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818013410 con 0x561b8e809570 2024-05-10T05:36:00.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.686+0000 7f4816000700 1 -- 172.21.15.131:0/4264268207 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4818013a30 con 0x561b8e809570 2024-05-10T05:36:00.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.686+0000 7f4816000700 1 -- 172.21.15.131:0/4264268207 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818012ab0 con 0x561b8e809570 2024-05-10T05:36:00.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.687+0000 7f483f754b80 1 -- 172.21.15.131:0/4264268207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809570 msgr2=0x561b8e809980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:00.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.687+0000 7f483f754b80 1 --2- 172.21.15.131:0/4264268207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809570 0x561b8e809980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f481800b450 tx=0x7f481800b760 comp rx=0 tx=0).stop 2024-05-10T05:36:00.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.687+0000 7f483f754b80 1 -- 172.21.15.131:0/4264268207 shutdown_connections 2024-05-10T05:36:00.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.687+0000 7f483f754b80 1 --2- 172.21.15.131:0/4264268207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809ec0 0x561b8e8107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.687+0000 7f483f754b80 1 --2- 172.21.15.131:0/4264268207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 0x561b8e808af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.687+0000 7f483f754b80 1 --2- 172.21.15.131:0/4264268207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809570 0x561b8e809980 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.687+0000 7f483f754b80 1 -- 172.21.15.131:0/4264268207 >> 172.21.15.131:0/4264268207 conn(0x561b8e76aa60 msgr2=0x561b8e802fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:00.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.688+0000 7f483f754b80 1 -- 172.21.15.131:0/4264268207 shutdown_connections 2024-05-10T05:36:00.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.688+0000 7f483f754b80 1 -- 172.21.15.131:0/4264268207 wait complete. 2024-05-10T05:36:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.690+0000 7f483f754b80 1 -- start start 2024-05-10T05:36:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.690+0000 7f4825800700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48040a1400 0x7f48040a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.690+0000 7f483f754b80 1 Processor -- start 2024-05-10T05:36:00.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.691+0000 7f483f754b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e823770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.693+0000 7f483f754b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e823cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.693+0000 7f483f754b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e82ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.693+0000 7f483f754b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e6c4560 con 0x561b8e809ec0 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.693+0000 7f483f754b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e6c4200 con 0x561b8e809570 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.693+0000 7f483f754b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e5ab6a0 con 0x561b8e8086a0 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.693+0000 7f483f754b80 10 client.0 _ll_register_callbacks cb 0x561b8e6cdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.694+0000 7f4824e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e823770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.694+0000 7f4824e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e823770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41244/0 (socket says 172.21.15.131:41244) 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.694+0000 7f4824e00700 1 -- 172.21.15.131:0/3025821167 learned_addr learned my addr 172.21.15.131:0/3025821167 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.694+0000 7f481fe00700 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e823cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.694+0000 7f4824e00700 1 -- 172.21.15.131:0/3025821167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 msgr2=0x561b8e823cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:00.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.694+0000 7f4824e00700 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e823cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.694+0000 7f4824e00700 1 -- 172.21.15.131:0/3025821167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 msgr2=0x561b8e82ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:00.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.694+0000 7f4824e00700 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e82ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.694+0000 7f4824e00700 1 -- 172.21.15.131:0/3025821167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48180099d0 con 0x561b8e8086a0 2024-05-10T05:36:00.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.694+0000 7f481fe00700 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e823cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:00.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.695+0000 7f480aa00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:36:00.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.695+0000 7f4824e00700 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e823770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4818013670 tx=0x7f4818013750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef1298a09f7e0742 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:00.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.696+0000 7f480aa00700 1 -- 172.21.15.131:0/3025821167 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48180093d0 con 0x561b8e8086a0 2024-05-10T05:36:00.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.696+0000 7f483f754b80 1 -- 172.21.15.131:0/3025821167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b8e681270 con 0x561b8e8086a0 2024-05-10T05:36:00.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.696+0000 7f480aa00700 1 -- 172.21.15.131:0/3025821167 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4818012a40 con 0x561b8e8086a0 2024-05-10T05:36:00.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.697+0000 7f480aa00700 1 -- 172.21.15.131:0/3025821167 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48180093d0 con 0x561b8e8086a0 2024-05-10T05:36:00.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.697+0000 7f483f754b80 10 client.25906 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:00.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.697+0000 7f483f754b80 1 -- 172.21.15.131:0/3025821167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561b8e555b60 con 0x561b8e8086a0 2024-05-10T05:36:00.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.697+0000 7f480aa00700 1 -- 172.21.15.131:0/3025821167 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b8e681270 con 0x561b8e8086a0 2024-05-10T05:36:00.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.697+0000 7f480aa00700 1 -- 172.21.15.131:0/3025821167 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f481801eb40 con 0x561b8e8086a0 2024-05-10T05:36:00.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.699+0000 7f483f754b80 10 client.25906 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:00.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.699+0000 7f483f754b80 10 client.25906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:00.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.699+0000 7f483f754b80 20 client.25906 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:00.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.699+0000 7f483f754b80 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561b8e82f0d0 0x561b8e831520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.699+0000 7f483f754b80 4 client.25906 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:00.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.699+0000 7f483f754b80 1 -- 172.21.15.131:0/3025821167 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b8e7d7780 con 0x561b8e82f0d0 2024-05-10T05:36:00.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.700+0000 7f481fe00700 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561b8e82f0d0 0x561b8e831520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.701+0000 7f481fe00700 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561b8e82f0d0 0x561b8e831520 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf421fcabf8649d server_cookie=12d954dba93f2eef in_seq=0 out_seq=0 2024-05-10T05:36:00.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.701+0000 7f4825800700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48040a1400 0x7f48040a3910 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f48100038c0 tx=0x7f4810009cd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:00.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.701+0000 7f480aa00700 10 client.25906 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:00.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.701+0000 7f480aa00700 1 -- 172.21.15.131:0/3025821167 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b8e7d7780 con 0x561b8e82f0d0 2024-05-10T05:36:00.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.701+0000 7f480aa00700 10 client.25906 handle_command_reply: tid=0 2024-05-10T05:36:00.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.702+0000 7f483f754b80 1 client.25906 shutdown 2024-05-10T05:36:00.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.702+0000 7f483f754b80 1 -- 172.21.15.131:0/3025821167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 msgr2=0x561b8e823770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:00.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.702+0000 7f483f754b80 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e823770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4818013670 tx=0x7f4818013750 comp rx=0 tx=0).stop 2024-05-10T05:36:00.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.703+0000 7f483f754b80 1 -- 172.21.15.131:0/3025821167 shutdown_connections 2024-05-10T05:36:00.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.703+0000 7f483f754b80 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561b8e82f0d0 0x561b8e831520 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.703+0000 7f483f754b80 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e82ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.703+0000 7f483f754b80 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e823770 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.703+0000 7f483f754b80 1 --2- 172.21.15.131:0/3025821167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e823cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.703+0000 7f483f754b80 1 -- 172.21.15.131:0/3025821167 >> 172.21.15.131:0/3025821167 conn(0x561b8e76aa60 msgr2=0x561b8e802fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:00.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.703+0000 7f480aa00700 0 client.25906 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:00.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.706+0000 7f483f754b80 1 -- 172.21.15.131:0/3025821167 shutdown_connections 2024-05-10T05:36:00.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.706+0000 7f483f754b80 1 -- 172.21.15.131:0/3025821167 wait complete. 2024-05-10T05:36:00.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.706+0000 7f483f754b80 20 client.25906 trim_cache size 0 max 16384 2024-05-10T05:36:00.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.805+0000 7f483f754b80 1 Processor -- start 2024-05-10T05:36:00.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.808+0000 7f483f754b80 1 -- start start 2024-05-10T05:36:00.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.809+0000 7f483f754b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e7ff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.809+0000 7f483f754b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e82abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.812+0000 7f4824e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e7ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.813+0000 7f483f754b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e821d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.814+0000 7f483f754b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e82b220 con 0x561b8e809ec0 2024-05-10T05:36:00.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.814+0000 7f483f754b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e82b390 con 0x561b8e809570 2024-05-10T05:36:00.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.814+0000 7f483f754b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e803ef0 con 0x561b8e8086a0 2024-05-10T05:36:00.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.815+0000 7f481fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e82abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.816+0000 7f481fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e82abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45332/0 (socket says 172.21.15.131:45332) 2024-05-10T05:36:00.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.816+0000 7f481fe00700 1 -- 172.21.15.131:0/562543035 learned_addr learned my addr 172.21.15.131:0/562543035 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:00.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.818+0000 7f4825800700 1 --2- 172.21.15.131:0/562543035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e821d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.819+0000 7f4824e00700 1 -- 172.21.15.131:0/562543035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 msgr2=0x561b8e82abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:00.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.819+0000 7f4824e00700 1 --2- 172.21.15.131:0/562543035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e82abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.819+0000 7f4824e00700 1 -- 172.21.15.131:0/562543035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 msgr2=0x561b8e821d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:00.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.819+0000 7f4824e00700 1 --2- 172.21.15.131:0/562543035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e821d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.819+0000 7f4824e00700 1 -- 172.21.15.131:0/562543035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b8e6ee560 con 0x561b8e8086a0 2024-05-10T05:36:00.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.820+0000 7f4825800700 1 --2- 172.21.15.131:0/562543035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e821d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:00.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.820+0000 7f481fe00700 1 --2- 172.21.15.131:0/562543035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e82abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:00.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.821+0000 7f4824e00700 1 --2- 172.21.15.131:0/562543035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e7ff180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4818013640 tx=0x7f48180096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc17c94b2ae4f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:00.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.823+0000 7f4816a00700 1 -- 172.21.15.131:0/562543035 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818013870 con 0x561b8e8086a0 2024-05-10T05:36:00.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.823+0000 7f4816a00700 1 -- 172.21.15.131:0/562543035 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481801c590 con 0x561b8e8086a0 2024-05-10T05:36:00.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.825+0000 7f483f754b80 1 -- 172.21.15.131:0/562543035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 msgr2=0x561b8e7ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:00.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.828+0000 7f4816a00700 1 -- 172.21.15.131:0/562543035 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818013870 con 0x561b8e8086a0 2024-05-10T05:36:00.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.828+0000 7f483f754b80 1 --2- 172.21.15.131:0/562543035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e7ff180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4818013640 tx=0x7f48180096b0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.828+0000 7f483f754b80 1 -- 172.21.15.131:0/562543035 shutdown_connections 2024-05-10T05:36:00.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.828+0000 7f483f754b80 1 --2- 172.21.15.131:0/562543035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e821d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.828+0000 7f483f754b80 1 --2- 172.21.15.131:0/562543035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e8086a0 0x561b8e7ff180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.828+0000 7f483f754b80 1 --2- 172.21.15.131:0/562543035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e809570 0x561b8e82abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.828+0000 7f483f754b80 1 -- 172.21.15.131:0/562543035 >> 172.21.15.131:0/562543035 conn(0x561b8e76aa60 msgr2=0x561b8e80d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:00.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.829+0000 7f483f754b80 1 -- 172.21.15.131:0/562543035 shutdown_connections 2024-05-10T05:36:00.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.829+0000 7f483f754b80 1 -- 172.21.15.131:0/562543035 wait complete. 2024-05-10T05:36:00.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.831+0000 7f483f754b80 1 -- start start 2024-05-10T05:36:00.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.833+0000 7f483f754b80 1 Processor -- start 2024-05-10T05:36:00.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.835+0000 7f483f754b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 0x561b8e827ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.841+0000 7f4824e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 0x561b8e827ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.841+0000 7f4824e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 0x561b8e827ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45342/0 (socket says 172.21.15.131:45342) 2024-05-10T05:36:00.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.841+0000 7f4824e00700 1 -- 172.21.15.131:0/2978297120 learned_addr learned my addr 172.21.15.131:0/2978297120 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:00.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.841+0000 7f483f754b80 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809570 0x561b8e8280e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.844+0000 7f481fe00700 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809570 0x561b8e8280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.844+0000 7f483f754b80 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e883570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.845+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e8287d0 con 0x561b8e809ec0 2024-05-10T05:36:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.845+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e883ae0 con 0x561b8e8086a0 2024-05-10T05:36:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.845+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8e883ca0 con 0x561b8e809570 2024-05-10T05:36:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.845+0000 7f483f754b80 10 client.0 _ll_register_callbacks cb 0x561b8e6cdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.849+0000 7f4824e00700 1 -- 172.21.15.131:0/2978297120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809570 msgr2=0x561b8e8280e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.849+0000 7f4824e00700 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809570 0x561b8e8280e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.849+0000 7f4824e00700 1 -- 172.21.15.131:0/2978297120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 msgr2=0x561b8e883570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:00.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.851+0000 7f4825800700 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e883570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.851+0000 7f4824e00700 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e883570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.851+0000 7f4824e00700 1 -- 172.21.15.131:0/2978297120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48201a6ac0 con 0x561b8e8086a0 2024-05-10T05:36:00.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.851+0000 7f481fe00700 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809570 0x561b8e8280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:36:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.853+0000 7f4825800700 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e883570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.853+0000 7f480aa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:36:00.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.853+0000 7f4824e00700 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 0x561b8e827ba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f481800bfd0 tx=0x7f48180125e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d4cabb07405b4b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.859+0000 7f480aa00700 1 -- 172.21.15.131:0/2978297120 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481801c990 con 0x561b8e8086a0 2024-05-10T05:36:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.863+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4818013a30 con 0x561b8e8086a0 2024-05-10T05:36:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.863+0000 7f480aa00700 1 -- 172.21.15.131:0/2978297120 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4818012a40 con 0x561b8e8086a0 2024-05-10T05:36:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.866+0000 7f480aa00700 1 -- 172.21.15.131:0/2978297120 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f481801c990 con 0x561b8e8086a0 2024-05-10T05:36:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.866+0000 7f483f754b80 10 client.25892 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.866+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561b8e555b60 con 0x561b8e8086a0 2024-05-10T05:36:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.867+0000 7f480aa00700 1 -- 172.21.15.131:0/2978297120 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4818013a30 con 0x561b8e8086a0 2024-05-10T05:36:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.867+0000 7f480aa00700 1 -- 172.21.15.131:0/2978297120 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f481800b480 con 0x561b8e8086a0 2024-05-10T05:36:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.871+0000 7f483f754b80 10 client.25892 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.871+0000 7f483f754b80 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:00.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.871+0000 7f483f754b80 20 client.25892 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:00.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.871+0000 7f483f754b80 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561b8e885360 0x561b8e887780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:00.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.871+0000 7f483f754b80 4 client.25892 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:36:00.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.871+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561b8e720ec0 con 0x561b8e885360 2024-05-10T05:36:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.872+0000 7f481fe00700 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561b8e885360 0x561b8e887780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.872+0000 7f480aa00700 10 client.25892 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.873+0000 7f481fe00700 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561b8e885360 0x561b8e887780 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8b83c858c671b0 server_cookie=f6beca44f4ab962e in_seq=0 out_seq=0 2024-05-10T05:36:00.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.874+0000 7f480aa00700 1 -- 172.21.15.131:0/2978297120 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561b8e720ec0 con 0x561b8e885360 2024-05-10T05:36:00.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.874+0000 7f480aa00700 10 client.25892 handle_command_reply: tid=0 2024-05-10T05:36:00.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.874+0000 7f483f754b80 1 client.25892 shutdown 2024-05-10T05:36:00.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.877+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 msgr2=0x561b8e827ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:00.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.877+0000 7f483f754b80 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 0x561b8e827ba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f481800bfd0 tx=0x7f48180125e0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.877+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 shutdown_connections 2024-05-10T05:36:00.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.877+0000 7f483f754b80 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561b8e885360 0x561b8e887780 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.877+0000 7f483f754b80 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561b8e809ec0 0x561b8e883570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.877+0000 7f483f754b80 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b8e8086a0 0x561b8e827ba0 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f481800bfd0 tx=0x7f48180125e0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.877+0000 7f483f754b80 1 --2- 172.21.15.131:0/2978297120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b8e809570 0x561b8e8280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.877+0000 7f480aa00700 0 client.25892 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:00.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.877+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 >> 172.21.15.131:0/2978297120 conn(0x561b8e76aa60 msgr2=0x561b8e821d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:00.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.879+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 shutdown_connections 2024-05-10T05:36:00.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.880+0000 7f483f754b80 1 -- 172.21.15.131:0/2978297120 wait complete. 2024-05-10T05:36:00.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.885+0000 7f483f754b80 20 client.25892 trim_cache size 0 max 16384 2024-05-10T05:36:00.886 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:36:00.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.888+0000 7f4827800700 1 -- 172.21.15.131:0/3069328087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48040a1400 msgr2=0x7f48040a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:00.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.889+0000 7f4827800700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48040a1400 0x7f48040a3910 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f48100038c0 tx=0x7f4810009cd0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.889+0000 7f4827800700 1 -- 172.21.15.131:0/3069328087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4820109330 msgr2=0x7f482019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:00.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.889+0000 7f4827800700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4820109330 0x7f482019ea70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c9b0 tx=0x7f480c00cd70 comp rx=0 tx=0).stop 2024-05-10T05:36:00.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.889+0000 7f4827800700 1 -- 172.21.15.131:0/3069328087 shutdown_connections 2024-05-10T05:36:00.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.889+0000 7f4827800700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f48040a1400 0x7f48040a3910 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.889+0000 7f4827800700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820108210 0x7f482019e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.889+0000 7f4827800700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4820109cd0 0x7f48201a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.889+0000 7f4827800700 1 --2- 172.21.15.131:0/3069328087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4820109330 0x7f482019ea70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:00.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.890+0000 7f4827800700 1 -- 172.21.15.131:0/3069328087 >> 172.21.15.131:0/3069328087 conn(0x7f4820077240 msgr2=0x7f482010c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:00.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.890+0000 7f4827800700 1 -- 172.21.15.131:0/3069328087 shutdown_connections 2024-05-10T05:36:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:00.899+0000 7f4827800700 1 -- 172.21.15.131:0/3069328087 wait complete. 2024-05-10T05:36:01.465 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:01 smithi131 ceph-mon[37264]: pgmap v365: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 237 MiB/s wr, 72 op/s 2024-05-10T05:36:01.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:01 smithi157 ceph-mon[78427]: pgmap v365: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 237 MiB/s wr, 72 op/s 2024-05-10T05:36:01.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:01 smithi142 ceph-mon[77667]: pgmap v365: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 237 MiB/s wr, 72 op/s 2024-05-10T05:36:01.980 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:36:01.980 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:36:02.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.643+0000 7ff569e00700 1 -- 172.21.15.131:0/1601478490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff564109750 msgr2=0x7ff564109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:02.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.643+0000 7ff569e00700 1 --2- 172.21.15.131:0/1601478490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff564109750 0x7ff564109b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00cac0 tx=0x7ff55c00cdd0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.648+0000 7ff569e00700 1 -- 172.21.15.131:0/1601478490 shutdown_connections 2024-05-10T05:36:02.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.648+0000 7ff569e00700 1 --2- 172.21.15.131:0/1601478490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff56410a0a0 0x7ff5641109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.648+0000 7ff569e00700 1 --2- 172.21.15.131:0/1601478490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5641085e0 0x7ff5641089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.648+0000 7ff569e00700 1 --2- 172.21.15.131:0/1601478490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff564109750 0x7ff564109b60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.649+0000 7ff569e00700 1 -- 172.21.15.131:0/1601478490 >> 172.21.15.131:0/1601478490 conn(0x7ff56406ea00 msgr2=0x7ff564070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:02.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.649+0000 7ff569e00700 1 -- 172.21.15.131:0/1601478490 shutdown_connections 2024-05-10T05:36:02.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.649+0000 7ff569e00700 1 -- 172.21.15.131:0/1601478490 wait complete. 2024-05-10T05:36:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.653+0000 7ff569e00700 1 Processor -- start 2024-05-10T05:36:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.654+0000 7ff569e00700 1 -- start start 2024-05-10T05:36:02.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.655+0000 7ff569e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5641085e0 0x7ff56419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.655+0000 7ff562a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5641085e0 0x7ff56419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.656+0000 7ff562a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5641085e0 0x7ff56419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41758/0 (socket says 172.21.15.131:41758) 2024-05-10T05:36:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.656+0000 7ff562a00700 1 -- 172.21.15.131:0/2438742662 learned_addr learned my addr 172.21.15.131:0/2438742662 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.656+0000 7ff569e00700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff564109750 0x7ff56419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.656+0000 7ff569e00700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff56410a0a0 0x7ff5641a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.656+0000 7ff562000700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff564109750 0x7ff56419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.656+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641a5ec0 con 0x7ff5641085e0 2024-05-10T05:36:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.656+0000 7ff563400700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff56410a0a0 0x7ff5641a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.656+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641a6030 con 0x7ff564109750 2024-05-10T05:36:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.656+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641a61f0 con 0x7ff56410a0a0 2024-05-10T05:36:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.657+0000 7ff562000700 1 -- 172.21.15.131:0/2438742662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff56410a0a0 msgr2=0x7ff5641a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.657+0000 7ff562000700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff56410a0a0 0x7ff5641a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.657+0000 7ff562000700 1 -- 172.21.15.131:0/2438742662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5641085e0 msgr2=0x7ff56419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.657+0000 7ff562000700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5641085e0 0x7ff56419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.657+0000 7ff562000700 1 -- 172.21.15.131:0/2438742662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c009d00 con 0x7ff564109750 2024-05-10T05:36:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.658+0000 7ff562000700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff564109750 0x7ff56419e970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00ca10 tx=0x7ff55c006080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.658+0000 7ff568a00700 1 -- 172.21.15.131:0/2438742662 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00b050 con 0x7ff564109750 2024-05-10T05:36:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.658+0000 7ff568a00700 1 -- 172.21.15.131:0/2438742662 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c00a5b0 con 0x7ff564109750 2024-05-10T05:36:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.658+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5641a6470 con 0x7ff564109750 2024-05-10T05:36:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.658+0000 7ff568a00700 1 -- 172.21.15.131:0/2438742662 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c005320 con 0x7ff564109750 2024-05-10T05:36:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.659+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5641a68f0 con 0x7ff564109750 2024-05-10T05:36:02.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.664+0000 7ff568a00700 1 -- 172.21.15.131:0/2438742662 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff55c00c530 con 0x7ff564109750 2024-05-10T05:36:02.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.666+0000 7ff581d0cb80 1 Processor -- start 2024-05-10T05:36:02.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.666+0000 7ff581d0cb80 1 -- start start 2024-05-10T05:36:02.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c4061f0 0x55918c406640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c3fc380 0x55918c3fc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c40d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff563400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c4061f0 0x55918c406640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff563400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c4061f0 0x55918c406640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41284/0 (socket says 172.21.15.131:41284) 2024-05-10T05:36:02.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff568a00700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5480a1240 0x7ff5480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff563400700 1 -- 172.21.15.131:0/504265344 learned_addr learned my addr 172.21.15.131:0/504265344 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:02.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff581d0cb80 1 -- 172.21.15.131:0/504265344 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c3231d0 con 0x55918c3fc380 2024-05-10T05:36:02.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff581d0cb80 1 -- 172.21.15.131:0/504265344 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c320b90 con 0x55918c3fccd0 2024-05-10T05:36:02.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff581d0cb80 1 -- 172.21.15.131:0/504265344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c31c480 con 0x55918c4061f0 2024-05-10T05:36:02.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.667+0000 7ff562000700 1 --2- 172.21.15.131:0/504265344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c40d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.668+0000 7ff562a00700 1 --2- 172.21.15.131:0/504265344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c3fc380 0x55918c3fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.668+0000 7ff562000700 1 -- 172.21.15.131:0/504265344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c4061f0 msgr2=0x55918c406640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:02.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.668+0000 7ff562000700 1 --2- 172.21.15.131:0/504265344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c4061f0 0x55918c406640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.668+0000 7ff562000700 1 -- 172.21.15.131:0/504265344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c3fc380 msgr2=0x55918c3fc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:02.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.668+0000 7ff562000700 1 --2- 172.21.15.131:0/504265344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c3fc380 0x55918c3fc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.668+0000 7ff562000700 1 -- 172.21.15.131:0/504265344 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55918c19d430 con 0x55918c3fccd0 2024-05-10T05:36:02.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.668+0000 7ff563400700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5480a1240 0x7ff5480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.669+0000 7ff568a00700 1 -- 172.21.15.131:0/2438742662 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff55c0cfea0 con 0x7ff564109750 2024-05-10T05:36:02.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.669+0000 7ff562000700 1 --2- 172.21.15.131:0/504265344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c40d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0cd9f0 tx=0x7ff55c0a2150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4490c29be56f2060 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:02.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.669+0000 7ff54fe00700 1 -- 172.21.15.131:0/504265344 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c009b80 con 0x55918c3fccd0 2024-05-10T05:36:02.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.669+0000 7ff54fe00700 1 -- 172.21.15.131:0/504265344 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c09d380 con 0x55918c3fccd0 2024-05-10T05:36:02.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.669+0000 7ff54fe00700 1 -- 172.21.15.131:0/504265344 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0a5770 con 0x55918c3fccd0 2024-05-10T05:36:02.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.670+0000 7ff581d0cb80 1 -- 172.21.15.131:0/504265344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 msgr2=0x55918c40d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:02.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.670+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/504265344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c40d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0cd9f0 tx=0x7ff55c0a2150 comp rx=0 tx=0).stop 2024-05-10T05:36:02.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.670+0000 7ff563400700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5480a1240 0x7ff5480a3750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55918c3fd6b0 tx=0x7ff5500047c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:02.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.670+0000 7ff581d0cb80 1 -- 172.21.15.131:0/504265344 shutdown_connections 2024-05-10T05:36:02.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.670+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/504265344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c40d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.670+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/504265344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c4061f0 0x55918c406640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.670+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/504265344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c3fc380 0x55918c3fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.670+0000 7ff581d0cb80 1 -- 172.21.15.131:0/504265344 >> 172.21.15.131:0/504265344 conn(0x55918c35eae0 msgr2=0x55918c3f7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:02.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.670+0000 7ff581d0cb80 1 -- 172.21.15.131:0/504265344 shutdown_connections 2024-05-10T05:36:02.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.670+0000 7ff581d0cb80 1 -- 172.21.15.131:0/504265344 wait complete. 2024-05-10T05:36:02.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.671+0000 7ff581d0cb80 1 -- start start 2024-05-10T05:36:02.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.671+0000 7ff581d0cb80 1 Processor -- start 2024-05-10T05:36:02.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.672+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 0x55918c40ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.672+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c40ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.672+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c4061f0 0x55918c415ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.672+0000 7ff581d0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c2b85e0 con 0x55918c4061f0 2024-05-10T05:36:02.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.672+0000 7ff581d0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c2b8280 con 0x55918c3fccd0 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.672+0000 7ff581d0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c19f720 con 0x55918c3fc380 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.672+0000 7ff581d0cb80 10 client.0 _ll_register_callbacks cb 0x55918c2c1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff563400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c4061f0 0x55918c415ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff563400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c4061f0 0x55918c415ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41780/0 (socket says 172.21.15.131:41780) 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff563400700 1 -- 172.21.15.131:0/3083082179 learned_addr learned my addr 172.21.15.131:0/3083082179 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff562000700 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c40ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff562a00700 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 0x55918c40ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff563400700 1 -- 172.21.15.131:0/3083082179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 msgr2=0x55918c40ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff563400700 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 0x55918c40ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff563400700 1 -- 172.21.15.131:0/3083082179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 msgr2=0x55918c40ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff563400700 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c40ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff563400700 1 -- 172.21.15.131:0/3083082179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5641a68f0 con 0x55918c4061f0 2024-05-10T05:36:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.673+0000 7ff562000700 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c40ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:02.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.674+0000 7ff563400700 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c4061f0 0x55918c415ea0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff550010730 tx=0x7ff550004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca2b30aa8a7d836 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:02.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.674+0000 7ff54cc00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:36:02.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.674+0000 7ff54cc00700 1 -- 172.21.15.131:0/3083082179 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff550004a30 con 0x55918c4061f0 2024-05-10T05:36:02.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.674+0000 7ff54cc00700 1 -- 172.21.15.131:0/3083082179 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff550004b90 con 0x55918c4061f0 2024-05-10T05:36:02.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.676+0000 7ff581d0cb80 1 -- 172.21.15.131:0/3083082179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55918c2752f0 con 0x55918c4061f0 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.676+0000 7ff54cc00700 1 -- 172.21.15.131:0/3083082179 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff550004a30 con 0x55918c4061f0 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.676+0000 7ff581d0cb80 10 client.15855 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.676+0000 7ff581d0cb80 1 -- 172.21.15.131:0/3083082179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55918c149be0 con 0x55918c4061f0 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.677+0000 7ff54cc00700 1 -- 172.21.15.131:0/3083082179 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55918c2752f0 con 0x55918c4061f0 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.677+0000 7ff54cc00700 1 -- 172.21.15.131:0/3083082179 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff550026030 con 0x55918c4061f0 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.677+0000 7ff581d0cb80 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.677+0000 7ff581d0cb80 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.677+0000 7ff581d0cb80 20 client.15855 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.677+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55918c41a700 0x55918c41cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.677+0000 7ff581d0cb80 4 client.15855 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.677+0000 7ff581d0cb80 1 -- 172.21.15.131:0/3083082179 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55918c3cb800 con 0x55918c41a700 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.678+0000 7ff562a00700 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55918c41a700 0x55918c41cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.679+0000 7ff562a00700 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55918c41a700 0x55918c41cb50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d9be3fe94d7d94 server_cookie=9d5630d295e882ed in_seq=0 out_seq=0 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.679+0000 7ff54cc00700 10 client.15855 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.680+0000 7ff54cc00700 1 -- 172.21.15.131:0/3083082179 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55918c3cb800 con 0x55918c41a700 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.680+0000 7ff54cc00700 10 client.15855 handle_command_reply: tid=0 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.680+0000 7ff581d0cb80 1 client.15855 shutdown 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.680+0000 7ff581d0cb80 1 -- 172.21.15.131:0/3083082179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c4061f0 msgr2=0x55918c415ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.680+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c4061f0 0x55918c415ea0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff550010730 tx=0x7ff550004750 comp rx=0 tx=0).stop 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.681+0000 7ff581d0cb80 1 -- 172.21.15.131:0/3083082179 shutdown_connections 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.681+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55918c41a700 0x55918c41cb50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.681+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c4061f0 0x55918c415ea0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.681+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 0x55918c40ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.681+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/3083082179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c40ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.681+0000 7ff581d0cb80 1 -- 172.21.15.131:0/3083082179 >> 172.21.15.131:0/3083082179 conn(0x55918c35eae0 msgr2=0x55918c3f7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.681+0000 7ff54cc00700 0 client.15855 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:02.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.681+0000 7ff581d0cb80 1 -- 172.21.15.131:0/3083082179 shutdown_connections 2024-05-10T05:36:02.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.681+0000 7ff581d0cb80 1 -- 172.21.15.131:0/3083082179 wait complete. 2024-05-10T05:36:02.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.682+0000 7ff581d0cb80 20 client.15855 trim_cache size 0 max 16384 2024-05-10T05:36:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.727+0000 7ff581d0cb80 1 Processor -- start 2024-05-10T05:36:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.727+0000 7ff581d0cb80 1 -- start start 2024-05-10T05:36:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.728+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 0x55918c40cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.728+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fc380 0x55918c415e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.728+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fccd0 0x55918c40d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.728+0000 7ff581d0cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c41a220 con 0x55918c35eae0 2024-05-10T05:36:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.728+0000 7ff581d0cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c40f460 con 0x55918c3fc380 2024-05-10T05:36:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.728+0000 7ff581d0cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c3f1840 con 0x55918c3fccd0 2024-05-10T05:36:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff563400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fccd0 0x55918c40d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff563400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fccd0 0x55918c40d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41306/0 (socket says 172.21.15.131:41306) 2024-05-10T05:36:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff563400700 1 -- 172.21.15.131:0/1896185710 learned_addr learned my addr 172.21.15.131:0/1896185710 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff562000700 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fc380 0x55918c415e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff562a00700 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 0x55918c40cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff563400700 1 -- 172.21.15.131:0/1896185710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fc380 msgr2=0x55918c415e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff563400700 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fc380 0x55918c415e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff563400700 1 -- 172.21.15.131:0/1896185710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 msgr2=0x55918c40cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff563400700 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 0x55918c40cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff563400700 1 -- 172.21.15.131:0/1896185710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55918c2e25e0 con 0x55918c3fccd0 2024-05-10T05:36:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff562000700 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fc380 0x55918c415e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:36:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.730+0000 7ff562a00700 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 0x55918c40cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:02.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.735+0000 7ff563400700 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fccd0 0x55918c40d4f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff55000d550 tx=0x7ff55000dad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1ac5c9b6f39cf6c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:02.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.735+0000 7ff559200700 1 -- 172.21.15.131:0/1896185710 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff550004a30 con 0x55918c3fccd0 2024-05-10T05:36:02.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.736+0000 7ff559200700 1 -- 172.21.15.131:0/1896185710 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff550013ed0 con 0x55918c3fccd0 2024-05-10T05:36:02.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.736+0000 7ff559200700 1 -- 172.21.15.131:0/1896185710 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff550020ba0 con 0x55918c3fccd0 2024-05-10T05:36:02.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.737+0000 7ff581d0cb80 1 -- 172.21.15.131:0/1896185710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fccd0 msgr2=0x55918c40d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:02.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.737+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fccd0 0x55918c40d4f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff55000d550 tx=0x7ff55000dad0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.737+0000 7ff581d0cb80 1 -- 172.21.15.131:0/1896185710 shutdown_connections 2024-05-10T05:36:02.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.737+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 0x55918c40cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.737+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fccd0 0x55918c40d4f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.737+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/1896185710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fc380 0x55918c415e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.737+0000 7ff581d0cb80 1 -- 172.21.15.131:0/1896185710 >> 172.21.15.131:0/1896185710 conn(0x55918c3f34d0 msgr2=0x55918c4098a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:02.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.738+0000 7ff581d0cb80 1 -- 172.21.15.131:0/1896185710 shutdown_connections 2024-05-10T05:36:02.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.738+0000 7ff581d0cb80 1 -- 172.21.15.131:0/1896185710 wait complete. 2024-05-10T05:36:02.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.740+0000 7ff581d0cb80 1 -- start start 2024-05-10T05:36:02.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.743+0000 7ff581d0cb80 1 Processor -- start 2024-05-10T05:36:02.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.745+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 0x55918c415750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.745+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 0x55918c415c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.745+0000 7ff581d0cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c413e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.746+0000 7ff562000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 0x55918c415c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.746+0000 7ff562000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 0x55918c415c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41318/0 (socket says 172.21.15.131:41318) 2024-05-10T05:36:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.750+0000 7ff562000700 1 -- 172.21.15.131:0/742222106 learned_addr learned my addr 172.21.15.131:0/742222106 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.750+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c414430 con 0x55918c35eae0 2024-05-10T05:36:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.750+0000 7ff563400700 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c413e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.750+0000 7ff562a00700 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 0x55918c415750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.750+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c4145a0 con 0x55918c3fccd0 2024-05-10T05:36:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.750+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918c414760 con 0x55918c3fc380 2024-05-10T05:36:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.750+0000 7ff581d0cb80 10 client.0 _ll_register_callbacks cb 0x55918c2c1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.753+0000 7ff562000700 1 -- 172.21.15.131:0/742222106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 msgr2=0x55918c413e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.753+0000 7ff562000700 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c413e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.753+0000 7ff562000700 1 -- 172.21.15.131:0/742222106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 msgr2=0x55918c415750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.753+0000 7ff562000700 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 0x55918c415750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.753+0000 7ff562000700 1 -- 172.21.15.131:0/742222106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55918c149be0 con 0x55918c3fc380 2024-05-10T05:36:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.753+0000 7ff562a00700 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 0x55918c415750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.753+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff55c09d380 con 0x55918c3fc380 2024-05-10T05:36:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.754+0000 7ff563400700 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c413e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.754+0000 7ff562000700 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 0x55918c415c90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0095d0 tx=0x7ff55c09dbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58b1e09a295b5dbd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.756+0000 7ff54cc00700 10 client.25930 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:36:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.756+0000 7ff54cc00700 1 -- 172.21.15.131:0/742222106 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0a54e0 con 0x55918c3fc380 2024-05-10T05:36:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.756+0000 7ff54cc00700 1 -- 172.21.15.131:0/742222106 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff55c0a5770 con 0x55918c3fc380 2024-05-10T05:36:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.756+0000 7ff54cc00700 1 -- 172.21.15.131:0/742222106 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff55c0a11b0 con 0x55918c3fc380 2024-05-10T05:36:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.756+0000 7ff581d0cb80 10 client.25930 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.756+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=28}) v3 -- 0x55918c1a1d50 con 0x55918c3fc380 2024-05-10T05:36:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.758+0000 7ff54cc00700 1 -- 172.21.15.131:0/742222106 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff55c007b40 con 0x55918c3fc380 2024-05-10T05:36:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.758+0000 7ff581d0cb80 10 client.25930 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.758+0000 7ff581d0cb80 10 client.25930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.758+0000 7ff581d0cb80 20 client.25930 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.758+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55918c4061f0 0x55918c4738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.758+0000 7ff581d0cb80 4 client.25930 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:36:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.758+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55918c314f40 con 0x55918c4061f0 2024-05-10T05:36:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.759+0000 7ff562a00700 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55918c4061f0 0x55918c4738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.760+0000 7ff54cc00700 10 client.25930 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:02.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.760+0000 7ff562a00700 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55918c4061f0 0x55918c4738e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87a00c961aeb4e0 server_cookie=298f88edf6900cba in_seq=0 out_seq=0 2024-05-10T05:36:02.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.761+0000 7ff54cc00700 1 -- 172.21.15.131:0/742222106 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55918c314f40 con 0x55918c4061f0 2024-05-10T05:36:02.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.761+0000 7ff54cc00700 10 client.25930 handle_command_reply: tid=0 2024-05-10T05:36:02.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.761+0000 7ff581d0cb80 1 client.25930 shutdown 2024-05-10T05:36:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.765+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 msgr2=0x55918c415c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.765+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 0x55918c415c90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0095d0 tx=0x7ff55c09dbc0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.766+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 shutdown_connections 2024-05-10T05:36:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.766+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55918c4061f0 0x55918c4738e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.766+0000 7ff54cc00700 0 client.25930 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.766+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55918c35eae0 0x55918c415750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.766+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918c3fccd0 0x55918c413e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.766+0000 7ff581d0cb80 1 --2- 172.21.15.131:0/742222106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55918c3fc380 0x55918c415c90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.766+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 >> 172.21.15.131:0/742222106 conn(0x55918c3f34d0 msgr2=0x55918c4098a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.768+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 shutdown_connections 2024-05-10T05:36:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.769+0000 7ff581d0cb80 1 -- 172.21.15.131:0/742222106 wait complete. 2024-05-10T05:36:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.770+0000 7ff581d0cb80 20 client.25930 trim_cache size 0 max 16384 2024-05-10T05:36:02.774 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:36:02.774 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:36:02.774 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "d9fc0556-f5cb-4b21-917d-5cb1e7cea838", 2024-05-10T05:36:02.775 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:36:02.775 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:36:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.779+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5480a1240 msgr2=0x7ff5480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.779+0000 7ff569e00700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5480a1240 0x7ff5480a3750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55918c3fd6b0 tx=0x7ff5500047c0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.779+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff564109750 msgr2=0x7ff56419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.779+0000 7ff569e00700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff564109750 0x7ff56419e970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00ca10 tx=0x7ff55c006080 comp rx=0 tx=0).stop 2024-05-10T05:36:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.780+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 shutdown_connections 2024-05-10T05:36:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.780+0000 7ff569e00700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff5480a1240 0x7ff5480a3750 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.780+0000 7ff569e00700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5641085e0 0x7ff56419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.780+0000 7ff569e00700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff56410a0a0 0x7ff5641a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.780+0000 7ff569e00700 1 --2- 172.21.15.131:0/2438742662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff564109750 0x7ff56419e970 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.780+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 >> 172.21.15.131:0/2438742662 conn(0x7ff56406ea00 msgr2=0x7ff56410c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:02.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.783+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 shutdown_connections 2024-05-10T05:36:02.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:02.783+0000 7ff569e00700 1 -- 172.21.15.131:0/2438742662 wait complete. 2024-05-10T05:36:02.860 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:36:03.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.340+0000 7fa0c4000700 1 -- 172.21.15.131:0/1750118731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b40a6ea0 msgr2=0x7fa0b40ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:03.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.340+0000 7fa0c4000700 1 --2- 172.21.15.131:0/1750118731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b40a6ea0 0x7fa0b40ab7b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc064500 tx=0x7fa0bc069d30 comp rx=0 tx=0).stop 2024-05-10T05:36:03.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.340+0000 7fa0c4000700 1 -- 172.21.15.131:0/1750118731 shutdown_connections 2024-05-10T05:36:03.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.340+0000 7fa0c4000700 1 --2- 172.21.15.131:0/1750118731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40a5570 0x7fa0b40a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.340+0000 7fa0c4000700 1 --2- 172.21.15.131:0/1750118731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b40a6ea0 0x7fa0b40ab7b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.340+0000 7fa0c4000700 1 --2- 172.21.15.131:0/1750118731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b40a6550 0x7fa0b40a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.340+0000 7fa0c4000700 1 -- 172.21.15.131:0/1750118731 >> 172.21.15.131:0/1750118731 conn(0x7fa0b40a0a00 msgr2=0x7fa0b40a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:03.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.341+0000 7fa0c4000700 1 -- 172.21.15.131:0/1750118731 shutdown_connections 2024-05-10T05:36:03.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.341+0000 7fa0c4000700 1 -- 172.21.15.131:0/1750118731 wait complete. 2024-05-10T05:36:03.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.342+0000 7fa0c4000700 1 Processor -- start 2024-05-10T05:36:03.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.343+0000 7fa0c4000700 1 -- start start 2024-05-10T05:36:03.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.343+0000 7fa0c4000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b40a5570 0x7fa0b4144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.343+0000 7fa0c4000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40a6550 0x7fa0b4144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.343+0000 7fa0c4000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b40a6ea0 0x7fa0b414b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.343+0000 7fa0c4000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b414bad0 con 0x7fa0b40a6550 2024-05-10T05:36:03.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.343+0000 7fa0c4000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b414bc10 con 0x7fa0b40a5570 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.343+0000 7fa0c4000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b414bda0 con 0x7fa0b40a6ea0 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.344+0000 7fa0c1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b40a5570 0x7fa0b4144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.344+0000 7fa0c1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b40a5570 0x7fa0b4144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41822/0 (socket says 172.21.15.131:41822) 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.344+0000 7fa0c1600700 1 -- 172.21.15.131:0/178640045 learned_addr learned my addr 172.21.15.131:0/178640045 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.344+0000 7fa0c2000700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b40a6ea0 0x7fa0b414b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.344+0000 7fa0c1600700 1 -- 172.21.15.131:0/178640045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b40a6ea0 msgr2=0x7fa0b414b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.344+0000 7fa0c1600700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b40a6ea0 0x7fa0b414b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.344+0000 7fa0c1600700 1 -- 172.21.15.131:0/178640045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40a6550 msgr2=0x7fa0b4144570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.344+0000 7fa0c1600700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40a6550 0x7fa0b4144570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.344+0000 7fa0c1600700 1 -- 172.21.15.131:0/178640045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc069700 con 0x7fa0b40a5570 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.345+0000 7fa0c1600700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b40a5570 0x7fa0b4144030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b4d0 tx=0x7fa0a800b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.345+0000 7fa0ba000700 1 -- 172.21.15.131:0/178640045 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8011830 con 0x7fa0b40a5570 2024-05-10T05:36:03.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.345+0000 7fa0ba000700 1 -- 172.21.15.131:0/178640045 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0a8011e50 con 0x7fa0b40a5570 2024-05-10T05:36:03.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.345+0000 7fa0ba000700 1 -- 172.21.15.131:0/178640045 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8010e80 con 0x7fa0b40a5570 2024-05-10T05:36:03.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.347+0000 7fa0c4000700 1 -- 172.21.15.131:0/178640045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b414c080 con 0x7fa0b40a5570 2024-05-10T05:36:03.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.347+0000 7fa0c4000700 1 -- 172.21.15.131:0/178640045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b414c5c0 con 0x7fa0b40a5570 2024-05-10T05:36:03.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.354+0000 7fa0ba000700 1 -- 172.21.15.131:0/178640045 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa0a80104f0 con 0x7fa0b40a5570 2024-05-10T05:36:03.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.356+0000 7fa0ba000700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0a40a11f0 0x7fa0a40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.357+0000 7fa0ba000700 1 -- 172.21.15.131:0/178640045 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa0a8099b30 con 0x7fa0b40a5570 2024-05-10T05:36:03.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.359+0000 7fa0dbf6ab80 1 Processor -- start 2024-05-10T05:36:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.361+0000 7fa0c0c00700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0a40a11f0 0x7fa0a40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.362+0000 7fa0c0c00700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0a40a11f0 0x7fa0a40a3700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa0b41450d0 tx=0x7fa0b0006cb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:03.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.365+0000 7fa0dbf6ab80 1 -- start start 2024-05-10T05:36:03.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.366+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 0x5558830b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.366+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b87a0 0x5558830b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.366+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b90f0 0x5558830bf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.366+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555882fde150 con 0x5558830b90f0 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.366+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555882fdbb10 con 0x5558830b7530 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.366+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555882fd7400 con 0x5558830b87a0 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.366+0000 7fa0c1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 0x5558830b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.366+0000 7fa0c1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 0x5558830b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41828/0 (socket says 172.21.15.131:41828) 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.366+0000 7fa0c1600700 1 -- 172.21.15.131:0/1087099612 learned_addr learned my addr 172.21.15.131:0/1087099612 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.367+0000 7fa0c1600700 1 -- 172.21.15.131:0/1087099612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b87a0 msgr2=0x5558830b8bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.367+0000 7fa0c1600700 1 --2- 172.21.15.131:0/1087099612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b87a0 0x5558830b8bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.367+0000 7fa0c1600700 1 -- 172.21.15.131:0/1087099612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b90f0 msgr2=0x5558830bf9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.367+0000 7fa0c1600700 1 --2- 172.21.15.131:0/1087099612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b90f0 0x5558830bf9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.367+0000 7fa0c1600700 1 -- 172.21.15.131:0/1087099612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555882e583b0 con 0x5558830b7530 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.367+0000 7fa0c1600700 1 --2- 172.21.15.131:0/1087099612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 0x5558830b7960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8096750 tx=0x7fa0a809e110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd74f016daac9f4c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.367+0000 7fa0ae600700 1 -- 172.21.15.131:0/1087099612 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8099200 con 0x5558830b7530 2024-05-10T05:36:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.367+0000 7fa0ae600700 1 -- 172.21.15.131:0/1087099612 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0a8098d90 con 0x5558830b7530 2024-05-10T05:36:03.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.367+0000 7fa0ae600700 1 -- 172.21.15.131:0/1087099612 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8099200 con 0x5558830b7530 2024-05-10T05:36:03.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.371+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1087099612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 msgr2=0x5558830b7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:03.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.371+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/1087099612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 0x5558830b7960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8096750 tx=0x7fa0a809e110 comp rx=0 tx=0).stop 2024-05-10T05:36:03.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.374+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1087099612 shutdown_connections 2024-05-10T05:36:03.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.374+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/1087099612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b90f0 0x5558830bf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.374+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/1087099612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 0x5558830b7960 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.374+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/1087099612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b87a0 0x5558830b8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.374+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1087099612 >> 172.21.15.131:0/1087099612 conn(0x555883019a60 msgr2=0x5558830b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:03.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.376+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1087099612 shutdown_connections 2024-05-10T05:36:03.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.376+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1087099612 wait complete. 2024-05-10T05:36:03.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.382+0000 7fa0dbf6ab80 1 -- start start 2024-05-10T05:36:03.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.386+0000 7fa0dbf6ab80 1 Processor -- start 2024-05-10T05:36:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.401+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 0x5558830c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.401+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b87a0 0x5558830c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.401+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x5558830c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.401+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555882f73560 con 0x5558830b87a0 2024-05-10T05:36:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.401+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555882f73200 con 0x5558830b7530 2024-05-10T05:36:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.401+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555882e5a6a0 con 0x5558830b90f0 2024-05-10T05:36:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.402+0000 7fa0dbf6ab80 10 client.0 _ll_register_callbacks cb 0x555882f7cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.429+0000 7fa0c2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x5558830c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.429+0000 7fa0c2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x5558830c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44190/0 (socket says 172.21.15.131:44190) 2024-05-10T05:36:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.429+0000 7fa0c2000700 1 -- 172.21.15.131:0/154930139 learned_addr learned my addr 172.21.15.131:0/154930139 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.429+0000 7fa0c1600700 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 0x5558830c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.429+0000 7fa0c0c00700 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b87a0 0x5558830c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.429+0000 7fa0c2000700 1 -- 172.21.15.131:0/154930139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 msgr2=0x5558830c18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.429+0000 7fa0c2000700 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 0x5558830c18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.429+0000 7fa0c2000700 1 -- 172.21.15.131:0/154930139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b87a0 msgr2=0x5558830c1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.440+0000 7fa0c2000700 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b87a0 0x5558830c1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.440+0000 7fa0c2000700 1 -- 172.21.15.131:0/154930139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b414c5c0 con 0x5558830b90f0 2024-05-10T05:36:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.440+0000 7fa0c0c00700 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b87a0 0x5558830c1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:03.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.441+0000 7fa0c2000700 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x5558830c8d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc04efb0 tx=0x7fa0bc067100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63250fe21fd8eb45 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:03.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.441+0000 7fa0a2a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:36:03.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.443+0000 7fa0a2a00700 1 -- 172.21.15.131:0/154930139 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc06fb70 con 0x5558830b90f0 2024-05-10T05:36:03.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.443+0000 7fa0a2a00700 1 -- 172.21.15.131:0/154930139 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa0bc068e90 con 0x5558830b90f0 2024-05-10T05:36:03.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.443+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/154930139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555882f30270 con 0x5558830b90f0 2024-05-10T05:36:03.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.445+0000 7fa0a2a00700 1 -- 172.21.15.131:0/154930139 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0bc06fcd0 con 0x5558830b90f0 2024-05-10T05:36:03.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.445+0000 7fa0dbf6ab80 10 client.25939 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:03.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.445+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/154930139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555882e04b60 con 0x5558830b90f0 2024-05-10T05:36:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.449+0000 7fa0a2a00700 1 -- 172.21.15.131:0/154930139 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555882f30270 con 0x5558830b90f0 2024-05-10T05:36:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.449+0000 7fa0a2a00700 1 -- 172.21.15.131:0/154930139 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa0bc07f440 con 0x5558830b90f0 2024-05-10T05:36:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.449+0000 7fa0dbf6ab80 10 client.25939 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.449+0000 7fa0dbf6ab80 10 client.25939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.449+0000 7fa0dbf6ab80 20 client.25939 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.449+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5558830cd550 0x5558830cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.449+0000 7fa0dbf6ab80 4 client.25939 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.449+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/154930139 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555883086780 con 0x5558830cd550 2024-05-10T05:36:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.451+0000 7fa0c1600700 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5558830cd550 0x5558830cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.455+0000 7fa0c1600700 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5558830cd550 0x5558830cf9a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54cbd40b506e8b67 server_cookie=4cebf09cf85539e2 in_seq=0 out_seq=0 2024-05-10T05:36:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.455+0000 7fa0a2a00700 10 client.25939 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.455+0000 7fa0a2a00700 1 -- 172.21.15.131:0/154930139 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555883086780 con 0x5558830cd550 2024-05-10T05:36:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.455+0000 7fa0a2a00700 10 client.25939 handle_command_reply: tid=0 2024-05-10T05:36:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.456+0000 7fa0dbf6ab80 1 client.25939 shutdown 2024-05-10T05:36:03.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.456+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/154930139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 msgr2=0x5558830c8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:03.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.456+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x5558830c8d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc04efb0 tx=0x7fa0bc067100 comp rx=0 tx=0).stop 2024-05-10T05:36:03.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.459+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/154930139 shutdown_connections 2024-05-10T05:36:03.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.459+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5558830cd550 0x5558830cf9a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.459+0000 7fa0a2a00700 0 client.25939 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.459+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x5558830c8d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.459+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b7530 0x5558830c18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.459+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/154930139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b87a0 0x5558830c1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.459+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/154930139 >> 172.21.15.131:0/154930139 conn(0x555883019a60 msgr2=0x5558830b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:03.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.463+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/154930139 shutdown_connections 2024-05-10T05:36:03.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.464+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/154930139 wait complete. 2024-05-10T05:36:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.467+0000 7fa0dbf6ab80 20 client.25939 trim_cache size 0 max 16384 2024-05-10T05:36:03.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.524+0000 7fa0dbf6ab80 1 Processor -- start 2024-05-10T05:36:03.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.529+0000 7fa0dbf6ab80 1 -- start start 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.529+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b7530 0x5558830c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.532+0000 7fa0c1600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b7530 0x5558830c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.536+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b87a0 0x5558830c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.536+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b90f0 0x5558830bfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.536+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558830a9d60 con 0x5558830b7530 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.536+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558830bc050 con 0x5558830b90f0 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.536+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558830c11d0 con 0x5558830b87a0 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.536+0000 7fa0c2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b90f0 0x5558830bfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.536+0000 7fa0c2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b90f0 0x5558830bfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41846/0 (socket says 172.21.15.131:41846) 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.536+0000 7fa0c2000700 1 -- 172.21.15.131:0/135687368 learned_addr learned my addr 172.21.15.131:0/135687368 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.537+0000 7fa0c2000700 1 -- 172.21.15.131:0/135687368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b87a0 msgr2=0x5558830c6ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.537+0000 7fa0c2000700 1 --2- 172.21.15.131:0/135687368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b87a0 0x5558830c6ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.537+0000 7fa0c2000700 1 -- 172.21.15.131:0/135687368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b7530 msgr2=0x5558830c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.537+0000 7fa0c2000700 1 --2- 172.21.15.131:0/135687368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b7530 0x5558830c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.537+0000 7fa0c2000700 1 -- 172.21.15.131:0/135687368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555882f9d560 con 0x5558830b90f0 2024-05-10T05:36:03.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.538+0000 7fa0c2000700 1 --2- 172.21.15.131:0/135687368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b90f0 0x5558830bfcc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc064e40 tx=0x7fa0bc066ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=378d6a2f6c37bc83 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:03.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.541+0000 7fa0af000700 1 -- 172.21.15.131:0/135687368 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc06fcd0 con 0x5558830b90f0 2024-05-10T05:36:03.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.541+0000 7fa0af000700 1 -- 172.21.15.131:0/135687368 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0bc068f50 con 0x5558830b90f0 2024-05-10T05:36:03.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.541+0000 7fa0af000700 1 -- 172.21.15.131:0/135687368 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc073940 con 0x5558830b90f0 2024-05-10T05:36:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.547+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/135687368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b90f0 msgr2=0x5558830bfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.547+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/135687368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b90f0 0x5558830bfcc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc064e40 tx=0x7fa0bc066ca0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.547+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/135687368 shutdown_connections 2024-05-10T05:36:03.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.547+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/135687368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b7530 0x5558830c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.547+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/135687368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b90f0 0x5558830bfcc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.547+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/135687368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b87a0 0x5558830c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.551+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/135687368 >> 172.21.15.131:0/135687368 conn(0x555883019a60 msgr2=0x5558830c0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:03.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.553+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/135687368 shutdown_connections 2024-05-10T05:36:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.553+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/135687368 wait complete. 2024-05-10T05:36:03.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.555+0000 7fa0dbf6ab80 1 -- start start 2024-05-10T05:36:03.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.555+0000 7fa0dbf6ab80 1 Processor -- start 2024-05-10T05:36:03.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.556+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b7530 0x55588311d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.556+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b87a0 0x55588311ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.557+0000 7fa0dbf6ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x555883122490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.557+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555883122a60 con 0x5558830b7530 2024-05-10T05:36:03.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.557+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555883122ba0 con 0x5558830b87a0 2024-05-10T05:36:03.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.557+0000 7fa0dbf6ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555883122d60 con 0x5558830b90f0 2024-05-10T05:36:03.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.557+0000 7fa0dbf6ab80 10 client.0 _ll_register_callbacks cb 0x555882f7cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:03.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.559+0000 7fa0c2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x555883122490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.559+0000 7fa0c2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x555883122490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44200/0 (socket says 172.21.15.131:44200) 2024-05-10T05:36:03.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.559+0000 7fa0c2000700 1 -- 172.21.15.131:0/1845510565 learned_addr learned my addr 172.21.15.131:0/1845510565 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:03.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.559+0000 7fa0c2000700 1 -- 172.21.15.131:0/1845510565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b87a0 msgr2=0x55588311ff40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:36:03.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.559+0000 7fa0c2000700 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b87a0 0x55588311ff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.559+0000 7fa0c2000700 1 -- 172.21.15.131:0/1845510565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b7530 msgr2=0x55588311d9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:03.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.559+0000 7fa0c2000700 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b7530 0x55588311d9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.559+0000 7fa0c2000700 1 -- 172.21.15.131:0/1845510565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555882e04b60 con 0x5558830b90f0 2024-05-10T05:36:03.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.562+0000 7fa0c2000700 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x555883122490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc066e70 tx=0x7fa0bc0674b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa352bade8bb6614 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:03.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.562+0000 7fa0a2a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:36:03.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.564+0000 7fa0a2a00700 1 -- 172.21.15.131:0/1845510565 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc06fed0 con 0x5558830b90f0 2024-05-10T05:36:03.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.564+0000 7fa0a2a00700 1 -- 172.21.15.131:0/1845510565 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa0bc07fae0 con 0x5558830b90f0 2024-05-10T05:36:03.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.565+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1845510565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555882fcfec0 con 0x5558830b90f0 2024-05-10T05:36:03.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.565+0000 7fa0a2a00700 1 -- 172.21.15.131:0/1845510565 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0bc073480 con 0x5558830b90f0 2024-05-10T05:36:03.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.565+0000 7fa0dbf6ab80 10 client.25942 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:03.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.566+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1845510565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555882e5ccd0 con 0x5558830b90f0 2024-05-10T05:36:03.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.566+0000 7fa0a2a00700 1 -- 172.21.15.131:0/1845510565 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555882fcfec0 con 0x5558830b90f0 2024-05-10T05:36:03.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.570+0000 7fa0a2a00700 1 -- 172.21.15.131:0/1845510565 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa0bc088780 con 0x5558830b90f0 2024-05-10T05:36:03.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.570+0000 7fa0dbf6ab80 10 client.25942 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:03.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.570+0000 7fa0dbf6ab80 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:03.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.570+0000 7fa0dbf6ab80 20 client.25942 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.570+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5558830cd550 0x555883126270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.570+0000 7fa0dbf6ab80 4 client.25942 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:36:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.570+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1845510565 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555882e25cf0 con 0x5558830cd550 2024-05-10T05:36:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.570+0000 7fa0c1600700 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5558830cd550 0x555883126270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.572+0000 7fa0c1600700 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5558830cd550 0x555883126270 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f8de5f51bef7a0 server_cookie=b7449e7916f08327 in_seq=0 out_seq=0 2024-05-10T05:36:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.572+0000 7fa0a2a00700 10 client.25942 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.572+0000 7fa0a2a00700 1 -- 172.21.15.131:0/1845510565 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555882e25cf0 con 0x5558830cd550 2024-05-10T05:36:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.572+0000 7fa0a2a00700 10 client.25942 handle_command_reply: tid=0 2024-05-10T05:36:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.573+0000 7fa0dbf6ab80 1 client.25942 shutdown 2024-05-10T05:36:03.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.579+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1845510565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 msgr2=0x555883122490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:03.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.579+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x555883122490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc066e70 tx=0x7fa0bc0674b0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.583+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1845510565 shutdown_connections 2024-05-10T05:36:03.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.583+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5558830cd550 0x555883126270 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.583+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558830b7530 0x55588311d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.583+0000 7fa0a2a00700 0 client.25942 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:03.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.583+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558830b90f0 0x555883122490 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.583+0000 7fa0dbf6ab80 1 --2- 172.21.15.131:0/1845510565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558830b87a0 0x55588311ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.583+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1845510565 >> 172.21.15.131:0/1845510565 conn(0x555883019a60 msgr2=0x5558830c0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:03.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.584+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1845510565 shutdown_connections 2024-05-10T05:36:03.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.584+0000 7fa0dbf6ab80 1 -- 172.21.15.131:0/1845510565 wait complete. 2024-05-10T05:36:03.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.585+0000 7fa0dbf6ab80 20 client.25942 trim_cache size 0 max 16384 2024-05-10T05:36:03.587 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:36:03.587 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:36:03.587 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:36:03.587 INFO:teuthology.orchestra.run.smithi131.stdout: "d9fc0556-f5cb-4b21-917d-5cb1e7cea838": { 2024-05-10T05:36:03.587 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:36:03.587 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "d9fc0556-f5cb-4b21-917d-5cb1e7cea838", 2024-05-10T05:36:03.587 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:36:03.587 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:36:03.587 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:36:03.588 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:36:03.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.588+0000 7fa0c4000700 1 -- 172.21.15.131:0/178640045 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0a40a11f0 msgr2=0x7fa0a40a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:03.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.588+0000 7fa0c4000700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0a40a11f0 0x7fa0a40a3700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa0b41450d0 tx=0x7fa0b0006cb0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.588+0000 7fa0c4000700 1 -- 172.21.15.131:0/178640045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b40a5570 msgr2=0x7fa0b4144030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:03.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.588+0000 7fa0c4000700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b40a5570 0x7fa0b4144030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b4d0 tx=0x7fa0a800b7e0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.591+0000 7fa0c4000700 1 -- 172.21.15.131:0/178640045 shutdown_connections 2024-05-10T05:36:03.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.591+0000 7fa0c4000700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0a40a11f0 0x7fa0a40a3700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.591+0000 7fa0c4000700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0b40a6ea0 0x7fa0b414b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.591+0000 7fa0c4000700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b40a5570 0x7fa0b4144030 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.591+0000 7fa0c4000700 1 --2- 172.21.15.131:0/178640045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40a6550 0x7fa0b4144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:03.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.591+0000 7fa0c4000700 1 -- 172.21.15.131:0/178640045 >> 172.21.15.131:0/178640045 conn(0x7fa0b40a0a00 msgr2=0x7fa0b40a7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:03.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.592+0000 7fa0c4000700 1 -- 172.21.15.131:0/178640045 shutdown_connections 2024-05-10T05:36:03.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:03.593+0000 7fa0c4000700 1 -- 172.21.15.131:0/178640045 wait complete. 2024-05-10T05:36:03.717 INFO:tasks.cephfs.filesystem:scrub status for tag:d9fc0556-f5cb-4b21-917d-5cb1e7cea838 - {'path': '/', 'tag': 'd9fc0556-f5cb-4b21-917d-5cb1e7cea838', 'options': 'recursive,force'} 2024-05-10T05:36:04.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:04 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:36:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:04 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:04 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:36:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:04 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:36:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:04 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:04 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:04.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:04 smithi157 ceph-mon[78427]: pgmap v366: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 237 MiB/s wr, 75 op/s 2024-05-10T05:36:04.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:04 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:36:04.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:04 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:04.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:04 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:36:04.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:04 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:36:04.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:04 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:04.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:04 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:04.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:04 smithi142 ceph-mon[77667]: pgmap v366: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 237 MiB/s wr, 75 op/s 2024-05-10T05:36:04.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:04 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:36:04.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:04 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:04.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:04 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:36:04.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:04 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:36:04.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:04 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:04.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:04 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:04.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:04 smithi131 ceph-mon[37264]: pgmap v366: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 237 MiB/s wr, 75 op/s 2024-05-10T05:36:06.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:06 smithi157 ceph-mon[78427]: pgmap v367: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 263 MiB/s wr, 80 op/s 2024-05-10T05:36:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:06 smithi142 ceph-mon[77667]: pgmap v367: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 263 MiB/s wr, 80 op/s 2024-05-10T05:36:06.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:06 smithi131 ceph-mon[37264]: pgmap v367: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 263 MiB/s wr, 80 op/s 2024-05-10T05:36:08.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:08 smithi157 ceph-mon[78427]: pgmap v368: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 218 MiB/s wr, 69 op/s 2024-05-10T05:36:08.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:08 smithi142 ceph-mon[77667]: pgmap v368: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 218 MiB/s wr, 69 op/s 2024-05-10T05:36:08.465 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:08 smithi131 ceph-mon[37264]: pgmap v368: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 218 MiB/s wr, 69 op/s 2024-05-10T05:36:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:10 smithi142 ceph-mon[77667]: pgmap v369: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:36:10.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:10 smithi131 ceph-mon[37264]: pgmap v369: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:36:10.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:10 smithi157 ceph-mon[78427]: pgmap v369: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:36:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:11 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:36:11.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:11 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:36:11.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:11 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:36:11.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:11 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:36:11.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:11 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:36:11.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:11 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:36:12.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:12 smithi142 ceph-mon[77667]: pgmap v370: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:36:12.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:12 smithi131 ceph-mon[37264]: pgmap v370: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:36:12.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:12 smithi157 ceph-mon[78427]: pgmap v370: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:36:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:14 smithi142 ceph-mon[77667]: pgmap v371: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 237 MiB/s wr, 75 op/s 2024-05-10T05:36:14.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:14 smithi131 ceph-mon[37264]: pgmap v371: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 237 MiB/s wr, 75 op/s 2024-05-10T05:36:14.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:14 smithi157 ceph-mon[78427]: pgmap v371: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 237 MiB/s wr, 75 op/s 2024-05-10T05:36:16.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:16 smithi142 ceph-mon[77667]: pgmap v372: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 266 MiB/s wr, 80 op/s 2024-05-10T05:36:16.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:16 smithi131 ceph-mon[37264]: pgmap v372: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 266 MiB/s wr, 80 op/s 2024-05-10T05:36:16.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:16 smithi157 ceph-mon[78427]: pgmap v372: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 266 MiB/s wr, 80 op/s 2024-05-10T05:36:17.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:17 smithi131 ceph-mon[37264]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 71 op/s 2024-05-10T05:36:17.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:17 smithi157 ceph-mon[78427]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 71 op/s 2024-05-10T05:36:17.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:17 smithi142 ceph-mon[77667]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 71 op/s 2024-05-10T05:36:20.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:20 smithi157 ceph-mon[78427]: pgmap v374: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 256 MiB/s wr, 77 op/s 2024-05-10T05:36:20.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:20 smithi142 ceph-mon[77667]: pgmap v374: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 256 MiB/s wr, 77 op/s 2024-05-10T05:36:20.465 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:20 smithi131 ceph-mon[37264]: pgmap v374: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 256 MiB/s wr, 77 op/s 2024-05-10T05:36:22.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:22 smithi142 ceph-mon[77667]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:36:22.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:22 smithi131 ceph-mon[37264]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:36:22.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:22 smithi157 ceph-mon[78427]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:36:24.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:24 smithi142 ceph-mon[77667]: pgmap v376: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 73 op/s 2024-05-10T05:36:24.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:24 smithi131 ceph-mon[37264]: pgmap v376: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 73 op/s 2024-05-10T05:36:24.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:24 smithi157 ceph-mon[78427]: pgmap v376: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 73 op/s 2024-05-10T05:36:26.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:26 smithi142 ceph-mon[77667]: pgmap v377: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 264 MiB/s wr, 80 op/s 2024-05-10T05:36:26.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:26 smithi131 ceph-mon[37264]: pgmap v377: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 264 MiB/s wr, 80 op/s 2024-05-10T05:36:26.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:26 smithi157 ceph-mon[78427]: pgmap v377: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 264 MiB/s wr, 80 op/s 2024-05-10T05:36:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:28 smithi142 ceph-mon[77667]: pgmap v378: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:36:28.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:28 smithi131 ceph-mon[37264]: pgmap v378: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:36:28.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:28 smithi157 ceph-mon[78427]: pgmap v378: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:36:30.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:30 smithi131 ceph-mon[37264]: pgmap v379: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 253 MiB/s wr, 76 op/s 2024-05-10T05:36:30.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:30 smithi157 ceph-mon[78427]: pgmap v379: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 253 MiB/s wr, 76 op/s 2024-05-10T05:36:30.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:30 smithi142 ceph-mon[77667]: pgmap v379: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 253 MiB/s wr, 76 op/s 2024-05-10T05:36:32.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:32 smithi131 ceph-mon[37264]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 238 MiB/s wr, 72 op/s 2024-05-10T05:36:32.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:32 smithi157 ceph-mon[78427]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 238 MiB/s wr, 72 op/s 2024-05-10T05:36:32.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:32 smithi142 ceph-mon[77667]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 238 MiB/s wr, 72 op/s 2024-05-10T05:36:33.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:33 smithi131 ceph-mon[37264]: pgmap v381: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:36:33.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:33 smithi157 ceph-mon[78427]: pgmap v381: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:36:33.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:33 smithi142 ceph-mon[77667]: pgmap v381: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:36:33.718 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:36:34.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.434+0000 7f2e9ec00700 1 -- 172.21.15.131:0/1824014914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98109780 msgr2=0x7f2e98109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:34.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.434+0000 7f2e9ec00700 1 --2- 172.21.15.131:0/1824014914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98109780 0x7f2e98109b90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000b0a0 tx=0x7f2e9000b3b0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.436+0000 7f2e9ec00700 1 -- 172.21.15.131:0/1824014914 shutdown_connections 2024-05-10T05:36:34.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.436+0000 7f2e9ec00700 1 --2- 172.21.15.131:0/1824014914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e98108610 0x7f2e98108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.436+0000 7f2e9ec00700 1 --2- 172.21.15.131:0/1824014914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e9810a0d0 0x7f2e981109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.436+0000 7f2e9ec00700 1 --2- 172.21.15.131:0/1824014914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98109780 0x7f2e98109b90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.437+0000 7f2e9ec00700 1 -- 172.21.15.131:0/1824014914 >> 172.21.15.131:0/1824014914 conn(0x7f2e9806ee80 msgr2=0x7f2e98071330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:34.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.442+0000 7f2e9ec00700 1 -- 172.21.15.131:0/1824014914 shutdown_connections 2024-05-10T05:36:34.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.442+0000 7f2e9ec00700 1 -- 172.21.15.131:0/1824014914 wait complete. 2024-05-10T05:36:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.450+0000 7f2e9ec00700 1 Processor -- start 2024-05-10T05:36:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.451+0000 7f2e9ec00700 1 -- start start 2024-05-10T05:36:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.453+0000 7f2e9ec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e98108610 0x7f2e981a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.453+0000 7f2e9ec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e98109780 0x7f2e981a7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.454+0000 7f2e97e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e98108610 0x7f2e981a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.454+0000 7f2e9ec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9810a0d0 0x7f2e981ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.458+0000 7f2e9ec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e981ae840 con 0x7f2e98109780 2024-05-10T05:36:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.458+0000 7f2e9cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9810a0d0 0x7f2e981ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.458+0000 7f2e9cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9810a0d0 0x7f2e981ae270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:33966/0 (socket says 172.21.15.131:33966) 2024-05-10T05:36:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.458+0000 7f2e9cc00700 1 -- 172.21.15.131:0/3156494484 learned_addr learned my addr 172.21.15.131:0/3156494484 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.458+0000 7f2e9ec00700 1 -- 172.21.15.131:0/3156494484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e981ae980 con 0x7f2e9810a0d0 2024-05-10T05:36:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.458+0000 7f2e9ec00700 1 -- 172.21.15.131:0/3156494484 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e981aeb40 con 0x7f2e98108610 2024-05-10T05:36:34.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.462+0000 7f2e9cc00700 1 -- 172.21.15.131:0/3156494484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e98108610 msgr2=0x7f2e981a6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:34.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.462+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e98108610 0x7f2e981a6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.462+0000 7f2e9cc00700 1 -- 172.21.15.131:0/3156494484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e98109780 msgr2=0x7f2e981a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:34.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.462+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e98109780 0x7f2e981a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.462+0000 7f2e9cc00700 1 -- 172.21.15.131:0/3156494484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e88009520 con 0x7f2e9810a0d0 2024-05-10T05:36:34.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.463+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9810a0d0 0x7f2e981ae270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800eab0 tx=0x7f2e8800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:34.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.463+0000 7f2e94c00700 1 -- 172.21.15.131:0/3156494484 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88004220 con 0x7f2e9810a0d0 2024-05-10T05:36:34.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.463+0000 7f2e94c00700 1 -- 172.21.15.131:0/3156494484 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e8800ce10 con 0x7f2e9810a0d0 2024-05-10T05:36:34.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.463+0000 7f2e94c00700 1 -- 172.21.15.131:0/3156494484 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88003e10 con 0x7f2e9810a0d0 2024-05-10T05:36:34.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.463+0000 7f2e9ec00700 1 -- 172.21.15.131:0/3156494484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e90009d00 con 0x7f2e9810a0d0 2024-05-10T05:36:34.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.468+0000 7f2e9ec00700 1 -- 172.21.15.131:0/3156494484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e981af150 con 0x7f2e9810a0d0 2024-05-10T05:36:34.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.476+0000 7f2e94c00700 1 -- 172.21.15.131:0/3156494484 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2e88018430 con 0x7f2e9810a0d0 2024-05-10T05:36:34.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.482+0000 7f2e94c00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2e7c0a1240 0x7f2e7c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.482+0000 7f2e94c00700 1 -- 172.21.15.131:0/3156494484 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2e88014070 con 0x7f2e9810a0d0 2024-05-10T05:36:34.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.486+0000 7f2e97e00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2e7c0a1240 0x7f2e7c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.487+0000 7f2eb6a84b80 1 Processor -- start 2024-05-10T05:36:34.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.488+0000 7f2e97e00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2e7c0a1240 0x7f2e7c0a3750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e981a81d0 tx=0x7f2e8400a480 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:34.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.489+0000 7f2eb6a84b80 1 -- start start 2024-05-10T05:36:34.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.492+0000 7f2eb6a84b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e806a0 0x558ef7e80af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.492+0000 7f2eb6a84b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e81980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.492+0000 7f2eb6a84b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.493+0000 7f2eb6a84b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7da7150 con 0x558ef7e81ec0 2024-05-10T05:36:34.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.493+0000 7f2eb6a84b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7da4b10 con 0x558ef7e806a0 2024-05-10T05:36:34.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.493+0000 7f2eb6a84b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7da0400 con 0x558ef7e81570 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.493+0000 7f2e9cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.494+0000 7f2e9cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e887c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35752/0 (socket says 172.21.15.131:35752) 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.494+0000 7f2e9cc00700 1 -- 172.21.15.131:0/3465856654 learned_addr learned my addr 172.21.15.131:0/3465856654 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.494+0000 7f2e97400700 1 --2- 172.21.15.131:0/3465856654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e81980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.494+0000 7f2e9cc00700 1 -- 172.21.15.131:0/3465856654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 msgr2=0x558ef7e81980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.494+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/3465856654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e81980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.494+0000 7f2e9cc00700 1 -- 172.21.15.131:0/3465856654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e806a0 msgr2=0x558ef7e80af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.494+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/3465856654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e806a0 0x558ef7e80af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.494+0000 7f2e9cc00700 1 -- 172.21.15.131:0/3465856654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ef7c213b0 con 0x558ef7e81ec0 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.495+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/3465856654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e887c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2e88095d10 tx=0x7f2e880954b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb053c075d3e9168 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.497+0000 7f2e8d200700 1 -- 172.21.15.131:0/3465856654 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88099170 con 0x558ef7e81ec0 2024-05-10T05:36:34.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.497+0000 7f2e8d200700 1 -- 172.21.15.131:0/3465856654 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e880044f0 con 0x558ef7e81ec0 2024-05-10T05:36:34.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.497+0000 7f2e8d200700 1 -- 172.21.15.131:0/3465856654 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88097bb0 con 0x558ef7e81ec0 2024-05-10T05:36:34.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.498+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/3465856654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 msgr2=0x558ef7e887c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:34.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.498+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/3465856654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e887c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2e88095d10 tx=0x7f2e880954b0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.502+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/3465856654 shutdown_connections 2024-05-10T05:36:34.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.502+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/3465856654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e887c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.502+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/3465856654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e806a0 0x558ef7e80af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.502+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/3465856654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e81980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.502+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/3465856654 >> 172.21.15.131:0/3465856654 conn(0x558ef7de2a60 msgr2=0x558ef7e7afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:34.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.502+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/3465856654 shutdown_connections 2024-05-10T05:36:34.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.502+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/3465856654 wait complete. 2024-05-10T05:36:34.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.507+0000 7f2eb6a84b80 1 -- start start 2024-05-10T05:36:34.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.508+0000 7f2eb6a84b80 1 Processor -- start 2024-05-10T05:36:34.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.509+0000 7f2eb6a84b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e806a0 0x558ef7e8a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.510+0000 7f2eb6a84b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81570 0x558ef7e8ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.510+0000 7f2eb6a84b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.511+0000 7f2e9cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.511+0000 7f2e9cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35762/0 (socket says 172.21.15.131:35762) 2024-05-10T05:36:34.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.511+0000 7f2e97e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e806a0 0x558ef7e8a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.512+0000 7f2eb6a84b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7d3c560 con 0x558ef7e81ec0 2024-05-10T05:36:34.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.512+0000 7f2e97e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e806a0 0x558ef7e8a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60764/0 (socket says 172.21.15.131:60764) 2024-05-10T05:36:34.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.512+0000 7f2e97e00700 1 -- 172.21.15.131:0/1369632089 learned_addr learned my addr 172.21.15.131:0/1369632089 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:34.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.512+0000 7f2e97400700 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81570 0x558ef7e8ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.512+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/1369632089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7d3c200 con 0x558ef7e81570 2024-05-10T05:36:34.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.512+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/1369632089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7c236a0 con 0x558ef7e806a0 2024-05-10T05:36:34.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.514+0000 7f2e9cc00700 1 -- 172.21.15.131:0/1369632089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e806a0 msgr2=0x558ef7e8a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:34.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.514+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e806a0 0x558ef7e8a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.514+0000 7f2e9cc00700 1 -- 172.21.15.131:0/1369632089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81570 msgr2=0x558ef7e8ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:34.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.514+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81570 0x558ef7e8ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.514+0000 7f2e9cc00700 1 -- 172.21.15.131:0/1369632089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e981af150 con 0x558ef7e81ec0 2024-05-10T05:36:34.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.514+0000 7f2eb6a84b80 10 client.0 _ll_register_callbacks cb 0x558ef7d45b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:34.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.514+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/1369632089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ef7cf9270 con 0x558ef7e81ec0 2024-05-10T05:36:34.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.515+0000 7f2e97e00700 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e806a0 0x558ef7e8a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:34.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.515+0000 7f2e97400700 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81570 0x558ef7e8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:34.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.515+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91bb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x558ef7e82340 tx=0x7f2e880a1d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f89325d92805a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:34.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.518+0000 7f2e81600700 10 client.15864 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:36:34.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.518+0000 7f2e81600700 1 -- 172.21.15.131:0/1369632089 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88099f10 con 0x558ef7e81ec0 2024-05-10T05:36:34.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.518+0000 7f2e81600700 1 -- 172.21.15.131:0/1369632089 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2e880b90d0 con 0x558ef7e81ec0 2024-05-10T05:36:34.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.518+0000 7f2e81600700 1 -- 172.21.15.131:0/1369632089 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e880b9290 con 0x558ef7e81ec0 2024-05-10T05:36:34.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.518+0000 7f2eb6a84b80 10 client.15864 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:34.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.518+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/1369632089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=28}) v3 -- 0x558ef7bcdb60 con 0x558ef7e81ec0 2024-05-10T05:36:34.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.520+0000 7f2e81600700 1 -- 172.21.15.131:0/1369632089 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2e88099a70 con 0x558ef7e81ec0 2024-05-10T05:36:34.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.524+0000 7f2eb6a84b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:34.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.524+0000 7f2eb6a84b80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:34.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.524+0000 7f2eb6a84b80 20 client.15864 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:34.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.524+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558ef7e95d00 0x558ef7e98150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.524+0000 7f2eb6a84b80 4 client.15864 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:34.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.524+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/1369632089 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ef7e4f780 con 0x558ef7e95d00 2024-05-10T05:36:34.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.524+0000 7f2e97e00700 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558ef7e95d00 0x558ef7e98150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.525+0000 7f2e97e00700 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558ef7e95d00 0x558ef7e98150 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69962ec65f5e649 server_cookie=c0df3c137c486cd1 in_seq=0 out_seq=0 2024-05-10T05:36:34.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.525+0000 7f2e81600700 10 client.15864 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:34.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.526+0000 7f2e81600700 1 -- 172.21.15.131:0/1369632089 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ef7e4f780 con 0x558ef7e95d00 2024-05-10T05:36:34.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.526+0000 7f2e81600700 10 client.15864 handle_command_reply: tid=0 2024-05-10T05:36:34.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.526+0000 7f2eb6a84b80 1 client.15864 shutdown 2024-05-10T05:36:34.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.527+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/1369632089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 msgr2=0x558ef7e91bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:34.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.527+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91bb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x558ef7e82340 tx=0x7f2e880a1d90 comp rx=0 tx=0).stop 2024-05-10T05:36:34.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.531+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/1369632089 shutdown_connections 2024-05-10T05:36:34.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.531+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558ef7e95d00 0x558ef7e98150 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.531+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91bb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.531+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e806a0 0x558ef7e8a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.531+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/1369632089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81570 0x558ef7e8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.531+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/1369632089 >> 172.21.15.131:0/1369632089 conn(0x558ef7de2a60 msgr2=0x558ef7e7afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:34.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.531+0000 7f2e81600700 0 client.15864 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:34.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.533+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/1369632089 shutdown_connections 2024-05-10T05:36:34.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.533+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/1369632089 wait complete. 2024-05-10T05:36:34.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.534+0000 7f2eb6a84b80 20 client.15864 trim_cache size 0 max 16384 2024-05-10T05:36:34.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.683+0000 7f2eb6a84b80 1 Processor -- start 2024-05-10T05:36:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.685+0000 7f2eb6a84b80 1 -- start start 2024-05-10T05:36:34.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.688+0000 7f2eb6a84b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e806a0 0x558ef7e88d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.690+0000 7f2e97e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e806a0 0x558ef7e88d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.690+0000 7f2e97e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e806a0 0x558ef7e88d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35770/0 (socket says 172.21.15.131:35770) 2024-05-10T05:36:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.690+0000 7f2e97e00700 1 -- 172.21.15.131:0/829553139 learned_addr learned my addr 172.21.15.131:0/829553139 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.691+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e91b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.691+0000 7f2e97400700 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e91b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.691+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81ec0 0x558ef7e89240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.692+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/829553139 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7e7c770 con 0x558ef7e806a0 2024-05-10T05:36:34.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.692+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/829553139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7e8b800 con 0x558ef7e81ec0 2024-05-10T05:36:34.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.693+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/829553139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7e7bad0 con 0x558ef7e81570 2024-05-10T05:36:34.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.693+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81ec0 0x558ef7e89240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.697+0000 7f2e97e00700 1 -- 172.21.15.131:0/829553139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 msgr2=0x558ef7e91b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:34.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.697+0000 7f2e97e00700 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e91b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.697+0000 7f2e97e00700 1 -- 172.21.15.131:0/829553139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81ec0 msgr2=0x558ef7e89240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:34.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.697+0000 7f2e97e00700 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81ec0 0x558ef7e89240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.698+0000 7f2e97e00700 1 -- 172.21.15.131:0/829553139 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ef7d66560 con 0x558ef7e806a0 2024-05-10T05:36:34.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.698+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81ec0 0x558ef7e89240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:34.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.700+0000 7f2e97e00700 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e806a0 0x558ef7e88d00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400aef0 tx=0x7f2e84012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9126598794e89819 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:34.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.702+0000 7f2e8dc00700 1 -- 172.21.15.131:0/829553139 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84004a30 con 0x558ef7e806a0 2024-05-10T05:36:34.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.703+0000 7f2e8dc00700 1 -- 172.21.15.131:0/829553139 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e84022040 con 0x558ef7e806a0 2024-05-10T05:36:34.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.705+0000 7f2e8dc00700 1 -- 172.21.15.131:0/829553139 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e840053a0 con 0x558ef7e806a0 2024-05-10T05:36:34.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.708+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/829553139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e806a0 msgr2=0x558ef7e88d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.708+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e806a0 0x558ef7e88d00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400aef0 tx=0x7f2e84012000 comp rx=0 tx=0).stop 2024-05-10T05:36:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.708+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/829553139 shutdown_connections 2024-05-10T05:36:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.708+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e806a0 0x558ef7e88d00 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400aef0 tx=0x7f2e84012000 comp rx=0 tx=0).stop 2024-05-10T05:36:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.708+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e81ec0 0x558ef7e89240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.708+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/829553139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e91b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.708+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/829553139 >> 172.21.15.131:0/829553139 conn(0x558ef7de2a60 msgr2=0x558ef7e8a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:34.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.710+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/829553139 shutdown_connections 2024-05-10T05:36:34.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.714+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/829553139 wait complete. 2024-05-10T05:36:34.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.714+0000 7f2eb6a84b80 1 -- start start 2024-05-10T05:36:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.715+0000 7f2eb6a84b80 1 Processor -- start 2024-05-10T05:36:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.718+0000 7f2eb6a84b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e91510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.718+0000 7f2eb6a84b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.718+0000 7f2eb6a84b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e8fc20 0x558ef7e90030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.718+0000 7f2eb6a84b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7e90650 con 0x558ef7e81ec0 2024-05-10T05:36:34.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.718+0000 7f2eb6a84b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7e90790 con 0x558ef7e8fc20 2024-05-10T05:36:34.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.718+0000 7f2eb6a84b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef7e90950 con 0x558ef7e81570 2024-05-10T05:36:34.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.718+0000 7f2eb6a84b80 10 client.0 _ll_register_callbacks cb 0x558ef7d45b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.718+0000 7f2e9cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e8fc20 0x558ef7e90030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.718+0000 7f2e9cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e8fc20 0x558ef7e90030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:33998/0 (socket says 172.21.15.131:33998) 2024-05-10T05:36:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.718+0000 7f2e9cc00700 1 -- 172.21.15.131:0/726631696 learned_addr learned my addr 172.21.15.131:0/726631696 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.719+0000 7f2e9cc00700 1 -- 172.21.15.131:0/726631696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 msgr2=0x558ef7e91510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.719+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e91510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.719+0000 7f2e9cc00700 1 -- 172.21.15.131:0/726631696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 msgr2=0x558ef7e91a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.719+0000 7f2e97400700 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.719+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.719+0000 7f2e9cc00700 1 -- 172.21.15.131:0/726631696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ef7bcdb60 con 0x558ef7e8fc20 2024-05-10T05:36:34.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.719+0000 7f2e97400700 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.720+0000 7f2e81600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:36:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.720+0000 7f2e9cc00700 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e8fc20 0x558ef7e90030 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e88096280 tx=0x7f2e8800cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd69b52fd87e17 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:34.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.724+0000 7f2e81600700 1 -- 172.21.15.131:0/726631696 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e880a1a00 con 0x558ef7e8fc20 2024-05-10T05:36:34.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.724+0000 7f2e81600700 1 -- 172.21.15.131:0/726631696 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2e88099f10 con 0x558ef7e8fc20 2024-05-10T05:36:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.725+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/726631696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e880044f0 con 0x558ef7e8fc20 2024-05-10T05:36:34.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.725+0000 7f2e81600700 1 -- 172.21.15.131:0/726631696 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e880a1a00 con 0x558ef7e8fc20 2024-05-10T05:36:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.727+0000 7f2eb6a84b80 10 client.25928 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.727+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/726631696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f2e8400fd20 con 0x558ef7e8fc20 2024-05-10T05:36:34.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.728+0000 7f2e81600700 1 -- 172.21.15.131:0/726631696 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e880044f0 con 0x558ef7e8fc20 2024-05-10T05:36:34.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.728+0000 7f2e81600700 1 -- 172.21.15.131:0/726631696 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2e8809d160 con 0x558ef7e8fc20 2024-05-10T05:36:34.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.729+0000 7f2eb6a84b80 10 client.25928 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:34.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.729+0000 7f2eb6a84b80 10 client.25928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:34.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.729+0000 7f2eb6a84b80 20 client.25928 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:34.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.729+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558ef7eed430 0x558ef7eef850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:34.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.729+0000 7f2eb6a84b80 4 client.25928 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:36:34.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.729+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/726631696 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558ef7d98ec0 con 0x558ef7eed430 2024-05-10T05:36:34.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.731+0000 7f2e97e00700 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558ef7eed430 0x558ef7eef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:34.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.733+0000 7f2e97e00700 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558ef7eed430 0x558ef7eef850 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb556cae1d86b96 server_cookie=8cab2e1142105fbf in_seq=0 out_seq=0 2024-05-10T05:36:34.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.733+0000 7f2e81600700 10 client.25928 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:34.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.737+0000 7f2e81600700 1 -- 172.21.15.131:0/726631696 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558ef7d98ec0 con 0x558ef7eed430 2024-05-10T05:36:34.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.737+0000 7f2e81600700 10 client.25928 handle_command_reply: tid=0 2024-05-10T05:36:34.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.737+0000 7f2eb6a84b80 1 client.25928 shutdown 2024-05-10T05:36:34.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.737+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/726631696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e8fc20 msgr2=0x558ef7e90030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.737+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e8fc20 0x558ef7e90030 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e88096280 tx=0x7f2e8800cf80 comp rx=0 tx=0).stop 2024-05-10T05:36:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.738+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/726631696 shutdown_connections 2024-05-10T05:36:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.738+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558ef7eed430 0x558ef7eef850 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.738+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ef7e8fc20 0x558ef7e90030 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.738+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ef7e81570 0x558ef7e91510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.738+0000 7f2eb6a84b80 1 --2- 172.21.15.131:0/726631696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ef7e81ec0 0x558ef7e91a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.738+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/726631696 >> 172.21.15.131:0/726631696 conn(0x558ef7de2a60 msgr2=0x558ef7e8a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.738+0000 7f2e81600700 0 client.25928 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.738+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/726631696 shutdown_connections 2024-05-10T05:36:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.738+0000 7f2eb6a84b80 1 -- 172.21.15.131:0/726631696 wait complete. 2024-05-10T05:36:34.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.741+0000 7f2eb6a84b80 20 client.25928 trim_cache size 0 max 16384 2024-05-10T05:36:34.743 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:36:34.743 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:36:34.743 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:36:34.743 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:36:34.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.742+0000 7f2e9ec00700 1 -- 172.21.15.131:0/3156494484 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2e7c0a1240 msgr2=0x7f2e7c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:34.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.742+0000 7f2e9ec00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2e7c0a1240 0x7f2e7c0a3750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e981a81d0 tx=0x7f2e8400a480 comp rx=0 tx=0).stop 2024-05-10T05:36:34.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.742+0000 7f2e9ec00700 1 -- 172.21.15.131:0/3156494484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9810a0d0 msgr2=0x7f2e981ae270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:34.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.742+0000 7f2e9ec00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9810a0d0 0x7f2e981ae270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800eab0 tx=0x7f2e8800edc0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.742+0000 7f2e9ec00700 1 -- 172.21.15.131:0/3156494484 shutdown_connections 2024-05-10T05:36:34.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.742+0000 7f2e9ec00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2e7c0a1240 0x7f2e7c0a3750 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.742+0000 7f2e9ec00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e9810a0d0 0x7f2e981ae270 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.742+0000 7f2e9ec00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e98108610 0x7f2e981a6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.742+0000 7f2e9ec00700 1 --2- 172.21.15.131:0/3156494484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e98109780 0x7f2e981a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.742+0000 7f2e9ec00700 1 -- 172.21.15.131:0/3156494484 >> 172.21.15.131:0/3156494484 conn(0x7f2e9806ee80 msgr2=0x7f2e9810c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.743+0000 7f2e9ec00700 1 -- 172.21.15.131:0/3156494484 shutdown_connections 2024-05-10T05:36:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:34.743+0000 7f2e9ec00700 1 -- 172.21.15.131:0/3156494484 wait complete. 2024-05-10T05:36:34.821 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:36:34.821 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:36:35.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.500+0000 7f7036a00700 1 -- 172.21.15.131:0/1721336870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7030109cd0 msgr2=0x7f7030110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:35.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.500+0000 7f7036a00700 1 --2- 172.21.15.131:0/1721336870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7030109cd0 0x7f7030110580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f70180099c0 tx=0x7f7018009cd0 comp rx=0 tx=0).stop 2024-05-10T05:36:35.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.500+0000 7f7036a00700 1 -- 172.21.15.131:0/1721336870 shutdown_connections 2024-05-10T05:36:35.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.500+0000 7f7036a00700 1 --2- 172.21.15.131:0/1721336870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7030109cd0 0x7f7030110580 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:35.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.500+0000 7f7036a00700 1 --2- 172.21.15.131:0/1721336870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7030108210 0x7f7030108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:35.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.500+0000 7f7036a00700 1 --2- 172.21.15.131:0/1721336870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7030109330 0x7f7030109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:35.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.500+0000 7f7036a00700 1 -- 172.21.15.131:0/1721336870 >> 172.21.15.131:0/1721336870 conn(0x7f7030077240 msgr2=0x7f7030106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:35.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.511+0000 7f7036a00700 1 -- 172.21.15.131:0/1721336870 shutdown_connections 2024-05-10T05:36:35.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.514+0000 7f7036a00700 1 -- 172.21.15.131:0/1721336870 wait complete. 2024-05-10T05:36:35.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.516+0000 7f7036a00700 1 Processor -- start 2024-05-10T05:36:35.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.528+0000 7f7036a00700 1 -- start start 2024-05-10T05:36:35.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.528+0000 7f7036a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7030108210 0x7f7030072840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:35.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.528+0000 7f7036a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7030109330 0x7f7030072d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:35.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.529+0000 7f7036a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7030109cd0 0x7f7030070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:35.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.529+0000 7f7036a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7030071600 con 0x7f7030109330 2024-05-10T05:36:35.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.529+0000 7f7036a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7030071770 con 0x7f7030108210 2024-05-10T05:36:35.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.529+0000 7f7036a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7030071930 con 0x7f7030109cd0 2024-05-10T05:36:35.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.529+0000 7f7034a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7030109cd0 0x7f7030070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:35.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.529+0000 7f7034a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7030109cd0 0x7f7030070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60806/0 (socket says 172.21.15.131:60806) 2024-05-10T05:36:35.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.529+0000 7f7034a00700 1 -- 172.21.15.131:0/4189486085 learned_addr learned my addr 172.21.15.131:0/4189486085 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:35.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.530+0000 7f702f400700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7030109330 0x7f7030072d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:35.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.532+0000 7f702fe00700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7030108210 0x7f7030072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:35.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.533+0000 7f702f400700 1 -- 172.21.15.131:0/4189486085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7030109cd0 msgr2=0x7f7030070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:35.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.533+0000 7f702f400700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7030109cd0 0x7f7030070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:35.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.533+0000 7f702f400700 1 -- 172.21.15.131:0/4189486085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7030108210 msgr2=0x7f7030072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:35.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.533+0000 7f702f400700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7030108210 0x7f7030072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:35.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.533+0000 7f702f400700 1 -- 172.21.15.131:0/4189486085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180095f0 con 0x7f7030109330 2024-05-10T05:36:35.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.534+0000 7f702f400700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7030109330 0x7f7030072d80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f702800cfd0 tx=0x7f7028007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:35.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.535+0000 7f702cc00700 1 -- 172.21.15.131:0/4189486085 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702800ab70 con 0x7f7030109330 2024-05-10T05:36:35.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.535+0000 7f7036a00700 1 -- 172.21.15.131:0/4189486085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7030071c10 con 0x7f7030109330 2024-05-10T05:36:35.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.535+0000 7f702cc00700 1 -- 172.21.15.131:0/4189486085 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f702800acd0 con 0x7f7030109330 2024-05-10T05:36:35.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.535+0000 7f702cc00700 1 -- 172.21.15.131:0/4189486085 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702800ab70 con 0x7f7030109330 2024-05-10T05:36:35.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.535+0000 7f7036a00700 1 -- 172.21.15.131:0/4189486085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70301af3f0 con 0x7f7030109330 2024-05-10T05:36:35.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.540+0000 7f7036a00700 1 -- 172.21.15.131:0/4189486085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703010f420 con 0x7f7030109330 2024-05-10T05:36:35.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.549+0000 7f702cc00700 1 -- 172.21.15.131:0/4189486085 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f702801b410 con 0x7f7030109330 2024-05-10T05:36:35.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.555+0000 7f702cc00700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70140a1280 0x7f70140a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:35.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.556+0000 7f702cc00700 1 -- 172.21.15.131:0/4189486085 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f70280cc410 con 0x7f7030109330 2024-05-10T05:36:35.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.559+0000 7f702fe00700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70140a1280 0x7f70140a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:35.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.562+0000 7f702cc00700 1 -- 172.21.15.131:0/4189486085 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7028095b40 con 0x7f7030109330 2024-05-10T05:36:35.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:35.565+0000 7f702fe00700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70140a1280 0x7f70140a3790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7030109230 tx=0x7f7020008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:36.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.131+0000 7f7036a00700 1 -- 172.21.15.131:0/4189486085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7030071da0 con 0x7f7030109330 2024-05-10T05:36:36.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.133+0000 7f702cc00700 1 -- 172.21.15.131:0/4189486085 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10931 (secure 0 0 0) 0x7f7028099160 con 0x7f7030109330 2024-05-10T05:36:36.134 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:36:36.134 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:35:26.081961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:36:36.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.138+0000 7f7036a00700 1 -- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70140a1280 msgr2=0x7f70140a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:36.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.138+0000 7f7036a00700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70140a1280 0x7f70140a3790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7030109230 tx=0x7f7020008040 comp rx=0 tx=0).stop 2024-05-10T05:36:36.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.138+0000 7f7036a00700 1 -- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7030109330 msgr2=0x7f7030072d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:36.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.138+0000 7f7036a00700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7030109330 0x7f7030072d80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f702800cfd0 tx=0x7f7028007f90 comp rx=0 tx=0).stop 2024-05-10T05:36:36.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.138+0000 7f7036a00700 1 -- 172.21.15.131:0/4189486085 shutdown_connections 2024-05-10T05:36:36.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.139+0000 7f7036a00700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70140a1280 0x7f70140a3790 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.139+0000 7f7036a00700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7030109cd0 0x7f7030070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.139+0000 7f7036a00700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7030108210 0x7f7030072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.139+0000 7f7036a00700 1 --2- 172.21.15.131:0/4189486085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7030109330 0x7f7030072d80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.139+0000 7f7036a00700 1 -- 172.21.15.131:0/4189486085 >> 172.21.15.131:0/4189486085 conn(0x7f7030077240 msgr2=0x7f703010c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.142+0000 7f7036a00700 1 -- 172.21.15.131:0/4189486085 shutdown_connections 2024-05-10T05:36:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.142+0000 7f7036a00700 1 -- 172.21.15.131:0/4189486085 wait complete. 2024-05-10T05:36:36.157 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 28 2024-05-10T05:36:36.237 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:36:36.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:36 smithi157 ceph-mon[78427]: pgmap v382: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 265 MiB/s wr, 80 op/s 2024-05-10T05:36:36.344 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:36 smithi131 ceph-mon[37264]: pgmap v382: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 265 MiB/s wr, 80 op/s 2024-05-10T05:36:36.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:36 smithi142 ceph-mon[77667]: pgmap v382: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 265 MiB/s wr, 80 op/s 2024-05-10T05:36:36.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.836+0000 7ff933c00700 1 -- 172.21.15.131:0/1230918171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff92409d6b0 msgr2=0x7ff9240a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:36.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.836+0000 7ff933c00700 1 --2- 172.21.15.131:0/1230918171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff92409d6b0 0x7ff9240a4fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff9200098f0 tx=0x7ff920009c00 comp rx=0 tx=0).stop 2024-05-10T05:36:36.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.839+0000 7ff933c00700 1 -- 172.21.15.131:0/1230918171 shutdown_connections 2024-05-10T05:36:36.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.839+0000 7ff933c00700 1 --2- 172.21.15.131:0/1230918171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff92409e140 0x7ff92409e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.839+0000 7ff933c00700 1 --2- 172.21.15.131:0/1230918171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff92409d6b0 0x7ff9240a4fc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.839+0000 7ff933c00700 1 --2- 172.21.15.131:0/1230918171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92409cd60 0x7ff92409d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.839+0000 7ff933c00700 1 -- 172.21.15.131:0/1230918171 >> 172.21.15.131:0/1230918171 conn(0x7ff92400bc40 msgr2=0x7ff92400c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:36.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.845+0000 7ff933c00700 1 -- 172.21.15.131:0/1230918171 shutdown_connections 2024-05-10T05:36:36.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.845+0000 7ff933c00700 1 -- 172.21.15.131:0/1230918171 wait complete. 2024-05-10T05:36:36.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.856+0000 7ff933c00700 1 Processor -- start 2024-05-10T05:36:36.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.858+0000 7ff933c00700 1 -- start start 2024-05-10T05:36:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.864+0000 7ff933c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff92409cd60 0x7ff924132e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.867+0000 7ff933c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff92409d6b0 0x7ff924133350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.867+0000 7ff933c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92409e140 0x7ff92413a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.867+0000 7ff92be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff92409d6b0 0x7ff924133350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.867+0000 7ff92be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff92409d6b0 0x7ff924133350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60826/0 (socket says 172.21.15.131:60826) 2024-05-10T05:36:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.867+0000 7ff931200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff92409cd60 0x7ff924132e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.867+0000 7ff931200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff92409cd60 0x7ff924132e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35820/0 (socket says 172.21.15.131:35820) 2024-05-10T05:36:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.868+0000 7ff92be00700 1 -- 172.21.15.131:0/4138974181 learned_addr learned my addr 172.21.15.131:0/4138974181 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.868+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92413a8b0 con 0x7ff92409cd60 2024-05-10T05:36:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.868+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92413a9f0 con 0x7ff92409e140 2024-05-10T05:36:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.868+0000 7ff932600700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92409e140 0x7ff92413a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.868+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92413ab80 con 0x7ff92409d6b0 2024-05-10T05:36:36.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.876+0000 7ff931200700 1 -- 172.21.15.131:0/4138974181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff92409d6b0 msgr2=0x7ff924133350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.876+0000 7ff931200700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff92409d6b0 0x7ff924133350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.876+0000 7ff931200700 1 -- 172.21.15.131:0/4138974181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92409e140 msgr2=0x7ff92413a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:36.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.877+0000 7ff931200700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92409e140 0x7ff92413a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.877+0000 7ff931200700 1 -- 172.21.15.131:0/4138974181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920009520 con 0x7ff92409cd60 2024-05-10T05:36:36.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.877+0000 7ff932600700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92409e140 0x7ff92413a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:36.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.880+0000 7ff931200700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff92409cd60 0x7ff924132e10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff92409de30 tx=0x7ff92c074f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.884+0000 7ff929600700 1 -- 172.21.15.131:0/4138974181 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c06a620 con 0x7ff92409cd60 2024-05-10T05:36:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.884+0000 7ff929600700 1 -- 172.21.15.131:0/4138974181 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff92c07c040 con 0x7ff92409cd60 2024-05-10T05:36:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.884+0000 7ff929600700 1 -- 172.21.15.131:0/4138974181 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0813b0 con 0x7ff92409cd60 2024-05-10T05:36:36.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.891+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92413ae60 con 0x7ff92409cd60 2024-05-10T05:36:36.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.891+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92413b1d0 con 0x7ff92409cd60 2024-05-10T05:36:36.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.913+0000 7ff94bbf6b80 1 Processor -- start 2024-05-10T05:36:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.917+0000 7ff94bbf6b80 1 -- start start 2024-05-10T05:36:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.919+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f361530 0x561f5f361960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.919+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f362bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.919+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3630f0 0x561f5f3699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.919+0000 7ff94bbf6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f288150 con 0x561f5f3627a0 2024-05-10T05:36:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.919+0000 7ff94bbf6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f285b10 con 0x561f5f3630f0 2024-05-10T05:36:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.919+0000 7ff94bbf6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f281400 con 0x561f5f361530 2024-05-10T05:36:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.920+0000 7ff932600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f361530 0x561f5f361960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.920+0000 7ff932600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f361530 0x561f5f361960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60828/0 (socket says 172.21.15.131:60828) 2024-05-10T05:36:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.918+0000 7ff929600700 1 -- 172.21.15.131:0/4138974181 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff92c081510 con 0x7ff92409cd60 2024-05-10T05:36:36.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.921+0000 7ff932600700 1 -- 172.21.15.131:0/1053756282 learned_addr learned my addr 172.21.15.131:0/1053756282 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:36.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.921+0000 7ff931200700 1 --2- 172.21.15.131:0/1053756282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f362bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.921+0000 7ff92be00700 1 --2- 172.21.15.131:0/1053756282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3630f0 0x561f5f3699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.922+0000 7ff92be00700 1 -- 172.21.15.131:0/1053756282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f361530 msgr2=0x561f5f361960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.922+0000 7ff92be00700 1 --2- 172.21.15.131:0/1053756282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f361530 0x561f5f361960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.922+0000 7ff92be00700 1 -- 172.21.15.131:0/1053756282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 msgr2=0x561f5f362bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.922+0000 7ff92be00700 1 --2- 172.21.15.131:0/1053756282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f362bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.922+0000 7ff92be00700 1 -- 172.21.15.131:0/1053756282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f5f1023b0 con 0x561f5f3630f0 2024-05-10T05:36:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.922+0000 7ff92be00700 1 --2- 172.21.15.131:0/1053756282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3630f0 0x561f5f3699f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff918009ab0 tx=0x7ff918009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2ac6d7bc1e7372 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.922+0000 7ff91e000700 1 -- 172.21.15.131:0/1053756282 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800e8d0 con 0x561f5f3630f0 2024-05-10T05:36:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.923+0000 7ff91e000700 1 -- 172.21.15.131:0/1053756282 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff918013470 con 0x561f5f3630f0 2024-05-10T05:36:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.924+0000 7ff91e000700 1 -- 172.21.15.131:0/1053756282 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800e8d0 con 0x561f5f3630f0 2024-05-10T05:36:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.926+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/1053756282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3630f0 msgr2=0x561f5f3699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.926+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/1053756282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3630f0 0x561f5f3699f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff918009ab0 tx=0x7ff918009dc0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.928+0000 7ff929600700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9140a12f0 0x7ff9140a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.928+0000 7ff929600700 1 -- 172.21.15.131:0/4138974181 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff92c136c60 con 0x7ff92409cd60 2024-05-10T05:36:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.929+0000 7ff932600700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9140a12f0 0x7ff9140a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.933+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/1053756282 shutdown_connections 2024-05-10T05:36:36.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.933+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/1053756282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3630f0 0x561f5f3699f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.933+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/1053756282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f361530 0x561f5f361960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.933+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/1053756282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f362bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.933+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/1053756282 >> 172.21.15.131:0/1053756282 conn(0x561f5f2c3a60 msgr2=0x561f5f35c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:36.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.933+0000 7ff932600700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9140a12f0 0x7ff9140a3800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x561f5f3612d0 tx=0x7ff920019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:36.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.935+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/1053756282 shutdown_connections 2024-05-10T05:36:36.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.936+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/1053756282 wait complete. 2024-05-10T05:36:36.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.937+0000 7ff94bbf6b80 1 -- start start 2024-05-10T05:36:36.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.938+0000 7ff94bbf6b80 1 Processor -- start 2024-05-10T05:36:36.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.940+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f361530 0x561f5f37c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.940+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3627a0 0x561f5f37cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.941+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f383d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.941+0000 7ff92be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3627a0 0x561f5f37cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.941+0000 7ff92be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3627a0 0x561f5f37cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34042/0 (socket says 172.21.15.131:34042) 2024-05-10T05:36:36.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.942+0000 7ff932600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f383d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.942+0000 7ff932600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f383d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60842/0 (socket says 172.21.15.131:60842) 2024-05-10T05:36:36.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.943+0000 7ff94bbf6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f21d560 con 0x561f5f361530 2024-05-10T05:36:36.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.943+0000 7ff94bbf6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f21d200 con 0x561f5f3627a0 2024-05-10T05:36:36.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.943+0000 7ff94bbf6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f1046a0 con 0x561f5f3630f0 2024-05-10T05:36:36.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.943+0000 7ff94bbf6b80 10 client.0 _ll_register_callbacks cb 0x561f5f226b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:36.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.943+0000 7ff92be00700 1 -- 172.21.15.131:0/949227208 learned_addr learned my addr 172.21.15.131:0/949227208 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:36.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.944+0000 7ff931200700 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f361530 0x561f5f37c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.944+0000 7ff931200700 1 -- 172.21.15.131:0/949227208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 msgr2=0x561f5f383d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:36.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.944+0000 7ff931200700 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f383d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.944+0000 7ff931200700 1 -- 172.21.15.131:0/949227208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3627a0 msgr2=0x561f5f37cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:36.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.945+0000 7ff931200700 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3627a0 0x561f5f37cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.946+0000 7ff931200700 1 -- 172.21.15.131:0/949227208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff918009710 con 0x561f5f361530 2024-05-10T05:36:36.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.946+0000 7ff932600700 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f383d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:36:36.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.946+0000 7ff931200700 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f361530 0x561f5f37c870 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff92c1346d0 tx=0x7ff92c067280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9124faf3e38ba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:36.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.946+0000 7ff92be00700 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3627a0 0x561f5f37cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:36.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.947+0000 7ff912a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:36:36.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.949+0000 7ff912a00700 1 -- 172.21.15.131:0/949227208 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c1044b0 con 0x561f5f361530 2024-05-10T05:36:36.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.949+0000 7ff912a00700 1 -- 172.21.15.131:0/949227208 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff92c107e80 con 0x561f5f361530 2024-05-10T05:36:36.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.949+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/949227208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f5f1da270 con 0x561f5f361530 2024-05-10T05:36:36.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.950+0000 7ff912a00700 1 -- 172.21.15.131:0/949227208 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff92c10c4d0 con 0x561f5f361530 2024-05-10T05:36:36.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.950+0000 7ff94bbf6b80 10 client.15879 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:36.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.950+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/949227208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561f5f0aeb60 con 0x561f5f361530 2024-05-10T05:36:36.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.950+0000 7ff912a00700 1 -- 172.21.15.131:0/949227208 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f5f1da270 con 0x561f5f361530 2024-05-10T05:36:36.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.951+0000 7ff912a00700 1 -- 172.21.15.131:0/949227208 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff92c10c040 con 0x561f5f361530 2024-05-10T05:36:36.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.951+0000 7ff94bbf6b80 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:36.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.951+0000 7ff94bbf6b80 10 client.15879 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:36:36.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.951+0000 7ff94bbf6b80 20 client.15879 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:36.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.951+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561f5f388520 0x561f5f38a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.951+0000 7ff94bbf6b80 4 client.15879 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:36.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.951+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/949227208 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f5f330780 con 0x561f5f388520 2024-05-10T05:36:36.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.952+0000 7ff92be00700 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561f5f388520 0x561f5f38a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.953+0000 7ff92be00700 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561f5f388520 0x561f5f38a970 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dfee8a86cbda9636 server_cookie=61c95e01cf519d7b in_seq=0 out_seq=0 2024-05-10T05:36:36.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.953+0000 7ff912a00700 10 client.15879 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:36:36.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.954+0000 7ff912a00700 1 -- 172.21.15.131:0/949227208 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f5f330780 con 0x561f5f388520 2024-05-10T05:36:36.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.954+0000 7ff912a00700 10 client.15879 handle_command_reply: tid=0 2024-05-10T05:36:36.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.954+0000 7ff94bbf6b80 1 client.15879 shutdown 2024-05-10T05:36:36.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.955+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/949227208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f361530 msgr2=0x561f5f37c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:36.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.955+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f361530 0x561f5f37c870 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff92c1346d0 tx=0x7ff92c067280 comp rx=0 tx=0).stop 2024-05-10T05:36:36.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.955+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/949227208 shutdown_connections 2024-05-10T05:36:36.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.955+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561f5f388520 0x561f5f38a970 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.955+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f361530 0x561f5f37c870 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.955+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f383d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.955+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/949227208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f3627a0 0x561f5f37cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.956+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/949227208 >> 172.21.15.131:0/949227208 conn(0x561f5f2c3a60 msgr2=0x561f5f360060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:36.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.956+0000 7ff912a00700 0 client.15879 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:36:36.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.956+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/949227208 shutdown_connections 2024-05-10T05:36:36.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.956+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/949227208 wait complete. 2024-05-10T05:36:36.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.956+0000 7ff94bbf6b80 20 client.15879 trim_cache size 0 max 16384 2024-05-10T05:36:36.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.984+0000 7ff94bbf6b80 1 Processor -- start 2024-05-10T05:36:36.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.984+0000 7ff94bbf6b80 1 -- start start 2024-05-10T05:36:36.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.985+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 0x561f5f366a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.985+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f383ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.985+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f35c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.986+0000 7ff94bbf6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f361e10 con 0x561f5f3627a0 2024-05-10T05:36:36.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.986+0000 7ff94bbf6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f362590 con 0x561f5f361530 2024-05-10T05:36:36.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.986+0000 7ff932600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f35c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.986+0000 7ff932600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f35c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60850/0 (socket says 172.21.15.131:60850) 2024-05-10T05:36:36.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.986+0000 7ff932600700 1 -- 172.21.15.131:0/4198232105 learned_addr learned my addr 172.21.15.131:0/4198232105 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:36.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.986+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/4198232105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f369720 con 0x561f5f3630f0 2024-05-10T05:36:36.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.986+0000 7ff931200700 1 --2- 172.21.15.131:0/4198232105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 0x561f5f366a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.988+0000 7ff92be00700 1 --2- 172.21.15.131:0/4198232105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f383ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:36.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.990+0000 7ff92be00700 1 -- 172.21.15.131:0/4198232105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 msgr2=0x561f5f35c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:36.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.990+0000 7ff92be00700 1 --2- 172.21.15.131:0/4198232105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f35c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.990+0000 7ff92be00700 1 -- 172.21.15.131:0/4198232105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 msgr2=0x561f5f366a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:36.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.990+0000 7ff92be00700 1 --2- 172.21.15.131:0/4198232105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 0x561f5f366a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.990+0000 7ff92be00700 1 -- 172.21.15.131:0/4198232105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f5f247560 con 0x561f5f3627a0 2024-05-10T05:36:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.992+0000 7ff92be00700 1 --2- 172.21.15.131:0/4198232105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f383ca0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff91800a010 tx=0x7ff918008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a7a196c0286545 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.993+0000 7ff91ea00700 1 -- 172.21.15.131:0/4198232105 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918021040 con 0x561f5f3627a0 2024-05-10T05:36:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.993+0000 7ff91ea00700 1 -- 172.21.15.131:0/4198232105 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff918023040 con 0x561f5f3627a0 2024-05-10T05:36:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.993+0000 7ff91ea00700 1 -- 172.21.15.131:0/4198232105 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800e8d0 con 0x561f5f3627a0 2024-05-10T05:36:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.994+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/4198232105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 msgr2=0x561f5f383ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.994+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/4198232105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f383ca0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff91800a010 tx=0x7ff918008040 comp rx=0 tx=0).stop 2024-05-10T05:36:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.995+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/4198232105 shutdown_connections 2024-05-10T05:36:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.995+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/4198232105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f35c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.995+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/4198232105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 0x561f5f366a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.995+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/4198232105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f383ca0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.995+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/4198232105 >> 172.21.15.131:0/4198232105 conn(0x561f5f2c3a60 msgr2=0x561f5f35bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.995+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/4198232105 shutdown_connections 2024-05-10T05:36:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.996+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/4198232105 wait complete. 2024-05-10T05:36:36.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.996+0000 7ff94bbf6b80 1 -- start start 2024-05-10T05:36:36.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.997+0000 7ff94bbf6b80 1 Processor -- start 2024-05-10T05:36:36.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.998+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 0x561f5f381b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.998+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f3820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:36.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.998+0000 7ff94bbf6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f382f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.998+0000 7ff931200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 0x561f5f381b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.998+0000 7ff931200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 0x561f5f381b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34060/0 (socket says 172.21.15.131:34060) 2024-05-10T05:36:37.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.998+0000 7ff931200700 1 -- 172.21.15.131:0/2177500607 learned_addr learned my addr 172.21.15.131:0/2177500607 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:37.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.998+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f383490 con 0x561f5f3627a0 2024-05-10T05:36:37.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.998+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f383600 con 0x561f5f361530 2024-05-10T05:36:37.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.998+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f5f3837c0 con 0x561f5f3630f0 2024-05-10T05:36:37.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.998+0000 7ff92be00700 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f3820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.999+0000 7ff94bbf6b80 10 client.0 _ll_register_callbacks cb 0x561f5f226b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:37.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.999+0000 7ff932600700 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f382f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.999+0000 7ff931200700 1 -- 172.21.15.131:0/2177500607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 msgr2=0x561f5f382f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:37.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.999+0000 7ff931200700 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f382f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:36.999+0000 7ff931200700 1 -- 172.21.15.131:0/2177500607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 msgr2=0x561f5f3820c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:37.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.000+0000 7ff931200700 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f3820c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.000+0000 7ff931200700 1 -- 172.21.15.131:0/2177500607 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f5f1023b0 con 0x561f5f361530 2024-05-10T05:36:37.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.000+0000 7ff92be00700 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f3820c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.005+0000 7ff931200700 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 0x561f5f381b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff92c050560 tx=0x7ff92c067100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53caec798fabe42b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.005+0000 7ff912a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:36:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.005+0000 7ff912a00700 1 -- 172.21.15.131:0/2177500607 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c122850 con 0x561f5f361530 2024-05-10T05:36:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.006+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff918013470 con 0x561f5f361530 2024-05-10T05:36:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.006+0000 7ff912a00700 1 -- 172.21.15.131:0/2177500607 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff92c107e80 con 0x561f5f361530 2024-05-10T05:36:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.006+0000 7ff912a00700 1 -- 172.21.15.131:0/2177500607 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff92c10c4d0 con 0x561f5f361530 2024-05-10T05:36:37.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.011+0000 7ff94bbf6b80 10 client.25946 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:37.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.012+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561f5f106cd0 con 0x561f5f361530 2024-05-10T05:36:37.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.012+0000 7ff912a00700 1 -- 172.21.15.131:0/2177500607 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff92c112020 con 0x561f5f361530 2024-05-10T05:36:37.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.013+0000 7ff912a00700 1 -- 172.21.15.131:0/2177500607 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff92c10c040 con 0x561f5f361530 2024-05-10T05:36:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.015+0000 7ff94bbf6b80 10 client.25946 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.015+0000 7ff94bbf6b80 10 client.25946 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:36:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.015+0000 7ff94bbf6b80 20 client.25946 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.015+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561f5f3df400 0x561f5f3e1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.015+0000 7ff94bbf6b80 4 client.25946 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:36:37.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.015+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561f5f279ec0 con 0x561f5f3df400 2024-05-10T05:36:37.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.017+0000 7ff92be00700 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561f5f3df400 0x561f5f3e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.018+0000 7ff92be00700 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561f5f3df400 0x561f5f3e1820 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aacd288a5b0f53ed server_cookie=6d9ab9d4dbd36f3e in_seq=0 out_seq=0 2024-05-10T05:36:37.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.018+0000 7ff912a00700 10 client.25946 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:36:37.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.018+0000 7ff912a00700 1 -- 172.21.15.131:0/2177500607 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561f5f279ec0 con 0x561f5f3df400 2024-05-10T05:36:37.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.018+0000 7ff912a00700 10 client.25946 handle_command_reply: tid=0 2024-05-10T05:36:37.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.020+0000 7ff94bbf6b80 1 client.25946 shutdown 2024-05-10T05:36:37.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.021+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 msgr2=0x561f5f381b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.021+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 0x561f5f381b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff92c050560 tx=0x7ff92c067100 comp rx=0 tx=0).stop 2024-05-10T05:36:37.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.023+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 shutdown_connections 2024-05-10T05:36:37.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.023+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561f5f3df400 0x561f5f3e1820 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.023+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5f3630f0 0x561f5f382f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.023+0000 7ff912a00700 0 client.25946 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:36:37.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.023+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f5f361530 0x561f5f381b80 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff92c050560 tx=0x7ff92c067100 comp rx=0 tx=0).stop 2024-05-10T05:36:37.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.023+0000 7ff94bbf6b80 1 --2- 172.21.15.131:0/2177500607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561f5f3627a0 0x561f5f3820c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.023+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 >> 172.21.15.131:0/2177500607 conn(0x561f5f2c3a60 msgr2=0x561f5f35c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:37.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.027+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 shutdown_connections 2024-05-10T05:36:37.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.029+0000 7ff94bbf6b80 1 -- 172.21.15.131:0/2177500607 wait complete. 2024-05-10T05:36:37.034 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:36:37.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.032+0000 7ff94bbf6b80 20 client.25946 trim_cache size 0 max 16384 2024-05-10T05:36:37.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.036+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9140a12f0 msgr2=0x7ff9140a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.036+0000 7ff933c00700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9140a12f0 0x7ff9140a3800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x561f5f3612d0 tx=0x7ff920019040 comp rx=0 tx=0).stop 2024-05-10T05:36:37.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.037+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff92409cd60 msgr2=0x7ff924132e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.037+0000 7ff933c00700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff92409cd60 0x7ff924132e10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff92409de30 tx=0x7ff92c074f90 comp rx=0 tx=0).stop 2024-05-10T05:36:37.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.037+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 shutdown_connections 2024-05-10T05:36:37.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.037+0000 7ff933c00700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9140a12f0 0x7ff9140a3800 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.037+0000 7ff933c00700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff92409cd60 0x7ff924132e10 secure :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff92409de30 tx=0x7ff92c074f90 comp rx=0 tx=0).stop 2024-05-10T05:36:37.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.037+0000 7ff933c00700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92409e140 0x7ff92413a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.038+0000 7ff933c00700 1 --2- 172.21.15.131:0/4138974181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff92409d6b0 0x7ff924133350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.038+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 >> 172.21.15.131:0/4138974181 conn(0x7ff92400bc40 msgr2=0x7ff9240a0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:37.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.039+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 shutdown_connections 2024-05-10T05:36:37.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.039+0000 7ff933c00700 1 -- 172.21.15.131:0/4138974181 wait complete. 2024-05-10T05:36:37.110 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:36:37.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:37 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4189486085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:36:37.467 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:37 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4189486085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:36:37.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:37 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4189486085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:36:37.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.667+0000 7fc22dc00700 1 -- 172.21.15.131:0/3253400375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2200980c0 msgr2=0x7fc22009e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.667+0000 7fc22dc00700 1 --2- 172.21.15.131:0/3253400375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2200980c0 0x7fc22009e970 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc22804efb0 tx=0x7fc228069cb0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.670+0000 7fc22dc00700 1 -- 172.21.15.131:0/3253400375 shutdown_connections 2024-05-10T05:36:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.670+0000 7fc22dc00700 1 --2- 172.21.15.131:0/3253400375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc220096600 0x7fc220096a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.670+0000 7fc22dc00700 1 --2- 172.21.15.131:0/3253400375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2200980c0 0x7fc22009e970 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.670+0000 7fc22dc00700 1 --2- 172.21.15.131:0/3253400375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc220097720 0x7fc220097b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.670+0000 7fc22dc00700 1 -- 172.21.15.131:0/3253400375 >> 172.21.15.131:0/3253400375 conn(0x7fc220091c10 msgr2=0x7fc2200940c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.670+0000 7fc22dc00700 1 -- 172.21.15.131:0/3253400375 shutdown_connections 2024-05-10T05:36:37.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.671+0000 7fc22dc00700 1 -- 172.21.15.131:0/3253400375 wait complete. 2024-05-10T05:36:37.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.673+0000 7fc22dc00700 1 Processor -- start 2024-05-10T05:36:37.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.673+0000 7fc22dc00700 1 -- start start 2024-05-10T05:36:37.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.674+0000 7fc22dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc220096600 0x7fc22012c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.674+0000 7fc22dc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc220097720 0x7fc22012cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.674+0000 7fc22dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2200980c0 0x7fc220133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.675+0000 7fc22dc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc220134400 con 0x7fc220097720 2024-05-10T05:36:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.675+0000 7fc22dc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc220134540 con 0x7fc220096600 2024-05-10T05:36:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.675+0000 7fc22dc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201346d0 con 0x7fc2200980c0 2024-05-10T05:36:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.675+0000 7fc226000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc220097720 0x7fc22012cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.675+0000 7fc226000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc220097720 0x7fc22012cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35878/0 (socket says 172.21.15.131:35878) 2024-05-10T05:36:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.675+0000 7fc226000700 1 -- 172.21.15.131:0/6104042 learned_addr learned my addr 172.21.15.131:0/6104042 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.675+0000 7fc226a00700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc220096600 0x7fc22012c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.675+0000 7fc227400700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2200980c0 0x7fc220133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.677+0000 7fc226000700 1 -- 172.21.15.131:0/6104042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2200980c0 msgr2=0x7fc220133ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.677+0000 7fc226000700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2200980c0 0x7fc220133ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.677+0000 7fc226000700 1 -- 172.21.15.131:0/6104042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc220096600 msgr2=0x7fc22012c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.677+0000 7fc226000700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc220096600 0x7fc22012c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.677+0000 7fc226000700 1 -- 172.21.15.131:0/6104042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc228069700 con 0x7fc220097720 2024-05-10T05:36:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.677+0000 7fc226000700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc220097720 0x7fc22012cf30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc21400c9e0 tx=0x7fc21400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.677+0000 7fc21f400700 1 -- 172.21.15.131:0/6104042 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214007ac0 con 0x7fc220097720 2024-05-10T05:36:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.677+0000 7fc21f400700 1 -- 172.21.15.131:0/6104042 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc21400ce80 con 0x7fc220097720 2024-05-10T05:36:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.677+0000 7fc21f400700 1 -- 172.21.15.131:0/6104042 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214018660 con 0x7fc220097720 2024-05-10T05:36:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.678+0000 7fc22dc00700 1 -- 172.21.15.131:0/6104042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc220007d80 con 0x7fc220097720 2024-05-10T05:36:37.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.678+0000 7fc22dc00700 1 -- 172.21.15.131:0/6104042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc220008240 con 0x7fc220097720 2024-05-10T05:36:37.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.685+0000 7fc21f400700 1 -- 172.21.15.131:0/6104042 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc21400f450 con 0x7fc220097720 2024-05-10T05:36:37.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.686+0000 7fc245a90b80 1 Processor -- start 2024-05-10T05:36:37.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.687+0000 7fc245a90b80 1 -- start start 2024-05-10T05:36:37.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.688+0000 7fc21f400700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc20c0a55b0 0x7fc20c0a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.688+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca5959960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.688+0000 7fc21f400700 1 -- 172.21.15.131:0/6104042 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc2140cc800 con 0x7fc220097720 2024-05-10T05:36:37.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.689+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca595abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.689+0000 7fc226a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca5959960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.689+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca59619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.689+0000 7fc226a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca5959960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34100/0 (socket says 172.21.15.131:34100) 2024-05-10T05:36:37.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.689+0000 7fc245a90b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca5880150 con 0x557ca595b0f0 2024-05-10T05:36:37.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.689+0000 7fc245a90b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca587db10 con 0x557ca5959530 2024-05-10T05:36:37.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.689+0000 7fc245a90b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca5879400 con 0x557ca595a7a0 2024-05-10T05:36:37.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.689+0000 7fc226a00700 1 -- 172.21.15.131:0/3063081709 learned_addr learned my addr 172.21.15.131:0/3063081709 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:37.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.690+0000 7fc227400700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc20c0a55b0 0x7fc20c0a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.690+0000 7fc227400700 1 --2- 172.21.15.131:0/3063081709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca59619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.690+0000 7fc226000700 1 --2- 172.21.15.131:0/3063081709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca595abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.691+0000 7fc227400700 1 -- 172.21.15.131:0/3063081709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 msgr2=0x557ca595abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:37.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.691+0000 7fc227400700 1 --2- 172.21.15.131:0/3063081709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca595abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.691+0000 7fc227400700 1 -- 172.21.15.131:0/3063081709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 msgr2=0x557ca5959960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:37.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.691+0000 7fc227400700 1 --2- 172.21.15.131:0/3063081709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca5959960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.691+0000 7fc227400700 1 -- 172.21.15.131:0/3063081709 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ca56fa3b0 con 0x557ca595b0f0 2024-05-10T05:36:37.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.692+0000 7fc227400700 1 --2- 172.21.15.131:0/3063081709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca59619f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc228050560 tx=0x7fc228067100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b994992cb29567 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:37.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.692+0000 7fc227400700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc20c0a55b0 0x7fc20c0a7ac0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc22804fb30 tx=0x7fc228088000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:37.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.693+0000 7fc213e00700 1 -- 172.21.15.131:0/3063081709 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22807b040 con 0x557ca595b0f0 2024-05-10T05:36:37.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.693+0000 7fc213e00700 1 -- 172.21.15.131:0/3063081709 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc228068bd0 con 0x557ca595b0f0 2024-05-10T05:36:37.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.693+0000 7fc213e00700 1 -- 172.21.15.131:0/3063081709 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228080420 con 0x557ca595b0f0 2024-05-10T05:36:37.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.694+0000 7fc245a90b80 1 -- 172.21.15.131:0/3063081709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 msgr2=0x557ca59619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.694+0000 7fc245a90b80 1 --2- 172.21.15.131:0/3063081709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca59619f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc228050560 tx=0x7fc228067100 comp rx=0 tx=0).stop 2024-05-10T05:36:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.695+0000 7fc245a90b80 1 -- 172.21.15.131:0/3063081709 shutdown_connections 2024-05-10T05:36:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.695+0000 7fc245a90b80 1 --2- 172.21.15.131:0/3063081709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca59619f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.695+0000 7fc245a90b80 1 --2- 172.21.15.131:0/3063081709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca5959960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.695+0000 7fc245a90b80 1 --2- 172.21.15.131:0/3063081709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca595abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.695+0000 7fc245a90b80 1 -- 172.21.15.131:0/3063081709 >> 172.21.15.131:0/3063081709 conn(0x557ca58bba60 msgr2=0x557ca5954330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.695+0000 7fc245a90b80 1 -- 172.21.15.131:0/3063081709 shutdown_connections 2024-05-10T05:36:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.695+0000 7fc245a90b80 1 -- 172.21.15.131:0/3063081709 wait complete. 2024-05-10T05:36:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.696+0000 7fc245a90b80 1 -- start start 2024-05-10T05:36:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.697+0000 7fc245a90b80 1 Processor -- start 2024-05-10T05:36:37.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.698+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca5959530 0x557ca596c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.698+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca596c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.698+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca595b0f0 0x557ca59735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.698+0000 7fc245a90b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca5815560 con 0x557ca5959530 2024-05-10T05:36:37.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.698+0000 7fc245a90b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca5815200 con 0x557ca595b0f0 2024-05-10T05:36:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.698+0000 7fc245a90b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca56fc6a0 con 0x557ca595a7a0 2024-05-10T05:36:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.698+0000 7fc245a90b80 10 client.0 _ll_register_callbacks cb 0x557ca581eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.699+0000 7fc226a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca5959530 0x557ca596c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.699+0000 7fc226a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca5959530 0x557ca596c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35902/0 (socket says 172.21.15.131:35902) 2024-05-10T05:36:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.699+0000 7fc226a00700 1 -- 172.21.15.131:0/2801560313 learned_addr learned my addr 172.21.15.131:0/2801560313 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.699+0000 7fc227400700 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca595b0f0 0x557ca59735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.699+0000 7fc226000700 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca596c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.699+0000 7fc226a00700 1 -- 172.21.15.131:0/2801560313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 msgr2=0x557ca596c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:37.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.700+0000 7fc226a00700 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca596c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.700+0000 7fc226a00700 1 -- 172.21.15.131:0/2801560313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca595b0f0 msgr2=0x557ca59735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:37.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.700+0000 7fc226a00700 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca595b0f0 0x557ca59735b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.700+0000 7fc226a00700 1 -- 172.21.15.131:0/2801560313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22009f400 con 0x557ca5959530 2024-05-10T05:36:37.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.701+0000 7fc226a00700 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca5959530 0x557ca596c110 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc218012a70 tx=0x7fc218012e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ea5eea6c53d8b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:37.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.701+0000 7fc210c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:36:37.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.701+0000 7fc210c00700 1 -- 172.21.15.131:0/2801560313 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218017070 con 0x557ca5959530 2024-05-10T05:36:37.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.701+0000 7fc210c00700 1 -- 172.21.15.131:0/2801560313 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc218012190 con 0x557ca5959530 2024-05-10T05:36:37.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.701+0000 7fc245a90b80 1 -- 172.21.15.131:0/2801560313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ca57d2270 con 0x557ca5959530 2024-05-10T05:36:37.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.702+0000 7fc210c00700 1 -- 172.21.15.131:0/2801560313 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc21800f890 con 0x557ca5959530 2024-05-10T05:36:37.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.702+0000 7fc245a90b80 10 client.15897 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:37.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.702+0000 7fc245a90b80 1 -- 172.21.15.131:0/2801560313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557ca56a6b60 con 0x557ca5959530 2024-05-10T05:36:37.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.703+0000 7fc210c00700 1 -- 172.21.15.131:0/2801560313 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ca57d2270 con 0x557ca5959530 2024-05-10T05:36:37.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.704+0000 7fc210c00700 1 -- 172.21.15.131:0/2801560313 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc218010a10 con 0x557ca5959530 2024-05-10T05:36:37.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.705+0000 7fc245a90b80 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:37.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.705+0000 7fc245a90b80 10 client.15897 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:36:37.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.705+0000 7fc245a90b80 20 client.15897 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:37.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.705+0000 7fc245a90b80 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557ca5967850 0x557ca5969ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.705+0000 7fc245a90b80 4 client.15897 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:37.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.705+0000 7fc245a90b80 1 -- 172.21.15.131:0/2801560313 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ca5928780 con 0x557ca5967850 2024-05-10T05:36:37.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.705+0000 7fc226000700 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557ca5967850 0x557ca5969ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.706+0000 7fc226000700 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557ca5967850 0x557ca5969ca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5abd0caa78224c75 server_cookie=10de30df133c906a in_seq=0 out_seq=0 2024-05-10T05:36:37.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.710+0000 7fc210c00700 10 client.15897 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:36:37.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.710+0000 7fc210c00700 1 -- 172.21.15.131:0/2801560313 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ca5928780 con 0x557ca5967850 2024-05-10T05:36:37.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.710+0000 7fc210c00700 10 client.15897 handle_command_reply: tid=0 2024-05-10T05:36:37.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.710+0000 7fc245a90b80 1 client.15897 shutdown 2024-05-10T05:36:37.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.710+0000 7fc245a90b80 1 -- 172.21.15.131:0/2801560313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca5959530 msgr2=0x557ca596c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.710+0000 7fc245a90b80 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca5959530 0x557ca596c110 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc218012a70 tx=0x7fc218012e30 comp rx=0 tx=0).stop 2024-05-10T05:36:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.711+0000 7fc245a90b80 1 -- 172.21.15.131:0/2801560313 shutdown_connections 2024-05-10T05:36:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.711+0000 7fc245a90b80 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557ca5967850 0x557ca5969ca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.711+0000 7fc245a90b80 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca5959530 0x557ca596c110 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.711+0000 7fc210c00700 0 client.15897 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:36:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.711+0000 7fc245a90b80 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca595b0f0 0x557ca59735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.711+0000 7fc245a90b80 1 --2- 172.21.15.131:0/2801560313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca596c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.711+0000 7fc245a90b80 1 -- 172.21.15.131:0/2801560313 >> 172.21.15.131:0/2801560313 conn(0x557ca58bba60 msgr2=0x557ca5958060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.712+0000 7fc245a90b80 1 -- 172.21.15.131:0/2801560313 shutdown_connections 2024-05-10T05:36:37.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.712+0000 7fc245a90b80 1 -- 172.21.15.131:0/2801560313 wait complete. 2024-05-10T05:36:37.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.716+0000 7fc245a90b80 20 client.15897 trim_cache size 0 max 16384 2024-05-10T05:36:37.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.760+0000 7fc245a90b80 1 Processor -- start 2024-05-10T05:36:37.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.761+0000 7fc245a90b80 1 -- start start 2024-05-10T05:36:37.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.762+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca595e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.762+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca5973570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.762+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca595ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.763+0000 7fc226000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca5973570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.763+0000 7fc226000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca5973570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60920/0 (socket says 172.21.15.131:60920) 2024-05-10T05:36:37.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.763+0000 7fc227400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca595ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.763+0000 7fc227400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca595ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35904/0 (socket says 172.21.15.131:35904) 2024-05-10T05:36:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.763+0000 7fc226000700 1 -- 172.21.15.131:0/804680652 learned_addr learned my addr 172.21.15.131:0/804680652 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.763+0000 7fc245a90b80 1 -- 172.21.15.131:0/804680652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca594e320 con 0x557ca595b0f0 2024-05-10T05:36:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.763+0000 7fc245a90b80 1 -- 172.21.15.131:0/804680652 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca594e900 con 0x557ca5959530 2024-05-10T05:36:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.763+0000 7fc245a90b80 1 -- 172.21.15.131:0/804680652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca5961a00 con 0x557ca595a7a0 2024-05-10T05:36:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.764+0000 7fc226a00700 1 --2- 172.21.15.131:0/804680652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca595e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.766+0000 7fc226000700 1 -- 172.21.15.131:0/804680652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 msgr2=0x557ca595e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:37.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.766+0000 7fc226000700 1 --2- 172.21.15.131:0/804680652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca595e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.766+0000 7fc226000700 1 -- 172.21.15.131:0/804680652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 msgr2=0x557ca595ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.766+0000 7fc226a00700 1 --2- 172.21.15.131:0/804680652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca595e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:36:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.766+0000 7fc226000700 1 --2- 172.21.15.131:0/804680652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca595ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.767+0000 7fc226000700 1 -- 172.21.15.131:0/804680652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ca583f560 con 0x557ca595a7a0 2024-05-10T05:36:37.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.767+0000 7fc227400700 1 --2- 172.21.15.131:0/804680652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca595ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:37.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.768+0000 7fc226000700 1 --2- 172.21.15.131:0/804680652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca5973570 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc214099be0 tx=0x7fc214000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de6602777cc351a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:37.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.768+0000 7fc21cc00700 1 -- 172.21.15.131:0/804680652 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21409a350 con 0x557ca595a7a0 2024-05-10T05:36:37.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.768+0000 7fc21cc00700 1 -- 172.21.15.131:0/804680652 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc214007c20 con 0x557ca595a7a0 2024-05-10T05:36:37.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.768+0000 7fc21cc00700 1 -- 172.21.15.131:0/804680652 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214096360 con 0x557ca595a7a0 2024-05-10T05:36:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.770+0000 7fc245a90b80 1 -- 172.21.15.131:0/804680652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 msgr2=0x557ca5973570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.770+0000 7fc245a90b80 1 --2- 172.21.15.131:0/804680652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca5973570 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc214099be0 tx=0x7fc214000bc0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.770+0000 7fc245a90b80 1 -- 172.21.15.131:0/804680652 shutdown_connections 2024-05-10T05:36:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.771+0000 7fc245a90b80 1 --2- 172.21.15.131:0/804680652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595b0f0 0x557ca595ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.771+0000 7fc245a90b80 1 --2- 172.21.15.131:0/804680652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca595e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.771+0000 7fc245a90b80 1 --2- 172.21.15.131:0/804680652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595a7a0 0x557ca5973570 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.771+0000 7fc245a90b80 1 -- 172.21.15.131:0/804680652 >> 172.21.15.131:0/804680652 conn(0x557ca58bba60 msgr2=0x557ca5962e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:37.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.771+0000 7fc245a90b80 1 -- 172.21.15.131:0/804680652 shutdown_connections 2024-05-10T05:36:37.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.771+0000 7fc245a90b80 1 -- 172.21.15.131:0/804680652 wait complete. 2024-05-10T05:36:37.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.772+0000 7fc245a90b80 1 -- start start 2024-05-10T05:36:37.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.772+0000 7fc245a90b80 1 Processor -- start 2024-05-10T05:36:37.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.774+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca5970b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.774+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595a7a0 0x557ca5971060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.774+0000 7fc245a90b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595b0f0 0x557ca5971f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.774+0000 7fc245a90b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca5972480 con 0x557ca595a7a0 2024-05-10T05:36:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.774+0000 7fc245a90b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca59725f0 con 0x557ca5959530 2024-05-10T05:36:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.774+0000 7fc245a90b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca59727b0 con 0x557ca595b0f0 2024-05-10T05:36:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.774+0000 7fc245a90b80 10 client.0 _ll_register_callbacks cb 0x557ca581eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.775+0000 7fc226a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca5970b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.775+0000 7fc227400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595b0f0 0x557ca5971f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.775+0000 7fc226a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca5970b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34124/0 (socket says 172.21.15.131:34124) 2024-05-10T05:36:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.775+0000 7fc226a00700 1 -- 172.21.15.131:0/1112657478 learned_addr learned my addr 172.21.15.131:0/1112657478 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.775+0000 7fc226000700 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595a7a0 0x557ca5971060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.777+0000 7fc226000700 1 -- 172.21.15.131:0/1112657478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595b0f0 msgr2=0x557ca5971f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.777+0000 7fc226000700 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595b0f0 0x557ca5971f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.777+0000 7fc226000700 1 -- 172.21.15.131:0/1112657478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 msgr2=0x557ca5970b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.777+0000 7fc226000700 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca5970b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.777+0000 7fc226000700 1 -- 172.21.15.131:0/1112657478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ca56fa3b0 con 0x557ca595a7a0 2024-05-10T05:36:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.778+0000 7fc210c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:36:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.778+0000 7fc226000700 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595a7a0 0x557ca5971060 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc2140a1ba0 tx=0x7fc2140a2150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d3e4bfb54ec3da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.779+0000 7fc210c00700 1 -- 172.21.15.131:0/1112657478 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21407e070 con 0x557ca595a7a0 2024-05-10T05:36:37.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.779+0000 7fc210c00700 1 -- 172.21.15.131:0/1112657478 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc21409a350 con 0x557ca595a7a0 2024-05-10T05:36:37.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.779+0000 7fc245a90b80 1 -- 172.21.15.131:0/1112657478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc228068bd0 con 0x557ca595a7a0 2024-05-10T05:36:37.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.780+0000 7fc210c00700 1 -- 172.21.15.131:0/1112657478 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc214099c50 con 0x557ca595a7a0 2024-05-10T05:36:37.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.780+0000 7fc245a90b80 10 client.15903 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:37.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.780+0000 7fc245a90b80 1 -- 172.21.15.131:0/1112657478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fc220008240 con 0x557ca595a7a0 2024-05-10T05:36:37.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.781+0000 7fc210c00700 1 -- 172.21.15.131:0/1112657478 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc228068bd0 con 0x557ca595a7a0 2024-05-10T05:36:37.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.781+0000 7fc210c00700 1 -- 172.21.15.131:0/1112657478 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc2140a1d50 con 0x557ca595a7a0 2024-05-10T05:36:37.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.781+0000 7fc245a90b80 10 client.15903 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:37.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.781+0000 7fc245a90b80 10 client.15903 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:36:37.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.781+0000 7fc245a90b80 20 client.15903 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:37.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.781+0000 7fc245a90b80 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557ca59d6510 0x557ca59d8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:37.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.781+0000 7fc245a90b80 4 client.15903 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:36:37.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.781+0000 7fc245a90b80 1 -- 172.21.15.131:0/1112657478 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ca5871ec0 con 0x557ca59d6510 2024-05-10T05:36:37.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.784+0000 7fc226a00700 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557ca59d6510 0x557ca59d8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:37.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.792+0000 7fc226a00700 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557ca59d6510 0x557ca59d8930 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef34a75516dca512 server_cookie=61b0f089483d4f0d in_seq=0 out_seq=0 2024-05-10T05:36:37.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.792+0000 7fc210c00700 10 client.15903 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:36:37.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.792+0000 7fc210c00700 1 -- 172.21.15.131:0/1112657478 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557ca5871ec0 con 0x557ca59d6510 2024-05-10T05:36:37.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.792+0000 7fc210c00700 10 client.15903 handle_command_reply: tid=0 2024-05-10T05:36:37.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.792+0000 7fc245a90b80 1 client.15903 shutdown 2024-05-10T05:36:37.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.794+0000 7fc245a90b80 1 -- 172.21.15.131:0/1112657478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595a7a0 msgr2=0x557ca5971060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.794+0000 7fc245a90b80 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595a7a0 0x557ca5971060 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc2140a1ba0 tx=0x7fc2140a2150 comp rx=0 tx=0).stop 2024-05-10T05:36:37.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.795+0000 7fc245a90b80 1 -- 172.21.15.131:0/1112657478 shutdown_connections 2024-05-10T05:36:37.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.795+0000 7fc245a90b80 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x557ca59d6510 0x557ca59d8930 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.795+0000 7fc245a90b80 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca595b0f0 0x557ca5971f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.795+0000 7fc245a90b80 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca5959530 0x557ca5970b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.795+0000 7fc245a90b80 1 --2- 172.21.15.131:0/1112657478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca595a7a0 0x557ca5971060 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.795+0000 7fc245a90b80 1 -- 172.21.15.131:0/1112657478 >> 172.21.15.131:0/1112657478 conn(0x557ca58bba60 msgr2=0x557ca5962dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:37.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.795+0000 7fc210c00700 0 client.15903 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:36:37.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.795+0000 7fc245a90b80 1 -- 172.21.15.131:0/1112657478 shutdown_connections 2024-05-10T05:36:37.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.796+0000 7fc245a90b80 1 -- 172.21.15.131:0/1112657478 wait complete. 2024-05-10T05:36:37.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.796+0000 7fc245a90b80 20 client.15903 trim_cache size 0 max 16384 2024-05-10T05:36:37.797 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:36:37.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.798+0000 7fc22dc00700 1 -- 172.21.15.131:0/6104042 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc20c0a55b0 msgr2=0x7fc20c0a7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.798+0000 7fc22dc00700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc20c0a55b0 0x7fc20c0a7ac0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc22804fb30 tx=0x7fc228088000 comp rx=0 tx=0).stop 2024-05-10T05:36:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.798+0000 7fc22dc00700 1 -- 172.21.15.131:0/6104042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc220097720 msgr2=0x7fc22012cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.798+0000 7fc22dc00700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc220097720 0x7fc22012cf30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc21400c9e0 tx=0x7fc21400ccf0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.799+0000 7fc22dc00700 1 -- 172.21.15.131:0/6104042 shutdown_connections 2024-05-10T05:36:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.799+0000 7fc22dc00700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc20c0a55b0 0x7fc20c0a7ac0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.799+0000 7fc22dc00700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2200980c0 0x7fc220133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.799+0000 7fc22dc00700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc220096600 0x7fc22012c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.799+0000 7fc22dc00700 1 --2- 172.21.15.131:0/6104042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc220097720 0x7fc22012cf30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:37.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.799+0000 7fc22dc00700 1 -- 172.21.15.131:0/6104042 >> 172.21.15.131:0/6104042 conn(0x7fc220091c10 msgr2=0x7fc22009a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:37.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.800+0000 7fc22dc00700 1 -- 172.21.15.131:0/6104042 shutdown_connections 2024-05-10T05:36:37.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:37.800+0000 7fc22dc00700 1 -- 172.21.15.131:0/6104042 wait complete. 2024-05-10T05:36:37.863 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:36:38.111 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:38 smithi131 ceph-mon[37264]: pgmap v383: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 224 MiB/s wr, 71 op/s 2024-05-10T05:36:38.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:38 smithi142 ceph-mon[77667]: pgmap v383: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 224 MiB/s wr, 71 op/s 2024-05-10T05:36:38.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:38 smithi157 ceph-mon[78427]: pgmap v383: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 224 MiB/s wr, 71 op/s 2024-05-10T05:36:38.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.636+0000 7fe297800700 1 -- 172.21.15.131:0/4032654566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe290102d00 msgr2=0x7fe290103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:38.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.636+0000 7fe297800700 1 --2- 172.21.15.131:0/4032654566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe290102d00 0x7fe290103110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe2800099c0 tx=0x7fe280009cd0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.636+0000 7fe297800700 1 -- 172.21.15.131:0/4032654566 shutdown_connections 2024-05-10T05:36:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.636+0000 7fe297800700 1 --2- 172.21.15.131:0/4032654566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2901036a0 0x7fe290109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.636+0000 7fe297800700 1 --2- 172.21.15.131:0/4032654566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe290101be0 0x7fe290101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.636+0000 7fe297800700 1 --2- 172.21.15.131:0/4032654566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe290102d00 0x7fe290103110 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.636+0000 7fe297800700 1 -- 172.21.15.131:0/4032654566 >> 172.21.15.131:0/4032654566 conn(0x7fe2900fd1d0 msgr2=0x7fe2900ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.637+0000 7fe297800700 1 -- 172.21.15.131:0/4032654566 shutdown_connections 2024-05-10T05:36:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.637+0000 7fe297800700 1 -- 172.21.15.131:0/4032654566 wait complete. 2024-05-10T05:36:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.638+0000 7fe297800700 1 Processor -- start 2024-05-10T05:36:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.639+0000 7fe297800700 1 -- start start 2024-05-10T05:36:38.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.639+0000 7fe297800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe290101be0 0x7fe29019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe297800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe290102d00 0x7fe29019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe297800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2901036a0 0x7fe2901a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe28fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe290102d00 0x7fe29019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe28fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe290102d00 0x7fe29019ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34148/0 (socket says 172.21.15.131:34148) 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe294e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe290101be0 0x7fe29019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe294e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe290101be0 0x7fe29019e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35926/0 (socket says 172.21.15.131:35926) 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe28fe00700 1 -- 172.21.15.131:0/1334189933 learned_addr learned my addr 172.21.15.131:0/1334189933 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe295800700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2901036a0 0x7fe2901a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901a6030 con 0x7fe290101be0 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901a61a0 con 0x7fe290102d00 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.640+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901a6360 con 0x7fe2901036a0 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.641+0000 7fe28fe00700 1 -- 172.21.15.131:0/1334189933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2901036a0 msgr2=0x7fe2901a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.641+0000 7fe28fe00700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2901036a0 0x7fe2901a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.641+0000 7fe28fe00700 1 -- 172.21.15.131:0/1334189933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe290101be0 msgr2=0x7fe29019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:38.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.641+0000 7fe28fe00700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe290101be0 0x7fe29019e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.641+0000 7fe28fe00700 1 -- 172.21.15.131:0/1334189933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe288009c40 con 0x7fe290102d00 2024-05-10T05:36:38.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.641+0000 7fe295800700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2901036a0 0x7fe2901a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:36:38.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.641+0000 7fe294e00700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe290101be0 0x7fe29019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:38.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.642+0000 7fe28fe00700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe290102d00 0x7fe29019ea90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe280009990 tx=0x7fe28000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:38.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.642+0000 7fe28d600700 1 -- 172.21.15.131:0/1334189933 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28001d070 con 0x7fe290102d00 2024-05-10T05:36:38.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.642+0000 7fe28d600700 1 -- 172.21.15.131:0/1334189933 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2800054f0 con 0x7fe290102d00 2024-05-10T05:36:38.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.642+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2800095f0 con 0x7fe290102d00 2024-05-10T05:36:38.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.642+0000 7fe28d600700 1 -- 172.21.15.131:0/1334189933 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280004a00 con 0x7fe290102d00 2024-05-10T05:36:38.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.642+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2901a68d0 con 0x7fe290102d00 2024-05-10T05:36:38.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.647+0000 7fe28d600700 1 -- 172.21.15.131:0/1334189933 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe280004bc0 con 0x7fe290102d00 2024-05-10T05:36:38.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.650+0000 7fe2af678b80 1 Processor -- start 2024-05-10T05:36:38.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.650+0000 7fe2af678b80 1 -- start start 2024-05-10T05:36:38.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.650+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f46a0 0x5626472f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.651+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x5626472f5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.651+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626472fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.651+0000 7fe2af678b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264721b150 con 0x5626472f46a0 2024-05-10T05:36:38.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.651+0000 7fe2af678b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562647218b10 con 0x5626472f5570 2024-05-10T05:36:38.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.651+0000 7fe2af678b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562647214400 con 0x5626472f5ec0 2024-05-10T05:36:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.651+0000 7fe295800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f46a0 0x5626472f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.651+0000 7fe295800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f46a0 0x5626472f4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35930/0 (socket says 172.21.15.131:35930) 2024-05-10T05:36:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.651+0000 7fe295800700 1 -- 172.21.15.131:0/3767780585 learned_addr learned my addr 172.21.15.131:0/3767780585 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.651+0000 7fe28d600700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2780a12c0 0x7fe2780a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.652+0000 7fe295800700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2780a12c0 0x7fe2780a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.652+0000 7fe295800700 1 -- 172.21.15.131:0/3767780585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 msgr2=0x5626472fc7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.652+0000 7fe295800700 1 --2- 172.21.15.131:0/3767780585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626472fc7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.652+0000 7fe295800700 1 -- 172.21.15.131:0/3767780585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 msgr2=0x5626472f5980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:36:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.652+0000 7fe295800700 1 --2- 172.21.15.131:0/3767780585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x5626472f5980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.652+0000 7fe295800700 1 -- 172.21.15.131:0/3767780585 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626470953b0 con 0x5626472f46a0 2024-05-10T05:36:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.653+0000 7fe28d600700 1 -- 172.21.15.131:0/1334189933 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe2800cd790 con 0x7fe290102d00 2024-05-10T05:36:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.653+0000 7fe295800700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2780a12c0 0x7fe2780a37d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe288009550 tx=0x7fe28800e000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.653+0000 7fe295800700 1 --2- 172.21.15.131:0/3767780585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f46a0 0x5626472f4af0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe2880194a0 tx=0x7fe288007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cbcfffcc5b9111 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.657+0000 7fe27e000700 1 -- 172.21.15.131:0/3767780585 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800e070 con 0x5626472f46a0 2024-05-10T05:36:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.657+0000 7fe27e000700 1 -- 172.21.15.131:0/3767780585 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe28800f040 con 0x5626472f46a0 2024-05-10T05:36:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.657+0000 7fe27e000700 1 -- 172.21.15.131:0/3767780585 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288004280 con 0x5626472f46a0 2024-05-10T05:36:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.659+0000 7fe2af678b80 1 -- 172.21.15.131:0/3767780585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f46a0 msgr2=0x5626472f4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.659+0000 7fe2af678b80 1 --2- 172.21.15.131:0/3767780585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f46a0 0x5626472f4af0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe2880194a0 tx=0x7fe288007f90 comp rx=0 tx=0).stop 2024-05-10T05:36:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.659+0000 7fe2af678b80 1 -- 172.21.15.131:0/3767780585 shutdown_connections 2024-05-10T05:36:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.659+0000 7fe2af678b80 1 --2- 172.21.15.131:0/3767780585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f46a0 0x5626472f4af0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.659+0000 7fe2af678b80 1 --2- 172.21.15.131:0/3767780585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626472fc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.659+0000 7fe2af678b80 1 --2- 172.21.15.131:0/3767780585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x5626472f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.659+0000 7fe2af678b80 1 -- 172.21.15.131:0/3767780585 >> 172.21.15.131:0/3767780585 conn(0x562647256a60 msgr2=0x5626472eefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.663+0000 7fe2af678b80 1 -- 172.21.15.131:0/3767780585 shutdown_connections 2024-05-10T05:36:38.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.665+0000 7fe2af678b80 1 -- 172.21.15.131:0/3767780585 wait complete. 2024-05-10T05:36:38.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.666+0000 7fe2af678b80 1 -- start start 2024-05-10T05:36:38.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.667+0000 7fe2af678b80 1 Processor -- start 2024-05-10T05:36:38.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.668+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f46a0 0x5626472fe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.668+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f5570 0x5626472fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.668+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626473059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.669+0000 7fe2af678b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626471b0560 con 0x5626472f5570 2024-05-10T05:36:38.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.669+0000 7fe2af678b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626471b0200 con 0x5626472f46a0 2024-05-10T05:36:38.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.669+0000 7fe2af678b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626470976a0 con 0x5626472f5ec0 2024-05-10T05:36:38.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.669+0000 7fe294e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f46a0 0x5626472fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.669+0000 7fe294e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f46a0 0x5626472fe530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34166/0 (socket says 172.21.15.131:34166) 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.669+0000 7fe294e00700 1 -- 172.21.15.131:0/3339414268 learned_addr learned my addr 172.21.15.131:0/3339414268 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.669+0000 7fe2af678b80 10 client.0 _ll_register_callbacks cb 0x5626471b9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.669+0000 7fe295800700 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626473059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.670+0000 7fe28fe00700 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f5570 0x5626472fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.670+0000 7fe294e00700 1 -- 172.21.15.131:0/3339414268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 msgr2=0x5626473059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.670+0000 7fe294e00700 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626473059d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.670+0000 7fe294e00700 1 -- 172.21.15.131:0/3339414268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f5570 msgr2=0x5626472fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.670+0000 7fe294e00700 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f5570 0x5626472fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.670+0000 7fe294e00700 1 -- 172.21.15.131:0/3339414268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe288007c70 con 0x5626472f46a0 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.670+0000 7fe28fe00700 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f5570 0x5626472fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.670+0000 7fe272a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:36:38.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.670+0000 7fe294e00700 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f46a0 0x5626472fe530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe2840119e0 tx=0x7fe284011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821c2ae9061bf35b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:38.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.672+0000 7fe272a00700 1 -- 172.21.15.131:0/3339414268 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284004520 con 0x5626472f46a0 2024-05-10T05:36:38.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.672+0000 7fe272a00700 1 -- 172.21.15.131:0/3339414268 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe284004830 con 0x5626472f46a0 2024-05-10T05:36:38.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.674+0000 7fe2af678b80 1 -- 172.21.15.131:0/3339414268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56264716d270 con 0x5626472f46a0 2024-05-10T05:36:38.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.675+0000 7fe272a00700 1 -- 172.21.15.131:0/3339414268 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2840049f0 con 0x5626472f46a0 2024-05-10T05:36:38.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.675+0000 7fe2af678b80 10 client.25967 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:38.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.675+0000 7fe2af678b80 1 -- 172.21.15.131:0/3339414268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562647041b60 con 0x5626472f46a0 2024-05-10T05:36:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.677+0000 7fe272a00700 1 -- 172.21.15.131:0/3339414268 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56264716d270 con 0x5626472f46a0 2024-05-10T05:36:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.678+0000 7fe272a00700 1 -- 172.21.15.131:0/3339414268 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe284003bb0 con 0x5626472f46a0 2024-05-10T05:36:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.679+0000 7fe2af678b80 10 client.25967 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.679+0000 7fe2af678b80 10 client.25967 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:36:38.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.679+0000 7fe2af678b80 20 client.25967 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:38.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.679+0000 7fe2af678b80 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56264730a1e0 0x56264730c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.679+0000 7fe2af678b80 4 client.25967 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:38.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.679+0000 7fe2af678b80 1 -- 172.21.15.131:0/3339414268 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626472c3780 con 0x56264730a1e0 2024-05-10T05:36:38.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.679+0000 7fe28fe00700 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56264730a1e0 0x56264730c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.683+0000 7fe28fe00700 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56264730a1e0 0x56264730c630 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45148679f8c09ab9 server_cookie=eda87e9feb84402a in_seq=0 out_seq=0 2024-05-10T05:36:38.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.684+0000 7fe272a00700 10 client.25967 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:36:38.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.685+0000 7fe272a00700 1 -- 172.21.15.131:0/3339414268 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626472c3780 con 0x56264730a1e0 2024-05-10T05:36:38.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.685+0000 7fe272a00700 10 client.25967 handle_command_reply: tid=0 2024-05-10T05:36:38.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.686+0000 7fe2af678b80 1 client.25967 shutdown 2024-05-10T05:36:38.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.687+0000 7fe2af678b80 1 -- 172.21.15.131:0/3339414268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f46a0 msgr2=0x5626472fe530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:38.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.687+0000 7fe2af678b80 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f46a0 0x5626472fe530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe2840119e0 tx=0x7fe284011cf0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.688+0000 7fe2af678b80 1 -- 172.21.15.131:0/3339414268 shutdown_connections 2024-05-10T05:36:38.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.688+0000 7fe2af678b80 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56264730a1e0 0x56264730c630 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.688+0000 7fe2af678b80 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626473059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.688+0000 7fe2af678b80 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f46a0 0x5626472fe530 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe2840119e0 tx=0x7fe284011cf0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.688+0000 7fe2af678b80 1 --2- 172.21.15.131:0/3339414268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626472f5570 0x5626472fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.688+0000 7fe2af678b80 1 -- 172.21.15.131:0/3339414268 >> 172.21.15.131:0/3339414268 conn(0x562647256a60 msgr2=0x5626472eefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:38.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.688+0000 7fe272a00700 0 client.25967 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:36:38.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.690+0000 7fe2af678b80 1 -- 172.21.15.131:0/3339414268 shutdown_connections 2024-05-10T05:36:38.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.690+0000 7fe2af678b80 1 -- 172.21.15.131:0/3339414268 wait complete. 2024-05-10T05:36:38.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.691+0000 7fe2af678b80 20 client.25967 trim_cache size 0 max 16384 2024-05-10T05:36:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.711+0000 7fe2af678b80 1 Processor -- start 2024-05-10T05:36:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.712+0000 7fe2af678b80 1 -- start start 2024-05-10T05:36:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.712+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x56264730c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.712+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626472efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.712+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 0x562647305da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.712+0000 7fe2af678b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626473117d0 con 0x562647305990 2024-05-10T05:36:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.712+0000 7fe2af678b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626472f4e60 con 0x5626472f5570 2024-05-10T05:36:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.712+0000 7fe2af678b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626472fc5a0 con 0x5626472f5ec0 2024-05-10T05:36:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.713+0000 7fe295800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 0x562647305da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.713+0000 7fe294e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x56264730c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.713+0000 7fe294e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x56264730c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34172/0 (socket says 172.21.15.131:34172) 2024-05-10T05:36:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.713+0000 7fe294e00700 1 -- 172.21.15.131:0/1868286001 learned_addr learned my addr 172.21.15.131:0/1868286001 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.713+0000 7fe28fe00700 1 --2- 172.21.15.131:0/1868286001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626472efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.714+0000 7fe294e00700 1 -- 172.21.15.131:0/1868286001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 msgr2=0x5626472efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.714+0000 7fe294e00700 1 --2- 172.21.15.131:0/1868286001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626472efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.714+0000 7fe294e00700 1 -- 172.21.15.131:0/1868286001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 msgr2=0x562647305da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.714+0000 7fe294e00700 1 --2- 172.21.15.131:0/1868286001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 0x562647305da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.714+0000 7fe294e00700 1 -- 172.21.15.131:0/1868286001 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626471da560 con 0x5626472f5570 2024-05-10T05:36:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.715+0000 7fe295800700 1 --2- 172.21.15.131:0/1868286001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 0x562647305da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:36:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.715+0000 7fe294e00700 1 --2- 172.21.15.131:0/1868286001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x56264730c600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe284009230 tx=0x7fe2840235e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e970edc24458245 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.716+0000 7fe27ea00700 1 -- 172.21.15.131:0/1868286001 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284005590 con 0x5626472f5570 2024-05-10T05:36:38.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.716+0000 7fe27ea00700 1 -- 172.21.15.131:0/1868286001 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2840056f0 con 0x5626472f5570 2024-05-10T05:36:38.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.716+0000 7fe27ea00700 1 -- 172.21.15.131:0/1868286001 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284003d30 con 0x5626472f5570 2024-05-10T05:36:38.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.717+0000 7fe2af678b80 1 -- 172.21.15.131:0/1868286001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 msgr2=0x56264730c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:38.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.717+0000 7fe2af678b80 1 --2- 172.21.15.131:0/1868286001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x56264730c600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe284009230 tx=0x7fe2840235e0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.717+0000 7fe2af678b80 1 -- 172.21.15.131:0/1868286001 shutdown_connections 2024-05-10T05:36:38.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.717+0000 7fe2af678b80 1 --2- 172.21.15.131:0/1868286001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 0x562647305da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.717+0000 7fe2af678b80 1 --2- 172.21.15.131:0/1868286001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x56264730c600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.717+0000 7fe2af678b80 1 --2- 172.21.15.131:0/1868286001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x5626472efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.717+0000 7fe2af678b80 1 -- 172.21.15.131:0/1868286001 >> 172.21.15.131:0/1868286001 conn(0x562647256a60 msgr2=0x5626472f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.717+0000 7fe2af678b80 1 -- 172.21.15.131:0/1868286001 shutdown_connections 2024-05-10T05:36:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.718+0000 7fe2af678b80 1 -- 172.21.15.131:0/1868286001 wait complete. 2024-05-10T05:36:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.718+0000 7fe2af678b80 1 -- start start 2024-05-10T05:36:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.719+0000 7fe2af678b80 1 Processor -- start 2024-05-10T05:36:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.720+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x562647303030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.720+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x562647303570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.720+0000 7fe2af678b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 0x562647304450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.720+0000 7fe2af678b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562647304990 con 0x562647305990 2024-05-10T05:36:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.720+0000 7fe2af678b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562647304b00 con 0x5626472f5570 2024-05-10T05:36:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.720+0000 7fe2af678b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562647304cc0 con 0x5626472f5ec0 2024-05-10T05:36:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.720+0000 7fe2af678b80 10 client.0 _ll_register_callbacks cb 0x5626471b9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.721+0000 7fe28fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x562647303570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.721+0000 7fe294e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x562647303030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.721+0000 7fe294e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x562647303030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34178/0 (socket says 172.21.15.131:34178) 2024-05-10T05:36:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.721+0000 7fe294e00700 1 -- 172.21.15.131:0/390889163 learned_addr learned my addr 172.21.15.131:0/390889163 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.721+0000 7fe295800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 0x562647304450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.722+0000 7fe294e00700 1 -- 172.21.15.131:0/390889163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 msgr2=0x562647303570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.722+0000 7fe294e00700 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x562647303570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.722+0000 7fe294e00700 1 -- 172.21.15.131:0/390889163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 msgr2=0x562647304450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:38.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.722+0000 7fe294e00700 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 0x562647304450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.722+0000 7fe294e00700 1 -- 172.21.15.131:0/390889163 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562647041b60 con 0x5626472f5570 2024-05-10T05:36:38.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.722+0000 7fe28fe00700 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x562647303570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:38.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.722+0000 7fe294e00700 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x562647303030 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe284009410 tx=0x7fe284005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=948c7d4154b71d5a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:38.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.723+0000 7fe272a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:36:38.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.723+0000 7fe272a00700 1 -- 172.21.15.131:0/390889163 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284023b70 con 0x5626472f5570 2024-05-10T05:36:38.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.723+0000 7fe272a00700 1 -- 172.21.15.131:0/390889163 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe284023cd0 con 0x5626472f5570 2024-05-10T05:36:38.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.723+0000 7fe2af678b80 1 -- 172.21.15.131:0/390889163 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe28800f040 con 0x5626472f5570 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.724+0000 7fe272a00700 1 -- 172.21.15.131:0/390889163 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe28400f6a0 con 0x5626472f5570 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.724+0000 7fe2af678b80 10 client.25973 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.724+0000 7fe2af678b80 1 -- 172.21.15.131:0/390889163 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562647099cd0 con 0x5626472f5570 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.724+0000 7fe272a00700 1 -- 172.21.15.131:0/390889163 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe28800f040 con 0x5626472f5570 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.724+0000 7fe272a00700 1 -- 172.21.15.131:0/390889163 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe28402a030 con 0x5626472f5570 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.726+0000 7fe2af678b80 10 client.25973 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.726+0000 7fe2af678b80 10 client.25973 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.726+0000 7fe2af678b80 20 client.25973 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.726+0000 7fe2af678b80 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x562647371fa0 0x5626473743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.726+0000 7fe2af678b80 4 client.25973 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:36:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.726+0000 7fe2af678b80 1 -- 172.21.15.131:0/390889163 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56264720cec0 con 0x562647371fa0 2024-05-10T05:36:38.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.731+0000 7fe28fe00700 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x562647371fa0 0x5626473743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:38.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.733+0000 7fe28fe00700 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x562647371fa0 0x5626473743c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7b0ff41ebcbfcd0 server_cookie=9463f108093d2811 in_seq=0 out_seq=0 2024-05-10T05:36:38.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.734+0000 7fe272a00700 10 client.25973 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:36:38.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.734+0000 7fe272a00700 1 -- 172.21.15.131:0/390889163 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56264720cec0 con 0x562647371fa0 2024-05-10T05:36:38.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.734+0000 7fe272a00700 10 client.25973 handle_command_reply: tid=0 2024-05-10T05:36:38.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.735+0000 7fe2af678b80 1 client.25973 shutdown 2024-05-10T05:36:38.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.735+0000 7fe2af678b80 1 -- 172.21.15.131:0/390889163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 msgr2=0x562647303030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:38.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.735+0000 7fe2af678b80 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x562647303030 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe284009410 tx=0x7fe284005c20 comp rx=0 tx=0).stop 2024-05-10T05:36:38.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.736+0000 7fe2af678b80 1 -- 172.21.15.131:0/390889163 shutdown_connections 2024-05-10T05:36:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.736+0000 7fe2af678b80 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x562647371fa0 0x5626473743c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.736+0000 7fe2af678b80 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562647305990 0x562647304450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.736+0000 7fe2af678b80 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626472f5570 0x562647303030 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.736+0000 7fe2af678b80 1 --2- 172.21.15.131:0/390889163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626472f5ec0 0x562647303570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.736+0000 7fe2af678b80 1 -- 172.21.15.131:0/390889163 >> 172.21.15.131:0/390889163 conn(0x562647256a60 msgr2=0x5626472f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.736+0000 7fe272a00700 0 client.25973 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:36:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.736+0000 7fe2af678b80 1 -- 172.21.15.131:0/390889163 shutdown_connections 2024-05-10T05:36:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.737+0000 7fe2af678b80 1 -- 172.21.15.131:0/390889163 wait complete. 2024-05-10T05:36:38.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.739+0000 7fe2af678b80 20 client.25973 trim_cache size 0 max 16384 2024-05-10T05:36:38.743 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:36:38.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.746+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2780a12c0 msgr2=0x7fe2780a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:38.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.746+0000 7fe297800700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2780a12c0 0x7fe2780a37d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe288009550 tx=0x7fe28800e000 comp rx=0 tx=0).stop 2024-05-10T05:36:38.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.746+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe290102d00 msgr2=0x7fe29019ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:38.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.746+0000 7fe297800700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe290102d00 0x7fe29019ea90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe280009990 tx=0x7fe28000f690 comp rx=0 tx=0).stop 2024-05-10T05:36:38.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.751+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 shutdown_connections 2024-05-10T05:36:38.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.751+0000 7fe297800700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe2780a12c0 0x7fe2780a37d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.752+0000 7fe297800700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe290101be0 0x7fe29019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.752+0000 7fe297800700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2901036a0 0x7fe2901a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.752+0000 7fe297800700 1 --2- 172.21.15.131:0/1334189933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe290102d00 0x7fe29019ea90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:38.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.752+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 >> 172.21.15.131:0/1334189933 conn(0x7fe2900fd1d0 msgr2=0x7fe290103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:38.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.753+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 shutdown_connections 2024-05-10T05:36:38.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:38.753+0000 7fe297800700 1 -- 172.21.15.131:0/1334189933 wait complete. 2024-05-10T05:36:38.928 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:36:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.710+0000 7f65ff600700 1 -- 172.21.15.131:0/3274861737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f810efa0 msgr2=0x7f65f81114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.710+0000 7f65ff600700 1 --2- 172.21.15.131:0/3274861737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f810efa0 0x7f65f81114b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009990 tx=0x7f65f0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.712+0000 7f65ff600700 1 -- 172.21.15.131:0/3274861737 shutdown_connections 2024-05-10T05:36:39.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.712+0000 7f65ff600700 1 --2- 172.21.15.131:0/3274861737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65f8109bb0 0x7f65f810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.712+0000 7f65ff600700 1 --2- 172.21.15.131:0/3274861737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f810efa0 0x7f65f81114b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.712+0000 7f65ff600700 1 --2- 172.21.15.131:0/3274861737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65f810c510 0x7f65f810e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.712+0000 7f65ff600700 1 -- 172.21.15.131:0/3274861737 >> 172.21.15.131:0/3274861737 conn(0x7f65f806ea00 msgr2=0x7f65f8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:39.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.712+0000 7f65ff600700 1 -- 172.21.15.131:0/3274861737 shutdown_connections 2024-05-10T05:36:39.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.712+0000 7f65ff600700 1 -- 172.21.15.131:0/3274861737 wait complete. 2024-05-10T05:36:39.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.721+0000 7f65ff600700 1 Processor -- start 2024-05-10T05:36:39.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.722+0000 7f65ff600700 1 -- start start 2024-05-10T05:36:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.722+0000 7f65ff600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65f8109bb0 0x7f65f819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.723+0000 7f65ff600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f810c510 0x7f65f819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.723+0000 7f65fcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65f8109bb0 0x7f65f819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.723+0000 7f65fcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65f8109bb0 0x7f65f819dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35970/0 (socket says 172.21.15.131:35970) 2024-05-10T05:36:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.723+0000 7f65fcc00700 1 -- 172.21.15.131:0/2998830422 learned_addr learned my addr 172.21.15.131:0/2998830422 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.723+0000 7f65ff600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65f810efa0 0x7f65f819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.723+0000 7f65f7e00700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f810c510 0x7f65f819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.724+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f819b0b0 con 0x7f65f8109bb0 2024-05-10T05:36:39.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.724+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f819b1f0 con 0x7f65f810efa0 2024-05-10T05:36:39.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.724+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f819b3b0 con 0x7f65f810c510 2024-05-10T05:36:39.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.724+0000 7f65fd600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65f810efa0 0x7f65f819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.725+0000 7f65fd600700 1 -- 172.21.15.131:0/2998830422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f810c510 msgr2=0x7f65f819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:39.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.725+0000 7f65fd600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f810c510 0x7f65f819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.725+0000 7f65fd600700 1 -- 172.21.15.131:0/2998830422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65f8109bb0 msgr2=0x7f65f819dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:39.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.725+0000 7f65fd600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65f8109bb0 0x7f65f819dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.725+0000 7f65fd600700 1 -- 172.21.15.131:0/2998830422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f00095f0 con 0x7f65f810efa0 2024-05-10T05:36:39.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.725+0000 7f65fcc00700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65f8109bb0 0x7f65f819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:39.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.725+0000 7f65fd600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65f810efa0 0x7f65f819aa00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65f0006010 tx=0x7f65f000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:39.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.726+0000 7f65f5600700 1 -- 172.21.15.131:0/2998830422 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f001d070 con 0x7f65f810efa0 2024-05-10T05:36:39.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.726+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f81a65d0 con 0x7f65f810efa0 2024-05-10T05:36:39.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.726+0000 7f65f5600700 1 -- 172.21.15.131:0/2998830422 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65f000f780 con 0x7f65f810efa0 2024-05-10T05:36:39.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.726+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f81a6aa0 con 0x7f65f810efa0 2024-05-10T05:36:39.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.728+0000 7f65f5600700 1 -- 172.21.15.131:0/2998830422 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f001d070 con 0x7f65f810efa0 2024-05-10T05:36:39.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.737+0000 7f65f5600700 1 -- 172.21.15.131:0/2998830422 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f65f0004890 con 0x7f65f810efa0 2024-05-10T05:36:39.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.739+0000 7f6617576b80 1 Processor -- start 2024-05-10T05:36:39.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.740+0000 7f65f5600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f65e00a1300 0x7f65e00a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.740+0000 7f65f5600700 1 -- 172.21.15.131:0/2998830422 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f65f00cea80 con 0x7f65f810efa0 2024-05-10T05:36:39.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.741+0000 7f65f7e00700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f65e00a1300 0x7f65e00a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.741+0000 7f6617576b80 1 -- start start 2024-05-10T05:36:39.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.741+0000 7f6617576b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382a6a0 0x558b4382aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.742+0000 7f65f7e00700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f65e00a1300 0x7f65e00a3810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f65f819b8d0 tx=0x7f65e802b500 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:39.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.742+0000 7f6617576b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 0x558b4382b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.742+0000 7f6617576b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382bec0 0x558b438327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.742+0000 7f6617576b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b43751150 con 0x558b4382a6a0 2024-05-10T05:36:39.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.742+0000 7f6617576b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4374eb10 con 0x558b4382bec0 2024-05-10T05:36:39.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.742+0000 7f6617576b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4374a400 con 0x558b4382b570 2024-05-10T05:36:39.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.742+0000 7f65fcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382a6a0 0x558b4382aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.743+0000 7f65fcc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382a6a0 0x558b4382aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35984/0 (socket says 172.21.15.131:35984) 2024-05-10T05:36:39.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.743+0000 7f65fcc00700 1 -- 172.21.15.131:0/3518521962 learned_addr learned my addr 172.21.15.131:0/3518521962 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.744+0000 7f65f7e00700 1 --2- 172.21.15.131:0/3518521962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 0x558b4382b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.744+0000 7f65fcc00700 1 -- 172.21.15.131:0/3518521962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 msgr2=0x558b4382b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.744+0000 7f65fcc00700 1 --2- 172.21.15.131:0/3518521962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 0x558b4382b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.744+0000 7f65fcc00700 1 -- 172.21.15.131:0/3518521962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382bec0 msgr2=0x558b438327c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:36:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.744+0000 7f65fcc00700 1 --2- 172.21.15.131:0/3518521962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382bec0 0x558b438327c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.744+0000 7f65fcc00700 1 -- 172.21.15.131:0/3518521962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b435cb3b0 con 0x558b4382a6a0 2024-05-10T05:36:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.745+0000 7f65f7e00700 1 --2- 172.21.15.131:0/3518521962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 0x558b4382b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:39.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.745+0000 7f65fcc00700 1 --2- 172.21.15.131:0/3518521962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382a6a0 0x558b4382aaf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f65ec010610 tx=0x7f65ec00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf52b4094028c36 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:39.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.746+0000 7f65e5c00700 1 -- 172.21.15.131:0/3518521962 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec01c680 con 0x558b4382a6a0 2024-05-10T05:36:39.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.746+0000 7f65e5c00700 1 -- 172.21.15.131:0/3518521962 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65ec015070 con 0x558b4382a6a0 2024-05-10T05:36:39.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.746+0000 7f65e5c00700 1 -- 172.21.15.131:0/3518521962 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec011c20 con 0x558b4382a6a0 2024-05-10T05:36:39.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.748+0000 7f6617576b80 1 -- 172.21.15.131:0/3518521962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382a6a0 msgr2=0x558b4382aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:39.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.748+0000 7f6617576b80 1 --2- 172.21.15.131:0/3518521962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382a6a0 0x558b4382aaf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f65ec010610 tx=0x7f65ec00bf90 comp rx=0 tx=0).stop 2024-05-10T05:36:39.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.748+0000 7f6617576b80 1 -- 172.21.15.131:0/3518521962 shutdown_connections 2024-05-10T05:36:39.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.748+0000 7f6617576b80 1 --2- 172.21.15.131:0/3518521962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382a6a0 0x558b4382aaf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.748+0000 7f6617576b80 1 --2- 172.21.15.131:0/3518521962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382bec0 0x558b438327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.748+0000 7f6617576b80 1 --2- 172.21.15.131:0/3518521962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 0x558b4382b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.749+0000 7f6617576b80 1 -- 172.21.15.131:0/3518521962 >> 172.21.15.131:0/3518521962 conn(0x558b4378ca60 msgr2=0x558b43824fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.749+0000 7f6617576b80 1 -- 172.21.15.131:0/3518521962 shutdown_connections 2024-05-10T05:36:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.749+0000 7f6617576b80 1 -- 172.21.15.131:0/3518521962 wait complete. 2024-05-10T05:36:39.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.750+0000 7f6617576b80 1 -- start start 2024-05-10T05:36:39.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.751+0000 7f6617576b80 1 Processor -- start 2024-05-10T05:36:39.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.752+0000 7f6617576b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382a6a0 0x558b43834720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.752+0000 7f6617576b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382b570 0x558b43834c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.752+0000 7f6617576b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382bec0 0x558b4383bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.752+0000 7f6617576b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b436e6560 con 0x558b4382b570 2024-05-10T05:36:39.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.752+0000 7f6617576b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b436e6200 con 0x558b4382a6a0 2024-05-10T05:36:39.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.752+0000 7f6617576b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b435cd6a0 con 0x558b4382bec0 2024-05-10T05:36:39.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.752+0000 7f6617576b80 10 client.0 _ll_register_callbacks cb 0x558b436efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:39.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.753+0000 7f65fcc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382a6a0 0x558b43834720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.753+0000 7f65fd600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382bec0 0x558b4383bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.754+0000 7f65fd600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382bec0 0x558b4383bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32778/0 (socket says 172.21.15.131:32778) 2024-05-10T05:36:39.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.754+0000 7f65fd600700 1 -- 172.21.15.131:0/2450521670 learned_addr learned my addr 172.21.15.131:0/2450521670 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:39.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.754+0000 7f65f7e00700 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382b570 0x558b43834c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.755+0000 7f65fcc00700 1 -- 172.21.15.131:0/2450521670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382bec0 msgr2=0x558b4383bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.755+0000 7f65fcc00700 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382bec0 0x558b4383bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.755+0000 7f65fcc00700 1 -- 172.21.15.131:0/2450521670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382b570 msgr2=0x558b43834c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.755+0000 7f65fcc00700 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382b570 0x558b43834c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.755+0000 7f65fcc00700 1 -- 172.21.15.131:0/2450521670 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec00be00 con 0x558b4382a6a0 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.756+0000 7f65f7e00700 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382b570 0x558b43834c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.756+0000 7f65fd600700 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382bec0 0x558b4383bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.756+0000 7f65da000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.756+0000 7f65fcc00700 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382a6a0 0x558b43834720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65ec010cb0 tx=0x7f65ec012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7529a3ea2d250fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.757+0000 7f65da000700 1 -- 172.21.15.131:0/2450521670 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec020070 con 0x558b4382a6a0 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.757+0000 7f6617576b80 1 -- 172.21.15.131:0/2450521670 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b436a3270 con 0x558b4382a6a0 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.757+0000 7f65da000700 1 -- 172.21.15.131:0/2450521670 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f65ec017070 con 0x558b4382a6a0 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.758+0000 7f65da000700 1 -- 172.21.15.131:0/2450521670 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65ec020070 con 0x558b4382a6a0 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.758+0000 7f6617576b80 10 client.25979 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:39.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.758+0000 7f6617576b80 1 -- 172.21.15.131:0/2450521670 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558b43577b60 con 0x558b4382a6a0 2024-05-10T05:36:39.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.759+0000 7f65da000700 1 -- 172.21.15.131:0/2450521670 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b436a3270 con 0x558b4382a6a0 2024-05-10T05:36:39.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.759+0000 7f65da000700 1 -- 172.21.15.131:0/2450521670 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f65ec004750 con 0x558b4382a6a0 2024-05-10T05:36:39.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.759+0000 7f6617576b80 10 client.25979 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:39.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.759+0000 7f6617576b80 10 client.25979 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:36:39.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.759+0000 7f6617576b80 20 client.25979 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:39.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.759+0000 7f6617576b80 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558b438402e0 0x558b43842730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.759+0000 7f6617576b80 4 client.25979 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:39.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.759+0000 7f6617576b80 1 -- 172.21.15.131:0/2450521670 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b437f9780 con 0x558b438402e0 2024-05-10T05:36:39.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.761+0000 7f65f7e00700 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558b438402e0 0x558b43842730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.763+0000 7f65da000700 10 client.25979 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:36:39.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.763+0000 7f65f7e00700 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558b438402e0 0x558b43842730 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c57538500fbe43a server_cookie=e3b637989122d74b in_seq=0 out_seq=0 2024-05-10T05:36:39.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.764+0000 7f65da000700 1 -- 172.21.15.131:0/2450521670 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b437f9780 con 0x558b438402e0 2024-05-10T05:36:39.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.764+0000 7f65da000700 10 client.25979 handle_command_reply: tid=0 2024-05-10T05:36:39.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.764+0000 7f6617576b80 1 client.25979 shutdown 2024-05-10T05:36:39.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.765+0000 7f6617576b80 1 -- 172.21.15.131:0/2450521670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382a6a0 msgr2=0x558b43834720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:39.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.765+0000 7f6617576b80 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382a6a0 0x558b43834720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65ec010cb0 tx=0x7f65ec012be0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.765+0000 7f6617576b80 1 -- 172.21.15.131:0/2450521670 shutdown_connections 2024-05-10T05:36:39.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.765+0000 7f6617576b80 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558b438402e0 0x558b43842730 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.765+0000 7f6617576b80 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382bec0 0x558b4383bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.765+0000 7f6617576b80 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382a6a0 0x558b43834720 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65ec010cb0 tx=0x7f65ec012be0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.765+0000 7f6617576b80 1 --2- 172.21.15.131:0/2450521670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382b570 0x558b43834c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.765+0000 7f6617576b80 1 -- 172.21.15.131:0/2450521670 >> 172.21.15.131:0/2450521670 conn(0x558b4378ca60 msgr2=0x558b43824fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:39.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.765+0000 7f65da000700 0 client.25979 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:36:39.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.766+0000 7f6617576b80 1 -- 172.21.15.131:0/2450521670 shutdown_connections 2024-05-10T05:36:39.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.766+0000 7f6617576b80 1 -- 172.21.15.131:0/2450521670 wait complete. 2024-05-10T05:36:39.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.767+0000 7f6617576b80 20 client.25979 trim_cache size 0 max 16384 2024-05-10T05:36:39.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.796+0000 7f6617576b80 1 Processor -- start 2024-05-10T05:36:39.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.797+0000 7f6617576b80 1 -- start start 2024-05-10T05:36:39.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.798+0000 7f6617576b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382b570 0x558b43824e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.799+0000 7f6617576b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382bec0 0x558b43839a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.800+0000 7f6617576b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b43839f40 0x558b4382f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.800+0000 7f6617576b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b43832700 con 0x558b4382bec0 2024-05-10T05:36:39.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.800+0000 7f6617576b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b43835770 con 0x558b4382b570 2024-05-10T05:36:39.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.800+0000 7f6617576b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4382bbd0 con 0x558b43839f40 2024-05-10T05:36:39.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.800+0000 7f65fcc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382b570 0x558b43824e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.800+0000 7f65fcc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382b570 0x558b43824e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34216/0 (socket says 172.21.15.131:34216) 2024-05-10T05:36:39.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.800+0000 7f65fcc00700 1 -- 172.21.15.131:0/1602567590 learned_addr learned my addr 172.21.15.131:0/1602567590 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:39.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.801+0000 7f65fd600700 1 --2- 172.21.15.131:0/1602567590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b43839f40 0x558b4382f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.801+0000 7f65f7e00700 1 --2- 172.21.15.131:0/1602567590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382bec0 0x558b43839a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.802+0000 7f65fcc00700 1 -- 172.21.15.131:0/1602567590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b43839f40 msgr2=0x558b4382f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:39.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.802+0000 7f65fcc00700 1 --2- 172.21.15.131:0/1602567590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b43839f40 0x558b4382f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.802+0000 7f65fcc00700 1 -- 172.21.15.131:0/1602567590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382bec0 msgr2=0x558b43839a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:39.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.802+0000 7f65fcc00700 1 --2- 172.21.15.131:0/1602567590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382bec0 0x558b43839a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.802+0000 7f65fcc00700 1 -- 172.21.15.131:0/1602567590 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b43710560 con 0x558b4382b570 2024-05-10T05:36:39.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.804+0000 7f65fcc00700 1 --2- 172.21.15.131:0/1602567590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382b570 0x558b43824e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65ec011590 tx=0x7f65ec00b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5024e21e0a600f7a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:39.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.804+0000 7f65e6600700 1 -- 172.21.15.131:0/1602567590 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec020070 con 0x558b4382b570 2024-05-10T05:36:39.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.804+0000 7f65e6600700 1 -- 172.21.15.131:0/1602567590 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65ec025c60 con 0x558b4382b570 2024-05-10T05:36:39.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.804+0000 7f65e6600700 1 -- 172.21.15.131:0/1602567590 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec031ea0 con 0x558b4382b570 2024-05-10T05:36:39.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.806+0000 7f6617576b80 1 -- 172.21.15.131:0/1602567590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382b570 msgr2=0x558b43824e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:39.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.806+0000 7f6617576b80 1 --2- 172.21.15.131:0/1602567590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382b570 0x558b43824e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65ec011590 tx=0x7f65ec00b5b0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.806+0000 7f6617576b80 1 -- 172.21.15.131:0/1602567590 shutdown_connections 2024-05-10T05:36:39.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.806+0000 7f6617576b80 1 --2- 172.21.15.131:0/1602567590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b43839f40 0x558b4382f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.806+0000 7f6617576b80 1 --2- 172.21.15.131:0/1602567590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b4382b570 0x558b43824e50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.806+0000 7f6617576b80 1 --2- 172.21.15.131:0/1602567590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382bec0 0x558b43839a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.806+0000 7f6617576b80 1 -- 172.21.15.131:0/1602567590 >> 172.21.15.131:0/1602567590 conn(0x558b4378ca60 msgr2=0x558b4382e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:39.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.807+0000 7f6617576b80 1 -- 172.21.15.131:0/1602567590 shutdown_connections 2024-05-10T05:36:39.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.807+0000 7f6617576b80 1 -- 172.21.15.131:0/1602567590 wait complete. 2024-05-10T05:36:39.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.808+0000 7f6617576b80 1 -- start start 2024-05-10T05:36:39.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.808+0000 7f6617576b80 1 Processor -- start 2024-05-10T05:36:39.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.812+0000 7f6617576b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 0x558b43838e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.812+0000 7f6617576b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382bec0 0x558b438393a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.813+0000 7f6617576b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b43839f40 0x558b43894a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.813+0000 7f6617576b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b43894fd0 con 0x558b4382bec0 2024-05-10T05:36:39.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.813+0000 7f6617576b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b43895110 con 0x558b43839f40 2024-05-10T05:36:39.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.813+0000 7f6617576b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b438952d0 con 0x558b4382b570 2024-05-10T05:36:39.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.813+0000 7f6617576b80 10 client.0 _ll_register_callbacks cb 0x558b436efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:39.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.813+0000 7f65fd600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b43839f40 0x558b43894a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.813+0000 7f65fd600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b43839f40 0x558b43894a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34232/0 (socket says 172.21.15.131:34232) 2024-05-10T05:36:39.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.814+0000 7f65fd600700 1 -- 172.21.15.131:0/2033787826 learned_addr learned my addr 172.21.15.131:0/2033787826 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:39.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.814+0000 7f65fcc00700 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 0x558b43838e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.814+0000 7f65f7e00700 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382bec0 0x558b438393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.816+0000 7f65fd600700 1 -- 172.21.15.131:0/2033787826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 msgr2=0x558b43838e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:39.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.816+0000 7f65fd600700 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 0x558b43838e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.816+0000 7f65fd600700 1 -- 172.21.15.131:0/2033787826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382bec0 msgr2=0x558b438393a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:39.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.816+0000 7f65fd600700 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382bec0 0x558b438393a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.816+0000 7f65fd600700 1 -- 172.21.15.131:0/2033787826 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b43577b60 con 0x558b43839f40 2024-05-10T05:36:39.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.819+0000 7f65fd600700 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b43839f40 0x558b43894a90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f65f000c010 tx=0x7f65f009b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7479e0a989336d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:39.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.819+0000 7f65da000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:36:39.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.821+0000 7f65da000700 1 -- 172.21.15.131:0/2033787826 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0004d40 con 0x558b43839f40 2024-05-10T05:36:39.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.821+0000 7f65da000700 1 -- 172.21.15.131:0/2033787826 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f65f009c600 con 0x558b43839f40 2024-05-10T05:36:39.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.821+0000 7f6617576b80 1 -- 172.21.15.131:0/2033787826 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65ec015070 con 0x558b43839f40 2024-05-10T05:36:39.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.822+0000 7f65da000700 1 -- 172.21.15.131:0/2033787826 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65f0004ea0 con 0x558b43839f40 2024-05-10T05:36:39.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.824+0000 7f6617576b80 10 client.25985 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:39.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.824+0000 7f6617576b80 1 -- 172.21.15.131:0/2033787826 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558b435cfcd0 con 0x558b43839f40 2024-05-10T05:36:39.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.826+0000 7f65da000700 1 -- 172.21.15.131:0/2033787826 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65f0065080 con 0x558b43839f40 2024-05-10T05:36:39.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.829+0000 7f65da000700 1 -- 172.21.15.131:0/2033787826 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f65f0096b10 con 0x558b43839f40 2024-05-10T05:36:39.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.829+0000 7f6617576b80 10 client.25985 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:39.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.829+0000 7f6617576b80 10 client.25985 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:36:39.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.829+0000 7f6617576b80 20 client.25985 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:39.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.829+0000 7f6617576b80 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558b43896910 0x558b43898d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:39.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.829+0000 7f6617576b80 4 client.25985 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:36:39.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.829+0000 7f6617576b80 1 -- 172.21.15.131:0/2033787826 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558b43742ec0 con 0x558b43896910 2024-05-10T05:36:39.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.831+0000 7f65fcc00700 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558b43896910 0x558b43898d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:39.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.836+0000 7f65da000700 10 client.25985 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:36:39.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.836+0000 7f65fcc00700 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558b43896910 0x558b43898d30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=989fd7a86991f84e server_cookie=a0d4b8a357fb7a47 in_seq=0 out_seq=0 2024-05-10T05:36:39.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.837+0000 7f65da000700 1 -- 172.21.15.131:0/2033787826 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558b43742ec0 con 0x558b43896910 2024-05-10T05:36:39.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.837+0000 7f65da000700 10 client.25985 handle_command_reply: tid=0 2024-05-10T05:36:39.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.837+0000 7f6617576b80 1 client.25985 shutdown 2024-05-10T05:36:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.852+0000 7f6617576b80 1 -- 172.21.15.131:0/2033787826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b43839f40 msgr2=0x558b43894a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.852+0000 7f6617576b80 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b43839f40 0x558b43894a90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f65f000c010 tx=0x7f65f009b6d0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.853+0000 7f6617576b80 1 -- 172.21.15.131:0/2033787826 shutdown_connections 2024-05-10T05:36:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.853+0000 7f6617576b80 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x558b43896910 0x558b43898d30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.853+0000 7f6617576b80 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b43839f40 0x558b43894a90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.853+0000 7f6617576b80 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558b4382b570 0x558b43838e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.853+0000 7f6617576b80 1 --2- 172.21.15.131:0/2033787826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558b4382bec0 0x558b438393a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.853+0000 7f6617576b80 1 -- 172.21.15.131:0/2033787826 >> 172.21.15.131:0/2033787826 conn(0x558b4378ca60 msgr2=0x558b4382f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.854+0000 7f65da000700 0 client.25985 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:36:39.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.877+0000 7f6617576b80 1 -- 172.21.15.131:0/2033787826 shutdown_connections 2024-05-10T05:36:39.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.877+0000 7f6617576b80 1 -- 172.21.15.131:0/2033787826 wait complete. 2024-05-10T05:36:39.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.882+0000 7f6617576b80 20 client.25985 trim_cache size 0 max 16384 2024-05-10T05:36:39.883 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:36:39.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.889+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f65e00a1300 msgr2=0x7f65e00a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:39.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.889+0000 7f65ff600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f65e00a1300 0x7f65e00a3810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f65f819b8d0 tx=0x7f65e802b500 comp rx=0 tx=0).stop 2024-05-10T05:36:39.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.889+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65f810efa0 msgr2=0x7f65f819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:39.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.889+0000 7f65ff600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65f810efa0 0x7f65f819aa00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65f0006010 tx=0x7f65f000f690 comp rx=0 tx=0).stop 2024-05-10T05:36:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.892+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 shutdown_connections 2024-05-10T05:36:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.892+0000 7f65ff600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f65e00a1300 0x7f65e00a3810 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.892+0000 7f65ff600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65f8109bb0 0x7f65f819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.892+0000 7f65ff600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65f810efa0 0x7f65f819aa00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.892+0000 7f65ff600700 1 --2- 172.21.15.131:0/2998830422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f810c510 0x7f65f819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:39.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.892+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 >> 172.21.15.131:0/2998830422 conn(0x7f65f806ea00 msgr2=0x7f65f8108aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:39.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.893+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 shutdown_connections 2024-05-10T05:36:39.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:39.893+0000 7f65ff600700 1 -- 172.21.15.131:0/2998830422 wait complete. 2024-05-10T05:36:40.049 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:36:40.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:40 smithi131 ceph-mon[37264]: pgmap v384: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:36:40.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:40 smithi142 ceph-mon[77667]: pgmap v384: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:36:40.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:40 smithi157 ceph-mon[78427]: pgmap v384: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:36:40.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.746+0000 7febda000700 1 -- 172.21.15.131:0/24363029 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8003bc0 con 0x7febdc109300 2024-05-10T05:36:40.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.747+0000 7febe2000700 1 -- 172.21.15.131:0/24363029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc109300 msgr2=0x7febdc109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:40.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.747+0000 7febe2000700 1 --2- 172.21.15.131:0/24363029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc109300 0x7febdc109710 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7febc801a9a0 tx=0x7febc801acb0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.747+0000 7febe2000700 1 -- 172.21.15.131:0/24363029 shutdown_connections 2024-05-10T05:36:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.747+0000 7febe2000700 1 --2- 172.21.15.131:0/24363029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febdc109c50 0x7febdc10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.747+0000 7febe2000700 1 --2- 172.21.15.131:0/24363029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febdc108630 0x7febdc108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.747+0000 7febe2000700 1 --2- 172.21.15.131:0/24363029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc109300 0x7febdc109710 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.747+0000 7febe2000700 1 -- 172.21.15.131:0/24363029 >> 172.21.15.131:0/24363029 conn(0x7febdc06ea00 msgr2=0x7febdc070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.748+0000 7febe2000700 1 -- 172.21.15.131:0/24363029 shutdown_connections 2024-05-10T05:36:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.748+0000 7febe2000700 1 -- 172.21.15.131:0/24363029 wait complete. 2024-05-10T05:36:40.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.751+0000 7febe2000700 1 Processor -- start 2024-05-10T05:36:40.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.753+0000 7febe2000700 1 -- start start 2024-05-10T05:36:40.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.755+0000 7febe2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febdc108630 0x7febdc19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.756+0000 7febe2000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc109300 0x7febdc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.756+0000 7febe2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febdc109c50 0x7febdc1a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.756+0000 7febe2000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1a5de0 con 0x7febdc109300 2024-05-10T05:36:40.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.756+0000 7febe2000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1a5f50 con 0x7febdc108630 2024-05-10T05:36:40.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.756+0000 7febe2000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1a6110 con 0x7febdc109c50 2024-05-10T05:36:40.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.756+0000 7febdb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febdc108630 0x7febdc19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.756+0000 7febdb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febdc108630 0x7febdc19e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34242/0 (socket says 172.21.15.131:34242) 2024-05-10T05:36:40.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.756+0000 7febdb400700 1 -- 172.21.15.131:0/3386032040 learned_addr learned my addr 172.21.15.131:0/3386032040 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:40.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.756+0000 7febdaa00700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc109300 0x7febdc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.757+0000 7febdbe00700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febdc109c50 0x7febdc1a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.758+0000 7febdb400700 1 -- 172.21.15.131:0/3386032040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febdc109c50 msgr2=0x7febdc1a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:40.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.758+0000 7febdb400700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febdc109c50 0x7febdc1a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.758+0000 7febdb400700 1 -- 172.21.15.131:0/3386032040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc109300 msgr2=0x7febdc19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:40.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.758+0000 7febdb400700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc109300 0x7febdc19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.758+0000 7febdb400700 1 -- 172.21.15.131:0/3386032040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc801a600 con 0x7febdc108630 2024-05-10T05:36:40.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.758+0000 7febdb400700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febdc108630 0x7febdc19e300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7febd400ada0 tx=0x7febd400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:40.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.759+0000 7febd3e00700 1 -- 172.21.15.131:0/3386032040 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd400f070 con 0x7febdc108630 2024-05-10T05:36:40.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.759+0000 7febd3e00700 1 -- 172.21.15.131:0/3386032040 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd400cd90 con 0x7febdc108630 2024-05-10T05:36:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.759+0000 7febd3e00700 1 -- 172.21.15.131:0/3386032040 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4005280 con 0x7febdc108630 2024-05-10T05:36:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.759+0000 7febe2000700 1 -- 172.21.15.131:0/3386032040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc1a63f0 con 0x7febdc108630 2024-05-10T05:36:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.759+0000 7febe2000700 1 -- 172.21.15.131:0/3386032040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc1a68b0 con 0x7febdc108630 2024-05-10T05:36:40.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.771+0000 7febd3e00700 1 -- 172.21.15.131:0/3386032040 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7febd400c8d0 con 0x7febdc108630 2024-05-10T05:36:40.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.771+0000 7febf9fbcb80 1 Processor -- start 2024-05-10T05:36:40.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.772+0000 7febf9fbcb80 1 -- start start 2024-05-10T05:36:40.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.772+0000 7febf9fbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df837f6a0 0x555df837faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.773+0000 7febf9fbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380570 0x555df8380980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.775+0000 7febd3e00700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7febc00a1310 0x7febc00a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.775+0000 7febd3e00700 1 -- 172.21.15.131:0/3386032040 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7febd40cbdc0 con 0x7febdc108630 2024-05-10T05:36:40.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.776+0000 7febdbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df837f6a0 0x555df837faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.776+0000 7febdaa00700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7febc00a1310 0x7febc00a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.776+0000 7febdbe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df837f6a0 0x555df837faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36044/0 (socket says 172.21.15.131:36044) 2024-05-10T05:36:40.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.777+0000 7febdbe00700 1 -- 172.21.15.131:0/3313774930 learned_addr learned my addr 172.21.15.131:0/3313774930 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:40.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.777+0000 7febdb400700 1 --2- 172.21.15.131:0/3313774930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380570 0x555df8380980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.777+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/3313774930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df83877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.777+0000 7febdaa00700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7febc00a1310 0x7febc00a3820 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7febc801af50 tx=0x7febc802a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:40.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.777+0000 7febf9fbcb80 1 -- 172.21.15.131:0/3313774930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df82a6150 con 0x555df837f6a0 2024-05-10T05:36:40.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.777+0000 7febf9fbcb80 1 -- 172.21.15.131:0/3313774930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df82a3b10 con 0x555df8380ec0 2024-05-10T05:36:40.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.777+0000 7febf9fbcb80 1 -- 172.21.15.131:0/3313774930 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df829f400 con 0x555df8380570 2024-05-10T05:36:40.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.778+0000 7febdb400700 1 -- 172.21.15.131:0/3313774930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 msgr2=0x555df83877c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:40.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.778+0000 7febdb400700 1 --2- 172.21.15.131:0/3313774930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df83877c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.778+0000 7febdb400700 1 -- 172.21.15.131:0/3313774930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df837f6a0 msgr2=0x555df837faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:40.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.778+0000 7febdb400700 1 --2- 172.21.15.131:0/3313774930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df837f6a0 0x555df837faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.778+0000 7febdb400700 1 -- 172.21.15.131:0/3313774930 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df81203b0 con 0x555df8380570 2024-05-10T05:36:40.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.778+0000 7febdbe00700 1 --2- 172.21.15.131:0/3313774930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df837f6a0 0x555df837faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:40.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.779+0000 7febdb400700 1 --2- 172.21.15.131:0/3313774930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380570 0x555df8380980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7febd4007760 tx=0x7febd40076d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3309ea78d3da69a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:40.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.781+0000 7febc7e00700 1 -- 172.21.15.131:0/3313774930 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd409d070 con 0x555df8380570 2024-05-10T05:36:40.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.781+0000 7febc7e00700 1 -- 172.21.15.131:0/3313774930 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd40a1630 con 0x555df8380570 2024-05-10T05:36:40.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.781+0000 7febc7e00700 1 -- 172.21.15.131:0/3313774930 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4095950 con 0x555df8380570 2024-05-10T05:36:40.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.782+0000 7febf9fbcb80 1 -- 172.21.15.131:0/3313774930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380570 msgr2=0x555df8380980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:40.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.782+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/3313774930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380570 0x555df8380980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7febd4007760 tx=0x7febd40076d0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.782+0000 7febf9fbcb80 1 -- 172.21.15.131:0/3313774930 shutdown_connections 2024-05-10T05:36:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.782+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/3313774930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df837f6a0 0x555df837faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.782+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/3313774930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df83877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.782+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/3313774930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380570 0x555df8380980 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.782+0000 7febf9fbcb80 1 -- 172.21.15.131:0/3313774930 >> 172.21.15.131:0/3313774930 conn(0x555df82e1a60 msgr2=0x555df8379fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.782+0000 7febf9fbcb80 1 -- 172.21.15.131:0/3313774930 shutdown_connections 2024-05-10T05:36:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.782+0000 7febf9fbcb80 1 -- 172.21.15.131:0/3313774930 wait complete. 2024-05-10T05:36:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.783+0000 7febf9fbcb80 1 -- start start 2024-05-10T05:36:40.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.784+0000 7febf9fbcb80 1 Processor -- start 2024-05-10T05:36:40.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.785+0000 7febf9fbcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df837f6a0 0x555df83897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.785+0000 7febf9fbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 0x555df8389d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.786+0000 7febdaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 0x555df8389d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.787+0000 7febdaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 0x555df8389d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36048/0 (socket says 172.21.15.131:36048) 2024-05-10T05:36:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.787+0000 7febdb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df837f6a0 0x555df83897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.788+0000 7febdb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df837f6a0 0x555df83897d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34246/0 (socket says 172.21.15.131:34246) 2024-05-10T05:36:40.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.788+0000 7febdaa00700 1 -- 172.21.15.131:0/2905148394 learned_addr learned my addr 172.21.15.131:0/2905148394 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:40.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.789+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380ec0 0x555df8390c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.789+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df823b560 con 0x555df8380570 2024-05-10T05:36:40.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.790+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df823b200 con 0x555df837f6a0 2024-05-10T05:36:40.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.790+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df81226a0 con 0x555df8380ec0 2024-05-10T05:36:40.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.790+0000 7febdbe00700 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380ec0 0x555df8390c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.795+0000 7febf9fbcb80 10 client.0 _ll_register_callbacks cb 0x555df8244b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:40.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.795+0000 7febdb400700 1 -- 172.21.15.131:0/2905148394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380ec0 msgr2=0x555df8390c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:40.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.795+0000 7febdb400700 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380ec0 0x555df8390c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.795+0000 7febdb400700 1 -- 172.21.15.131:0/2905148394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 msgr2=0x555df8389d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:40.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.795+0000 7febdb400700 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 0x555df8389d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.796+0000 7febdb400700 1 -- 172.21.15.131:0/2905148394 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc009620 con 0x555df837f6a0 2024-05-10T05:36:40.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.796+0000 7febdaa00700 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 0x555df8389d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:40.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.796+0000 7febdbe00700 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380ec0 0x555df8390c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:36:40.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.796+0000 7febdb400700 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df837f6a0 0x555df83897d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7febd4007760 tx=0x7febd40c9800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1adcec4ed0c9190b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:40.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.796+0000 7febc4c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:36:40.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.796+0000 7febc4c00700 1 -- 172.21.15.131:0/2905148394 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd40b1070 con 0x555df837f6a0 2024-05-10T05:36:40.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.797+0000 7febc4c00700 1 -- 172.21.15.131:0/2905148394 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7febd409d070 con 0x555df837f6a0 2024-05-10T05:36:40.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.797+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555df81f8270 con 0x555df837f6a0 2024-05-10T05:36:40.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.802+0000 7febc4c00700 1 -- 172.21.15.131:0/2905148394 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7febd40994a0 con 0x555df837f6a0 2024-05-10T05:36:40.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.803+0000 7febf9fbcb80 10 client.25991 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:40.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.803+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7febdc1a68b0 con 0x555df837f6a0 2024-05-10T05:36:40.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.803+0000 7febc4c00700 1 -- 172.21.15.131:0/2905148394 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555df81f8270 con 0x555df837f6a0 2024-05-10T05:36:40.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.804+0000 7febc4c00700 1 -- 172.21.15.131:0/2905148394 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7febd40c9d20 con 0x555df837f6a0 2024-05-10T05:36:40.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.804+0000 7febf9fbcb80 10 client.25991 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:40.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.804+0000 7febf9fbcb80 10 client.25991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:40.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.804+0000 7febf9fbcb80 20 client.25991 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:40.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.804+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555df8394f10 0x555df8397330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.804+0000 7febf9fbcb80 4 client.25991 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:40.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.804+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555df834e780 con 0x555df8394f10 2024-05-10T05:36:40.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.805+0000 7febdaa00700 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555df8394f10 0x555df8397330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.805+0000 7febdaa00700 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555df8394f10 0x555df8397330 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b066276543ce031 server_cookie=4b01929e7c7cd0da in_seq=0 out_seq=0 2024-05-10T05:36:40.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.805+0000 7febc4c00700 10 client.25991 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:40.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.806+0000 7febc4c00700 1 -- 172.21.15.131:0/2905148394 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555df834e780 con 0x555df8394f10 2024-05-10T05:36:40.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.806+0000 7febc4c00700 10 client.25991 handle_command_reply: tid=0 2024-05-10T05:36:40.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.806+0000 7febf9fbcb80 1 client.25991 shutdown 2024-05-10T05:36:40.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.810+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df837f6a0 msgr2=0x555df83897d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:40.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.810+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df837f6a0 0x555df83897d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7febd4007760 tx=0x7febd40c9800 comp rx=0 tx=0).stop 2024-05-10T05:36:40.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.811+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 shutdown_connections 2024-05-10T05:36:40.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.811+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555df8394f10 0x555df8397330 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.811+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380ec0 0x555df8390c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.811+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df837f6a0 0x555df83897d0 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7febd4007760 tx=0x7febd40c9800 comp rx=0 tx=0).stop 2024-05-10T05:36:40.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.811+0000 7febc4c00700 0 client.25991 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:40.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.811+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2905148394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 0x555df8389d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.811+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 >> 172.21.15.131:0/2905148394 conn(0x555df82e1a60 msgr2=0x555df8379fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:40.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.814+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 shutdown_connections 2024-05-10T05:36:40.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.815+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2905148394 wait complete. 2024-05-10T05:36:40.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.815+0000 7febf9fbcb80 20 client.25991 trim_cache size 0 max 16384 2024-05-10T05:36:40.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.910+0000 7febf9fbcb80 1 Processor -- start 2024-05-10T05:36:40.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.910+0000 7febf9fbcb80 1 -- start start 2024-05-10T05:36:40.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.910+0000 7febf9fbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380570 0x555df8391280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.910+0000 7febf9fbcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df8390c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.910+0000 7febf9fbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df83844b0 0x555df83848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.910+0000 7febf9fbcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8384e00 con 0x555df83844b0 2024-05-10T05:36:40.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.910+0000 7febf9fbcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8397580 con 0x555df8380ec0 2024-05-10T05:36:40.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.910+0000 7febf9fbcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df837ad20 con 0x555df8380570 2024-05-10T05:36:40.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.911+0000 7febdaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df8390c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.911+0000 7febdaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df8390c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34262/0 (socket says 172.21.15.131:34262) 2024-05-10T05:36:40.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.911+0000 7febdaa00700 1 -- 172.21.15.131:0/2015786009 learned_addr learned my addr 172.21.15.131:0/2015786009 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:40.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.912+0000 7febdbe00700 1 --2- 172.21.15.131:0/2015786009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df83844b0 0x555df83848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.912+0000 7febdaa00700 1 -- 172.21.15.131:0/2015786009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380570 msgr2=0x555df8391280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:36:40.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.912+0000 7febdaa00700 1 --2- 172.21.15.131:0/2015786009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380570 0x555df8391280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.912+0000 7febdaa00700 1 -- 172.21.15.131:0/2015786009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df83844b0 msgr2=0x555df83848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:40.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.912+0000 7febdaa00700 1 --2- 172.21.15.131:0/2015786009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df83844b0 0x555df83848c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.912+0000 7febdaa00700 1 -- 172.21.15.131:0/2015786009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df80ccb60 con 0x555df8380ec0 2024-05-10T05:36:40.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.914+0000 7febdaa00700 1 --2- 172.21.15.131:0/2015786009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df8390c30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febc801a270 tx=0x7febc8030000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61f7163fc1d09996 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:40.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.914+0000 7febd1200700 1 -- 172.21.15.131:0/2015786009 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80054b0 con 0x555df8380ec0 2024-05-10T05:36:40.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.914+0000 7febd1200700 1 -- 172.21.15.131:0/2015786009 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555df834e780 con 0x555df8380ec0 2024-05-10T05:36:40.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.914+0000 7febd1200700 1 -- 172.21.15.131:0/2015786009 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80432d0 con 0x555df8380ec0 2024-05-10T05:36:40.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.916+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2015786009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 msgr2=0x555df8390c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:40.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.916+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2015786009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df8390c30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febc801a270 tx=0x7febc8030000 comp rx=0 tx=0).stop 2024-05-10T05:36:40.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.920+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2015786009 shutdown_connections 2024-05-10T05:36:40.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.920+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2015786009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df83844b0 0x555df83848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.920+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2015786009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df8380570 0x555df8391280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.920+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/2015786009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df8390c30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.920+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2015786009 >> 172.21.15.131:0/2015786009 conn(0x555df82e1a60 msgr2=0x555df8379fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:40.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.921+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2015786009 shutdown_connections 2024-05-10T05:36:40.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.923+0000 7febf9fbcb80 1 -- 172.21.15.131:0/2015786009 wait complete. 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.924+0000 7febf9fbcb80 1 -- start start 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.924+0000 7febf9fbcb80 1 Processor -- start 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.926+0000 7febf9fbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 0x555df838ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.926+0000 7febf9fbcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df838f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.926+0000 7febf9fbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df83844b0 0x555df8390020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.926+0000 7febf9fbcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8390560 con 0x555df8380570 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.926+0000 7febf9fbcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df83906d0 con 0x555df8380ec0 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.926+0000 7febf9fbcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8390890 con 0x555df83844b0 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.926+0000 7febf9fbcb80 10 client.0 _ll_register_callbacks cb 0x555df8244b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.927+0000 7febdbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df83844b0 0x555df8390020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.927+0000 7febdbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df83844b0 0x555df8390020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32868/0 (socket says 172.21.15.131:32868) 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.927+0000 7febdbe00700 1 -- 172.21.15.131:0/490078615 learned_addr learned my addr 172.21.15.131:0/490078615 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:40.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.927+0000 7febdaa00700 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df838f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.927+0000 7febdbe00700 1 -- 172.21.15.131:0/490078615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 msgr2=0x555df838f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.927+0000 7febdbe00700 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df838f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.927+0000 7febdbe00700 1 -- 172.21.15.131:0/490078615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 msgr2=0x555df838ebb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.927+0000 7febdbe00700 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 0x555df838ebb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.928+0000 7febdbe00700 1 -- 172.21.15.131:0/490078615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc10efc0 con 0x555df83844b0 2024-05-10T05:36:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.929+0000 7febdbe00700 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df83844b0 0x555df8390020 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7febcc0157b0 tx=0x7febcc015890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d3984f761afd1c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:40.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.929+0000 7febc4c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:36:40.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.929+0000 7febc4c00700 1 -- 172.21.15.131:0/490078615 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc016640 con 0x555df83844b0 2024-05-10T05:36:40.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.929+0000 7febc4c00700 1 -- 172.21.15.131:0/490078615 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7febcc0167a0 con 0x555df83844b0 2024-05-10T05:36:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.934+0000 7febf9fbcb80 1 -- 172.21.15.131:0/490078615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febd40a1630 con 0x555df83844b0 2024-05-10T05:36:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.934+0000 7febc4c00700 1 -- 172.21.15.131:0/490078615 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7febcc016960 con 0x555df83844b0 2024-05-10T05:36:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.935+0000 7febf9fbcb80 10 client.26035 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:40.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.935+0000 7febf9fbcb80 1 -- 172.21.15.131:0/490078615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555df8265560 con 0x555df83844b0 2024-05-10T05:36:40.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.937+0000 7febc4c00700 1 -- 172.21.15.131:0/490078615 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd40a1630 con 0x555df83844b0 2024-05-10T05:36:40.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.937+0000 7febc4c00700 1 -- 172.21.15.131:0/490078615 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7febcc013460 con 0x555df83844b0 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.937+0000 7febf9fbcb80 10 client.26035 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.937+0000 7febf9fbcb80 10 client.26035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.937+0000 7febf9fbcb80 20 client.26035 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.937+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555df83ec4f0 0x555df83ee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.937+0000 7febf9fbcb80 4 client.26035 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.937+0000 7febf9fbcb80 1 -- 172.21.15.131:0/490078615 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555df8297ec0 con 0x555df83ec4f0 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.938+0000 7febdb400700 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555df83ec4f0 0x555df83ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.938+0000 7febdb400700 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555df83ec4f0 0x555df83ee910 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=852a76bb390a5731 server_cookie=9c416f8927000612 in_seq=0 out_seq=0 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.939+0000 7febc4c00700 10 client.26035 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.939+0000 7febc4c00700 1 -- 172.21.15.131:0/490078615 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555df8297ec0 con 0x555df83ec4f0 2024-05-10T05:36:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.939+0000 7febc4c00700 10 client.26035 handle_command_reply: tid=0 2024-05-10T05:36:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.939+0000 7febf9fbcb80 1 client.26035 shutdown 2024-05-10T05:36:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.940+0000 7febf9fbcb80 1 -- 172.21.15.131:0/490078615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df83844b0 msgr2=0x555df8390020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.940+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df83844b0 0x555df8390020 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7febcc0157b0 tx=0x7febcc015890 comp rx=0 tx=0).stop 2024-05-10T05:36:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.941+0000 7febf9fbcb80 1 -- 172.21.15.131:0/490078615 shutdown_connections 2024-05-10T05:36:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.941+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555df83ec4f0 0x555df83ee910 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.941+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555df8380570 0x555df838ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.941+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555df83844b0 0x555df8390020 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.941+0000 7febf9fbcb80 1 --2- 172.21.15.131:0/490078615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555df8380ec0 0x555df838f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.941+0000 7febf9fbcb80 1 -- 172.21.15.131:0/490078615 >> 172.21.15.131:0/490078615 conn(0x555df82e1a60 msgr2=0x555df837e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.942+0000 7febc4c00700 0 client.26035 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.942+0000 7febf9fbcb80 1 -- 172.21.15.131:0/490078615 shutdown_connections 2024-05-10T05:36:40.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.942+0000 7febf9fbcb80 1 -- 172.21.15.131:0/490078615 wait complete. 2024-05-10T05:36:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.947+0000 7febf9fbcb80 20 client.26035 trim_cache size 0 max 16384 2024-05-10T05:36:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:36:40.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.950+0000 7febe2000700 1 -- 172.21.15.131:0/3386032040 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7febc00a1310 msgr2=0x7febc00a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.950+0000 7febe2000700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7febc00a1310 0x7febc00a3820 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7febc801af50 tx=0x7febc802a040 comp rx=0 tx=0).stop 2024-05-10T05:36:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.950+0000 7febe2000700 1 -- 172.21.15.131:0/3386032040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febdc108630 msgr2=0x7febdc19e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.950+0000 7febe2000700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febdc108630 0x7febdc19e300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7febd400ada0 tx=0x7febd400c330 comp rx=0 tx=0).stop 2024-05-10T05:36:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.954+0000 7febe2000700 1 -- 172.21.15.131:0/3386032040 shutdown_connections 2024-05-10T05:36:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.954+0000 7febe2000700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7febc00a1310 0x7febc00a3820 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.954+0000 7febe2000700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febdc109c50 0x7febdc1a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.954+0000 7febe2000700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febdc108630 0x7febdc19e300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.954+0000 7febe2000700 1 --2- 172.21.15.131:0/3386032040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febdc109300 0x7febdc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.954+0000 7febe2000700 1 -- 172.21.15.131:0/3386032040 >> 172.21.15.131:0/3386032040 conn(0x7febdc06ea00 msgr2=0x7febdc10c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.956+0000 7febe2000700 1 -- 172.21.15.131:0/3386032040 shutdown_connections 2024-05-10T05:36:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:40.956+0000 7febe2000700 1 -- 172.21.15.131:0/3386032040 wait complete. 2024-05-10T05:36:42.046 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:36:42.046 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:36:42.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:42 smithi142 ceph-mon[77667]: pgmap v385: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 70 op/s 2024-05-10T05:36:42.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:42 smithi131 ceph-mon[37264]: pgmap v385: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 70 op/s 2024-05-10T05:36:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:42 smithi157 ceph-mon[78427]: pgmap v385: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 70 op/s 2024-05-10T05:36:42.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.729+0000 7fbb4f600700 1 -- 172.21.15.131:0/1962758088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb48101be0 msgr2=0x7fbb48101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:42.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.729+0000 7fbb4f600700 1 --2- 172.21.15.131:0/1962758088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb48101be0 0x7fbb48101ff0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbb340099c0 tx=0x7fbb34009cd0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.730+0000 7fbb4f600700 1 -- 172.21.15.131:0/1962758088 shutdown_connections 2024-05-10T05:36:42.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.730+0000 7fbb4f600700 1 --2- 172.21.15.131:0/1962758088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb48100ac0 0x7fbb48100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.730+0000 7fbb4f600700 1 --2- 172.21.15.131:0/1962758088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb48102580 0x7fbb48108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.730+0000 7fbb4f600700 1 --2- 172.21.15.131:0/1962758088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb48101be0 0x7fbb48101ff0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.730+0000 7fbb4f600700 1 -- 172.21.15.131:0/1962758088 >> 172.21.15.131:0/1962758088 conn(0x7fbb480fc0d0 msgr2=0x7fbb480fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:42.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.730+0000 7fbb4f600700 1 -- 172.21.15.131:0/1962758088 shutdown_connections 2024-05-10T05:36:42.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.730+0000 7fbb4f600700 1 -- 172.21.15.131:0/1962758088 wait complete. 2024-05-10T05:36:42.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.732+0000 7fbb4f600700 1 Processor -- start 2024-05-10T05:36:42.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.732+0000 7fbb4f600700 1 -- start start 2024-05-10T05:36:42.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.732+0000 7fbb4f600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb48100ac0 0x7fbb48197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.732+0000 7fbb4f600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb48101be0 0x7fbb48198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.732+0000 7fbb4f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb48102580 0x7fbb4819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.732+0000 7fbb4f600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4819fa00 con 0x7fbb48101be0 2024-05-10T05:36:42.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.732+0000 7fbb4f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4819fb70 con 0x7fbb48102580 2024-05-10T05:36:42.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.732+0000 7fbb4f600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4819fd30 con 0x7fbb48100ac0 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb4cc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb48100ac0 0x7fbb48197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb4cc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb48100ac0 0x7fbb48197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32882/0 (socket says 172.21.15.131:32882) 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb4cc00700 1 -- 172.21.15.131:0/2019741553 learned_addr learned my addr 172.21.15.131:0/2019741553 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb4d600700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb48102580 0x7fbb4819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb47e00700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb48101be0 0x7fbb48198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb4cc00700 1 -- 172.21.15.131:0/2019741553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb48102580 msgr2=0x7fbb4819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb48102580 0x7fbb4819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb4cc00700 1 -- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb48101be0 msgr2=0x7fbb48198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb48101be0 0x7fbb48198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb4cc00700 1 -- 172.21.15.131:0/2019741553 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340095f0 con 0x7fbb48100ac0 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.733+0000 7fbb47e00700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb48101be0 0x7fbb48198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.734+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb48100ac0 0x7fbb48197f20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00d950 tx=0x7fbb3c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:42.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.734+0000 7fbb45600700 1 -- 172.21.15.131:0/2019741553 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c00b950 con 0x7fbb48100ac0 2024-05-10T05:36:42.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.734+0000 7fbb45600700 1 -- 172.21.15.131:0/2019741553 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb3c007620 con 0x7fbb48100ac0 2024-05-10T05:36:42.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.734+0000 7fbb45600700 1 -- 172.21.15.131:0/2019741553 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c00f3c0 con 0x7fbb48100ac0 2024-05-10T05:36:42.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.735+0000 7fbb4f600700 1 -- 172.21.15.131:0/2019741553 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4819ffb0 con 0x7fbb48100ac0 2024-05-10T05:36:42.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.736+0000 7fbb4f600700 1 -- 172.21.15.131:0/2019741553 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb481a0380 con 0x7fbb48100ac0 2024-05-10T05:36:42.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.739+0000 7fbb45600700 1 -- 172.21.15.131:0/2019741553 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbb3c0105d0 con 0x7fbb48100ac0 2024-05-10T05:36:42.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.742+0000 7fbb45600700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb300a1240 0x7fbb300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.742+0000 7fbb45600700 1 -- 172.21.15.131:0/2019741553 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbb3c0cc8e0 con 0x7fbb48100ac0 2024-05-10T05:36:42.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.743+0000 7fbb47e00700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb300a1240 0x7fbb300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.743+0000 7fbb67602b80 1 Processor -- start 2024-05-10T05:36:42.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.743+0000 7fbb67602b80 1 -- start start 2024-05-10T05:36:42.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.743+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 0x55c053c8db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.743+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8e5f0 0x55c053c8ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 0x55c053c95840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb67602b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053bb41d0 con 0x55c053c8d720 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb67602b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053bb1b90 con 0x55c053c8ef40 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb67602b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053bad480 con 0x55c053c8e5f0 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb47e00700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb300a1240 0x7fbb300a3750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb34006010 tx=0x7fbb34009510 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb47e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8e5f0 0x55c053c8ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb4cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 0x55c053c8db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb47e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8e5f0 0x55c053c8ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32888/0 (socket says 172.21.15.131:32888) 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb47e00700 1 -- 172.21.15.131:0/3163302314 learned_addr learned my addr 172.21.15.131:0/3163302314 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/3163302314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 0x55c053c8db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36094/0 (socket says 172.21.15.131:36094) 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.744+0000 7fbb4d600700 1 --2- 172.21.15.131:0/3163302314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 0x55c053c95840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.745+0000 7fbb47e00700 1 -- 172.21.15.131:0/3163302314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 msgr2=0x55c053c95840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.745+0000 7fbb47e00700 1 --2- 172.21.15.131:0/3163302314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 0x55c053c95840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.745+0000 7fbb47e00700 1 -- 172.21.15.131:0/3163302314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 msgr2=0x55c053c8db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:42.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.745+0000 7fbb47e00700 1 --2- 172.21.15.131:0/3163302314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 0x55c053c8db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.745+0000 7fbb47e00700 1 -- 172.21.15.131:0/3163302314 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c053a2e430 con 0x55c053c8e5f0 2024-05-10T05:36:42.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.745+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/3163302314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 0x55c053c8db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:36:42.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.745+0000 7fbb47e00700 1 --2- 172.21.15.131:0/3163302314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8e5f0 0x55c053c8ea00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbb340099c0 tx=0x7fbb3401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28fbc2fd2d8e8178 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.745+0000 7fbb39c00700 1 -- 172.21.15.131:0/3163302314 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3402c070 con 0x55c053c8e5f0 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.745+0000 7fbb39c00700 1 -- 172.21.15.131:0/3163302314 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb34004ca0 con 0x55c053c8e5f0 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.745+0000 7fbb39c00700 1 -- 172.21.15.131:0/3163302314 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3401c070 con 0x55c053c8e5f0 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.746+0000 7fbb67602b80 1 -- 172.21.15.131:0/3163302314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8e5f0 msgr2=0x55c053c8ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.746+0000 7fbb67602b80 1 --2- 172.21.15.131:0/3163302314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8e5f0 0x55c053c8ea00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbb340099c0 tx=0x7fbb3401c000 comp rx=0 tx=0).stop 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.747+0000 7fbb67602b80 1 -- 172.21.15.131:0/3163302314 shutdown_connections 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.747+0000 7fbb67602b80 1 --2- 172.21.15.131:0/3163302314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 0x55c053c8db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.747+0000 7fbb67602b80 1 --2- 172.21.15.131:0/3163302314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 0x55c053c95840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.747+0000 7fbb67602b80 1 --2- 172.21.15.131:0/3163302314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8e5f0 0x55c053c8ea00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.747+0000 7fbb67602b80 1 -- 172.21.15.131:0/3163302314 >> 172.21.15.131:0/3163302314 conn(0x55c053befae0 msgr2=0x55c053c88060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:42.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.747+0000 7fbb67602b80 1 -- 172.21.15.131:0/3163302314 shutdown_connections 2024-05-10T05:36:42.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.747+0000 7fbb67602b80 1 -- 172.21.15.131:0/3163302314 wait complete. 2024-05-10T05:36:42.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.748+0000 7fbb67602b80 1 -- start start 2024-05-10T05:36:42.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.748+0000 7fbb67602b80 1 Processor -- start 2024-05-10T05:36:42.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.749+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 0x55c053c97690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.749+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8e5f0 0x55c053c97bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.749+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8ef40 0x55c053c9eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.750+0000 7fbb67602b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053b495e0 con 0x55c053c8d720 2024-05-10T05:36:42.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.750+0000 7fbb67602b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053b49280 con 0x55c053c8e5f0 2024-05-10T05:36:42.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.750+0000 7fbb67602b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053a30720 con 0x55c053c8ef40 2024-05-10T05:36:42.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.750+0000 7fbb47e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8e5f0 0x55c053c97bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.750+0000 7fbb4d600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8ef40 0x55c053c9eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.750+0000 7fbb47e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8e5f0 0x55c053c97bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34298/0 (socket says 172.21.15.131:34298) 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.750+0000 7fbb47e00700 1 -- 172.21.15.131:0/3514640972 learned_addr learned my addr 172.21.15.131:0/3514640972 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.750+0000 7fbb67602b80 10 client.0 _ll_register_callbacks cb 0x55c053b52bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.750+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 0x55c053c97690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.751+0000 7fbb47e00700 1 -- 172.21.15.131:0/3514640972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8ef40 msgr2=0x55c053c9eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.751+0000 7fbb47e00700 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8ef40 0x55c053c9eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.751+0000 7fbb47e00700 1 -- 172.21.15.131:0/3514640972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 msgr2=0x55c053c97690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.751+0000 7fbb47e00700 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 0x55c053c97690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.751+0000 7fbb47e00700 1 -- 172.21.15.131:0/3514640972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb48068580 con 0x55c053c8e5f0 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.751+0000 7fbb4d600700 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8ef40 0x55c053c9eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.751+0000 7fbb47e00700 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8e5f0 0x55c053c97bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb34005850 tx=0x7fbb34004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba9e6eb504a6253 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:42.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.751+0000 7fbb2e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:36:42.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.751+0000 7fbb2e000700 1 -- 172.21.15.131:0/3514640972 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3402c070 con 0x55c053c8e5f0 2024-05-10T05:36:42.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.751+0000 7fbb2e000700 1 -- 172.21.15.131:0/3514640972 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbb34025b00 con 0x55c053c8e5f0 2024-05-10T05:36:42.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb67602b80 1 -- 172.21.15.131:0/3514640972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c053b062f0 con 0x55c053c8e5f0 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb2e000700 1 -- 172.21.15.131:0/3514640972 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb34025cc0 con 0x55c053c8e5f0 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb67602b80 10 client.26009 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb67602b80 1 -- 172.21.15.131:0/3514640972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c0539dabe0 con 0x55c053c8e5f0 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb2e000700 1 -- 172.21.15.131:0/3514640972 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c053b062f0 con 0x55c053c8e5f0 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb2e000700 1 -- 172.21.15.131:0/3514640972 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbb34038660 con 0x55c053c8e5f0 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb67602b80 10 client.26009 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb67602b80 10 client.26009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb67602b80 20 client.26009 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb67602b80 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c053ca2df0 0x55c053ca5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb67602b80 4 client.26009 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.752+0000 7fbb67602b80 1 -- 172.21.15.131:0/3514640972 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c053c5c800 con 0x55c053ca2df0 2024-05-10T05:36:42.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.753+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c053ca2df0 0x55c053ca5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.753+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c053ca2df0 0x55c053ca5240 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb5ea9ff1a5dafbe server_cookie=7e4696ba1f8acb2f in_seq=0 out_seq=0 2024-05-10T05:36:42.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.753+0000 7fbb2e000700 10 client.26009 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:42.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.754+0000 7fbb2e000700 1 -- 172.21.15.131:0/3514640972 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c053c5c800 con 0x55c053ca2df0 2024-05-10T05:36:42.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.754+0000 7fbb2e000700 10 client.26009 handle_command_reply: tid=0 2024-05-10T05:36:42.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.754+0000 7fbb67602b80 1 client.26009 shutdown 2024-05-10T05:36:42.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb67602b80 1 -- 172.21.15.131:0/3514640972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8e5f0 msgr2=0x55c053c97bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:42.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb67602b80 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8e5f0 0x55c053c97bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb34005850 tx=0x7fbb34004ec0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb67602b80 1 -- 172.21.15.131:0/3514640972 shutdown_connections 2024-05-10T05:36:42.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb67602b80 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c053ca2df0 0x55c053ca5240 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb67602b80 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8d720 0x55c053c97690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb67602b80 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8ef40 0x55c053c9eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb67602b80 1 --2- 172.21.15.131:0/3514640972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8e5f0 0x55c053c97bd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb2e000700 0 client.26009 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:42.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb67602b80 1 -- 172.21.15.131:0/3514640972 >> 172.21.15.131:0/3514640972 conn(0x55c053befae0 msgr2=0x55c053c88030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:42.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb67602b80 1 -- 172.21.15.131:0/3514640972 shutdown_connections 2024-05-10T05:36:42.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.755+0000 7fbb67602b80 1 -- 172.21.15.131:0/3514640972 wait complete. 2024-05-10T05:36:42.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.756+0000 7fbb67602b80 20 client.26009 trim_cache size 0 max 16384 2024-05-10T05:36:42.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.774+0000 7fbb67602b80 1 Processor -- start 2024-05-10T05:36:42.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.774+0000 7fbb67602b80 1 -- start start 2024-05-10T05:36:42.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.775+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8d720 0x55c053c923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.775+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8ef40 0x55c053c92910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.775+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c97da0 0x55c053c9e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.775+0000 7fbb67602b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053c897f0 con 0x55c053c8ef40 2024-05-10T05:36:42.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.775+0000 7fbb67602b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053c98290 con 0x55c053c8d720 2024-05-10T05:36:42.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.775+0000 7fbb67602b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053c956f0 con 0x55c053c97da0 2024-05-10T05:36:42.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8d720 0x55c053c923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8d720 0x55c053c923d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34312/0 (socket says 172.21.15.131:34312) 2024-05-10T05:36:42.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4cc00700 1 -- 172.21.15.131:0/1480303160 learned_addr learned my addr 172.21.15.131:0/1480303160 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:42.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4d600700 1 --2- 172.21.15.131:0/1480303160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c97da0 0x55c053c9e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4cc00700 1 -- 172.21.15.131:0/1480303160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c97da0 msgr2=0x55c053c9e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:42.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/1480303160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c97da0 0x55c053c9e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4cc00700 1 -- 172.21.15.131:0/1480303160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8ef40 msgr2=0x55c053c92910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:42.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/1480303160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8ef40 0x55c053c92910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4cc00700 1 -- 172.21.15.131:0/1480303160 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c053b735e0 con 0x55c053c8d720 2024-05-10T05:36:42.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4d600700 1 --2- 172.21.15.131:0/1480303160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c97da0 0x55c053c9e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:42.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/1480303160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8d720 0x55c053c923d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0ca380 tx=0x7fbb3c0109d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d78b987cc0bf02 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:42.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.776+0000 7fbb3a600700 1 -- 172.21.15.131:0/1480303160 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0a2150 con 0x55c053c8d720 2024-05-10T05:36:42.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.777+0000 7fbb3a600700 1 -- 172.21.15.131:0/1480303160 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb3c097e10 con 0x55c053c8d720 2024-05-10T05:36:42.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.777+0000 7fbb3a600700 1 -- 172.21.15.131:0/1480303160 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c099420 con 0x55c053c8d720 2024-05-10T05:36:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.777+0000 7fbb67602b80 1 -- 172.21.15.131:0/1480303160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8d720 msgr2=0x55c053c923d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.777+0000 7fbb67602b80 1 --2- 172.21.15.131:0/1480303160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8d720 0x55c053c923d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0ca380 tx=0x7fbb3c0109d0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.778+0000 7fbb67602b80 1 -- 172.21.15.131:0/1480303160 shutdown_connections 2024-05-10T05:36:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.778+0000 7fbb67602b80 1 --2- 172.21.15.131:0/1480303160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c97da0 0x55c053c9e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.778+0000 7fbb67602b80 1 --2- 172.21.15.131:0/1480303160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8d720 0x55c053c923d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.778+0000 7fbb67602b80 1 --2- 172.21.15.131:0/1480303160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c8ef40 0x55c053c92910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.778+0000 7fbb67602b80 1 -- 172.21.15.131:0/1480303160 >> 172.21.15.131:0/1480303160 conn(0x55c053befae0 msgr2=0x55c053c97690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:42.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.778+0000 7fbb67602b80 1 -- 172.21.15.131:0/1480303160 shutdown_connections 2024-05-10T05:36:42.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.778+0000 7fbb67602b80 1 -- 172.21.15.131:0/1480303160 wait complete. 2024-05-10T05:36:42.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.778+0000 7fbb67602b80 1 -- start start 2024-05-10T05:36:42.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.779+0000 7fbb67602b80 1 Processor -- start 2024-05-10T05:36:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.780+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8d720 0x55c053c9ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.780+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 0x55c053c9d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.780+0000 7fbb67602b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c97da0 0x55c053c9d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.780+0000 7fbb67602b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053c9dd50 con 0x55c053c97da0 2024-05-10T05:36:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.780+0000 7fbb67602b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053c9de90 con 0x55c053c8ef40 2024-05-10T05:36:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.780+0000 7fbb67602b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c053c9e050 con 0x55c053c8d720 2024-05-10T05:36:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.780+0000 7fbb67602b80 10 client.0 _ll_register_callbacks cb 0x55c053b52bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.780+0000 7fbb4d600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c97da0 0x55c053c9d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.780+0000 7fbb4d600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c97da0 0x55c053c9d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36114/0 (socket says 172.21.15.131:36114) 2024-05-10T05:36:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.780+0000 7fbb4d600700 1 -- 172.21.15.131:0/1817774649 learned_addr learned my addr 172.21.15.131:0/1817774649 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:42.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.781+0000 7fbb47e00700 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 0x55c053c9d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.781+0000 7fbb4d600700 1 -- 172.21.15.131:0/1817774649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8d720 msgr2=0x55c053c9ec20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.781+0000 7fbb4d600700 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8d720 0x55c053c9ec20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.781+0000 7fbb4d600700 1 -- 172.21.15.131:0/1817774649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 msgr2=0x55c053c9d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.781+0000 7fbb4d600700 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 0x55c053c9d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.781+0000 7fbb4d600700 1 -- 172.21.15.131:0/1817774649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb481a0380 con 0x55c053c97da0 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.781+0000 7fbb47e00700 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 0x55c053c9d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.781+0000 7fbb4d600700 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c97da0 0x55c053c9d7d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbb40007f00 tx=0x7fbb40016620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a43dc383153bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.781+0000 7fbb2e000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.782+0000 7fbb2e000700 1 -- 172.21.15.131:0/1817774649 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000e040 con 0x55c053c97da0 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.782+0000 7fbb2e000700 1 -- 172.21.15.131:0/1817774649 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fbb4001c070 con 0x55c053c97da0 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.782+0000 7fbb67602b80 1 -- 172.21.15.131:0/1817774649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb34004ca0 con 0x55c053c97da0 2024-05-10T05:36:42.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.782+0000 7fbb2e000700 1 -- 172.21.15.131:0/1817774649 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb40009410 con 0x55c053c97da0 2024-05-10T05:36:42.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.783+0000 7fbb67602b80 10 client.15963 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:42.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.783+0000 7fbb67602b80 1 -- 172.21.15.131:0/1817774649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c053a32d50 con 0x55c053c97da0 2024-05-10T05:36:42.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.783+0000 7fbb2e000700 1 -- 172.21.15.131:0/1817774649 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb34004ca0 con 0x55c053c97da0 2024-05-10T05:36:42.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.783+0000 7fbb2e000700 1 -- 172.21.15.131:0/1817774649 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbb40020430 con 0x55c053c97da0 2024-05-10T05:36:42.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.783+0000 7fbb67602b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:42.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.783+0000 7fbb67602b80 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:42.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.783+0000 7fbb67602b80 20 client.15963 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:42.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.783+0000 7fbb67602b80 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c053d032c0 0x55c053d056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:42.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.783+0000 7fbb67602b80 4 client.15963 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:36:42.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.783+0000 7fbb67602b80 1 -- 172.21.15.131:0/1817774649 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c053ba5f40 con 0x55c053d032c0 2024-05-10T05:36:42.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.784+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c053d032c0 0x55c053d056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:42.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.784+0000 7fbb2e000700 10 client.15963 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:42.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.784+0000 7fbb4cc00700 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c053d032c0 0x55c053d056e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44cf56cb7d5a8681 server_cookie=37a1e6b26ddac376 in_seq=0 out_seq=0 2024-05-10T05:36:42.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.786+0000 7fbb2e000700 1 -- 172.21.15.131:0/1817774649 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c053ba5f40 con 0x55c053d032c0 2024-05-10T05:36:42.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.786+0000 7fbb2e000700 10 client.15963 handle_command_reply: tid=0 2024-05-10T05:36:42.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.786+0000 7fbb67602b80 1 client.15963 shutdown 2024-05-10T05:36:42.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.787+0000 7fbb67602b80 1 -- 172.21.15.131:0/1817774649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c97da0 msgr2=0x55c053c9d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:42.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.787+0000 7fbb67602b80 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c97da0 0x55c053c9d7d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbb40007f00 tx=0x7fbb40016620 comp rx=0 tx=0).stop 2024-05-10T05:36:42.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.787+0000 7fbb67602b80 1 -- 172.21.15.131:0/1817774649 shutdown_connections 2024-05-10T05:36:42.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.787+0000 7fbb67602b80 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c053d032c0 0x55c053d056e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.787+0000 7fbb67602b80 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c053c97da0 0x55c053c9d7d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.787+0000 7fbb67602b80 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c053c8d720 0x55c053c9ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.787+0000 7fbb67602b80 1 --2- 172.21.15.131:0/1817774649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c053c8ef40 0x55c053c9d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.787+0000 7fbb67602b80 1 -- 172.21.15.131:0/1817774649 >> 172.21.15.131:0/1817774649 conn(0x55c053befae0 msgr2=0x55c053c97660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:42.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.787+0000 7fbb2e000700 0 client.15963 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:42.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.787+0000 7fbb67602b80 1 -- 172.21.15.131:0/1817774649 shutdown_connections 2024-05-10T05:36:42.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.788+0000 7fbb67602b80 1 -- 172.21.15.131:0/1817774649 wait complete. 2024-05-10T05:36:42.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.788+0000 7fbb67602b80 20 client.15963 trim_cache size 0 max 16384 2024-05-10T05:36:42.788 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:36:42.789 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:36:42.789 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "22b5a4ca-9b8e-4ecb-b026-3f0bdb019d68", 2024-05-10T05:36:42.789 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:36:42.789 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:36:42.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 -- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb300a1240 msgr2=0x7fbb300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:42.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb300a1240 0x7fbb300a3750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb34006010 tx=0x7fbb34009510 comp rx=0 tx=0).stop 2024-05-10T05:36:42.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 -- 172.21.15.131:0/2019741553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb48100ac0 msgr2=0x7fbb48197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:42.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb48100ac0 0x7fbb48197f20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00d950 tx=0x7fbb3c00dc60 comp rx=0 tx=0).stop 2024-05-10T05:36:42.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 -- 172.21.15.131:0/2019741553 shutdown_connections 2024-05-10T05:36:42.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fbb300a1240 0x7fbb300a3750 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb48102580 0x7fbb4819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb48100ac0 0x7fbb48197f20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 --2- 172.21.15.131:0/2019741553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb48101be0 0x7fbb48198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:42.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 -- 172.21.15.131:0/2019741553 >> 172.21.15.131:0/2019741553 conn(0x7fbb480fc0d0 msgr2=0x7fbb48104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:42.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 -- 172.21.15.131:0/2019741553 shutdown_connections 2024-05-10T05:36:42.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:42.791+0000 7fbb4f600700 1 -- 172.21.15.131:0/2019741553 wait complete. 2024-05-10T05:36:42.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:36:43.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.436+0000 7f6ec5000700 1 -- 172.21.15.131:0/82741549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec0109130 msgr2=0x7f6ec0109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:43.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.436+0000 7f6ec5000700 1 --2- 172.21.15.131:0/82741549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec0109130 0x7f6ec0109540 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0098f0 tx=0x7f6eac009c00 comp rx=0 tx=0).stop 2024-05-10T05:36:43.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.437+0000 7f6ec5000700 1 -- 172.21.15.131:0/82741549 shutdown_connections 2024-05-10T05:36:43.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.437+0000 7f6ec5000700 1 --2- 172.21.15.131:0/82741549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec0109ad0 0x7f6ec0110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.437+0000 7f6ec5000700 1 --2- 172.21.15.131:0/82741549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0108010 0x7f6ec0108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.437+0000 7f6ec5000700 1 --2- 172.21.15.131:0/82741549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec0109130 0x7f6ec0109540 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.437+0000 7f6ec5000700 1 -- 172.21.15.131:0/82741549 >> 172.21.15.131:0/82741549 conn(0x7f6ec0077040 msgr2=0x7f6ec0106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:43.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.437+0000 7f6ec5000700 1 -- 172.21.15.131:0/82741549 shutdown_connections 2024-05-10T05:36:43.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.437+0000 7f6ec5000700 1 -- 172.21.15.131:0/82741549 wait complete. 2024-05-10T05:36:43.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.439+0000 7f6ec5000700 1 Processor -- start 2024-05-10T05:36:43.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.440+0000 7f6ec5000700 1 -- start start 2024-05-10T05:36:43.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.441+0000 7f6ec5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0108010 0x7f6ec006ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.441+0000 7f6ec5000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec0109130 0x7f6ec00704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.441+0000 7f6ebf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0108010 0x7f6ec006ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.441+0000 7f6ebf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0108010 0x7f6ec006ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34244/0 (socket says 172.21.15.131:34244) 2024-05-10T05:36:43.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.441+0000 7f6ec5000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec0109ad0 0x7f6ec01ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.442+0000 7f6ec5000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec01ae680 con 0x7f6ec0109ad0 2024-05-10T05:36:43.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.442+0000 7f6ec5000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec01ae7f0 con 0x7f6ec0108010 2024-05-10T05:36:43.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.442+0000 7f6ec5000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec01ae9b0 con 0x7f6ec0109130 2024-05-10T05:36:43.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.443+0000 7f6ebea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec0109130 0x7f6ec00704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.443+0000 7f6ebf400700 1 -- 172.21.15.131:0/2474462216 learned_addr learned my addr 172.21.15.131:0/2474462216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:43.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.444+0000 7f6ebfe00700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec0109ad0 0x7f6ec01ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.445+0000 7f6ebfe00700 1 -- 172.21.15.131:0/2474462216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec0109130 msgr2=0x7f6ec00704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:43.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.445+0000 7f6ebfe00700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec0109130 0x7f6ec00704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.445+0000 7f6ebfe00700 1 -- 172.21.15.131:0/2474462216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0108010 msgr2=0x7f6ec006ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:43.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.445+0000 7f6ebfe00700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0108010 0x7f6ec006ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.445+0000 7f6ebfe00700 1 -- 172.21.15.131:0/2474462216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac009520 con 0x7f6ec0109ad0 2024-05-10T05:36:43.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.446+0000 7f6ebfe00700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec0109ad0 0x7f6ec01ae110 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800efc0 tx=0x7f6eb800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:43.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.447+0000 7f6eb7e00700 1 -- 172.21.15.131:0/2474462216 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8018070 con 0x7f6ec0109ad0 2024-05-10T05:36:43.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.447+0000 7f6ec5000700 1 -- 172.21.15.131:0/2474462216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec01aec30 con 0x7f6ec0109ad0 2024-05-10T05:36:43.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.447+0000 7f6ec5000700 1 -- 172.21.15.131:0/2474462216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec01af140 con 0x7f6ec0109ad0 2024-05-10T05:36:43.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.448+0000 7f6eb7e00700 1 -- 172.21.15.131:0/2474462216 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eb800f040 con 0x7f6ec0109ad0 2024-05-10T05:36:43.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.448+0000 7f6eb7e00700 1 -- 172.21.15.131:0/2474462216 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8018070 con 0x7f6ec0109ad0 2024-05-10T05:36:43.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.456+0000 7f6ede5dfb80 1 Processor -- start 2024-05-10T05:36:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.457+0000 7f6ede5dfb80 1 -- start start 2024-05-10T05:36:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82a530 0x561bff82a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff82bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff8329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff751150 con 0x561bff82c0f0 2024-05-10T05:36:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff74eb10 con 0x561bff82b7a0 2024-05-10T05:36:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff74a400 con 0x561bff82a530 2024-05-10T05:36:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ebf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff82bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ebf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff82bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34250/0 (socket says 172.21.15.131:34250) 2024-05-10T05:36:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ebf400700 1 -- 172.21.15.131:0/1016966057 learned_addr learned my addr 172.21.15.131:0/1016966057 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ebf400700 1 -- 172.21.15.131:0/1016966057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82a530 msgr2=0x561bff82a960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ebf400700 1 --2- 172.21.15.131:0/1016966057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82a530 0x561bff82a960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ebf400700 1 -- 172.21.15.131:0/1016966057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 msgr2=0x561bff8329f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ebf400700 1 --2- 172.21.15.131:0/1016966057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff8329f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.458+0000 7f6ebf400700 1 -- 172.21.15.131:0/1016966057 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bff5cb3b0 con 0x561bff82b7a0 2024-05-10T05:36:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.459+0000 7f6ebf400700 1 --2- 172.21.15.131:0/1016966057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff82bbb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000ad10 tx=0x7f6eb000f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9b8fe175994fd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.459+0000 7f6eb4c00700 1 -- 172.21.15.131:0/1016966057 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0018640 con 0x561bff82b7a0 2024-05-10T05:36:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.459+0000 7f6eb4c00700 1 -- 172.21.15.131:0/1016966057 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eb000fec0 con 0x561bff82b7a0 2024-05-10T05:36:43.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.459+0000 7f6eb4c00700 1 -- 172.21.15.131:0/1016966057 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0010ac0 con 0x561bff82b7a0 2024-05-10T05:36:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.460+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1016966057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 msgr2=0x561bff82bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.460+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/1016966057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff82bbb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000ad10 tx=0x7f6eb000f6a0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.465+0000 7f6eb7e00700 1 -- 172.21.15.131:0/2474462216 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6eb80092e0 con 0x7f6ec0109ad0 2024-05-10T05:36:43.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.467+0000 7f6eb7e00700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6ea40a1250 0x7f6ea40a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.468+0000 7f6eb7e00700 1 -- 172.21.15.131:0/2474462216 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6eb80cc160 con 0x7f6ec0109ad0 2024-05-10T05:36:43.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.468+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1016966057 shutdown_connections 2024-05-10T05:36:43.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.468+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/1016966057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff8329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.468+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/1016966057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82a530 0x561bff82a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.468+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/1016966057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff82bbb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.468+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1016966057 >> 172.21.15.131:0/1016966057 conn(0x561bff78ca60 msgr2=0x561bff825330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:43.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.469+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1016966057 shutdown_connections 2024-05-10T05:36:43.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.469+0000 7f6ebea00700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6ea40a1250 0x7f6ea40a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.470+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1016966057 wait complete. 2024-05-10T05:36:43.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.471+0000 7f6ebea00700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6ea40a1250 0x7f6ea40a3760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0098c0 tx=0x7f6eac019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:43.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.474+0000 7f6ede5dfb80 1 -- start start 2024-05-10T05:36:43.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.474+0000 7f6ede5dfb80 1 Processor -- start 2024-05-10T05:36:43.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.475+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82a530 0x561bff834820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.475+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82b7a0 0x561bff834d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.476+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff83bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.476+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff6e6560 con 0x561bff82c0f0 2024-05-10T05:36:43.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.476+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff6e6200 con 0x561bff82a530 2024-05-10T05:36:43.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.476+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff5cd6a0 con 0x561bff82b7a0 2024-05-10T05:36:43.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.476+0000 7f6ede5dfb80 10 client.0 _ll_register_callbacks cb 0x561bff6efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:43.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.476+0000 7f6ebf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82a530 0x561bff834820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.476+0000 7f6ebf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82a530 0x561bff834820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34266/0 (socket says 172.21.15.131:34266) 2024-05-10T05:36:43.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.476+0000 7f6ebf400700 1 -- 172.21.15.131:0/1823726831 learned_addr learned my addr 172.21.15.131:0/1823726831 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:43.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.477+0000 7f6ebf400700 1 -- 172.21.15.131:0/1823726831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82b7a0 msgr2=0x561bff834d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:36:43.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.477+0000 7f6ebf400700 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82b7a0 0x561bff834d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.477+0000 7f6ebf400700 1 -- 172.21.15.131:0/1823726831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 msgr2=0x561bff83bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:43.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.477+0000 7f6ebf400700 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff83bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.477+0000 7f6ebf400700 1 -- 172.21.15.131:0/1823726831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb000a970 con 0x561bff82a530 2024-05-10T05:36:43.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.477+0000 7f6ebf400700 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82a530 0x561bff834820 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000a730 tx=0x7f6eb0010460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60f3570b0be375df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:43.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.477+0000 7f6ea9600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:36:43.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.478+0000 7f6ea9600700 1 -- 172.21.15.131:0/1823726831 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00053d0 con 0x561bff82a530 2024-05-10T05:36:43.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.478+0000 7f6ea9600700 1 -- 172.21.15.131:0/1823726831 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6eb0005530 con 0x561bff82a530 2024-05-10T05:36:43.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.479+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1823726831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bff7f9780 con 0x561bff82a530 2024-05-10T05:36:43.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.482+0000 7f6ea9600700 1 -- 172.21.15.131:0/1823726831 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eb00056f0 con 0x561bff82a530 2024-05-10T05:36:43.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.483+0000 7f6ede5dfb80 10 client.26021 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:43.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.483+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1823726831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561bff577b60 con 0x561bff82a530 2024-05-10T05:36:43.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.485+0000 7f6ea9600700 1 -- 172.21.15.131:0/1823726831 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eb002f030 con 0x561bff82a530 2024-05-10T05:36:43.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.485+0000 7f6ea9600700 1 -- 172.21.15.131:0/1823726831 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6eb00036a0 con 0x561bff82a530 2024-05-10T05:36:43.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.485+0000 7f6ede5dfb80 10 client.26021 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:43.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.485+0000 7f6ede5dfb80 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.485+0000 7f6ede5dfb80 20 client.26021 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.485+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561bff851470 0x561bff853900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.485+0000 7f6ede5dfb80 4 client.26021 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:36:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.485+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1823726831 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bff7f9780 con 0x561bff851470 2024-05-10T05:36:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.485+0000 7f6ebea00700 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561bff851470 0x561bff853900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.486+0000 7f6ea9600700 10 client.26021 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:43.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.488+0000 7f6ebea00700 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561bff851470 0x561bff853900 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a05451befe36ce36 server_cookie=d396e9bf633ff962 in_seq=0 out_seq=0 2024-05-10T05:36:43.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.488+0000 7f6ea9600700 1 -- 172.21.15.131:0/1823726831 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561bff7f9780 con 0x561bff851470 2024-05-10T05:36:43.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.488+0000 7f6ea9600700 10 client.26021 handle_command_reply: tid=0 2024-05-10T05:36:43.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.489+0000 7f6ede5dfb80 1 client.26021 shutdown 2024-05-10T05:36:43.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.489+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1823726831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82a530 msgr2=0x561bff834820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:43.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.489+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82a530 0x561bff834820 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000a730 tx=0x7f6eb0010460 comp rx=0 tx=0).stop 2024-05-10T05:36:43.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.489+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1823726831 shutdown_connections 2024-05-10T05:36:43.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.489+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561bff851470 0x561bff853900 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.490+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff83bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.490+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82a530 0x561bff834820 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.490+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/1823726831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82b7a0 0x561bff834d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.490+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1823726831 >> 172.21.15.131:0/1823726831 conn(0x561bff78ca60 msgr2=0x561bff829060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:43.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.490+0000 7f6ea9600700 0 client.26021 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:43.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.495+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1823726831 shutdown_connections 2024-05-10T05:36:43.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.495+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/1823726831 wait complete. 2024-05-10T05:36:43.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.496+0000 7f6ede5dfb80 20 client.26021 trim_cache size 0 max 16384 2024-05-10T05:36:43.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.539+0000 7f6ede5dfb80 1 Processor -- start 2024-05-10T05:36:43.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.539+0000 7f6ede5dfb80 1 -- start start 2024-05-10T05:36:43.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.540+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82a530 0x561bff853890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.540+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff83bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.540+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82c0f0 0x561bff825190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.540+0000 7f6ebea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff83bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.541+0000 7f6ebea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff83bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34268/0 (socket says 172.21.15.131:34268) 2024-05-10T05:36:43.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.541+0000 7f6ebf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82a530 0x561bff853890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.541+0000 7f6ebf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82a530 0x561bff853890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60948/0 (socket says 172.21.15.131:60948) 2024-05-10T05:36:43.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.547+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff832d50 con 0x561bff82a530 2024-05-10T05:36:43.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.547+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff835190 con 0x561bff82b7a0 2024-05-10T05:36:43.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.547+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff82c500 con 0x561bff82c0f0 2024-05-10T05:36:43.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.547+0000 7f6ebea00700 1 -- 172.21.15.131:0/4199308284 learned_addr learned my addr 172.21.15.131:0/4199308284 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:43.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.548+0000 7f6ebfe00700 1 --2- 172.21.15.131:0/4199308284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82c0f0 0x561bff825190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.548+0000 7f6ebfe00700 1 -- 172.21.15.131:0/4199308284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 msgr2=0x561bff83bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:43.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.548+0000 7f6ebfe00700 1 --2- 172.21.15.131:0/4199308284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff83bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.548+0000 7f6ebfe00700 1 -- 172.21.15.131:0/4199308284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82a530 msgr2=0x561bff853890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:43.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.549+0000 7f6ebfe00700 1 --2- 172.21.15.131:0/4199308284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82a530 0x561bff853890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.549+0000 7f6ebfe00700 1 -- 172.21.15.131:0/4199308284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bff710560 con 0x561bff82c0f0 2024-05-10T05:36:43.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.549+0000 7f6ebf400700 1 --2- 172.21.15.131:0/4199308284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82a530 0x561bff853890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:43.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.550+0000 7f6ebfe00700 1 --2- 172.21.15.131:0/4199308284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82c0f0 0x561bff825190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80994e0 tx=0x7f6eb80993a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d74fb80508392dca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:43.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.552+0000 7f6eb5600700 1 -- 172.21.15.131:0/4199308284 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8018070 con 0x561bff82c0f0 2024-05-10T05:36:43.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.552+0000 7f6eb5600700 1 -- 172.21.15.131:0/4199308284 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eb8098eb0 con 0x561bff82c0f0 2024-05-10T05:36:43.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.552+0000 7f6eb5600700 1 -- 172.21.15.131:0/4199308284 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80c9d10 con 0x561bff82c0f0 2024-05-10T05:36:43.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.554+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4199308284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82c0f0 msgr2=0x561bff825190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:43.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.554+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/4199308284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82c0f0 0x561bff825190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80994e0 tx=0x7f6eb80993a0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.558+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4199308284 shutdown_connections 2024-05-10T05:36:43.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.558+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/4199308284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82a530 0x561bff853890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.558+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/4199308284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82c0f0 0x561bff825190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.558+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/4199308284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff83bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.558+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4199308284 >> 172.21.15.131:0/4199308284 conn(0x561bff78ca60 msgr2=0x561bff82e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:43.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.562+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4199308284 shutdown_connections 2024-05-10T05:36:43.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.562+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4199308284 wait complete. 2024-05-10T05:36:43.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.570+0000 7f6ede5dfb80 1 -- start start 2024-05-10T05:36:43.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.572+0000 7f6ede5dfb80 1 Processor -- start 2024-05-10T05:36:43.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.578+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82a530 0x561bff8a1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.578+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff8a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.578+0000 7f6ede5dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff8a6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.578+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff8a6970 con 0x561bff82c0f0 2024-05-10T05:36:43.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.578+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff8a6ae0 con 0x561bff82b7a0 2024-05-10T05:36:43.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.578+0000 7f6ede5dfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff8a6ca0 con 0x561bff82a530 2024-05-10T05:36:43.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.579+0000 7f6ede5dfb80 10 client.0 _ll_register_callbacks cb 0x7f6eac009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:36:43.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.580+0000 7f6ebfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff8a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.580+0000 7f6ebfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff8a6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60964/0 (socket says 172.21.15.131:60964) 2024-05-10T05:36:43.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.580+0000 7f6ebfe00700 1 -- 172.21.15.131:0/4092396895 learned_addr learned my addr 172.21.15.131:0/4092396895 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:36:43.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.581+0000 7f6ebf400700 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82a530 0x561bff8a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.582+0000 7f6ebea00700 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff8a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.582+0000 7f6ebf400700 1 -- 172.21.15.131:0/4092396895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 msgr2=0x561bff8a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:43.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.582+0000 7f6ebf400700 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff8a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.582+0000 7f6ebf400700 1 -- 172.21.15.131:0/4092396895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 msgr2=0x561bff8a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:36:43.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.582+0000 7f6ebf400700 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff8a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.582+0000 7f6ebf400700 1 -- 172.21.15.131:0/4092396895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bff577b60 con 0x561bff82a530 2024-05-10T05:36:43.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.582+0000 7f6ebea00700 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff8a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:36:43.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.583+0000 7f6ebf400700 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82a530 0x561bff8a1990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000ac60 tx=0x7f6eb0022670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9577341d345e8fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:36:43.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.583+0000 7f6ea9600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:36:43.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.584+0000 7f6ea9600700 1 -- 172.21.15.131:0/4092396895 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00055e0 con 0x561bff82a530 2024-05-10T05:36:43.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.584+0000 7f6ea9600700 1 -- 172.21.15.131:0/4092396895 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6eb0003fb0 con 0x561bff82a530 2024-05-10T05:36:43.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.584+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4092396895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bff6a3270 con 0x561bff82a530 2024-05-10T05:36:43.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.584+0000 7f6ea9600700 1 -- 172.21.15.131:0/4092396895 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eb0004170 con 0x561bff82a530 2024-05-10T05:36:43.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.585+0000 7f6ede5dfb80 10 client.26056 fetch_fsmap learned FSMap version 28 2024-05-10T05:36:43.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.585+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4092396895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f6ec01af140 con 0x561bff82a530 2024-05-10T05:36:43.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.585+0000 7f6ea9600700 1 -- 172.21.15.131:0/4092396895 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eb0043020 con 0x561bff82a530 2024-05-10T05:36:43.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.585+0000 7f6ea9600700 1 -- 172.21.15.131:0/4092396895 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6eb0022790 con 0x561bff82a530 2024-05-10T05:36:43.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.586+0000 7f6ede5dfb80 10 client.26056 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:36:43.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.586+0000 7f6ede5dfb80 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:36:43.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.586+0000 7f6ede5dfb80 20 client.26056 populate_metadata read hostname 'smithi131' 2024-05-10T05:36:43.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.586+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561bff8a8340 0x561bff8aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:36:43.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.586+0000 7f6ede5dfb80 4 client.26056 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:36:43.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.586+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4092396895 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561bff6a3270 con 0x561bff8a8340 2024-05-10T05:36:43.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.586+0000 7f6ebea00700 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561bff8a8340 0x561bff8aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:36:43.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.587+0000 7f6ea9600700 10 client.26056 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:43.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.588+0000 7f6ebea00700 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561bff8a8340 0x561bff8aa760 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=632988d1eb066f04 server_cookie=eb547c59c21181e5 in_seq=0 out_seq=0 2024-05-10T05:36:43.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.588+0000 7f6ea9600700 1 -- 172.21.15.131:0/4092396895 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561bff6a3270 con 0x561bff8a8340 2024-05-10T05:36:43.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.588+0000 7f6ea9600700 10 client.26056 handle_command_reply: tid=0 2024-05-10T05:36:43.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.589+0000 7f6ede5dfb80 1 client.26056 shutdown 2024-05-10T05:36:43.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.589+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4092396895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82a530 msgr2=0x561bff8a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:43.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.589+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82a530 0x561bff8a1990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000ac60 tx=0x7f6eb0022670 comp rx=0 tx=0).stop 2024-05-10T05:36:43.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.590+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4092396895 shutdown_connections 2024-05-10T05:36:43.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.590+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561bff8a8340 0x561bff8aa760 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.590+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561bff82c0f0 0x561bff8a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.590+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561bff82a530 0x561bff8a1990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.590+0000 7f6ede5dfb80 1 --2- 172.21.15.131:0/4092396895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bff82b7a0 0x561bff8a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.590+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4092396895 >> 172.21.15.131:0/4092396895 conn(0x561bff78ca60 msgr2=0x561bff82e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:43.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.590+0000 7f6ea9600700 0 client.26056 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:36:43.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.591+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4092396895 shutdown_connections 2024-05-10T05:36:43.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.591+0000 7f6ede5dfb80 1 -- 172.21.15.131:0/4092396895 wait complete. 2024-05-10T05:36:43.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.592+0000 7f6ede5dfb80 20 client.26056 trim_cache size 0 max 16384 2024-05-10T05:36:43.592 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:36:43.592 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:36:43.592 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:36:43.592 INFO:teuthology.orchestra.run.smithi131.stdout: "22b5a4ca-9b8e-4ecb-b026-3f0bdb019d68": { 2024-05-10T05:36:43.592 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:36:43.592 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "22b5a4ca-9b8e-4ecb-b026-3f0bdb019d68", 2024-05-10T05:36:43.592 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:36:43.593 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:36:43.593 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:36:43.593 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:36:43.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.594+0000 7f6ec5000700 1 -- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6ea40a1250 msgr2=0x7f6ea40a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:43.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.594+0000 7f6ec5000700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6ea40a1250 0x7f6ea40a3760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0098c0 tx=0x7f6eac019040 comp rx=0 tx=0).stop 2024-05-10T05:36:43.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.594+0000 7f6ec5000700 1 -- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec0109ad0 msgr2=0x7f6ec01ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:36:43.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.594+0000 7f6ec5000700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec0109ad0 0x7f6ec01ae110 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800efc0 tx=0x7f6eb800c3a0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.595+0000 7f6ec5000700 1 -- 172.21.15.131:0/2474462216 shutdown_connections 2024-05-10T05:36:43.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.595+0000 7f6ec5000700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6ea40a1250 0x7f6ea40a3760 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.595+0000 7f6ec5000700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec0109ad0 0x7f6ec01ae110 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.595+0000 7f6ec5000700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec0108010 0x7f6ec006ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.595+0000 7f6ec5000700 1 --2- 172.21.15.131:0/2474462216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec0109130 0x7f6ec00704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:36:43.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.595+0000 7f6ec5000700 1 -- 172.21.15.131:0/2474462216 >> 172.21.15.131:0/2474462216 conn(0x7f6ec0077040 msgr2=0x7f6ec010c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:36:43.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.595+0000 7f6ec5000700 1 -- 172.21.15.131:0/2474462216 shutdown_connections 2024-05-10T05:36:43.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:36:43.595+0000 7f6ec5000700 1 -- 172.21.15.131:0/2474462216 wait complete. 2024-05-10T05:36:43.696 INFO:tasks.cephfs.filesystem:scrub status for tag:22b5a4ca-9b8e-4ecb-b026-3f0bdb019d68 - {'path': '/', 'tag': '22b5a4ca-9b8e-4ecb-b026-3f0bdb019d68', 'options': 'recursive,force'} 2024-05-10T05:36:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:44 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:36:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:44 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:44 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:36:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:44 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:36:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:44 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:44 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:44 smithi142 ceph-mon[77667]: pgmap v386: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 MiB/s wr, 71 op/s 2024-05-10T05:36:44.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:44 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:36:44.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:44 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:44.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:44 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:36:44.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:44 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:36:44.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:44 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:44.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:44 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:44.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:44 smithi131 ceph-mon[37264]: pgmap v386: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 MiB/s wr, 71 op/s 2024-05-10T05:36:44.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:44 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:36:44.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:44 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:44.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:44 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:36:44.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:44 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:36:44.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:44 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:44.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:44 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:36:44.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:44 smithi157 ceph-mon[78427]: pgmap v386: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 MiB/s wr, 71 op/s 2024-05-10T05:36:46.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:46 smithi131 ceph-mon[37264]: pgmap v387: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 263 MiB/s wr, 78 op/s 2024-05-10T05:36:46.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:46 smithi157 ceph-mon[78427]: pgmap v387: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 263 MiB/s wr, 78 op/s 2024-05-10T05:36:46.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:46 smithi142 ceph-mon[77667]: pgmap v387: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 263 MiB/s wr, 78 op/s 2024-05-10T05:36:47.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:47 smithi131 ceph-mon[37264]: pgmap v388: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 228 MiB/s wr, 70 op/s 2024-05-10T05:36:47.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:47 smithi157 ceph-mon[78427]: pgmap v388: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 228 MiB/s wr, 70 op/s 2024-05-10T05:36:47.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:47 smithi142 ceph-mon[77667]: pgmap v388: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 228 MiB/s wr, 70 op/s 2024-05-10T05:36:50.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:50 smithi142 ceph-mon[77667]: pgmap v389: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 249 MiB/s wr, 74 op/s 2024-05-10T05:36:50.466 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:50 smithi131 ceph-mon[37264]: pgmap v389: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 249 MiB/s wr, 74 op/s 2024-05-10T05:36:50.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:50 smithi157 ceph-mon[78427]: pgmap v389: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 249 MiB/s wr, 74 op/s 2024-05-10T05:36:51.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:51 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:36:51.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:51 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:36:51.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:51 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:36:51.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:51 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:36:51.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:51 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:36:51.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:51 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:36:52.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:52 smithi142 ceph-mon[77667]: pgmap v390: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 244 MiB/s wr, 72 op/s 2024-05-10T05:36:52.463 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:52 smithi131 ceph-mon[37264]: pgmap v390: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 244 MiB/s wr, 72 op/s 2024-05-10T05:36:52.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:52 smithi157 ceph-mon[78427]: pgmap v390: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 244 MiB/s wr, 72 op/s 2024-05-10T05:36:54.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:54 smithi142 ceph-mon[77667]: pgmap v391: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 241 MiB/s wr, 73 op/s 2024-05-10T05:36:54.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:54 smithi131 ceph-mon[37264]: pgmap v391: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 241 MiB/s wr, 73 op/s 2024-05-10T05:36:54.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:54 smithi157 ceph-mon[78427]: pgmap v391: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 241 MiB/s wr, 73 op/s 2024-05-10T05:36:56.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:56 smithi142 ceph-mon[77667]: pgmap v392: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 80 op/s 2024-05-10T05:36:56.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:56 smithi131 ceph-mon[37264]: pgmap v392: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 80 op/s 2024-05-10T05:36:56.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:56 smithi157 ceph-mon[78427]: pgmap v392: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 80 op/s 2024-05-10T05:36:58.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:36:58 smithi131 ceph-mon[37264]: pgmap v393: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:36:58.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:36:58 smithi157 ceph-mon[78427]: pgmap v393: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:36:58.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:36:58 smithi142 ceph-mon[77667]: pgmap v393: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:37:00.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:00 smithi131 ceph-mon[37264]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 79 op/s 2024-05-10T05:37:00.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:00 smithi157 ceph-mon[78427]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 79 op/s 2024-05-10T05:37:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:00 smithi142 ceph-mon[77667]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 79 op/s 2024-05-10T05:37:01.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:01 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:37:01.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:01 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:01 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:01 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:01 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:01 smithi131 ceph-mon[37264]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 267 MiB/s wr, 77 op/s 2024-05-10T05:37:01.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:01 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:37:01.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:01 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:01 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:01 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:01 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:01 smithi157 ceph-mon[78427]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 267 MiB/s wr, 77 op/s 2024-05-10T05:37:01.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:01 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:37:01.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:01 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:01 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:01 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:01 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:01.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:01 smithi142 ceph-mon[77667]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 267 MiB/s wr, 77 op/s 2024-05-10T05:37:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:02.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:02 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:02 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:02.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:02 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:02.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:02 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:03.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:03 smithi157 ceph-mon[78427]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 258 MiB/s wr, 78 op/s 2024-05-10T05:37:03.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:37:03.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:37:03.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:03 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:03.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:03 smithi142 ceph-mon[77667]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 258 MiB/s wr, 78 op/s 2024-05-10T05:37:03.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:03 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:37:03.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:03 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:37:03.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:03 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:03.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:03 smithi131 ceph-mon[37264]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 258 MiB/s wr, 78 op/s 2024-05-10T05:37:03.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:37:03.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:37:03.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:03 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:37:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:06 smithi142 ceph-mon[77667]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 282 MiB/s wr, 83 op/s 2024-05-10T05:37:06.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:06 smithi131 ceph-mon[37264]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 282 MiB/s wr, 83 op/s 2024-05-10T05:37:06.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:06 smithi157 ceph-mon[78427]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 282 MiB/s wr, 83 op/s 2024-05-10T05:37:08.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:08 smithi142 ceph-mon[77667]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:37:08.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:08 smithi131 ceph-mon[37264]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:37:08.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:08 smithi157 ceph-mon[78427]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:37:10.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:10 smithi142 ceph-mon[77667]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 75 op/s 2024-05-10T05:37:10.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:10 smithi131 ceph-mon[37264]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 75 op/s 2024-05-10T05:37:10.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:10 smithi157 ceph-mon[78427]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 75 op/s 2024-05-10T05:37:12.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:12 smithi142 ceph-mon[77667]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 229 MiB/s wr, 69 op/s 2024-05-10T05:37:12.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:12 smithi131 ceph-mon[37264]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 229 MiB/s wr, 69 op/s 2024-05-10T05:37:12.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:12 smithi157 ceph-mon[78427]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 229 MiB/s wr, 69 op/s 2024-05-10T05:37:13.697 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:37:14.144 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:14 smithi131 ceph-mon[37264]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 225 MiB/s wr, 71 op/s 2024-05-10T05:37:14.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:14 smithi142 ceph-mon[77667]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 225 MiB/s wr, 71 op/s 2024-05-10T05:37:14.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.489+0000 7f2556800700 1 -- 172.21.15.131:0/148499473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2550108630 msgr2=0x7f2550108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:14.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.490+0000 7f2556800700 1 --2- 172.21.15.131:0/148499473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2550108630 0x7f2550108a40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2540009990 tx=0x7f2540009ca0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.494+0000 7f2556800700 1 -- 172.21.15.131:0/148499473 shutdown_connections 2024-05-10T05:37:14.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.495+0000 7f2556800700 1 --2- 172.21.15.131:0/148499473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2550109c50 0x7f255010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.495+0000 7f2556800700 1 --2- 172.21.15.131:0/148499473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2550108630 0x7f2550108a40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.495+0000 7f2556800700 1 --2- 172.21.15.131:0/148499473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2550109300 0x7f2550109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.495+0000 7f2556800700 1 -- 172.21.15.131:0/148499473 >> 172.21.15.131:0/148499473 conn(0x7f255006ea00 msgr2=0x7f2550070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:14.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.500+0000 7f2556800700 1 -- 172.21.15.131:0/148499473 shutdown_connections 2024-05-10T05:37:14.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.501+0000 7f2556800700 1 -- 172.21.15.131:0/148499473 wait complete. 2024-05-10T05:37:14.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.508+0000 7f2556800700 1 Processor -- start 2024-05-10T05:37:14.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.509+0000 7f2556800700 1 -- start start 2024-05-10T05:37:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.512+0000 7f2556800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2550108630 0x7f25501a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.512+0000 7f2556800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2550109300 0x7f25501a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.512+0000 7f2556800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2550109c50 0x7f25501ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.512+0000 7f2556800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501ae890 con 0x7f2550109c50 2024-05-10T05:37:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.512+0000 7f2556800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501aea00 con 0x7f2550108630 2024-05-10T05:37:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.512+0000 7f2556800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501aebc0 con 0x7f2550109300 2024-05-10T05:37:14.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.514+0000 7f254fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2550109c50 0x7f25501ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.514+0000 7f254fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2550109c50 0x7f25501ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43900/0 (socket says 172.21.15.131:43900) 2024-05-10T05:37:14.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.514+0000 7f254fe00700 1 -- 172.21.15.131:0/3405703813 learned_addr learned my addr 172.21.15.131:0/3405703813 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:14.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.515+0000 7f254f400700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2550108630 0x7f25501a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.515+0000 7f254ea00700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2550109300 0x7f25501a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.515+0000 7f254ea00700 1 -- 172.21.15.131:0/3405703813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2550108630 msgr2=0x7f25501a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:14.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.515+0000 7f254ea00700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2550108630 0x7f25501a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.515+0000 7f254ea00700 1 -- 172.21.15.131:0/3405703813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2550109c50 msgr2=0x7f25501ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:14.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.516+0000 7f254ea00700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2550109c50 0x7f25501ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.516+0000 7f254ea00700 1 -- 172.21.15.131:0/3405703813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25400095f0 con 0x7f2550109300 2024-05-10T05:37:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.516+0000 7f254fe00700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2550109c50 0x7f25501ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:37:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.516+0000 7f254ea00700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2550109300 0x7f25501a72f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f253c00c970 tx=0x7f253c00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.517+0000 7f2547e00700 1 -- 172.21.15.131:0/3405703813 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c007b00 con 0x7f2550109300 2024-05-10T05:37:14.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.517+0000 7f2547e00700 1 -- 172.21.15.131:0/3405703813 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f253c00ce60 con 0x7f2550109300 2024-05-10T05:37:14.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.517+0000 7f2547e00700 1 -- 172.21.15.131:0/3405703813 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c005440 con 0x7f2550109300 2024-05-10T05:37:14.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.517+0000 7f2556800700 1 -- 172.21.15.131:0/3405703813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25501aeea0 con 0x7f2550109300 2024-05-10T05:37:14.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.518+0000 7f2556800700 1 -- 172.21.15.131:0/3405703813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501af290 con 0x7f2550109300 2024-05-10T05:37:14.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.537+0000 7f2547e00700 1 -- 172.21.15.131:0/3405703813 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f253c0055a0 con 0x7f2550109300 2024-05-10T05:37:14.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.543+0000 7f256e797b80 1 Processor -- start 2024-05-10T05:37:14.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.548+0000 7f2547e00700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f25340a1330 0x7f25340a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.548+0000 7f2547e00700 1 -- 172.21.15.131:0/3405703813 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f253c0cc320 con 0x7f2550109300 2024-05-10T05:37:14.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.549+0000 7f256e797b80 1 -- start start 2024-05-10T05:37:14.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.550+0000 7f256e797b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff603af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.550+0000 7f256e797b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff604570 0x5590ff604980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.550+0000 7f256e797b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff604ec0 0x5590ff60b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.550+0000 7f256e797b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff52a150 con 0x5590ff604ec0 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.550+0000 7f256e797b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff527b10 con 0x5590ff6036a0 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.550+0000 7f256e797b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff523400 con 0x5590ff604570 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.550+0000 7f254f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff603af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.550+0000 7f254f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff603af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56192/0 (socket says 172.21.15.131:56192) 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.550+0000 7f254f400700 1 -- 172.21.15.131:0/2995329703 learned_addr learned my addr 172.21.15.131:0/2995329703 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.551+0000 7f254f400700 1 -- 172.21.15.131:0/2995329703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff604570 msgr2=0x5590ff604980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.551+0000 7f254f400700 1 --2- 172.21.15.131:0/2995329703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff604570 0x5590ff604980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.551+0000 7f254f400700 1 -- 172.21.15.131:0/2995329703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff604ec0 msgr2=0x5590ff60b7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.551+0000 7f254f400700 1 --2- 172.21.15.131:0/2995329703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff604ec0 0x5590ff60b7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.551+0000 7f254f400700 1 -- 172.21.15.131:0/2995329703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590ff3a43b0 con 0x5590ff6036a0 2024-05-10T05:37:14.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.551+0000 7f254f400700 1 --2- 172.21.15.131:0/2995329703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff603af0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f25400098e0 tx=0x7f2540005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6922f967c9bdc281 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:14.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:14 smithi157 ceph-mon[78427]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 225 MiB/s wr, 71 op/s 2024-05-10T05:37:14.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.551+0000 7f254fe00700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f25340a1330 0x7f25340a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.552+0000 7f254fe00700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f25340a1330 0x7f25340a3840 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2548009df0 tx=0x7f25480097d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:14.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.556+0000 7f2544c00700 1 -- 172.21.15.131:0/2995329703 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000b9d0 con 0x5590ff6036a0 2024-05-10T05:37:14.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.557+0000 7f2544c00700 1 -- 172.21.15.131:0/2995329703 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254001e040 con 0x5590ff6036a0 2024-05-10T05:37:14.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.557+0000 7f2544c00700 1 -- 172.21.15.131:0/2995329703 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000f460 con 0x5590ff6036a0 2024-05-10T05:37:14.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.562+0000 7f256e797b80 1 -- 172.21.15.131:0/2995329703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 msgr2=0x5590ff603af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:14.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.562+0000 7f256e797b80 1 --2- 172.21.15.131:0/2995329703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff603af0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f25400098e0 tx=0x7f2540005040 comp rx=0 tx=0).stop 2024-05-10T05:37:14.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.563+0000 7f256e797b80 1 -- 172.21.15.131:0/2995329703 shutdown_connections 2024-05-10T05:37:14.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.563+0000 7f256e797b80 1 --2- 172.21.15.131:0/2995329703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff604ec0 0x5590ff60b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.563+0000 7f256e797b80 1 --2- 172.21.15.131:0/2995329703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff603af0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.563+0000 7f256e797b80 1 --2- 172.21.15.131:0/2995329703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff604570 0x5590ff604980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.564+0000 7f256e797b80 1 -- 172.21.15.131:0/2995329703 >> 172.21.15.131:0/2995329703 conn(0x5590ff565a60 msgr2=0x5590ff5fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:14.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.566+0000 7f256e797b80 1 -- 172.21.15.131:0/2995329703 shutdown_connections 2024-05-10T05:37:14.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.567+0000 7f256e797b80 1 -- 172.21.15.131:0/2995329703 wait complete. 2024-05-10T05:37:14.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.568+0000 7f256e797b80 1 -- start start 2024-05-10T05:37:14.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.569+0000 7f256e797b80 1 Processor -- start 2024-05-10T05:37:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.575+0000 7f256e797b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff61e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.575+0000 7f256e797b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff604570 0x5590ff61ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.575+0000 7f256e797b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff604ec0 0x5590ff625bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.575+0000 7f254f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff61e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.575+0000 7f254ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff604570 0x5590ff61ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.576+0000 7f254ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff604570 0x5590ff61ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59420/0 (socket says 172.21.15.131:59420) 2024-05-10T05:37:14.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.578+0000 7f254f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff61e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56196/0 (socket says 172.21.15.131:56196) 2024-05-10T05:37:14.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.578+0000 7f254ea00700 1 -- 172.21.15.131:0/4255428592 learned_addr learned my addr 172.21.15.131:0/4255428592 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:14.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.578+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff4bf560 con 0x5590ff604ec0 2024-05-10T05:37:14.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.578+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff4bf200 con 0x5590ff6036a0 2024-05-10T05:37:14.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.578+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff3a66a0 con 0x5590ff604570 2024-05-10T05:37:14.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.579+0000 7f256e797b80 10 client.0 _ll_register_callbacks cb 0x5590ff4c8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:14.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.579+0000 7f254fe00700 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff604ec0 0x5590ff625bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.579+0000 7f254f400700 1 -- 172.21.15.131:0/4255428592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff604570 msgr2=0x5590ff61ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:14.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.579+0000 7f254f400700 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff604570 0x5590ff61ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.579+0000 7f254f400700 1 -- 172.21.15.131:0/4255428592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff604ec0 msgr2=0x5590ff625bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:14.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.579+0000 7f254f400700 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff604ec0 0x5590ff625bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.579+0000 7f254f400700 1 -- 172.21.15.131:0/4255428592 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255010efc0 con 0x5590ff6036a0 2024-05-10T05:37:14.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.582+0000 7f254f400700 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff61e780 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f254000b5c0 tx=0x7f254000ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333866b5aadd48cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:14.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.582+0000 7f2539600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:37:14.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.582+0000 7f2539600700 1 -- 172.21.15.131:0/4255428592 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540003d70 con 0x5590ff6036a0 2024-05-10T05:37:14.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.582+0000 7f2539600700 1 -- 172.21.15.131:0/4255428592 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f254000f460 con 0x5590ff6036a0 2024-05-10T05:37:14.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.582+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590ff47c270 con 0x5590ff6036a0 2024-05-10T05:37:14.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.583+0000 7f2539600700 1 -- 172.21.15.131:0/4255428592 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f254000f620 con 0x5590ff6036a0 2024-05-10T05:37:14.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.583+0000 7f256e797b80 10 client.26036 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:14.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.583+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5590ff350b60 con 0x5590ff6036a0 2024-05-10T05:37:14.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.583+0000 7f2539600700 1 -- 172.21.15.131:0/4255428592 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254003b080 con 0x5590ff6036a0 2024-05-10T05:37:14.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.584+0000 7f2539600700 1 -- 172.21.15.131:0/4255428592 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f254002c840 con 0x5590ff6036a0 2024-05-10T05:37:14.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.584+0000 7f256e797b80 10 client.26036 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:14.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.584+0000 7f256e797b80 10 client.26036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:37:14.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.584+0000 7f256e797b80 20 client.26036 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:14.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.584+0000 7f256e797b80 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590ff629dd0 0x5590ff62c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.584+0000 7f256e797b80 4 client.26036 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:14.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.584+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590ff47c270 con 0x5590ff629dd0 2024-05-10T05:37:14.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.585+0000 7f254ea00700 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590ff629dd0 0x5590ff62c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.587+0000 7f254ea00700 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590ff629dd0 0x5590ff62c1f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a786824843ba2a server_cookie=ed5e5b0365dcdf21 in_seq=0 out_seq=0 2024-05-10T05:37:14.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.587+0000 7f2539600700 10 client.26036 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:14.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.588+0000 7f2539600700 1 -- 172.21.15.131:0/4255428592 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590ff47c270 con 0x5590ff629dd0 2024-05-10T05:37:14.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.588+0000 7f2539600700 10 client.26036 handle_command_reply: tid=0 2024-05-10T05:37:14.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.588+0000 7f256e797b80 1 client.26036 shutdown 2024-05-10T05:37:14.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.590+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 msgr2=0x5590ff61e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:14.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.590+0000 7f256e797b80 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff61e780 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f254000b5c0 tx=0x7f254000ba00 comp rx=0 tx=0).stop 2024-05-10T05:37:14.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.590+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 shutdown_connections 2024-05-10T05:37:14.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.590+0000 7f256e797b80 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590ff629dd0 0x5590ff62c1f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.591+0000 7f256e797b80 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff604ec0 0x5590ff625bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.591+0000 7f2539600700 0 client.26036 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:14.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.591+0000 7f256e797b80 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff6036a0 0x5590ff61e780 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.591+0000 7f256e797b80 1 --2- 172.21.15.131:0/4255428592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff604570 0x5590ff61ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.591+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 >> 172.21.15.131:0/4255428592 conn(0x5590ff565a60 msgr2=0x5590ff5fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:14.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.591+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 shutdown_connections 2024-05-10T05:37:14.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.592+0000 7f256e797b80 1 -- 172.21.15.131:0/4255428592 wait complete. 2024-05-10T05:37:14.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.596+0000 7f256e797b80 20 client.26036 trim_cache size 0 max 16384 2024-05-10T05:37:14.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.639+0000 7f256e797b80 1 Processor -- start 2024-05-10T05:37:14.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.640+0000 7f256e797b80 1 -- start start 2024-05-10T05:37:14.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.640+0000 7f256e797b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff565a60 0x5590ff6083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.640+0000 7f256e797b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff625b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.640+0000 7f256e797b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 0x5590ff608930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.640+0000 7f256e797b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff6043c0 con 0x5590ff565a60 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.640+0000 7f256e797b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff60b710 con 0x5590ff604570 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.640+0000 7f256e797b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff61f380 con 0x5590ff6036a0 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.641+0000 7f254ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff625b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.641+0000 7f254ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff625b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59430/0 (socket says 172.21.15.131:59430) 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.641+0000 7f254ea00700 1 -- 172.21.15.131:0/577356840 learned_addr learned my addr 172.21.15.131:0/577356840 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.641+0000 7f254fe00700 1 --2- 172.21.15.131:0/577356840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 0x5590ff608930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.641+0000 7f254ea00700 1 -- 172.21.15.131:0/577356840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 msgr2=0x5590ff608930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.641+0000 7f254ea00700 1 --2- 172.21.15.131:0/577356840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 0x5590ff608930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.641+0000 7f254ea00700 1 -- 172.21.15.131:0/577356840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff565a60 msgr2=0x5590ff6083f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:14.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.641+0000 7f254ea00700 1 --2- 172.21.15.131:0/577356840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff565a60 0x5590ff6083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.641+0000 7f254ea00700 1 -- 172.21.15.131:0/577356840 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590ff4e9560 con 0x5590ff6036a0 2024-05-10T05:37:14.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.642+0000 7f254ea00700 1 --2- 172.21.15.131:0/577356840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff625b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f253c00c940 tx=0x7f253c0b9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616379978dfb586f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:14.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.642+0000 7f2545600700 1 -- 172.21.15.131:0/577356840 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c09d5a0 con 0x5590ff6036a0 2024-05-10T05:37:14.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.645+0000 7f2545600700 1 -- 172.21.15.131:0/577356840 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590ff47c270 con 0x5590ff6036a0 2024-05-10T05:37:14.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.645+0000 7f2545600700 1 -- 172.21.15.131:0/577356840 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c09d5a0 con 0x5590ff6036a0 2024-05-10T05:37:14.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.646+0000 7f256e797b80 1 -- 172.21.15.131:0/577356840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 msgr2=0x5590ff625b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:14.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.646+0000 7f256e797b80 1 --2- 172.21.15.131:0/577356840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff625b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f253c00c940 tx=0x7f253c0b9040 comp rx=0 tx=0).stop 2024-05-10T05:37:14.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.646+0000 7f256e797b80 1 -- 172.21.15.131:0/577356840 shutdown_connections 2024-05-10T05:37:14.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.646+0000 7f256e797b80 1 --2- 172.21.15.131:0/577356840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff565a60 0x5590ff6083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.646+0000 7f256e797b80 1 --2- 172.21.15.131:0/577356840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 0x5590ff608930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.646+0000 7f256e797b80 1 --2- 172.21.15.131:0/577356840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff625b90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.646+0000 7f256e797b80 1 -- 172.21.15.131:0/577356840 >> 172.21.15.131:0/577356840 conn(0x5590ff5fa450 msgr2=0x5590ff6077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:14.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.647+0000 7f256e797b80 1 -- 172.21.15.131:0/577356840 shutdown_connections 2024-05-10T05:37:14.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.647+0000 7f256e797b80 1 -- 172.21.15.131:0/577356840 wait complete. 2024-05-10T05:37:14.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.647+0000 7f256e797b80 1 -- start start 2024-05-10T05:37:14.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.648+0000 7f256e797b80 1 Processor -- start 2024-05-10T05:37:14.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.649+0000 7f256e797b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff565a60 0x5590ff6238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.649+0000 7f256e797b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff623e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.649+0000 7f256e797b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 0x5590ff624d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f254ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff623e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f254fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 0x5590ff624d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f254fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 0x5590ff624d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56220/0 (socket says 172.21.15.131:56220) 2024-05-10T05:37:14.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f254ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff623e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59440/0 (socket says 172.21.15.131:59440) 2024-05-10T05:37:14.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f254fe00700 1 -- 172.21.15.131:0/2907909608 learned_addr learned my addr 172.21.15.131:0/2907909608 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:14.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff625280 con 0x5590ff565a60 2024-05-10T05:37:14.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff6253f0 con 0x5590ff604570 2024-05-10T05:37:14.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ff6255b0 con 0x5590ff6036a0 2024-05-10T05:37:14.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f256e797b80 10 client.0 _ll_register_callbacks cb 0x5590ff4c8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:14.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f254ea00700 1 -- 172.21.15.131:0/2907909608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 msgr2=0x5590ff624d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:14.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f254ea00700 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 0x5590ff624d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f254ea00700 1 -- 172.21.15.131:0/2907909608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff565a60 msgr2=0x5590ff6238d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:14.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f254ea00700 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff565a60 0x5590ff6238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.650+0000 7f254ea00700 1 -- 172.21.15.131:0/2907909608 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501af290 con 0x5590ff6036a0 2024-05-10T05:37:14.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.651+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f254001e040 con 0x5590ff6036a0 2024-05-10T05:37:14.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.651+0000 7f254ea00700 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff623e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f253c00c940 tx=0x7f253c095470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2331e026b255c53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:14.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.651+0000 7f2539600700 10 client.26071 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:37:14.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.654+0000 7f2539600700 1 -- 172.21.15.131:0/2907909608 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c095e80 con 0x5590ff6036a0 2024-05-10T05:37:14.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.654+0000 7f2539600700 1 -- 172.21.15.131:0/2907909608 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f253c0a1e40 con 0x5590ff6036a0 2024-05-10T05:37:14.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.654+0000 7f2539600700 1 -- 172.21.15.131:0/2907909608 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f253c095fe0 con 0x5590ff6036a0 2024-05-10T05:37:14.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.655+0000 7f256e797b80 10 client.26071 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:14.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.655+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=28}) v3 -- 0x5590ff3a8cd0 con 0x5590ff6036a0 2024-05-10T05:37:14.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.656+0000 7f2539600700 1 -- 172.21.15.131:0/2907909608 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f253c09d120 con 0x5590ff6036a0 2024-05-10T05:37:14.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.659+0000 7f256e797b80 10 client.26071 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:14.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.659+0000 7f256e797b80 10 client.26071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:37:14.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.659+0000 7f256e797b80 20 client.26071 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.659+0000 7f256e797b80 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590ff604ec0 0x5590ff682980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.659+0000 7f256e797b80 4 client.26071 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:37:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.659+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590ff5d2780 con 0x5590ff604ec0 2024-05-10T05:37:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.660+0000 7f254f400700 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590ff604ec0 0x5590ff682980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:14.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.661+0000 7f254f400700 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590ff604ec0 0x5590ff682980 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12fafb886b43e414 server_cookie=80ef985460375414 in_seq=0 out_seq=0 2024-05-10T05:37:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.661+0000 7f2539600700 10 client.26071 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.662+0000 7f2539600700 1 -- 172.21.15.131:0/2907909608 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5590ff5d2780 con 0x5590ff604ec0 2024-05-10T05:37:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.663+0000 7f2539600700 10 client.26071 handle_command_reply: tid=0 2024-05-10T05:37:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.663+0000 7f256e797b80 1 client.26071 shutdown 2024-05-10T05:37:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.665+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 msgr2=0x5590ff623e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.665+0000 7f256e797b80 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff623e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f253c00c940 tx=0x7f253c095470 comp rx=0 tx=0).stop 2024-05-10T05:37:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.665+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 shutdown_connections 2024-05-10T05:37:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.665+0000 7f256e797b80 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5590ff604ec0 0x5590ff682980 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.665+0000 7f256e797b80 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590ff565a60 0x5590ff6238d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.665+0000 7f256e797b80 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ff604570 0x5590ff624d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.665+0000 7f256e797b80 1 --2- 172.21.15.131:0/2907909608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590ff6036a0 0x5590ff623e10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.665+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 >> 172.21.15.131:0/2907909608 conn(0x5590ff5fa450 msgr2=0x5590ff6077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.665+0000 7f2539600700 0 client.26071 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.666+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 shutdown_connections 2024-05-10T05:37:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.666+0000 7f256e797b80 1 -- 172.21.15.131:0/2907909608 wait complete. 2024-05-10T05:37:14.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.667+0000 7f256e797b80 20 client.26071 trim_cache size 0 max 16384 2024-05-10T05:37:14.668 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:37:14.668 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:37:14.668 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:37:14.668 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:37:14.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.670+0000 7f2556800700 1 -- 172.21.15.131:0/3405703813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f25340a1330 msgr2=0x7f25340a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:14.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.670+0000 7f2556800700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f25340a1330 0x7f25340a3840 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2548009df0 tx=0x7f25480097d0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.671+0000 7f2556800700 1 -- 172.21.15.131:0/3405703813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2550109300 msgr2=0x7f25501a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:14.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.671+0000 7f2556800700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2550109300 0x7f25501a72f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f253c00c970 tx=0x7f253c00cc80 comp rx=0 tx=0).stop 2024-05-10T05:37:14.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.671+0000 7f2556800700 1 -- 172.21.15.131:0/3405703813 shutdown_connections 2024-05-10T05:37:14.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.672+0000 7f2556800700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f25340a1330 0x7f25340a3840 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.672+0000 7f2556800700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2550109c50 0x7f25501ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.672+0000 7f2556800700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2550108630 0x7f25501a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.672+0000 7f2556800700 1 --2- 172.21.15.131:0/3405703813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2550109300 0x7f25501a72f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.672+0000 7f2556800700 1 -- 172.21.15.131:0/3405703813 >> 172.21.15.131:0/3405703813 conn(0x7f255006ea00 msgr2=0x7f255006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.674+0000 7f2556800700 1 -- 172.21.15.131:0/3405703813 shutdown_connections 2024-05-10T05:37:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:14.674+0000 7f2556800700 1 -- 172.21.15.131:0/3405703813 wait complete. 2024-05-10T05:37:14.746 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:37:14.746 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:37:15.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.270+0000 7fb257c00700 1 -- 172.21.15.131:0/2763391406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb250108210 msgr2=0x7fb250108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:15.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.270+0000 7fb257c00700 1 --2- 172.21.15.131:0/2763391406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb250108210 0x7fb250108620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb24000f560 tx=0x7fb24000f870 comp rx=0 tx=0).stop 2024-05-10T05:37:15.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.271+0000 7fb257c00700 1 -- 172.21.15.131:0/2763391406 shutdown_connections 2024-05-10T05:37:15.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.271+0000 7fb257c00700 1 --2- 172.21.15.131:0/2763391406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb250110320 0x7fb250112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:15.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.271+0000 7fb257c00700 1 --2- 172.21.15.131:0/2763391406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb250108210 0x7fb250108620 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:15.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.271+0000 7fb257c00700 1 --2- 172.21.15.131:0/2763391406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb250108b60 0x7fb25010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:15.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.271+0000 7fb257c00700 1 -- 172.21.15.131:0/2763391406 >> 172.21.15.131:0/2763391406 conn(0x7fb25006ea00 msgr2=0x7fb250070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.273+0000 7fb257c00700 1 -- 172.21.15.131:0/2763391406 shutdown_connections 2024-05-10T05:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.277+0000 7fb257c00700 1 -- 172.21.15.131:0/2763391406 wait complete. 2024-05-10T05:37:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.278+0000 7fb257c00700 1 Processor -- start 2024-05-10T05:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.281+0000 7fb257c00700 1 -- start start 2024-05-10T05:37:15.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.282+0000 7fb257c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb250108210 0x7fb2501a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:15.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.283+0000 7fb257c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb250110320 0x7fb2501a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:15.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.283+0000 7fb255200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb250108210 0x7fb2501a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:15.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.283+0000 7fb255200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb250108210 0x7fb2501a6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43952/0 (socket says 172.21.15.131:43952) 2024-05-10T05:37:15.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.283+0000 7fb255200700 1 -- 172.21.15.131:0/3514943903 learned_addr learned my addr 172.21.15.131:0/3514943903 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:15.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.283+0000 7fb257c00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2501ac260 0x7fb2501ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:15.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.283+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501aec90 con 0x7fb250108210 2024-05-10T05:37:15.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.283+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501aee00 con 0x7fb250110320 2024-05-10T05:37:15.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.284+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501aefc0 con 0x7fb2501ac260 2024-05-10T05:37:15.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.284+0000 7fb255c00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2501ac260 0x7fb2501ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:15.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.285+0000 7fb24fe00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb250110320 0x7fb2501a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:15.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.286+0000 7fb255200700 1 -- 172.21.15.131:0/3514943903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2501ac260 msgr2=0x7fb2501ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:15.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.286+0000 7fb255200700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2501ac260 0x7fb2501ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:15.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.286+0000 7fb255200700 1 -- 172.21.15.131:0/3514943903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb250110320 msgr2=0x7fb2501a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:15.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.286+0000 7fb255200700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb250110320 0x7fb2501a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:15.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.286+0000 7fb255200700 1 -- 172.21.15.131:0/3514943903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb240009d00 con 0x7fb250108210 2024-05-10T05:37:15.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.286+0000 7fb255200700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb250108210 0x7fb2501a6da0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb240000f80 tx=0x7fb240018f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:15.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.287+0000 7fb24d600700 1 -- 172.21.15.131:0/3514943903 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240009b80 con 0x7fb250108210 2024-05-10T05:37:15.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.287+0000 7fb24d600700 1 -- 172.21.15.131:0/3514943903 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb24000f320 con 0x7fb250108210 2024-05-10T05:37:15.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.287+0000 7fb24d600700 1 -- 172.21.15.131:0/3514943903 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240005750 con 0x7fb250108210 2024-05-10T05:37:15.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.288+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2501af240 con 0x7fb250108210 2024-05-10T05:37:15.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.288+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2501af6f0 con 0x7fb250108210 2024-05-10T05:37:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.294+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22c005320 con 0x7fb250108210 2024-05-10T05:37:15.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.298+0000 7fb24d600700 1 -- 172.21.15.131:0/3514943903 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb2400058b0 con 0x7fb250108210 2024-05-10T05:37:15.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.301+0000 7fb24d600700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2340a1220 0x7fb2340a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:15.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.301+0000 7fb24d600700 1 -- 172.21.15.131:0/3514943903 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb2400cd530 con 0x7fb250108210 2024-05-10T05:37:15.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.302+0000 7fb24fe00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2340a1220 0x7fb2340a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:15.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.303+0000 7fb24fe00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2340a1220 0x7fb2340a3730 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb2501a7e40 tx=0x7fb238009190 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:15.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:15.311+0000 7fb24d600700 1 -- 172.21.15.131:0/3514943903 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb240096cf0 con 0x7fb250108210 2024-05-10T05:37:16.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.073+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb22c005b50 con 0x7fb250108210 2024-05-10T05:37:16.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.077+0000 7fb24d600700 1 -- 172.21.15.131:0/3514943903 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10931 (secure 0 0 0) 0x7fb240005040 con 0x7fb250108210 2024-05-10T05:37:16.079 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:37:16.080 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:35:26.081961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:37:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.087+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2340a1220 msgr2=0x7fb2340a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.087+0000 7fb257c00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2340a1220 0x7fb2340a3730 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb2501a7e40 tx=0x7fb238009190 comp rx=0 tx=0).stop 2024-05-10T05:37:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.087+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb250108210 msgr2=0x7fb2501a6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.087+0000 7fb257c00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb250108210 0x7fb2501a6da0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb240000f80 tx=0x7fb240018f00 comp rx=0 tx=0).stop 2024-05-10T05:37:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.092+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 shutdown_connections 2024-05-10T05:37:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.092+0000 7fb257c00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb2340a1220 0x7fb2340a3730 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.092+0000 7fb257c00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb250108210 0x7fb2501a6da0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.092+0000 7fb257c00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2501ac260 0x7fb2501ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.092+0000 7fb257c00700 1 --2- 172.21.15.131:0/3514943903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb250110320 0x7fb2501a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.092+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 >> 172.21.15.131:0/3514943903 conn(0x7fb25006ea00 msgr2=0x7fb25006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.094+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 shutdown_connections 2024-05-10T05:37:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:16.096+0000 7fb257c00700 1 -- 172.21.15.131:0/3514943903 wait complete. 2024-05-10T05:37:16.121 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 28 2024-05-10T05:37:16.202 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:37:16.439 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:16 smithi131 ceph-mon[37264]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 260 MiB/s wr, 79 op/s 2024-05-10T05:37:16.439 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:16 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3514943903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:37:16.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:16 smithi157 ceph-mon[78427]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 260 MiB/s wr, 79 op/s 2024-05-10T05:37:16.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:16 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3514943903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:37:16.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:16 smithi142 ceph-mon[77667]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 260 MiB/s wr, 79 op/s 2024-05-10T05:37:16.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:16 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3514943903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:37:17.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.142+0000 7f5aeb000700 1 -- 172.21.15.131:0/4172288614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae4108b60 msgr2=0x7f5ae410fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.142+0000 7f5aeb000700 1 --2- 172.21.15.131:0/4172288614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae4108b60 0x7f5ae410fde0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00099c0 tx=0x7f5ad0009cd0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.142+0000 7f5aeb000700 1 -- 172.21.15.131:0/4172288614 shutdown_connections 2024-05-10T05:37:17.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.142+0000 7f5aeb000700 1 --2- 172.21.15.131:0/4172288614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae4108210 0x7f5ae4108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.142+0000 7f5aeb000700 1 --2- 172.21.15.131:0/4172288614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae4110320 0x7f5ae4112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.142+0000 7f5aeb000700 1 --2- 172.21.15.131:0/4172288614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae4108b60 0x7f5ae410fde0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.142+0000 7f5aeb000700 1 -- 172.21.15.131:0/4172288614 >> 172.21.15.131:0/4172288614 conn(0x7f5ae406ea00 msgr2=0x7f5ae4070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:17.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.143+0000 7f5aeb000700 1 -- 172.21.15.131:0/4172288614 shutdown_connections 2024-05-10T05:37:17.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.143+0000 7f5aeb000700 1 -- 172.21.15.131:0/4172288614 wait complete. 2024-05-10T05:37:17.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.157+0000 7f5aeb000700 1 Processor -- start 2024-05-10T05:37:17.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.161+0000 7f5aeb000700 1 -- start start 2024-05-10T05:37:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.163+0000 7f5aeb000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae4108210 0x7f5ae419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.163+0000 7f5aeb000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae4108b60 0x7f5ae419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.163+0000 7f5aeb000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae4110320 0x7f5ae41a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.163+0000 7f5aeb000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae41a6010 con 0x7f5ae4110320 2024-05-10T05:37:17.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.163+0000 7f5aeb000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae41a6180 con 0x7f5ae4108b60 2024-05-10T05:37:17.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.163+0000 7f5aeb000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae41a6340 con 0x7f5ae4108210 2024-05-10T05:37:17.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.164+0000 7f5ae3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae4108b60 0x7f5ae419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.164+0000 7f5ae3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae4108b60 0x7f5ae419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56260/0 (socket says 172.21.15.131:56260) 2024-05-10T05:37:17.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.165+0000 7f5ae3400700 1 -- 172.21.15.131:0/3113244267 learned_addr learned my addr 172.21.15.131:0/3113244267 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.166+0000 7f5ae9000700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae4110320 0x7f5ae41a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.167+0000 7f5ae9000700 1 -- 172.21.15.131:0/3113244267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae4108210 msgr2=0x7f5ae419e530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:37:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.167+0000 7f5ae9000700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae4108210 0x7f5ae419e530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.167+0000 7f5ae9000700 1 -- 172.21.15.131:0/3113244267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae4108b60 msgr2=0x7f5ae419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.167+0000 7f5ae9000700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae4108b60 0x7f5ae419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.167+0000 7f5ae9000700 1 -- 172.21.15.131:0/3113244267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad00095f0 con 0x7f5ae4110320 2024-05-10T05:37:17.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.170+0000 7f5ae9000700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae4110320 0x7f5ae41a5a00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00efc0 tx=0x7f5adc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.172+0000 7f5ae0c00700 1 -- 172.21.15.131:0/3113244267 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc018070 con 0x7f5ae4110320 2024-05-10T05:37:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.174+0000 7f5ae0c00700 1 -- 172.21.15.131:0/3113244267 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5adc00f040 con 0x7f5ae4110320 2024-05-10T05:37:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.174+0000 7f5ae0c00700 1 -- 172.21.15.131:0/3113244267 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc004020 con 0x7f5ae4110320 2024-05-10T05:37:17.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.175+0000 7f5aeb000700 1 -- 172.21.15.131:0/3113244267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae41aedd0 con 0x7f5ae4110320 2024-05-10T05:37:17.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.175+0000 7f5aeb000700 1 -- 172.21.15.131:0/3113244267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae41af1d0 con 0x7f5ae4110320 2024-05-10T05:37:17.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.183+0000 7f5ae0c00700 1 -- 172.21.15.131:0/3113244267 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5adc004ad0 con 0x7f5ae4110320 2024-05-10T05:37:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.187+0000 7f5ae0c00700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5ac80a1310 0x7f5ac80a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.187+0000 7f5ae0c00700 1 -- 172.21.15.131:0/3113244267 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5adc0cc3c0 con 0x7f5ae4110320 2024-05-10T05:37:17.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.188+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5ac80a1310 0x7f5ac80a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.191+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5ac80a1310 0x7f5ac80a3820 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5ae41095f0 tx=0x7f5ad4005c10 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.193+0000 7f5b02f57b80 1 Processor -- start 2024-05-10T05:37:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.195+0000 7f5b02f57b80 1 -- start start 2024-05-10T05:37:17.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.196+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff276a0 0x55b5dff27af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.196+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 0x55b5dff28980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.196+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28ec0 0x55b5dff2f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.196+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dfe4e150 con 0x55b5dff28ec0 2024-05-10T05:37:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.196+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dfe4bb10 con 0x55b5dff28570 2024-05-10T05:37:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.196+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dfe47400 con 0x55b5dff276a0 2024-05-10T05:37:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.197+0000 7f5ae3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 0x55b5dff28980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.197+0000 7f5ae3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 0x55b5dff28980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56268/0 (socket says 172.21.15.131:56268) 2024-05-10T05:37:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.197+0000 7f5ae3400700 1 -- 172.21.15.131:0/3126852617 learned_addr learned my addr 172.21.15.131:0/3126852617 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.198+0000 7f5ae3400700 1 -- 172.21.15.131:0/3126852617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff276a0 msgr2=0x55b5dff27af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:37:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.198+0000 7f5ae3400700 1 --2- 172.21.15.131:0/3126852617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff276a0 0x55b5dff27af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.198+0000 7f5ae3400700 1 -- 172.21.15.131:0/3126852617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28ec0 msgr2=0x55b5dff2f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.198+0000 7f5ae3400700 1 --2- 172.21.15.131:0/3126852617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28ec0 0x55b5dff2f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.198+0000 7f5ae3400700 1 -- 172.21.15.131:0/3126852617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5dfcc83b0 con 0x55b5dff28570 2024-05-10T05:37:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.199+0000 7f5ae3400700 1 --2- 172.21.15.131:0/3126852617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 0x55b5dff28980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000bf50 tx=0x7f5ad0005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16e8be6838b6dcdc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.199+0000 7f5ad9600700 1 -- 172.21.15.131:0/3126852617 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad001d040 con 0x55b5dff28570 2024-05-10T05:37:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.199+0000 7f5ad9600700 1 -- 172.21.15.131:0/3126852617 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ad000bbf0 con 0x55b5dff28570 2024-05-10T05:37:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.199+0000 7f5ad9600700 1 -- 172.21.15.131:0/3126852617 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000f460 con 0x55b5dff28570 2024-05-10T05:37:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.201+0000 7f5b02f57b80 1 -- 172.21.15.131:0/3126852617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 msgr2=0x55b5dff28980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.201+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/3126852617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 0x55b5dff28980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000bf50 tx=0x7f5ad0005040 comp rx=0 tx=0).stop 2024-05-10T05:37:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.202+0000 7f5b02f57b80 1 -- 172.21.15.131:0/3126852617 shutdown_connections 2024-05-10T05:37:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.202+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/3126852617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28ec0 0x55b5dff2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.202+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/3126852617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff276a0 0x55b5dff27af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.202+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/3126852617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 0x55b5dff28980 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.202+0000 7f5b02f57b80 1 -- 172.21.15.131:0/3126852617 >> 172.21.15.131:0/3126852617 conn(0x55b5dfe89a60 msgr2=0x55b5dff21fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:17.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.202+0000 7f5b02f57b80 1 -- 172.21.15.131:0/3126852617 shutdown_connections 2024-05-10T05:37:17.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.202+0000 7f5b02f57b80 1 -- 172.21.15.131:0/3126852617 wait complete. 2024-05-10T05:37:17.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.204+0000 7f5b02f57b80 1 -- start start 2024-05-10T05:37:17.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.205+0000 7f5b02f57b80 1 Processor -- start 2024-05-10T05:37:17.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.208+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff31720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.208+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28570 0x55b5dff31c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.208+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28ec0 0x55b5dff38bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.208+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dfde3560 con 0x55b5dff28ec0 2024-05-10T05:37:17.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.208+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dfde3200 con 0x55b5dff276a0 2024-05-10T05:37:17.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.208+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dfcca6a0 con 0x55b5dff28570 2024-05-10T05:37:17.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.208+0000 7f5b02f57b80 10 client.0 _ll_register_callbacks cb 0x55b5dfdecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:17.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.209+0000 7f5ae3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff31720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.209+0000 7f5ae3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28570 0x55b5dff31c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.209+0000 7f5ae3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28570 0x55b5dff31c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59498/0 (socket says 172.21.15.131:59498) 2024-05-10T05:37:17.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.209+0000 7f5ae3400700 1 -- 172.21.15.131:0/2270757736 learned_addr learned my addr 172.21.15.131:0/2270757736 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:17.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.210+0000 7f5ae9000700 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28ec0 0x55b5dff38bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.210+0000 7f5ae3e00700 1 -- 172.21.15.131:0/2270757736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28570 msgr2=0x55b5dff31c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:17.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.210+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28570 0x55b5dff31c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.210+0000 7f5ae3e00700 1 -- 172.21.15.131:0/2270757736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28ec0 msgr2=0x55b5dff38bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:17.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.210+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28ec0 0x55b5dff38bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.210+0000 7f5ae3e00700 1 -- 172.21.15.131:0/2270757736 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae41131d0 con 0x55b5dff276a0 2024-05-10T05:37:17.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.210+0000 7f5ae9000700 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28ec0 0x55b5dff38bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:17.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.212+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff31720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4005950 tx=0x7f5ad40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d654609f21e0ec1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:17.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.212+0000 7f5ace000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:37:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.212+0000 7f5ace000700 1 -- 172.21.15.131:0/2270757736 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad401c5f0 con 0x55b5dff276a0 2024-05-10T05:37:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.212+0000 7f5ace000700 1 -- 172.21.15.131:0/2270757736 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5ad4017040 con 0x55b5dff276a0 2024-05-10T05:37:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.212+0000 7f5b02f57b80 1 -- 172.21.15.131:0/2270757736 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5dfda0270 con 0x55b5dff276a0 2024-05-10T05:37:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.213+0000 7f5ace000700 1 -- 172.21.15.131:0/2270757736 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad400fe40 con 0x55b5dff276a0 2024-05-10T05:37:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.213+0000 7f5b02f57b80 10 client.26060 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:17.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.213+0000 7f5b02f57b80 1 -- 172.21.15.131:0/2270757736 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b5dfc74b60 con 0x55b5dff276a0 2024-05-10T05:37:17.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.213+0000 7f5ace000700 1 -- 172.21.15.131:0/2270757736 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5dfda0270 con 0x55b5dff276a0 2024-05-10T05:37:17.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.213+0000 7f5ace000700 1 -- 172.21.15.131:0/2270757736 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5ad400e940 con 0x55b5dff276a0 2024-05-10T05:37:17.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.214+0000 7f5b02f57b80 10 client.26060 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:17.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.214+0000 7f5b02f57b80 10 client.26060 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:37:17.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.214+0000 7f5b02f57b80 20 client.26060 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:17.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.214+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b5dff3d370 0x55b5dff3f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.214+0000 7f5b02f57b80 4 client.26060 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:17.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.214+0000 7f5b02f57b80 1 -- 172.21.15.131:0/2270757736 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5dfef6780 con 0x55b5dff3d370 2024-05-10T05:37:17.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.214+0000 7f5ae3400700 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b5dff3d370 0x55b5dff3f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.215+0000 7f5ace000700 10 client.26060 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:37:17.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.215+0000 7f5ae3400700 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b5dff3d370 0x55b5dff3f7c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f744fbb7c6e5d64 server_cookie=98e2244069ede482 in_seq=0 out_seq=0 2024-05-10T05:37:17.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5ace000700 1 -- 172.21.15.131:0/2270757736 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5dfef6780 con 0x55b5dff3d370 2024-05-10T05:37:17.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5ace000700 10 client.26060 handle_command_reply: tid=0 2024-05-10T05:37:17.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5b02f57b80 1 client.26060 shutdown 2024-05-10T05:37:17.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5b02f57b80 1 -- 172.21.15.131:0/2270757736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 msgr2=0x55b5dff31720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff31720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4005950 tx=0x7f5ad40047c0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5b02f57b80 1 -- 172.21.15.131:0/2270757736 shutdown_connections 2024-05-10T05:37:17.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b5dff3d370 0x55b5dff3f7c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28ec0 0x55b5dff38bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff31720 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/2270757736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28570 0x55b5dff31c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5b02f57b80 1 -- 172.21.15.131:0/2270757736 >> 172.21.15.131:0/2270757736 conn(0x55b5dfe89a60 msgr2=0x55b5dff21fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:17.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.216+0000 7f5ace000700 0 client.26060 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:37:17.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.217+0000 7f5b02f57b80 1 -- 172.21.15.131:0/2270757736 shutdown_connections 2024-05-10T05:37:17.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.217+0000 7f5b02f57b80 1 -- 172.21.15.131:0/2270757736 wait complete. 2024-05-10T05:37:17.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.217+0000 7f5b02f57b80 20 client.26060 trim_cache size 0 max 16384 2024-05-10T05:37:17.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.232+0000 7f5b02f57b80 1 Processor -- start 2024-05-10T05:37:17.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.232+0000 7f5b02f57b80 1 -- start start 2024-05-10T05:37:17.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.233+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff3fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.233+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28570 0x55b5dff38b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.233+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff2fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.233+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dff22ad0 con 0x55b5dff28570 2024-05-10T05:37:17.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.233+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dff1c7c0 con 0x55b5dff276a0 2024-05-10T05:37:17.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.233+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dff2f6f0 con 0x55b5dff28ec0 2024-05-10T05:37:17.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.233+0000 7f5ae3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff3fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff3fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56292/0 (socket says 172.21.15.131:56292) 2024-05-10T05:37:17.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae3e00700 1 -- 172.21.15.131:0/340740568 learned_addr learned my addr 172.21.15.131:0/340740568 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:17.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae9000700 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff2fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae3e00700 1 -- 172.21.15.131:0/340740568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 msgr2=0x55b5dff2fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:17.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae3400700 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28570 0x55b5dff38b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff2fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae3e00700 1 -- 172.21.15.131:0/340740568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28570 msgr2=0x55b5dff38b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:17.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28570 0x55b5dff38b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae3e00700 1 -- 172.21.15.131:0/340740568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5dfe0d560 con 0x55b5dff276a0 2024-05-10T05:37:17.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae3400700 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28570 0x55b5dff38b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:17.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae9000700 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff2fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:37:17.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.234+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff3fa90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4011040 tx=0x7f5ad400eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f6ac3c5d732562 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:17.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.235+0000 7f5ada000700 1 -- 172.21.15.131:0/340740568 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4004020 con 0x55b5dff276a0 2024-05-10T05:37:17.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.235+0000 7f5ada000700 1 -- 172.21.15.131:0/340740568 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ad4004830 con 0x55b5dff276a0 2024-05-10T05:37:17.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.235+0000 7f5ada000700 1 -- 172.21.15.131:0/340740568 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4005240 con 0x55b5dff276a0 2024-05-10T05:37:17.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.236+0000 7f5b02f57b80 1 -- 172.21.15.131:0/340740568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 msgr2=0x55b5dff3fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.236+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff3fa90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4011040 tx=0x7f5ad400eec0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.236+0000 7f5b02f57b80 1 -- 172.21.15.131:0/340740568 shutdown_connections 2024-05-10T05:37:17.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.236+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff2fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.236+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff276a0 0x55b5dff3fa90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.236+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/340740568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff28570 0x55b5dff38b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.236+0000 7f5b02f57b80 1 -- 172.21.15.131:0/340740568 >> 172.21.15.131:0/340740568 conn(0x55b5dfe89a60 msgr2=0x55b5dff3f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:17.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.236+0000 7f5b02f57b80 1 -- 172.21.15.131:0/340740568 shutdown_connections 2024-05-10T05:37:17.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.237+0000 7f5b02f57b80 1 -- 172.21.15.131:0/340740568 wait complete. 2024-05-10T05:37:17.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.237+0000 7f5b02f57b80 1 -- start start 2024-05-10T05:37:17.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.238+0000 7f5b02f57b80 1 Processor -- start 2024-05-10T05:37:17.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.239+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff276a0 0x55b5dff35ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.239+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 0x55b5dff36400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.239+0000 7f5b02f57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.239+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dff376e0 con 0x55b5dff276a0 2024-05-10T05:37:17.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.239+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dff37850 con 0x55b5dff28570 2024-05-10T05:37:17.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.239+0000 7f5b02f57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dff37a10 con 0x55b5dff28ec0 2024-05-10T05:37:17.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.239+0000 7f5b02f57b80 10 client.0 _ll_register_callbacks cb 0x55b5dfdecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:17.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.239+0000 7f5ae9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.239+0000 7f5ae9000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff371a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59520/0 (socket says 172.21.15.131:59520) 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.239+0000 7f5ae9000700 1 -- 172.21.15.131:0/47414120 learned_addr learned my addr 172.21.15.131:0/47414120 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.240+0000 7f5ae3400700 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 0x55b5dff36400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.240+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff276a0 0x55b5dff35ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.240+0000 7f5ae9000700 1 -- 172.21.15.131:0/47414120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 msgr2=0x55b5dff36400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.240+0000 7f5ae9000700 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 0x55b5dff36400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.240+0000 7f5ae9000700 1 -- 172.21.15.131:0/47414120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff276a0 msgr2=0x55b5dff35ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.240+0000 7f5ae9000700 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff276a0 0x55b5dff35ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.240+0000 7f5ae9000700 1 -- 172.21.15.131:0/47414120 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5dfc74b60 con 0x55b5dff28ec0 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.241+0000 7f5ae9000700 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff371a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0962b0 tx=0x7f5adc01b600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf18b45c8172dd28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:17.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.241+0000 7f5ace000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:37:17.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.241+0000 7f5ace000700 1 -- 172.21.15.131:0/47414120 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc099f40 con 0x55b5dff28ec0 2024-05-10T05:37:17.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.241+0000 7f5ace000700 1 -- 172.21.15.131:0/47414120 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5adc099840 con 0x55b5dff28ec0 2024-05-10T05:37:17.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.241+0000 7f5b02f57b80 1 -- 172.21.15.131:0/47414120 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ad000bbf0 con 0x55b5dff28ec0 2024-05-10T05:37:17.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5ace000700 1 -- 172.21.15.131:0/47414120 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5adc09d670 con 0x55b5dff28ec0 2024-05-10T05:37:17.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5b02f57b80 10 client.26086 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:17.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5b02f57b80 1 -- 172.21.15.131:0/47414120 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b5dfccccd0 con 0x55b5dff28ec0 2024-05-10T05:37:17.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5ace000700 1 -- 172.21.15.131:0/47414120 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ad000bbf0 con 0x55b5dff28ec0 2024-05-10T05:37:17.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5ace000700 1 -- 172.21.15.131:0/47414120 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5adc0a17a0 con 0x55b5dff28ec0 2024-05-10T05:37:17.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5b02f57b80 10 client.26086 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5b02f57b80 10 client.26086 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5b02f57b80 20 client.26086 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b5dff939c0 0x55b5dff95de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5b02f57b80 4 client.26086 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.242+0000 7f5b02f57b80 1 -- 172.21.15.131:0/47414120 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b5dfe3fec0 con 0x55b5dff939c0 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.243+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b5dff939c0 0x55b5dff95de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.243+0000 7f5ace000700 10 client.26086 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.244+0000 7f5ae3e00700 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b5dff939c0 0x55b5dff95de0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46a4a3f42b7e8f97 server_cookie=682f6a9761420968 in_seq=0 out_seq=0 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.244+0000 7f5ace000700 1 -- 172.21.15.131:0/47414120 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b5dfe3fec0 con 0x55b5dff939c0 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.244+0000 7f5ace000700 10 client.26086 handle_command_reply: tid=0 2024-05-10T05:37:17.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.244+0000 7f5b02f57b80 1 client.26086 shutdown 2024-05-10T05:37:17.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5b02f57b80 1 -- 172.21.15.131:0/47414120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 msgr2=0x55b5dff371a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff371a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0962b0 tx=0x7f5adc01b600 comp rx=0 tx=0).stop 2024-05-10T05:37:17.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5b02f57b80 1 -- 172.21.15.131:0/47414120 shutdown_connections 2024-05-10T05:37:17.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55b5dff939c0 0x55b5dff95de0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b5dff276a0 0x55b5dff35ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5dff28ec0 0x55b5dff371a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5b02f57b80 1 --2- 172.21.15.131:0/47414120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5dff28570 0x55b5dff36400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5b02f57b80 1 -- 172.21.15.131:0/47414120 >> 172.21.15.131:0/47414120 conn(0x55b5dfe89a60 msgr2=0x55b5dff2f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:17.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5ace000700 0 client.26086 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:37:17.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5b02f57b80 1 -- 172.21.15.131:0/47414120 shutdown_connections 2024-05-10T05:37:17.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.246+0000 7f5b02f57b80 1 -- 172.21.15.131:0/47414120 wait complete. 2024-05-10T05:37:17.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.247+0000 7f5b02f57b80 20 client.26086 trim_cache size 0 max 16384 2024-05-10T05:37:17.249 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:37:17.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.248+0000 7f5aeb000700 1 -- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5ac80a1310 msgr2=0x7f5ac80a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.248+0000 7f5aeb000700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5ac80a1310 0x7f5ac80a3820 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5ae41095f0 tx=0x7f5ad4005c10 comp rx=0 tx=0).stop 2024-05-10T05:37:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.248+0000 7f5aeb000700 1 -- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae4110320 msgr2=0x7f5ae41a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.248+0000 7f5aeb000700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae4110320 0x7f5ae41a5a00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00efc0 tx=0x7f5adc00c3a0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.248+0000 7f5aeb000700 1 -- 172.21.15.131:0/3113244267 shutdown_connections 2024-05-10T05:37:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.248+0000 7f5aeb000700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5ac80a1310 0x7f5ac80a3820 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.248+0000 7f5aeb000700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae4110320 0x7f5ae41a5a00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.248+0000 7f5aeb000700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae4108210 0x7f5ae419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.248+0000 7f5aeb000700 1 --2- 172.21.15.131:0/3113244267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ae4108b60 0x7f5ae419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.248+0000 7f5aeb000700 1 -- 172.21.15.131:0/3113244267 >> 172.21.15.131:0/3113244267 conn(0x7f5ae406ea00 msgr2=0x7f5ae410bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.249+0000 7f5aeb000700 1 -- 172.21.15.131:0/3113244267 shutdown_connections 2024-05-10T05:37:17.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.249+0000 7f5aeb000700 1 -- 172.21.15.131:0/3113244267 wait complete. 2024-05-10T05:37:17.283 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:37:17.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.883+0000 7f8587e00700 1 -- 172.21.15.131:0/1016729236 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858000b810 con 0x7f8588077620 2024-05-10T05:37:17.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.892+0000 7f858ec00700 1 -- 172.21.15.131:0/1016729236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8588077620 msgr2=0x7f8588077f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.892+0000 7f858ec00700 1 --2- 172.21.15.131:0/1016729236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8588077620 0x7f8588077f10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f85800098f0 tx=0x7f8580009c00 comp rx=0 tx=0).stop 2024-05-10T05:37:17.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.892+0000 7f858ec00700 1 -- 172.21.15.131:0/1016729236 shutdown_connections 2024-05-10T05:37:17.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.892+0000 7f858ec00700 1 --2- 172.21.15.131:0/1016729236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8588077620 0x7f8588077f10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.892+0000 7f858ec00700 1 --2- 172.21.15.131:0/1016729236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8588075b60 0x7f8588075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.892+0000 7f858ec00700 1 --2- 172.21.15.131:0/1016729236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8588076cd0 0x7f85880770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.892+0000 7f858ec00700 1 -- 172.21.15.131:0/1016729236 >> 172.21.15.131:0/1016729236 conn(0x7f85880ff220 msgr2=0x7f85881016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:17.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.895+0000 7f858ec00700 1 -- 172.21.15.131:0/1016729236 shutdown_connections 2024-05-10T05:37:17.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.895+0000 7f858ec00700 1 -- 172.21.15.131:0/1016729236 wait complete. 2024-05-10T05:37:17.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.898+0000 7f858ec00700 1 Processor -- start 2024-05-10T05:37:17.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.904+0000 7f858ec00700 1 -- start start 2024-05-10T05:37:17.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.904+0000 7f858ec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8588075b60 0x7f858810e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.905+0000 7f858ec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8588076cd0 0x7f858810ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.905+0000 7f858ec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8588077620 0x7f85881ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.905+0000 7f858ec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858810f250 con 0x7f8588076cd0 2024-05-10T05:37:17.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.905+0000 7f858ec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858810f390 con 0x7f8588077620 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.905+0000 7f858ec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858806ac60 con 0x7f8588075b60 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.905+0000 7f858e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8588077620 0x7f85881ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.905+0000 7f858e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8588077620 0x7f85881ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56310/0 (socket says 172.21.15.131:56310) 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.905+0000 7f858e200700 1 -- 172.21.15.131:0/900349800 learned_addr learned my addr 172.21.15.131:0/900349800 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.906+0000 7f858d800700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8588075b60 0x7f858810e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.906+0000 7f858e200700 1 -- 172.21.15.131:0/900349800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8588075b60 msgr2=0x7f858810e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.906+0000 7f858e200700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8588075b60 0x7f858810e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.906+0000 7f858e200700 1 -- 172.21.15.131:0/900349800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8588076cd0 msgr2=0x7f858810ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.906+0000 7f858e200700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8588076cd0 0x7f858810ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.906+0000 7f858e200700 1 -- 172.21.15.131:0/900349800 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8580009520 con 0x7f8588077620 2024-05-10T05:37:17.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.907+0000 7f858e200700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8588077620 0x7f85881ae140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f858000c010 tx=0x7f858000b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.908+0000 7f8586000700 1 -- 172.21.15.131:0/900349800 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580005450 con 0x7f8588077620 2024-05-10T05:37:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.908+0000 7f8586000700 1 -- 172.21.15.131:0/900349800 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f858000bcc0 con 0x7f8588077620 2024-05-10T05:37:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.908+0000 7f8586000700 1 -- 172.21.15.131:0/900349800 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580017860 con 0x7f8588077620 2024-05-10T05:37:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.908+0000 7f858ec00700 1 -- 172.21.15.131:0/900349800 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858806aee0 con 0x7f8588077620 2024-05-10T05:37:17.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.908+0000 7f858ec00700 1 -- 172.21.15.131:0/900349800 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858806b340 con 0x7f8588077620 2024-05-10T05:37:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.927+0000 7f8586000700 1 -- 172.21.15.131:0/900349800 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8580003680 con 0x7f8588077620 2024-05-10T05:37:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.929+0000 7f8586000700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85700a12f0 0x7f85700a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.929+0000 7f8586000700 1 -- 172.21.15.131:0/900349800 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f85800cdc00 con 0x7f8588077620 2024-05-10T05:37:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.939+0000 7f858d800700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85700a12f0 0x7f85700a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.939+0000 7f85a813ab80 1 Processor -- start 2024-05-10T05:37:17.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.940+0000 7f85a813ab80 1 -- start start 2024-05-10T05:37:17.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.940+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570e530 0x55626570e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.940+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55626570f7a0 0x55626570fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.943+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562657100f0 0x5562657169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.943+0000 7f85a813ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556265635150 con 0x55626570f7a0 2024-05-10T05:37:17.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.943+0000 7f85a813ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556265632b10 con 0x55626570e530 2024-05-10T05:37:17.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.943+0000 7f85a813ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626562e400 con 0x5562657100f0 2024-05-10T05:37:17.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.944+0000 7f858d800700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85700a12f0 0x7f85700a3800 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8588075850 tx=0x7f857c009040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:17.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.944+0000 7f858d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570e530 0x55626570e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.944+0000 7f858d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570e530 0x55626570e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56320/0 (socket says 172.21.15.131:56320) 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.944+0000 7f858d800700 1 -- 172.21.15.131:0/4081252102 learned_addr learned my addr 172.21.15.131:0/4081252102 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.945+0000 7f858d800700 1 -- 172.21.15.131:0/4081252102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562657100f0 msgr2=0x5562657169f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.945+0000 7f858d800700 1 --2- 172.21.15.131:0/4081252102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562657100f0 0x5562657169f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.945+0000 7f858d800700 1 -- 172.21.15.131:0/4081252102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55626570f7a0 msgr2=0x55626570fbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.948+0000 7f858ce00700 1 --2- 172.21.15.131:0/4081252102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55626570f7a0 0x55626570fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.948+0000 7f858d800700 1 --2- 172.21.15.131:0/4081252102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55626570f7a0 0x55626570fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.948+0000 7f858d800700 1 -- 172.21.15.131:0/4081252102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562654af3b0 con 0x55626570e530 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.948+0000 7f858ce00700 1 --2- 172.21.15.131:0/4081252102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55626570f7a0 0x55626570fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.948+0000 7f858d800700 1 --2- 172.21.15.131:0/4081252102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570e530 0x55626570e960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f857c016470 tx=0x7f857c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83923976dbae9241 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.948+0000 7f857aa00700 1 -- 172.21.15.131:0/4081252102 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c021600 con 0x55626570e530 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.948+0000 7f857aa00700 1 -- 172.21.15.131:0/4081252102 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f857c021c20 con 0x55626570e530 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.948+0000 7f857aa00700 1 -- 172.21.15.131:0/4081252102 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c014b90 con 0x55626570e530 2024-05-10T05:37:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.951+0000 7f85a813ab80 1 -- 172.21.15.131:0/4081252102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570e530 msgr2=0x55626570e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:17.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.951+0000 7f85a813ab80 1 --2- 172.21.15.131:0/4081252102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570e530 0x55626570e960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f857c016470 tx=0x7f857c012000 comp rx=0 tx=0).stop 2024-05-10T05:37:17.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.964+0000 7f85a813ab80 1 -- 172.21.15.131:0/4081252102 shutdown_connections 2024-05-10T05:37:17.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.964+0000 7f85a813ab80 1 --2- 172.21.15.131:0/4081252102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562657100f0 0x5562657169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.964+0000 7f85a813ab80 1 --2- 172.21.15.131:0/4081252102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570e530 0x55626570e960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.964+0000 7f85a813ab80 1 --2- 172.21.15.131:0/4081252102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55626570f7a0 0x55626570fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.964+0000 7f85a813ab80 1 -- 172.21.15.131:0/4081252102 >> 172.21.15.131:0/4081252102 conn(0x556265670a60 msgr2=0x556265709330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:17.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.964+0000 7f85a813ab80 1 -- 172.21.15.131:0/4081252102 shutdown_connections 2024-05-10T05:37:17.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.964+0000 7f85a813ab80 1 -- 172.21.15.131:0/4081252102 wait complete. 2024-05-10T05:37:17.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.968+0000 7f85a813ab80 1 -- start start 2024-05-10T05:37:17.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.968+0000 7f85a813ab80 1 Processor -- start 2024-05-10T05:37:17.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.976+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570e530 0x556265718980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.977+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 0x556265718ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.977+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562657100f0 0x55626571fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.977+0000 7f85a813ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562655ca560 con 0x5562657100f0 2024-05-10T05:37:17.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.977+0000 7f85a813ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562655ca200 con 0x55626570f7a0 2024-05-10T05:37:17.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.977+0000 7f85a813ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562654b16a0 con 0x55626570e530 2024-05-10T05:37:17.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.977+0000 7f85a813ab80 10 client.0 _ll_register_callbacks cb 0x5562655d3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:17.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.978+0000 7f858d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570e530 0x556265718980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:17.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.978+0000 7f858d800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570e530 0x556265718980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59548/0 (socket says 172.21.15.131:59548) 2024-05-10T05:37:17.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.978+0000 7f858d800700 1 -- 172.21.15.131:0/384846409 learned_addr learned my addr 172.21.15.131:0/384846409 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:17.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.981+0000 7f858d800700 1 -- 172.21.15.131:0/384846409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 msgr2=0x556265718ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:17.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.981+0000 7f858d800700 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 0x556265718ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.981+0000 7f858d800700 1 -- 172.21.15.131:0/384846409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562657100f0 msgr2=0x55626571fe20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:17.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.981+0000 7f858d800700 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562657100f0 0x55626571fe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:17.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.981+0000 7f858d800700 1 -- 172.21.15.131:0/384846409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c009dc0 con 0x55626570e530 2024-05-10T05:37:17.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.982+0000 7f856f400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:37:17.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.982+0000 7f858d800700 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570e530 0x556265718980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f857c014660 tx=0x7f857c014f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df03bae0dfb89d49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:17.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.984+0000 7f856f400700 1 -- 172.21.15.131:0/384846409 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c004c50 con 0x55626570e530 2024-05-10T05:37:17.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.985+0000 7f856f400700 1 -- 172.21.15.131:0/384846409 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f857c004db0 con 0x55626570e530 2024-05-10T05:37:17.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.985+0000 7f85a813ab80 1 -- 172.21.15.131:0/384846409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556265587270 con 0x55626570e530 2024-05-10T05:37:17.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.986+0000 7f856f400700 1 -- 172.21.15.131:0/384846409 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f857c014c30 con 0x55626570e530 2024-05-10T05:37:17.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.986+0000 7f85a813ab80 10 client.26095 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:17.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.986+0000 7f85a813ab80 1 -- 172.21.15.131:0/384846409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55626545bb60 con 0x55626570e530 2024-05-10T05:37:17.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.987+0000 7f856f400700 1 -- 172.21.15.131:0/384846409 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f857c005020 con 0x55626570e530 2024-05-10T05:37:17.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.987+0000 7f856f400700 1 -- 172.21.15.131:0/384846409 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f857c02b410 con 0x55626570e530 2024-05-10T05:37:17.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.990+0000 7f85a813ab80 10 client.26095 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:17.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.990+0000 7f85a813ab80 10 client.26095 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:37:17.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.990+0000 7f85a813ab80 20 client.26095 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:17.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.990+0000 7f85a813ab80 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x556265724510 0x556265726930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:17.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.992+0000 7f85a813ab80 4 client.26095 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:17.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.992+0000 7f85a813ab80 1 -- 172.21.15.131:0/384846409 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556265587270 con 0x556265724510 2024-05-10T05:37:17.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.992+0000 7f858ce00700 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x556265724510 0x556265726930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:17.996+0000 7f856f400700 10 client.26095 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:37:18.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.005+0000 7f858ce00700 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x556265724510 0x556265726930 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90d6ad7338d25c5a server_cookie=fe23038460efa89e in_seq=0 out_seq=0 2024-05-10T05:37:18.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.006+0000 7f856f400700 1 -- 172.21.15.131:0/384846409 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556265587270 con 0x556265724510 2024-05-10T05:37:18.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.006+0000 7f856f400700 10 client.26095 handle_command_reply: tid=0 2024-05-10T05:37:18.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.006+0000 7f85a813ab80 1 client.26095 shutdown 2024-05-10T05:37:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.020+0000 7f85a813ab80 1 -- 172.21.15.131:0/384846409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570e530 msgr2=0x556265718980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.020+0000 7f85a813ab80 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570e530 0x556265718980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f857c014660 tx=0x7f857c014f00 comp rx=0 tx=0).stop 2024-05-10T05:37:18.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.021+0000 7f85a813ab80 1 -- 172.21.15.131:0/384846409 shutdown_connections 2024-05-10T05:37:18.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.021+0000 7f85a813ab80 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x556265724510 0x556265726930 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.021+0000 7f85a813ab80 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562657100f0 0x55626571fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.021+0000 7f85a813ab80 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570e530 0x556265718980 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f857c014660 tx=0x7f857c014f00 comp rx=0 tx=0).stop 2024-05-10T05:37:18.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.021+0000 7f85a813ab80 1 --2- 172.21.15.131:0/384846409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 0x556265718ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.021+0000 7f85a813ab80 1 -- 172.21.15.131:0/384846409 >> 172.21.15.131:0/384846409 conn(0x556265670a60 msgr2=0x55626570d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:18.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.021+0000 7f856f400700 0 client.26095 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:37:18.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.024+0000 7f85a813ab80 1 -- 172.21.15.131:0/384846409 shutdown_connections 2024-05-10T05:37:18.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.024+0000 7f85a813ab80 1 -- 172.21.15.131:0/384846409 wait complete. 2024-05-10T05:37:18.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.024+0000 7f85a813ab80 20 client.26095 trim_cache size 0 max 16384 2024-05-10T05:37:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.064+0000 7f85a813ab80 1 Processor -- start 2024-05-10T05:37:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.065+0000 7f85a813ab80 1 -- start start 2024-05-10T05:37:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.066+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 0x5562657203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.067+0000 7f858d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 0x5562657203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.067+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562657100f0 0x55626571fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:18.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.067+0000 7f858d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 0x5562657203b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56334/0 (socket says 172.21.15.131:56334) 2024-05-10T05:37:18.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.067+0000 7f858d800700 1 -- 172.21.15.131:0/2244501266 learned_addr learned my addr 172.21.15.131:0/2244501266 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:18.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.068+0000 7f858ce00700 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562657100f0 0x55626571fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.068+0000 7f85a813ab80 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556265716ca0 0x5562657170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:18.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.068+0000 7f858e200700 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556265716ca0 0x5562657170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.068+0000 7f85a813ab80 1 -- 172.21.15.131:0/2244501266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556265726ec0 con 0x556265716ca0 2024-05-10T05:37:18.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.068+0000 7f85a813ab80 1 -- 172.21.15.131:0/2244501266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556265720f20 con 0x55626570f7a0 2024-05-10T05:37:18.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.069+0000 7f85a813ab80 1 -- 172.21.15.131:0/2244501266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562657098a0 con 0x5562657100f0 2024-05-10T05:37:18.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.070+0000 7f858e200700 1 -- 172.21.15.131:0/2244501266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562657100f0 msgr2=0x55626571fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:18.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.070+0000 7f858e200700 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562657100f0 0x55626571fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.070+0000 7f858e200700 1 -- 172.21.15.131:0/2244501266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 msgr2=0x5562657203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:18.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.070+0000 7f858e200700 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 0x5562657203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.070+0000 7f858e200700 1 -- 172.21.15.131:0/2244501266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562655f4560 con 0x556265716ca0 2024-05-10T05:37:18.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.071+0000 7f858e200700 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556265716ca0 0x5562657170b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f858009b010 tx=0x7f858000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77be5ec107b9754 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:18.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.071+0000 7f858d800700 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 0x5562657203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:18.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.072+0000 7f857b400700 1 -- 172.21.15.131:0/2244501266 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858009b080 con 0x556265716ca0 2024-05-10T05:37:18.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.072+0000 7f857b400700 1 -- 172.21.15.131:0/2244501266 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f858009eeb0 con 0x556265716ca0 2024-05-10T05:37:18.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.072+0000 7f857b400700 1 -- 172.21.15.131:0/2244501266 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858009b2e0 con 0x556265716ca0 2024-05-10T05:37:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.073+0000 7f85a813ab80 1 -- 172.21.15.131:0/2244501266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556265716ca0 msgr2=0x5562657170b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.073+0000 7f85a813ab80 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556265716ca0 0x5562657170b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f858009b010 tx=0x7f858000bfa0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.074+0000 7f85a813ab80 1 -- 172.21.15.131:0/2244501266 shutdown_connections 2024-05-10T05:37:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.074+0000 7f85a813ab80 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556265716ca0 0x5562657170b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.074+0000 7f85a813ab80 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626570f7a0 0x5562657203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.074+0000 7f85a813ab80 1 --2- 172.21.15.131:0/2244501266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562657100f0 0x55626571fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.074+0000 7f85a813ab80 1 -- 172.21.15.131:0/2244501266 >> 172.21.15.131:0/2244501266 conn(0x556265670a60 msgr2=0x556265709190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:18.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.074+0000 7f85a813ab80 1 -- 172.21.15.131:0/2244501266 shutdown_connections 2024-05-10T05:37:18.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.074+0000 7f85a813ab80 1 -- 172.21.15.131:0/2244501266 wait complete. 2024-05-10T05:37:18.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.075+0000 7f85a813ab80 1 -- start start 2024-05-10T05:37:18.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.075+0000 7f85a813ab80 1 Processor -- start 2024-05-10T05:37:18.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.076+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570f7a0 0x55626571dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:18.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.076+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562657100f0 0x55626571e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:18.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.076+0000 7f85a813ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556265716ca0 0x55626571f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:18.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.076+0000 7f85a813ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626571f680 con 0x5562657100f0 2024-05-10T05:37:18.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.076+0000 7f85a813ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626571f7f0 con 0x556265716ca0 2024-05-10T05:37:18.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.076+0000 7f85a813ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626571f9b0 con 0x55626570f7a0 2024-05-10T05:37:18.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.076+0000 7f85a813ab80 10 client.0 _ll_register_callbacks cb 0x5562655d3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.077+0000 7f858e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556265716ca0 0x55626571f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.077+0000 7f858e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556265716ca0 0x55626571f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56344/0 (socket says 172.21.15.131:56344) 2024-05-10T05:37:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.077+0000 7f858e200700 1 -- 172.21.15.131:0/1413759256 learned_addr learned my addr 172.21.15.131:0/1413759256 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.077+0000 7f858d800700 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570f7a0 0x55626571dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.078+0000 7f858ce00700 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562657100f0 0x55626571e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.078+0000 7f858e200700 1 -- 172.21.15.131:0/1413759256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570f7a0 msgr2=0x55626571dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:18.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.078+0000 7f858e200700 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570f7a0 0x55626571dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.078+0000 7f858e200700 1 -- 172.21.15.131:0/1413759256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562657100f0 msgr2=0x55626571e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:18.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.078+0000 7f858e200700 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562657100f0 0x55626571e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.078+0000 7f858e200700 1 -- 172.21.15.131:0/1413759256 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858806b340 con 0x556265716ca0 2024-05-10T05:37:18.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.078+0000 7f858d800700 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570f7a0 0x55626571dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:18.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.079+0000 7f858e200700 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556265716ca0 0x55626571f140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f858009ad80 tx=0x7f8580003ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f061559bcaf55bcd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:18.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.079+0000 7f856f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:37:18.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.079+0000 7f856f400700 1 -- 172.21.15.131:0/1413759256 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858009b080 con 0x556265716ca0 2024-05-10T05:37:18.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.080+0000 7f85a813ab80 1 -- 172.21.15.131:0/1413759256 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f857c021c20 con 0x556265716ca0 2024-05-10T05:37:18.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.079+0000 7f856f400700 1 -- 172.21.15.131:0/1413759256 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f858009b2e0 con 0x556265716ca0 2024-05-10T05:37:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.080+0000 7f856f400700 1 -- 172.21.15.131:0/1413759256 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f858009b4a0 con 0x556265716ca0 2024-05-10T05:37:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.081+0000 7f85a813ab80 10 client.26078 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.081+0000 7f85a813ab80 1 -- 172.21.15.131:0/1413759256 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5562654b3cd0 con 0x556265716ca0 2024-05-10T05:37:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.081+0000 7f856f400700 1 -- 172.21.15.131:0/1413759256 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85800a9080 con 0x556265716ca0 2024-05-10T05:37:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.081+0000 7f856f400700 1 -- 172.21.15.131:0/1413759256 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f85800cbb60 con 0x556265716ca0 2024-05-10T05:37:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.081+0000 7f85a813ab80 10 client.26078 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:18.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.081+0000 7f85a813ab80 10 client.26078 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:37:18.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.081+0000 7f85a813ab80 20 client.26078 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:18.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.081+0000 7f85a813ab80 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55626577b2c0 0x55626577d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:18.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.081+0000 7f85a813ab80 4 client.26078 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:37:18.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.081+0000 7f85a813ab80 1 -- 172.21.15.131:0/1413759256 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5562656dd780 con 0x55626577b2c0 2024-05-10T05:37:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.083+0000 7f858d800700 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55626577b2c0 0x55626577d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.083+0000 7f858d800700 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55626577b2c0 0x55626577d6e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=726915d958905af9 server_cookie=b9bba7da3726205e in_seq=0 out_seq=0 2024-05-10T05:37:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.083+0000 7f856f400700 10 client.26078 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:37:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f856f400700 1 -- 172.21.15.131:0/1413759256 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5562656dd780 con 0x55626577b2c0 2024-05-10T05:37:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f856f400700 10 client.26078 handle_command_reply: tid=0 2024-05-10T05:37:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f85a813ab80 1 client.26078 shutdown 2024-05-10T05:37:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f85a813ab80 1 -- 172.21.15.131:0/1413759256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556265716ca0 msgr2=0x55626571f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f85a813ab80 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556265716ca0 0x55626571f140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f858009ad80 tx=0x7f8580003ee0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f85a813ab80 1 -- 172.21.15.131:0/1413759256 shutdown_connections 2024-05-10T05:37:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f85a813ab80 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55626577b2c0 0x55626577d6e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f85a813ab80 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556265716ca0 0x55626571f140 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f85a813ab80 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55626570f7a0 0x55626571dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f85a813ab80 1 --2- 172.21.15.131:0/1413759256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562657100f0 0x55626571e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f85a813ab80 1 -- 172.21.15.131:0/1413759256 >> 172.21.15.131:0/1413759256 conn(0x556265670a60 msgr2=0x556265709170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.084+0000 7f856f400700 0 client.26078 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:37:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.085+0000 7f85a813ab80 1 -- 172.21.15.131:0/1413759256 shutdown_connections 2024-05-10T05:37:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.085+0000 7f85a813ab80 1 -- 172.21.15.131:0/1413759256 wait complete. 2024-05-10T05:37:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.085+0000 7f85a813ab80 20 client.26078 trim_cache size 0 max 16384 2024-05-10T05:37:18.090 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:37:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.086+0000 7f858ec00700 1 -- 172.21.15.131:0/900349800 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85700a12f0 msgr2=0x7f85700a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.086+0000 7f858ec00700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85700a12f0 0x7f85700a3800 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8588075850 tx=0x7f857c009040 comp rx=0 tx=0).stop 2024-05-10T05:37:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.086+0000 7f858ec00700 1 -- 172.21.15.131:0/900349800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8588077620 msgr2=0x7f85881ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.086+0000 7f858ec00700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8588077620 0x7f85881ae140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f858000c010 tx=0x7f858000b9e0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.087+0000 7f858ec00700 1 -- 172.21.15.131:0/900349800 shutdown_connections 2024-05-10T05:37:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.087+0000 7f858ec00700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f85700a12f0 0x7f85700a3800 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.087+0000 7f858ec00700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8588077620 0x7f85881ae140 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.087+0000 7f858ec00700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8588075b60 0x7f858810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.087+0000 7f858ec00700 1 --2- 172.21.15.131:0/900349800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8588076cd0 0x7f858810ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.087+0000 7f858ec00700 1 -- 172.21.15.131:0/900349800 >> 172.21.15.131:0/900349800 conn(0x7f85880ff220 msgr2=0x7f8588101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.087+0000 7f858ec00700 1 -- 172.21.15.131:0/900349800 shutdown_connections 2024-05-10T05:37:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.087+0000 7f858ec00700 1 -- 172.21.15.131:0/900349800 wait complete. 2024-05-10T05:37:18.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:37:18.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:18 smithi131 ceph-mon[37264]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 228 MiB/s wr, 73 op/s 2024-05-10T05:37:18.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:18 smithi157 ceph-mon[78427]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 228 MiB/s wr, 73 op/s 2024-05-10T05:37:18.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:18 smithi142 ceph-mon[77667]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 228 MiB/s wr, 73 op/s 2024-05-10T05:37:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.900+0000 7f3ba3a00700 1 -- 172.21.15.131:0/4135574208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9c109300 msgr2=0x7f3b9c109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.900+0000 7f3ba3a00700 1 --2- 172.21.15.131:0/4135574208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9c109300 0x7f3b9c109710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3b8801a9a0 tx=0x7f3b8801acb0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.901+0000 7f3ba3a00700 1 -- 172.21.15.131:0/4135574208 shutdown_connections 2024-05-10T05:37:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.901+0000 7f3ba3a00700 1 --2- 172.21.15.131:0/4135574208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9c109c50 0x7f3b9c10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.901+0000 7f3ba3a00700 1 --2- 172.21.15.131:0/4135574208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c108630 0x7f3b9c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.901+0000 7f3ba3a00700 1 --2- 172.21.15.131:0/4135574208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9c109300 0x7f3b9c109710 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.901+0000 7f3ba3a00700 1 -- 172.21.15.131:0/4135574208 >> 172.21.15.131:0/4135574208 conn(0x7f3b9c06ea00 msgr2=0x7f3b9c070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:18.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.902+0000 7f3ba3a00700 1 -- 172.21.15.131:0/4135574208 shutdown_connections 2024-05-10T05:37:18.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.902+0000 7f3ba3a00700 1 -- 172.21.15.131:0/4135574208 wait complete. 2024-05-10T05:37:18.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.905+0000 7f3ba3a00700 1 Processor -- start 2024-05-10T05:37:18.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.914+0000 7f3ba3a00700 1 -- start start 2024-05-10T05:37:18.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.914+0000 7f3ba3a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c108630 0x7f3b9c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:18.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.915+0000 7f3ba3a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9c109300 0x7f3b9c19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:18.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.915+0000 7f3ba1000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c108630 0x7f3b9c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.916+0000 7f3ba1000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c108630 0x7f3b9c19e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59576/0 (socket says 172.21.15.131:59576) 2024-05-10T05:37:18.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.919+0000 7f3ba3a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9c109c50 0x7f3b9c1a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:18.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.919+0000 7f3ba3a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c1a5ff0 con 0x7f3b9c109c50 2024-05-10T05:37:18.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.919+0000 7f3ba3a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c1a6130 con 0x7f3b9c109300 2024-05-10T05:37:18.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.919+0000 7f3ba3a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c1a62c0 con 0x7f3b9c108630 2024-05-10T05:37:18.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.922+0000 7f3b9be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9c109300 0x7f3b9c19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.922+0000 7f3ba1000700 1 -- 172.21.15.131:0/3501457903 learned_addr learned my addr 172.21.15.131:0/3501457903 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:18.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.922+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9c109c50 0x7f3b9c1a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:18.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.922+0000 7f3ba1000700 1 -- 172.21.15.131:0/3501457903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9c109300 msgr2=0x7f3b9c19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:18.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.923+0000 7f3ba1000700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9c109300 0x7f3b9c19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.923+0000 7f3ba1000700 1 -- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9c109c50 msgr2=0x7f3b9c1a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:18.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.923+0000 7f3ba1000700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9c109c50 0x7f3b9c1a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:18.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.923+0000 7f3ba1000700 1 -- 172.21.15.131:0/3501457903 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8801a600 con 0x7f3b9c108630 2024-05-10T05:37:18.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.923+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9c109c50 0x7f3b9c1a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:37:18.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:18.923+0000 7f3b9be00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9c109300 0x7f3b9c19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:19.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.133+0000 7f3ba1000700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c108630 0x7f3b9c19e550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000b550 tx=0x7f3b9000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:19.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.133+0000 7f3b99600700 1 -- 172.21.15.131:0/3501457903 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b90011830 con 0x7f3b9c108630 2024-05-10T05:37:19.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.133+0000 7f3b99600700 1 -- 172.21.15.131:0/3501457903 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b90011e50 con 0x7f3b9c108630 2024-05-10T05:37:19.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.135+0000 7f3b99600700 1 -- 172.21.15.131:0/3501457903 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b90010e90 con 0x7f3b9c108630 2024-05-10T05:37:19.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.135+0000 7f3ba3a00700 1 -- 172.21.15.131:0/3501457903 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9c1a65a0 con 0x7f3b9c108630 2024-05-10T05:37:19.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.135+0000 7f3ba3a00700 1 -- 172.21.15.131:0/3501457903 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c1a6a30 con 0x7f3b9c108630 2024-05-10T05:37:19.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.141+0000 7f3b99600700 1 -- 172.21.15.131:0/3501457903 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3b90011990 con 0x7f3b9c108630 2024-05-10T05:37:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.144+0000 7f3b99600700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b840a1240 0x7f3b840a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.144+0000 7f3b99600700 1 -- 172.21.15.131:0/3501457903 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3b900cc3d0 con 0x7f3b9c108630 2024-05-10T05:37:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.145+0000 7f3b9be00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b840a1240 0x7f3b840a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.150+0000 7f3b9be00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b840a1240 0x7f3b840a3750 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3b88006010 tx=0x7f3b8801af10 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:19.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.151+0000 7f3bbb911b80 1 Processor -- start 2024-05-10T05:37:19.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.151+0000 7f3bbb911b80 1 -- start start 2024-05-10T05:37:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.152+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.152+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.152+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.152+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d64f3150 con 0x5627d65cdec0 2024-05-10T05:37:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.152+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d64f0b10 con 0x5627d65cd570 2024-05-10T05:37:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.152+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d64ec400 con 0x5627d65cc6a0 2024-05-10T05:37:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.153+0000 7f3ba1000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.153+0000 7f3ba1000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59588/0 (socket says 172.21.15.131:59588) 2024-05-10T05:37:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.153+0000 7f3ba1000700 1 -- 172.21.15.131:0/596085169 learned_addr learned my addr 172.21.15.131:0/596085169 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.153+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.153+0000 7f3b9be00700 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.154+0000 7f3ba1000700 1 -- 172.21.15.131:0/596085169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 msgr2=0x5627d65cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.154+0000 7f3ba1000700 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.154+0000 7f3ba1000700 1 -- 172.21.15.131:0/596085169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 msgr2=0x5627d65d47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.154+0000 7f3ba1000700 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65d47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.154+0000 7f3ba1000700 1 -- 172.21.15.131:0/596085169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627d636d3b0 con 0x5627d65cc6a0 2024-05-10T05:37:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.154+0000 7f3b9be00700 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:37:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.154+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65d47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.156+0000 7f3ba1000700 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ccaf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3b90095cc0 tx=0x7f3b900996c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e196a644cca4d6d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:19.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.156+0000 7f3b8e000700 1 -- 172.21.15.131:0/596085169 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b900997c0 con 0x5627d65cc6a0 2024-05-10T05:37:19.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.156+0000 7f3b8e000700 1 -- 172.21.15.131:0/596085169 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b9009d5f0 con 0x5627d65cc6a0 2024-05-10T05:37:19.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.156+0000 7f3b8e000700 1 -- 172.21.15.131:0/596085169 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9009d170 con 0x5627d65cc6a0 2024-05-10T05:37:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.158+0000 7f3bbb911b80 1 -- 172.21.15.131:0/596085169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 msgr2=0x5627d65ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.158+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ccaf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3b90095cc0 tx=0x7f3b900996c0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.160+0000 7f3bbb911b80 1 -- 172.21.15.131:0/596085169 shutdown_connections 2024-05-10T05:37:19.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.160+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65d47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.160+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ccaf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.160+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/596085169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.160+0000 7f3bbb911b80 1 -- 172.21.15.131:0/596085169 >> 172.21.15.131:0/596085169 conn(0x5627d652ea60 msgr2=0x5627d65c6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.161+0000 7f3bbb911b80 1 -- 172.21.15.131:0/596085169 shutdown_connections 2024-05-10T05:37:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.161+0000 7f3bbb911b80 1 -- 172.21.15.131:0/596085169 wait complete. 2024-05-10T05:37:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.162+0000 7f3bbb911b80 1 -- start start 2024-05-10T05:37:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.162+0000 7f3bbb911b80 1 Processor -- start 2024-05-10T05:37:19.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.170+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cc6a0 0x5627d65d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.170+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cd570 0x5627d65d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.170+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.170+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d6488560 con 0x5627d65cdec0 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.170+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d6488200 con 0x5627d65cc6a0 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.170+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d636f6a0 con 0x5627d65cd570 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.170+0000 7f3bbb911b80 10 client.0 _ll_register_callbacks cb 0x5627d6491b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.172+0000 7f3ba1a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.173+0000 7f3ba1a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65ddc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44088/0 (socket says 172.21.15.131:44088) 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.173+0000 7f3ba1a00700 1 -- 172.21.15.131:0/1383610623 learned_addr learned my addr 172.21.15.131:0/1383610623 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.176+0000 7f3ba1a00700 1 -- 172.21.15.131:0/1383610623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cd570 msgr2=0x5627d65d6d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.176+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cd570 0x5627d65d6d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.176+0000 7f3ba1a00700 1 -- 172.21.15.131:0/1383610623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cc6a0 msgr2=0x5627d65d67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:37:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.176+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cc6a0 0x5627d65d67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.176+0000 7f3ba1a00700 1 -- 172.21.15.131:0/1383610623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9400f040 con 0x5627d65cdec0 2024-05-10T05:37:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.177+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65ddc90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3b94011d60 tx=0x7f3b9400f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afe1e0e76d6cb17 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.177+0000 7f3b82a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:37:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.178+0000 7f3b82a00700 1 -- 172.21.15.131:0/1383610623 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9400fcd0 con 0x5627d65cdec0 2024-05-10T05:37:19.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.179+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1383610623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627d6445270 con 0x5627d65cdec0 2024-05-10T05:37:19.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.179+0000 7f3b82a00700 1 -- 172.21.15.131:0/1383610623 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3b9400fe30 con 0x5627d65cdec0 2024-05-10T05:37:19.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.180+0000 7f3b82a00700 1 -- 172.21.15.131:0/1383610623 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b94004bb0 con 0x5627d65cdec0 2024-05-10T05:37:19.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.180+0000 7f3bbb911b80 10 client.16005 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:19.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.180+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1383610623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f3b9c1a6a30 con 0x5627d65cdec0 2024-05-10T05:37:19.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.181+0000 7f3b82a00700 1 -- 172.21.15.131:0/1383610623 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627d6445270 con 0x5627d65cdec0 2024-05-10T05:37:19.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.181+0000 7f3b82a00700 1 -- 172.21.15.131:0/1383610623 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3b94005020 con 0x5627d65cdec0 2024-05-10T05:37:19.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.181+0000 7f3bbb911b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:19.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.181+0000 7f3bbb911b80 10 client.16005 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:37:19.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.181+0000 7f3bbb911b80 20 client.16005 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:19.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.181+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5627d65e2580 0x5627d65e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.182+0000 7f3bbb911b80 4 client.16005 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:19.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.182+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1383610623 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627d659b780 con 0x5627d65e2580 2024-05-10T05:37:19.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.182+0000 7f3ba1000700 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5627d65e2580 0x5627d65e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.183+0000 7f3ba1000700 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5627d65e2580 0x5627d65e49d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39c75f73b97ab219 server_cookie=be0370c7afb7c1e9 in_seq=0 out_seq=0 2024-05-10T05:37:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.183+0000 7f3b82a00700 10 client.16005 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:37:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.184+0000 7f3b82a00700 1 -- 172.21.15.131:0/1383610623 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f3b900a1e90 con 0x5627d65e2580 2024-05-10T05:37:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.184+0000 7f3b82a00700 10 client.16005 handle_command_reply: tid=0 2024-05-10T05:37:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.184+0000 7f3bbb911b80 1 client.16005 shutdown 2024-05-10T05:37:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.185+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1383610623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 msgr2=0x5627d65ddc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.185+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65ddc90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3b94011d60 tx=0x7f3b9400f4b0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.186+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1383610623 shutdown_connections 2024-05-10T05:37:19.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.186+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5627d65e2580 0x5627d65e49d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.186+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d65cdec0 0x5627d65ddc90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.186+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cc6a0 0x5627d65d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.186+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1383610623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cd570 0x5627d65d6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.186+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1383610623 >> 172.21.15.131:0/1383610623 conn(0x5627d652ea60 msgr2=0x5627d65c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:19.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.186+0000 7f3b82a00700 0 client.16005 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:37:19.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.186+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1383610623 shutdown_connections 2024-05-10T05:37:19.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.186+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1383610623 wait complete. 2024-05-10T05:37:19.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.187+0000 7f3bbb911b80 20 client.16005 trim_cache size 0 max 16384 2024-05-10T05:37:19.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.221+0000 7f3bbb911b80 1 Processor -- start 2024-05-10T05:37:19.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.224+0000 7f3bbb911b80 1 -- start start 2024-05-10T05:37:19.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.227+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.228+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.228+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.228+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d65c6fd0 con 0x5627d652ea60 2024-05-10T05:37:19.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.229+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d65c7140 con 0x5627d65cd570 2024-05-10T05:37:19.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.229+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d65d0e20 con 0x5627d65cc6a0 2024-05-10T05:37:19.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.229+0000 7f3ba1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.229+0000 7f3ba1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65e4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56404/0 (socket says 172.21.15.131:56404) 2024-05-10T05:37:19.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.229+0000 7f3ba1a00700 1 -- 172.21.15.131:0/585677015 learned_addr learned my addr 172.21.15.131:0/585677015 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:19.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.229+0000 7f3b9be00700 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.229+0000 7f3ba1000700 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.230+0000 7f3ba1a00700 1 -- 172.21.15.131:0/585677015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 msgr2=0x5627d65ddb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:19.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.230+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ddb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.230+0000 7f3ba1a00700 1 -- 172.21.15.131:0/585677015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 msgr2=0x5627d65e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:19.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.230+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.230+0000 7f3ba1a00700 1 -- 172.21.15.131:0/585677015 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627d6319b60 con 0x5627d65cd570 2024-05-10T05:37:19.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.230+0000 7f3b9be00700 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:19.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.230+0000 7f3ba1000700 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:37:19.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.231+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65e4ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b94008f70 tx=0x7f3b94008fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe7930db1342e84 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:19.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.231+0000 7f3b8ea00700 1 -- 172.21.15.131:0/585677015 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b940056a0 con 0x5627d65cd570 2024-05-10T05:37:19.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.231+0000 7f3b8ea00700 1 -- 172.21.15.131:0/585677015 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b94005020 con 0x5627d65cd570 2024-05-10T05:37:19.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.231+0000 7f3b8ea00700 1 -- 172.21.15.131:0/585677015 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94028720 con 0x5627d65cd570 2024-05-10T05:37:19.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.233+0000 7f3bbb911b80 1 -- 172.21.15.131:0/585677015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 msgr2=0x5627d65e4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:19.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.233+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65e4ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b94008f70 tx=0x7f3b94008fa0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.234+0000 7f3bbb911b80 1 -- 172.21.15.131:0/585677015 shutdown_connections 2024-05-10T05:37:19.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.234+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.234+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65e4ad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.234+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/585677015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65ddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.234+0000 7f3bbb911b80 1 -- 172.21.15.131:0/585677015 >> 172.21.15.131:0/585677015 conn(0x5627d65c3450 msgr2=0x5627d65d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:19.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.235+0000 7f3bbb911b80 1 -- 172.21.15.131:0/585677015 shutdown_connections 2024-05-10T05:37:19.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.235+0000 7f3bbb911b80 1 -- 172.21.15.131:0/585677015 wait complete. 2024-05-10T05:37:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.236+0000 7f3bbb911b80 1 -- start start 2024-05-10T05:37:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.236+0000 7f3bbb911b80 1 Processor -- start 2024-05-10T05:37:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.238+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65dba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.241+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65dbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.241+0000 7f3bbb911b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65dce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.241+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d65dd340 con 0x5627d652ea60 2024-05-10T05:37:19.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.241+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d65dd4b0 con 0x5627d65cd570 2024-05-10T05:37:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.242+0000 7f3bbb911b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d65dd670 con 0x5627d65cc6a0 2024-05-10T05:37:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.242+0000 7f3bbb911b80 10 client.0 _ll_register_callbacks cb 0x5627d6491b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.242+0000 7f3ba1000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65dba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.242+0000 7f3ba1000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65dba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44106/0 (socket says 172.21.15.131:44106) 2024-05-10T05:37:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.242+0000 7f3ba1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65dce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.242+0000 7f3ba1000700 1 -- 172.21.15.131:0/1714617783 learned_addr learned my addr 172.21.15.131:0/1714617783 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.243+0000 7f3ba1a00700 1 -- 172.21.15.131:0/1714617783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 msgr2=0x5627d65dbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.243+0000 7f3b9be00700 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65dbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.247+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65dbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.247+0000 7f3ba1a00700 1 -- 172.21.15.131:0/1714617783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 msgr2=0x5627d65dba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:19.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.247+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65dba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.247+0000 7f3ba1a00700 1 -- 172.21.15.131:0/1714617783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c10efc0 con 0x5627d65cd570 2024-05-10T05:37:19.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.249+0000 7f3b9be00700 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65dbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:19.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.249+0000 7f3ba1000700 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65dba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:19.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.250+0000 7f3ba1a00700 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65dce00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5627d64e4ae0 tx=0x7f3b94011a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a425f14cdd7fa21 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:19.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.250+0000 7f3b82a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:37:19.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.250+0000 7f3b82a00700 1 -- 172.21.15.131:0/1714617783 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b940286a0 con 0x5627d65cd570 2024-05-10T05:37:19.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.250+0000 7f3b82a00700 1 -- 172.21.15.131:0/1714617783 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f3b940056a0 con 0x5627d65cd570 2024-05-10T05:37:19.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.252+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1714617783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b9009d5f0 con 0x5627d65cd570 2024-05-10T05:37:19.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.252+0000 7f3b82a00700 1 -- 172.21.15.131:0/1714617783 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b940058e0 con 0x5627d65cd570 2024-05-10T05:37:19.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.252+0000 7f3bbb911b80 10 client.26090 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:19.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.252+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1714617783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f3b9c1a6a30 con 0x5627d65cd570 2024-05-10T05:37:19.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.253+0000 7f3b82a00700 1 -- 172.21.15.131:0/1714617783 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b9009d5f0 con 0x5627d65cd570 2024-05-10T05:37:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.253+0000 7f3b82a00700 1 -- 172.21.15.131:0/1714617783 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3b94038050 con 0x5627d65cd570 2024-05-10T05:37:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.253+0000 7f3bbb911b80 10 client.26090 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.253+0000 7f3bbb911b80 10 client.26090 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:37:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.253+0000 7f3bbb911b80 20 client.26090 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.253+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5627d65cdec0 0x5627d663b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.253+0000 7f3bbb911b80 4 client.26090 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:37:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.253+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1714617783 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5627d64e4ec0 con 0x5627d65cdec0 2024-05-10T05:37:19.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.254+0000 7f3ba1000700 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5627d65cdec0 0x5627d663b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:19.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.257+0000 7f3ba1000700 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5627d65cdec0 0x5627d663b410 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed428c3ca303447e server_cookie=121aeb07820d1ae4 in_seq=0 out_seq=0 2024-05-10T05:37:19.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.257+0000 7f3b82a00700 10 client.26090 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:37:19.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.258+0000 7f3b82a00700 1 -- 172.21.15.131:0/1714617783 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5627d64e4ec0 con 0x5627d65cdec0 2024-05-10T05:37:19.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.258+0000 7f3b82a00700 10 client.26090 handle_command_reply: tid=0 2024-05-10T05:37:19.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.258+0000 7f3bbb911b80 1 client.26090 shutdown 2024-05-10T05:37:19.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.259+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1714617783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 msgr2=0x5627d65dce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:19.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.259+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65dce00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5627d64e4ae0 tx=0x7f3b94011a10 comp rx=0 tx=0).stop 2024-05-10T05:37:19.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.259+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1714617783 shutdown_connections 2024-05-10T05:37:19.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.259+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5627d65cdec0 0x5627d663b410 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.259+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627d652ea60 0x5627d65dba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.259+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5627d65cd570 0x5627d65dce00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.259+0000 7f3bbb911b80 1 --2- 172.21.15.131:0/1714617783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627d65cc6a0 0x5627d65dbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.259+0000 7f3b82a00700 0 client.26090 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:37:19.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.259+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1714617783 >> 172.21.15.131:0/1714617783 conn(0x5627d65c3450 msgr2=0x5627d65d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:19.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.260+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1714617783 shutdown_connections 2024-05-10T05:37:19.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.260+0000 7f3bbb911b80 1 -- 172.21.15.131:0/1714617783 wait complete. 2024-05-10T05:37:19.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.260+0000 7f3bbb911b80 20 client.26090 trim_cache size 0 max 16384 2024-05-10T05:37:19.262 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:37:19.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.266+0000 7f3ba3a00700 1 -- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b840a1240 msgr2=0x7f3b840a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:19.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.266+0000 7f3ba3a00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b840a1240 0x7f3b840a3750 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3b88006010 tx=0x7f3b8801af10 comp rx=0 tx=0).stop 2024-05-10T05:37:19.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.266+0000 7f3ba3a00700 1 -- 172.21.15.131:0/3501457903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c108630 msgr2=0x7f3b9c19e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:19.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.266+0000 7f3ba3a00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c108630 0x7f3b9c19e550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000b550 tx=0x7f3b9000b910 comp rx=0 tx=0).stop 2024-05-10T05:37:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.269+0000 7f3ba3a00700 1 -- 172.21.15.131:0/3501457903 shutdown_connections 2024-05-10T05:37:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.270+0000 7f3ba3a00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f3b840a1240 0x7f3b840a3750 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.270+0000 7f3ba3a00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b9c109c50 0x7f3b9c1a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.270+0000 7f3ba3a00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c108630 0x7f3b9c19e550 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.270+0000 7f3ba3a00700 1 --2- 172.21.15.131:0/3501457903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9c109300 0x7f3b9c19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:19.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.270+0000 7f3ba3a00700 1 -- 172.21.15.131:0/3501457903 >> 172.21.15.131:0/3501457903 conn(0x7f3b9c06ea00 msgr2=0x7f3b9c10c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:19.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.270+0000 7f3ba3a00700 1 -- 172.21.15.131:0/3501457903 shutdown_connections 2024-05-10T05:37:19.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:19.271+0000 7f3ba3a00700 1 -- 172.21.15.131:0/3501457903 wait complete. 2024-05-10T05:37:19.338 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:37:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.154+0000 7fb9b8a00700 1 -- 172.21.15.131:0/212505442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4108b60 msgr2=0x7fb9b410e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.154+0000 7fb9b8a00700 1 --2- 172.21.15.131:0/212505442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4108b60 0x7fb9b410e480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00b0a0 tx=0x7fb9ac00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.157+0000 7fb9b8a00700 1 -- 172.21.15.131:0/212505442 shutdown_connections 2024-05-10T05:37:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.157+0000 7fb9b8a00700 1 --2- 172.21.15.131:0/212505442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4108b60 0x7fb9b410e480 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.157+0000 7fb9b8a00700 1 --2- 172.21.15.131:0/212505442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9b4109500 0x7fb9b4109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.157+0000 7fb9b8a00700 1 --2- 172.21.15.131:0/212505442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b4108210 0x7fb9b4108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.157+0000 7fb9b8a00700 1 -- 172.21.15.131:0/212505442 >> 172.21.15.131:0/212505442 conn(0x7fb9b406e800 msgr2=0x7fb9b4070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.157+0000 7fb9b8a00700 1 -- 172.21.15.131:0/212505442 shutdown_connections 2024-05-10T05:37:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.158+0000 7fb9b8a00700 1 -- 172.21.15.131:0/212505442 wait complete. 2024-05-10T05:37:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.159+0000 7fb9b8a00700 1 Processor -- start 2024-05-10T05:37:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.161+0000 7fb9b8a00700 1 -- start start 2024-05-10T05:37:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.161+0000 7fb9b8a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b4108210 0x7fb9b41a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.161+0000 7fb9b8a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4108b60 0x7fb9b41a7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.162+0000 7fb9b8a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9b4109500 0x7fb9b41ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.162+0000 7fb9b8a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41ae720 con 0x7fb9b4108210 2024-05-10T05:37:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.162+0000 7fb9b2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4108b60 0x7fb9b41a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.162+0000 7fb9b2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4108b60 0x7fb9b41a7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56422/0 (socket says 172.21.15.131:56422) 2024-05-10T05:37:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.162+0000 7fb9b2a00700 1 -- 172.21.15.131:0/4169811058 learned_addr learned my addr 172.21.15.131:0/4169811058 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.162+0000 7fb9b3400700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b4108210 0x7fb9b41a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.162+0000 7fb9b8a00700 1 -- 172.21.15.131:0/4169811058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41ae860 con 0x7fb9b4108b60 2024-05-10T05:37:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.162+0000 7fb9b8a00700 1 -- 172.21.15.131:0/4169811058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41aea20 con 0x7fb9b4109500 2024-05-10T05:37:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.163+0000 7fb9b2a00700 1 -- 172.21.15.131:0/4169811058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9b4109500 msgr2=0x7fb9b41ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.163+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9b4109500 0x7fb9b41ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.163+0000 7fb9b2a00700 1 -- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b4108210 msgr2=0x7fb9b41a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.163+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b4108210 0x7fb9b41a6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.163+0000 7fb9b2a00700 1 -- 172.21.15.131:0/4169811058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac009d00 con 0x7fb9b4108b60 2024-05-10T05:37:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.165+0000 7fb9b3400700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b4108210 0x7fb9b41a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.165+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4108b60 0x7fb9b41a7290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000c9e0 tx=0x7fb9a000cda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:20.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.167+0000 7fb9abe00700 1 -- 172.21.15.131:0/4169811058 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0007ac0 con 0x7fb9b4108b60 2024-05-10T05:37:20.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.168+0000 7fb9abe00700 1 -- 172.21.15.131:0/4169811058 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a0007c20 con 0x7fb9b4108b60 2024-05-10T05:37:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.168+0000 7fb9abe00700 1 -- 172.21.15.131:0/4169811058 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0018660 con 0x7fb9b4108b60 2024-05-10T05:37:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.168+0000 7fb9b8a00700 1 -- 172.21.15.131:0/4169811058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b41aed00 con 0x7fb9b4108b60 2024-05-10T05:37:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.168+0000 7fb9b8a00700 1 -- 172.21.15.131:0/4169811058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b41af160 con 0x7fb9b4108b60 2024-05-10T05:37:20.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.180+0000 7fb9abe00700 1 -- 172.21.15.131:0/4169811058 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb9a00187c0 con 0x7fb9b4108b60 2024-05-10T05:37:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.186+0000 7fb9abe00700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb9980a1270 0x7fb9980a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.186+0000 7fb9abe00700 1 -- 172.21.15.131:0/4169811058 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb9a00ccdb0 con 0x7fb9b4108b60 2024-05-10T05:37:20.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.196+0000 7fb9b3400700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb9980a1270 0x7fb9980a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.206+0000 7fb9d1e55b80 1 Processor -- start 2024-05-10T05:37:20.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.207+0000 7fb9d1e55b80 1 -- start start 2024-05-10T05:37:20.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.209+0000 7fb9b3400700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb9980a1270 0x7fb9980a3780 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb9b41a8140 tx=0x7fb9a4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:20.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.210+0000 7fb9d1e55b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4d5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.211+0000 7fb9b3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4d5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.211+0000 7fb9b3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4d5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44132/0 (socket says 172.21.15.131:44132) 2024-05-10T05:37:20.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.211+0000 7fb9b3400700 1 -- 172.21.15.131:0/3545860703 learned_addr learned my addr 172.21.15.131:0/3545860703 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.211+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.212+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 0x55855d4dd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.212+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.213+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3545860703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d3fc150 con 0x55855d4d5530 2024-05-10T05:37:20.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.213+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3545860703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d3f9b10 con 0x55855d4d67a0 2024-05-10T05:37:20.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.213+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3545860703 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d3f5400 con 0x55855d4d70f0 2024-05-10T05:37:20.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.213+0000 7fb9b3e00700 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 0x55855d4dd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.214+0000 7fb9b2a00700 1 -- 172.21.15.131:0/3545860703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 msgr2=0x55855d4dd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.214+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 0x55855d4dd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.214+0000 7fb9b2a00700 1 -- 172.21.15.131:0/3545860703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 msgr2=0x55855d4d5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.214+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4d5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.214+0000 7fb9b2a00700 1 -- 172.21.15.131:0/3545860703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55855d2763b0 con 0x55855d4d67a0 2024-05-10T05:37:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.215+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4d6bb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00ca900 tx=0x7fb9a0099fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1722e75b0c7acab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.215+0000 7fb99fe00700 1 -- 172.21.15.131:0/3545860703 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0099840 con 0x55855d4d67a0 2024-05-10T05:37:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.215+0000 7fb99fe00700 1 -- 172.21.15.131:0/3545860703 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a00999a0 con 0x55855d4d67a0 2024-05-10T05:37:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.215+0000 7fb99fe00700 1 -- 172.21.15.131:0/3545860703 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a00992b0 con 0x55855d4d67a0 2024-05-10T05:37:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.216+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3545860703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 msgr2=0x55855d4d6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.217+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4d6bb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00ca900 tx=0x7fb9a0099fc0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.217+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3545860703 shutdown_connections 2024-05-10T05:37:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.217+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4d5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.217+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 0x55855d4dd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.218+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3545860703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4d6bb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.218+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3545860703 >> 172.21.15.131:0/3545860703 conn(0x55855d437a60 msgr2=0x55855d4d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.218+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3545860703 shutdown_connections 2024-05-10T05:37:20.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.218+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3545860703 wait complete. 2024-05-10T05:37:20.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.219+0000 7fb9d1e55b80 1 -- start start 2024-05-10T05:37:20.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.222+0000 7fb9d1e55b80 1 Processor -- start 2024-05-10T05:37:20.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.224+0000 7fb9d1e55b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4df940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.225+0000 7fb9d1e55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4dfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.225+0000 7fb9b3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4df940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.226+0000 7fb9b3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4df940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44146/0 (socket says 172.21.15.131:44146) 2024-05-10T05:37:20.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.226+0000 7fb9b3400700 1 -- 172.21.15.131:0/3087676435 learned_addr learned my addr 172.21.15.131:0/3087676435 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.226+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 0x55855d4e6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.227+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4dfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.227+0000 7fb9b3e00700 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 0x55855d4e6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.227+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d391560 con 0x55855d4d5530 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.227+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d391200 con 0x55855d4d67a0 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.227+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d2786a0 con 0x55855d4d70f0 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.228+0000 7fb9b3e00700 1 -- 172.21.15.131:0/3087676435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 msgr2=0x55855d4dfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.228+0000 7fb9b3e00700 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4dfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.228+0000 7fb9b3e00700 1 -- 172.21.15.131:0/3087676435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 msgr2=0x55855d4df940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.229+0000 7fb9b3e00700 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4df940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.229+0000 7fb9b3e00700 1 -- 172.21.15.131:0/3087676435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b41af160 con 0x55855d4d70f0 2024-05-10T05:37:20.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.229+0000 7fb9b3e00700 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 0x55855d4e6de0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009fd0 tx=0x7fb9ac007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26cc524dc52ee30 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:20.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.229+0000 7fb99cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:37:20.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.230+0000 7fb9b3400700 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4df940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:37:20.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.230+0000 7fb99cc00700 1 -- 172.21.15.131:0/3087676435 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00e070 con 0x55855d4d70f0 2024-05-10T05:37:20.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.234+0000 7fb99cc00700 1 -- 172.21.15.131:0/3087676435 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb9ac003a00 con 0x55855d4d70f0 2024-05-10T05:37:20.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.234+0000 7fb99cc00700 10 client.0 inactive, discarding osd_map(82..82 src has 1..82) v4 2024-05-10T05:37:20.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.234+0000 7fb9d1e55b80 10 client.0 _ll_register_callbacks cb 0x55855d39ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:20.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.235+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55855d34e270 con 0x55855d4d70f0 2024-05-10T05:37:20.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.235+0000 7fb99cc00700 1 -- 172.21.15.131:0/3087676435 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9ac00be70 con 0x55855d4d70f0 2024-05-10T05:37:20.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.235+0000 7fb9d1e55b80 10 client.26119 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:20.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.235+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=28}) v3 -- 0x55855d222b60 con 0x55855d4d70f0 2024-05-10T05:37:20.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.236+0000 7fb99cc00700 1 -- 172.21.15.131:0/3087676435 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb9ac00b0d0 con 0x55855d4d70f0 2024-05-10T05:37:20.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.236+0000 7fb9d1e55b80 10 client.26119 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:20.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.236+0000 7fb9d1e55b80 10 client.26119 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:37:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.236+0000 7fb9d1e55b80 20 client.26119 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.236+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55855d4eb270 0x55855d4ed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.236+0000 7fb9d1e55b80 4 client.26119 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.236+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55855d4a4780 con 0x55855d4eb270 2024-05-10T05:37:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.237+0000 7fb9b3400700 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55855d4eb270 0x55855d4ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.239+0000 7fb9b3400700 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55855d4eb270 0x55855d4ed780 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12033921db2b3df5 server_cookie=5ab9fe8f670bbd2 in_seq=0 out_seq=0 2024-05-10T05:37:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.239+0000 7fb99cc00700 10 client.26119 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:37:20.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.240+0000 7fb99cc00700 1 -- 172.21.15.131:0/3087676435 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55855d4a4780 con 0x55855d4eb270 2024-05-10T05:37:20.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.240+0000 7fb99cc00700 10 client.26119 handle_command_reply: tid=0 2024-05-10T05:37:20.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.241+0000 7fb9d1e55b80 1 client.26119 shutdown 2024-05-10T05:37:20.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.242+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 msgr2=0x55855d4e6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.242+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 0x55855d4e6de0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009fd0 tx=0x7fb9ac007930 comp rx=0 tx=0).stop 2024-05-10T05:37:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.242+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 shutdown_connections 2024-05-10T05:37:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.242+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55855d4eb270 0x55855d4ed780 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.242+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d5530 0x55855d4df940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.242+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d70f0 0x55855d4e6de0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.242+0000 7fb99cc00700 0 client.26119 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:37:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.242+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/3087676435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4dfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.242+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 >> 172.21.15.131:0/3087676435 conn(0x55855d437a60 msgr2=0x55855d4d4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:20.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.251+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 shutdown_connections 2024-05-10T05:37:20.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.251+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/3087676435 wait complete. 2024-05-10T05:37:20.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.251+0000 7fb9d1e55b80 20 client.26119 trim_cache size 0 max 16384 2024-05-10T05:37:20.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.281+0000 7fb9d1e55b80 1 Processor -- start 2024-05-10T05:37:20.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.281+0000 7fb9d1e55b80 1 -- start start 2024-05-10T05:37:20.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.282+0000 7fb9d1e55b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4deb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.282+0000 7fb9d1e55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.282+0000 7fb9d1e55b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d70f0 0x55855d4df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.282+0000 7fb9b2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.282+0000 7fb9b2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4e6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56458/0 (socket says 172.21.15.131:56458) 2024-05-10T05:37:20.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9b3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d70f0 0x55855d4df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9b3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4deb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9b3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d70f0 0x55855d4df0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44150/0 (socket says 172.21.15.131:44150) 2024-05-10T05:37:20.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9b2a00700 1 -- 172.21.15.131:0/4131140203 learned_addr learned my addr 172.21.15.131:0/4131140203 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:20.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/4131140203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d4dd900 con 0x55855d4d70f0 2024-05-10T05:37:20.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/4131140203 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d4da050 con 0x55855d4d67a0 2024-05-10T05:37:20.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/4131140203 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d4eafc0 con 0x55855d4d5530 2024-05-10T05:37:20.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9b3e00700 1 -- 172.21.15.131:0/4131140203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 msgr2=0x55855d4deb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9b3e00700 1 --2- 172.21.15.131:0/4131140203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4deb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9b3e00700 1 -- 172.21.15.131:0/4131140203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 msgr2=0x55855d4e6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9b3e00700 1 --2- 172.21.15.131:0/4131140203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4e6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.283+0000 7fb9b3e00700 1 -- 172.21.15.131:0/4131140203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55855d3bb560 con 0x55855d4d70f0 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.284+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/4131140203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.284+0000 7fb9b3400700 1 --2- 172.21.15.131:0/4131140203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.284+0000 7fb9b3e00700 1 --2- 172.21.15.131:0/4131140203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d70f0 0x55855d4df0a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009ba0 tx=0x7fb9ac01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70417342f0e855c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.285+0000 7fb9a9200700 1 -- 172.21.15.131:0/4131140203 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac02ed40 con 0x55855d4d70f0 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.285+0000 7fb9a9200700 1 -- 172.21.15.131:0/4131140203 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55855d34e270 con 0x55855d4d70f0 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.285+0000 7fb9a9200700 1 -- 172.21.15.131:0/4131140203 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac02eab0 con 0x55855d4d70f0 2024-05-10T05:37:20.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.286+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/4131140203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d70f0 msgr2=0x55855d4df0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.286+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/4131140203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d70f0 0x55855d4df0a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009ba0 tx=0x7fb9ac01b000 comp rx=0 tx=0).stop 2024-05-10T05:37:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.287+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/4131140203 shutdown_connections 2024-05-10T05:37:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.287+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/4131140203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d70f0 0x55855d4df0a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.287+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/4131140203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.287+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/4131140203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d67a0 0x55855d4e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.287+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/4131140203 >> 172.21.15.131:0/4131140203 conn(0x55855d437a60 msgr2=0x55855d4da570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.287+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/4131140203 shutdown_connections 2024-05-10T05:37:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.287+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/4131140203 wait complete. 2024-05-10T05:37:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.288+0000 7fb9d1e55b80 1 -- start start 2024-05-10T05:37:20.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.288+0000 7fb9d1e55b80 1 Processor -- start 2024-05-10T05:37:20.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.289+0000 7fb9d1e55b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.289+0000 7fb9d1e55b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d67a0 0x55855d4e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.290+0000 7fb9d1e55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d70f0 0x55855d4e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.290+0000 7fb9d1e55b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d4e63e0 con 0x55855d4d67a0 2024-05-10T05:37:20.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.290+0000 7fb9d1e55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d4e6550 con 0x55855d4d70f0 2024-05-10T05:37:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.290+0000 7fb9d1e55b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855d4e6710 con 0x55855d4d5530 2024-05-10T05:37:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.290+0000 7fb9d1e55b80 10 client.0 _ll_register_callbacks cb 0x55855d39ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.289+0000 7fb9b3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.290+0000 7fb9b3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59670/0 (socket says 172.21.15.131:59670) 2024-05-10T05:37:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.290+0000 7fb9b3400700 1 -- 172.21.15.131:0/1689189474 learned_addr learned my addr 172.21.15.131:0/1689189474 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.290+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d67a0 0x55855d4e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.290+0000 7fb9b3400700 1 -- 172.21.15.131:0/1689189474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d70f0 msgr2=0x55855d4e5ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:37:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.291+0000 7fb9b3400700 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d70f0 0x55855d4e5ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.291+0000 7fb9b3400700 1 -- 172.21.15.131:0/1689189474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d67a0 msgr2=0x55855d4e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.291+0000 7fb9b3400700 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d67a0 0x55855d4e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.291+0000 7fb9b3400700 1 -- 172.21.15.131:0/1689189474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55855d222b60 con 0x55855d4d5530 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.291+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d67a0 0x55855d4e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.292+0000 7fb9b3400700 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4e4a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005d50 tx=0x7fb9a40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71ecaf608c2ad07f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.292+0000 7fb99cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.292+0000 7fb99cc00700 1 -- 172.21.15.131:0/1689189474 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4004250 con 0x55855d4d5530 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.292+0000 7fb99cc00700 1 -- 172.21.15.131:0/1689189474 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb9a40043b0 con 0x55855d4d5530 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.292+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/1689189474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9a00999a0 con 0x55855d4d5530 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.292+0000 7fb99cc00700 1 -- 172.21.15.131:0/1689189474 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a4004570 con 0x55855d4d5530 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.292+0000 7fb9d1e55b80 10 client.26122 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.292+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/1689189474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55855d27acd0 con 0x55855d4d5530 2024-05-10T05:37:20.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.293+0000 7fb99cc00700 1 -- 172.21.15.131:0/1689189474 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a00999a0 con 0x55855d4d5530 2024-05-10T05:37:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.293+0000 7fb99cc00700 1 -- 172.21.15.131:0/1689189474 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb9a4027410 con 0x55855d4d5530 2024-05-10T05:37:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.293+0000 7fb9d1e55b80 10 client.26122 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.293+0000 7fb9d1e55b80 10 client.26122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:37:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.293+0000 7fb9d1e55b80 20 client.26122 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.293+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55855d542490 0x55855d5448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.293+0000 7fb9d1e55b80 4 client.26122 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:37:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.293+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/1689189474 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55855d3edec0 con 0x55855d542490 2024-05-10T05:37:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.294+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55855d542490 0x55855d5448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:20.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.295+0000 7fb99cc00700 10 client.26122 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:37:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.295+0000 7fb9b2a00700 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55855d542490 0x55855d5448b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df2e41ab2efd337e server_cookie=94563c8681e8b614 in_seq=0 out_seq=0 2024-05-10T05:37:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.296+0000 7fb99cc00700 1 -- 172.21.15.131:0/1689189474 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55855d3edec0 con 0x55855d542490 2024-05-10T05:37:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.296+0000 7fb99cc00700 10 client.26122 handle_command_reply: tid=0 2024-05-10T05:37:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.296+0000 7fb9d1e55b80 1 client.26122 shutdown 2024-05-10T05:37:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.296+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/1689189474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 msgr2=0x55855d4e4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.296+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4e4a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005d50 tx=0x7fb9a40047c0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.297+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/1689189474 shutdown_connections 2024-05-10T05:37:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.297+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55855d542490 0x55855d5448b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.297+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55855d4d70f0 0x55855d4e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.297+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55855d4d5530 0x55855d4e4a30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.297+0000 7fb9d1e55b80 1 --2- 172.21.15.131:0/1689189474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55855d4d67a0 0x55855d4e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.297+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/1689189474 >> 172.21.15.131:0/1689189474 conn(0x55855d437a60 msgr2=0x55855d4da570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:20.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.297+0000 7fb99cc00700 0 client.26122 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:37:20.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.297+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/1689189474 shutdown_connections 2024-05-10T05:37:20.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.298+0000 7fb9d1e55b80 1 -- 172.21.15.131:0/1689189474 wait complete. 2024-05-10T05:37:20.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.298+0000 7fb9d1e55b80 20 client.26122 trim_cache size 0 max 16384 2024-05-10T05:37:20.302 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:37:20.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.299+0000 7fb9b8a00700 1 -- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb9980a1270 msgr2=0x7fb9980a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:20.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.299+0000 7fb9b8a00700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb9980a1270 0x7fb9980a3780 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb9b41a8140 tx=0x7fb9a4009260 comp rx=0 tx=0).stop 2024-05-10T05:37:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.300+0000 7fb9b8a00700 1 -- 172.21.15.131:0/4169811058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4108b60 msgr2=0x7fb9b41a7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.300+0000 7fb9b8a00700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4108b60 0x7fb9b41a7290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000c9e0 tx=0x7fb9a000cda0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.300+0000 7fb9b8a00700 1 -- 172.21.15.131:0/4169811058 shutdown_connections 2024-05-10T05:37:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.300+0000 7fb9b8a00700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb9980a1270 0x7fb9980a3780 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.300+0000 7fb9b8a00700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9b4108210 0x7fb9b41a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.300+0000 7fb9b8a00700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9b4109500 0x7fb9b41ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.300+0000 7fb9b8a00700 1 --2- 172.21.15.131:0/4169811058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4108b60 0x7fb9b41a7290 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.300+0000 7fb9b8a00700 1 -- 172.21.15.131:0/4169811058 >> 172.21.15.131:0/4169811058 conn(0x7fb9b406e800 msgr2=0x7fb9b406f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.300+0000 7fb9b8a00700 1 -- 172.21.15.131:0/4169811058 shutdown_connections 2024-05-10T05:37:20.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:20.300+0000 7fb9b8a00700 1 -- 172.21.15.131:0/4169811058 wait complete. 2024-05-10T05:37:20.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:37:20.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:20 smithi131 ceph-mon[37264]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 243 MiB/s wr, 75 op/s 2024-05-10T05:37:20.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:20 smithi157 ceph-mon[78427]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 243 MiB/s wr, 75 op/s 2024-05-10T05:37:20.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:20 smithi142 ceph-mon[77667]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 243 MiB/s wr, 75 op/s 2024-05-10T05:37:21.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.092+0000 7fe9d4e00700 1 -- 172.21.15.131:0/1602788942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9d0073b40 msgr2=0x7fe9d0074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:21.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.092+0000 7fe9d4e00700 1 --2- 172.21.15.131:0/1602788942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9d0073b40 0x7fe9d0074010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009990 tx=0x7fe9c0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.092+0000 7fe9d4e00700 1 -- 172.21.15.131:0/1602788942 shutdown_connections 2024-05-10T05:37:21.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.092+0000 7fe9d4e00700 1 --2- 172.21.15.131:0/1602788942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d010e220 0x7fe9d0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.092+0000 7fe9d4e00700 1 --2- 172.21.15.131:0/1602788942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d0075140 0x7fe9d0073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.092+0000 7fe9d4e00700 1 --2- 172.21.15.131:0/1602788942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9d0073b40 0x7fe9d0074010 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.092+0000 7fe9d4e00700 1 -- 172.21.15.131:0/1602788942 >> 172.21.15.131:0/1602788942 conn(0x7fe9d00fd1d0 msgr2=0x7fe9d00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:21.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.093+0000 7fe9d4e00700 1 -- 172.21.15.131:0/1602788942 shutdown_connections 2024-05-10T05:37:21.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.093+0000 7fe9d4e00700 1 -- 172.21.15.131:0/1602788942 wait complete. 2024-05-10T05:37:21.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.095+0000 7fe9d4e00700 1 Processor -- start 2024-05-10T05:37:21.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.095+0000 7fe9d4e00700 1 -- start start 2024-05-10T05:37:21.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.096+0000 7fe9d4e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d0073b40 0x7fe9d010ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.096+0000 7fe9d4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9d0075140 0x7fe9d010b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.096+0000 7fe9d4e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d010e220 0x7fe9d01ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.096+0000 7fe9cde00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d0073b40 0x7fe9d010ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.096+0000 7fe9cd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9d0075140 0x7fe9d010b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.096+0000 7fe9cd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9d0075140 0x7fe9d010b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56488/0 (socket says 172.21.15.131:56488) 2024-05-10T05:37:21.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.096+0000 7fe9cde00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d0073b40 0x7fe9d010ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59682/0 (socket says 172.21.15.131:59682) 2024-05-10T05:37:21.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.097+0000 7fe9cd400700 1 -- 172.21.15.131:0/3476535260 learned_addr learned my addr 172.21.15.131:0/3476535260 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:21.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.097+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d01ae880 con 0x7fe9d010e220 2024-05-10T05:37:21.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.097+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d01ae9c0 con 0x7fe9d0075140 2024-05-10T05:37:21.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.097+0000 7fe9ce800700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d010e220 0x7fe9d01ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.097+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d01aeb80 con 0x7fe9d0073b40 2024-05-10T05:37:21.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.098+0000 7fe9cd400700 1 -- 172.21.15.131:0/3476535260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d0073b40 msgr2=0x7fe9d010ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:21.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.098+0000 7fe9cd400700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d0073b40 0x7fe9d010ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.098+0000 7fe9cd400700 1 -- 172.21.15.131:0/3476535260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d010e220 msgr2=0x7fe9d01ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:21.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.098+0000 7fe9cd400700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d010e220 0x7fe9d01ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.098+0000 7fe9cd400700 1 -- 172.21.15.131:0/3476535260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc009520 con 0x7fe9d0075140 2024-05-10T05:37:21.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.098+0000 7fe9cde00700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d0073b40 0x7fe9d010ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:37:21.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.098+0000 7fe9cd400700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9d0075140 0x7fe9d010b270 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0000c00 tx=0x7fe9c0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:21.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.099+0000 7fe9c6a00700 1 -- 172.21.15.131:0/3476535260 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c001d070 con 0x7fe9d0075140 2024-05-10T05:37:21.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.099+0000 7fe9c6a00700 1 -- 172.21.15.131:0/3476535260 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c0021410 con 0x7fe9d0075140 2024-05-10T05:37:21.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.099+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c00095f0 con 0x7fe9d0075140 2024-05-10T05:37:21.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.099+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d01af090 con 0x7fe9d0075140 2024-05-10T05:37:21.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.099+0000 7fe9c6a00700 1 -- 172.21.15.131:0/3476535260 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0017650 con 0x7fe9d0075140 2024-05-10T05:37:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.106+0000 7fe9c6a00700 1 -- 172.21.15.131:0/3476535260 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe9c0021580 con 0x7fe9d0075140 2024-05-10T05:37:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.109+0000 7fe9c6a00700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9b40a1240 0x7fe9b40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.110+0000 7fe9c6a00700 1 -- 172.21.15.131:0/3476535260 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe9c00cdf40 con 0x7fe9d0075140 2024-05-10T05:37:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.110+0000 7fe9ecd06b80 1 Processor -- start 2024-05-10T05:37:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.110+0000 7fe9cde00700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9b40a1240 0x7fe9b40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.110+0000 7fe9ecd06b80 1 -- start start 2024-05-10T05:37:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.111+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.111+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5570 0x5556850b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.111+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5ec0 0x5556850bc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.112+0000 7fe9cde00700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9b40a1240 0x7fe9b40a3750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0074e30 tx=0x7fe9c8009040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.112+0000 7fe9ecd06b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555684fdb150 con 0x5556850b5ec0 2024-05-10T05:37:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.112+0000 7fe9ecd06b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555684fd8b10 con 0x5556850b46a0 2024-05-10T05:37:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.112+0000 7fe9ecd06b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555684fd4400 con 0x5556850b5570 2024-05-10T05:37:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.112+0000 7fe9cde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.112+0000 7fe9ce800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5ec0 0x5556850bc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.112+0000 7fe9cde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56498/0 (socket says 172.21.15.131:56498) 2024-05-10T05:37:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.112+0000 7fe9cde00700 1 -- 172.21.15.131:0/1802848760 learned_addr learned my addr 172.21.15.131:0/1802848760 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.113+0000 7fe9ce800700 1 -- 172.21.15.131:0/1802848760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5570 msgr2=0x5556850b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.113+0000 7fe9cd400700 1 --2- 172.21.15.131:0/1802848760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5570 0x5556850b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.113+0000 7fe9ce800700 1 --2- 172.21.15.131:0/1802848760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5570 0x5556850b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.113+0000 7fe9ce800700 1 -- 172.21.15.131:0/1802848760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 msgr2=0x5556850b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.113+0000 7fe9ce800700 1 --2- 172.21.15.131:0/1802848760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.113+0000 7fe9ce800700 1 -- 172.21.15.131:0/1802848760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555684e553b0 con 0x5556850b5ec0 2024-05-10T05:37:21.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.113+0000 7fe9cde00700 1 --2- 172.21.15.131:0/1802848760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:21.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.113+0000 7fe9ce800700 1 --2- 172.21.15.131:0/1802848760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5ec0 0x5556850bc7c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009d00 tx=0x7fe9bc011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579462f03f870b74 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:21.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.114+0000 7fe9cd400700 1 --2- 172.21.15.131:0/1802848760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5570 0x5556850b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:21.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.114+0000 7fe9bb400700 1 -- 172.21.15.131:0/1802848760 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc019630 con 0x5556850b5ec0 2024-05-10T05:37:21.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.114+0000 7fe9bb400700 1 -- 172.21.15.131:0/1802848760 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9bc019c50 con 0x5556850b5ec0 2024-05-10T05:37:21.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.114+0000 7fe9bb400700 1 -- 172.21.15.131:0/1802848760 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00ece0 con 0x5556850b5ec0 2024-05-10T05:37:21.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.115+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1802848760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5ec0 msgr2=0x5556850bc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:21.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.115+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/1802848760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5ec0 0x5556850bc7c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009d00 tx=0x7fe9bc011720 comp rx=0 tx=0).stop 2024-05-10T05:37:21.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.116+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1802848760 shutdown_connections 2024-05-10T05:37:21.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.116+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/1802848760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5ec0 0x5556850bc7c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.116+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/1802848760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.116+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/1802848760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5570 0x5556850b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.116+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1802848760 >> 172.21.15.131:0/1802848760 conn(0x555685016a60 msgr2=0x5556850aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:21.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.116+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1802848760 shutdown_connections 2024-05-10T05:37:21.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.116+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1802848760 wait complete. 2024-05-10T05:37:21.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.117+0000 7fe9ecd06b80 1 -- start start 2024-05-10T05:37:21.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.118+0000 7fe9ecd06b80 1 Processor -- start 2024-05-10T05:37:21.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.119+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b46a0 0x5556850be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.119+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x5556850bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.119+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b5ec0 0x5556850c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.120+0000 7fe9ecd06b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555684f70560 con 0x5556850b5570 2024-05-10T05:37:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.120+0000 7fe9ecd06b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555684f70200 con 0x5556850b5ec0 2024-05-10T05:37:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.120+0000 7fe9ce800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b5ec0 0x5556850c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.120+0000 7fe9ce800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b5ec0 0x5556850c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56510/0 (socket says 172.21.15.131:56510) 2024-05-10T05:37:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.120+0000 7fe9ce800700 1 -- 172.21.15.131:0/1363832945 learned_addr learned my addr 172.21.15.131:0/1363832945 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.120+0000 7fe9cde00700 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b46a0 0x5556850be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.120+0000 7fe9ecd06b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555684e576a0 con 0x5556850b46a0 2024-05-10T05:37:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.121+0000 7fe9ce800700 1 -- 172.21.15.131:0/1363832945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b46a0 msgr2=0x5556850be710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.121+0000 7fe9ce800700 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b46a0 0x5556850be710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.121+0000 7fe9ce800700 1 -- 172.21.15.131:0/1363832945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 msgr2=0x5556850bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.121+0000 7fe9ce800700 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x5556850bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.121+0000 7fe9ce800700 1 -- 172.21.15.131:0/1363832945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c8007de0 con 0x5556850b5ec0 2024-05-10T05:37:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.122+0000 7fe9ce800700 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b5ec0 0x5556850c5bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00e720 tx=0x7fe9bc00e750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52159be2c905936a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.122+0000 7fe9afe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:37:21.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.122+0000 7fe9ecd06b80 10 client.0 _ll_register_callbacks cb 0x555684f79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:21.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.122+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1363832945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555684f2d270 con 0x5556850b5ec0 2024-05-10T05:37:21.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.122+0000 7fe9afe00700 1 -- 172.21.15.131:0/1363832945 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc019630 con 0x5556850b5ec0 2024-05-10T05:37:21.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.122+0000 7fe9afe00700 1 -- 172.21.15.131:0/1363832945 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe9bc019790 con 0x5556850b5ec0 2024-05-10T05:37:21.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.123+0000 7fe9afe00700 1 -- 172.21.15.131:0/1363832945 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9bc019950 con 0x5556850b5ec0 2024-05-10T05:37:21.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.123+0000 7fe9ecd06b80 10 client.26114 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:21.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.123+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1363832945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=28}) v3 -- 0x7fe9bc0099b0 con 0x5556850b5ec0 2024-05-10T05:37:21.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.123+0000 7fe9afe00700 1 -- 172.21.15.131:0/1363832945 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe9bc005250 con 0x5556850b5ec0 2024-05-10T05:37:21.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.124+0000 7fe9ecd06b80 10 client.26114 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:21.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.124+0000 7fe9ecd06b80 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:37:21.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.124+0000 7fe9ecd06b80 20 client.26114 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:21.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.124+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5556850c9fc0 0x5556850cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.124+0000 7fe9ecd06b80 4 client.26114 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:21.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.124+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1363832945 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555685083780 con 0x5556850c9fc0 2024-05-10T05:37:21.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.124+0000 7fe9cde00700 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5556850c9fc0 0x5556850cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.129+0000 7fe9afe00700 10 client.26114 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:21.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.129+0000 7fe9cde00700 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5556850c9fc0 0x5556850cc490 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9bf8d4bf6c0121 server_cookie=1d6c23155736848c in_seq=0 out_seq=0 2024-05-10T05:37:21.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.129+0000 7fe9afe00700 1 -- 172.21.15.131:0/1363832945 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555685083780 con 0x5556850c9fc0 2024-05-10T05:37:21.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.129+0000 7fe9afe00700 10 client.26114 handle_command_reply: tid=0 2024-05-10T05:37:21.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.130+0000 7fe9ecd06b80 1 client.26114 shutdown 2024-05-10T05:37:21.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.130+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1363832945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b5ec0 msgr2=0x5556850c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:21.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.130+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b5ec0 0x5556850c5bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00e720 tx=0x7fe9bc00e750 comp rx=0 tx=0).stop 2024-05-10T05:37:21.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.130+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1363832945 shutdown_connections 2024-05-10T05:37:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.130+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5556850c9fc0 0x5556850cc490 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.130+0000 7fe9afe00700 0 client.26114 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.130+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b5ec0 0x5556850c5bb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.130+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b46a0 0x5556850be710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.130+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/1363832945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x5556850bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.131+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1363832945 >> 172.21.15.131:0/1363832945 conn(0x555685016a60 msgr2=0x5556850aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.131+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1363832945 shutdown_connections 2024-05-10T05:37:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.131+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/1363832945 wait complete. 2024-05-10T05:37:21.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.132+0000 7fe9ecd06b80 20 client.26114 trim_cache size 0 max 16384 2024-05-10T05:37:21.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.191+0000 7fe9ecd06b80 1 Processor -- start 2024-05-10T05:37:21.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.191+0000 7fe9ecd06b80 1 -- start start 2024-05-10T05:37:21.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.191+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.191+0000 7fe9cde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.192+0000 7fe9cde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56518/0 (socket says 172.21.15.131:56518) 2024-05-10T05:37:21.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.192+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x5556850c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.192+0000 7fe9cde00700 1 -- 172.21.15.131:0/3907502706 learned_addr learned my addr 172.21.15.131:0/3907502706 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:21.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.192+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x5556850bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.192+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3907502706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556850b0190 con 0x5556850b5570 2024-05-10T05:37:21.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.192+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3907502706 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556850afd20 con 0x5556850b46a0 2024-05-10T05:37:21.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.192+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3907502706 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556850c9ce0 con 0x5556850b5ec0 2024-05-10T05:37:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.192+0000 7fe9ce800700 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x5556850bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.192+0000 7fe9cd400700 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x5556850c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.193+0000 7fe9ce800700 1 -- 172.21.15.131:0/3907502706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 msgr2=0x5556850c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.193+0000 7fe9ce800700 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.193+0000 7fe9ce800700 1 -- 172.21.15.131:0/3907502706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 msgr2=0x5556850c3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:21.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.193+0000 7fe9ce800700 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x5556850c3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.193+0000 7fe9ce800700 1 -- 172.21.15.131:0/3907502706 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555684e01b60 con 0x5556850b5ec0 2024-05-10T05:37:21.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.193+0000 7fe9cd400700 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x5556850c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:21.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.194+0000 7fe9ce800700 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x5556850bd930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc005950 tx=0x7fe9bc024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd7e1ffef0d81ffc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:21.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.195+0000 7fe9bbe00700 1 -- 172.21.15.131:0/3907502706 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0056c0 con 0x5556850b5ec0 2024-05-10T05:37:21.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.195+0000 7fe9bbe00700 1 -- 172.21.15.131:0/3907502706 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555684f2d270 con 0x5556850b5ec0 2024-05-10T05:37:21.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.195+0000 7fe9bbe00700 1 -- 172.21.15.131:0/3907502706 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0056c0 con 0x5556850b5ec0 2024-05-10T05:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.196+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3907502706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 msgr2=0x5556850bd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.196+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x5556850bd930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc005950 tx=0x7fe9bc024000 comp rx=0 tx=0).stop 2024-05-10T05:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.197+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3907502706 shutdown_connections 2024-05-10T05:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.197+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x5556850bd930 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.197+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x5556850c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.197+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3907502706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x5556850c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.197+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3907502706 >> 172.21.15.131:0/3907502706 conn(0x555685016a60 msgr2=0x5556850b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.197+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3907502706 shutdown_connections 2024-05-10T05:37:21.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.197+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3907502706 wait complete. 2024-05-10T05:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.198+0000 7fe9ecd06b80 1 -- start start 2024-05-10T05:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.199+0000 7fe9ecd06b80 1 Processor -- start 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.201+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x55568511a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.201+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x55568511c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.201+0000 7fe9ecd06b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x55568511eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.202+0000 7fe9cd400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x55568511c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.202+0000 7fe9cd400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x55568511c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44228/0 (socket says 172.21.15.131:44228) 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.202+0000 7fe9cde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x55568511a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.202+0000 7fe9cde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x55568511a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56534/0 (socket says 172.21.15.131:56534) 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.202+0000 7fe9ce800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x55568511eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.202+0000 7fe9ce800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x55568511eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59730/0 (socket says 172.21.15.131:59730) 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.202+0000 7fe9cd400700 1 -- 172.21.15.131:0/3985422445 learned_addr learned my addr 172.21.15.131:0/3985422445 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.203+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568511f400 con 0x5556850b5570 2024-05-10T05:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.203+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568511f540 con 0x5556850b46a0 2024-05-10T05:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.203+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568511f680 con 0x5556850b5ec0 2024-05-10T05:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.203+0000 7fe9ecd06b80 10 client.0 _ll_register_callbacks cb 0x555684f79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.204+0000 7fe9cd400700 1 -- 172.21.15.131:0/3985422445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 msgr2=0x55568511eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.204+0000 7fe9cd400700 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x55568511eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.204+0000 7fe9cd400700 1 -- 172.21.15.131:0/3985422445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 msgr2=0x55568511a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.204+0000 7fe9cd400700 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x55568511a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.204+0000 7fe9cd400700 1 -- 172.21.15.131:0/3985422445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d01af090 con 0x5556850b5570 2024-05-10T05:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.205+0000 7fe9cde00700 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x55568511a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.205+0000 7fe9ce800700 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x55568511eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.205+0000 7fe9cd400700 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x55568511c970 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0097e30 tx=0x7fe9c0021b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d90e6ac71eebc68 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.205+0000 7fe9afe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:37:21.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.206+0000 7fe9afe00700 1 -- 172.21.15.131:0/3985422445 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00cbea0 con 0x5556850b5570 2024-05-10T05:37:21.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.207+0000 7fe9afe00700 1 -- 172.21.15.131:0/3985422445 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe9c009bac0 con 0x5556850b5570 2024-05-10T05:37:21.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.207+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9bc019c50 con 0x5556850b5570 2024-05-10T05:37:21.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.207+0000 7fe9afe00700 1 -- 172.21.15.131:0/3985422445 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9c009b3c0 con 0x5556850b5570 2024-05-10T05:37:21.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.209+0000 7fe9ecd06b80 10 client.16038 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:21.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.209+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fe9bc0099b0 con 0x5556850b5570 2024-05-10T05:37:21.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.210+0000 7fe9afe00700 1 -- 172.21.15.131:0/3985422445 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9bc019c50 con 0x5556850b5570 2024-05-10T05:37:21.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.210+0000 7fe9afe00700 1 -- 172.21.15.131:0/3985422445 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe9c00a3460 con 0x5556850b5570 2024-05-10T05:37:21.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.210+0000 7fe9ecd06b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:21.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.211+0000 7fe9ecd06b80 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:37:21.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.211+0000 7fe9ecd06b80 20 client.16038 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:21.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.211+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555685120c40 0x555685123060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:21.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.211+0000 7fe9ecd06b80 4 client.16038 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:37:21.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.211+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555684fccec0 con 0x555685120c40 2024-05-10T05:37:21.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.212+0000 7fe9cde00700 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555685120c40 0x555685123060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:21.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.213+0000 7fe9afe00700 10 client.16038 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:21.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.213+0000 7fe9cde00700 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555685120c40 0x555685123060 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc656f30dd6e0db server_cookie=c331d845ee176d4c in_seq=0 out_seq=0 2024-05-10T05:37:21.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9afe00700 1 -- 172.21.15.131:0/3985422445 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555684fccec0 con 0x555685120c40 2024-05-10T05:37:21.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9afe00700 10 client.16038 handle_command_reply: tid=0 2024-05-10T05:37:21.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9ecd06b80 1 client.16038 shutdown 2024-05-10T05:37:21.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 msgr2=0x55568511c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:21.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x55568511c970 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0097e30 tx=0x7fe9c0021b40 comp rx=0 tx=0).stop 2024-05-10T05:37:21.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 shutdown_connections 2024-05-10T05:37:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555685120c40 0x555685123060 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556850b5ec0 0x55568511eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556850b46a0 0x55568511a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9ecd06b80 1 --2- 172.21.15.131:0/3985422445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556850b5570 0x55568511c970 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 >> 172.21.15.131:0/3985422445 conn(0x555685016a60 msgr2=0x5556850bd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.214+0000 7fe9afe00700 0 client.16038 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.215+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 shutdown_connections 2024-05-10T05:37:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.215+0000 7fe9ecd06b80 1 -- 172.21.15.131:0/3985422445 wait complete. 2024-05-10T05:37:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.215+0000 7fe9ecd06b80 20 client.16038 trim_cache size 0 max 16384 2024-05-10T05:37:21.217 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:37:21.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.216+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9b40a1240 msgr2=0x7fe9b40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:21.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.216+0000 7fe9d4e00700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9b40a1240 0x7fe9b40a3750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0074e30 tx=0x7fe9c8009040 comp rx=0 tx=0).stop 2024-05-10T05:37:21.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.216+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9d0075140 msgr2=0x7fe9d010b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:21.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.216+0000 7fe9d4e00700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9d0075140 0x7fe9d010b270 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0000c00 tx=0x7fe9c0009580 comp rx=0 tx=0).stop 2024-05-10T05:37:21.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.217+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 shutdown_connections 2024-05-10T05:37:21.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.217+0000 7fe9d4e00700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9b40a1240 0x7fe9b40a3750 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.217+0000 7fe9d4e00700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d010e220 0x7fe9d01ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.217+0000 7fe9d4e00700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9d0073b40 0x7fe9d010ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.217+0000 7fe9d4e00700 1 --2- 172.21.15.131:0/3476535260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9d0075140 0x7fe9d010b270 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:21.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.217+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 >> 172.21.15.131:0/3476535260 conn(0x7fe9d00fd1d0 msgr2=0x7fe9d01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:21.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.217+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 shutdown_connections 2024-05-10T05:37:21.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:21.217+0000 7fe9d4e00700 1 -- 172.21.15.131:0/3476535260 wait complete. 2024-05-10T05:37:22.273 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:37:22.274 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:37:22.467 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:22 smithi131 ceph-mon[37264]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 239 MiB/s wr, 74 op/s 2024-05-10T05:37:22.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:22 smithi157 ceph-mon[78427]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 239 MiB/s wr, 74 op/s 2024-05-10T05:37:22.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:22 smithi142 ceph-mon[77667]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 239 MiB/s wr, 74 op/s 2024-05-10T05:37:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.100+0000 7f12e6400700 1 -- 172.21.15.131:0/847971237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0109300 msgr2=0x7f12e0109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.100+0000 7f12e6400700 1 --2- 172.21.15.131:0/847971237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0109300 0x7f12e0109710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009990 tx=0x7f12d0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.101+0000 7f12e6400700 1 -- 172.21.15.131:0/847971237 shutdown_connections 2024-05-10T05:37:23.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.101+0000 7f12e6400700 1 --2- 172.21.15.131:0/847971237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12e0109c50 0x7f12e010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.101+0000 7f12e6400700 1 --2- 172.21.15.131:0/847971237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e0108630 0x7f12e0108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.101+0000 7f12e6400700 1 --2- 172.21.15.131:0/847971237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0109300 0x7f12e0109710 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.101+0000 7f12e6400700 1 -- 172.21.15.131:0/847971237 >> 172.21.15.131:0/847971237 conn(0x7f12e006ea00 msgr2=0x7f12e0070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:23.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.104+0000 7f12e6400700 1 -- 172.21.15.131:0/847971237 shutdown_connections 2024-05-10T05:37:23.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.104+0000 7f12e6400700 1 -- 172.21.15.131:0/847971237 wait complete. 2024-05-10T05:37:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.108+0000 7f12e6400700 1 Processor -- start 2024-05-10T05:37:23.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.110+0000 7f12e6400700 1 -- start start 2024-05-10T05:37:23.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.111+0000 7f12e6400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e0108630 0x7f12e01a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.112+0000 7f12df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e0108630 0x7f12e01a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.112+0000 7f12df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e0108630 0x7f12e01a6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:33396/0 (socket says 172.21.15.131:33396) 2024-05-10T05:37:23.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.113+0000 7f12df400700 1 -- 172.21.15.131:0/931680071 learned_addr learned my addr 172.21.15.131:0/931680071 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:23.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.113+0000 7f12e6400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12e0109300 0x7f12e01a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.113+0000 7f12e6400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0109c50 0x7f12e01ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.113+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e01ae850 con 0x7f12e0109300 2024-05-10T05:37:23.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.114+0000 7f12dea00700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12e0109300 0x7f12e01a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.114+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e01ae990 con 0x7f12e0108630 2024-05-10T05:37:23.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.114+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e01aeb20 con 0x7f12e0109c50 2024-05-10T05:37:23.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.114+0000 7f12df400700 1 -- 172.21.15.131:0/931680071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0109c50 msgr2=0x7f12e01ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.114+0000 7f12dfe00700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0109c50 0x7f12e01ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.114+0000 7f12df400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0109c50 0x7f12e01ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.114+0000 7f12df400700 1 -- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12e0109300 msgr2=0x7f12e01a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.115+0000 7f12df400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12e0109300 0x7f12e01a72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.115+0000 7f12df400700 1 -- 172.21.15.131:0/931680071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d00095f0 con 0x7f12e0108630 2024-05-10T05:37:23.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.115+0000 7f12dea00700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12e0109300 0x7f12e01a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:23.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.115+0000 7f12dfe00700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0109c50 0x7f12e01ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:23.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.115+0000 7f12df400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e0108630 0x7f12e01a6db0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00d950 tx=0x7f12cc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:23.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.117+0000 7f12d7e00700 1 -- 172.21.15.131:0/931680071 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc009930 con 0x7f12e0108630 2024-05-10T05:37:23.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.117+0000 7f12d7e00700 1 -- 172.21.15.131:0/931680071 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12cc010460 con 0x7f12e0108630 2024-05-10T05:37:23.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.117+0000 7f12d7e00700 1 -- 172.21.15.131:0/931680071 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc00f620 con 0x7f12e0108630 2024-05-10T05:37:23.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.117+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e01aee00 con 0x7f12e0108630 2024-05-10T05:37:23.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.118+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e01af230 con 0x7f12e0108630 2024-05-10T05:37:23.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.124+0000 7f12d7e00700 1 -- 172.21.15.131:0/931680071 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f12cc00f780 con 0x7f12e0108630 2024-05-10T05:37:23.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.126+0000 7f12d7e00700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12c40a1380 0x7f12c40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.127+0000 7f12d7e00700 1 -- 172.21.15.131:0/931680071 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f12cc0cc270 con 0x7f12e0108630 2024-05-10T05:37:23.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.128+0000 7f12dea00700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12c40a1380 0x7f12c40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.128+0000 7f12fe298b80 1 Processor -- start 2024-05-10T05:37:23.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.129+0000 7f12fe298b80 1 -- start start 2024-05-10T05:37:23.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.129+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.129+0000 7f12dea00700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12c40a1380 0x7f12c40a3890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f12d0006010 tx=0x7f12d001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:23.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.130+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 0x5653d16e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.130+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4f50 0x5653d16eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.130+0000 7f12fe298b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d15f91d0 con 0x5653d16e4600 2024-05-10T05:37:23.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.130+0000 7f12fe298b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d15f6b90 con 0x5653d16e3730 2024-05-10T05:37:23.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.130+0000 7f12fe298b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d15f2480 con 0x5653d16e4f50 2024-05-10T05:37:23.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.131+0000 7f12df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.131+0000 7f12df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16e3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:33398/0 (socket says 172.21.15.131:33398) 2024-05-10T05:37:23.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.131+0000 7f12df400700 1 -- 172.21.15.131:0/1485381461 learned_addr learned my addr 172.21.15.131:0/1485381461 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:23.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.131+0000 7f12dea00700 1 --2- 172.21.15.131:0/1485381461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 0x5653d16e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.131+0000 7f12dfe00700 1 --2- 172.21.15.131:0/1485381461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4f50 0x5653d16eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.132+0000 7f12dea00700 1 -- 172.21.15.131:0/1485381461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4f50 msgr2=0x5653d16eb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.132+0000 7f12dea00700 1 --2- 172.21.15.131:0/1485381461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4f50 0x5653d16eb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.132+0000 7f12dea00700 1 -- 172.21.15.131:0/1485381461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 msgr2=0x5653d16e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.132+0000 7f12dea00700 1 --2- 172.21.15.131:0/1485381461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16e3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.132+0000 7f12dea00700 1 -- 172.21.15.131:0/1485381461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d1473430 con 0x5653d16e4600 2024-05-10T05:37:23.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.132+0000 7f12df400700 1 --2- 172.21.15.131:0/1485381461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:23.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.132+0000 7f12dea00700 1 --2- 172.21.15.131:0/1485381461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 0x5653d16e4a10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f12e01a7e50 tx=0x7f12d001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e890ffc36ce9e87 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:23.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.133+0000 7f12d4c00700 1 -- 172.21.15.131:0/1485381461 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d002b070 con 0x5653d16e4600 2024-05-10T05:37:23.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.133+0000 7f12d4c00700 1 -- 172.21.15.131:0/1485381461 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12d001e040 con 0x5653d16e4600 2024-05-10T05:37:23.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.133+0000 7f12d4c00700 1 -- 172.21.15.131:0/1485381461 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0005330 con 0x5653d16e4600 2024-05-10T05:37:23.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.135+0000 7f12fe298b80 1 -- 172.21.15.131:0/1485381461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 msgr2=0x5653d16e4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.135+0000 7f12fe298b80 1 --2- 172.21.15.131:0/1485381461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 0x5653d16e4a10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f12e01a7e50 tx=0x7f12d001f000 comp rx=0 tx=0).stop 2024-05-10T05:37:23.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.135+0000 7f12fe298b80 1 -- 172.21.15.131:0/1485381461 shutdown_connections 2024-05-10T05:37:23.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.135+0000 7f12fe298b80 1 --2- 172.21.15.131:0/1485381461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4f50 0x5653d16eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.135+0000 7f12fe298b80 1 --2- 172.21.15.131:0/1485381461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.135+0000 7f12fe298b80 1 --2- 172.21.15.131:0/1485381461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 0x5653d16e4a10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.135+0000 7f12fe298b80 1 -- 172.21.15.131:0/1485381461 >> 172.21.15.131:0/1485381461 conn(0x5653d1634ae0 msgr2=0x5653d16cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:23.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.135+0000 7f12fe298b80 1 -- 172.21.15.131:0/1485381461 shutdown_connections 2024-05-10T05:37:23.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.135+0000 7f12fe298b80 1 -- 172.21.15.131:0/1485381461 wait complete. 2024-05-10T05:37:23.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.136+0000 7f12fe298b80 1 -- start start 2024-05-10T05:37:23.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.136+0000 7f12fe298b80 1 Processor -- start 2024-05-10T05:37:23.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.138+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e3730 0x5653d16ed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.138+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 0x5653d16edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.138+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e4f50 0x5653d16f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.138+0000 7f12fe298b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d158e5e0 con 0x5653d16e4600 2024-05-10T05:37:23.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.138+0000 7f12fe298b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d158e280 con 0x5653d16e4f50 2024-05-10T05:37:23.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.138+0000 7f12fe298b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d1475720 con 0x5653d16e3730 2024-05-10T05:37:23.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.138+0000 7f12fe298b80 10 client.0 _ll_register_callbacks cb 0x5653d1597bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:23.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.139+0000 7f12df400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e3730 0x5653d16ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.139+0000 7f12df400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e3730 0x5653d16ed750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43218/0 (socket says 172.21.15.131:43218) 2024-05-10T05:37:23.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.139+0000 7f12df400700 1 -- 172.21.15.131:0/1810242119 learned_addr learned my addr 172.21.15.131:0/1810242119 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.139+0000 7f12dea00700 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 0x5653d16edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.140+0000 7f12dfe00700 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e4f50 0x5653d16f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.140+0000 7f12dea00700 1 -- 172.21.15.131:0/1810242119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e3730 msgr2=0x5653d16ed750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.140+0000 7f12dea00700 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e3730 0x5653d16ed750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.140+0000 7f12dea00700 1 -- 172.21.15.131:0/1810242119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e4f50 msgr2=0x5653d16f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.140+0000 7f12dea00700 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e4f50 0x5653d16f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.140+0000 7f12dea00700 1 -- 172.21.15.131:0/1810242119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e01af230 con 0x5653d16e4600 2024-05-10T05:37:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.140+0000 7f12dea00700 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 0x5653d16edc90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f12e01a7e50 tx=0x7f12d000fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3262676a6b5e1b44 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.141+0000 7f12c9600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:37:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.141+0000 7f12c9600700 1 -- 172.21.15.131:0/1810242119 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d002b040 con 0x5653d16e4600 2024-05-10T05:37:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.141+0000 7f12c9600700 1 -- 172.21.15.131:0/1810242119 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f12d00054e0 con 0x5653d16e4600 2024-05-10T05:37:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.141+0000 7f12fe298b80 1 -- 172.21.15.131:0/1810242119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653d154b2f0 con 0x5653d16e4600 2024-05-10T05:37:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.141+0000 7f12c9600700 1 -- 172.21.15.131:0/1810242119 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12d00056a0 con 0x5653d16e4600 2024-05-10T05:37:23.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.141+0000 7f12fe298b80 10 client.16047 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:23.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.141+0000 7f12fe298b80 1 -- 172.21.15.131:0/1810242119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f12e010efc0 con 0x5653d16e4600 2024-05-10T05:37:23.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.142+0000 7f12c9600700 1 -- 172.21.15.131:0/1810242119 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653d154b2f0 con 0x5653d16e4600 2024-05-10T05:37:23.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.143+0000 7f12c9600700 1 -- 172.21.15.131:0/1810242119 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f12d0024670 con 0x5653d16e4600 2024-05-10T05:37:23.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.143+0000 7f12fe298b80 10 client.16047 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:23.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.143+0000 7f12fe298b80 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:37:23.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.143+0000 7f12fe298b80 20 client.16047 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:23.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.143+0000 7f12fe298b80 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5653d16f8fe0 0x5653d16fb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.143+0000 7f12fe298b80 4 client.16047 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:23.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.143+0000 7f12fe298b80 1 -- 172.21.15.131:0/1810242119 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653d16a1800 con 0x5653d16f8fe0 2024-05-10T05:37:23.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.144+0000 7f12df400700 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5653d16f8fe0 0x5653d16fb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.144+0000 7f12df400700 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5653d16f8fe0 0x5653d16fb430 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b20d4c0c48536 server_cookie=93516033b9c2cdb0 in_seq=0 out_seq=0 2024-05-10T05:37:23.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.144+0000 7f12c9600700 10 client.16047 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:23.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.145+0000 7f12c9600700 1 -- 172.21.15.131:0/1810242119 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653d16a1800 con 0x5653d16f8fe0 2024-05-10T05:37:23.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.145+0000 7f12c9600700 10 client.16047 handle_command_reply: tid=0 2024-05-10T05:37:23.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.145+0000 7f12fe298b80 1 client.16047 shutdown 2024-05-10T05:37:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.145+0000 7f12fe298b80 1 -- 172.21.15.131:0/1810242119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 msgr2=0x5653d16edc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.145+0000 7f12fe298b80 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 0x5653d16edc90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f12e01a7e50 tx=0x7f12d000fca0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.146+0000 7f12fe298b80 1 -- 172.21.15.131:0/1810242119 shutdown_connections 2024-05-10T05:37:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.146+0000 7f12fe298b80 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5653d16f8fe0 0x5653d16fb430 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.146+0000 7f12fe298b80 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e4f50 0x5653d16f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.146+0000 7f12fe298b80 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e3730 0x5653d16ed750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.146+0000 7f12fe298b80 1 --2- 172.21.15.131:0/1810242119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e4600 0x5653d16edc90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.146+0000 7f12fe298b80 1 -- 172.21.15.131:0/1810242119 >> 172.21.15.131:0/1810242119 conn(0x5653d1634ae0 msgr2=0x5653d16cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:23.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.146+0000 7f12c9600700 0 client.16047 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:23.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.146+0000 7f12fe298b80 1 -- 172.21.15.131:0/1810242119 shutdown_connections 2024-05-10T05:37:23.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.146+0000 7f12fe298b80 1 -- 172.21.15.131:0/1810242119 wait complete. 2024-05-10T05:37:23.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.146+0000 7f12fe298b80 20 client.16047 trim_cache size 0 max 16384 2024-05-10T05:37:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.165+0000 7f12fe298b80 1 Processor -- start 2024-05-10T05:37:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.165+0000 7f12fe298b80 1 -- start start 2024-05-10T05:37:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.165+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d1634ae0 0x5653d16faff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.166+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16fb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.166+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4600 0x5653d16f4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.166+0000 7f12fe298b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d16e8a70 con 0x5653d1634ae0 2024-05-10T05:37:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.166+0000 7f12fe298b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d16c7860 con 0x5653d16e3730 2024-05-10T05:37:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.166+0000 7f12fe298b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d16e7eb0 con 0x5653d16e4600 2024-05-10T05:37:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.166+0000 7f12dea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.166+0000 7f12dea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16fb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:33410/0 (socket says 172.21.15.131:33410) 2024-05-10T05:37:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.166+0000 7f12dea00700 1 -- 172.21.15.131:0/47161883 learned_addr learned my addr 172.21.15.131:0/47161883 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.166+0000 7f12dfe00700 1 --2- 172.21.15.131:0/47161883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4600 0x5653d16f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.166+0000 7f12df400700 1 --2- 172.21.15.131:0/47161883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d1634ae0 0x5653d16faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.167+0000 7f12dea00700 1 -- 172.21.15.131:0/47161883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4600 msgr2=0x5653d16f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.167+0000 7f12dea00700 1 --2- 172.21.15.131:0/47161883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4600 0x5653d16f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.167+0000 7f12dea00700 1 -- 172.21.15.131:0/47161883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d1634ae0 msgr2=0x5653d16faff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.167+0000 7f12dea00700 1 --2- 172.21.15.131:0/47161883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d1634ae0 0x5653d16faff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.167+0000 7f12dea00700 1 -- 172.21.15.131:0/47161883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d141fbe0 con 0x5653d16e3730 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.167+0000 7f12dea00700 1 --2- 172.21.15.131:0/47161883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16fb530 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12d000be70 tx=0x7f12d000f570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a27f9051bfbc1fee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.167+0000 7f12d5600700 1 -- 172.21.15.131:0/47161883 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d00052e0 con 0x5653d16e3730 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.168+0000 7f12d5600700 1 -- 172.21.15.131:0/47161883 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12d0005440 con 0x5653d16e3730 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.168+0000 7f12d5600700 1 -- 172.21.15.131:0/47161883 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d00399d0 con 0x5653d16e3730 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.168+0000 7f12fe298b80 1 -- 172.21.15.131:0/47161883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 msgr2=0x5653d16fb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.168+0000 7f12fe298b80 1 --2- 172.21.15.131:0/47161883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16fb530 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12d000be70 tx=0x7f12d000f570 comp rx=0 tx=0).stop 2024-05-10T05:37:23.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.169+0000 7f12fe298b80 1 -- 172.21.15.131:0/47161883 shutdown_connections 2024-05-10T05:37:23.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.169+0000 7f12fe298b80 1 --2- 172.21.15.131:0/47161883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d1634ae0 0x5653d16faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.169+0000 7f12fe298b80 1 --2- 172.21.15.131:0/47161883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4600 0x5653d16f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.169+0000 7f12fe298b80 1 --2- 172.21.15.131:0/47161883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d16e3730 0x5653d16fb530 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.169+0000 7f12fe298b80 1 -- 172.21.15.131:0/47161883 >> 172.21.15.131:0/47161883 conn(0x5653d16c94d0 msgr2=0x5653d16e8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:23.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.169+0000 7f12fe298b80 1 -- 172.21.15.131:0/47161883 shutdown_connections 2024-05-10T05:37:23.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.169+0000 7f12fe298b80 1 -- 172.21.15.131:0/47161883 wait complete. 2024-05-10T05:37:23.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.169+0000 7f12fe298b80 1 -- start start 2024-05-10T05:37:23.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.170+0000 7f12fe298b80 1 Processor -- start 2024-05-10T05:37:23.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.172+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d1634ae0 0x5653d16f2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.172+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e3730 0x5653d16f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.172+0000 7f12fe298b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4600 0x5653d16f3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.172+0000 7f12fe298b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d16f3e80 con 0x5653d16e3730 2024-05-10T05:37:23.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.172+0000 7f12fe298b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d16f3ff0 con 0x5653d1634ae0 2024-05-10T05:37:23.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.172+0000 7f12fe298b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d16f41b0 con 0x5653d16e4600 2024-05-10T05:37:23.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.172+0000 7f12fe298b80 10 client.0 _ll_register_callbacks cb 0x5653d1597bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:23.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.173+0000 7f12dea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e3730 0x5653d16f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.173+0000 7f12df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d1634ae0 0x5653d16f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.173+0000 7f12dea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e3730 0x5653d16f2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55084/0 (socket says 172.21.15.131:55084) 2024-05-10T05:37:23.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.173+0000 7f12df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d1634ae0 0x5653d16f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:33412/0 (socket says 172.21.15.131:33412) 2024-05-10T05:37:23.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.173+0000 7f12dea00700 1 -- 172.21.15.131:0/522882729 learned_addr learned my addr 172.21.15.131:0/522882729 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:23.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.173+0000 7f12dea00700 1 -- 172.21.15.131:0/522882729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4600 msgr2=0x5653d16f3940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:37:23.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.173+0000 7f12dea00700 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4600 0x5653d16f3940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.173+0000 7f12dea00700 1 -- 172.21.15.131:0/522882729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d1634ae0 msgr2=0x5653d16f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.173+0000 7f12dea00700 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d1634ae0 0x5653d16f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.173+0000 7f12dea00700 1 -- 172.21.15.131:0/522882729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e010efc0 con 0x5653d16e3730 2024-05-10T05:37:23.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.174+0000 7f12dea00700 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e3730 0x5653d16f2a60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12d0024e20 tx=0x7f12d000fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc890c9ca13b424 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:23.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.174+0000 7f12c9600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:37:23.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.174+0000 7f12c9600700 1 -- 172.21.15.131:0/522882729 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d002b030 con 0x5653d16e3730 2024-05-10T05:37:23.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.174+0000 7f12c9600700 1 -- 172.21.15.131:0/522882729 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f12d00398d0 con 0x5653d16e3730 2024-05-10T05:37:23.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.174+0000 7f12fe298b80 1 -- 172.21.15.131:0/522882729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12d001e040 con 0x5653d16e3730 2024-05-10T05:37:23.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.174+0000 7f12c9600700 1 -- 172.21.15.131:0/522882729 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12d0039a90 con 0x5653d16e3730 2024-05-10T05:37:23.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.174+0000 7f12fe298b80 10 client.16053 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:23.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.175+0000 7f12fe298b80 1 -- 172.21.15.131:0/522882729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5653d15b85e0 con 0x5653d16e3730 2024-05-10T05:37:23.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.175+0000 7f12c9600700 1 -- 172.21.15.131:0/522882729 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12d001e040 con 0x5653d16e3730 2024-05-10T05:37:23.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.175+0000 7f12c9600700 1 -- 172.21.15.131:0/522882729 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f12d0044cf0 con 0x5653d16e3730 2024-05-10T05:37:23.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.175+0000 7f12fe298b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:23.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.175+0000 7f12fe298b80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:37:23.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.175+0000 7f12fe298b80 20 client.16053 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:23.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.176+0000 7f12fe298b80 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5653d16e4f50 0x5653d17519e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.176+0000 7f12fe298b80 4 client.16053 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:37:23.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.176+0000 7f12fe298b80 1 -- 172.21.15.131:0/522882729 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5653d15eaf40 con 0x5653d16e4f50 2024-05-10T05:37:23.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.176+0000 7f12df400700 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5653d16e4f50 0x5653d17519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.178+0000 7f12df400700 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5653d16e4f50 0x5653d17519e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b21aaf7726ebca server_cookie=539cc679e4e329a1 in_seq=0 out_seq=0 2024-05-10T05:37:23.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.178+0000 7f12c9600700 10 client.16053 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:23.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.180+0000 7f12c9600700 1 -- 172.21.15.131:0/522882729 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5653d15eaf40 con 0x5653d16e4f50 2024-05-10T05:37:23.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.180+0000 7f12c9600700 10 client.16053 handle_command_reply: tid=0 2024-05-10T05:37:23.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.180+0000 7f12fe298b80 1 client.16053 shutdown 2024-05-10T05:37:23.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12fe298b80 1 -- 172.21.15.131:0/522882729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e3730 msgr2=0x5653d16f2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12fe298b80 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e3730 0x5653d16f2a60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12d0024e20 tx=0x7f12d000fca0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12fe298b80 1 -- 172.21.15.131:0/522882729 shutdown_connections 2024-05-10T05:37:23.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12fe298b80 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5653d16e4f50 0x5653d17519e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12fe298b80 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653d16e4600 0x5653d16f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12fe298b80 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5653d1634ae0 0x5653d16f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12fe298b80 1 --2- 172.21.15.131:0/522882729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d16e3730 0x5653d16f2a60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12fe298b80 1 -- 172.21.15.131:0/522882729 >> 172.21.15.131:0/522882729 conn(0x5653d16c94d0 msgr2=0x5653d16e8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:23.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12c9600700 0 client.16053 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:23.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12fe298b80 1 -- 172.21.15.131:0/522882729 shutdown_connections 2024-05-10T05:37:23.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.181+0000 7f12fe298b80 1 -- 172.21.15.131:0/522882729 wait complete. 2024-05-10T05:37:23.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.182+0000 7f12fe298b80 20 client.16053 trim_cache size 0 max 16384 2024-05-10T05:37:23.187 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:37:23.187 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:37:23.187 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "fdd49a09-16bc-4e85-ae7b-d6d083ddaff2", 2024-05-10T05:37:23.187 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:37:23.187 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:37:23.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.183+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12c40a1380 msgr2=0x7f12c40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.183+0000 7f12e6400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12c40a1380 0x7f12c40a3890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f12d0006010 tx=0x7f12d001a040 comp rx=0 tx=0).stop 2024-05-10T05:37:23.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.183+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e0108630 msgr2=0x7f12e01a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.183+0000 7f12e6400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e0108630 0x7f12e01a6db0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00d950 tx=0x7f12cc00dc60 comp rx=0 tx=0).stop 2024-05-10T05:37:23.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.183+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 shutdown_connections 2024-05-10T05:37:23.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.183+0000 7f12e6400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12c40a1380 0x7f12c40a3890 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.183+0000 7f12e6400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0109c50 0x7f12e01ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.183+0000 7f12e6400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12e0108630 0x7f12e01a6db0 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00d950 tx=0x7f12cc00dc60 comp rx=0 tx=0).stop 2024-05-10T05:37:23.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.183+0000 7f12e6400700 1 --2- 172.21.15.131:0/931680071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12e0109300 0x7f12e01a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.183+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 >> 172.21.15.131:0/931680071 conn(0x7f12e006ea00 msgr2=0x7f12e006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:23.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.184+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 shutdown_connections 2024-05-10T05:37:23.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.184+0000 7f12e6400700 1 -- 172.21.15.131:0/931680071 wait complete. 2024-05-10T05:37:23.227 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:37:23.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:23 smithi157 ceph-mon[78427]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 236 MiB/s wr, 75 op/s 2024-05-10T05:37:23.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:23 smithi142 ceph-mon[77667]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 236 MiB/s wr, 75 op/s 2024-05-10T05:37:23.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:23 smithi131 ceph-mon[37264]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 236 MiB/s wr, 75 op/s 2024-05-10T05:37:23.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.819+0000 7f8fd8c00700 1 -- 172.21.15.131:0/974719913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd41085e0 msgr2=0x7f8fd41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.819+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/974719913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd41085e0 0x7f8fd41089f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0098f0 tx=0x7f8fcc009c00 comp rx=0 tx=0).stop 2024-05-10T05:37:23.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.831+0000 7f8fd8c00700 1 -- 172.21.15.131:0/974719913 shutdown_connections 2024-05-10T05:37:23.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.831+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/974719913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd410a0a0 0x7f8fd41109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.831+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/974719913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd41085e0 0x7f8fd41089f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.831+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/974719913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd4109750 0x7f8fd4109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.831+0000 7f8fd8c00700 1 -- 172.21.15.131:0/974719913 >> 172.21.15.131:0/974719913 conn(0x7f8fd406ea00 msgr2=0x7f8fd4070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:23.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.836+0000 7f8fd8c00700 1 -- 172.21.15.131:0/974719913 shutdown_connections 2024-05-10T05:37:23.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.841+0000 7f8fd8c00700 1 -- 172.21.15.131:0/974719913 wait complete. 2024-05-10T05:37:23.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.852+0000 7f8fd8c00700 1 Processor -- start 2024-05-10T05:37:23.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.855+0000 7f8fd8c00700 1 -- start start 2024-05-10T05:37:23.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.856+0000 7f8fd8c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd41085e0 0x7f8fd410c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.859+0000 7f8fd1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd41085e0 0x7f8fd410c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.859+0000 7f8fd1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd41085e0 0x7f8fd410c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:33436/0 (socket says 172.21.15.131:33436) 2024-05-10T05:37:23.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.859+0000 7f8fd8c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd4109750 0x7f8fd410aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.859+0000 7f8fd1e00700 1 -- 172.21.15.131:0/813140018 learned_addr learned my addr 172.21.15.131:0/813140018 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:23.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.859+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd410a0a0 0x7f8fd410afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.859+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd410b6b0 con 0x7f8fd4109750 2024-05-10T05:37:23.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.859+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd410b820 con 0x7f8fd41085e0 2024-05-10T05:37:23.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.859+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd41aa810 con 0x7f8fd410a0a0 2024-05-10T05:37:23.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.860+0000 7f8fd2800700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd410a0a0 0x7f8fd410afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.860+0000 7f8fd1400700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd4109750 0x7f8fd410aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.861+0000 7f8fd1400700 1 -- 172.21.15.131:0/813140018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd410a0a0 msgr2=0x7f8fd410afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.861+0000 7f8fd1400700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd410a0a0 0x7f8fd410afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.861+0000 7f8fd1400700 1 -- 172.21.15.131:0/813140018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd41085e0 msgr2=0x7f8fd410c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.861+0000 7f8fd1400700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd41085e0 0x7f8fd410c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.861+0000 7f8fd1400700 1 -- 172.21.15.131:0/813140018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0009520 con 0x7f8fd4109750 2024-05-10T05:37:23.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.862+0000 7f8fd1400700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd4109750 0x7f8fd410aa90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000e9f0 tx=0x7f8fc000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:23.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.862+0000 7f8fd1e00700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd41085e0 0x7f8fd410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:23.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.863+0000 7f8fcaa00700 1 -- 172.21.15.131:0/813140018 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000cb70 con 0x7f8fd4109750 2024-05-10T05:37:23.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.865+0000 7f8fcaa00700 1 -- 172.21.15.131:0/813140018 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc00044e0 con 0x7f8fd4109750 2024-05-10T05:37:23.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.865+0000 7f8fcaa00700 1 -- 172.21.15.131:0/813140018 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0005020 con 0x7f8fd4109750 2024-05-10T05:37:23.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.865+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc4009d00 con 0x7f8fd4109750 2024-05-10T05:37:23.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.865+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc009520 con 0x7f8fd4109750 2024-05-10T05:37:23.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.876+0000 7f8fcaa00700 1 -- 172.21.15.131:0/813140018 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8fc0005180 con 0x7f8fd4109750 2024-05-10T05:37:23.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.877+0000 7f8ff0be0b80 1 Processor -- start 2024-05-10T05:37:23.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.879+0000 7f8fcaa00700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8fb80a1240 0x7f8fb80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.880+0000 7f8ff0be0b80 1 -- start start 2024-05-10T05:37:23.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.880+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7295eb0 0x557ca7296300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.880+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca7296d80 0x557ca7297190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.880+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca72976d0 0x557ca729dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.880+0000 7f8ff0be0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca71b4150 con 0x557ca7295eb0 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.880+0000 7f8ff0be0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca71b1b10 con 0x557ca72976d0 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.880+0000 7f8ff0be0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca71ad400 con 0x557ca7296d80 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.880+0000 7f8fd2800700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8fb80a1240 0x7f8fb80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.881+0000 7f8fd2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca72976d0 0x557ca729dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.881+0000 7f8fd2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca72976d0 0x557ca729dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:33438/0 (socket says 172.21.15.131:33438) 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.881+0000 7f8fd2800700 1 -- 172.21.15.131:0/3359981181 learned_addr learned my addr 172.21.15.131:0/3359981181 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.881+0000 7f8fd1e00700 1 --2- 172.21.15.131:0/3359981181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7295eb0 0x557ca7296300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.881+0000 7f8fd2800700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8fb80a1240 0x7f8fb80a3750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4007830 tx=0x7f8fc40099e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.882+0000 7f8fd2800700 1 -- 172.21.15.131:0/3359981181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca7296d80 msgr2=0x557ca7297190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.882+0000 7f8fd2800700 1 --2- 172.21.15.131:0/3359981181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca7296d80 0x557ca7297190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.882+0000 7f8fd2800700 1 -- 172.21.15.131:0/3359981181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7295eb0 msgr2=0x557ca7296300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.882+0000 7f8fd2800700 1 --2- 172.21.15.131:0/3359981181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7295eb0 0x557ca7296300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.882+0000 7f8fd2800700 1 -- 172.21.15.131:0/3359981181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ca702e3b0 con 0x557ca72976d0 2024-05-10T05:37:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.883+0000 7f8fcaa00700 1 -- 172.21.15.131:0/813140018 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8fc0014070 con 0x7f8fd4109750 2024-05-10T05:37:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.883+0000 7f8fd2800700 1 --2- 172.21.15.131:0/3359981181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca72976d0 0x557ca729dfd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4018860 tx=0x7f8fc401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea02784de84a72b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.883+0000 7f8fbf000700 1 -- 172.21.15.131:0/3359981181 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4010040 con 0x557ca72976d0 2024-05-10T05:37:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.884+0000 7f8fbf000700 1 -- 172.21.15.131:0/3359981181 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc401bb30 con 0x557ca72976d0 2024-05-10T05:37:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.884+0000 7f8fbf000700 1 -- 172.21.15.131:0/3359981181 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4027bc0 con 0x557ca72976d0 2024-05-10T05:37:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.886+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3359981181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca72976d0 msgr2=0x557ca729dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.886+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3359981181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca72976d0 0x557ca729dfd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4018860 tx=0x7f8fc401c000 comp rx=0 tx=0).stop 2024-05-10T05:37:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.887+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3359981181 shutdown_connections 2024-05-10T05:37:23.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.887+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3359981181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7295eb0 0x557ca7296300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.887+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3359981181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca72976d0 0x557ca729dfd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.887+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3359981181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca7296d80 0x557ca7297190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.887+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3359981181 >> 172.21.15.131:0/3359981181 conn(0x557ca71efa60 msgr2=0x557ca7287fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.887+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3359981181 shutdown_connections 2024-05-10T05:37:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.887+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3359981181 wait complete. 2024-05-10T05:37:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.889+0000 7f8ff0be0b80 1 -- start start 2024-05-10T05:37:23.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.889+0000 7f8ff0be0b80 1 Processor -- start 2024-05-10T05:37:23.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.891+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca7295eb0 0x557ca729fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.891+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7296d80 0x557ca72a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.891+0000 7f8fd1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca7295eb0 0x557ca729fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.891+0000 7f8fd1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca7295eb0 0x557ca729fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43278/0 (socket says 172.21.15.131:43278) 2024-05-10T05:37:23.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.891+0000 7f8fd1e00700 1 -- 172.21.15.131:0/3327560285 learned_addr learned my addr 172.21.15.131:0/3327560285 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:23.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.895+0000 7f8fd1400700 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7296d80 0x557ca72a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.895+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca72976d0 0x557ca72a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.896+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca7149560 con 0x557ca72976d0 2024-05-10T05:37:23.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.896+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca7149200 con 0x557ca7296d80 2024-05-10T05:37:23.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.896+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca70306a0 con 0x557ca7295eb0 2024-05-10T05:37:23.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.896+0000 7f8ff0be0b80 10 client.0 _ll_register_callbacks cb 0x557ca7152b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:23.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.897+0000 7f8fd2800700 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca72976d0 0x557ca72a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.899+0000 7f8fd2800700 1 -- 172.21.15.131:0/3327560285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca7295eb0 msgr2=0x557ca729fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.899+0000 7f8fd2800700 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca7295eb0 0x557ca729fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.899+0000 7f8fd2800700 1 -- 172.21.15.131:0/3327560285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7296d80 msgr2=0x557ca72a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.899+0000 7f8fd2800700 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7296d80 0x557ca72a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.899+0000 7f8fd2800700 1 -- 172.21.15.131:0/3327560285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc40185d0 con 0x557ca72976d0 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.899+0000 7f8fd1400700 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7296d80 0x557ca72a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.900+0000 7f8fb3400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.900+0000 7f8fd2800700 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca72976d0 0x557ca72a71c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009cd0 tx=0x7f8fc401beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13972ea586928216 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.901+0000 7f8fb3400700 1 -- 172.21.15.131:0/3327560285 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc401b9c0 con 0x557ca72976d0 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.901+0000 7f8fb3400700 1 -- 172.21.15.131:0/3327560285 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8fc4027940 con 0x557ca72976d0 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.901+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ca7106270 con 0x557ca72976d0 2024-05-10T05:37:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.902+0000 7f8fb3400700 1 -- 172.21.15.131:0/3327560285 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fc4027b00 con 0x557ca72976d0 2024-05-10T05:37:23.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.904+0000 7f8ff0be0b80 10 client.16065 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:23.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.904+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557ca6fdab60 con 0x557ca72976d0 2024-05-10T05:37:23.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.914+0000 7f8fb3400700 1 -- 172.21.15.131:0/3327560285 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ca7106270 con 0x557ca72976d0 2024-05-10T05:37:23.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.915+0000 7f8fb3400700 1 -- 172.21.15.131:0/3327560285 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8fc403a390 con 0x557ca72976d0 2024-05-10T05:37:23.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.915+0000 7f8ff0be0b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:23.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.915+0000 7f8ff0be0b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:37:23.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.915+0000 7f8ff0be0b80 20 client.16065 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:23.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.915+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557ca72ab890 0x557ca72adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:23.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.915+0000 7f8ff0be0b80 4 client.16065 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:23.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.915+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ca725c780 con 0x557ca72ab890 2024-05-10T05:37:23.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.916+0000 7f8fd1e00700 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557ca72ab890 0x557ca72adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:23.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.918+0000 7f8fd1e00700 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557ca72ab890 0x557ca72adce0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370278e3aa9459a2 server_cookie=d7111efeedd2dfce in_seq=0 out_seq=0 2024-05-10T05:37:23.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.918+0000 7f8fb3400700 10 client.16065 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:23.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.920+0000 7f8fb3400700 1 -- 172.21.15.131:0/3327560285 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ca725c780 con 0x557ca72ab890 2024-05-10T05:37:23.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.920+0000 7f8fb3400700 10 client.16065 handle_command_reply: tid=0 2024-05-10T05:37:23.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.920+0000 7f8ff0be0b80 1 client.16065 shutdown 2024-05-10T05:37:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.921+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca72976d0 msgr2=0x557ca72a71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.921+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca72976d0 0x557ca72a71c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009cd0 tx=0x7f8fc401beb0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.921+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 shutdown_connections 2024-05-10T05:37:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.921+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557ca72ab890 0x557ca72adce0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.921+0000 7f8fb3400700 0 client.16065 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.921+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca72976d0 0x557ca72a71c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.922+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca7295eb0 0x557ca729fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.922+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/3327560285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7296d80 0x557ca72a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:23.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.922+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 >> 172.21.15.131:0/3327560285 conn(0x557ca71efa60 msgr2=0x557ca7287fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:23.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.922+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 shutdown_connections 2024-05-10T05:37:23.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.922+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/3327560285 wait complete. 2024-05-10T05:37:23.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.923+0000 7f8ff0be0b80 20 client.16065 trim_cache size 0 max 16384 2024-05-10T05:37:24.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.993+0000 7f8ff0be0b80 1 Processor -- start 2024-05-10T05:37:24.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.997+0000 7f8ff0be0b80 1 -- start start 2024-05-10T05:37:24.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.998+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 0x557ca729a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:24.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.998+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7296d80 0x557ca72a5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:24.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.998+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 0x557ca72a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:24.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.998+0000 7f8ff0be0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca72a82c0 con 0x557ca7296d80 2024-05-10T05:37:24.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.998+0000 7f8ff0be0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca72a7c20 con 0x557ca7295eb0 2024-05-10T05:37:24.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:23.998+0000 7f8ff0be0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca7297ae0 con 0x557ca72976d0 2024-05-10T05:37:24.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.001+0000 7f8fd2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 0x557ca72a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:24.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.001+0000 7f8fd2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 0x557ca72a56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43286/0 (socket says 172.21.15.131:43286) 2024-05-10T05:37:24.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.001+0000 7f8fd2800700 1 -- 172.21.15.131:0/2011637609 learned_addr learned my addr 172.21.15.131:0/2011637609 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:24.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.002+0000 7f8fd2800700 1 -- 172.21.15.131:0/2011637609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 msgr2=0x557ca729a9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:24.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.003+0000 7f8fd2800700 1 --2- 172.21.15.131:0/2011637609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 0x557ca729a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.003+0000 7f8fd2800700 1 -- 172.21.15.131:0/2011637609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7296d80 msgr2=0x557ca72a5170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:24.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.006+0000 7f8fd1400700 1 --2- 172.21.15.131:0/2011637609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7296d80 0x557ca72a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:24.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.008+0000 7f8fd2800700 1 --2- 172.21.15.131:0/2011637609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7296d80 0x557ca72a5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.008+0000 7f8fd2800700 1 -- 172.21.15.131:0/2011637609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd41a4650 con 0x557ca72976d0 2024-05-10T05:37:24.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.008+0000 7f8fd2800700 1 --2- 172.21.15.131:0/2011637609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 0x557ca72a56b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40090d0 tx=0x7f8fc4004b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5003ec7423230e34 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:24.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.009+0000 7f8fbfa00700 1 -- 172.21.15.131:0/2011637609 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc403aa10 con 0x557ca72976d0 2024-05-10T05:37:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.010+0000 7f8fbfa00700 1 -- 172.21.15.131:0/2011637609 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc401b9c0 con 0x557ca72976d0 2024-05-10T05:37:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.010+0000 7f8fbfa00700 1 -- 172.21.15.131:0/2011637609 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4030430 con 0x557ca72976d0 2024-05-10T05:37:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.014+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/2011637609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 msgr2=0x557ca72a56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.014+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/2011637609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 0x557ca72a56b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40090d0 tx=0x7f8fc4004b10 comp rx=0 tx=0).stop 2024-05-10T05:37:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.016+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/2011637609 shutdown_connections 2024-05-10T05:37:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.016+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/2011637609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 0x557ca72a56b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.016+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/2011637609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 0x557ca729a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.016+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/2011637609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7296d80 0x557ca72a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.016+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/2011637609 >> 172.21.15.131:0/2011637609 conn(0x557ca71efa60 msgr2=0x557ca72adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.025+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/2011637609 shutdown_connections 2024-05-10T05:37:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.027+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/2011637609 wait complete. 2024-05-10T05:37:24.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.028+0000 7f8ff0be0b80 1 -- start start 2024-05-10T05:37:24.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.029+0000 7f8ff0be0b80 1 Processor -- start 2024-05-10T05:37:24.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.035+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 0x557ca72a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.035+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7296d80 0x557ca72a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.036+0000 7f8ff0be0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 0x557ca72a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.036+0000 7f8ff0be0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca72a4e10 con 0x557ca7296d80 2024-05-10T05:37:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.036+0000 7f8ff0be0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca72a4f80 con 0x557ca7295eb0 2024-05-10T05:37:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.036+0000 7f8ff0be0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ca72a5140 con 0x557ca72976d0 2024-05-10T05:37:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.036+0000 7f8ff0be0b80 10 client.0 _ll_register_callbacks cb 0x557ca7152b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:24.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.036+0000 7f8fd1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 0x557ca72a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:24.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.036+0000 7f8fd1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 0x557ca72a34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:33466/0 (socket says 172.21.15.131:33466) 2024-05-10T05:37:24.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.036+0000 7f8fd1e00700 1 -- 172.21.15.131:0/379848266 learned_addr learned my addr 172.21.15.131:0/379848266 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:24.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.036+0000 7f8fd2800700 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 0x557ca72a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:24.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.038+0000 7f8fd1e00700 1 -- 172.21.15.131:0/379848266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 msgr2=0x557ca72a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.038+0000 7f8fd1e00700 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 0x557ca72a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.038+0000 7f8fd1e00700 1 -- 172.21.15.131:0/379848266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7296d80 msgr2=0x557ca72a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.038+0000 7f8fd1e00700 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7296d80 0x557ca72a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.038+0000 7f8fd1e00700 1 -- 172.21.15.131:0/379848266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ca6fdab60 con 0x557ca7295eb0 2024-05-10T05:37:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.039+0000 7f8fb3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:37:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.039+0000 7f8fd1e00700 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 0x557ca72a34b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009230 tx=0x7f8fcc009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e51826d19555b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.040+0000 7f8fb3400700 1 -- 172.21.15.131:0/379848266 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc01c040 con 0x557ca7295eb0 2024-05-10T05:37:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.040+0000 7f8fb3400700 1 -- 172.21.15.131:0/379848266 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8fcc00b560 con 0x557ca7295eb0 2024-05-10T05:37:24.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.040+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/379848266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fc401bb30 con 0x557ca7295eb0 2024-05-10T05:37:24.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.041+0000 7f8fb3400700 1 -- 172.21.15.131:0/379848266 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fcc00b720 con 0x557ca7295eb0 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.041+0000 7f8ff0be0b80 10 client.26147 fetch_fsmap learned FSMap version 28 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.041+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/379848266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557ca7173560 con 0x557ca7295eb0 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.042+0000 7f8fb3400700 1 -- 172.21.15.131:0/379848266 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc401bb30 con 0x557ca7295eb0 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.042+0000 7f8fb3400700 1 -- 172.21.15.131:0/379848266 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8fcc015020 con 0x557ca7295eb0 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.042+0000 7f8ff0be0b80 10 client.26147 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.042+0000 7f8ff0be0b80 10 client.26147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.043+0000 7f8ff0be0b80 20 client.26147 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.043+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557ca72ab890 0x557ca72911e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.043+0000 7f8ff0be0b80 4 client.26147 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.044+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/379848266 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557ca71a5ec0 con 0x557ca72ab890 2024-05-10T05:37:24.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.045+0000 7f8fd1400700 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557ca72ab890 0x557ca72911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:24.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.048+0000 7f8fb3400700 10 client.26147 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:24.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.051+0000 7f8fd1400700 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557ca72ab890 0x557ca72911e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908156dc74d85102 server_cookie=9cb76e892a98bc8b in_seq=0 out_seq=0 2024-05-10T05:37:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.051+0000 7f8fb3400700 1 -- 172.21.15.131:0/379848266 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557ca71a5ec0 con 0x557ca72ab890 2024-05-10T05:37:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.051+0000 7f8fb3400700 10 client.26147 handle_command_reply: tid=0 2024-05-10T05:37:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.051+0000 7f8ff0be0b80 1 client.26147 shutdown 2024-05-10T05:37:24.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.053+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/379848266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 msgr2=0x557ca72a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:24.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.053+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 0x557ca72a34b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009230 tx=0x7f8fcc009490 comp rx=0 tx=0).stop 2024-05-10T05:37:24.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.057+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/379848266 shutdown_connections 2024-05-10T05:37:24.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.057+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557ca72ab890 0x557ca72911e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.057+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ca72976d0 0x557ca72a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.057+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ca7295eb0 0x557ca72a34b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.057+0000 7f8ff0be0b80 1 --2- 172.21.15.131:0/379848266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ca7296d80 0x557ca72a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.057+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/379848266 >> 172.21.15.131:0/379848266 conn(0x557ca71efa60 msgr2=0x557ca72adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:24.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.057+0000 7f8fb3400700 0 client.26147 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:24.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.057+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/379848266 shutdown_connections 2024-05-10T05:37:24.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.057+0000 7f8ff0be0b80 1 -- 172.21.15.131:0/379848266 wait complete. 2024-05-10T05:37:24.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.058+0000 7f8ff0be0b80 20 client.26147 trim_cache size 0 max 16384 2024-05-10T05:37:24.059 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:37:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:37:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:37:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "fdd49a09-16bc-4e85-ae7b-d6d083ddaff2": { 2024-05-10T05:37:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:37:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "fdd49a09-16bc-4e85-ae7b-d6d083ddaff2", 2024-05-10T05:37:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:37:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:37:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:37:24.060 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:37:24.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.060+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8fb80a1240 msgr2=0x7f8fb80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:24.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.060+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8fb80a1240 0x7f8fb80a3750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4007830 tx=0x7f8fc40099e0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.060+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd4109750 msgr2=0x7f8fd410aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:24.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.060+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd4109750 0x7f8fd410aa90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000e9f0 tx=0x7f8fc000ed00 comp rx=0 tx=0).stop 2024-05-10T05:37:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.060+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 shutdown_connections 2024-05-10T05:37:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.060+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8fb80a1240 0x7f8fb80a3750 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.060+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd410a0a0 0x7f8fd410afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.060+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd41085e0 0x7f8fd410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.060+0000 7f8fd8c00700 1 --2- 172.21.15.131:0/813140018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd4109750 0x7f8fd410aa90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.060+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 >> 172.21.15.131:0/813140018 conn(0x7f8fd406ea00 msgr2=0x7f8fd410eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:24.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.061+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 shutdown_connections 2024-05-10T05:37:24.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:24.061+0000 7f8fd8c00700 1 -- 172.21.15.131:0/813140018 wait complete. 2024-05-10T05:37:24.115 INFO:tasks.cephfs.filesystem:scrub status for tag:fdd49a09-16bc-4e85-ae7b-d6d083ddaff2 - {'path': '/', 'tag': 'fdd49a09-16bc-4e85-ae7b-d6d083ddaff2', 'options': 'recursive,force'} 2024-05-10T05:37:24.465 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:24 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:37:24.465 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:24 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:37:24.466 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:24 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:37:24.466 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:24 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:37:24.466 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:24 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:37:24.466 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:24 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:37:24.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:24 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:37:24.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:24 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:37:24.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:24 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:37:24.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:24 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:37:24.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:24 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:37:24.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:24 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:37:24.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:24 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:37:24.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:24 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:37:24.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:24 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:37:24.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:24 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:37:24.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:24 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:37:24.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:24 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:37:25.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:25 smithi157 ceph-mon[78427]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-05-10T05:37:25.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:25 smithi142 ceph-mon[77667]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-05-10T05:37:25.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:25 smithi131 ceph-mon[37264]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-05-10T05:37:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:28 smithi142 ceph-mon[77667]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 220 MiB/s wr, 70 op/s 2024-05-10T05:37:28.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:28 smithi131 ceph-mon[37264]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 220 MiB/s wr, 70 op/s 2024-05-10T05:37:28.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:28 smithi157 ceph-mon[78427]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 220 MiB/s wr, 70 op/s 2024-05-10T05:37:30.398 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:30 smithi142 ceph-mon[77667]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:37:30.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:30 smithi131 ceph-mon[37264]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:37:30.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:30 smithi157 ceph-mon[78427]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:37:31.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:31 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:37:31.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:31 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:37:31.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:31 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:37:31.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:31 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:37:31.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:31 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:37:31.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:31 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:37:32.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:32 smithi131 ceph-mon[37264]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:37:32.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:32 smithi131 ceph-mon[37264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:37:32.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:32 smithi157 ceph-mon[78427]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:37:32.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:32 smithi157 ceph-mon[78427]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:37:32.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:32 smithi142 ceph-mon[77667]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:37:32.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:32 smithi142 ceph-mon[77667]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:37:34.466 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:34 smithi131 ceph-mon[37264]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 73 op/s 2024-05-10T05:37:34.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:34 smithi157 ceph-mon[78427]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 73 op/s 2024-05-10T05:37:34.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:34 smithi142 ceph-mon[77667]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 73 op/s 2024-05-10T05:37:36.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:36 smithi131 ceph-mon[37264]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 266 MiB/s wr, 80 op/s 2024-05-10T05:37:36.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:36 smithi157 ceph-mon[78427]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 266 MiB/s wr, 80 op/s 2024-05-10T05:37:36.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:36 smithi142 ceph-mon[77667]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 266 MiB/s wr, 80 op/s 2024-05-10T05:37:38.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:38 smithi131 ceph-mon[37264]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 235 MiB/s wr, 73 op/s 2024-05-10T05:37:38.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:38 smithi157 ceph-mon[78427]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 235 MiB/s wr, 73 op/s 2024-05-10T05:37:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:38 smithi142 ceph-mon[77667]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 235 MiB/s wr, 73 op/s 2024-05-10T05:37:40.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:40 smithi157 ceph-mon[78427]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 262 MiB/s wr, 78 op/s 2024-05-10T05:37:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:40 smithi142 ceph-mon[77667]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 262 MiB/s wr, 78 op/s 2024-05-10T05:37:40.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:40 smithi131 ceph-mon[37264]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 262 MiB/s wr, 78 op/s 2024-05-10T05:37:41.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:41 smithi157 ceph-mon[78427]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:37:41.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:41 smithi142 ceph-mon[77667]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:37:41.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:41 smithi131 ceph-mon[37264]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:37:44.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:44 smithi142 ceph-mon[77667]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:37:44.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:44 smithi131 ceph-mon[37264]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:37:44.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:44 smithi157 ceph-mon[78427]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:37:46.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:46 smithi131 ceph-mon[37264]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 81 op/s 2024-05-10T05:37:46.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:46 smithi157 ceph-mon[78427]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 81 op/s 2024-05-10T05:37:46.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:46 smithi142 ceph-mon[77667]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 81 op/s 2024-05-10T05:37:48.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:48 smithi131 ceph-mon[37264]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:37:48.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:48 smithi157 ceph-mon[78427]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:37:48.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:48 smithi142 ceph-mon[77667]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:37:50.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:50 smithi131 ceph-mon[37264]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 258 MiB/s wr, 76 op/s 2024-05-10T05:37:50.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:50 smithi157 ceph-mon[78427]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 258 MiB/s wr, 76 op/s 2024-05-10T05:37:50.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:50 smithi142 ceph-mon[77667]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 258 MiB/s wr, 76 op/s 2024-05-10T05:37:52.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:52 smithi131 ceph-mon[37264]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 244 MiB/s wr, 72 op/s 2024-05-10T05:37:52.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:52 smithi157 ceph-mon[78427]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 244 MiB/s wr, 72 op/s 2024-05-10T05:37:52.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:52 smithi142 ceph-mon[77667]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 244 MiB/s wr, 72 op/s 2024-05-10T05:37:54.116 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:37:54.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:54 smithi131 ceph-mon[37264]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 249 MiB/s wr, 75 op/s 2024-05-10T05:37:54.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:54 smithi157 ceph-mon[78427]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 249 MiB/s wr, 75 op/s 2024-05-10T05:37:54.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:54 smithi142 ceph-mon[77667]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 249 MiB/s wr, 75 op/s 2024-05-10T05:37:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.826+0000 7f2025600700 1 -- 172.21.15.131:0/299967044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2020109720 msgr2=0x7f2020109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.826+0000 7f2025600700 1 --2- 172.21.15.131:0/299967044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2020109720 0x7f2020109b30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f200800aad0 tx=0x7f200800ade0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.827+0000 7f2025600700 1 -- 172.21.15.131:0/299967044 shutdown_connections 2024-05-10T05:37:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.827+0000 7f2025600700 1 --2- 172.21.15.131:0/299967044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202010a0c0 0x7f2020110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.827+0000 7f2025600700 1 --2- 172.21.15.131:0/299967044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2020108630 0x7f2020108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.827+0000 7f2025600700 1 --2- 172.21.15.131:0/299967044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2020109720 0x7f2020109b30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.827+0000 7f2025600700 1 -- 172.21.15.131:0/299967044 >> 172.21.15.131:0/299967044 conn(0x7f202006ee40 msgr2=0x7f20200712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.828+0000 7f2025600700 1 -- 172.21.15.131:0/299967044 shutdown_connections 2024-05-10T05:37:54.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.829+0000 7f2025600700 1 -- 172.21.15.131:0/299967044 wait complete. 2024-05-10T05:37:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.831+0000 7f2025600700 1 Processor -- start 2024-05-10T05:37:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.832+0000 7f2025600700 1 -- start start 2024-05-10T05:37:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.834+0000 7f2025600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2020108630 0x7f20201183b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.834+0000 7f2025600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2020109720 0x7f20201188f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.834+0000 7f2025600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202010a0c0 0x7f2020119850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.835+0000 7f201ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2020108630 0x7f20201183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:54.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.835+0000 7f201ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2020108630 0x7f20201183b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:40864/0 (socket says 172.21.15.131:40864) 2024-05-10T05:37:54.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.835+0000 7f201ea00700 1 -- 172.21.15.131:0/753846652 learned_addr learned my addr 172.21.15.131:0/753846652 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:54.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.835+0000 7f201f400700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202010a0c0 0x7f2020119850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:54.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.835+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2020119e60 con 0x7f2020109720 2024-05-10T05:37:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.836+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2020119fd0 con 0x7f2020108630 2024-05-10T05:37:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.836+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202011a190 con 0x7f202010a0c0 2024-05-10T05:37:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.836+0000 7f201f400700 1 -- 172.21.15.131:0/753846652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2020108630 msgr2=0x7f20201183b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.837+0000 7f201f400700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2020108630 0x7f20201183b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.837+0000 7f201f400700 1 -- 172.21.15.131:0/753846652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2020109720 msgr2=0x7f20201188f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.837+0000 7f201f400700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2020109720 0x7f20201188f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.837+0000 7f201f400700 1 -- 172.21.15.131:0/753846652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200800a700 con 0x7f202010a0c0 2024-05-10T05:37:54.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.837+0000 7f201ea00700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2020108630 0x7f20201183b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:54.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.838+0000 7f201f400700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202010a0c0 0x7f2020119850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2018007f10 tx=0x7f201800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:54.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.838+0000 7f2017400700 1 -- 172.21.15.131:0/753846652 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800ddd0 con 0x7f202010a0c0 2024-05-10T05:37:54.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.841+0000 7f2017400700 1 -- 172.21.15.131:0/753846652 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f201800f040 con 0x7f202010a0c0 2024-05-10T05:37:54.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.841+0000 7f2017400700 1 -- 172.21.15.131:0/753846652 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20180059f0 con 0x7f202010a0c0 2024-05-10T05:37:54.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.844+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202011a470 con 0x7f202010a0c0 2024-05-10T05:37:54.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.845+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20201b7c10 con 0x7f202010a0c0 2024-05-10T05:37:54.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.863+0000 7f2017400700 1 -- 172.21.15.131:0/753846652 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2018004ad0 con 0x7f202010a0c0 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.863+0000 7f203d41db80 1 Processor -- start 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.865+0000 7f2017400700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20040a1310 0x7f20040a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.865+0000 7f203d41db80 1 -- start start 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.866+0000 7f203d41db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82d6a0 0x5588ab82daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.866+0000 7f203d41db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82e570 0x5588ab82e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.866+0000 7f203d41db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82eec0 0x5588ab8357c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.866+0000 7f201f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82eec0 0x5588ab8357c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.866+0000 7f203d41db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab754150 con 0x5588ab82eec0 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.866+0000 7f203d41db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab751b10 con 0x5588ab82d6a0 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.866+0000 7f203d41db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab74d400 con 0x5588ab82e570 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.867+0000 7f201ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82d6a0 0x5588ab82daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:54.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.867+0000 7f201ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82d6a0 0x5588ab82daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:40868/0 (socket says 172.21.15.131:40868) 2024-05-10T05:37:54.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.867+0000 7f201ea00700 1 -- 172.21.15.131:0/3874455238 learned_addr learned my addr 172.21.15.131:0/3874455238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:54.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.868+0000 7f201f400700 1 -- 172.21.15.131:0/3874455238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82e570 msgr2=0x5588ab82e980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:54.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.868+0000 7f201f400700 1 --2- 172.21.15.131:0/3874455238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82e570 0x5588ab82e980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.868+0000 7f201f400700 1 -- 172.21.15.131:0/3874455238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82d6a0 msgr2=0x5588ab82daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:54.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.868+0000 7f201f400700 1 --2- 172.21.15.131:0/3874455238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82d6a0 0x5588ab82daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.868+0000 7f201f400700 1 -- 172.21.15.131:0/3874455238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588ab5ce3b0 con 0x5588ab82eec0 2024-05-10T05:37:54.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.869+0000 7f201f400700 1 --2- 172.21.15.131:0/3874455238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82eec0 0x5588ab8357c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f20180c9a80 tx=0x7f20180cfef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e81d0e8ff8220c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:54.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.871+0000 7f200f400700 1 -- 172.21.15.131:0/3874455238 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20180ce2e0 con 0x5588ab82eec0 2024-05-10T05:37:54.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.873+0000 7f200f400700 1 -- 172.21.15.131:0/3874455238 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2018093de0 con 0x5588ab82eec0 2024-05-10T05:37:54.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.873+0000 7f200f400700 1 -- 172.21.15.131:0/3874455238 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018099400 con 0x5588ab82eec0 2024-05-10T05:37:54.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.874+0000 7f2017400700 1 -- 172.21.15.131:0/753846652 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f20180cbfb0 con 0x7f202010a0c0 2024-05-10T05:37:54.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.875+0000 7f201e000700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20040a1310 0x7f20040a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:54.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.878+0000 7f203d41db80 1 -- 172.21.15.131:0/3874455238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82eec0 msgr2=0x5588ab8357c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:54.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.878+0000 7f203d41db80 1 --2- 172.21.15.131:0/3874455238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82eec0 0x5588ab8357c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f20180c9a80 tx=0x7f20180cfef0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.882+0000 7f201e000700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20040a1310 0x7f20040a3820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f200800aaa0 tx=0x7f2008005f00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:54.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.886+0000 7f203d41db80 1 -- 172.21.15.131:0/3874455238 shutdown_connections 2024-05-10T05:37:54.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.886+0000 7f203d41db80 1 --2- 172.21.15.131:0/3874455238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82eec0 0x5588ab8357c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.886+0000 7f203d41db80 1 --2- 172.21.15.131:0/3874455238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82d6a0 0x5588ab82daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.886+0000 7f203d41db80 1 --2- 172.21.15.131:0/3874455238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82e570 0x5588ab82e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.886+0000 7f203d41db80 1 -- 172.21.15.131:0/3874455238 >> 172.21.15.131:0/3874455238 conn(0x5588ab78fa60 msgr2=0x5588ab827fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:54.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.888+0000 7f203d41db80 1 -- 172.21.15.131:0/3874455238 shutdown_connections 2024-05-10T05:37:54.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.888+0000 7f203d41db80 1 -- 172.21.15.131:0/3874455238 wait complete. 2024-05-10T05:37:54.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.889+0000 7f203d41db80 1 -- start start 2024-05-10T05:37:54.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.890+0000 7f203d41db80 1 Processor -- start 2024-05-10T05:37:54.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.896+0000 7f203d41db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82d6a0 0x5588ab859700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.896+0000 7f203d41db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82e570 0x5588ab859c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.896+0000 7f203d41db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.896+0000 7f203d41db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab6e9560 con 0x5588ab82d6a0 2024-05-10T05:37:54.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.897+0000 7f203d41db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab6e9200 con 0x5588ab82eec0 2024-05-10T05:37:54.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.897+0000 7f203d41db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab5d06a0 con 0x5588ab82e570 2024-05-10T05:37:54.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.897+0000 7f201f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:54.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.897+0000 7f201f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:40872/0 (socket says 172.21.15.131:40872) 2024-05-10T05:37:54.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.897+0000 7f201f400700 1 -- 172.21.15.131:0/1644819992 learned_addr learned my addr 172.21.15.131:0/1644819992 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:54.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.898+0000 7f203d41db80 10 client.0 _ll_register_callbacks cb 0x5588ab6f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:54.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.898+0000 7f201ea00700 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82d6a0 0x5588ab859700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:54.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.901+0000 7f201ea00700 1 -- 172.21.15.131:0/1644819992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82e570 msgr2=0x5588ab859c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:54.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.901+0000 7f201ea00700 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82e570 0x5588ab859c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.901+0000 7f201ea00700 1 -- 172.21.15.131:0/1644819992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 msgr2=0x5588ab860ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:54.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.901+0000 7f201ea00700 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.901+0000 7f201ea00700 1 -- 172.21.15.131:0/1644819992 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20201b7c10 con 0x5588ab82d6a0 2024-05-10T05:37:54.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.901+0000 7f201f400700 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:37:54.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.903+0000 7f201ea00700 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82d6a0 0x5588ab859700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2010012cf0 tx=0x7f2010009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a48ac02294d91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:54.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.903+0000 7f2003e00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:37:54.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.905+0000 7f2003e00700 1 -- 172.21.15.131:0/1644819992 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010017070 con 0x5588ab82d6a0 2024-05-10T05:37:54.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.905+0000 7f203d41db80 1 -- 172.21.15.131:0/1644819992 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588ab6a6270 con 0x5588ab82d6a0 2024-05-10T05:37:54.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.906+0000 7f2003e00700 1 -- 172.21.15.131:0/1644819992 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f201000f8f0 con 0x5588ab82d6a0 2024-05-10T05:37:54.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.906+0000 7f2003e00700 1 -- 172.21.15.131:0/1644819992 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f201000fab0 con 0x5588ab82d6a0 2024-05-10T05:37:54.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.906+0000 7f203d41db80 10 client.16074 fetch_fsmap learned FSMap version 29 2024-05-10T05:37:54.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.906+0000 7f203d41db80 1 -- 172.21.15.131:0/1644819992 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5588ab57ab60 con 0x5588ab82d6a0 2024-05-10T05:37:54.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.907+0000 7f2003e00700 1 -- 172.21.15.131:0/1644819992 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5588ab6a6270 con 0x5588ab82d6a0 2024-05-10T05:37:54.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.908+0000 7f2003e00700 1 -- 172.21.15.131:0/1644819992 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2010010d00 con 0x5588ab82d6a0 2024-05-10T05:37:54.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.909+0000 7f203d41db80 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:37:54.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.909+0000 7f203d41db80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:37:54.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.909+0000 7f203d41db80 20 client.16074 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:54.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.909+0000 7f203d41db80 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5588ab865400 0x5588ab867850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:54.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.909+0000 7f203d41db80 4 client.16074 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:54.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.909+0000 7f203d41db80 1 -- 172.21.15.131:0/1644819992 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588ab7fc780 con 0x5588ab865400 2024-05-10T05:37:54.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.911+0000 7f201e000700 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5588ab865400 0x5588ab867850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:54.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.916+0000 7f2003e00700 10 client.16074 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:54.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.916+0000 7f201e000700 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5588ab865400 0x5588ab867850 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=518361fc4b1fe2b3 server_cookie=b2f8e64a99cd75ce in_seq=0 out_seq=0 2024-05-10T05:37:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.917+0000 7f2003e00700 1 -- 172.21.15.131:0/1644819992 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588ab7fc780 con 0x5588ab865400 2024-05-10T05:37:54.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.924+0000 7f2003e00700 10 client.16074 handle_command_reply: tid=0 2024-05-10T05:37:54.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.925+0000 7f203d41db80 1 client.16074 shutdown 2024-05-10T05:37:54.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.927+0000 7f203d41db80 1 -- 172.21.15.131:0/1644819992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82d6a0 msgr2=0x5588ab859700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:54.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.927+0000 7f203d41db80 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82d6a0 0x5588ab859700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2010012cf0 tx=0x7f2010009e30 comp rx=0 tx=0).stop 2024-05-10T05:37:54.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.928+0000 7f203d41db80 1 -- 172.21.15.131:0/1644819992 shutdown_connections 2024-05-10T05:37:54.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.928+0000 7f203d41db80 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5588ab865400 0x5588ab867850 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.928+0000 7f203d41db80 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82d6a0 0x5588ab859700 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.928+0000 7f203d41db80 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.928+0000 7f203d41db80 1 --2- 172.21.15.131:0/1644819992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82e570 0x5588ab859c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:54.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.928+0000 7f203d41db80 1 -- 172.21.15.131:0/1644819992 >> 172.21.15.131:0/1644819992 conn(0x5588ab78fa60 msgr2=0x5588ab827fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:54.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.929+0000 7f2003e00700 0 client.16074 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:54.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.930+0000 7f203d41db80 1 -- 172.21.15.131:0/1644819992 shutdown_connections 2024-05-10T05:37:54.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.930+0000 7f203d41db80 1 -- 172.21.15.131:0/1644819992 wait complete. 2024-05-10T05:37:54.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:54.930+0000 7f203d41db80 20 client.16074 trim_cache size 0 max 16384 2024-05-10T05:37:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.010+0000 7f203d41db80 1 Processor -- start 2024-05-10T05:37:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.012+0000 7f203d41db80 1 -- start start 2024-05-10T05:37:55.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.015+0000 7f203d41db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82e570 0x5588ab828d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.016+0000 7f203d41db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.016+0000 7f203d41db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab858ca0 0x5588ab824180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.018+0000 7f203d41db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab86ca10 con 0x5588ab82e570 2024-05-10T05:37:55.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.018+0000 7f203d41db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab832d00 con 0x5588ab82eec0 2024-05-10T05:37:55.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.018+0000 7f203d41db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab8246c0 con 0x5588ab858ca0 2024-05-10T05:37:55.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.018+0000 7f201ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82e570 0x5588ab828d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:55.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.019+0000 7f201ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82e570 0x5588ab828d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47980/0 (socket says 172.21.15.131:47980) 2024-05-10T05:37:55.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.019+0000 7f201ea00700 1 -- 172.21.15.131:0/1606453847 learned_addr learned my addr 172.21.15.131:0/1606453847 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:55.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.019+0000 7f201f400700 1 --2- 172.21.15.131:0/1606453847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab858ca0 0x5588ab824180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:55.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.020+0000 7f201e000700 1 --2- 172.21.15.131:0/1606453847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:55.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.021+0000 7f201e000700 1 -- 172.21.15.131:0/1606453847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab858ca0 msgr2=0x5588ab824180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:55.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.021+0000 7f201e000700 1 --2- 172.21.15.131:0/1606453847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab858ca0 0x5588ab824180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.021+0000 7f201e000700 1 -- 172.21.15.131:0/1606453847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82e570 msgr2=0x5588ab828d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:55.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.021+0000 7f201e000700 1 --2- 172.21.15.131:0/1606453847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82e570 0x5588ab828d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.021+0000 7f201e000700 1 -- 172.21.15.131:0/1606453847 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588ab713560 con 0x5588ab82eec0 2024-05-10T05:37:55.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.023+0000 7f201e000700 1 --2- 172.21.15.131:0/1606453847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860b60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2008006010 tx=0x7f2008020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e762bd086a8de2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:55.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.023+0000 7f200fe00700 1 -- 172.21.15.131:0/1606453847 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080055f0 con 0x5588ab82eec0 2024-05-10T05:37:55.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.024+0000 7f200fe00700 1 -- 172.21.15.131:0/1606453847 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2008021440 con 0x5588ab82eec0 2024-05-10T05:37:55.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.025+0000 7f203d41db80 1 -- 172.21.15.131:0/1606453847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 msgr2=0x5588ab860b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:55.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.025+0000 7f203d41db80 1 --2- 172.21.15.131:0/1606453847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860b60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2008006010 tx=0x7f2008020000 comp rx=0 tx=0).stop 2024-05-10T05:37:55.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.024+0000 7f200fe00700 1 -- 172.21.15.131:0/1606453847 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080055f0 con 0x5588ab82eec0 2024-05-10T05:37:55.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.028+0000 7f203d41db80 1 -- 172.21.15.131:0/1606453847 shutdown_connections 2024-05-10T05:37:55.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.028+0000 7f203d41db80 1 --2- 172.21.15.131:0/1606453847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab82e570 0x5588ab828d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.031+0000 7f203d41db80 1 --2- 172.21.15.131:0/1606453847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab858ca0 0x5588ab824180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.032+0000 7f203d41db80 1 --2- 172.21.15.131:0/1606453847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82eec0 0x5588ab860b60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.032+0000 7f203d41db80 1 -- 172.21.15.131:0/1606453847 >> 172.21.15.131:0/1606453847 conn(0x5588ab78fa60 msgr2=0x5588ab8317a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:55.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.033+0000 7f203d41db80 1 -- 172.21.15.131:0/1606453847 shutdown_connections 2024-05-10T05:37:55.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.033+0000 7f203d41db80 1 -- 172.21.15.131:0/1606453847 wait complete. 2024-05-10T05:37:55.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.037+0000 7f203d41db80 1 -- start start 2024-05-10T05:37:55.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.038+0000 7f203d41db80 1 Processor -- start 2024-05-10T05:37:55.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.043+0000 7f203d41db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82e570 0x5588ab85e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.046+0000 7f203d41db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82eec0 0x5588ab85ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.046+0000 7f203d41db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab858ca0 0x5588ab85fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.046+0000 7f203d41db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab860250 con 0x5588ab858ca0 2024-05-10T05:37:55.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.046+0000 7f203d41db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab8603c0 con 0x5588ab82e570 2024-05-10T05:37:55.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.046+0000 7f203d41db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab860580 con 0x5588ab82eec0 2024-05-10T05:37:55.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.046+0000 7f201e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82eec0 0x5588ab85ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:55.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.046+0000 7f201e000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82eec0 0x5588ab85ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54958/0 (socket says 172.21.15.131:54958) 2024-05-10T05:37:55.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.046+0000 7f201e000700 1 -- 172.21.15.131:0/734927034 learned_addr learned my addr 172.21.15.131:0/734927034 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:55.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.047+0000 7f201e000700 1 -- 172.21.15.131:0/734927034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82e570 msgr2=0x5588ab85e8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:37:55.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.047+0000 7f201e000700 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82e570 0x5588ab85e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.047+0000 7f201e000700 1 -- 172.21.15.131:0/734927034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab858ca0 msgr2=0x5588ab85fd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:55.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.047+0000 7f203d41db80 10 client.0 _ll_register_callbacks cb 0x5588ab6f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:55.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.048+0000 7f201e000700 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab858ca0 0x5588ab85fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.050+0000 7f201e000700 1 -- 172.21.15.131:0/734927034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202006a380 con 0x5588ab82eec0 2024-05-10T05:37:55.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.050+0000 7f203d41db80 1 -- 172.21.15.131:0/734927034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2018093de0 con 0x5588ab82eec0 2024-05-10T05:37:55.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.051+0000 7f201e000700 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82eec0 0x5588ab85ee30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2008005fe0 tx=0x7f2008021ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da340bb3b50bee58 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:55.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.052+0000 7f2003e00700 10 client.26176 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:37:55.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.052+0000 7f2003e00700 1 -- 172.21.15.131:0/734927034 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200802d480 con 0x5588ab82eec0 2024-05-10T05:37:55.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.053+0000 7f2003e00700 1 -- 172.21.15.131:0/734927034 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2008012070 con 0x5588ab82eec0 2024-05-10T05:37:55.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.053+0000 7f2003e00700 1 -- 172.21.15.131:0/734927034 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f200802d5e0 con 0x5588ab82eec0 2024-05-10T05:37:55.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.054+0000 7f203d41db80 10 client.26176 fetch_fsmap learned FSMap version 29 2024-05-10T05:37:55.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.054+0000 7f203d41db80 1 -- 172.21.15.131:0/734927034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x5588ab5d2cd0 con 0x5588ab82eec0 2024-05-10T05:37:55.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.055+0000 7f2003e00700 1 -- 172.21.15.131:0/734927034 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f200802d740 con 0x5588ab82eec0 2024-05-10T05:37:55.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.057+0000 7f203d41db80 10 client.26176 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:37:55.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.057+0000 7f203d41db80 10 client.26176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:37:55.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.057+0000 7f203d41db80 20 client.26176 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:55.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.057+0000 7f203d41db80 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5588ab8bc320 0x5588ab8be790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.057+0000 7f203d41db80 4 client.26176 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:37:55.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.057+0000 7f203d41db80 1 -- 172.21.15.131:0/734927034 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5588ab745ec0 con 0x5588ab8bc320 2024-05-10T05:37:55.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.058+0000 7f201ea00700 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5588ab8bc320 0x5588ab8be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:55.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.060+0000 7f2003e00700 10 client.26176 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:55.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.060+0000 7f201ea00700 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5588ab8bc320 0x5588ab8be790 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd493e715223845f server_cookie=8910f4c94c2bf53d in_seq=0 out_seq=0 2024-05-10T05:37:55.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.060+0000 7f2003e00700 1 -- 172.21.15.131:0/734927034 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5588ab745ec0 con 0x5588ab8bc320 2024-05-10T05:37:55.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.060+0000 7f2003e00700 10 client.26176 handle_command_reply: tid=0 2024-05-10T05:37:55.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.061+0000 7f203d41db80 1 client.26176 shutdown 2024-05-10T05:37:55.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.061+0000 7f203d41db80 1 -- 172.21.15.131:0/734927034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82eec0 msgr2=0x5588ab85ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:55.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.061+0000 7f203d41db80 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82eec0 0x5588ab85ee30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2008005fe0 tx=0x7f2008021ec0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.062+0000 7f203d41db80 1 -- 172.21.15.131:0/734927034 shutdown_connections 2024-05-10T05:37:55.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.062+0000 7f203d41db80 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5588ab8bc320 0x5588ab8be790 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.062+0000 7f203d41db80 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5588ab858ca0 0x5588ab85fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.062+0000 7f203d41db80 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ab82e570 0x5588ab85e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.062+0000 7f203d41db80 1 --2- 172.21.15.131:0/734927034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588ab82eec0 0x5588ab85ee30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.062+0000 7f203d41db80 1 -- 172.21.15.131:0/734927034 >> 172.21.15.131:0/734927034 conn(0x5588ab78fa60 msgr2=0x5588ab857d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:55.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.064+0000 7f2003e00700 0 client.26176 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:37:55.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.069+0000 7f203d41db80 1 -- 172.21.15.131:0/734927034 shutdown_connections 2024-05-10T05:37:55.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.069+0000 7f203d41db80 1 -- 172.21.15.131:0/734927034 wait complete. 2024-05-10T05:37:55.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.072+0000 7f203d41db80 20 client.26176 trim_cache size 0 max 16384 2024-05-10T05:37:55.076 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:37:55.076 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:37:55.076 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:37:55.076 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:37:55.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.080+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20040a1310 msgr2=0x7f20040a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:55.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.080+0000 7f2025600700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20040a1310 0x7f20040a3820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f200800aaa0 tx=0x7f2008005f00 comp rx=0 tx=0).stop 2024-05-10T05:37:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.080+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202010a0c0 msgr2=0x7f2020119850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.080+0000 7f2025600700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202010a0c0 0x7f2020119850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2018007f10 tx=0x7f201800d3b0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.081+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 shutdown_connections 2024-05-10T05:37:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.081+0000 7f2025600700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20040a1310 0x7f20040a3820 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.081+0000 7f2025600700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202010a0c0 0x7f2020119850 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.081+0000 7f2025600700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2020108630 0x7f20201183b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.081+0000 7f2025600700 1 --2- 172.21.15.131:0/753846652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2020109720 0x7f20201188f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.081+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 >> 172.21.15.131:0/753846652 conn(0x7f202006ee40 msgr2=0x7f202010c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:55.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.085+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 shutdown_connections 2024-05-10T05:37:55.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.089+0000 7f2025600700 1 -- 172.21.15.131:0/753846652 wait complete. 2024-05-10T05:37:55.155 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:37:55.155 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:37:55.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.752+0000 7f7058a00700 1 -- 172.21.15.131:0/702090489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054073b40 msgr2=0x7f7054074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:55.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.752+0000 7f7058a00700 1 --2- 172.21.15.131:0/702090489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054073b40 0x7f7054074010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f703c009990 tx=0x7f703c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.752+0000 7f7058a00700 1 -- 172.21.15.131:0/702090489 shutdown_connections 2024-05-10T05:37:55.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.752+0000 7f7058a00700 1 --2- 172.21.15.131:0/702090489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f705410e220 0x7f7054110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.752+0000 7f7058a00700 1 --2- 172.21.15.131:0/702090489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7054075140 0x7f7054073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.752+0000 7f7058a00700 1 --2- 172.21.15.131:0/702090489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054073b40 0x7f7054074010 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.752+0000 7f7058a00700 1 -- 172.21.15.131:0/702090489 >> 172.21.15.131:0/702090489 conn(0x7f70540fd1d0 msgr2=0x7f70540ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:55.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.753+0000 7f7058a00700 1 -- 172.21.15.131:0/702090489 shutdown_connections 2024-05-10T05:37:55.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.753+0000 7f7058a00700 1 -- 172.21.15.131:0/702090489 wait complete. 2024-05-10T05:37:55.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.754+0000 7f7058a00700 1 Processor -- start 2024-05-10T05:37:55.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.758+0000 7f7058a00700 1 -- start start 2024-05-10T05:37:55.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.762+0000 7f7058a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054073b40 0x7f705419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.762+0000 7f7058a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7054075140 0x7f705419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.762+0000 7f7058a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f705410e220 0x7f70541a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.762+0000 7f7058a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70541a5fc0 con 0x7f705410e220 2024-05-10T05:37:55.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.762+0000 7f7058a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70541a6100 con 0x7f7054073b40 2024-05-10T05:37:55.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.762+0000 7f7058a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70541a6290 con 0x7f7054075140 2024-05-10T05:37:55.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.765+0000 7f7049e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054073b40 0x7f705419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:55.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.766+0000 7f7051e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7054075140 0x7f705419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:55.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.766+0000 7f7051e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7054075140 0x7f705419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54972/0 (socket says 172.21.15.131:54972) 2024-05-10T05:37:55.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.766+0000 7f7051e00700 1 -- 172.21.15.131:0/752619004 learned_addr learned my addr 172.21.15.131:0/752619004 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:55.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.770+0000 7f7051e00700 1 -- 172.21.15.131:0/752619004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054073b40 msgr2=0x7f705419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:55.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.770+0000 7f7052800700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f705410e220 0x7f70541a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:55.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.771+0000 7f7051e00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054073b40 0x7f705419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.771+0000 7f7051e00700 1 -- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f705410e220 msgr2=0x7f70541a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:55.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.771+0000 7f7051e00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f705410e220 0x7f70541a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:55.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.771+0000 7f7051e00700 1 -- 172.21.15.131:0/752619004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c0095f0 con 0x7f7054075140 2024-05-10T05:37:55.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.772+0000 7f7049e00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054073b40 0x7f705419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:55.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.773+0000 7f7052800700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f705410e220 0x7f70541a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:55.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.775+0000 7f7051e00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7054075140 0x7f705419eaf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f703c0098e0 tx=0x7f703c00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:55.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.779+0000 7f704b400700 1 -- 172.21.15.131:0/752619004 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c01d070 con 0x7f7054075140 2024-05-10T05:37:55.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.779+0000 7f704b400700 1 -- 172.21.15.131:0/752619004 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f703c00fb70 con 0x7f7054075140 2024-05-10T05:37:55.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.779+0000 7f704b400700 1 -- 172.21.15.131:0/752619004 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c0173f0 con 0x7f7054075140 2024-05-10T05:37:55.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.782+0000 7f7058a00700 1 -- 172.21.15.131:0/752619004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70541a6510 con 0x7f7054075140 2024-05-10T05:37:55.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.782+0000 7f7058a00700 1 -- 172.21.15.131:0/752619004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70541a68e0 con 0x7f7054075140 2024-05-10T05:37:55.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.794+0000 7f7058a00700 1 -- 172.21.15.131:0/752619004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7054104480 con 0x7f7054075140 2024-05-10T05:37:55.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.818+0000 7f704b400700 1 -- 172.21.15.131:0/752619004 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f703c00fce0 con 0x7f7054075140 2024-05-10T05:37:55.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.822+0000 7f704b400700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70380a12a0 0x7f70380a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:55.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.823+0000 7f704b400700 1 -- 172.21.15.131:0/752619004 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f703c0cd630 con 0x7f7054075140 2024-05-10T05:37:55.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.825+0000 7f704b400700 1 -- 172.21.15.131:0/752619004 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f703c09a8e0 con 0x7f7054075140 2024-05-10T05:37:55.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.824+0000 7f7049e00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70380a12a0 0x7f70380a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:55.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:55.826+0000 7f7049e00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70380a12a0 0x7f70380a37b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7044005fd0 tx=0x7f7044005e20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.519+0000 7f7058a00700 1 -- 172.21.15.131:0/752619004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7054113c30 con 0x7f7054075140 2024-05-10T05:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.521+0000 7f704b400700 1 -- 172.21.15.131:0/752619004 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f703c09a470 con 0x7f7054075140 2024-05-10T05:37:56.523 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:37:56.524 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:37:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.529+0000 7f7058a00700 1 -- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70380a12a0 msgr2=0x7f70380a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.529+0000 7f7058a00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70380a12a0 0x7f70380a37b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7044005fd0 tx=0x7f7044005e20 comp rx=0 tx=0).stop 2024-05-10T05:37:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.529+0000 7f7058a00700 1 -- 172.21.15.131:0/752619004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7054075140 msgr2=0x7f705419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:56.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.529+0000 7f7058a00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7054075140 0x7f705419eaf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f703c0098e0 tx=0x7f703c00f740 comp rx=0 tx=0).stop 2024-05-10T05:37:56.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.530+0000 7f7058a00700 1 -- 172.21.15.131:0/752619004 shutdown_connections 2024-05-10T05:37:56.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.530+0000 7f7058a00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f70380a12a0 0x7f70380a37b0 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7044005fd0 tx=0x7f7044005e20 comp rx=0 tx=0).stop 2024-05-10T05:37:56.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.530+0000 7f7058a00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f705410e220 0x7f70541a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:56.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.530+0000 7f7058a00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054073b40 0x7f705419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:56.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.530+0000 7f7058a00700 1 --2- 172.21.15.131:0/752619004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7054075140 0x7f705419eaf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:56.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.530+0000 7f7058a00700 1 -- 172.21.15.131:0/752619004 >> 172.21.15.131:0/752619004 conn(0x7f70540fd1d0 msgr2=0x7f70541015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:56.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.533+0000 7f7058a00700 1 -- 172.21.15.131:0/752619004 shutdown_connections 2024-05-10T05:37:56.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:56.533+0000 7f7058a00700 1 -- 172.21.15.131:0/752619004 wait complete. 2024-05-10T05:37:56.549 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:37:56.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:56 smithi157 ceph-mon[78427]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 83 op/s 2024-05-10T05:37:56.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:56 smithi142 ceph-mon[77667]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 83 op/s 2024-05-10T05:37:56.664 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:56 smithi131 ceph-mon[37264]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 83 op/s 2024-05-10T05:37:56.670 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:37:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.351+0000 7fe0e1600700 1 -- 172.21.15.131:0/1527567940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc1087e0 msgr2=0x7fe0dc108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:57.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.351+0000 7fe0e1600700 1 --2- 172.21.15.131:0/1527567940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc1087e0 0x7fe0dc108c30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400fb60 tx=0x7fe0c400fe70 comp rx=0 tx=0).stop 2024-05-10T05:37:57.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.355+0000 7fe0e1600700 1 -- 172.21.15.131:0/1527567940 shutdown_connections 2024-05-10T05:37:57.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.355+0000 7fe0e1600700 1 --2- 172.21.15.131:0/1527567940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0dc10fec0 0x7fe0dc112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.355+0000 7fe0e1600700 1 --2- 172.21.15.131:0/1527567940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0dc109730 0x7fe0dc108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.355+0000 7fe0e1600700 1 --2- 172.21.15.131:0/1527567940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc1087e0 0x7fe0dc108c30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.355+0000 7fe0e1600700 1 -- 172.21.15.131:0/1527567940 >> 172.21.15.131:0/1527567940 conn(0x7fe0dc06e800 msgr2=0x7fe0dc070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:57.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.356+0000 7fe0e1600700 1 -- 172.21.15.131:0/1527567940 shutdown_connections 2024-05-10T05:37:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.356+0000 7fe0e1600700 1 -- 172.21.15.131:0/1527567940 wait complete. 2024-05-10T05:37:57.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.359+0000 7fe0e1600700 1 Processor -- start 2024-05-10T05:37:57.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.360+0000 7fe0e1600700 1 -- start start 2024-05-10T05:37:57.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.363+0000 7fe0e1600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0dc1087e0 0x7fe0dc19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.363+0000 7fe0e1600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0dc109730 0x7fe0dc19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.363+0000 7fe0e1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc10fec0 0x7fe0dc1a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.363+0000 7fe0e1600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1a5f10 con 0x7fe0dc109730 2024-05-10T05:37:57.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.363+0000 7fe0e1600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1a6050 con 0x7fe0dc10fec0 2024-05-10T05:37:57.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.363+0000 7fe0e1600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1a61e0 con 0x7fe0dc1087e0 2024-05-10T05:37:57.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.364+0000 7fe0e0c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc10fec0 0x7fe0dc1a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.364+0000 7fe0e0c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc10fec0 0x7fe0dc1a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:40926/0 (socket says 172.21.15.131:40926) 2024-05-10T05:37:57.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.364+0000 7fe0e0c00700 1 -- 172.21.15.131:0/4080280522 learned_addr learned my addr 172.21.15.131:0/4080280522 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:57.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.364+0000 7fe0dbe00700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0dc1087e0 0x7fe0dc19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.365+0000 7fe0d3400700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0dc109730 0x7fe0dc19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.366+0000 7fe0d3400700 1 -- 172.21.15.131:0/4080280522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0dc1087e0 msgr2=0x7fe0dc19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.366+0000 7fe0d3400700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0dc1087e0 0x7fe0dc19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.366+0000 7fe0d3400700 1 -- 172.21.15.131:0/4080280522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc10fec0 msgr2=0x7fe0dc1a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.366+0000 7fe0d3400700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc10fec0 0x7fe0dc1a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.366+0000 7fe0d3400700 1 -- 172.21.15.131:0/4080280522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c400f790 con 0x7fe0dc109730 2024-05-10T05:37:57.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.369+0000 7fe0d3400700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0dc109730 0x7fe0dc19e9b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400fb60 tx=0x7fe0c4018b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:57.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.371+0000 7fe0d9600700 1 -- 172.21.15.131:0/4080280522 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40051d0 con 0x7fe0dc109730 2024-05-10T05:37:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.371+0000 7fe0d9600700 1 -- 172.21.15.131:0/4080280522 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c4003680 con 0x7fe0dc109730 2024-05-10T05:37:57.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.372+0000 7fe0d9600700 1 -- 172.21.15.131:0/4080280522 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c400e700 con 0x7fe0dc109730 2024-05-10T05:37:57.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.373+0000 7fe0e1600700 1 -- 172.21.15.131:0/4080280522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc1a6460 con 0x7fe0dc109730 2024-05-10T05:37:57.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.373+0000 7fe0e1600700 1 -- 172.21.15.131:0/4080280522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc1a6900 con 0x7fe0dc109730 2024-05-10T05:37:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.392+0000 7fe0d9600700 1 -- 172.21.15.131:0/4080280522 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe0c400e860 con 0x7fe0dc109730 2024-05-10T05:37:57.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.406+0000 7fe0d9600700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe0c00a1220 0x7fe0c00a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.407+0000 7fe0d9600700 1 -- 172.21.15.131:0/4080280522 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe0c40037f0 con 0x7fe0dc109730 2024-05-10T05:37:57.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.412+0000 7fe0dbe00700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe0c00a1220 0x7fe0c00a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.414+0000 7fe0dbe00700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe0c00a1220 0x7fe0c00a3730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc072e70 tx=0x7fe0cc009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:57.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.421+0000 7fe0fab9bb80 1 Processor -- start 2024-05-10T05:37:57.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.422+0000 7fe0fab9bb80 1 -- start start 2024-05-10T05:37:57.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.422+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd71960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.422+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd727a0 0x5612bdd72bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.422+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd730f0 0x5612bdd799f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.422+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdc98150 con 0x5612bdd730f0 2024-05-10T05:37:57.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.422+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdc95b10 con 0x5612bdd71530 2024-05-10T05:37:57.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.422+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdc91400 con 0x5612bdd727a0 2024-05-10T05:37:57.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.423+0000 7fe0dbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd71960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.423+0000 7fe0dbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd71960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:40940/0 (socket says 172.21.15.131:40940) 2024-05-10T05:37:57.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.423+0000 7fe0dbe00700 1 -- 172.21.15.131:0/884948245 learned_addr learned my addr 172.21.15.131:0/884948245 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:57.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.423+0000 7fe0d3400700 1 --2- 172.21.15.131:0/884948245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd727a0 0x5612bdd72bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.423+0000 7fe0dbe00700 1 -- 172.21.15.131:0/884948245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd727a0 msgr2=0x5612bdd72bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:57.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.423+0000 7fe0dbe00700 1 --2- 172.21.15.131:0/884948245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd727a0 0x5612bdd72bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.423+0000 7fe0dbe00700 1 -- 172.21.15.131:0/884948245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd730f0 msgr2=0x5612bdd799f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:57.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.423+0000 7fe0dbe00700 1 --2- 172.21.15.131:0/884948245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd730f0 0x5612bdd799f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.423+0000 7fe0dbe00700 1 -- 172.21.15.131:0/884948245 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612bdb123b0 con 0x5612bdd71530 2024-05-10T05:37:57.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.424+0000 7fe0dbe00700 1 --2- 172.21.15.131:0/884948245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd71960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0041d0 tx=0x7fe0cc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f042522ee33d972b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:57.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.424+0000 7fe0d1200700 1 -- 172.21.15.131:0/884948245 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc009ea0 con 0x5612bdd71530 2024-05-10T05:37:57.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.424+0000 7fe0d1200700 1 -- 172.21.15.131:0/884948245 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0cc004590 con 0x5612bdd71530 2024-05-10T05:37:57.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.426+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/884948245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 msgr2=0x5612bdd71960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:57.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.426+0000 7fe0d1200700 1 -- 172.21.15.131:0/884948245 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc009ea0 con 0x5612bdd71530 2024-05-10T05:37:57.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.427+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/884948245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd71960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0041d0 tx=0x7fe0cc014000 comp rx=0 tx=0).stop 2024-05-10T05:37:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.432+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/884948245 shutdown_connections 2024-05-10T05:37:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.432+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/884948245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd730f0 0x5612bdd799f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.432+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/884948245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd71960 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.432+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/884948245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd727a0 0x5612bdd72bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.432+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/884948245 >> 172.21.15.131:0/884948245 conn(0x5612bdcd3a60 msgr2=0x5612bdd6c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:57.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.434+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/884948245 shutdown_connections 2024-05-10T05:37:57.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.438+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/884948245 wait complete. 2024-05-10T05:37:57.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.439+0000 7fe0fab9bb80 1 -- start start 2024-05-10T05:37:57.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.439+0000 7fe0fab9bb80 1 Processor -- start 2024-05-10T05:37:57.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.448+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd7b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.448+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 0x5612bdd7bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.448+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 0x5612bdd83140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.448+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdc2d560 con 0x5612bdd80d20 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.448+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdc2d200 con 0x5612bdd71530 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.448+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdb146a0 con 0x5612bdd730f0 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.449+0000 7fe0fab9bb80 10 client.0 _ll_register_callbacks cb 0x5612bdc36b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.450+0000 7fe0e0c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 0x5612bdd83140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.451+0000 7fe0e0c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 0x5612bdd83140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48038/0 (socket says 172.21.15.131:48038) 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.451+0000 7fe0e0c00700 1 -- 172.21.15.131:0/977264767 learned_addr learned my addr 172.21.15.131:0/977264767 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.451+0000 7fe0d3400700 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 0x5612bdd7bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.451+0000 7fe0e0c00700 1 -- 172.21.15.131:0/977264767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 msgr2=0x5612bdd7bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.451+0000 7fe0e0c00700 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 0x5612bdd7bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.451+0000 7fe0e0c00700 1 -- 172.21.15.131:0/977264767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 msgr2=0x5612bdd7b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.451+0000 7fe0e0c00700 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd7b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.451+0000 7fe0e0c00700 1 -- 172.21.15.131:0/977264767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0cc017070 con 0x5612bdd80d20 2024-05-10T05:37:57.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.457+0000 7fe0c9600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:37:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.458+0000 7fe0e0c00700 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 0x5612bdd83140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400ebe0 tx=0x7fe0d400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3237df631f5af5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:57.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.459+0000 7fe0c9600700 1 -- 172.21.15.131:0/977264767 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4009bc0 con 0x5612bdd80d20 2024-05-10T05:37:57.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.459+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/977264767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612bdd40780 con 0x5612bdd80d20 2024-05-10T05:37:57.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.460+0000 7fe0c9600700 1 -- 172.21.15.131:0/977264767 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe0d4006e40 con 0x5612bdd80d20 2024-05-10T05:37:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.462+0000 7fe0c9600700 1 -- 172.21.15.131:0/977264767 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0d4009bc0 con 0x5612bdd80d20 2024-05-10T05:37:57.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.464+0000 7fe0fab9bb80 10 client.16083 fetch_fsmap learned FSMap version 29 2024-05-10T05:37:57.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.464+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/977264767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5612bdabeb60 con 0x5612bdd80d20 2024-05-10T05:37:57.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.470+0000 7fe0c9600700 1 -- 172.21.15.131:0/977264767 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612bdd40780 con 0x5612bdd80d20 2024-05-10T05:37:57.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.471+0000 7fe0c9600700 1 -- 172.21.15.131:0/977264767 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe0d4014bd0 con 0x5612bdd80d20 2024-05-10T05:37:57.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.472+0000 7fe0fab9bb80 10 client.16083 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:37:57.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.472+0000 7fe0fab9bb80 10 client.16083 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:37:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.472+0000 7fe0fab9bb80 20 client.16083 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.472+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5612bdd879c0 0x5612bdd89e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.473+0000 7fe0fab9bb80 4 client.16083 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.473+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/977264767 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612bdc89ec0 con 0x5612bdd879c0 2024-05-10T05:37:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.474+0000 7fe0dbe00700 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5612bdd879c0 0x5612bdd89e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.475+0000 7fe0dbe00700 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5612bdd879c0 0x5612bdd89e10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6084f5687004d34f server_cookie=ae9fe4f34c638933 in_seq=0 out_seq=0 2024-05-10T05:37:57.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.475+0000 7fe0c9600700 10 client.16083 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:37:57.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.476+0000 7fe0c9600700 1 -- 172.21.15.131:0/977264767 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612bdc89ec0 con 0x5612bdd879c0 2024-05-10T05:37:57.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.476+0000 7fe0c9600700 10 client.16083 handle_command_reply: tid=0 2024-05-10T05:37:57.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.476+0000 7fe0fab9bb80 1 client.16083 shutdown 2024-05-10T05:37:57.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.477+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/977264767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 msgr2=0x5612bdd83140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:57.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.477+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 0x5612bdd83140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400ebe0 tx=0x7fe0d400ecc0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.478+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/977264767 shutdown_connections 2024-05-10T05:37:57.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.478+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5612bdd879c0 0x5612bdd89e10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.478+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 0x5612bdd83140 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.478+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd7b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.478+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/977264767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 0x5612bdd7bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.478+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/977264767 >> 172.21.15.131:0/977264767 conn(0x5612bdcd3a60 msgr2=0x5612bdd70060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:57.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.478+0000 7fe0c9600700 0 client.16083 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:37:57.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.479+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/977264767 shutdown_connections 2024-05-10T05:37:57.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.480+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/977264767 wait complete. 2024-05-10T05:37:57.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.480+0000 7fe0fab9bb80 20 client.16083 trim_cache size 0 max 16384 2024-05-10T05:37:57.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.520+0000 7fe0fab9bb80 1 Processor -- start 2024-05-10T05:37:57.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.520+0000 7fe0fab9bb80 1 -- start start 2024-05-10T05:37:57.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.521+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd97ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.521+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 0x5612bdd98220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.522+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 0x5612bdd6bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.522+0000 7fe0d3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 0x5612bdd98220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.522+0000 7fe0e0c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 0x5612bdd6bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.522+0000 7fe0d3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 0x5612bdd98220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55002/0 (socket says 172.21.15.131:55002) 2024-05-10T05:37:57.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.522+0000 7fe0dbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd97ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.522+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdd84240 con 0x5612bdd80d20 2024-05-10T05:37:57.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.522+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdd668d0 con 0x5612bdd71530 2024-05-10T05:37:57.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.522+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdd721f0 con 0x5612bdd730f0 2024-05-10T05:37:57.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.522+0000 7fe0d3400700 1 -- 172.21.15.131:0/823095602 learned_addr learned my addr 172.21.15.131:0/823095602 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:57.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.523+0000 7fe0d3400700 1 -- 172.21.15.131:0/823095602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 msgr2=0x5612bdd97ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:57.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.523+0000 7fe0d3400700 1 --2- 172.21.15.131:0/823095602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd97ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.523+0000 7fe0d3400700 1 -- 172.21.15.131:0/823095602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 msgr2=0x5612bdd6bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:57.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.523+0000 7fe0d3400700 1 --2- 172.21.15.131:0/823095602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 0x5612bdd6bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.523+0000 7fe0d3400700 1 -- 172.21.15.131:0/823095602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612bdc57560 con 0x5612bdd730f0 2024-05-10T05:37:57.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.527+0000 7fe0d3400700 1 --2- 172.21.15.131:0/823095602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 0x5612bdd98220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400fb30 tx=0x7fe0c40a4430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=564e0b66e2665fe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:57.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.528+0000 7fe0d1c00700 1 -- 172.21.15.131:0/823095602 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40a8880 con 0x5612bdd730f0 2024-05-10T05:37:57.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.529+0000 7fe0d1c00700 1 -- 172.21.15.131:0/823095602 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c40ace40 con 0x5612bdd730f0 2024-05-10T05:37:57.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.529+0000 7fe0d1c00700 1 -- 172.21.15.131:0/823095602 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40a8880 con 0x5612bdd730f0 2024-05-10T05:37:57.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.533+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/823095602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 msgr2=0x5612bdd98220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:57.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.533+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/823095602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 0x5612bdd98220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400fb30 tx=0x7fe0c40a4430 comp rx=0 tx=0).stop 2024-05-10T05:37:57.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.538+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/823095602 shutdown_connections 2024-05-10T05:37:57.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.538+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/823095602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd80d20 0x5612bdd6bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.538+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/823095602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd71530 0x5612bdd97ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.538+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/823095602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd730f0 0x5612bdd98220 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.538+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/823095602 >> 172.21.15.131:0/823095602 conn(0x5612bdcd3a60 msgr2=0x5612bdd83140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:57.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.540+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/823095602 shutdown_connections 2024-05-10T05:37:57.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.541+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/823095602 wait complete. 2024-05-10T05:37:57.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.541+0000 7fe0fab9bb80 1 -- start start 2024-05-10T05:37:57.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.546+0000 7fe0fab9bb80 1 Processor -- start 2024-05-10T05:37:57.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.549+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd71530 0x5612bdd975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.549+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd730f0 0x5612bdd97ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.549+0000 7fe0fab9bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd80d20 0x5612bdddcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.549+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdddd140 con 0x5612bdd71530 2024-05-10T05:37:57.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.549+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bdddd280 con 0x5612bdd730f0 2024-05-10T05:37:57.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.549+0000 7fe0fab9bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bddee420 con 0x5612bdd80d20 2024-05-10T05:37:57.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.549+0000 7fe0fab9bb80 10 client.0 _ll_register_callbacks cb 0x5612bdc36b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:57.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.554+0000 7fe0d3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd730f0 0x5612bdd97ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.554+0000 7fe0e0c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd80d20 0x5612bdddcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.554+0000 7fe0e0c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd80d20 0x5612bdddcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55014/0 (socket says 172.21.15.131:55014) 2024-05-10T05:37:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.554+0000 7fe0e0c00700 1 -- 172.21.15.131:0/2127021599 learned_addr learned my addr 172.21.15.131:0/2127021599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.563+0000 7fe0d3400700 1 -- 172.21.15.131:0/2127021599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd80d20 msgr2=0x5612bdddcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.563+0000 7fe0d3400700 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd80d20 0x5612bdddcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.563+0000 7fe0d3400700 1 -- 172.21.15.131:0/2127021599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd71530 msgr2=0x5612bdd975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.563+0000 7fe0d3400700 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd71530 0x5612bdd975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.563+0000 7fe0d3400700 1 -- 172.21.15.131:0/2127021599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc1a6900 con 0x5612bdd730f0 2024-05-10T05:37:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.567+0000 7fe0d3400700 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd730f0 0x5612bdd97ae0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5612bdd80c90 tx=0x7fe0c40a0e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b60c05adc6647639 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.567+0000 7fe0c9600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:37:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.567+0000 7fe0c9600700 1 -- 172.21.15.131:0/2127021599 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40a8880 con 0x5612bdd730f0 2024-05-10T05:37:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.567+0000 7fe0c9600700 1 -- 172.21.15.131:0/2127021599 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe0c40a4150 con 0x5612bdd730f0 2024-05-10T05:37:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.573+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/2127021599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612bdbea270 con 0x5612bdd730f0 2024-05-10T05:37:57.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.578+0000 7fe0c9600700 1 -- 172.21.15.131:0/2127021599 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0c40ac9b0 con 0x5612bdd730f0 2024-05-10T05:37:57.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.578+0000 7fe0fab9bb80 10 client.26180 fetch_fsmap learned FSMap version 29 2024-05-10T05:37:57.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.578+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/2127021599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5612bdb16cd0 con 0x5612bdd730f0 2024-05-10T05:37:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.578+0000 7fe0c9600700 1 -- 172.21.15.131:0/2127021599 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612bdbea270 con 0x5612bdd730f0 2024-05-10T05:37:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.578+0000 7fe0c9600700 1 -- 172.21.15.131:0/2127021599 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe0c40a0840 con 0x5612bdd730f0 2024-05-10T05:37:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.579+0000 7fe0fab9bb80 10 client.26180 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:37:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.579+0000 7fe0fab9bb80 10 client.26180 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:37:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.579+0000 7fe0fab9bb80 20 client.26180 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.579+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5612bddef8a0 0x5612bddf1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.579+0000 7fe0fab9bb80 4 client.26180 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:37:57.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.579+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/2127021599 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612bdadfcf0 con 0x5612bddef8a0 2024-05-10T05:37:57.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.580+0000 7fe0dbe00700 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5612bddef8a0 0x5612bddf1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:57.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.583+0000 7fe0c9600700 10 client.26180 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:37:57.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.583+0000 7fe0dbe00700 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5612bddef8a0 0x5612bddf1cc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4da74dc18f27d4af server_cookie=743e5feb051eba69 in_seq=0 out_seq=0 2024-05-10T05:37:57.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.583+0000 7fe0c9600700 1 -- 172.21.15.131:0/2127021599 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612bdadfcf0 con 0x5612bddef8a0 2024-05-10T05:37:57.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.583+0000 7fe0c9600700 10 client.26180 handle_command_reply: tid=0 2024-05-10T05:37:57.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.584+0000 7fe0fab9bb80 1 client.26180 shutdown 2024-05-10T05:37:57.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.584+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/2127021599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd730f0 msgr2=0x5612bdd97ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:57.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.584+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd730f0 0x5612bdd97ae0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5612bdd80c90 tx=0x7fe0c40a0e40 comp rx=0 tx=0).stop 2024-05-10T05:37:57.589 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:37:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.586+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/2127021599 shutdown_connections 2024-05-10T05:37:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.586+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5612bddef8a0 0x5612bddf1cc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.586+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612bdd71530 0x5612bdd975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.586+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612bdd80d20 0x5612bdddcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.586+0000 7fe0fab9bb80 1 --2- 172.21.15.131:0/2127021599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612bdd730f0 0x5612bdd97ae0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.586+0000 7fe0c9600700 0 client.26180 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:37:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.586+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/2127021599 >> 172.21.15.131:0/2127021599 conn(0x5612bdcd3a60 msgr2=0x5612bdd6bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.587+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/2127021599 shutdown_connections 2024-05-10T05:37:57.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.587+0000 7fe0fab9bb80 1 -- 172.21.15.131:0/2127021599 wait complete. 2024-05-10T05:37:57.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.587+0000 7fe0fab9bb80 20 client.26180 trim_cache size 0 max 16384 2024-05-10T05:37:57.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.588+0000 7fe0e1600700 1 -- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe0c00a1220 msgr2=0x7fe0c00a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:57.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.588+0000 7fe0e1600700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe0c00a1220 0x7fe0c00a3730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc072e70 tx=0x7fe0cc009310 comp rx=0 tx=0).stop 2024-05-10T05:37:57.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.589+0000 7fe0e1600700 1 -- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0dc109730 msgr2=0x7fe0dc19e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:57.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.589+0000 7fe0e1600700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0dc109730 0x7fe0dc19e9b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400fb60 tx=0x7fe0c4018b00 comp rx=0 tx=0).stop 2024-05-10T05:37:57.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.589+0000 7fe0e1600700 1 -- 172.21.15.131:0/4080280522 shutdown_connections 2024-05-10T05:37:57.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.589+0000 7fe0e1600700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe0c00a1220 0x7fe0c00a3730 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.589+0000 7fe0e1600700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc10fec0 0x7fe0dc1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.589+0000 7fe0e1600700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0dc1087e0 0x7fe0dc19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.589+0000 7fe0e1600700 1 --2- 172.21.15.131:0/4080280522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0dc109730 0x7fe0dc19e9b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:57.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.589+0000 7fe0e1600700 1 -- 172.21.15.131:0/4080280522 >> 172.21.15.131:0/4080280522 conn(0x7fe0dc06e800 msgr2=0x7fe0dc10bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:57.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.590+0000 7fe0e1600700 1 -- 172.21.15.131:0/4080280522 shutdown_connections 2024-05-10T05:37:57.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:57.590+0000 7fe0e1600700 1 -- 172.21.15.131:0/4080280522 wait complete. 2024-05-10T05:37:57.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:57 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/752619004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:37:57.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:37:57 smithi142 ceph-mon[77667]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 238 MiB/s wr, 73 op/s 2024-05-10T05:37:57.646 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:57 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/752619004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:37:57.647 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:37:57 smithi131 ceph-mon[37264]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 238 MiB/s wr, 73 op/s 2024-05-10T05:37:57.653 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:37:57.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:57 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/752619004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:37:57.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:37:57 smithi157 ceph-mon[78427]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 238 MiB/s wr, 73 op/s 2024-05-10T05:37:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.551+0000 7faefe600700 1 -- 172.21.15.131:0/1857333915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef8110320 msgr2=0x7faef8112740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.551+0000 7faefe600700 1 --2- 172.21.15.131:0/1857333915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef8110320 0x7faef8112740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7faee4009990 tx=0x7faee4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.551+0000 7faefe600700 1 -- 172.21.15.131:0/1857333915 shutdown_connections 2024-05-10T05:37:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.552+0000 7faefe600700 1 --2- 172.21.15.131:0/1857333915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef8110320 0x7faef8112740 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.552+0000 7faefe600700 1 --2- 172.21.15.131:0/1857333915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef8108210 0x7faef8108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.552+0000 7faefe600700 1 --2- 172.21.15.131:0/1857333915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef8108b60 0x7faef810fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.552+0000 7faefe600700 1 -- 172.21.15.131:0/1857333915 >> 172.21.15.131:0/1857333915 conn(0x7faef806ea00 msgr2=0x7faef8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:58.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.552+0000 7faefe600700 1 -- 172.21.15.131:0/1857333915 shutdown_connections 2024-05-10T05:37:58.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.552+0000 7faefe600700 1 -- 172.21.15.131:0/1857333915 wait complete. 2024-05-10T05:37:58.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.555+0000 7faefe600700 1 Processor -- start 2024-05-10T05:37:58.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.555+0000 7faefe600700 1 -- start start 2024-05-10T05:37:58.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.556+0000 7faefe600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef8108210 0x7faef81a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.556+0000 7faefe600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef8108b60 0x7faef81a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.557+0000 7faef7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef8108210 0x7faef81a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.557+0000 7faef7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef8108210 0x7faef81a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55038/0 (socket says 172.21.15.131:55038) 2024-05-10T05:37:58.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.557+0000 7faefe600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef8110320 0x7faef81a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.557+0000 7faef7400700 1 -- 172.21.15.131:0/2004155497 learned_addr learned my addr 172.21.15.131:0/2004155497 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:58.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.557+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef81a82b0 con 0x7faef8108b60 2024-05-10T05:37:58.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.557+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef81a8420 con 0x7faef8110320 2024-05-10T05:37:58.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.557+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef81a85e0 con 0x7faef8108210 2024-05-10T05:37:58.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.557+0000 7faef7e00700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef8110320 0x7faef81a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.558+0000 7faef7400700 1 -- 172.21.15.131:0/2004155497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef8110320 msgr2=0x7faef81a7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:58.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.558+0000 7faef7400700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef8110320 0x7faef81a7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.558+0000 7faef7400700 1 -- 172.21.15.131:0/2004155497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef8108b60 msgr2=0x7faef81a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:58.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.558+0000 7faef7400700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef8108b60 0x7faef81a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.558+0000 7faef7400700 1 -- 172.21.15.131:0/2004155497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee40095f0 con 0x7faef8108210 2024-05-10T05:37:58.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.559+0000 7faef7400700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef8108210 0x7faef81a07d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faef000b9b0 tx=0x7faef000bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:58.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.560+0000 7faeefe00700 1 -- 172.21.15.131:0/2004155497 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef0010d60 con 0x7faef8108210 2024-05-10T05:37:58.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.560+0000 7faeefe00700 1 -- 172.21.15.131:0/2004155497 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faef0018900 con 0x7faef8108210 2024-05-10T05:37:58.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.560+0000 7faeefe00700 1 -- 172.21.15.131:0/2004155497 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef0016880 con 0x7faef8108210 2024-05-10T05:37:58.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.562+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef8109c10 con 0x7faef8108210 2024-05-10T05:37:58.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.562+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef810a120 con 0x7faef8108210 2024-05-10T05:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.568+0000 7faeefe00700 1 -- 172.21.15.131:0/2004155497 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7faef0018a70 con 0x7faef8108210 2024-05-10T05:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.570+0000 7faeefe00700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faedc0a1400 0x7faedc0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.570+0000 7faeefe00700 1 -- 172.21.15.131:0/2004155497 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7faef00cff90 con 0x7faef8108210 2024-05-10T05:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.572+0000 7faef6a00700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faedc0a1400 0x7faedc0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.574+0000 7faef6a00700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faedc0a1400 0x7faedc0a3910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faef81a1870 tx=0x7faee8006cd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.574+0000 7faf164a1b80 1 Processor -- start 2024-05-10T05:37:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.575+0000 7faf164a1b80 1 -- start start 2024-05-10T05:37:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.576+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585696156a0 0x558569615af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.576+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x558569616980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.576+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x55856961d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.576+0000 7faf164a1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856953c150 con 0x558569616ec0 2024-05-10T05:37:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.576+0000 7faf164a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558569539b10 con 0x558569616570 2024-05-10T05:37:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.576+0000 7faf164a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558569535400 con 0x5585696156a0 2024-05-10T05:37:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.576+0000 7faef6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x558569616980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.576+0000 7faef6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x558569616980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41024/0 (socket says 172.21.15.131:41024) 2024-05-10T05:37:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.576+0000 7faef6a00700 1 -- 172.21.15.131:0/2305685004 learned_addr learned my addr 172.21.15.131:0/2305685004 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.577+0000 7faef7400700 1 --2- 172.21.15.131:0/2305685004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585696156a0 0x558569615af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.577+0000 7faef6a00700 1 -- 172.21.15.131:0/2305685004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585696156a0 msgr2=0x558569615af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.577+0000 7faef6a00700 1 --2- 172.21.15.131:0/2305685004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585696156a0 0x558569615af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.577+0000 7faef6a00700 1 -- 172.21.15.131:0/2305685004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 msgr2=0x55856961d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.577+0000 7faef6a00700 1 --2- 172.21.15.131:0/2305685004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x55856961d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.577+0000 7faef6a00700 1 -- 172.21.15.131:0/2305685004 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585693b63b0 con 0x558569616570 2024-05-10T05:37:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.577+0000 7faef6a00700 1 --2- 172.21.15.131:0/2305685004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x558569616980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faee8015900 tx=0x7faee800e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c08ef914d46c84 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.578+0000 7faeecc00700 1 -- 172.21.15.131:0/2305685004 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee801a070 con 0x558569616570 2024-05-10T05:37:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.578+0000 7faeecc00700 1 -- 172.21.15.131:0/2305685004 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee8012970 con 0x558569616570 2024-05-10T05:37:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.578+0000 7faeecc00700 1 -- 172.21.15.131:0/2305685004 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8013880 con 0x558569616570 2024-05-10T05:37:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.579+0000 7faf164a1b80 1 -- 172.21.15.131:0/2305685004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 msgr2=0x558569616980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.579+0000 7faf164a1b80 1 --2- 172.21.15.131:0/2305685004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x558569616980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faee8015900 tx=0x7faee800e000 comp rx=0 tx=0).stop 2024-05-10T05:37:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.580+0000 7faf164a1b80 1 -- 172.21.15.131:0/2305685004 shutdown_connections 2024-05-10T05:37:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.580+0000 7faf164a1b80 1 --2- 172.21.15.131:0/2305685004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x55856961d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.580+0000 7faf164a1b80 1 --2- 172.21.15.131:0/2305685004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585696156a0 0x558569615af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.580+0000 7faf164a1b80 1 --2- 172.21.15.131:0/2305685004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x558569616980 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.580+0000 7faf164a1b80 1 -- 172.21.15.131:0/2305685004 >> 172.21.15.131:0/2305685004 conn(0x558569577a60 msgr2=0x55856960ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.580+0000 7faf164a1b80 1 -- 172.21.15.131:0/2305685004 shutdown_connections 2024-05-10T05:37:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.580+0000 7faf164a1b80 1 -- 172.21.15.131:0/2305685004 wait complete. 2024-05-10T05:37:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.581+0000 7faf164a1b80 1 -- start start 2024-05-10T05:37:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.581+0000 7faf164a1b80 1 Processor -- start 2024-05-10T05:37:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.582+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585696156a0 0x558569627d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.582+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x558569628240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.583+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616ec0 0x55856962f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.583+0000 7faef7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585696156a0 0x558569627d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.583+0000 7faef7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585696156a0 0x558569627d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48100/0 (socket says 172.21.15.131:48100) 2024-05-10T05:37:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.583+0000 7faef7400700 1 -- 172.21.15.131:0/135950164 learned_addr learned my addr 172.21.15.131:0/135950164 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.584+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585694d1560 con 0x5585696156a0 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.584+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585694d1200 con 0x558569616570 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.584+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585693b86a0 con 0x558569616ec0 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.584+0000 7faf164a1b80 10 client.0 _ll_register_callbacks cb 0x5585694dab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.585+0000 7faef6a00700 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x558569628240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.585+0000 7faef7e00700 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616ec0 0x55856962f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.585+0000 7faef7400700 1 -- 172.21.15.131:0/135950164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616ec0 msgr2=0x55856962f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.585+0000 7faef7400700 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616ec0 0x55856962f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.585+0000 7faef7400700 1 -- 172.21.15.131:0/135950164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 msgr2=0x558569628240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.585+0000 7faef7400700 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x558569628240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.585+0000 7faef7400700 1 -- 172.21.15.131:0/135950164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee80155b0 con 0x5585696156a0 2024-05-10T05:37:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.585+0000 7faef7e00700 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616ec0 0x55856962f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.587+0000 7faef7400700 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585696156a0 0x558569627d00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faef009d080 tx=0x7faef009d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833cd27c0f4c9e01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.587+0000 7faee1600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:37:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.588+0000 7faee1600700 1 -- 172.21.15.131:0/135950164 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00a5800 con 0x5585696156a0 2024-05-10T05:37:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.588+0000 7faee1600700 1 -- 172.21.15.131:0/135950164 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7faef00cdb40 con 0x5585696156a0 2024-05-10T05:37:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.589+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55856948e270 con 0x5585696156a0 2024-05-10T05:37:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.590+0000 7faee1600700 1 -- 172.21.15.131:0/135950164 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7faef00a5370 con 0x5585696156a0 2024-05-10T05:37:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.590+0000 7faf164a1b80 10 client.16092 fetch_fsmap learned FSMap version 29 2024-05-10T05:37:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.590+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558569362b60 con 0x5585696156a0 2024-05-10T05:37:58.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.590+0000 7faee1600700 1 -- 172.21.15.131:0/135950164 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55856948e270 con 0x5585696156a0 2024-05-10T05:37:58.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.591+0000 7faee1600700 1 -- 172.21.15.131:0/135950164 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faef0099290 con 0x5585696156a0 2024-05-10T05:37:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.592+0000 7faf164a1b80 10 client.16092 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:37:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.592+0000 7faf164a1b80 10 client.16092 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:37:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.592+0000 7faf164a1b80 20 client.16092 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.592+0000 7faf164a1b80 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558569633940 0x558569635d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.592+0000 7faf164a1b80 4 client.16092 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.592+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585695e4780 con 0x558569633940 2024-05-10T05:37:58.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.593+0000 7faef6a00700 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558569633940 0x558569635d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.603+0000 7faef6a00700 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558569633940 0x558569635d90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a5b0cd0e5e0c2731 server_cookie=c13f8c95615238c8 in_seq=0 out_seq=0 2024-05-10T05:37:58.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.603+0000 7faee1600700 10 client.16092 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:37:58.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.604+0000 7faee1600700 1 -- 172.21.15.131:0/135950164 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585695e4780 con 0x558569633940 2024-05-10T05:37:58.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.604+0000 7faee1600700 10 client.16092 handle_command_reply: tid=0 2024-05-10T05:37:58.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.604+0000 7faf164a1b80 1 client.16092 shutdown 2024-05-10T05:37:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.607+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585696156a0 msgr2=0x558569627d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.607+0000 7faf164a1b80 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585696156a0 0x558569627d00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faef009d080 tx=0x7faef009d8c0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.608+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 shutdown_connections 2024-05-10T05:37:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.608+0000 7faf164a1b80 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x558569633940 0x558569635d90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.608+0000 7faf164a1b80 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585696156a0 0x558569627d00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.608+0000 7faf164a1b80 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616ec0 0x55856962f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.608+0000 7faf164a1b80 1 --2- 172.21.15.131:0/135950164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x558569628240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.608+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 >> 172.21.15.131:0/135950164 conn(0x558569577a60 msgr2=0x55856960ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.608+0000 7faee1600700 0 client.16092 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:37:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.610+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 shutdown_connections 2024-05-10T05:37:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.613+0000 7faf164a1b80 1 -- 172.21.15.131:0/135950164 wait complete. 2024-05-10T05:37:58.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.614+0000 7faf164a1b80 20 client.16092 trim_cache size 0 max 16384 2024-05-10T05:37:58.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.685+0000 7faf164a1b80 1 Processor -- start 2024-05-10T05:37:58.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.685+0000 7faf164a1b80 1 -- start start 2024-05-10T05:37:58.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.685+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585696156a0 0x558569616980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.685+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616570 0x55856962f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.688+0000 7faef6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616570 0x55856962f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.688+0000 7faef6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616570 0x55856962f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55056/0 (socket says 172.21.15.131:55056) 2024-05-10T05:37:58.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.690+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x558569615ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.690+0000 7faf164a1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856961da90 con 0x558569616ec0 2024-05-10T05:37:58.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.690+0000 7faf164a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856962f7b0 con 0x5585696156a0 2024-05-10T05:37:58.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.690+0000 7faf164a1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856962f970 con 0x558569616570 2024-05-10T05:37:58.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.690+0000 7faef6a00700 1 -- 172.21.15.131:0/2062608574 learned_addr learned my addr 172.21.15.131:0/2062608574 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:58.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.691+0000 7faef7e00700 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x558569615ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.691+0000 7faef6a00700 1 -- 172.21.15.131:0/2062608574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585696156a0 msgr2=0x558569616980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:58.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.692+0000 7faef7400700 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585696156a0 0x558569616980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.692+0000 7faef6a00700 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585696156a0 0x558569616980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.692+0000 7faef6a00700 1 -- 172.21.15.131:0/2062608574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 msgr2=0x558569615ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.692+0000 7faef6a00700 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x558569615ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.692+0000 7faef6a00700 1 -- 172.21.15.131:0/2062608574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585694fb560 con 0x558569616570 2024-05-10T05:37:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.692+0000 7faef7e00700 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x558569615ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:58.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.693+0000 7faef7400700 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585696156a0 0x558569616980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.693+0000 7faef6a00700 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616570 0x55856962f160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faee8015fa0 tx=0x7faee8009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb7516627a50a3a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.695+0000 7faeed600700 1 -- 172.21.15.131:0/2062608574 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee801a040 con 0x558569616570 2024-05-10T05:37:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.695+0000 7faeed600700 1 -- 172.21.15.131:0/2062608574 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee80093f0 con 0x558569616570 2024-05-10T05:37:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.695+0000 7faeed600700 1 -- 172.21.15.131:0/2062608574 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80136f0 con 0x558569616570 2024-05-10T05:37:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.696+0000 7faf164a1b80 1 -- 172.21.15.131:0/2062608574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616570 msgr2=0x55856962f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:58.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.696+0000 7faf164a1b80 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616570 0x55856962f160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faee8015fa0 tx=0x7faee8009380 comp rx=0 tx=0).stop 2024-05-10T05:37:58.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.697+0000 7faf164a1b80 1 -- 172.21.15.131:0/2062608574 shutdown_connections 2024-05-10T05:37:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.697+0000 7faf164a1b80 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x558569615ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.697+0000 7faf164a1b80 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585696156a0 0x558569616980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.697+0000 7faf164a1b80 1 --2- 172.21.15.131:0/2062608574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558569616570 0x55856962f160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.697+0000 7faf164a1b80 1 -- 172.21.15.131:0/2062608574 >> 172.21.15.131:0/2062608574 conn(0x558569577a60 msgr2=0x55856961ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.698+0000 7faf164a1b80 1 -- 172.21.15.131:0/2062608574 shutdown_connections 2024-05-10T05:37:58.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.698+0000 7faf164a1b80 1 -- 172.21.15.131:0/2062608574 wait complete. 2024-05-10T05:37:58.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.699+0000 7faf164a1b80 1 -- start start 2024-05-10T05:37:58.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.699+0000 7faf164a1b80 1 Processor -- start 2024-05-10T05:37:58.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.703+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585696156a0 0x55856962cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.703+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x55856962d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.704+0000 7faf164a1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x55856962df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.704+0000 7faef6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x55856962d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.704+0000 7faef6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x55856962d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41060/0 (socket says 172.21.15.131:41060) 2024-05-10T05:37:58.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.704+0000 7faef6a00700 1 -- 172.21.15.131:0/3978625930 learned_addr learned my addr 172.21.15.131:0/3978625930 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:58.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.704+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856962e490 con 0x558569616ec0 2024-05-10T05:37:58.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.704+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856962e600 con 0x558569616570 2024-05-10T05:37:58.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.704+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856962e7c0 con 0x5585696156a0 2024-05-10T05:37:58.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.705+0000 7faf164a1b80 10 client.0 _ll_register_callbacks cb 0x5585694dab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:58.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.705+0000 7faef7e00700 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x55856962df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.706+0000 7faef7e00700 1 -- 172.21.15.131:0/3978625930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585696156a0 msgr2=0x55856962cae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:37:58.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.706+0000 7faef7e00700 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585696156a0 0x55856962cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.706+0000 7faef7e00700 1 -- 172.21.15.131:0/3978625930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 msgr2=0x55856962d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.707+0000 7faef7e00700 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x55856962d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.707+0000 7faef7e00700 1 -- 172.21.15.131:0/3978625930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585693b63b0 con 0x558569616ec0 2024-05-10T05:37:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.708+0000 7faef6a00700 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x55856962d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:37:58.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.708+0000 7faef7e00700 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x55856962df50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5585696106c0 tx=0x7faee40050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c1db467b402994 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:58.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.708+0000 7faee1600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:37:58.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.709+0000 7faee1600700 1 -- 172.21.15.131:0/3978625930 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4004710 con 0x558569616ec0 2024-05-10T05:37:58.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.709+0000 7faee1600700 1 -- 172.21.15.131:0/3978625930 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7faee4004870 con 0x558569616ec0 2024-05-10T05:37:58.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.709+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faee8012970 con 0x558569616ec0 2024-05-10T05:37:58.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.710+0000 7faee1600700 1 -- 172.21.15.131:0/3978625930 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7faee4004a30 con 0x558569616ec0 2024-05-10T05:37:58.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.710+0000 7faf164a1b80 10 client.16095 fetch_fsmap learned FSMap version 29 2024-05-10T05:37:58.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.710+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5585693bacd0 con 0x558569616ec0 2024-05-10T05:37:58.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.711+0000 7faee1600700 1 -- 172.21.15.131:0/3978625930 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee8012970 con 0x558569616ec0 2024-05-10T05:37:58.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.711+0000 7faee1600700 1 -- 172.21.15.131:0/3978625930 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faee4021070 con 0x558569616ec0 2024-05-10T05:37:58.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.716+0000 7faf164a1b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:37:58.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.716+0000 7faf164a1b80 10 client.16095 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:37:58.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.716+0000 7faf164a1b80 20 client.16095 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:58.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.716+0000 7faf164a1b80 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55856968a680 0x55856968caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:58.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.716+0000 7faf164a1b80 4 client.16095 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:37:58.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.716+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55856952dec0 con 0x55856968a680 2024-05-10T05:37:58.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.717+0000 7faef7400700 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55856968a680 0x55856968caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:58.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.717+0000 7faef7400700 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55856968a680 0x55856968caa0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=484a881e89344511 server_cookie=55709dcbd34e5f35 in_seq=0 out_seq=0 2024-05-10T05:37:58.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.717+0000 7faee1600700 10 client.16095 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:37:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.718+0000 7faee1600700 1 -- 172.21.15.131:0/3978625930 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55856952dec0 con 0x55856968a680 2024-05-10T05:37:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.718+0000 7faee1600700 10 client.16095 handle_command_reply: tid=0 2024-05-10T05:37:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.718+0000 7faf164a1b80 1 client.16095 shutdown 2024-05-10T05:37:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.718+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 msgr2=0x55856962df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.718+0000 7faf164a1b80 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x55856962df50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5585696106c0 tx=0x7faee40050f0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.719+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 shutdown_connections 2024-05-10T05:37:58.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.719+0000 7faf164a1b80 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55856968a680 0x55856968caa0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.719+0000 7faf164a1b80 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558569616ec0 0x55856962df50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.719+0000 7faf164a1b80 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5585696156a0 0x55856962cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.719+0000 7faf164a1b80 1 --2- 172.21.15.131:0/3978625930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558569616570 0x55856962d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.719+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 >> 172.21.15.131:0/3978625930 conn(0x558569577a60 msgr2=0x55856960c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:58.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.720+0000 7faee1600700 0 client.16095 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:37:58.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.720+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 shutdown_connections 2024-05-10T05:37:58.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.720+0000 7faf164a1b80 1 -- 172.21.15.131:0/3978625930 wait complete. 2024-05-10T05:37:58.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.721+0000 7faf164a1b80 20 client.16095 trim_cache size 0 max 16384 2024-05-10T05:37:58.724 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:37:58.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.727+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faedc0a1400 msgr2=0x7faedc0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:58.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.727+0000 7faefe600700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faedc0a1400 0x7faedc0a3910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faef81a1870 tx=0x7faee8006cd0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.727+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef8108210 msgr2=0x7faef81a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:58.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.727+0000 7faefe600700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef8108210 0x7faef81a07d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faef000b9b0 tx=0x7faef000bcc0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.736+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 shutdown_connections 2024-05-10T05:37:58.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.736+0000 7faefe600700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7faedc0a1400 0x7faedc0a3910 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.736+0000 7faefe600700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef8110320 0x7faef81a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.736+0000 7faefe600700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef8108210 0x7faef81a07d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.736+0000 7faefe600700 1 --2- 172.21.15.131:0/2004155497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef8108b60 0x7faef81a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:58.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.736+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 >> 172.21.15.131:0/2004155497 conn(0x7faef806ea00 msgr2=0x7faef810bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:58.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.739+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 shutdown_connections 2024-05-10T05:37:58.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:58.739+0000 7faefe600700 1 -- 172.21.15.131:0/2004155497 wait complete. 2024-05-10T05:37:58.838 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:37:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.539+0000 7f2a67e00700 1 -- 172.21.15.131:0/2155128023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a58094f00 msgr2=0x7f2a58097320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.539+0000 7f2a67e00700 1 --2- 172.21.15.131:0/2155128023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a58094f00 0x7f2a58097320 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009940 tx=0x7f2a54009c50 comp rx=0 tx=0).stop 2024-05-10T05:37:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.539+0000 7f2a67e00700 1 -- 172.21.15.131:0/2155128023 shutdown_connections 2024-05-10T05:37:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.539+0000 7f2a67e00700 1 --2- 172.21.15.131:0/2155128023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a58094f00 0x7f2a58097320 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.539+0000 7f2a67e00700 1 --2- 172.21.15.131:0/2155128023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5809a1c0 0x7f2a5809a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.539+0000 7f2a67e00700 1 --2- 172.21.15.131:0/2155128023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a58097860 0x7f2a58099c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.539+0000 7f2a67e00700 1 -- 172.21.15.131:0/2155128023 >> 172.21.15.131:0/2155128023 conn(0x7f2a58090b10 msgr2=0x7f2a58092fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:59.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.543+0000 7f2a67e00700 1 -- 172.21.15.131:0/2155128023 shutdown_connections 2024-05-10T05:37:59.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.544+0000 7f2a67e00700 1 -- 172.21.15.131:0/2155128023 wait complete. 2024-05-10T05:37:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.549+0000 7f2a67e00700 1 Processor -- start 2024-05-10T05:37:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.549+0000 7f2a67e00700 1 -- start start 2024-05-10T05:37:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.550+0000 7f2a67e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a58094f00 0x7f2a58133160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.550+0000 7f2a67e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5809a1c0 0x7f2a581336a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.550+0000 7f2a67e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a58138620 0x7f2a5813aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.550+0000 7f2a67e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5813b010 con 0x7f2a5809a1c0 2024-05-10T05:37:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.550+0000 7f2a67e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5813b150 con 0x7f2a58094f00 2024-05-10T05:37:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.550+0000 7f2a67e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5813b2e0 con 0x7f2a58138620 2024-05-10T05:37:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.550+0000 7f2a65400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a58094f00 0x7f2a58133160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.551+0000 7f2a64a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5809a1c0 0x7f2a581336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.551+0000 7f2a65400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a58094f00 0x7f2a58133160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41092/0 (socket says 172.21.15.131:41092) 2024-05-10T05:37:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.551+0000 7f2a65400700 1 -- 172.21.15.131:0/3509861791 learned_addr learned my addr 172.21.15.131:0/3509861791 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.551+0000 7f2a65400700 1 -- 172.21.15.131:0/3509861791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a58138620 msgr2=0x7f2a5813aa40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:37:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.551+0000 7f2a65400700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a58138620 0x7f2a5813aa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.551+0000 7f2a65400700 1 -- 172.21.15.131:0/3509861791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5809a1c0 msgr2=0x7f2a581336a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.551+0000 7f2a65400700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5809a1c0 0x7f2a581336a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.551+0000 7f2a65400700 1 -- 172.21.15.131:0/3509861791 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a54009520 con 0x7f2a58094f00 2024-05-10T05:37:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.552+0000 7f2a65400700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a58094f00 0x7f2a58133160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2a54005950 tx=0x7f2a5400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.552+0000 7f2a5e000700 1 -- 172.21.15.131:0/3509861791 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5401c070 con 0x7f2a58094f00 2024-05-10T05:37:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.552+0000 7f2a5e000700 1 -- 172.21.15.131:0/3509861791 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a54005ce0 con 0x7f2a58094f00 2024-05-10T05:37:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.552+0000 7f2a5e000700 1 -- 172.21.15.131:0/3509861791 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54003c70 con 0x7f2a58094f00 2024-05-10T05:37:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.552+0000 7f2a67e00700 1 -- 172.21.15.131:0/3509861791 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5813b560 con 0x7f2a58094f00 2024-05-10T05:37:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.554+0000 7f2a67e00700 1 -- 172.21.15.131:0/3509861791 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5813b9b0 con 0x7f2a58094f00 2024-05-10T05:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.562+0000 7f2a5e000700 1 -- 172.21.15.131:0/3509861791 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2a5402a440 con 0x7f2a58094f00 2024-05-10T05:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.564+0000 7f2a5e000700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a480a11a0 0x7f2a480a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.566+0000 7f2a64a00700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a480a11a0 0x7f2a480a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.566+0000 7f2a7fdfdb80 1 Processor -- start 2024-05-10T05:37:59.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.567+0000 7f2a5e000700 1 -- 172.21.15.131:0/3509861791 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2a540cd810 con 0x7f2a58094f00 2024-05-10T05:37:59.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.568+0000 7f2a7fdfdb80 1 -- start start 2024-05-10T05:37:59.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.569+0000 7f2a64a00700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a480a11a0 0x7f2a480a36b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2a58134200 tx=0x7f2a4c009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:59.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.569+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a39530 0x55b4f9a39960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.569+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a3abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.569+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a419f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.569+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f9960150 con 0x55b4f9a3b0f0 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.569+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f995db10 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.569+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f9959400 con 0x55b4f9a39530 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.570+0000 7f2a64a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a3abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.570+0000 7f2a64a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a3abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41100/0 (socket says 172.21.15.131:41100) 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.570+0000 7f2a64a00700 1 -- 172.21.15.131:0/4106293449 learned_addr learned my addr 172.21.15.131:0/4106293449 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.570+0000 7f2a65400700 1 --2- 172.21.15.131:0/4106293449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a39530 0x55b4f9a39960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.570+0000 7f2a64a00700 1 -- 172.21.15.131:0/4106293449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a39530 msgr2=0x55b4f9a39960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:59.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.570+0000 7f2a64a00700 1 --2- 172.21.15.131:0/4106293449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a39530 0x55b4f9a39960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.570+0000 7f2a64a00700 1 -- 172.21.15.131:0/4106293449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3b0f0 msgr2=0x55b4f9a419f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.570+0000 7f2a64a00700 1 --2- 172.21.15.131:0/4106293449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a419f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.570+0000 7f2a64a00700 1 -- 172.21.15.131:0/4106293449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4f97da3b0 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.571+0000 7f2a64a00700 1 --2- 172.21.15.131:0/4106293449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a3abb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c011c90 tx=0x7f2a4c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac5c7ba1262a620 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.572+0000 7f2a52600700 1 -- 172.21.15.131:0/4106293449 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c004a30 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.572+0000 7f2a52600700 1 -- 172.21.15.131:0/4106293449 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a4c004580 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.572+0000 7f2a52600700 1 -- 172.21.15.131:0/4106293449 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0039f0 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.579+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/4106293449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3a7a0 msgr2=0x55b4f9a3abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:59.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.580+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/4106293449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a3abb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c011c90 tx=0x7f2a4c014000 comp rx=0 tx=0).stop 2024-05-10T05:37:59.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.581+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/4106293449 shutdown_connections 2024-05-10T05:37:59.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.581+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/4106293449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.581+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/4106293449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a39530 0x55b4f9a39960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.581+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/4106293449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a3abb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.581+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/4106293449 >> 172.21.15.131:0/4106293449 conn(0x55b4f999ba60 msgr2=0x55b4f9a34330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.583+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/4106293449 shutdown_connections 2024-05-10T05:37:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.584+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/4106293449 wait complete. 2024-05-10T05:37:59.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.591+0000 7f2a7fdfdb80 1 -- start start 2024-05-10T05:37:59.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.602+0000 7f2a7fdfdb80 1 Processor -- start 2024-05-10T05:37:59.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.612+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a39530 0x55b4f9a54920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.612+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a54e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.612+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a5bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.613+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f98f5560 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.613+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f98f5200 con 0x55b4f9a3b0f0 2024-05-10T05:37:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.613+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f97dc6a0 con 0x55b4f9a39530 2024-05-10T05:37:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.613+0000 7f2a7fdfdb80 10 client.0 _ll_register_callbacks cb 0x55b4f98feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.614+0000 7f2a64a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a54e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.614+0000 7f2a64a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a54e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48156/0 (socket says 172.21.15.131:48156) 2024-05-10T05:37:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.614+0000 7f2a64a00700 1 -- 172.21.15.131:0/579309458 learned_addr learned my addr 172.21.15.131:0/579309458 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.615+0000 7f2a64a00700 1 -- 172.21.15.131:0/579309458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a39530 msgr2=0x55b4f9a54920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:37:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.615+0000 7f2a64a00700 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a39530 0x55b4f9a54920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.615+0000 7f2a64a00700 1 -- 172.21.15.131:0/579309458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3b0f0 msgr2=0x55b4f9a5bdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:37:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.615+0000 7f2a64a00700 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a5bdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.615+0000 7f2a64a00700 1 -- 172.21.15.131:0/579309458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c017070 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.616+0000 7f2a46a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:37:59.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.617+0000 7f2a64a00700 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a54e60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0097c0 tx=0x7f2a4c0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2744143930432d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:59.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.617+0000 7f2a46a00700 1 -- 172.21.15.131:0/579309458 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c02c070 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.618+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/579309458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4f98b2270 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.618+0000 7f2a46a00700 1 -- 172.21.15.131:0/579309458 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2a4c0036f0 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.619+0000 7f2a46a00700 1 -- 172.21.15.131:0/579309458 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a4c02c070 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.619+0000 7f2a7fdfdb80 10 client.16104 fetch_fsmap learned FSMap version 29 2024-05-10T05:37:59.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.619+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/579309458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b4f9786b60 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.621+0000 7f2a46a00700 1 -- 172.21.15.131:0/579309458 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a4c038030 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.622+0000 7f2a46a00700 1 -- 172.21.15.131:0/579309458 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2a4c024030 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.623+0000 7f2a7fdfdb80 10 client.16104 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:37:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.623+0000 7f2a7fdfdb80 10 client.16104 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:37:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.624+0000 7f2a7fdfdb80 20 client.16104 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.624+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55b4f9a60580 0x55b4f9a629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.624+0000 7f2a7fdfdb80 4 client.16104 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:37:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.624+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/579309458 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4f9a08780 con 0x55b4f9a60580 2024-05-10T05:37:59.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.626+0000 7f2a65400700 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55b4f9a60580 0x55b4f9a629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.628+0000 7f2a65400700 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55b4f9a60580 0x55b4f9a629d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7185e9b1574d1e80 server_cookie=bf20c4f704466749 in_seq=0 out_seq=0 2024-05-10T05:37:59.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.628+0000 7f2a46a00700 10 client.16104 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:37:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.629+0000 7f2a46a00700 1 -- 172.21.15.131:0/579309458 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4f9a08780 con 0x55b4f9a60580 2024-05-10T05:37:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.629+0000 7f2a46a00700 10 client.16104 handle_command_reply: tid=0 2024-05-10T05:37:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.630+0000 7f2a7fdfdb80 1 client.16104 shutdown 2024-05-10T05:37:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.631+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/579309458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3a7a0 msgr2=0x55b4f9a54e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.631+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a54e60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0097c0 tx=0x7f2a4c0052e0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.631+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/579309458 shutdown_connections 2024-05-10T05:37:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.631+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55b4f9a60580 0x55b4f9a629d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.631+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a5bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.631+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a39530 0x55b4f9a54920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.631+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/579309458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a54e60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.631+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/579309458 >> 172.21.15.131:0/579309458 conn(0x55b4f999ba60 msgr2=0x55b4f9a38060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.632+0000 7f2a46a00700 0 client.16104 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:37:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.632+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/579309458 shutdown_connections 2024-05-10T05:37:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.632+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/579309458 wait complete. 2024-05-10T05:37:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.633+0000 7f2a7fdfdb80 20 client.16104 trim_cache size 0 max 16384 2024-05-10T05:37:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.655+0000 7f2a7fdfdb80 1 Processor -- start 2024-05-10T05:37:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.655+0000 7f2a7fdfdb80 1 -- start start 2024-05-10T05:37:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.656+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a39530 0x55b4f9a62ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.657+0000 7f2a65400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a39530 0x55b4f9a62ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.657+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a59d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.657+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a5a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.657+0000 7f2a65400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a39530 0x55b4f9a62ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41118/0 (socket says 172.21.15.131:41118) 2024-05-10T05:37:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.657+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f9a35be0 con 0x55b4f9a3b0f0 2024-05-10T05:37:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.657+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f9a2e900 con 0x55b4f9a39530 2024-05-10T05:37:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.657+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f9a5c820 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.657+0000 7f2a65400700 1 -- 172.21.15.131:0/1899864748 learned_addr learned my addr 172.21.15.131:0/1899864748 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:59.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.657+0000 7f2a64a00700 1 --2- 172.21.15.131:0/1899864748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a59d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.657+0000 7f2a65e00700 1 --2- 172.21.15.131:0/1899864748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a5a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.658+0000 7f2a65400700 1 -- 172.21.15.131:0/1899864748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 msgr2=0x55b4f9a59d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:59.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.658+0000 7f2a65400700 1 --2- 172.21.15.131:0/1899864748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a59d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.658+0000 7f2a65400700 1 -- 172.21.15.131:0/1899864748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3b0f0 msgr2=0x55b4f9a5a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:37:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.658+0000 7f2a65400700 1 --2- 172.21.15.131:0/1899864748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a5a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.658+0000 7f2a65400700 1 -- 172.21.15.131:0/1899864748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4f991f560 con 0x55b4f9a39530 2024-05-10T05:37:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.658+0000 7f2a65e00700 1 --2- 172.21.15.131:0/1899864748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a5a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.658+0000 7f2a65400700 1 --2- 172.21.15.131:0/1899864748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a39530 0x55b4f9a62ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400c010 tx=0x7f2a5400bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb53c084a1f894f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.659+0000 7f2a53000700 1 -- 172.21.15.131:0/1899864748 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5409eac0 con 0x55b4f9a39530 2024-05-10T05:37:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.659+0000 7f2a53000700 1 -- 172.21.15.131:0/1899864748 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a540a3080 con 0x55b4f9a39530 2024-05-10T05:37:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.659+0000 7f2a53000700 1 -- 172.21.15.131:0/1899864748 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540055c0 con 0x55b4f9a39530 2024-05-10T05:37:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.660+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/1899864748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a39530 msgr2=0x55b4f9a62ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.660+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/1899864748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a39530 0x55b4f9a62ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400c010 tx=0x7f2a5400bd70 comp rx=0 tx=0).stop 2024-05-10T05:37:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.660+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/1899864748 shutdown_connections 2024-05-10T05:37:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.660+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/1899864748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9a5a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.660+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/1899864748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a39530 0x55b4f9a62ca0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.660+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/1899864748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a59d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.660+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/1899864748 >> 172.21.15.131:0/1899864748 conn(0x55b4f999ba60 msgr2=0x55b4f9a62590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.660+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/1899864748 shutdown_connections 2024-05-10T05:37:59.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.660+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/1899864748 wait complete. 2024-05-10T05:37:59.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.661+0000 7f2a7fdfdb80 1 -- start start 2024-05-10T05:37:59.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.661+0000 7f2a7fdfdb80 1 Processor -- start 2024-05-10T05:37:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.662+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a39530 0x55b4f9a59750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.662+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a59c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.662+0000 7f2a7fdfdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9ab4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.662+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f9ab5280 con 0x55b4f9a39530 2024-05-10T05:37:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.662+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f9ab53c0 con 0x55b4f9a3b0f0 2024-05-10T05:37:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.662+0000 7f2a7fdfdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f9ab5550 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.662+0000 7f2a7fdfdb80 10 client.0 _ll_register_callbacks cb 0x55b4f98feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:37:59.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.663+0000 7f2a64a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a59c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.663+0000 7f2a64a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a59c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55100/0 (socket says 172.21.15.131:55100) 2024-05-10T05:37:59.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.663+0000 7f2a64a00700 1 -- 172.21.15.131:0/3137583348 learned_addr learned my addr 172.21.15.131:0/3137583348 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:37:59.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.663+0000 7f2a64a00700 1 -- 172.21.15.131:0/3137583348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3b0f0 msgr2=0x55b4f9ab4d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:59.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.663+0000 7f2a65e00700 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9ab4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.663+0000 7f2a64a00700 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9ab4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.663+0000 7f2a64a00700 1 -- 172.21.15.131:0/3137583348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a39530 msgr2=0x55b4f9a59750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:37:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.663+0000 7f2a65400700 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a39530 0x55b4f9a59750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.664+0000 7f2a64a00700 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a39530 0x55b4f9a59750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.664+0000 7f2a64a00700 1 -- 172.21.15.131:0/3137583348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5813b9b0 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.664+0000 7f2a65400700 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a39530 0x55b4f9a59750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:37:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.664+0000 7f2a64a00700 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a59c90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009a80 tx=0x7f2a4c005920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae0b6956a6367638 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:37:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.664+0000 7f2a46a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:37:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.664+0000 7f2a46a00700 1 -- 172.21.15.131:0/3137583348 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c02c040 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.664+0000 7f2a46a00700 1 -- 172.21.15.131:0/3137583348 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2a4c0036f0 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.664+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/3137583348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a4c004580 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.665+0000 7f2a46a00700 1 -- 172.21.15.131:0/3137583348 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a4c0255c0 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.665+0000 7f2a7fdfdb80 10 client.26218 fetch_fsmap learned FSMap version 29 2024-05-10T05:37:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.665+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/3137583348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b4f97decd0 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.665+0000 7f2a46a00700 1 -- 172.21.15.131:0/3137583348 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a4c004580 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.666+0000 7f2a46a00700 1 -- 172.21.15.131:0/3137583348 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2a4c0040f0 con 0x55b4f9a3a7a0 2024-05-10T05:37:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.666+0000 7f2a7fdfdb80 10 client.26218 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:37:59.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.666+0000 7f2a7fdfdb80 10 client.26218 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:37:59.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.666+0000 7f2a7fdfdb80 20 client.26218 populate_metadata read hostname 'smithi131' 2024-05-10T05:37:59.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.666+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55b4f9ac7900 0x55b4f9ac9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:37:59.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.666+0000 7f2a7fdfdb80 4 client.26218 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:37:59.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.666+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/3137583348 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b4f9951ec0 con 0x55b4f9ac7900 2024-05-10T05:37:59.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.666+0000 7f2a65400700 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55b4f9ac7900 0x55b4f9ac9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:37:59.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.667+0000 7f2a46a00700 10 client.26218 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:37:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.667+0000 7f2a65400700 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55b4f9ac7900 0x55b4f9ac9d20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6eddd3e78fcae94 server_cookie=775349db9ce06e97 in_seq=0 out_seq=0 2024-05-10T05:37:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.668+0000 7f2a46a00700 1 -- 172.21.15.131:0/3137583348 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b4f9951ec0 con 0x55b4f9ac7900 2024-05-10T05:37:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.668+0000 7f2a46a00700 10 client.26218 handle_command_reply: tid=0 2024-05-10T05:37:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.668+0000 7f2a7fdfdb80 1 client.26218 shutdown 2024-05-10T05:37:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.669+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/3137583348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 msgr2=0x55b4f9a59c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.669+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a59c90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009a80 tx=0x7f2a4c005920 comp rx=0 tx=0).stop 2024-05-10T05:37:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.669+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/3137583348 shutdown_connections 2024-05-10T05:37:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.669+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55b4f9ac7900 0x55b4f9ac9d20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.669+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4f9a39530 0x55b4f9a59750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.669+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4f9a3b0f0 0x55b4f9ab4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.669+0000 7f2a7fdfdb80 1 --2- 172.21.15.131:0/3137583348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4f9a3a7a0 0x55b4f9a59c90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.669+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/3137583348 >> 172.21.15.131:0/3137583348 conn(0x55b4f999ba60 msgr2=0x55b4f9a33bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.669+0000 7f2a46a00700 0 client.26218 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:37:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.669+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/3137583348 shutdown_connections 2024-05-10T05:37:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.670+0000 7f2a7fdfdb80 1 -- 172.21.15.131:0/3137583348 wait complete. 2024-05-10T05:37:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.670+0000 7f2a7fdfdb80 20 client.26218 trim_cache size 0 max 16384 2024-05-10T05:37:59.671 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:37:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.671+0000 7f2a67e00700 1 -- 172.21.15.131:0/3509861791 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a480a11a0 msgr2=0x7f2a480a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.671+0000 7f2a67e00700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a480a11a0 0x7f2a480a36b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2a58134200 tx=0x7f2a4c009310 comp rx=0 tx=0).stop 2024-05-10T05:37:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.671+0000 7f2a67e00700 1 -- 172.21.15.131:0/3509861791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a58094f00 msgr2=0x7f2a58133160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:37:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.671+0000 7f2a67e00700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a58094f00 0x7f2a58133160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2a54005950 tx=0x7f2a5400b8c0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.671+0000 7f2a67e00700 1 -- 172.21.15.131:0/3509861791 shutdown_connections 2024-05-10T05:37:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.671+0000 7f2a67e00700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a480a11a0 0x7f2a480a36b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.671+0000 7f2a67e00700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a58138620 0x7f2a5813aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.671+0000 7f2a67e00700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a58094f00 0x7f2a58133160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.671+0000 7f2a67e00700 1 --2- 172.21.15.131:0/3509861791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5809a1c0 0x7f2a581336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:37:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.671+0000 7f2a67e00700 1 -- 172.21.15.131:0/3509861791 >> 172.21.15.131:0/3509861791 conn(0x7f2a58090b10 msgr2=0x7f2a58098350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:37:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.672+0000 7f2a67e00700 1 -- 172.21.15.131:0/3509861791 shutdown_connections 2024-05-10T05:37:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:37:59.672+0000 7f2a67e00700 1 -- 172.21.15.131:0/3509861791 wait complete. 2024-05-10T05:37:59.724 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:38:00.123 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:00 smithi131 ceph-mon[37264]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 255 MiB/s wr, 76 op/s 2024-05-10T05:38:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:00 smithi142 ceph-mon[77667]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 255 MiB/s wr, 76 op/s 2024-05-10T05:38:00.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.454+0000 7f5ecf000700 1 -- 172.21.15.131:0/4233143105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8102580 msgr2=0x7f5ec80683c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:00.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.454+0000 7f5ecf000700 1 --2- 172.21.15.131:0/4233143105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8102580 0x7f5ec80683c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009f90 tx=0x7f5ec000b330 comp rx=0 tx=0).stop 2024-05-10T05:38:00.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.454+0000 7f5ecf000700 1 -- 172.21.15.131:0/4233143105 shutdown_connections 2024-05-10T05:38:00.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.454+0000 7f5ecf000700 1 --2- 172.21.15.131:0/4233143105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8102580 0x7f5ec80683c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.454+0000 7f5ecf000700 1 --2- 172.21.15.131:0/4233143105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ec8100ac0 0x7f5ec8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.454+0000 7f5ecf000700 1 --2- 172.21.15.131:0/4233143105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec8101be0 0x7f5ec8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.454+0000 7f5ecf000700 1 -- 172.21.15.131:0/4233143105 >> 172.21.15.131:0/4233143105 conn(0x7f5ec80fc0d0 msgr2=0x7f5ec80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.456+0000 7f5ecf000700 1 -- 172.21.15.131:0/4233143105 shutdown_connections 2024-05-10T05:38:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.456+0000 7f5ecf000700 1 -- 172.21.15.131:0/4233143105 wait complete. 2024-05-10T05:38:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.468+0000 7f5ecf000700 1 Processor -- start 2024-05-10T05:38:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.468+0000 7f5ecf000700 1 -- start start 2024-05-10T05:38:00.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.471+0000 7f5ecf000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ec8100ac0 0x7f5ec810e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.471+0000 7f5ecf000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8101be0 0x7f5ec810edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.471+0000 7f5ec7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ec8100ac0 0x7f5ec810e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.471+0000 7f5ec7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ec8100ac0 0x7f5ec810e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41130/0 (socket says 172.21.15.131:41130) 2024-05-10T05:38:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.472+0000 7f5ebf400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8101be0 0x7f5ec810edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.474+0000 7f5ec7e00700 1 -- 172.21.15.131:0/1420610291 learned_addr learned my addr 172.21.15.131:0/1420610291 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.474+0000 7f5ecf000700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec8102580 0x7f5ec810b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.474+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec810bb80 con 0x7f5ec8102580 2024-05-10T05:38:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.474+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec810bcf0 con 0x7f5ec8100ac0 2024-05-10T05:38:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.474+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec810beb0 con 0x7f5ec8101be0 2024-05-10T05:38:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.475+0000 7f5ecd000700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec8102580 0x7f5ec810b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.475+0000 7f5ec7e00700 1 -- 172.21.15.131:0/1420610291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8101be0 msgr2=0x7f5ec810edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.475+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8101be0 0x7f5ec810edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.475+0000 7f5ec7e00700 1 -- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec8102580 msgr2=0x7f5ec810b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.475+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec8102580 0x7f5ec810b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.475+0000 7f5ec7e00700 1 -- 172.21.15.131:0/1420610291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0009520 con 0x7f5ec8100ac0 2024-05-10T05:38:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.475+0000 7f5ebf400700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8101be0 0x7f5ec810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:38:00.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.477+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ec8100ac0 0x7f5ec810e870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800e9c0 tx=0x7f5eb800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:00.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.477+0000 7f5ec5600700 1 -- 172.21.15.131:0/1420610291 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8004220 con 0x7f5ec8100ac0 2024-05-10T05:38:00.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.477+0000 7f5ec5600700 1 -- 172.21.15.131:0/1420610291 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb8004380 con 0x7f5ec8100ac0 2024-05-10T05:38:00.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.477+0000 7f5ec5600700 1 -- 172.21.15.131:0/1420610291 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8003c00 con 0x7f5ec8100ac0 2024-05-10T05:38:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.478+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec0009c40 con 0x7f5ec8100ac0 2024-05-10T05:38:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.479+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec81a6700 con 0x7f5ec8100ac0 2024-05-10T05:38:00.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.491+0000 7f5ec5600700 1 -- 172.21.15.131:0/1420610291 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5eb8018430 con 0x7f5ec8100ac0 2024-05-10T05:38:00.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.492+0000 7f5ee6f12b80 1 Processor -- start 2024-05-10T05:38:00.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.493+0000 7f5ee6f12b80 1 -- start start 2024-05-10T05:38:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.493+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4387af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.493+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4388570 0x5587d4388980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.494+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d438f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.494+0000 7f5ecd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4387af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.494+0000 7f5ecd000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4387af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55126/0 (socket says 172.21.15.131:55126) 2024-05-10T05:38:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.495+0000 7f5ecd000700 1 -- 172.21.15.131:0/154898152 learned_addr learned my addr 172.21.15.131:0/154898152 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.495+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/154898152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d42ae150 con 0x5587d4388ec0 2024-05-10T05:38:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.495+0000 7f5ebf400700 1 --2- 172.21.15.131:0/154898152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d438f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.495+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/154898152 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d42abb10 con 0x5587d4388570 2024-05-10T05:38:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.495+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/154898152 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d42a7400 con 0x5587d43876a0 2024-05-10T05:38:00.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.496+0000 7f5ec5600700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5eac0a12c0 0x7f5eac0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.496+0000 7f5ec5600700 1 -- 172.21.15.131:0/1420610291 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5eb8014070 con 0x7f5ec8100ac0 2024-05-10T05:38:00.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.496+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/154898152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4388570 0x5587d4388980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.498+0000 7f5ec7e00700 1 -- 172.21.15.131:0/154898152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 msgr2=0x5587d4387af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:00.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.498+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/154898152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4387af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.498+0000 7f5ec7e00700 1 -- 172.21.15.131:0/154898152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 msgr2=0x5587d438f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:00.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.498+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/154898152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d438f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.498+0000 7f5ec7e00700 1 -- 172.21.15.131:0/154898152 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587d41283b0 con 0x5587d4388570 2024-05-10T05:38:00.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.498+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/154898152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4388570 0x5587d4388980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80ca4e0 tx=0x7f5eb8099860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb760df519487d8a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:00.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.498+0000 7f5ebf400700 1 --2- 172.21.15.131:0/154898152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d438f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:00.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.498+0000 7f5ecd000700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5eac0a12c0 0x7f5eac0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.500+0000 7f5ebd600700 1 -- 172.21.15.131:0/154898152 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80b1070 con 0x5587d4388570 2024-05-10T05:38:00.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.500+0000 7f5ebd600700 1 -- 172.21.15.131:0/154898152 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb8099e30 con 0x5587d4388570 2024-05-10T05:38:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.500+0000 7f5ebd600700 1 -- 172.21.15.131:0/154898152 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb801f340 con 0x5587d4388570 2024-05-10T05:38:00.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.501+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/154898152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4388570 msgr2=0x5587d4388980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:00.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.501+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/154898152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4388570 0x5587d4388980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80ca4e0 tx=0x7f5eb8099860 comp rx=0 tx=0).stop 2024-05-10T05:38:00.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.503+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/154898152 shutdown_connections 2024-05-10T05:38:00.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.503+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/154898152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d438f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.504+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/154898152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4387af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.504+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/154898152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4388570 0x5587d4388980 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.504+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/154898152 >> 172.21.15.131:0/154898152 conn(0x5587d42e9a60 msgr2=0x5587d4381fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:00.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.505+0000 7f5ecd000700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5eac0a12c0 0x7f5eac0a37d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5587d4387380 tx=0x7f5ec000b330 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:00.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.506+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/154898152 shutdown_connections 2024-05-10T05:38:00.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.506+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/154898152 wait complete. 2024-05-10T05:38:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.509+0000 7f5ee6f12b80 1 -- start start 2024-05-10T05:38:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.509+0000 7f5ee6f12b80 1 Processor -- start 2024-05-10T05:38:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.514+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4391770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.514+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4388570 0x5587d4391cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.514+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d4398c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.514+0000 7f5ee6f12b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d4243560 con 0x5587d4388ec0 2024-05-10T05:38:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.514+0000 7f5ee6f12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d4243200 con 0x5587d4388570 2024-05-10T05:38:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.514+0000 7f5ee6f12b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d412a6a0 con 0x5587d43876a0 2024-05-10T05:38:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.514+0000 7f5ee6f12b80 10 client.0 _ll_register_callbacks cb 0x5587d424cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.515+0000 7f5ec7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4391770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.515+0000 7f5ec7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4391770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55134/0 (socket says 172.21.15.131:55134) 2024-05-10T05:38:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.515+0000 7f5ec7e00700 1 -- 172.21.15.131:0/1011251597 learned_addr learned my addr 172.21.15.131:0/1011251597 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.515+0000 7f5ecd000700 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d4398c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.516+0000 7f5ec7e00700 1 -- 172.21.15.131:0/1011251597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4388570 msgr2=0x5587d4391cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:38:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.516+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4388570 0x5587d4391cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.516+0000 7f5ec7e00700 1 -- 172.21.15.131:0/1011251597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 msgr2=0x5587d4398c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:00.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.517+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d4398c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.517+0000 7f5ec7e00700 1 -- 172.21.15.131:0/1011251597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec81a6700 con 0x5587d43876a0 2024-05-10T05:38:00.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.517+0000 7f5ecd000700 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d4398c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:38:00.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.518+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4391770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80968a0 tx=0x7f5eb80a2360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27aa68702ab42c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:00.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.518+0000 7f5eb6000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:38:00.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.518+0000 7f5eb6000700 1 -- 172.21.15.131:0/1011251597 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80b1070 con 0x5587d43876a0 2024-05-10T05:38:00.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.519+0000 7f5eb6000700 1 -- 172.21.15.131:0/1011251597 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5eb809ae40 con 0x5587d43876a0 2024-05-10T05:38:00.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.521+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/1011251597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587d4200270 con 0x5587d43876a0 2024-05-10T05:38:00.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.522+0000 7f5eb6000700 1 -- 172.21.15.131:0/1011251597 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5eb8003e20 con 0x5587d43876a0 2024-05-10T05:38:00.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.522+0000 7f5ee6f12b80 10 client.26230 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:00.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.522+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/1011251597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5587d40d4b60 con 0x5587d43876a0 2024-05-10T05:38:00.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.523+0000 7f5eb6000700 1 -- 172.21.15.131:0/1011251597 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb807d020 con 0x5587d43876a0 2024-05-10T05:38:00.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.523+0000 7f5eb6000700 1 -- 172.21.15.131:0/1011251597 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5eb80a1f40 con 0x5587d43876a0 2024-05-10T05:38:00.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.523+0000 7f5ee6f12b80 10 client.26230 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:00.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.523+0000 7f5ee6f12b80 10 client.26230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:38:00.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.523+0000 7f5ee6f12b80 20 client.26230 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:00.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.523+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5587d43ae010 0x5587d43b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.523+0000 7f5ee6f12b80 4 client.26230 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:00.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.523+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/1011251597 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587d4356780 con 0x5587d43ae010 2024-05-10T05:38:00.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.524+0000 7f5ebf400700 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5587d43ae010 0x5587d43b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.528+0000 7f5eb6000700 10 client.26230 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:38:00.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.529+0000 7f5ebf400700 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5587d43ae010 0x5587d43b0460 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9682cb527013c76 server_cookie=9f70f142e21d7575 in_seq=0 out_seq=0 2024-05-10T05:38:00.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.533+0000 7f5eb6000700 1 -- 172.21.15.131:0/1011251597 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587d4356780 con 0x5587d43ae010 2024-05-10T05:38:00.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.533+0000 7f5eb6000700 10 client.26230 handle_command_reply: tid=0 2024-05-10T05:38:00.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.533+0000 7f5ee6f12b80 1 client.26230 shutdown 2024-05-10T05:38:00.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.535+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/1011251597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 msgr2=0x5587d4391770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:00.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.535+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4391770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80968a0 tx=0x7f5eb80a2360 comp rx=0 tx=0).stop 2024-05-10T05:38:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.537+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/1011251597 shutdown_connections 2024-05-10T05:38:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.537+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5587d43ae010 0x5587d43b0460 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.537+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d4398c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.537+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d43876a0 0x5587d4391770 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.537+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/1011251597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4388570 0x5587d4391cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.537+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/1011251597 >> 172.21.15.131:0/1011251597 conn(0x5587d42e9a60 msgr2=0x5587d4381fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:00.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.537+0000 7f5eb6000700 0 client.26230 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:38:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.541+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/1011251597 shutdown_connections 2024-05-10T05:38:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.541+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/1011251597 wait complete. 2024-05-10T05:38:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.543+0000 7f5ee6f12b80 20 client.26230 trim_cache size 0 max 16384 2024-05-10T05:38:00.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:00 smithi157 ceph-mon[78427]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 255 MiB/s wr, 76 op/s 2024-05-10T05:38:00.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.592+0000 7f5ee6f12b80 1 Processor -- start 2024-05-10T05:38:00.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.595+0000 7f5ee6f12b80 1 -- start start 2024-05-10T05:38:00.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.599+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43876a0 0x5587d4390ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.601+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d4388570 0x5587d4398bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.601+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d43b0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.602+0000 7f5ee6f12b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d4387ff0 con 0x5587d4388ec0 2024-05-10T05:38:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.602+0000 7f5ee6f12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d438fae0 con 0x5587d43876a0 2024-05-10T05:38:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.602+0000 7f5ee6f12b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d438f730 con 0x5587d4388570 2024-05-10T05:38:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.602+0000 7f5ebf400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d4388570 0x5587d4398bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.602+0000 7f5ec7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43876a0 0x5587d4390ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.602+0000 7f5ecd000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d43b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.602+0000 7f5ec7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43876a0 0x5587d4390ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41166/0 (socket says 172.21.15.131:41166) 2024-05-10T05:38:00.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.602+0000 7f5ec7e00700 1 -- 172.21.15.131:0/2706293803 learned_addr learned my addr 172.21.15.131:0/2706293803 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:00.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.604+0000 7f5ebf400700 1 -- 172.21.15.131:0/2706293803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43876a0 msgr2=0x5587d4390ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:00.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.604+0000 7f5ebf400700 1 --2- 172.21.15.131:0/2706293803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43876a0 0x5587d4390ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.604+0000 7f5ebf400700 1 -- 172.21.15.131:0/2706293803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 msgr2=0x5587d43b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:00.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.604+0000 7f5ebf400700 1 --2- 172.21.15.131:0/2706293803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d43b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.604+0000 7f5ebf400700 1 -- 172.21.15.131:0/2706293803 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587d426d560 con 0x5587d4388570 2024-05-10T05:38:00.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.604+0000 7f5ecd000700 1 --2- 172.21.15.131:0/2706293803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d43b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:38:00.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.609+0000 7f5ebf400700 1 --2- 172.21.15.131:0/2706293803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d4388570 0x5587d4398bd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x5587d43826c0 tx=0x7f5eb0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f17330f0bd5b4278 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:00.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.610+0000 7f5ebe000700 1 -- 172.21.15.131:0/2706293803 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0018040 con 0x5587d4388570 2024-05-10T05:38:00.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.610+0000 7f5ebe000700 1 -- 172.21.15.131:0/2706293803 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb00044f0 con 0x5587d4388570 2024-05-10T05:38:00.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.610+0000 7f5ebe000700 1 -- 172.21.15.131:0/2706293803 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0004d00 con 0x5587d4388570 2024-05-10T05:38:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.611+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/2706293803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d4388570 msgr2=0x5587d4398bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.611+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/2706293803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d4388570 0x5587d4398bd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x5587d43826c0 tx=0x7f5eb0009260 comp rx=0 tx=0).stop 2024-05-10T05:38:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.613+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/2706293803 shutdown_connections 2024-05-10T05:38:00.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.613+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/2706293803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d4388ec0 0x5587d43b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.613+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/2706293803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d43876a0 0x5587d4390ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.613+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/2706293803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d4388570 0x5587d4398bd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.613+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/2706293803 >> 172.21.15.131:0/2706293803 conn(0x5587d42e9a60 msgr2=0x5587d438c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:00.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.614+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/2706293803 shutdown_connections 2024-05-10T05:38:00.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.614+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/2706293803 wait complete. 2024-05-10T05:38:00.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.616+0000 7f5ee6f12b80 1 -- start start 2024-05-10T05:38:00.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.619+0000 7f5ee6f12b80 1 Processor -- start 2024-05-10T05:38:00.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.621+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d43876a0 0x5587d4396060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.621+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d4388ec0 0x5587d43965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.621+0000 7f5ee6f12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4397430 0x5587d4397840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.621+0000 7f5ee6f12b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d4397d80 con 0x5587d43876a0 2024-05-10T05:38:00.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.621+0000 7f5ee6f12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d4397ef0 con 0x5587d4397430 2024-05-10T05:38:00.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.621+0000 7f5ee6f12b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587d43980b0 con 0x5587d4388ec0 2024-05-10T05:38:00.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.621+0000 7f5ee6f12b80 10 client.0 _ll_register_callbacks cb 0x5587d424cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:00.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.627+0000 7f5ec7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d43876a0 0x5587d4396060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.627+0000 7f5ec7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d43876a0 0x5587d4396060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48206/0 (socket says 172.21.15.131:48206) 2024-05-10T05:38:00.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.627+0000 7f5ec7e00700 1 -- 172.21.15.131:0/889452423 learned_addr learned my addr 172.21.15.131:0/889452423 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:00.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.628+0000 7f5ec7e00700 1 -- 172.21.15.131:0/889452423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d4388ec0 msgr2=0x5587d43965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:00.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.628+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d4388ec0 0x5587d43965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.628+0000 7f5ec7e00700 1 -- 172.21.15.131:0/889452423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4397430 msgr2=0x5587d4397840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:38:00.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.628+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4397430 0x5587d4397840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.628+0000 7f5ec7e00700 1 -- 172.21.15.131:0/889452423 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec8068eb0 con 0x5587d43876a0 2024-05-10T05:38:00.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.628+0000 7f5ec7e00700 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d43876a0 0x5587d4396060 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80968a0 tx=0x7f5eb809dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efaeee3dcc921ee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:00.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.629+0000 7f5eb6000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:38:00.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.632+0000 7f5eb6000700 1 -- 172.21.15.131:0/889452423 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80b1070 con 0x5587d43876a0 2024-05-10T05:38:00.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.632+0000 7f5eb6000700 1 -- 172.21.15.131:0/889452423 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5eb809ae40 con 0x5587d43876a0 2024-05-10T05:38:00.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.632+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/889452423 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5eb8099e30 con 0x5587d43876a0 2024-05-10T05:38:00.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.632+0000 7f5eb6000700 1 -- 172.21.15.131:0/889452423 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5eb8099790 con 0x5587d43876a0 2024-05-10T05:38:00.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.633+0000 7f5ee6f12b80 10 client.16119 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:00.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.633+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/889452423 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f5eb001ab70 con 0x5587d43876a0 2024-05-10T05:38:00.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.634+0000 7f5eb6000700 1 -- 172.21.15.131:0/889452423 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb80a8020 con 0x5587d43876a0 2024-05-10T05:38:00.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.635+0000 7f5eb6000700 1 -- 172.21.15.131:0/889452423 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5eb80a1e20 con 0x5587d43876a0 2024-05-10T05:38:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.635+0000 7f5ee6f12b80 10 client.16119 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.635+0000 7f5ee6f12b80 10 client.16119 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:38:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.635+0000 7f5ee6f12b80 20 client.16119 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.635+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5587d4404950 0x5587d4406dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.635+0000 7f5ee6f12b80 4 client.16119 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:38:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.635+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/889452423 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5587d429fec0 con 0x5587d4404950 2024-05-10T05:38:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.642+0000 7f5ebf400700 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5587d4404950 0x5587d4406dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:00.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.646+0000 7f5ebf400700 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5587d4404950 0x5587d4406dd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6307c6379ada755 server_cookie=aab6e279eec4a191 in_seq=0 out_seq=0 2024-05-10T05:38:00.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.648+0000 7f5eb6000700 10 client.16119 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:38:00.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.650+0000 7f5eb6000700 1 -- 172.21.15.131:0/889452423 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587d429fec0 con 0x5587d4404950 2024-05-10T05:38:00.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.650+0000 7f5eb6000700 10 client.16119 handle_command_reply: tid=0 2024-05-10T05:38:00.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.650+0000 7f5ee6f12b80 1 client.16119 shutdown 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.651+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/889452423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d43876a0 msgr2=0x5587d4396060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.651+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d43876a0 0x5587d4396060 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80968a0 tx=0x7f5eb809dd10 comp rx=0 tx=0).stop 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.651+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/889452423 shutdown_connections 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.651+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5587d4404950 0x5587d4406dd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.651+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d43876a0 0x5587d4396060 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.651+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d4397430 0x5587d4397840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.651+0000 7f5ee6f12b80 1 --2- 172.21.15.131:0/889452423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d4388ec0 0x5587d43965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.651+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/889452423 >> 172.21.15.131:0/889452423 conn(0x5587d42e9a60 msgr2=0x5587d438c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.651+0000 7f5eb6000700 0 client.16119 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.652+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/889452423 shutdown_connections 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.653+0000 7f5ee6f12b80 1 -- 172.21.15.131:0/889452423 wait complete. 2024-05-10T05:38:00.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.653+0000 7f5ee6f12b80 20 client.16119 trim_cache size 0 max 16384 2024-05-10T05:38:00.657 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:38:00.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.655+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5eac0a12c0 msgr2=0x7f5eac0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:00.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.655+0000 7f5ecf000700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5eac0a12c0 0x7f5eac0a37d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5587d4387380 tx=0x7f5ec000b330 comp rx=0 tx=0).stop 2024-05-10T05:38:00.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.655+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ec8100ac0 msgr2=0x7f5ec810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:00.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.655+0000 7f5ecf000700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ec8100ac0 0x7f5ec810e870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800e9c0 tx=0x7f5eb800ecd0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.655+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 shutdown_connections 2024-05-10T05:38:00.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.655+0000 7f5ecf000700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5eac0a12c0 0x7f5eac0a37d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.655+0000 7f5ecf000700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec8102580 0x7f5ec810b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.655+0000 7f5ecf000700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ec8100ac0 0x7f5ec810e870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.655+0000 7f5ecf000700 1 --2- 172.21.15.131:0/1420610291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ec8101be0 0x7f5ec810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:00.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.655+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 >> 172.21.15.131:0/1420610291 conn(0x7f5ec80fc0d0 msgr2=0x7f5ec80fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:00.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.656+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 shutdown_connections 2024-05-10T05:38:00.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:00.657+0000 7f5ecf000700 1 -- 172.21.15.131:0/1420610291 wait complete. 2024-05-10T05:38:00.750 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:38:01.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.643+0000 7fa1bf200700 1 -- 172.21.15.131:0/1632530350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b8109c50 msgr2=0x7fa1b810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:01.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.643+0000 7fa1bf200700 1 --2- 172.21.15.131:0/1632530350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b8109c50 0x7fa1b810e560 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009990 tx=0x7fa1a8009ca0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.649+0000 7fa1bf200700 1 -- 172.21.15.131:0/1632530350 shutdown_connections 2024-05-10T05:38:01.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.649+0000 7fa1bf200700 1 --2- 172.21.15.131:0/1632530350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1b8108630 0x7fa1b8108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.649+0000 7fa1bf200700 1 --2- 172.21.15.131:0/1632530350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b8109c50 0x7fa1b810e560 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.649+0000 7fa1bf200700 1 --2- 172.21.15.131:0/1632530350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1b8109300 0x7fa1b8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.649+0000 7fa1bf200700 1 -- 172.21.15.131:0/1632530350 >> 172.21.15.131:0/1632530350 conn(0x7fa1b806ea00 msgr2=0x7fa1b8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:01.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.649+0000 7fa1bf200700 1 -- 172.21.15.131:0/1632530350 shutdown_connections 2024-05-10T05:38:01.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.649+0000 7fa1bf200700 1 -- 172.21.15.131:0/1632530350 wait complete. 2024-05-10T05:38:01.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.651+0000 7fa1bf200700 1 Processor -- start 2024-05-10T05:38:01.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.651+0000 7fa1bf200700 1 -- start start 2024-05-10T05:38:01.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.656+0000 7fa1bf200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b8108630 0x7fa1b8115340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.656+0000 7fa1bf200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1b8109300 0x7fa1b8115880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.656+0000 7fa1bf200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1b8109c50 0x7fa1b81b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.656+0000 7fa1bf200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81b7150 con 0x7fa1b8109300 2024-05-10T05:38:01.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.656+0000 7fa1bf200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81b7290 con 0x7fa1b8109c50 2024-05-10T05:38:01.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.656+0000 7fa1bf200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81b73d0 con 0x7fa1b8108630 2024-05-10T05:38:01.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.656+0000 7fa1b7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b8108630 0x7fa1b8115340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.657+0000 7fa1bd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1b8109c50 0x7fa1b81b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.657+0000 7fa1bd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1b8109c50 0x7fa1b81b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41194/0 (socket says 172.21.15.131:41194) 2024-05-10T05:38:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.657+0000 7fa1bd200700 1 -- 172.21.15.131:0/4214909106 learned_addr learned my addr 172.21.15.131:0/4214909106 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.657+0000 7fa1b7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b8108630 0x7fa1b8115340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55176/0 (socket says 172.21.15.131:55176) 2024-05-10T05:38:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.657+0000 7fa1bd200700 1 -- 172.21.15.131:0/4214909106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b8108630 msgr2=0x7fa1b8115340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.657+0000 7fa1bd200700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b8108630 0x7fa1b8115340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.657+0000 7fa1bd200700 1 -- 172.21.15.131:0/4214909106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1b8109300 msgr2=0x7fa1b8115880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:38:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.657+0000 7fa1bd200700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1b8109300 0x7fa1b8115880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.657+0000 7fa1bd200700 1 -- 172.21.15.131:0/4214909106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a80095f0 con 0x7fa1b8109c50 2024-05-10T05:38:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.658+0000 7fa1bd200700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1b8109c50 0x7fa1b81b6c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8003ff0 tx=0x7fa1a8004020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:01.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.659+0000 7fa1b4c00700 1 -- 172.21.15.131:0/4214909106 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a801d070 con 0x7fa1b8109c50 2024-05-10T05:38:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.659+0000 7fa1b4c00700 1 -- 172.21.15.131:0/4214909106 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a8004400 con 0x7fa1b8109c50 2024-05-10T05:38:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.659+0000 7fa1b4c00700 1 -- 172.21.15.131:0/4214909106 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8017650 con 0x7fa1b8109c50 2024-05-10T05:38:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.661+0000 7fa1bf200700 1 -- 172.21.15.131:0/4214909106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b81b75d0 con 0x7fa1b8109c50 2024-05-10T05:38:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.661+0000 7fa1bf200700 1 -- 172.21.15.131:0/4214909106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b81b7940 con 0x7fa1b8109c50 2024-05-10T05:38:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.679+0000 7fa1b4c00700 1 -- 172.21.15.131:0/4214909106 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa1a800f460 con 0x7fa1b8109c50 2024-05-10T05:38:01.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.681+0000 7fa1b4c00700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa19c0a1370 0x7fa19c0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.682+0000 7fa1b4c00700 1 -- 172.21.15.131:0/4214909106 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa1a80cdb70 con 0x7fa1b8109c50 2024-05-10T05:38:01.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.683+0000 7fa1b7e00700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa19c0a1370 0x7fa19c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.683+0000 7fa1d7121b80 1 Processor -- start 2024-05-10T05:38:01.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.684+0000 7fa1d7121b80 1 -- start start 2024-05-10T05:38:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.684+0000 7fa1d7121b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9a6a0 0x55d605f9aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.684+0000 7fa1d7121b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9b570 0x55d605f9b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.684+0000 7fa1d7121b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9bec0 0x55d605fa27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.684+0000 7fa1d7121b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d605ec1150 con 0x55d605f9bec0 2024-05-10T05:38:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.684+0000 7fa1d7121b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d605ebeb10 con 0x55d605f9a6a0 2024-05-10T05:38:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.684+0000 7fa1d7121b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d605eba400 con 0x55d605f9b570 2024-05-10T05:38:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.685+0000 7fa1b7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9b570 0x55d605f9b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.685+0000 7fa1b7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9b570 0x55d605f9b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55188/0 (socket says 172.21.15.131:55188) 2024-05-10T05:38:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.685+0000 7fa1b7400700 1 -- 172.21.15.131:0/2962034693 learned_addr learned my addr 172.21.15.131:0/2962034693 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.685+0000 7fa1b7e00700 1 --2- 172.21.15.131:0/2962034693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9a6a0 0x55d605f9aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.685+0000 7fa1b7400700 1 -- 172.21.15.131:0/2962034693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9a6a0 msgr2=0x55d605f9aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.685+0000 7fa1b7400700 1 --2- 172.21.15.131:0/2962034693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9a6a0 0x55d605f9aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.685+0000 7fa1b7400700 1 -- 172.21.15.131:0/2962034693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9bec0 msgr2=0x55d605fa27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.685+0000 7fa1b7400700 1 --2- 172.21.15.131:0/2962034693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9bec0 0x55d605fa27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.685+0000 7fa1b7400700 1 -- 172.21.15.131:0/2962034693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d605d3b3b0 con 0x55d605f9b570 2024-05-10T05:38:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.685+0000 7fa1b7e00700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa19c0a1370 0x7fa19c0a3880 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8108320 tx=0x7fa1b000d040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.688+0000 7fa1b7400700 1 --2- 172.21.15.131:0/2962034693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9b570 0x55d605f9b980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009840 tx=0x7fa1a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebf2132913cc0e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.688+0000 7fa1ad600700 1 -- 172.21.15.131:0/2962034693 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4011830 con 0x55d605f9b570 2024-05-10T05:38:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.688+0000 7fa1ad600700 1 -- 172.21.15.131:0/2962034693 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a4011e50 con 0x55d605f9b570 2024-05-10T05:38:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.688+0000 7fa1ad600700 1 -- 172.21.15.131:0/2962034693 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4010380 con 0x55d605f9b570 2024-05-10T05:38:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.692+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2962034693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9b570 msgr2=0x55d605f9b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.692+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/2962034693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9b570 0x55d605f9b980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009840 tx=0x7fa1a4009b50 comp rx=0 tx=0).stop 2024-05-10T05:38:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.693+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2962034693 shutdown_connections 2024-05-10T05:38:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.693+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/2962034693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9bec0 0x55d605fa27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.693+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/2962034693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9a6a0 0x55d605f9aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.693+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/2962034693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9b570 0x55d605f9b980 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.693+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2962034693 >> 172.21.15.131:0/2962034693 conn(0x55d605efca60 msgr2=0x55d605f94fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.694+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2962034693 shutdown_connections 2024-05-10T05:38:01.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.696+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2962034693 wait complete. 2024-05-10T05:38:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.697+0000 7fa1d7121b80 1 -- start start 2024-05-10T05:38:01.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.698+0000 7fa1d7121b80 1 Processor -- start 2024-05-10T05:38:01.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.699+0000 7fa1d7121b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9a6a0 0x55d605fa4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.699+0000 7fa1d7121b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9b570 0x55d605fa4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.699+0000 7fa1d7121b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 0x55d605fabbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.701+0000 7fa1b7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9b570 0x55d605fa4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.701+0000 7fa1b7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9b570 0x55d605fa4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48256/0 (socket says 172.21.15.131:48256) 2024-05-10T05:38:01.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.702+0000 7fa1b7400700 1 -- 172.21.15.131:0/2101649525 learned_addr learned my addr 172.21.15.131:0/2101649525 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:01.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.702+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d605e56560 con 0x55d605f9b570 2024-05-10T05:38:01.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.702+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d605e56200 con 0x55d605f9bec0 2024-05-10T05:38:01.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.702+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d605d3d6a0 con 0x55d605f9a6a0 2024-05-10T05:38:01.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.702+0000 7fa1d7121b80 10 client.0 _ll_register_callbacks cb 0x55d605e5fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:01.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.703+0000 7fa1b7e00700 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9a6a0 0x55d605fa4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.703+0000 7fa1bd200700 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 0x55d605fabbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.703+0000 7fa1b7400700 1 -- 172.21.15.131:0/2101649525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9a6a0 msgr2=0x55d605fa4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.703+0000 7fa1b7400700 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9a6a0 0x55d605fa4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.703+0000 7fa1b7400700 1 -- 172.21.15.131:0/2101649525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 msgr2=0x55d605fabbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.703+0000 7fa1b7400700 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 0x55d605fabbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.703+0000 7fa1b7400700 1 -- 172.21.15.131:0/2101649525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a4009470 con 0x55d605f9b570 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.704+0000 7fa1b7400700 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9b570 0x55d605fa4c50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009810 tx=0x7fa1a400b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2603f74404adeff3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.705+0000 7fa1a2000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.705+0000 7fa1a2000700 1 -- 172.21.15.131:0/2101649525 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4011a40 con 0x55d605f9b570 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.705+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d605e13270 con 0x55d605f9b570 2024-05-10T05:38:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.705+0000 7fa1a2000700 1 -- 172.21.15.131:0/2101649525 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa1a4011ba0 con 0x55d605f9b570 2024-05-10T05:38:01.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.706+0000 7fa1a2000700 1 -- 172.21.15.131:0/2101649525 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1a401a420 con 0x55d605f9b570 2024-05-10T05:38:01.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.706+0000 7fa1d7121b80 10 client.16122 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:01.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.706+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d605ce7b60 con 0x55d605f9b570 2024-05-10T05:38:01.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.707+0000 7fa1a2000700 1 -- 172.21.15.131:0/2101649525 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d605e13270 con 0x55d605f9b570 2024-05-10T05:38:01.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.709+0000 7fa1a2000700 1 -- 172.21.15.131:0/2101649525 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa1a4022030 con 0x55d605f9b570 2024-05-10T05:38:01.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.710+0000 7fa1d7121b80 10 client.16122 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:01.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.710+0000 7fa1d7121b80 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:01.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.710+0000 7fa1d7121b80 20 client.16122 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:01.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.710+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d605fb02b0 0x55d605fb2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.710+0000 7fa1d7121b80 4 client.16122 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:01.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.710+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d605f69780 con 0x55d605fb02b0 2024-05-10T05:38:01.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.710+0000 7fa1b7e00700 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d605fb02b0 0x55d605fb2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.723+0000 7fa1b7e00700 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d605fb02b0 0x55d605fb2700 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5933d2c64dd7e2fa server_cookie=a999ce42bb0461e1 in_seq=0 out_seq=0 2024-05-10T05:38:01.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.723+0000 7fa1a2000700 10 client.16122 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:01.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.725+0000 7fa1a2000700 1 -- 172.21.15.131:0/2101649525 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d605f69780 con 0x55d605fb02b0 2024-05-10T05:38:01.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.725+0000 7fa1a2000700 10 client.16122 handle_command_reply: tid=0 2024-05-10T05:38:01.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.725+0000 7fa1d7121b80 1 client.16122 shutdown 2024-05-10T05:38:01.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.727+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9b570 msgr2=0x55d605fa4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:01.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.727+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9b570 0x55d605fa4c50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009810 tx=0x7fa1a400b490 comp rx=0 tx=0).stop 2024-05-10T05:38:01.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.727+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 shutdown_connections 2024-05-10T05:38:01.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.727+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d605fb02b0 0x55d605fb2700 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.727+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 0x55d605fabbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.727+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605f9a6a0 0x55d605fa4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.727+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/2101649525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9b570 0x55d605fa4c50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.727+0000 7fa1a2000700 0 client.16122 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:01.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.727+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 >> 172.21.15.131:0/2101649525 conn(0x55d605efca60 msgr2=0x55d605f94fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:01.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.728+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 shutdown_connections 2024-05-10T05:38:01.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.728+0000 7fa1d7121b80 1 -- 172.21.15.131:0/2101649525 wait complete. 2024-05-10T05:38:01.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.729+0000 7fa1d7121b80 20 client.16122 trim_cache size 0 max 16384 2024-05-10T05:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.765+0000 7fa1d7121b80 1 Processor -- start 2024-05-10T05:38:01.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.766+0000 7fa1d7121b80 1 -- start start 2024-05-10T05:38:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.767+0000 7fa1d7121b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9a6a0 0x55d605f95d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.767+0000 7fa1d7121b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9bec0 0x55d605f9e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.767+0000 7fa1d7121b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d605fa9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.767+0000 7fa1d7121b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d605fac610 con 0x55d605f9bec0 2024-05-10T05:38:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.767+0000 7fa1d7121b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d605f96770 con 0x55d605f9a6a0 2024-05-10T05:38:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.767+0000 7fa1d7121b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d605f8f7c0 con 0x55d605fa9b60 2024-05-10T05:38:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.767+0000 7fa1bd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d605fa9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.767+0000 7fa1bd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d605fa9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55202/0 (socket says 172.21.15.131:55202) 2024-05-10T05:38:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.767+0000 7fa1bd200700 1 -- 172.21.15.131:0/406789848 learned_addr learned my addr 172.21.15.131:0/406789848 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:01.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.768+0000 7fa1bd200700 1 -- 172.21.15.131:0/406789848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9a6a0 msgr2=0x55d605f95d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:38:01.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.768+0000 7fa1bd200700 1 --2- 172.21.15.131:0/406789848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9a6a0 0x55d605f95d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.768+0000 7fa1bd200700 1 -- 172.21.15.131:0/406789848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9bec0 msgr2=0x55d605f9e7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:38:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.769+0000 7fa1b7400700 1 --2- 172.21.15.131:0/406789848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9bec0 0x55d605f9e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.769+0000 7fa1bd200700 1 --2- 172.21.15.131:0/406789848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9bec0 0x55d605f9e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.770+0000 7fa1bd200700 1 -- 172.21.15.131:0/406789848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d605e80560 con 0x55d605fa9b60 2024-05-10T05:38:01.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.770+0000 7fa1b7400700 1 --2- 172.21.15.131:0/406789848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9bec0 0x55d605f9e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:01.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.771+0000 7fa1bd200700 1 --2- 172.21.15.131:0/406789848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d605fa9f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8097ac0 tx=0x7fa1a8005790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4daef102fa20782 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:01.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.771+0000 7fa1ae000700 1 -- 172.21.15.131:0/406789848 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8020e30 con 0x55d605fa9b60 2024-05-10T05:38:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.771+0000 7fa1ae000700 1 -- 172.21.15.131:0/406789848 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a80aee90 con 0x55d605fa9b60 2024-05-10T05:38:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.771+0000 7fa1ae000700 1 -- 172.21.15.131:0/406789848 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80a10c0 con 0x55d605fa9b60 2024-05-10T05:38:01.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.773+0000 7fa1d7121b80 1 -- 172.21.15.131:0/406789848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 msgr2=0x55d605fa9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:01.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.773+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/406789848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d605fa9f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8097ac0 tx=0x7fa1a8005790 comp rx=0 tx=0).stop 2024-05-10T05:38:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.773+0000 7fa1d7121b80 1 -- 172.21.15.131:0/406789848 shutdown_connections 2024-05-10T05:38:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.773+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/406789848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d605fa9f70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.773+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/406789848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9a6a0 0x55d605f95d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.773+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/406789848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9bec0 0x55d605f9e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.773+0000 7fa1d7121b80 1 -- 172.21.15.131:0/406789848 >> 172.21.15.131:0/406789848 conn(0x55d605efca60 msgr2=0x55d605f94f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:01.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.775+0000 7fa1d7121b80 1 -- 172.21.15.131:0/406789848 shutdown_connections 2024-05-10T05:38:01.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.776+0000 7fa1d7121b80 1 -- 172.21.15.131:0/406789848 wait complete. 2024-05-10T05:38:01.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.778+0000 7fa1d7121b80 1 -- start start 2024-05-10T05:38:01.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.779+0000 7fa1d7121b80 1 Processor -- start 2024-05-10T05:38:01.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.786+0000 7fa1d7121b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9a6a0 0x55d6060006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.787+0000 7fa1b7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9a6a0 0x55d6060006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.787+0000 7fa1b7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9a6a0 0x55d6060006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48272/0 (socket says 172.21.15.131:48272) 2024-05-10T05:38:01.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.787+0000 7fa1b7e00700 1 -- 172.21.15.131:0/876395103 learned_addr learned my addr 172.21.15.131:0/876395103 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:01.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.787+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 0x55d606002c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.788+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d606005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.788+0000 7fa1b7400700 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 0x55d606002c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.788+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d605fa99c0 con 0x55d605f9a6a0 2024-05-10T05:38:01.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.788+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6060056d0 con 0x55d605f9bec0 2024-05-10T05:38:01.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.788+0000 7fa1bd200700 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d606005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.789+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d606005860 con 0x55d605fa9b60 2024-05-10T05:38:01.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.792+0000 7fa1bd200700 1 -- 172.21.15.131:0/876395103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 msgr2=0x55d606002c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:01.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.792+0000 7fa1bd200700 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 0x55d606002c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.792+0000 7fa1bd200700 1 -- 172.21.15.131:0/876395103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9a6a0 msgr2=0x55d6060006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:01.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.792+0000 7fa1bd200700 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9a6a0 0x55d6060006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.792+0000 7fa1bd200700 1 -- 172.21.15.131:0/876395103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b81b7940 con 0x55d605fa9b60 2024-05-10T05:38:01.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.793+0000 7fa1d7121b80 10 client.0 _ll_register_callbacks cb 0x55d605e5fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:01.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.793+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1a4011e50 con 0x55d605fa9b60 2024-05-10T05:38:01.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.793+0000 7fa1b7400700 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 0x55d606002c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:38:01.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.794+0000 7fa1bd200700 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d606005190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80cb6c0 tx=0x7fa1a809b0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1403d7f7c0dbe8b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:01.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.794+0000 7fa1a2000700 10 client.26251 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:38:01.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.796+0000 7fa1a2000700 1 -- 172.21.15.131:0/876395103 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8020e30 con 0x55d605fa9b60 2024-05-10T05:38:01.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.796+0000 7fa1a2000700 1 -- 172.21.15.131:0/876395103 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa1a809b750 con 0x55d605fa9b60 2024-05-10T05:38:01.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.796+0000 7fa1a2000700 1 -- 172.21.15.131:0/876395103 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1a80a1050 con 0x55d605fa9b60 2024-05-10T05:38:01.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.797+0000 7fa1d7121b80 10 client.26251 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:01.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.797+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55d605d3fcd0 con 0x55d605fa9b60 2024-05-10T05:38:01.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.801+0000 7fa1a2000700 1 -- 172.21.15.131:0/876395103 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa1a809a540 con 0x55d605fa9b60 2024-05-10T05:38:01.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.801+0000 7fa1d7121b80 10 client.26251 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:01.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.801+0000 7fa1d7121b80 10 client.26251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:01.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.801+0000 7fa1d7121b80 20 client.26251 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:01.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.801+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d606006b10 0x55d606008f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:01.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.801+0000 7fa1d7121b80 4 client.26251 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:38:01.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.801+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d605eb2ec0 con 0x55d606006b10 2024-05-10T05:38:01.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.802+0000 7fa1b7e00700 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d606006b10 0x55d606008f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:01.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.803+0000 7fa1a2000700 10 client.26251 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:01.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.803+0000 7fa1b7e00700 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d606006b10 0x55d606008f30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78bd1eb82d650eed server_cookie=fe83715bd504afcc in_seq=0 out_seq=0 2024-05-10T05:38:01.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.804+0000 7fa1a2000700 1 -- 172.21.15.131:0/876395103 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d605eb2ec0 con 0x55d606006b10 2024-05-10T05:38:01.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.804+0000 7fa1a2000700 10 client.26251 handle_command_reply: tid=0 2024-05-10T05:38:01.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.804+0000 7fa1d7121b80 1 client.26251 shutdown 2024-05-10T05:38:01.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.805+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 msgr2=0x55d606005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:01.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.805+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d606005190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80cb6c0 tx=0x7fa1a809b0d0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.806+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 shutdown_connections 2024-05-10T05:38:01.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.806+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d606006b10 0x55d606008f30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.806+0000 7fa1a2000700 0 client.26251 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:01.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.806+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d605f9a6a0 0x55d6060006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.806+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d605fa9b60 0x55d606005190 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.806+0000 7fa1d7121b80 1 --2- 172.21.15.131:0/876395103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d605f9bec0 0x55d606002c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.806+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 >> 172.21.15.131:0/876395103 conn(0x55d605efca60 msgr2=0x55d605f94f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:01.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.809+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 shutdown_connections 2024-05-10T05:38:01.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.809+0000 7fa1d7121b80 1 -- 172.21.15.131:0/876395103 wait complete. 2024-05-10T05:38:01.811 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:38:01.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.810+0000 7fa1d7121b80 20 client.26251 trim_cache size 0 max 16384 2024-05-10T05:38:01.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.811+0000 7fa1bf200700 1 -- 172.21.15.131:0/4214909106 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa19c0a1370 msgr2=0x7fa19c0a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:01.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.811+0000 7fa1bf200700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa19c0a1370 0x7fa19c0a3880 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8108320 tx=0x7fa1b000d040 comp rx=0 tx=0).stop 2024-05-10T05:38:01.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.812+0000 7fa1bf200700 1 -- 172.21.15.131:0/4214909106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1b8109c50 msgr2=0x7fa1b81b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.812+0000 7fa1bf200700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1b8109c50 0x7fa1b81b6c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8003ff0 tx=0x7fa1a8004020 comp rx=0 tx=0).stop 2024-05-10T05:38:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.812+0000 7fa1bf200700 1 -- 172.21.15.131:0/4214909106 shutdown_connections 2024-05-10T05:38:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.812+0000 7fa1bf200700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa19c0a1370 0x7fa19c0a3880 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.812+0000 7fa1bf200700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1b8109c50 0x7fa1b81b6c10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.812+0000 7fa1bf200700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b8108630 0x7fa1b8115340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.812+0000 7fa1bf200700 1 --2- 172.21.15.131:0/4214909106 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1b8109300 0x7fa1b8115880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:01.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.812+0000 7fa1bf200700 1 -- 172.21.15.131:0/4214909106 >> 172.21.15.131:0/4214909106 conn(0x7fa1b806ea00 msgr2=0x7fa1b810a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:01.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.813+0000 7fa1bf200700 1 -- 172.21.15.131:0/4214909106 shutdown_connections 2024-05-10T05:38:01.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:01.813+0000 7fa1bf200700 1 -- 172.21.15.131:0/4214909106 wait complete. 2024-05-10T05:38:02.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:02 smithi131 ceph-mon[37264]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 251 MiB/s wr, 76 op/s 2024-05-10T05:38:02.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:02 smithi157 ceph-mon[78427]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 251 MiB/s wr, 76 op/s 2024-05-10T05:38:02.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:02 smithi142 ceph-mon[77667]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 251 MiB/s wr, 76 op/s 2024-05-10T05:38:02.907 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:38:02.908 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:38:03.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.718+0000 7f53c1a00700 1 -- 172.21.15.131:0/2768019643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53bc108630 msgr2=0x7f53bc108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:03.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.718+0000 7f53c1a00700 1 --2- 172.21.15.131:0/2768019643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53bc108630 0x7f53bc108a40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009990 tx=0x7f53ac009ca0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.719+0000 7f53c1a00700 1 -- 172.21.15.131:0/2768019643 shutdown_connections 2024-05-10T05:38:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.719+0000 7f53c1a00700 1 --2- 172.21.15.131:0/2768019643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53bc109c50 0x7f53bc10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.719+0000 7f53c1a00700 1 --2- 172.21.15.131:0/2768019643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53bc108630 0x7f53bc108a40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.719+0000 7f53c1a00700 1 --2- 172.21.15.131:0/2768019643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53bc109300 0x7f53bc109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.719+0000 7f53c1a00700 1 -- 172.21.15.131:0/2768019643 >> 172.21.15.131:0/2768019643 conn(0x7f53bc06ea00 msgr2=0x7f53bc070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:03.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.721+0000 7f53c1a00700 1 -- 172.21.15.131:0/2768019643 shutdown_connections 2024-05-10T05:38:03.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.721+0000 7f53c1a00700 1 -- 172.21.15.131:0/2768019643 wait complete. 2024-05-10T05:38:03.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.725+0000 7f53c1a00700 1 Processor -- start 2024-05-10T05:38:03.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.725+0000 7f53c1a00700 1 -- start start 2024-05-10T05:38:03.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.727+0000 7f53c1a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53bc108630 0x7f53bc1a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.728+0000 7f53c1a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53bc109c50 0x7f53bc1a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.728+0000 7f53c1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53bc1ac450 0x7f53bc1ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.729+0000 7f53c1a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc1aee80 con 0x7f53bc109c50 2024-05-10T05:38:03.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.729+0000 7f53c1a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc1aeff0 con 0x7f53bc1ac450 2024-05-10T05:38:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.730+0000 7f53c1a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc1af1b0 con 0x7f53bc108630 2024-05-10T05:38:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.729+0000 7f53baa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53bc108630 0x7f53bc1a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.730+0000 7f53baa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53bc108630 0x7f53bc1a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55894/0 (socket says 172.21.15.131:55894) 2024-05-10T05:38:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.730+0000 7f53baa00700 1 -- 172.21.15.131:0/2413625559 learned_addr learned my addr 172.21.15.131:0/2413625559 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.730+0000 7f53bb400700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53bc1ac450 0x7f53bc1ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.731+0000 7f53baa00700 1 -- 172.21.15.131:0/2413625559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53bc1ac450 msgr2=0x7f53bc1ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.731+0000 7f53baa00700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53bc1ac450 0x7f53bc1ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.731+0000 7f53baa00700 1 -- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53bc109c50 msgr2=0x7f53bc1a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:03.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.731+0000 7f53ba000700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53bc109c50 0x7f53bc1a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.731+0000 7f53baa00700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53bc109c50 0x7f53bc1a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.731+0000 7f53baa00700 1 -- 172.21.15.131:0/2413625559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac0095f0 con 0x7f53bc108630 2024-05-10T05:38:03.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.731+0000 7f53ba000700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53bc109c50 0x7f53bc1a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:03.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.732+0000 7f53baa00700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53bc108630 0x7f53bc1a6f90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f53ac000c00 tx=0x7f53ac003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:03.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.736+0000 7f53b3400700 1 -- 172.21.15.131:0/2413625559 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac01d070 con 0x7f53bc108630 2024-05-10T05:38:03.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.736+0000 7f53b3400700 1 -- 172.21.15.131:0/2413625559 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53ac00f460 con 0x7f53bc108630 2024-05-10T05:38:03.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.736+0000 7f53b3400700 1 -- 172.21.15.131:0/2413625559 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac017650 con 0x7f53bc108630 2024-05-10T05:38:03.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.736+0000 7f53c1a00700 1 -- 172.21.15.131:0/2413625559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc1af430 con 0x7f53bc108630 2024-05-10T05:38:03.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.736+0000 7f53c1a00700 1 -- 172.21.15.131:0/2413625559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc1af820 con 0x7f53bc108630 2024-05-10T05:38:03.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.749+0000 7f53b3400700 1 -- 172.21.15.131:0/2413625559 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f53ac0177b0 con 0x7f53bc108630 2024-05-10T05:38:03.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.758+0000 7f53b3400700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53a00a1400 0x7f53a00a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.758+0000 7f53b3400700 1 -- 172.21.15.131:0/2413625559 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f53ac0cd9f0 con 0x7f53bc108630 2024-05-10T05:38:03.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.764+0000 7f53d98ebb80 1 Processor -- start 2024-05-10T05:38:03.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.764+0000 7f53d98ebb80 1 -- start start 2024-05-10T05:38:03.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.767+0000 7f53ba000700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53a00a1400 0x7f53a00a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.767+0000 7f53d98ebb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 0x56415565fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.768+0000 7f53d98ebb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x564155660a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.769+0000 7f53ba000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x564155660a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.769+0000 7f53ba000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x564155660a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55904/0 (socket says 172.21.15.131:55904) 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.769+0000 7f53d98ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x564155667840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.770+0000 7f53ba000700 1 -- 172.21.15.131:0/588438867 learned_addr learned my addr 172.21.15.131:0/588438867 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.770+0000 7f53bb400700 1 --2- 172.21.15.131:0/588438867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x564155667840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.770+0000 7f53d98ebb80 1 -- 172.21.15.131:0/588438867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641555861d0 con 0x56415565f720 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.770+0000 7f53d98ebb80 1 -- 172.21.15.131:0/588438867 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564155583b90 con 0x564155660f40 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.770+0000 7f53d98ebb80 1 -- 172.21.15.131:0/588438867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56415557f480 con 0x5641556605f0 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.770+0000 7f53baa00700 1 --2- 172.21.15.131:0/588438867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 0x56415565fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.771+0000 7f53ba000700 1 -- 172.21.15.131:0/588438867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 msgr2=0x564155667840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.771+0000 7f53ba000700 1 --2- 172.21.15.131:0/588438867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x564155667840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.771+0000 7f53ba000700 1 -- 172.21.15.131:0/588438867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 msgr2=0x56415565fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.771+0000 7f53ba000700 1 --2- 172.21.15.131:0/588438867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 0x56415565fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.771+0000 7f53ba000700 1 -- 172.21.15.131:0/588438867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564155400430 con 0x5641556605f0 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.772+0000 7f53bb400700 1 --2- 172.21.15.131:0/588438867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x564155667840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.772+0000 7f53ba000700 1 --2- 172.21.15.131:0/588438867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x564155660a00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f53a8013740 tx=0x7f53a8013a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=432ba1c1be6411ee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:03.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.773+0000 7f53a7e00700 1 -- 172.21.15.131:0/588438867 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a801d030 con 0x5641556605f0 2024-05-10T05:38:03.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.773+0000 7f53a7e00700 1 -- 172.21.15.131:0/588438867 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53a800c050 con 0x5641556605f0 2024-05-10T05:38:03.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.773+0000 7f53a7e00700 1 -- 172.21.15.131:0/588438867 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8018d00 con 0x5641556605f0 2024-05-10T05:38:03.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.774+0000 7f53d98ebb80 1 -- 172.21.15.131:0/588438867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 msgr2=0x564155660a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:03.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.774+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/588438867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x564155660a00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f53a8013740 tx=0x7f53a8013a50 comp rx=0 tx=0).stop 2024-05-10T05:38:03.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.776+0000 7f53ba000700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53a00a1400 0x7f53a00a3910 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f53a801d000 tx=0x7f53a80176c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:03.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.777+0000 7f53d98ebb80 1 -- 172.21.15.131:0/588438867 shutdown_connections 2024-05-10T05:38:03.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.777+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/588438867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 0x56415565fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.777+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/588438867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x564155667840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.777+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/588438867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x564155660a00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.777+0000 7f53d98ebb80 1 -- 172.21.15.131:0/588438867 >> 172.21.15.131:0/588438867 conn(0x5641555c1ae0 msgr2=0x56415565a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:03.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.777+0000 7f53d98ebb80 1 -- 172.21.15.131:0/588438867 shutdown_connections 2024-05-10T05:38:03.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.777+0000 7f53d98ebb80 1 -- 172.21.15.131:0/588438867 wait complete. 2024-05-10T05:38:03.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.779+0000 7f53d98ebb80 1 -- start start 2024-05-10T05:38:03.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.780+0000 7f53d98ebb80 1 Processor -- start 2024-05-10T05:38:03.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.782+0000 7f53d98ebb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56415565f720 0x564155669580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.783+0000 7f53baa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56415565f720 0x564155669580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.783+0000 7f53baa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56415565f720 0x564155669580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55918/0 (socket says 172.21.15.131:55918) 2024-05-10T05:38:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.783+0000 7f53baa00700 1 -- 172.21.15.131:0/4157549782 learned_addr learned my addr 172.21.15.131:0/4157549782 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.783+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 0x564155669ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.784+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x564155670a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.784+0000 7f53bb400700 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x564155670a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.789+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56415551b5e0 con 0x5641556605f0 2024-05-10T05:38:03.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.789+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56415551b280 con 0x564155660f40 2024-05-10T05:38:03.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.789+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564155402720 con 0x56415565f720 2024-05-10T05:38:03.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.789+0000 7f53d98ebb80 10 client.0 _ll_register_callbacks cb 0x564155524bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:03.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.790+0000 7f53ba000700 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 0x564155669ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.790+0000 7f53baa00700 1 -- 172.21.15.131:0/4157549782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 msgr2=0x564155670a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.791+0000 7f53baa00700 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x564155670a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.791+0000 7f53baa00700 1 -- 172.21.15.131:0/4157549782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 msgr2=0x564155669ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.794+0000 7f53baa00700 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 0x564155669ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.794+0000 7f53baa00700 1 -- 172.21.15.131:0/4157549782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a8013370 con 0x56415565f720 2024-05-10T05:38:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.794+0000 7f53bb400700 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x564155670a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:38:03.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.795+0000 7f53ba000700 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 0x564155669ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:38:03.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.798+0000 7f53baa00700 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56415565f720 0x564155669580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0cb530 tx=0x7f53ac09a870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a78bae56ea064c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:03.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.798+0000 7f53a4c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:38:03.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.800+0000 7f53a4c00700 1 -- 172.21.15.131:0/4157549782 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac09af10 con 0x56415565f720 2024-05-10T05:38:03.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.800+0000 7f53a4c00700 1 -- 172.21.15.131:0/4157549782 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f53ac09ed40 con 0x56415565f720 2024-05-10T05:38:03.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.800+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641554d82f0 con 0x56415565f720 2024-05-10T05:38:03.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.800+0000 7f53a4c00700 1 -- 172.21.15.131:0/4157549782 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53ac0a3300 con 0x56415565f720 2024-05-10T05:38:03.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.802+0000 7f53d98ebb80 10 client.26263 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:03.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.802+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5641553acbe0 con 0x56415565f720 2024-05-10T05:38:03.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.806+0000 7f53a4c00700 1 -- 172.21.15.131:0/4157549782 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53ac0a7080 con 0x56415565f720 2024-05-10T05:38:03.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.806+0000 7f53a4c00700 1 -- 172.21.15.131:0/4157549782 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f53ac09a4e0 con 0x56415565f720 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.806+0000 7f53d98ebb80 10 client.26263 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.806+0000 7f53d98ebb80 10 client.26263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.806+0000 7f53d98ebb80 20 client.26263 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.806+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564155674eb0 0x564155677300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.806+0000 7f53d98ebb80 4 client.26263 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.806+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56415562e800 con 0x564155674eb0 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.807+0000 7f53ba000700 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564155674eb0 0x564155677300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.808+0000 7f53ba000700 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564155674eb0 0x564155677300 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662d151d1ad7a26f server_cookie=aa9aea4716acc9b6 in_seq=0 out_seq=0 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.809+0000 7f53a4c00700 10 client.26263 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.811+0000 7f53a4c00700 1 -- 172.21.15.131:0/4157549782 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56415562e800 con 0x564155674eb0 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.811+0000 7f53a4c00700 10 client.26263 handle_command_reply: tid=0 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.811+0000 7f53d98ebb80 1 client.26263 shutdown 2024-05-10T05:38:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.816+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56415565f720 msgr2=0x564155669580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.816+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56415565f720 0x564155669580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0cb530 tx=0x7f53ac09a870 comp rx=0 tx=0).stop 2024-05-10T05:38:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.817+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 shutdown_connections 2024-05-10T05:38:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.817+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564155674eb0 0x564155677300 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.817+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x564155670a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.817+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56415565f720 0x564155669580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.817+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/4157549782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 0x564155669ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.817+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 >> 172.21.15.131:0/4157549782 conn(0x5641555c1ae0 msgr2=0x56415565a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.817+0000 7f53a4c00700 0 client.26263 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:03.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.818+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 shutdown_connections 2024-05-10T05:38:03.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.818+0000 7f53d98ebb80 1 -- 172.21.15.131:0/4157549782 wait complete. 2024-05-10T05:38:03.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.819+0000 7f53d98ebb80 20 client.26263 trim_cache size 0 max 16384 2024-05-10T05:38:03.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.915+0000 7f53d98ebb80 1 Processor -- start 2024-05-10T05:38:03.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.918+0000 7f53d98ebb80 1 -- start start 2024-05-10T05:38:03.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.918+0000 7f53d98ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56415565f720 0x564155669860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.918+0000 7f53d98ebb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 0x56415566e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.919+0000 7f53baa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56415565f720 0x564155669860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.919+0000 7f53d98ebb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564155660f40 0x56415566ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.919+0000 7f53baa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56415565f720 0x564155669860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:59220/0 (socket says 172.21.15.131:59220) 2024-05-10T05:38:03.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.920+0000 7f53bb400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564155660f40 0x56415566ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.920+0000 7f53bb400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564155660f40 0x56415566ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55924/0 (socket says 172.21.15.131:55924) 2024-05-10T05:38:03.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.923+0000 7f53bb400700 1 -- 172.21.15.131:0/2798566744 learned_addr learned my addr 172.21.15.131:0/2798566744 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:03.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.923+0000 7f53d98ebb80 1 -- 172.21.15.131:0/2798566744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564155656db0 con 0x5641556605f0 2024-05-10T05:38:03.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.923+0000 7f53d98ebb80 1 -- 172.21.15.131:0/2798566744 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564155671480 con 0x56415565f720 2024-05-10T05:38:03.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.923+0000 7f53d98ebb80 1 -- 172.21.15.131:0/2798566744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564155671b20 con 0x564155660f40 2024-05-10T05:38:03.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.923+0000 7f53ba000700 1 --2- 172.21.15.131:0/2798566744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 0x56415566e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.924+0000 7f53bb400700 1 -- 172.21.15.131:0/2798566744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56415565f720 msgr2=0x564155669860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.925+0000 7f53bb400700 1 --2- 172.21.15.131:0/2798566744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56415565f720 0x564155669860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.925+0000 7f53bb400700 1 -- 172.21.15.131:0/2798566744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 msgr2=0x56415566e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.925+0000 7f53bb400700 1 --2- 172.21.15.131:0/2798566744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 0x56415566e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.925+0000 7f53bb400700 1 -- 172.21.15.131:0/2798566744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56415565a300 con 0x564155660f40 2024-05-10T05:38:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.925+0000 7f53ba000700 1 --2- 172.21.15.131:0/2798566744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 0x56415566e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:38:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.925+0000 7f53baa00700 1 --2- 172.21.15.131:0/2798566744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56415565f720 0x564155669860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:38:03.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.927+0000 7f53bb400700 1 --2- 172.21.15.131:0/2798566744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564155660f40 0x56415566ef10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53b400a750 tx=0x7f53b4009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbe0b24e6a3517cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:03.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.928+0000 7f53b0c00700 1 -- 172.21.15.131:0/2798566744 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4017470 con 0x564155660f40 2024-05-10T05:38:03.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.928+0000 7f53b0c00700 1 -- 172.21.15.131:0/2798566744 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53b4011050 con 0x564155660f40 2024-05-10T05:38:03.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.928+0000 7f53b0c00700 1 -- 172.21.15.131:0/2798566744 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b40169c0 con 0x564155660f40 2024-05-10T05:38:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.930+0000 7f53d98ebb80 1 -- 172.21.15.131:0/2798566744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564155660f40 msgr2=0x56415566ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.930+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/2798566744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564155660f40 0x56415566ef10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53b400a750 tx=0x7f53b4009650 comp rx=0 tx=0).stop 2024-05-10T05:38:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.931+0000 7f53d98ebb80 1 -- 172.21.15.131:0/2798566744 shutdown_connections 2024-05-10T05:38:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.931+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/2798566744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564155660f40 0x56415566ef10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.931+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/2798566744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56415565f720 0x564155669860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.931+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/2798566744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641556605f0 0x56415566e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.931+0000 7f53d98ebb80 1 -- 172.21.15.131:0/2798566744 >> 172.21.15.131:0/2798566744 conn(0x5641555c1ae0 msgr2=0x564155669150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.931+0000 7f53d98ebb80 1 -- 172.21.15.131:0/2798566744 shutdown_connections 2024-05-10T05:38:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.931+0000 7f53d98ebb80 1 -- 172.21.15.131:0/2798566744 wait complete. 2024-05-10T05:38:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.932+0000 7f53d98ebb80 1 -- start start 2024-05-10T05:38:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.933+0000 7f53d98ebb80 1 Processor -- start 2024-05-10T05:38:03.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.934+0000 7f53d98ebb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 0x5641556c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.934+0000 7f53d98ebb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x5641556c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.934+0000 7f53d98ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x5641556cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.935+0000 7f53bb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x5641556cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.935+0000 7f53bb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x5641556cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:59228/0 (socket says 172.21.15.131:59228) 2024-05-10T05:38:03.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.935+0000 7f53ba000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x5641556c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.935+0000 7f53ba000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x5641556c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55940/0 (socket says 172.21.15.131:55940) 2024-05-10T05:38:03.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.935+0000 7f53baa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 0x5641556c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.935+0000 7f53baa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 0x5641556c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34796/0 (socket says 172.21.15.131:34796) 2024-05-10T05:38:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.935+0000 7f53d98ebb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641556cba80 con 0x56415565f720 2024-05-10T05:38:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.936+0000 7f53bb400700 1 -- 172.21.15.131:0/3711650291 learned_addr learned my addr 172.21.15.131:0/3711650291 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.936+0000 7f53d98ebb80 1 -- 172.21.15.131:0/3711650291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641556cbbc0 con 0x564155660f40 2024-05-10T05:38:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.936+0000 7f53d98ebb80 1 -- 172.21.15.131:0/3711650291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641556cbd50 con 0x5641556605f0 2024-05-10T05:38:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.936+0000 7f53d98ebb80 10 client.0 _ll_register_callbacks cb 0x564155524bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.936+0000 7f53bb400700 1 -- 172.21.15.131:0/3711650291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 msgr2=0x5641556c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.936+0000 7f53bb400700 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x5641556c45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.936+0000 7f53bb400700 1 -- 172.21.15.131:0/3711650291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 msgr2=0x5641556c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.936+0000 7f53bb400700 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 0x5641556c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.937+0000 7f53bb400700 1 -- 172.21.15.131:0/3711650291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641553acbe0 con 0x564155660f40 2024-05-10T05:38:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.937+0000 7f53bb400700 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x5641556cb540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f53b400a010 tx=0x7f53b4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b4f260f455d07c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.937+0000 7f53a4c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:38:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.937+0000 7f53a4c00700 1 -- 172.21.15.131:0/3711650291 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4017880 con 0x564155660f40 2024-05-10T05:38:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.937+0000 7f53a4c00700 1 -- 172.21.15.131:0/3711650291 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f53b4006e40 con 0x564155660f40 2024-05-10T05:38:03.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.940+0000 7f53d98ebb80 1 -- 172.21.15.131:0/3711650291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53a800c050 con 0x564155660f40 2024-05-10T05:38:03.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.940+0000 7f53a4c00700 1 -- 172.21.15.131:0/3711650291 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53b40179e0 con 0x564155660f40 2024-05-10T05:38:03.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.940+0000 7f53d98ebb80 10 client.26237 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:03.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.940+0000 7f53d98ebb80 1 -- 172.21.15.131:0/3711650291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564155400430 con 0x564155660f40 2024-05-10T05:38:03.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.941+0000 7f53a4c00700 1 -- 172.21.15.131:0/3711650291 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53a800c050 con 0x564155660f40 2024-05-10T05:38:03.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.941+0000 7f53a4c00700 1 -- 172.21.15.131:0/3711650291 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f53b4021400 con 0x564155660f40 2024-05-10T05:38:03.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.941+0000 7f53d98ebb80 10 client.26237 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:03.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.941+0000 7f53d98ebb80 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:03.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.941+0000 7f53d98ebb80 20 client.26237 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:03.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.941+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5641556cd700 0x5641556cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:03.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.941+0000 7f53d98ebb80 4 client.26237 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:38:03.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.941+0000 7f53d98ebb80 1 -- 172.21.15.131:0/3711650291 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564155577f40 con 0x5641556cd700 2024-05-10T05:38:03.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.942+0000 7f53baa00700 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5641556cd700 0x5641556cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:03.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.943+0000 7f53a4c00700 10 client.26237 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:03.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.943+0000 7f53baa00700 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5641556cd700 0x5641556cfb20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ddd74dd74d180d0 server_cookie=4f40fd262a54f6e0 in_seq=0 out_seq=0 2024-05-10T05:38:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53a4c00700 1 -- 172.21.15.131:0/3711650291 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564155577f40 con 0x5641556cd700 2024-05-10T05:38:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53a4c00700 10 client.26237 handle_command_reply: tid=0 2024-05-10T05:38:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53d98ebb80 1 client.26237 shutdown 2024-05-10T05:38:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53d98ebb80 1 -- 172.21.15.131:0/3711650291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 msgr2=0x5641556cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x5641556cb540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f53b400a010 tx=0x7f53b4004ad0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53d98ebb80 1 -- 172.21.15.131:0/3711650291 shutdown_connections 2024-05-10T05:38:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5641556cd700 0x5641556cfb20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56415565f720 0x5641556c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564155660f40 0x5641556cb540 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53d98ebb80 1 --2- 172.21.15.131:0/3711650291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641556605f0 0x5641556c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53d98ebb80 1 -- 172.21.15.131:0/3711650291 >> 172.21.15.131:0/3711650291 conn(0x5641555c1ae0 msgr2=0x564155669050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.946+0000 7f53a4c00700 0 client.26237 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.947+0000 7f53d98ebb80 1 -- 172.21.15.131:0/3711650291 shutdown_connections 2024-05-10T05:38:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.947+0000 7f53d98ebb80 1 -- 172.21.15.131:0/3711650291 wait complete. 2024-05-10T05:38:03.949 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:38:03.949 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:38:03.949 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "f33808c6-5684-43da-a9b8-c4486162cf76", 2024-05-10T05:38:03.949 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:38:03.950 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:38:03.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.947+0000 7f53d98ebb80 20 client.26237 trim_cache size 0 max 16384 2024-05-10T05:38:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.951+0000 7f53c1a00700 1 -- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53a00a1400 msgr2=0x7f53a00a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.951+0000 7f53c1a00700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53a00a1400 0x7f53a00a3910 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f53a801d000 tx=0x7f53a80176c0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.951+0000 7f53c1a00700 1 -- 172.21.15.131:0/2413625559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53bc108630 msgr2=0x7f53bc1a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.951+0000 7f53c1a00700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53bc108630 0x7f53bc1a6f90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f53ac000c00 tx=0x7f53ac003c00 comp rx=0 tx=0).stop 2024-05-10T05:38:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.952+0000 7f53c1a00700 1 -- 172.21.15.131:0/2413625559 shutdown_connections 2024-05-10T05:38:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.952+0000 7f53c1a00700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f53a00a1400 0x7f53a00a3910 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.952+0000 7f53c1a00700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53bc1ac450 0x7f53bc1ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.952+0000 7f53c1a00700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53bc108630 0x7f53bc1a6f90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.952+0000 7f53c1a00700 1 --2- 172.21.15.131:0/2413625559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53bc109c50 0x7f53bc1a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:03.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.952+0000 7f53c1a00700 1 -- 172.21.15.131:0/2413625559 >> 172.21.15.131:0/2413625559 conn(0x7f53bc06ea00 msgr2=0x7f53bc06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:03.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.952+0000 7f53c1a00700 1 -- 172.21.15.131:0/2413625559 shutdown_connections 2024-05-10T05:38:03.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:03.952+0000 7f53c1a00700 1 -- 172.21.15.131:0/2413625559 wait complete. 2024-05-10T05:38:04.015 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:38:04.467 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:04 smithi131 ceph-mon[37264]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 243 MiB/s wr, 76 op/s 2024-05-10T05:38:04.467 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:04 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:38:04.496 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:04 smithi157 ceph-mon[78427]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 243 MiB/s wr, 76 op/s 2024-05-10T05:38:04.496 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:04 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:38:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:04 smithi142 ceph-mon[77667]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 243 MiB/s wr, 76 op/s 2024-05-10T05:38:04.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:04 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:38:04.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.804+0000 7f15ed000700 1 -- 172.21.15.131:0/1309361370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e0095520 msgr2=0x7f15e0095930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:04.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.804+0000 7f15ed000700 1 --2- 172.21.15.131:0/1309361370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e0095520 0x7f15e0095930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f15e804efb0 tx=0x7f15e8069cb0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.804+0000 7f15ed000700 1 -- 172.21.15.131:0/1309361370 shutdown_connections 2024-05-10T05:38:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.804+0000 7f15ed000700 1 --2- 172.21.15.131:0/1309361370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e0096fe0 0x7f15e009d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.804+0000 7f15ed000700 1 --2- 172.21.15.131:0/1309361370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e0095520 0x7f15e0095930 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.804+0000 7f15ed000700 1 --2- 172.21.15.131:0/1309361370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e0096640 0x7f15e0096a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.804+0000 7f15ed000700 1 -- 172.21.15.131:0/1309361370 >> 172.21.15.131:0/1309361370 conn(0x7f15e0090b10 msgr2=0x7f15e0092fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.808+0000 7f15ed000700 1 -- 172.21.15.131:0/1309361370 shutdown_connections 2024-05-10T05:38:04.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.808+0000 7f15ed000700 1 -- 172.21.15.131:0/1309361370 wait complete. 2024-05-10T05:38:04.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.818+0000 7f15ed000700 1 Processor -- start 2024-05-10T05:38:04.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.818+0000 7f15ed000700 1 -- start start 2024-05-10T05:38:04.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.819+0000 7f15ed000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e0095520 0x7f15e00a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.819+0000 7f15ed000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e0096640 0x7f15e00a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.819+0000 7f15ed000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e0096fe0 0x7f15e00ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.820+0000 7f15e5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e0095520 0x7f15e00a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:04.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.820+0000 7f15e5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e0095520 0x7f15e00a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:59244/0 (socket says 172.21.15.131:59244) 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.820+0000 7f15e5e00700 1 -- 172.21.15.131:0/2823829720 learned_addr learned my addr 172.21.15.131:0/2823829720 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.820+0000 7f15e7200700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e0096fe0 0x7f15e00ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.820+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e00ad660 con 0x7f15e0096640 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.820+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e00ad7d0 con 0x7f15e0095520 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.820+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e00ad990 con 0x7f15e0096fe0 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.820+0000 7f15e5e00700 1 -- 172.21.15.131:0/2823829720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e0096fe0 msgr2=0x7f15e00ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.820+0000 7f15e5e00700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e0096fe0 0x7f15e00ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.821+0000 7f15e5e00700 1 -- 172.21.15.131:0/2823829720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e0096640 msgr2=0x7f15e00a6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.821+0000 7f15e5e00700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e0096640 0x7f15e00a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.821+0000 7f15e5e00700 1 -- 172.21.15.131:0/2823829720 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e8069700 con 0x7f15e0095520 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.821+0000 7f15e5e00700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e0095520 0x7f15e00a5c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f15e8064500 tx=0x7f15e8069300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.821+0000 7f15dea00700 1 -- 172.21.15.131:0/2823829720 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e807b040 con 0x7f15e0095520 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.821+0000 7f15dea00700 1 -- 172.21.15.131:0/2823829720 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e8074a80 con 0x7f15e0095520 2024-05-10T05:38:04.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.821+0000 7f15dea00700 1 -- 172.21.15.131:0/2823829720 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e807f860 con 0x7f15e0095520 2024-05-10T05:38:04.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.821+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e01439c0 con 0x7f15e0095520 2024-05-10T05:38:04.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.822+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e0143d30 con 0x7f15e0095520 2024-05-10T05:38:04.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.834+0000 7f15dea00700 1 -- 172.21.15.131:0/2823829720 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f15e8071070 con 0x7f15e0095520 2024-05-10T05:38:04.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.836+0000 7f15dea00700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f15cc0a1220 0x7f15cc0a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.837+0000 7f15dea00700 1 -- 172.21.15.131:0/2823829720 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f15e806f070 con 0x7f15e0095520 2024-05-10T05:38:04.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.837+0000 7f15e5400700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f15cc0a1220 0x7f15cc0a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:04.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.838+0000 7f15e5400700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f15cc0a1220 0x7f15cc0a3730 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f15e00a6c60 tx=0x7f15d8006d20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.918+0000 7f1604fffb80 1 Processor -- start 2024-05-10T05:38:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.921+0000 7f1604fffb80 1 -- start start 2024-05-10T05:38:04.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.928+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c34960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.928+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c357a0 0x55e567c35bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.929+0000 7f15e5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c34960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:04.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.930+0000 7f15e5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c34960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55974/0 (socket says 172.21.15.131:55974) 2024-05-10T05:38:04.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.930+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 0x55e567c3c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.930+0000 7f15e5e00700 1 -- 172.21.15.131:0/3773208131 learned_addr learned my addr 172.21.15.131:0/3773208131 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:04.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.930+0000 7f15e7200700 1 --2- 172.21.15.131:0/3773208131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 0x55e567c3c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:04.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.930+0000 7f1604fffb80 1 -- 172.21.15.131:0/3773208131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567b5b150 con 0x55e567c357a0 2024-05-10T05:38:04.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.930+0000 7f1604fffb80 1 -- 172.21.15.131:0/3773208131 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567b58b10 con 0x55e567c360f0 2024-05-10T05:38:04.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.930+0000 7f1604fffb80 1 -- 172.21.15.131:0/3773208131 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567b54400 con 0x55e567c34530 2024-05-10T05:38:04.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.931+0000 7f15e5e00700 1 -- 172.21.15.131:0/3773208131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 msgr2=0x55e567c3c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:04.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.931+0000 7f15e5e00700 1 --2- 172.21.15.131:0/3773208131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 0x55e567c3c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.931+0000 7f15e5e00700 1 -- 172.21.15.131:0/3773208131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c357a0 msgr2=0x55e567c35bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:38:04.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.931+0000 7f15e5e00700 1 --2- 172.21.15.131:0/3773208131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c357a0 0x55e567c35bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.931+0000 7f15e5e00700 1 -- 172.21.15.131:0/3773208131 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5679d53b0 con 0x55e567c34530 2024-05-10T05:38:04.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.932+0000 7f15e5e00700 1 --2- 172.21.15.131:0/3773208131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c34960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f15e8050560 tx=0x7f15e8113120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6fbc678fb7c90a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:04.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.935+0000 7f15d3400700 1 -- 172.21.15.131:0/3773208131 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e810c830 con 0x55e567c34530 2024-05-10T05:38:04.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.935+0000 7f15d3400700 1 -- 172.21.15.131:0/3773208131 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e810d5d0 con 0x55e567c34530 2024-05-10T05:38:04.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.935+0000 7f15d3400700 1 -- 172.21.15.131:0/3773208131 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e810c620 con 0x55e567c34530 2024-05-10T05:38:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.943+0000 7f1604fffb80 1 -- 172.21.15.131:0/3773208131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 msgr2=0x55e567c34960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.943+0000 7f1604fffb80 1 --2- 172.21.15.131:0/3773208131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c34960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f15e8050560 tx=0x7f15e8113120 comp rx=0 tx=0).stop 2024-05-10T05:38:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.945+0000 7f1604fffb80 1 -- 172.21.15.131:0/3773208131 shutdown_connections 2024-05-10T05:38:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.945+0000 7f1604fffb80 1 --2- 172.21.15.131:0/3773208131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 0x55e567c3c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.945+0000 7f1604fffb80 1 --2- 172.21.15.131:0/3773208131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c34960 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.945+0000 7f1604fffb80 1 --2- 172.21.15.131:0/3773208131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c357a0 0x55e567c35bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.945+0000 7f1604fffb80 1 -- 172.21.15.131:0/3773208131 >> 172.21.15.131:0/3773208131 conn(0x55e567b96a60 msgr2=0x55e567c2f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.946+0000 7f1604fffb80 1 -- 172.21.15.131:0/3773208131 shutdown_connections 2024-05-10T05:38:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.946+0000 7f1604fffb80 1 -- 172.21.15.131:0/3773208131 wait complete. 2024-05-10T05:38:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.948+0000 7f1604fffb80 1 -- start start 2024-05-10T05:38:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.949+0000 7f1604fffb80 1 Processor -- start 2024-05-10T05:38:04.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.950+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c34530 0x55e567c3e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.951+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c3ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.951+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c360f0 0x55e567c45960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.953+0000 7f15e5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c3ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.953+0000 7f15e5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c3ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55988/0 (socket says 172.21.15.131:55988) 2024-05-10T05:38:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.953+0000 7f1604fffb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567af0560 con 0x55e567c360f0 2024-05-10T05:38:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.953+0000 7f1604fffb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567af0200 con 0x55e567c34530 2024-05-10T05:38:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.953+0000 7f1604fffb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5679d76a0 con 0x55e567c357a0 2024-05-10T05:38:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.953+0000 7f1604fffb80 10 client.0 _ll_register_callbacks cb 0x55e567af9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.955+0000 7f15e5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c34530 0x55e567c3e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.955+0000 7f15e5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c34530 0x55e567c3e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:59272/0 (socket says 172.21.15.131:59272) 2024-05-10T05:38:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.955+0000 7f15e5e00700 1 -- 172.21.15.131:0/2671334126 learned_addr learned my addr 172.21.15.131:0/2671334126 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.956+0000 7f15e5e00700 1 -- 172.21.15.131:0/2671334126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 msgr2=0x55e567c3ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.956+0000 7f15e5e00700 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c3ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.956+0000 7f15e5e00700 1 -- 172.21.15.131:0/2671334126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c360f0 msgr2=0x55e567c45960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.956+0000 7f15e5e00700 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c360f0 0x55e567c45960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.956+0000 7f15e5e00700 1 -- 172.21.15.131:0/2671334126 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e0143d30 con 0x55e567c34530 2024-05-10T05:38:04.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.960+0000 7f15e5e00700 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c34530 0x55e567c3e4c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f15e8066e70 tx=0x7f15e80747f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2037dcb366160916 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:04.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.961+0000 7f15c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:38:04.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.962+0000 7f15c7e00700 1 -- 172.21.15.131:0/2671334126 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e81152c0 con 0x55e567c34530 2024-05-10T05:38:04.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.962+0000 7f15c7e00700 1 -- 172.21.15.131:0/2671334126 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f15e810ced0 con 0x55e567c34530 2024-05-10T05:38:04.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.962+0000 7f1604fffb80 1 -- 172.21.15.131:0/2671334126 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e567aad270 con 0x55e567c34530 2024-05-10T05:38:04.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.964+0000 7f15c7e00700 1 -- 172.21.15.131:0/2671334126 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e810c830 con 0x55e567c34530 2024-05-10T05:38:04.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.964+0000 7f1604fffb80 10 client.26249 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:04.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.964+0000 7f1604fffb80 1 -- 172.21.15.131:0/2671334126 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e567981b60 con 0x55e567c34530 2024-05-10T05:38:04.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.965+0000 7f15c7e00700 1 -- 172.21.15.131:0/2671334126 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e567aad270 con 0x55e567c34530 2024-05-10T05:38:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.969+0000 7f15c7e00700 1 -- 172.21.15.131:0/2671334126 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f15e810bff0 con 0x55e567c34530 2024-05-10T05:38:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.969+0000 7f1604fffb80 10 client.26249 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.969+0000 7f1604fffb80 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.969+0000 7f1604fffb80 20 client.26249 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.969+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e567c49d80 0x55e567c4c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.969+0000 7f1604fffb80 4 client.26249 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.969+0000 7f1604fffb80 1 -- 172.21.15.131:0/2671334126 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e567c03780 con 0x55e567c49d80 2024-05-10T05:38:04.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.971+0000 7f15e5400700 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e567c49d80 0x55e567c4c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:04.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.973+0000 7f15e5400700 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e567c49d80 0x55e567c4c1d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1861755e6b3ba37a server_cookie=90c44694c8dd061c in_seq=0 out_seq=0 2024-05-10T05:38:04.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.973+0000 7f15c7e00700 10 client.26249 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:04.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.974+0000 7f15c7e00700 1 -- 172.21.15.131:0/2671334126 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e567c03780 con 0x55e567c49d80 2024-05-10T05:38:04.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.974+0000 7f15c7e00700 10 client.26249 handle_command_reply: tid=0 2024-05-10T05:38:04.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.974+0000 7f1604fffb80 1 client.26249 shutdown 2024-05-10T05:38:04.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.975+0000 7f1604fffb80 1 -- 172.21.15.131:0/2671334126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c34530 msgr2=0x55e567c3e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:04.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.975+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c34530 0x55e567c3e4c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f15e8066e70 tx=0x7f15e80747f0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.975+0000 7f1604fffb80 1 -- 172.21.15.131:0/2671334126 shutdown_connections 2024-05-10T05:38:04.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.975+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e567c49d80 0x55e567c4c1d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.975+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c360f0 0x55e567c45960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.976+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c34530 0x55e567c3e4c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.976+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2671334126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c3ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:04.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.976+0000 7f1604fffb80 1 -- 172.21.15.131:0/2671334126 >> 172.21.15.131:0/2671334126 conn(0x55e567b96a60 msgr2=0x55e567c33060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:04.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.976+0000 7f15c7e00700 0 client.26249 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:04.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.983+0000 7f1604fffb80 1 -- 172.21.15.131:0/2671334126 shutdown_connections 2024-05-10T05:38:04.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.983+0000 7f1604fffb80 1 -- 172.21.15.131:0/2671334126 wait complete. 2024-05-10T05:38:04.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:04.984+0000 7f1604fffb80 20 client.26249 trim_cache size 0 max 16384 2024-05-10T05:38:05.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.095+0000 7f1604fffb80 1 Processor -- start 2024-05-10T05:38:05.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.095+0000 7f1604fffb80 1 -- start start 2024-05-10T05:38:05.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.096+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c2ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:05.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.096+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c357a0 0x55e567c45920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:05.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.096+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c360f0 0x55e567c389d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:05.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.096+0000 7f1604fffb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567c298d0 con 0x55e567c360f0 2024-05-10T05:38:05.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.096+0000 7f1604fffb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567c351f0 con 0x55e567c357a0 2024-05-10T05:38:05.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.096+0000 7f1604fffb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567c4c790 con 0x55e567c34530 2024-05-10T05:38:05.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.098+0000 7f15e5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c2ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:05.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.098+0000 7f15e5e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c2ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55996/0 (socket says 172.21.15.131:55996) 2024-05-10T05:38:05.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.098+0000 7f15e5e00700 1 -- 172.21.15.131:0/3602576126 learned_addr learned my addr 172.21.15.131:0/3602576126 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:05.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.100+0000 7f15e5400700 1 --2- 172.21.15.131:0/3602576126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c357a0 0x55e567c45920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.104+0000 7f15e5400700 1 -- 172.21.15.131:0/3602576126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 msgr2=0x55e567c2ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.104+0000 7f15e5400700 1 --2- 172.21.15.131:0/3602576126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c2ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.104+0000 7f15e5400700 1 -- 172.21.15.131:0/3602576126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c360f0 msgr2=0x55e567c389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.104+0000 7f15e5400700 1 --2- 172.21.15.131:0/3602576126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c360f0 0x55e567c389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.104+0000 7f15e5400700 1 -- 172.21.15.131:0/3602576126 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e567b1a560 con 0x55e567c357a0 2024-05-10T05:38:05.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.108+0000 7f15e5400700 1 --2- 172.21.15.131:0/3602576126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c357a0 0x55e567c45920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55e567c4c950 tx=0x7f15d8011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33a73821000fd4d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:05.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.109+0000 7f15d3e00700 1 -- 172.21.15.131:0/3602576126 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d801a040 con 0x55e567c357a0 2024-05-10T05:38:05.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.109+0000 7f15d3e00700 1 -- 172.21.15.131:0/3602576126 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15d801b040 con 0x55e567c357a0 2024-05-10T05:38:05.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.109+0000 7f15d3e00700 1 -- 172.21.15.131:0/3602576126 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8024540 con 0x55e567c357a0 2024-05-10T05:38:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.114+0000 7f1604fffb80 1 -- 172.21.15.131:0/3602576126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c357a0 msgr2=0x55e567c45920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.114+0000 7f1604fffb80 1 --2- 172.21.15.131:0/3602576126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c357a0 0x55e567c45920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55e567c4c950 tx=0x7f15d8011000 comp rx=0 tx=0).stop 2024-05-10T05:38:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.127+0000 7f1604fffb80 1 -- 172.21.15.131:0/3602576126 shutdown_connections 2024-05-10T05:38:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.127+0000 7f1604fffb80 1 --2- 172.21.15.131:0/3602576126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c360f0 0x55e567c389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.127+0000 7f1604fffb80 1 --2- 172.21.15.131:0/3602576126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c34530 0x55e567c2ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.127+0000 7f1604fffb80 1 --2- 172.21.15.131:0/3602576126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c357a0 0x55e567c45920 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.127+0000 7f1604fffb80 1 -- 172.21.15.131:0/3602576126 >> 172.21.15.131:0/3602576126 conn(0x55e567b96a60 msgr2=0x55e567c392f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:05.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.137+0000 7f1604fffb80 1 -- 172.21.15.131:0/3602576126 shutdown_connections 2024-05-10T05:38:05.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.138+0000 7f1604fffb80 1 -- 172.21.15.131:0/3602576126 wait complete. 2024-05-10T05:38:05.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.143+0000 7f1604fffb80 1 -- start start 2024-05-10T05:38:05.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.148+0000 7f1604fffb80 1 Processor -- start 2024-05-10T05:38:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.160+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c34530 0x55e567c45140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:05.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.161+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c45680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:05.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.162+0000 7f1604fffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 0x55e567c9e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:05.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.162+0000 7f15e5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c45680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:05.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.162+0000 7f15e5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c45680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56010/0 (socket says 172.21.15.131:56010) 2024-05-10T05:38:05.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.162+0000 7f15e5400700 1 -- 172.21.15.131:0/2376633207 learned_addr learned my addr 172.21.15.131:0/2376633207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:05.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.162+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567c9ed10 con 0x55e567c34530 2024-05-10T05:38:05.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.162+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567c9ee80 con 0x55e567c360f0 2024-05-10T05:38:05.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.162+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e567c9f040 con 0x55e567c357a0 2024-05-10T05:38:05.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.162+0000 7f1604fffb80 10 client.0 _ll_register_callbacks cb 0x55e567af9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:05.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.162+0000 7f15e5e00700 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c34530 0x55e567c45140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:05.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.163+0000 7f15e7200700 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 0x55e567c9e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.163+0000 7f15e7200700 1 -- 172.21.15.131:0/2376633207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 msgr2=0x55e567c45680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.165+0000 7f15e7200700 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c45680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.165+0000 7f15e7200700 1 -- 172.21.15.131:0/2376633207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c34530 msgr2=0x55e567c45140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.165+0000 7f15e7200700 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c34530 0x55e567c45140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.165+0000 7f15e7200700 1 -- 172.21.15.131:0/2376633207 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d8015900 con 0x55e567c360f0 2024-05-10T05:38:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.165+0000 7f15e5400700 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c45680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.165+0000 7f15e5e00700 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c34530 0x55e567c45140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.166+0000 7f15e7200700 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 0x55e567c9e7d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55e567c45f10 tx=0x7f15d4012a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30311ffa1190e0ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:05.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.169+0000 7f15c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:38:05.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.169+0000 7f15c7e00700 1 -- 172.21.15.131:0/2376633207 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d4017070 con 0x55e567c360f0 2024-05-10T05:38:05.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.169+0000 7f15c7e00700 1 -- 172.21.15.131:0/2376633207 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f15d4004050 con 0x55e567c360f0 2024-05-10T05:38:05.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.169+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15e810d5d0 con 0x55e567c360f0 2024-05-10T05:38:05.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.198+0000 7f15c7e00700 1 -- 172.21.15.131:0/2376633207 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15d4004210 con 0x55e567c360f0 2024-05-10T05:38:05.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.202+0000 7f1604fffb80 10 client.26255 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:05.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.202+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e5679d9cd0 con 0x55e567c360f0 2024-05-10T05:38:05.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.202+0000 7f15c7e00700 1 -- 172.21.15.131:0/2376633207 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e810d5d0 con 0x55e567c360f0 2024-05-10T05:38:05.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.203+0000 7f15c7e00700 1 -- 172.21.15.131:0/2376633207 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f15d4010430 con 0x55e567c360f0 2024-05-10T05:38:05.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.203+0000 7f1604fffb80 10 client.26255 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:05.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.203+0000 7f1604fffb80 10 client.26255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:05.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.203+0000 7f1604fffb80 20 client.26255 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:05.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.203+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e567ca0a60 0x55e567ca2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:05.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.203+0000 7f1604fffb80 4 client.26255 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:38:05.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.203+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e567b4cec0 con 0x55e567ca0a60 2024-05-10T05:38:05.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.204+0000 7f15e5e00700 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e567ca0a60 0x55e567ca2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:05.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.206+0000 7f15c7e00700 10 client.26255 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:05.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.208+0000 7f15e5e00700 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e567ca0a60 0x55e567ca2e80 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ac0f03996b38c46 server_cookie=aa34be4cb0a2e364 in_seq=0 out_seq=0 2024-05-10T05:38:05.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.210+0000 7f15c7e00700 1 -- 172.21.15.131:0/2376633207 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e567b4cec0 con 0x55e567ca0a60 2024-05-10T05:38:05.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.210+0000 7f15c7e00700 10 client.26255 handle_command_reply: tid=0 2024-05-10T05:38:05.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.214+0000 7f1604fffb80 1 client.26255 shutdown 2024-05-10T05:38:05.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.219+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 msgr2=0x55e567c9e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:05.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.219+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 0x55e567c9e7d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55e567c45f10 tx=0x7f15d4012a60 comp rx=0 tx=0).stop 2024-05-10T05:38:05.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.219+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 shutdown_connections 2024-05-10T05:38:05.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.219+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e567ca0a60 0x55e567ca2e80 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.219+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e567c34530 0x55e567c45140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.219+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e567c360f0 0x55e567c9e7d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.219+0000 7f1604fffb80 1 --2- 172.21.15.131:0/2376633207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e567c357a0 0x55e567c45680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.219+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 >> 172.21.15.131:0/2376633207 conn(0x55e567b96a60 msgr2=0x55e567c33060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:05.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.219+0000 7f15c7e00700 0 client.26255 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:05.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.220+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 shutdown_connections 2024-05-10T05:38:05.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.224+0000 7f1604fffb80 1 -- 172.21.15.131:0/2376633207 wait complete. 2024-05-10T05:38:05.228 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:38:05.228 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:38:05.228 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:38:05.228 INFO:teuthology.orchestra.run.smithi131.stdout: "f33808c6-5684-43da-a9b8-c4486162cf76": { 2024-05-10T05:38:05.228 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:38:05.229 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "f33808c6-5684-43da-a9b8-c4486162cf76", 2024-05-10T05:38:05.229 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:38:05.229 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:38:05.229 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:38:05.229 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:38:05.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.227+0000 7f1604fffb80 20 client.26255 trim_cache size 0 max 16384 2024-05-10T05:38:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.234+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f15cc0a1220 msgr2=0x7f15cc0a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.234+0000 7f15ed000700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f15cc0a1220 0x7f15cc0a3730 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f15e00a6c60 tx=0x7f15d8006d20 comp rx=0 tx=0).stop 2024-05-10T05:38:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.234+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e0095520 msgr2=0x7f15e00a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.234+0000 7f15ed000700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e0095520 0x7f15e00a5c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f15e8064500 tx=0x7f15e8069300 comp rx=0 tx=0).stop 2024-05-10T05:38:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.234+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 shutdown_connections 2024-05-10T05:38:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.234+0000 7f15ed000700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f15cc0a1220 0x7f15cc0a3730 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.234+0000 7f15ed000700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e0096fe0 0x7f15e00ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.234+0000 7f15ed000700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e0095520 0x7f15e00a5c10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.234+0000 7f15ed000700 1 --2- 172.21.15.131:0/2823829720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e0096640 0x7f15e00a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.234+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 >> 172.21.15.131:0/2823829720 conn(0x7f15e0090b10 msgr2=0x7f15e0099870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.235+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 shutdown_connections 2024-05-10T05:38:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:05.235+0000 7f15ed000700 1 -- 172.21.15.131:0/2823829720 wait complete. 2024-05-10T05:38:05.315 INFO:tasks.cephfs.filesystem:scrub status for tag:f33808c6-5684-43da-a9b8-c4486162cf76 - {'path': '/', 'tag': 'f33808c6-5684-43da-a9b8-c4486162cf76', 'options': 'recursive,force'} 2024-05-10T05:38:05.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:05 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:38:05.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:05 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:05.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:05 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:38:05.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:05 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:38:05.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:05 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:05.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:05 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:05.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:05 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:38:05.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:05 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:05.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:05 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:38:05.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:05 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:38:05.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:05 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:05.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:05 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:05.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:05 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:38:05.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:05 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:05.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:05 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:38:05.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:05 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:38:05.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:05 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:05.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:05 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:06.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:06 smithi131 ceph-mon[37264]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 82 op/s 2024-05-10T05:38:06.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:06 smithi157 ceph-mon[78427]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 82 op/s 2024-05-10T05:38:06.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:06 smithi142 ceph-mon[77667]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 82 op/s 2024-05-10T05:38:07.468 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:38:07.468 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:38:07.468 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:38:07.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:38:07.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:38:07.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:38:07.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:38:07.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:38:07.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:38:08.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:08 smithi131 ceph-mon[37264]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 238 MiB/s wr, 74 op/s 2024-05-10T05:38:08.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:08 smithi157 ceph-mon[78427]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 238 MiB/s wr, 74 op/s 2024-05-10T05:38:08.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:08 smithi142 ceph-mon[77667]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 238 MiB/s wr, 74 op/s 2024-05-10T05:38:10.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:10 smithi157 ceph-mon[78427]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 261 MiB/s wr, 79 op/s 2024-05-10T05:38:10.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:10 smithi142 ceph-mon[77667]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 261 MiB/s wr, 79 op/s 2024-05-10T05:38:10.716 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:10 smithi131 ceph-mon[37264]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 261 MiB/s wr, 79 op/s 2024-05-10T05:38:11.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:11 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:38:11.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:11 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:38:11.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:11 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:38:11.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:11 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:38:11.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:11 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:38:11.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:11 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:38:12.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:12 smithi157 ceph-mon[78427]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 264 MiB/s wr, 81 op/s 2024-05-10T05:38:12.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:12 smithi142 ceph-mon[77667]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 264 MiB/s wr, 81 op/s 2024-05-10T05:38:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:12 smithi131 ceph-mon[37264]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 264 MiB/s wr, 81 op/s 2024-05-10T05:38:13.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:13 smithi142 ceph-mon[77667]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 77 op/s 2024-05-10T05:38:13.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:13 smithi131 ceph-mon[37264]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 77 op/s 2024-05-10T05:38:13.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:13 smithi157 ceph-mon[78427]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 77 op/s 2024-05-10T05:38:16.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:16 smithi131 ceph-mon[37264]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 285 MiB/s wr, 85 op/s 2024-05-10T05:38:16.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:16 smithi157 ceph-mon[78427]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 285 MiB/s wr, 85 op/s 2024-05-10T05:38:16.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:16 smithi142 ceph-mon[77667]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 285 MiB/s wr, 85 op/s 2024-05-10T05:38:18.470 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:18 smithi131 ceph-mon[37264]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 246 MiB/s wr, 77 op/s 2024-05-10T05:38:18.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:18 smithi157 ceph-mon[78427]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 246 MiB/s wr, 77 op/s 2024-05-10T05:38:18.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:18 smithi142 ceph-mon[77667]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 246 MiB/s wr, 77 op/s 2024-05-10T05:38:20.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:20 smithi131 ceph-mon[37264]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 270 MiB/s wr, 82 op/s 2024-05-10T05:38:20.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:20 smithi157 ceph-mon[78427]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 270 MiB/s wr, 82 op/s 2024-05-10T05:38:20.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:20 smithi142 ceph-mon[77667]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 270 MiB/s wr, 82 op/s 2024-05-10T05:38:22.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:22 smithi131 ceph-mon[37264]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 266 MiB/s wr, 83 op/s 2024-05-10T05:38:22.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:22 smithi157 ceph-mon[78427]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 266 MiB/s wr, 83 op/s 2024-05-10T05:38:22.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:22 smithi142 ceph-mon[77667]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 266 MiB/s wr, 83 op/s 2024-05-10T05:38:24.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:24 smithi157 ceph-mon[78427]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 243 MiB/s wr, 75 op/s 2024-05-10T05:38:24.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:24 smithi142 ceph-mon[77667]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 243 MiB/s wr, 75 op/s 2024-05-10T05:38:24.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:24 smithi131 ceph-mon[37264]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 243 MiB/s wr, 75 op/s 2024-05-10T05:38:26.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:26 smithi157 ceph-mon[78427]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 294 MiB/s wr, 89 op/s 2024-05-10T05:38:26.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:26 smithi142 ceph-mon[77667]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 294 MiB/s wr, 89 op/s 2024-05-10T05:38:26.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:26 smithi131 ceph-mon[37264]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 294 MiB/s wr, 89 op/s 2024-05-10T05:38:28.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:28 smithi157 ceph-mon[78427]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 260 MiB/s wr, 81 op/s 2024-05-10T05:38:28.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:28 smithi142 ceph-mon[77667]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 260 MiB/s wr, 81 op/s 2024-05-10T05:38:28.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:28 smithi131 ceph-mon[37264]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 260 MiB/s wr, 81 op/s 2024-05-10T05:38:30.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:30 smithi157 ceph-mon[78427]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 280 MiB/s wr, 84 op/s 2024-05-10T05:38:30.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:30 smithi142 ceph-mon[77667]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 280 MiB/s wr, 84 op/s 2024-05-10T05:38:30.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:30 smithi131 ceph-mon[37264]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 280 MiB/s wr, 84 op/s 2024-05-10T05:38:31.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:31 smithi142 ceph-mon[77667]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 296 MiB/s wr, 91 op/s 2024-05-10T05:38:31.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:31 smithi131 ceph-mon[37264]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 296 MiB/s wr, 91 op/s 2024-05-10T05:38:31.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:31 smithi157 ceph-mon[78427]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 296 MiB/s wr, 91 op/s 2024-05-10T05:38:34.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:34 smithi131 ceph-mon[37264]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 262 MiB/s wr, 79 op/s 2024-05-10T05:38:34.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:34 smithi157 ceph-mon[78427]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 262 MiB/s wr, 79 op/s 2024-05-10T05:38:34.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:34 smithi142 ceph-mon[77667]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 262 MiB/s wr, 79 op/s 2024-05-10T05:38:35.317 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:38:36.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.080+0000 7f331fe00700 1 -- 172.21.15.131:0/181042121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33201055a0 msgr2=0x7f33201059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:36.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.080+0000 7f331fe00700 1 --2- 172.21.15.131:0/181042121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33201055a0 0x7f33201059b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3318009f90 tx=0x7f331800b330 comp rx=0 tx=0).stop 2024-05-10T05:38:36.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.080+0000 7f331fe00700 1 -- 172.21.15.131:0/181042121 shutdown_connections 2024-05-10T05:38:36.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.080+0000 7f331fe00700 1 --2- 172.21.15.131:0/181042121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33201055a0 0x7f33201059b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.080+0000 7f331fe00700 1 --2- 172.21.15.131:0/181042121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33201002e0 0x7f3320102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.080+0000 7f331fe00700 1 --2- 172.21.15.131:0/181042121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3320102c40 0x7f3320105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.080+0000 7f331fe00700 1 -- 172.21.15.131:0/181042121 >> 172.21.15.131:0/181042121 conn(0x7f33200fbf30 msgr2=0x7f33200fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.081+0000 7f331fe00700 1 -- 172.21.15.131:0/181042121 shutdown_connections 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.081+0000 7f331fe00700 1 -- 172.21.15.131:0/181042121 wait complete. 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.083+0000 7f331fe00700 1 Processor -- start 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.087+0000 7f331fe00700 1 -- start start 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.088+0000 7f331fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33201002e0 0x7f332019e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.088+0000 7f331fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320102c40 0x7f332019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.088+0000 7f331fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33201055a0 0x7f33201a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.088+0000 7f331fe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33201a5fa0 con 0x7f33201055a0 2024-05-10T05:38:36.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.088+0000 7f331fe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33201a60e0 con 0x7f33201002e0 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.088+0000 7f331fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33201a6270 con 0x7f3320102c40 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.088+0000 7f331ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33201002e0 0x7f332019e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.088+0000 7f331ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33201002e0 0x7f332019e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39098/0 (socket says 172.21.15.131:39098) 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.088+0000 7f331ea00700 1 -- 172.21.15.131:0/791625386 learned_addr learned my addr 172.21.15.131:0/791625386 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.089+0000 7f331ea00700 1 -- 172.21.15.131:0/791625386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320102c40 msgr2=0x7f332019ea40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.089+0000 7f331ea00700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320102c40 0x7f332019ea40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.089+0000 7f331ea00700 1 -- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33201055a0 msgr2=0x7f33201a59d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.089+0000 7f331f400700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33201055a0 0x7f33201a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.089+0000 7f331ea00700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33201055a0 0x7f33201a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.089+0000 7f331ea00700 1 -- 172.21.15.131:0/791625386 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318009c40 con 0x7f33201002e0 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.089+0000 7f331f400700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33201055a0 0x7f33201a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.090+0000 7f331ea00700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33201002e0 0x7f332019e500 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f331000d980 tx=0x7f331000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:36.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.090+0000 7f3317400700 1 -- 172.21.15.131:0/791625386 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310009b40 con 0x7f33201002e0 2024-05-10T05:38:36.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.091+0000 7f3317400700 1 -- 172.21.15.131:0/791625386 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f331000de60 con 0x7f33201002e0 2024-05-10T05:38:36.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.091+0000 7f331fe00700 1 -- 172.21.15.131:0/791625386 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33201a6550 con 0x7f33201002e0 2024-05-10T05:38:36.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.093+0000 7f331fe00700 1 -- 172.21.15.131:0/791625386 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33201a6ab0 con 0x7f33201002e0 2024-05-10T05:38:36.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.094+0000 7f3317400700 1 -- 172.21.15.131:0/791625386 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331000f620 con 0x7f33201002e0 2024-05-10T05:38:36.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.096+0000 7f3317400700 1 -- 172.21.15.131:0/791625386 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f331000f7c0 con 0x7f33201002e0 2024-05-10T05:38:36.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.107+0000 7f333d790b80 1 Processor -- start 2024-05-10T05:38:36.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.112+0000 7f3317400700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33040a11a0 0x7f33040a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.112+0000 7f3317400700 1 -- 172.21.15.131:0/791625386 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f33100d0170 con 0x7f33201002e0 2024-05-10T05:38:36.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.113+0000 7f331f400700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33040a11a0 0x7f33040a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.114+0000 7f333d790b80 1 -- start start 2024-05-10T05:38:36.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.115+0000 7f331f400700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33040a11a0 0x7f33040a36b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3318006070 tx=0x7f3318006000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:36.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.117+0000 7f333d790b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee7625960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.117+0000 7f333d790b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 0x55bee7626bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.117+0000 7f333d790b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76270f0 0x55bee762d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.117+0000 7f333d790b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee754c150 con 0x55bee76267a0 2024-05-10T05:38:36.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.117+0000 7f333d790b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee7549b10 con 0x55bee76270f0 2024-05-10T05:38:36.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.117+0000 7f333d790b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee7545400 con 0x55bee7625530 2024-05-10T05:38:36.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.117+0000 7f331f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76270f0 0x55bee762d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.117+0000 7f331f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76270f0 0x55bee762d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39108/0 (socket says 172.21.15.131:39108) 2024-05-10T05:38:36.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.117+0000 7f331f400700 1 -- 172.21.15.131:0/2647008883 learned_addr learned my addr 172.21.15.131:0/2647008883 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:36.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.117+0000 7f331ea00700 1 --2- 172.21.15.131:0/2647008883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee7625960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.118+0000 7f331e000700 1 --2- 172.21.15.131:0/2647008883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 0x55bee7626bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.118+0000 7f331f400700 1 -- 172.21.15.131:0/2647008883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 msgr2=0x55bee7625960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.118+0000 7f331f400700 1 --2- 172.21.15.131:0/2647008883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee7625960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.118+0000 7f331f400700 1 -- 172.21.15.131:0/2647008883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 msgr2=0x55bee7626bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.118+0000 7f331f400700 1 --2- 172.21.15.131:0/2647008883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 0x55bee7626bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.118+0000 7f331f400700 1 -- 172.21.15.131:0/2647008883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bee73c63b0 con 0x55bee76270f0 2024-05-10T05:38:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.118+0000 7f331e000700 1 --2- 172.21.15.131:0/2647008883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 0x55bee7626bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:36.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.118+0000 7f331f400700 1 --2- 172.21.15.131:0/2647008883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76270f0 0x55bee762d9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3318007680 tx=0x7f331801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23fa04f3d6ceb116 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.122+0000 7f330fe00700 1 -- 172.21.15.131:0/2647008883 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331800e040 con 0x55bee76270f0 2024-05-10T05:38:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.122+0000 7f330fe00700 1 -- 172.21.15.131:0/2647008883 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3318004c50 con 0x55bee76270f0 2024-05-10T05:38:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.122+0000 7f330fe00700 1 -- 172.21.15.131:0/2647008883 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331801da90 con 0x55bee76270f0 2024-05-10T05:38:36.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.123+0000 7f333d790b80 1 -- 172.21.15.131:0/2647008883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76270f0 msgr2=0x55bee762d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:36.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.123+0000 7f333d790b80 1 --2- 172.21.15.131:0/2647008883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76270f0 0x55bee762d9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3318007680 tx=0x7f331801a000 comp rx=0 tx=0).stop 2024-05-10T05:38:36.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.127+0000 7f333d790b80 1 -- 172.21.15.131:0/2647008883 shutdown_connections 2024-05-10T05:38:36.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.127+0000 7f333d790b80 1 --2- 172.21.15.131:0/2647008883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76270f0 0x55bee762d9f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.127+0000 7f333d790b80 1 --2- 172.21.15.131:0/2647008883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee7625960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.127+0000 7f333d790b80 1 --2- 172.21.15.131:0/2647008883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 0x55bee7626bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.127+0000 7f333d790b80 1 -- 172.21.15.131:0/2647008883 >> 172.21.15.131:0/2647008883 conn(0x55bee7587a60 msgr2=0x55bee7620330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:36.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.131+0000 7f333d790b80 1 -- 172.21.15.131:0/2647008883 shutdown_connections 2024-05-10T05:38:36.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.132+0000 7f333d790b80 1 -- 172.21.15.131:0/2647008883 wait complete. 2024-05-10T05:38:36.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.133+0000 7f333d790b80 1 -- start start 2024-05-10T05:38:36.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.135+0000 7f333d790b80 1 Processor -- start 2024-05-10T05:38:36.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.137+0000 7f333d790b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee762f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.138+0000 7f333d790b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee762fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.138+0000 7f333d790b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 0x55bee7636d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.138+0000 7f331e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee762fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.138+0000 7f331e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee762fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39118/0 (socket says 172.21.15.131:39118) 2024-05-10T05:38:36.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.139+0000 7f331ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee762f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.139+0000 7f331ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee762f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37680/0 (socket says 172.21.15.131:37680) 2024-05-10T05:38:36.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.139+0000 7f331e000700 1 -- 172.21.15.131:0/1723666717 learned_addr learned my addr 172.21.15.131:0/1723666717 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:36.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.139+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee74e1560 con 0x55bee76270f0 2024-05-10T05:38:36.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.139+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee74e1200 con 0x55bee76267a0 2024-05-10T05:38:36.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.139+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee73c86a0 con 0x55bee7625530 2024-05-10T05:38:36.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.139+0000 7f333d790b80 10 client.0 _ll_register_callbacks cb 0x55bee74eab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.140+0000 7f331f400700 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 0x55bee7636d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.140+0000 7f331ea00700 1 -- 172.21.15.131:0/1723666717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 msgr2=0x55bee762fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.140+0000 7f331ea00700 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee762fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.140+0000 7f331ea00700 1 -- 172.21.15.131:0/1723666717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 msgr2=0x55bee7636d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.140+0000 7f331ea00700 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 0x55bee7636d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.140+0000 7f331ea00700 1 -- 172.21.15.131:0/1723666717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33201065d0 con 0x55bee7625530 2024-05-10T05:38:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.140+0000 7f331f400700 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 0x55bee7636d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.141+0000 7f331ea00700 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee762f8b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f33201a6c40 tx=0x7f331009a390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83c3620391a47236 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:36.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.141+0000 7f330cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:38:36.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.141+0000 7f330cc00700 1 -- 172.21.15.131:0/1723666717 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310014020 con 0x55bee7625530 2024-05-10T05:38:36.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.141+0000 7f330cc00700 1 -- 172.21.15.131:0/1723666717 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f33100a2110 con 0x55bee7625530 2024-05-10T05:38:36.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.141+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bee749e270 con 0x55bee7625530 2024-05-10T05:38:36.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.143+0000 7f330cc00700 1 -- 172.21.15.131:0/1723666717 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f331009a0f0 con 0x55bee7625530 2024-05-10T05:38:36.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.143+0000 7f333d790b80 10 client.26290 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:36.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.143+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bee7372b60 con 0x55bee7625530 2024-05-10T05:38:36.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.144+0000 7f330cc00700 1 -- 172.21.15.131:0/1723666717 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bee749e270 con 0x55bee7625530 2024-05-10T05:38:36.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.144+0000 7f330cc00700 1 -- 172.21.15.131:0/1723666717 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3310098bf0 con 0x55bee7625530 2024-05-10T05:38:36.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.147+0000 7f333d790b80 10 client.26290 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:36.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.147+0000 7f333d790b80 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:36.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.147+0000 7f333d790b80 20 client.26290 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:36.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.147+0000 7f333d790b80 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bee763b2c0 0x55bee763d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.147+0000 7f333d790b80 4 client.26290 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:36.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.147+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bee75f4780 con 0x55bee763b2c0 2024-05-10T05:38:36.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.147+0000 7f331e000700 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bee763b2c0 0x55bee763d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.149+0000 7f331e000700 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bee763b2c0 0x55bee763d710 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1c7eb42114e41 server_cookie=b2bab4343431778d in_seq=0 out_seq=0 2024-05-10T05:38:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.149+0000 7f330cc00700 10 client.26290 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.150+0000 7f330cc00700 1 -- 172.21.15.131:0/1723666717 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bee75f4780 con 0x55bee763b2c0 2024-05-10T05:38:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.150+0000 7f330cc00700 10 client.26290 handle_command_reply: tid=0 2024-05-10T05:38:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.150+0000 7f333d790b80 1 client.26290 shutdown 2024-05-10T05:38:36.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 msgr2=0x55bee762f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:36.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f333d790b80 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee762f8b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f33201a6c40 tx=0x7f331009a390 comp rx=0 tx=0).stop 2024-05-10T05:38:36.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 shutdown_connections 2024-05-10T05:38:36.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f333d790b80 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bee763b2c0 0x55bee763d710 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f333d790b80 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 0x55bee7636d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f333d790b80 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee762f8b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f333d790b80 1 --2- 172.21.15.131:0/1723666717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee762fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f330cc00700 0 client.26290 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:36.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 >> 172.21.15.131:0/1723666717 conn(0x55bee7587a60 msgr2=0x55bee7624060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:36.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 shutdown_connections 2024-05-10T05:38:36.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.152+0000 7f333d790b80 1 -- 172.21.15.131:0/1723666717 wait complete. 2024-05-10T05:38:36.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.153+0000 7f333d790b80 20 client.26290 trim_cache size 0 max 16384 2024-05-10T05:38:36.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.190+0000 7f333d790b80 1 Processor -- start 2024-05-10T05:38:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.190+0000 7f333d790b80 1 -- start start 2024-05-10T05:38:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.192+0000 7f333d790b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee7625530 0x55bee762dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.192+0000 7f333d790b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 0x55bee7636d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.192+0000 7f333d790b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee76270f0 0x55bee762e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.192+0000 7f333d790b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee7637e60 con 0x55bee76267a0 2024-05-10T05:38:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.192+0000 7f333d790b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee761c340 con 0x55bee7625530 2024-05-10T05:38:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.192+0000 7f333d790b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee763afc0 con 0x55bee76270f0 2024-05-10T05:38:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.193+0000 7f331ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee7625530 0x55bee762dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.193+0000 7f331f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee76270f0 0x55bee762e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.193+0000 7f331f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee76270f0 0x55bee762e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37694/0 (socket says 172.21.15.131:37694) 2024-05-10T05:38:36.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.193+0000 7f331f400700 1 -- 172.21.15.131:0/3697022658 learned_addr learned my addr 172.21.15.131:0/3697022658 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:36.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.194+0000 7f331f400700 1 -- 172.21.15.131:0/3697022658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee7625530 msgr2=0x55bee762dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:36.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.194+0000 7f331e000700 1 --2- 172.21.15.131:0/3697022658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 0x55bee7636d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.194+0000 7f331f400700 1 --2- 172.21.15.131:0/3697022658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee7625530 0x55bee762dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.194+0000 7f331f400700 1 -- 172.21.15.131:0/3697022658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 msgr2=0x55bee7636d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.195+0000 7f331f400700 1 --2- 172.21.15.131:0/3697022658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 0x55bee7636d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.195+0000 7f331f400700 1 -- 172.21.15.131:0/3697022658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bee750b560 con 0x55bee76270f0 2024-05-10T05:38:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.195+0000 7f331e000700 1 --2- 172.21.15.131:0/3697022658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 0x55bee7636d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.196+0000 7f331f400700 1 --2- 172.21.15.131:0/3697022658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee76270f0 0x55bee762e140 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3318009510 tx=0x7f3318009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6605e81e83a8f6e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:36.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.196+0000 7f3314c00700 1 -- 172.21.15.131:0/3697022658 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331800e040 con 0x55bee76270f0 2024-05-10T05:38:36.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.197+0000 7f3314c00700 1 -- 172.21.15.131:0/3697022658 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3318004dc0 con 0x55bee76270f0 2024-05-10T05:38:36.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.198+0000 7f333d790b80 1 -- 172.21.15.131:0/3697022658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee76270f0 msgr2=0x55bee762e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:36.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.198+0000 7f333d790b80 1 --2- 172.21.15.131:0/3697022658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee76270f0 0x55bee762e140 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3318009510 tx=0x7f3318009920 comp rx=0 tx=0).stop 2024-05-10T05:38:36.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.199+0000 7f333d790b80 1 -- 172.21.15.131:0/3697022658 shutdown_connections 2024-05-10T05:38:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.199+0000 7f333d790b80 1 --2- 172.21.15.131:0/3697022658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee76270f0 0x55bee762e140 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.199+0000 7f333d790b80 1 --2- 172.21.15.131:0/3697022658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee7625530 0x55bee762dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.199+0000 7f333d790b80 1 --2- 172.21.15.131:0/3697022658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76267a0 0x55bee7636d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.199+0000 7f333d790b80 1 -- 172.21.15.131:0/3697022658 >> 172.21.15.131:0/3697022658 conn(0x55bee7587a60 msgr2=0x55bee762a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.200+0000 7f333d790b80 1 -- 172.21.15.131:0/3697022658 shutdown_connections 2024-05-10T05:38:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.200+0000 7f333d790b80 1 -- 172.21.15.131:0/3697022658 wait complete. 2024-05-10T05:38:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.201+0000 7f333d790b80 1 -- start start 2024-05-10T05:38:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.201+0000 7f333d790b80 1 Processor -- start 2024-05-10T05:38:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.202+0000 7f333d790b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee7634890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.202+0000 7f333d790b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee7634dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.202+0000 7f333d790b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 0x55bee7635d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.203+0000 7f331ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee7634890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.203+0000 7f331ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee7634890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37700/0 (socket says 172.21.15.131:37700) 2024-05-10T05:38:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.203+0000 7f331e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee7634dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.203+0000 7f331e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee7634dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39138/0 (socket says 172.21.15.131:39138) 2024-05-10T05:38:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.203+0000 7f331ea00700 1 -- 172.21.15.131:0/3373144875 learned_addr learned my addr 172.21.15.131:0/3373144875 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.203+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee7636240 con 0x55bee76270f0 2024-05-10T05:38:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.203+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee76363b0 con 0x55bee76267a0 2024-05-10T05:38:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.203+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bee7636570 con 0x55bee7625530 2024-05-10T05:38:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.203+0000 7f333d790b80 10 client.0 _ll_register_callbacks cb 0x55bee74eab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.204+0000 7f331f400700 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 0x55bee7635d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.204+0000 7f331ea00700 1 -- 172.21.15.131:0/3373144875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 msgr2=0x55bee7634dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.204+0000 7f331ea00700 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee7634dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.204+0000 7f331ea00700 1 -- 172.21.15.131:0/3373144875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 msgr2=0x55bee7635d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.204+0000 7f331ea00700 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 0x55bee7635d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.204+0000 7f331ea00700 1 -- 172.21.15.131:0/3373144875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bee73c63b0 con 0x55bee7625530 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.204+0000 7f331e000700 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee7634dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.204+0000 7f331f400700 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 0x55bee7635d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.205+0000 7f330cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.205+0000 7f331ea00700 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee7634890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3310096700 tx=0x7f33100ca8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bc6b60bf69a8aad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:36.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.205+0000 7f330cc00700 1 -- 172.21.15.131:0/3373144875 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310014040 con 0x55bee7625530 2024-05-10T05:38:36.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.205+0000 7f330cc00700 1 -- 172.21.15.131:0/3373144875 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f33100a2110 con 0x55bee7625530 2024-05-10T05:38:36.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.206+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3318004c50 con 0x55bee7625530 2024-05-10T05:38:36.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.206+0000 7f330cc00700 1 -- 172.21.15.131:0/3373144875 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3310014040 con 0x55bee7625530 2024-05-10T05:38:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.206+0000 7f333d790b80 10 client.26296 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.206+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bee73cacd0 con 0x55bee7625530 2024-05-10T05:38:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.206+0000 7f330cc00700 1 -- 172.21.15.131:0/3373144875 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f331007e080 con 0x55bee7625530 2024-05-10T05:38:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.207+0000 7f330cc00700 1 -- 172.21.15.131:0/3373144875 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3310098bf0 con 0x55bee7625530 2024-05-10T05:38:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.207+0000 7f333d790b80 10 client.26296 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.207+0000 7f333d790b80 10 client.26296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.207+0000 7f333d790b80 20 client.26296 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.207+0000 7f333d790b80 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bee7691fb0 0x55bee76943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.207+0000 7f333d790b80 4 client.26296 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:38:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.207+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bee753dec0 con 0x55bee7691fb0 2024-05-10T05:38:36.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.207+0000 7f331e000700 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bee7691fb0 0x55bee76943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.208+0000 7f331e000700 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bee7691fb0 0x55bee76943d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c83351322181a1a server_cookie=4c6ce2e7cf97a40e in_seq=0 out_seq=0 2024-05-10T05:38:36.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.208+0000 7f330cc00700 10 client.26296 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:36.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.208+0000 7f330cc00700 1 -- 172.21.15.131:0/3373144875 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bee753dec0 con 0x55bee7691fb0 2024-05-10T05:38:36.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.208+0000 7f330cc00700 10 client.26296 handle_command_reply: tid=0 2024-05-10T05:38:36.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.208+0000 7f333d790b80 1 client.26296 shutdown 2024-05-10T05:38:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.209+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 msgr2=0x55bee7634890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.209+0000 7f333d790b80 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee7634890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3310096700 tx=0x7f33100ca8e0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.209+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 shutdown_connections 2024-05-10T05:38:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.209+0000 7f333d790b80 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bee7691fb0 0x55bee76943d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.209+0000 7f333d790b80 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bee76270f0 0x55bee7635d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.209+0000 7f333d790b80 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bee7625530 0x55bee7634890 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.209+0000 7f333d790b80 1 --2- 172.21.15.131:0/3373144875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee76267a0 0x55bee7634dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.209+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 >> 172.21.15.131:0/3373144875 conn(0x55bee7587a60 msgr2=0x55bee761fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:36.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.210+0000 7f330cc00700 0 client.26296 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:36.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.210+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 shutdown_connections 2024-05-10T05:38:36.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.210+0000 7f333d790b80 1 -- 172.21.15.131:0/3373144875 wait complete. 2024-05-10T05:38:36.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.210+0000 7f333d790b80 20 client.26296 trim_cache size 0 max 16384 2024-05-10T05:38:36.212 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:38:36.212 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:38:36.212 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:38:36.212 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:38:36.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.211+0000 7f331fe00700 1 -- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33040a11a0 msgr2=0x7f33040a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:36.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.211+0000 7f331fe00700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33040a11a0 0x7f33040a36b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3318006070 tx=0x7f3318006000 comp rx=0 tx=0).stop 2024-05-10T05:38:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.211+0000 7f331fe00700 1 -- 172.21.15.131:0/791625386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33201002e0 msgr2=0x7f332019e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.211+0000 7f331fe00700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33201002e0 0x7f332019e500 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f331000d980 tx=0x7f331000dd40 comp rx=0 tx=0).stop 2024-05-10T05:38:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.213+0000 7f331fe00700 1 -- 172.21.15.131:0/791625386 shutdown_connections 2024-05-10T05:38:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.213+0000 7f331fe00700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f33040a11a0 0x7f33040a36b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.213+0000 7f331fe00700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33201055a0 0x7f33201a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.213+0000 7f331fe00700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33201002e0 0x7f332019e500 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.213+0000 7f331fe00700 1 --2- 172.21.15.131:0/791625386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3320102c40 0x7f332019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.213+0000 7f331fe00700 1 -- 172.21.15.131:0/791625386 >> 172.21.15.131:0/791625386 conn(0x7f33200fbf30 msgr2=0x7f3320100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.213+0000 7f331fe00700 1 -- 172.21.15.131:0/791625386 shutdown_connections 2024-05-10T05:38:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.213+0000 7f331fe00700 1 -- 172.21.15.131:0/791625386 wait complete. 2024-05-10T05:38:36.273 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:38:36.274 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:38:36.471 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:36 smithi131 ceph-mon[37264]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 298 MiB/s wr, 89 op/s 2024-05-10T05:38:36.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:36 smithi157 ceph-mon[78427]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 298 MiB/s wr, 89 op/s 2024-05-10T05:38:36.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:36 smithi142 ceph-mon[77667]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 298 MiB/s wr, 89 op/s 2024-05-10T05:38:36.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.894+0000 7f4945e00700 1 -- 172.21.15.131:0/854626873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4940073b40 msgr2=0x7f4940074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:36.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.894+0000 7f4945e00700 1 --2- 172.21.15.131:0/854626873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4940073b40 0x7f4940074010 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f492c009990 tx=0x7f492c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.895+0000 7f4945e00700 1 -- 172.21.15.131:0/854626873 shutdown_connections 2024-05-10T05:38:36.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.895+0000 7f4945e00700 1 --2- 172.21.15.131:0/854626873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4940109ac0 0x7f494010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.895+0000 7f4945e00700 1 --2- 172.21.15.131:0/854626873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4940075140 0x7f4940073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.895+0000 7f4945e00700 1 --2- 172.21.15.131:0/854626873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4940073b40 0x7f4940074010 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.895+0000 7f4945e00700 1 -- 172.21.15.131:0/854626873 >> 172.21.15.131:0/854626873 conn(0x7f49400fd1d0 msgr2=0x7f49400ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:36.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.895+0000 7f4945e00700 1 -- 172.21.15.131:0/854626873 shutdown_connections 2024-05-10T05:38:36.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.895+0000 7f4945e00700 1 -- 172.21.15.131:0/854626873 wait complete. 2024-05-10T05:38:36.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.897+0000 7f4945e00700 1 Processor -- start 2024-05-10T05:38:36.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.897+0000 7f4945e00700 1 -- start start 2024-05-10T05:38:36.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.898+0000 7f4945e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4940075140 0x7f4940111230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.898+0000 7f4945e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4940109ac0 0x7f4940111830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.898+0000 7f4945e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49401166f0 0x7f4940116b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.898+0000 7f4945e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49401171e0 con 0x7f4940109ac0 2024-05-10T05:38:36.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.898+0000 7f4945e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4940117350 con 0x7f4940075140 2024-05-10T05:38:36.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.898+0000 7f4945e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4940117510 con 0x7f49401166f0 2024-05-10T05:38:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.899+0000 7f493f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49401166f0 0x7f4940116b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.899+0000 7f493f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49401166f0 0x7f4940116b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37718/0 (socket says 172.21.15.131:37718) 2024-05-10T05:38:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.899+0000 7f493f400700 1 -- 172.21.15.131:0/2520384579 learned_addr learned my addr 172.21.15.131:0/2520384579 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.899+0000 7f493ea00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4940075140 0x7f4940111230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.899+0000 7f493f400700 1 -- 172.21.15.131:0/2520384579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4940075140 msgr2=0x7f4940111230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.899+0000 7f493f400700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4940075140 0x7f4940111230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.899+0000 7f493f400700 1 -- 172.21.15.131:0/2520384579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4940109ac0 msgr2=0x7f4940111830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.899+0000 7f493f400700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4940109ac0 0x7f4940111830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.899+0000 7f493f400700 1 -- 172.21.15.131:0/2520384579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0095f0 con 0x7f49401166f0 2024-05-10T05:38:36.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.899+0000 7f493ea00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4940075140 0x7f4940111230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:38:36.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.900+0000 7f493f400700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49401166f0 0x7f4940116b00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4938007ee0 tx=0x7f493800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:36.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.900+0000 7f4944a00700 1 -- 172.21.15.131:0/2520384579 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938017070 con 0x7f49401166f0 2024-05-10T05:38:36.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.900+0000 7f4944a00700 1 -- 172.21.15.131:0/2520384579 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f493800f040 con 0x7f49401166f0 2024-05-10T05:38:36.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.900+0000 7f4944a00700 1 -- 172.21.15.131:0/2520384579 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938005740 con 0x7f49401166f0 2024-05-10T05:38:36.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.900+0000 7f4945e00700 1 -- 172.21.15.131:0/2520384579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49401177f0 con 0x7f49401166f0 2024-05-10T05:38:36.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.900+0000 7f4945e00700 1 -- 172.21.15.131:0/2520384579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49401afab0 con 0x7f49401166f0 2024-05-10T05:38:36.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.902+0000 7f4945e00700 1 -- 172.21.15.131:0/2520384579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4940105180 con 0x7f49401166f0 2024-05-10T05:38:36.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.905+0000 7f4944a00700 1 -- 172.21.15.131:0/2520384579 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f49380058a0 con 0x7f49401166f0 2024-05-10T05:38:36.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.908+0000 7f4944a00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49240a11f0 0x7f49240a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:36.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.909+0000 7f493ea00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49240a11f0 0x7f49240a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.909+0000 7f4944a00700 1 -- 172.21.15.131:0/2520384579 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f49380cc110 con 0x7f49401166f0 2024-05-10T05:38:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.910+0000 7f493ea00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49240a11f0 0x7f49240a3700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f49401122d0 tx=0x7f4930008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:36.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:36.915+0000 7f4944a00700 1 -- 172.21.15.131:0/2520384579 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f49380958a0 con 0x7f49401166f0 2024-05-10T05:38:37.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.486+0000 7f4945e00700 1 -- 172.21.15.131:0/2520384579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4940101a20 con 0x7f49401166f0 2024-05-10T05:38:37.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.491+0000 7f4944a00700 1 -- 172.21.15.131:0/2520384579 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f4938095020 con 0x7f49401166f0 2024-05-10T05:38:37.493 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:38:37.494 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:38:37.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.499+0000 7f4945e00700 1 -- 172.21.15.131:0/2520384579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49240a11f0 msgr2=0x7f49240a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:37.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.499+0000 7f4945e00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49240a11f0 0x7f49240a3700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f49401122d0 tx=0x7f4930008040 comp rx=0 tx=0).stop 2024-05-10T05:38:37.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.499+0000 7f4945e00700 1 -- 172.21.15.131:0/2520384579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49401166f0 msgr2=0x7f4940116b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:37.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.499+0000 7f4945e00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49401166f0 0x7f4940116b00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4938007ee0 tx=0x7f493800d3b0 comp rx=0 tx=0).stop 2024-05-10T05:38:37.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.499+0000 7f4945e00700 1 -- 172.21.15.131:0/2520384579 shutdown_connections 2024-05-10T05:38:37.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.499+0000 7f4945e00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49240a11f0 0x7f49240a3700 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f49401122d0 tx=0x7f4930008040 comp rx=0 tx=0).stop 2024-05-10T05:38:37.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.499+0000 7f4945e00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49401166f0 0x7f4940116b00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:37.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.499+0000 7f4945e00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4940075140 0x7f4940111230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:37.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.499+0000 7f4945e00700 1 --2- 172.21.15.131:0/2520384579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4940109ac0 0x7f4940111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:37.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.499+0000 7f4945e00700 1 -- 172.21.15.131:0/2520384579 >> 172.21.15.131:0/2520384579 conn(0x7f49400fd1d0 msgr2=0x7f49401022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:37.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.500+0000 7f4945e00700 1 -- 172.21.15.131:0/2520384579 shutdown_connections 2024-05-10T05:38:37.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:37.500+0000 7f4945e00700 1 -- 172.21.15.131:0/2520384579 wait complete. 2024-05-10T05:38:37.503 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:38:37.555 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:38:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.295+0000 7fa993600700 1 -- 172.21.15.131:0/3128157040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c073b40 msgr2=0x7fa98c074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.295+0000 7fa993600700 1 --2- 172.21.15.131:0/3128157040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c073b40 0x7fa98c074010 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa978009940 tx=0x7fa978009c50 comp rx=0 tx=0).stop 2024-05-10T05:38:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.298+0000 7fa993600700 1 -- 172.21.15.131:0/3128157040 shutdown_connections 2024-05-10T05:38:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.298+0000 7fa993600700 1 --2- 172.21.15.131:0/3128157040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa98c10e220 0x7fa98c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.298+0000 7fa993600700 1 --2- 172.21.15.131:0/3128157040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c075140 0x7fa98c073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.298+0000 7fa993600700 1 --2- 172.21.15.131:0/3128157040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c073b40 0x7fa98c074010 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.298+0000 7fa993600700 1 -- 172.21.15.131:0/3128157040 >> 172.21.15.131:0/3128157040 conn(0x7fa98c0fd1d0 msgr2=0x7fa98c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.299+0000 7fa993600700 1 -- 172.21.15.131:0/3128157040 shutdown_connections 2024-05-10T05:38:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.300+0000 7fa993600700 1 -- 172.21.15.131:0/3128157040 wait complete. 2024-05-10T05:38:38.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.301+0000 7fa993600700 1 Processor -- start 2024-05-10T05:38:38.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.306+0000 7fa993600700 1 -- start start 2024-05-10T05:38:38.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.306+0000 7fa993600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa98c073b40 0x7fa98c105c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.306+0000 7fa993600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c075140 0x7fa98c1061b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.306+0000 7fa993600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c10e220 0x7fa98c10cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.306+0000 7fa993600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c10d3a0 con 0x7fa98c075140 2024-05-10T05:38:38.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.306+0000 7fa993600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c10d510 con 0x7fa98c10e220 2024-05-10T05:38:38.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.306+0000 7fa993600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c10d6d0 con 0x7fa98c073b40 2024-05-10T05:38:38.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.307+0000 7fa990c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa98c073b40 0x7fa98c105c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.307+0000 7fa991600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c10e220 0x7fa98c10cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.307+0000 7fa991600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c10e220 0x7fa98c10cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39182/0 (socket says 172.21.15.131:39182) 2024-05-10T05:38:38.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.307+0000 7fa990c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa98c073b40 0x7fa98c105c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37734/0 (socket says 172.21.15.131:37734) 2024-05-10T05:38:38.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.307+0000 7fa991600700 1 -- 172.21.15.131:0/4152132162 learned_addr learned my addr 172.21.15.131:0/4152132162 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:38.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.307+0000 7fa98be00700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c075140 0x7fa98c1061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.308+0000 7fa98be00700 1 -- 172.21.15.131:0/4152132162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa98c073b40 msgr2=0x7fa98c105c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:38.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.308+0000 7fa98be00700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa98c073b40 0x7fa98c105c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.308+0000 7fa98be00700 1 -- 172.21.15.131:0/4152132162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c10e220 msgr2=0x7fa98c10cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:38.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.308+0000 7fa98be00700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c10e220 0x7fa98c10cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.308+0000 7fa98be00700 1 -- 172.21.15.131:0/4152132162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa978009520 con 0x7fa98c075140 2024-05-10T05:38:38.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.310+0000 7fa98be00700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c075140 0x7fa98c1061b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa978000c00 tx=0x7fa9780055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:38.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.314+0000 7fa989600700 1 -- 172.21.15.131:0/4152132162 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97800bbe0 con 0x7fa98c075140 2024-05-10T05:38:38.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.318+0000 7fa989600700 1 -- 172.21.15.131:0/4152132162 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa97800bd40 con 0x7fa98c075140 2024-05-10T05:38:38.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.318+0000 7fa989600700 1 -- 172.21.15.131:0/4152132162 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978017840 con 0x7fa98c075140 2024-05-10T05:38:38.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.318+0000 7fa993600700 1 -- 172.21.15.131:0/4152132162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c10d950 con 0x7fa98c075140 2024-05-10T05:38:38.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.318+0000 7fa993600700 1 -- 172.21.15.131:0/4152132162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c10de30 con 0x7fa98c075140 2024-05-10T05:38:38.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.339+0000 7fa989600700 1 -- 172.21.15.131:0/4152132162 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa9780179a0 con 0x7fa98c075140 2024-05-10T05:38:38.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.341+0000 7fa9ab56bb80 1 Processor -- start 2024-05-10T05:38:38.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.342+0000 7fa9ab56bb80 1 -- start start 2024-05-10T05:38:38.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.342+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cec6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.342+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 0x55a18cec7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.342+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7ec0 0x55a18cece7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.343+0000 7fa991600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cec6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.343+0000 7fa991600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cec6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37750/0 (socket says 172.21.15.131:37750) 2024-05-10T05:38:38.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.343+0000 7fa98be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7ec0 0x55a18cece7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.343+0000 7fa98be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7ec0 0x55a18cece7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39192/0 (socket says 172.21.15.131:39192) 2024-05-10T05:38:38.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.343+0000 7fa990c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 0x55a18cec7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.344+0000 7fa990c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 0x55a18cec7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42868/0 (socket says 172.21.15.131:42868) 2024-05-10T05:38:38.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.344+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2115454622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cded150 con 0x55a18cec7570 2024-05-10T05:38:38.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.344+0000 7fa991600700 1 -- 172.21.15.131:0/2115454622 learned_addr learned my addr 172.21.15.131:0/2115454622 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:38.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.344+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2115454622 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cdeab10 con 0x55a18cec7ec0 2024-05-10T05:38:38.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.344+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2115454622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cde6400 con 0x55a18cec66a0 2024-05-10T05:38:38.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.353+0000 7fa990c00700 1 -- 172.21.15.131:0/2115454622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 msgr2=0x55a18cec6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:38.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.353+0000 7fa990c00700 1 --2- 172.21.15.131:0/2115454622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cec6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.353+0000 7fa990c00700 1 -- 172.21.15.131:0/2115454622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7ec0 msgr2=0x55a18cece7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:38.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.353+0000 7fa990c00700 1 --2- 172.21.15.131:0/2115454622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7ec0 0x55a18cece7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.353+0000 7fa990c00700 1 -- 172.21.15.131:0/2115454622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a18cc673b0 con 0x55a18cec7570 2024-05-10T05:38:38.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.354+0000 7fa990c00700 1 --2- 172.21.15.131:0/2115454622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 0x55a18cec7980 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa98c074e30 tx=0x7fa98000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240af64407322de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:38.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.355+0000 7fa989600700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa9740a1390 0x7fa9740a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.355+0000 7fa97dc00700 1 -- 172.21.15.131:0/2115454622 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980003940 con 0x55a18cec7570 2024-05-10T05:38:38.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.356+0000 7fa991600700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa9740a1390 0x7fa9740a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.356+0000 7fa97dc00700 1 -- 172.21.15.131:0/2115454622 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa98001a470 con 0x55a18cec7570 2024-05-10T05:38:38.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.356+0000 7fa97dc00700 1 -- 172.21.15.131:0/2115454622 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9800116a0 con 0x55a18cec7570 2024-05-10T05:38:38.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.359+0000 7fa989600700 1 -- 172.21.15.131:0/4152132162 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa9780cdb30 con 0x7fa98c075140 2024-05-10T05:38:38.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.359+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2115454622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 msgr2=0x55a18cec7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:38.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.360+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/2115454622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 0x55a18cec7980 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa98c074e30 tx=0x7fa98000ba10 comp rx=0 tx=0).stop 2024-05-10T05:38:38.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.360+0000 7fa991600700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa9740a1390 0x7fa9740a38a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55a18cec6380 tx=0x7fa9840098b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:38.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.362+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2115454622 shutdown_connections 2024-05-10T05:38:38.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.362+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/2115454622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7ec0 0x55a18cece7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.362+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/2115454622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cec6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.362+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/2115454622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 0x55a18cec7980 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.362+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2115454622 >> 172.21.15.131:0/2115454622 conn(0x55a18ce28a60 msgr2=0x55a18cec0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:38.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.363+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2115454622 shutdown_connections 2024-05-10T05:38:38.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.363+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2115454622 wait complete. 2024-05-10T05:38:38.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.364+0000 7fa9ab56bb80 1 -- start start 2024-05-10T05:38:38.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.364+0000 7fa9ab56bb80 1 Processor -- start 2024-05-10T05:38:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cef24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7570 0x55a18cef2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7ec0 0x55a18cef9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa9ab56bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cd82560 con 0x55a18cec7ec0 2024-05-10T05:38:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa9ab56bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cd82200 con 0x55a18cec7570 2024-05-10T05:38:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa9ab56bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cc696a0 con 0x55a18cec66a0 2024-05-10T05:38:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa9ab56bb80 10 client.0 _ll_register_callbacks cb 0x55a18cd8bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa990c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cef24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa990c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cef24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37760/0 (socket says 172.21.15.131:37760) 2024-05-10T05:38:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa990c00700 1 -- 172.21.15.131:0/4237640409 learned_addr learned my addr 172.21.15.131:0/4237640409 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.367+0000 7fa98be00700 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7570 0x55a18cef2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.368+0000 7fa990c00700 1 -- 172.21.15.131:0/4237640409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7570 msgr2=0x55a18cef2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.368+0000 7fa990c00700 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7570 0x55a18cef2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.368+0000 7fa990c00700 1 -- 172.21.15.131:0/4237640409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7ec0 msgr2=0x55a18cef9990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:38:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.368+0000 7fa990c00700 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7ec0 0x55a18cef9990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.368+0000 7fa990c00700 1 -- 172.21.15.131:0/4237640409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa980004250 con 0x55a18cec66a0 2024-05-10T05:38:38.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.368+0000 7fa990c00700 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cef24f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa980005e40 tx=0x7fa980005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2bde0bc54e7371e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:38.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.369+0000 7fa972000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:38:38.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.369+0000 7fa972000700 1 -- 172.21.15.131:0/4237640409 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98001f070 con 0x55a18cec66a0 2024-05-10T05:38:38.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.369+0000 7fa972000700 1 -- 172.21.15.131:0/4237640409 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa980003b10 con 0x55a18cec66a0 2024-05-10T05:38:38.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.369+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/4237640409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a18cd3f270 con 0x55a18cec66a0 2024-05-10T05:38:38.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.370+0000 7fa972000700 1 -- 172.21.15.131:0/4237640409 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa980003d10 con 0x55a18cec66a0 2024-05-10T05:38:38.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.370+0000 7fa9ab56bb80 10 client.26308 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:38.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.370+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/4237640409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a18cc13b60 con 0x55a18cec66a0 2024-05-10T05:38:38.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.370+0000 7fa972000700 1 -- 172.21.15.131:0/4237640409 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a18cd3f270 con 0x55a18cec66a0 2024-05-10T05:38:38.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.372+0000 7fa972000700 1 -- 172.21.15.131:0/4237640409 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa980023400 con 0x55a18cec66a0 2024-05-10T05:38:38.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.372+0000 7fa9ab56bb80 10 client.26308 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:38.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.372+0000 7fa9ab56bb80 10 client.26308 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:38:38.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.372+0000 7fa9ab56bb80 20 client.26308 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:38.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.372+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55a18cefdf60 0x55a18cf003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.372+0000 7fa9ab56bb80 4 client.26308 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:38.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.372+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/4237640409 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a18ce95780 con 0x55a18cefdf60 2024-05-10T05:38:38.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.373+0000 7fa98be00700 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55a18cefdf60 0x55a18cf003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.373+0000 7fa98be00700 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55a18cefdf60 0x55a18cf003b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=29d07ccc46212528 server_cookie=ebeef862fc3239c3 in_seq=0 out_seq=0 2024-05-10T05:38:38.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.373+0000 7fa972000700 10 client.26308 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:38:38.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.374+0000 7fa972000700 1 -- 172.21.15.131:0/4237640409 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a18ce95780 con 0x55a18cefdf60 2024-05-10T05:38:38.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.374+0000 7fa972000700 10 client.26308 handle_command_reply: tid=0 2024-05-10T05:38:38.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.374+0000 7fa9ab56bb80 1 client.26308 shutdown 2024-05-10T05:38:38.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.375+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/4237640409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 msgr2=0x55a18cef24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:38.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.375+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cef24f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa980005e40 tx=0x7fa980005810 comp rx=0 tx=0).stop 2024-05-10T05:38:38.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.375+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/4237640409 shutdown_connections 2024-05-10T05:38:38.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.375+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55a18cefdf60 0x55a18cf003b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.375+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7ec0 0x55a18cef9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.375+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec66a0 0x55a18cef24f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.375+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/4237640409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec7570 0x55a18cef2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.375+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/4237640409 >> 172.21.15.131:0/4237640409 conn(0x55a18ce28a60 msgr2=0x55a18cec0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:38.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.375+0000 7fa972000700 0 client.26308 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:38:38.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.376+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/4237640409 shutdown_connections 2024-05-10T05:38:38.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.376+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/4237640409 wait complete. 2024-05-10T05:38:38.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.376+0000 7fa9ab56bb80 20 client.26308 trim_cache size 0 max 16384 2024-05-10T05:38:38.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.405+0000 7fa9ab56bb80 1 Processor -- start 2024-05-10T05:38:38.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.406+0000 7fa9ab56bb80 1 -- start start 2024-05-10T05:38:38.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.406+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18ce28a60 0x55a18cef7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.407+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec66a0 0x55a18cef7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.408+0000 7fa98be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec66a0 0x55a18cef7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.408+0000 7fa98be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec66a0 0x55a18cef7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39204/0 (socket says 172.21.15.131:39204) 2024-05-10T05:38:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.408+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec7570 0x55a18cecb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.408+0000 7fa9ab56bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cef33c0 con 0x55a18ce28a60 2024-05-10T05:38:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.408+0000 7fa9ab56bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cece770 con 0x55a18cec66a0 2024-05-10T05:38:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.408+0000 7fa9ab56bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cebd180 con 0x55a18cec7570 2024-05-10T05:38:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.408+0000 7fa98be00700 1 -- 172.21.15.131:0/1168864208 learned_addr learned my addr 172.21.15.131:0/1168864208 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.409+0000 7fa991600700 1 --2- 172.21.15.131:0/1168864208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec7570 0x55a18cecb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.409+0000 7fa991600700 1 -- 172.21.15.131:0/1168864208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec66a0 msgr2=0x55a18cef7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.409+0000 7fa990c00700 1 --2- 172.21.15.131:0/1168864208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18ce28a60 0x55a18cef7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.409+0000 7fa991600700 1 --2- 172.21.15.131:0/1168864208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec66a0 0x55a18cef7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.409+0000 7fa991600700 1 -- 172.21.15.131:0/1168864208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18ce28a60 msgr2=0x55a18cef7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.409+0000 7fa991600700 1 --2- 172.21.15.131:0/1168864208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18ce28a60 0x55a18cef7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.409+0000 7fa991600700 1 -- 172.21.15.131:0/1168864208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a18cdac560 con 0x55a18cec7570 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.409+0000 7fa990c00700 1 --2- 172.21.15.131:0/1168864208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18ce28a60 0x55a18cef7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.410+0000 7fa991600700 1 --2- 172.21.15.131:0/1168864208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec7570 0x55a18cecb060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa984009f10 tx=0x7fa984013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9df3883cc71f0632 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.410+0000 7fa97e600700 1 -- 172.21.15.131:0/1168864208 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984017de0 con 0x55a18cec7570 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.410+0000 7fa97e600700 1 -- 172.21.15.131:0/1168864208 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9840228c0 con 0x55a18cec7570 2024-05-10T05:38:38.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.411+0000 7fa97e600700 1 -- 172.21.15.131:0/1168864208 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984017de0 con 0x55a18cec7570 2024-05-10T05:38:38.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.412+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/1168864208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec7570 msgr2=0x55a18cecb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:38.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.412+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/1168864208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec7570 0x55a18cecb060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa984009f10 tx=0x7fa984013000 comp rx=0 tx=0).stop 2024-05-10T05:38:38.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.412+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/1168864208 shutdown_connections 2024-05-10T05:38:38.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.412+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/1168864208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18ce28a60 0x55a18cef7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.412+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/1168864208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18cec7570 0x55a18cecb060 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.412+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/1168864208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec66a0 0x55a18cef7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.412+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/1168864208 >> 172.21.15.131:0/1168864208 conn(0x55a18cebd450 msgr2=0x55a18cef0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:38.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.417+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/1168864208 shutdown_connections 2024-05-10T05:38:38.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.417+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/1168864208 wait complete. 2024-05-10T05:38:38.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.418+0000 7fa9ab56bb80 1 -- start start 2024-05-10T05:38:38.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.418+0000 7fa9ab56bb80 1 Processor -- start 2024-05-10T05:38:38.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.448+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18ce28a60 0x55a18cef74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.449+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec66a0 0x55a18cef79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.449+0000 7fa9ab56bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 0x55a18cf52940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.449+0000 7fa9ab56bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cf52e80 con 0x55a18cec7570 2024-05-10T05:38:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.449+0000 7fa9ab56bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cf52fc0 con 0x55a18cec66a0 2024-05-10T05:38:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.449+0000 7fa9ab56bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18cf53150 con 0x55a18ce28a60 2024-05-10T05:38:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.450+0000 7fa9ab56bb80 10 client.0 _ll_register_callbacks cb 0x55a18cd8bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:38.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.451+0000 7fa990c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18ce28a60 0x55a18cef74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.451+0000 7fa990c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18ce28a60 0x55a18cef74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37780/0 (socket says 172.21.15.131:37780) 2024-05-10T05:38:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.451+0000 7fa990c00700 1 -- 172.21.15.131:0/2478201504 learned_addr learned my addr 172.21.15.131:0/2478201504 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.452+0000 7fa991600700 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 0x55a18cf52940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.454+0000 7fa990c00700 1 -- 172.21.15.131:0/2478201504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec66a0 msgr2=0x55a18cef79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:38.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.454+0000 7fa990c00700 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec66a0 0x55a18cef79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.454+0000 7fa990c00700 1 -- 172.21.15.131:0/2478201504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 msgr2=0x55a18cf52940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:38.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.454+0000 7fa990c00700 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 0x55a18cf52940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.454+0000 7fa990c00700 1 -- 172.21.15.131:0/2478201504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa984011040 con 0x55a18ce28a60 2024-05-10T05:38:38.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:38 smithi131 ceph-mon[37264]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 246 MiB/s wr, 77 op/s 2024-05-10T05:38:38.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:38 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2520384579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:38:38.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.461+0000 7fa990c00700 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18ce28a60 0x55a18cef74b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa980000c00 tx=0x7fa980003dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=476f54859e700ef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:38.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.461+0000 7fa972000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:38:38.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.463+0000 7fa972000700 1 -- 172.21.15.131:0/2478201504 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98001f040 con 0x55a18ce28a60 2024-05-10T05:38:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.463+0000 7fa972000700 1 -- 172.21.15.131:0/2478201504 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa980023a80 con 0x55a18ce28a60 2024-05-10T05:38:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.463+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2478201504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa98001a470 con 0x55a18ce28a60 2024-05-10T05:38:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.464+0000 7fa972000700 1 -- 172.21.15.131:0/2478201504 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa980005040 con 0x55a18ce28a60 2024-05-10T05:38:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.464+0000 7fa9ab56bb80 10 client.26314 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.464+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2478201504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a18cc6bcd0 con 0x55a18ce28a60 2024-05-10T05:38:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.464+0000 7fa972000700 1 -- 172.21.15.131:0/2478201504 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa98001a470 con 0x55a18ce28a60 2024-05-10T05:38:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.466+0000 7fa972000700 1 -- 172.21.15.131:0/2478201504 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa98003d020 con 0x55a18ce28a60 2024-05-10T05:38:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.466+0000 7fa9ab56bb80 10 client.26314 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.466+0000 7fa9ab56bb80 10 client.26314 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:38:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.466+0000 7fa9ab56bb80 20 client.26314 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.467+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55a18cec7ec0 0x55a18cf56880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.467+0000 7fa9ab56bb80 4 client.26314 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:38:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.467+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2478201504 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a18cddeec0 con 0x55a18cec7ec0 2024-05-10T05:38:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.468+0000 7fa98be00700 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55a18cec7ec0 0x55a18cf56880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:38.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.469+0000 7fa98be00700 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55a18cec7ec0 0x55a18cf56880 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3bb6e3e6d97dd653 server_cookie=41700f2242240f8c in_seq=0 out_seq=0 2024-05-10T05:38:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.470+0000 7fa972000700 10 client.26314 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:38:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.474+0000 7fa972000700 1 -- 172.21.15.131:0/2478201504 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a18cddeec0 con 0x55a18cec7ec0 2024-05-10T05:38:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.474+0000 7fa972000700 10 client.26314 handle_command_reply: tid=0 2024-05-10T05:38:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.476+0000 7fa9ab56bb80 1 client.26314 shutdown 2024-05-10T05:38:38.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.476+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2478201504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18ce28a60 msgr2=0x55a18cef74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.476+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18ce28a60 0x55a18cef74b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa980000c00 tx=0x7fa980003dc0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.477+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2478201504 shutdown_connections 2024-05-10T05:38:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.477+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55a18cec7ec0 0x55a18cf56880 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.477+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a18cec7570 0x55a18cf52940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.477+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a18ce28a60 0x55a18cef74b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.477+0000 7fa972000700 0 client.26314 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:38:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.477+0000 7fa9ab56bb80 1 --2- 172.21.15.131:0/2478201504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a18cec66a0 0x55a18cef79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.477+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2478201504 >> 172.21.15.131:0/2478201504 conn(0x55a18cebd450 msgr2=0x55a18cef0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:38.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.477+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2478201504 shutdown_connections 2024-05-10T05:38:38.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.478+0000 7fa9ab56bb80 1 -- 172.21.15.131:0/2478201504 wait complete. 2024-05-10T05:38:38.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.479+0000 7fa9ab56bb80 20 client.26314 trim_cache size 0 max 16384 2024-05-10T05:38:38.484 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:38:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.496+0000 7fa993600700 1 -- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa9740a1390 msgr2=0x7fa9740a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.496+0000 7fa993600700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa9740a1390 0x7fa9740a38a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55a18cec6380 tx=0x7fa9840098b0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.496+0000 7fa993600700 1 -- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c075140 msgr2=0x7fa98c1061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.496+0000 7fa993600700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c075140 0x7fa98c1061b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa978000c00 tx=0x7fa9780055e0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.497+0000 7fa993600700 1 -- 172.21.15.131:0/4152132162 shutdown_connections 2024-05-10T05:38:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.497+0000 7fa993600700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa9740a1390 0x7fa9740a38a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.497+0000 7fa993600700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c10e220 0x7fa98c10cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.497+0000 7fa993600700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa98c073b40 0x7fa98c105c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.497+0000 7fa993600700 1 --2- 172.21.15.131:0/4152132162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c075140 0x7fa98c1061b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:38.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.497+0000 7fa993600700 1 -- 172.21.15.131:0/4152132162 >> 172.21.15.131:0/4152132162 conn(0x7fa98c0fd1d0 msgr2=0x7fa98c1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:38.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.498+0000 7fa993600700 1 -- 172.21.15.131:0/4152132162 shutdown_connections 2024-05-10T05:38:38.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:38.498+0000 7fa993600700 1 -- 172.21.15.131:0/4152132162 wait complete. 2024-05-10T05:38:38.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:38 smithi157 ceph-mon[78427]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 246 MiB/s wr, 77 op/s 2024-05-10T05:38:38.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:38 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2520384579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:38:38.585 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:38:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:38 smithi142 ceph-mon[77667]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 246 MiB/s wr, 77 op/s 2024-05-10T05:38:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:38 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2520384579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:38:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.267+0000 7f1273600700 1 -- 172.21.15.131:0/893333414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c105010 msgr2=0x7f126c107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:39.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.268+0000 7f1273600700 1 --2- 172.21.15.131:0/893333414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c105010 0x7f126c107430 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f125c0098f0 tx=0x7f125c009c00 comp rx=0 tx=0).stop 2024-05-10T05:38:39.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.268+0000 7f1273600700 1 -- 172.21.15.131:0/893333414 shutdown_connections 2024-05-10T05:38:39.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.268+0000 7f1273600700 1 --2- 172.21.15.131:0/893333414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c107970 0x7f126c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.268+0000 7f1273600700 1 --2- 172.21.15.131:0/893333414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c1026b0 0x7f126c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.268+0000 7f1273600700 1 --2- 172.21.15.131:0/893333414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c105010 0x7f126c107430 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.268+0000 7f1273600700 1 -- 172.21.15.131:0/893333414 >> 172.21.15.131:0/893333414 conn(0x7f126c0fc0d0 msgr2=0x7f126c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:39.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.268+0000 7f1273600700 1 -- 172.21.15.131:0/893333414 shutdown_connections 2024-05-10T05:38:39.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.269+0000 7f1273600700 1 -- 172.21.15.131:0/893333414 wait complete. 2024-05-10T05:38:39.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.275+0000 7f1273600700 1 Processor -- start 2024-05-10T05:38:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.276+0000 7f1273600700 1 -- start start 2024-05-10T05:38:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.279+0000 7f1273600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c1026b0 0x7f126c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.279+0000 7f1273600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c105010 0x7f126c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.279+0000 7f1273600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c107970 0x7f126c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.279+0000 7f1273600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c1a1cb0 con 0x7f126c1026b0 2024-05-10T05:38:39.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.279+0000 7f1271600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c107970 0x7f126c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.279+0000 7f1271600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c107970 0x7f126c1a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37786/0 (socket says 172.21.15.131:37786) 2024-05-10T05:38:39.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.279+0000 7f1271600700 1 -- 172.21.15.131:0/3343644476 learned_addr learned my addr 172.21.15.131:0/3343644476 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:39.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.280+0000 7f1273600700 1 -- 172.21.15.131:0/3343644476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c1a1e20 con 0x7f126c105010 2024-05-10T05:38:39.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.280+0000 7f1273600700 1 -- 172.21.15.131:0/3343644476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c1a1fe0 con 0x7f126c107970 2024-05-10T05:38:39.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.280+0000 7f126be00700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c105010 0x7f126c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.281+0000 7f1270c00700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c1026b0 0x7f126c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.281+0000 7f1271600700 1 -- 172.21.15.131:0/3343644476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c105010 msgr2=0x7f126c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:39.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.281+0000 7f1271600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c105010 0x7f126c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.281+0000 7f1271600700 1 -- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c1026b0 msgr2=0x7f126c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:39.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.281+0000 7f1271600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c1026b0 0x7f126c19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.281+0000 7f1271600700 1 -- 172.21.15.131:0/3343644476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125c009520 con 0x7f126c107970 2024-05-10T05:38:39.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.281+0000 7f1270c00700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c1026b0 0x7f126c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:39.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.281+0000 7f1271600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c107970 0x7f126c1a16a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f126400ef90 tx=0x7f126400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:39.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.281+0000 7f1269600700 1 -- 172.21.15.131:0/3343644476 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1264018070 con 0x7f126c107970 2024-05-10T05:38:39.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.282+0000 7f1269600700 1 -- 172.21.15.131:0/3343644476 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f126400f040 con 0x7f126c107970 2024-05-10T05:38:39.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.282+0000 7f1269600700 1 -- 172.21.15.131:0/3343644476 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1264007960 con 0x7f126c107970 2024-05-10T05:38:39.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.283+0000 7f1273600700 1 -- 172.21.15.131:0/3343644476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c1a2260 con 0x7f126c107970 2024-05-10T05:38:39.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.283+0000 7f1273600700 1 -- 172.21.15.131:0/3343644476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c1a2720 con 0x7f126c107970 2024-05-10T05:38:39.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.295+0000 7f128b5ddb80 1 Processor -- start 2024-05-10T05:38:39.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.296+0000 7f1269600700 1 -- 172.21.15.131:0/3343644476 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1264007b20 con 0x7f126c107970 2024-05-10T05:38:39.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.298+0000 7f128b5ddb80 1 -- start start 2024-05-10T05:38:39.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.298+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 0x564fa082d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.298+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.298+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa0834fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.298+0000 7f128b5ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa074b150 con 0x564fa082ceb0 2024-05-10T05:38:39.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.298+0000 7f128b5ddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa0748b10 con 0x564fa082dd80 2024-05-10T05:38:39.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.299+0000 7f128b5ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa0744400 con 0x564fa082e6d0 2024-05-10T05:38:39.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.299+0000 7f1269600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12500a1240 0x7f12500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.299+0000 7f1270c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.299+0000 7f1270c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39252/0 (socket says 172.21.15.131:39252) 2024-05-10T05:38:39.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.299+0000 7f1270c00700 1 -- 172.21.15.131:0/4184792818 learned_addr learned my addr 172.21.15.131:0/4184792818 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:39.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.300+0000 7f1269600700 1 -- 172.21.15.131:0/3343644476 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f12640cc680 con 0x7f126c107970 2024-05-10T05:38:39.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.300+0000 7f126be00700 1 --2- 172.21.15.131:0/4184792818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa0834fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.300+0000 7f126be00700 1 --2- 172.21.15.131:0/4184792818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 0x564fa082d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.300+0000 7f1270c00700 1 -- 172.21.15.131:0/4184792818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 msgr2=0x564fa0834fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:39.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.300+0000 7f1270c00700 1 --2- 172.21.15.131:0/4184792818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa0834fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.300+0000 7f1270c00700 1 -- 172.21.15.131:0/4184792818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 msgr2=0x564fa082d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:39.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.300+0000 7f1270c00700 1 --2- 172.21.15.131:0/4184792818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 0x564fa082d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.300+0000 7f1270c00700 1 -- 172.21.15.131:0/4184792818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fa05c53b0 con 0x564fa082dd80 2024-05-10T05:38:39.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.301+0000 7f1271600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12500a1240 0x7f12500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.301+0000 7f1270c00700 1 --2- 172.21.15.131:0/4184792818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082e190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f125800bbe0 tx=0x7f125800bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dccb62ad664cc19f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:39.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.301+0000 7f1261c00700 1 -- 172.21.15.131:0/4184792818 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125800dd00 con 0x564fa082dd80 2024-05-10T05:38:39.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.302+0000 7f1261c00700 1 -- 172.21.15.131:0/4184792818 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f125800b690 con 0x564fa082dd80 2024-05-10T05:38:39.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.302+0000 7f1261c00700 1 -- 172.21.15.131:0/4184792818 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125800f7a0 con 0x564fa082dd80 2024-05-10T05:38:39.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.302+0000 7f1271600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12500a1240 0x7f12500a3750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1264096540 tx=0x7f126400c980 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:39.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.303+0000 7f128b5ddb80 1 -- 172.21.15.131:0/4184792818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 msgr2=0x564fa082e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:39.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.303+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/4184792818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082e190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f125800bbe0 tx=0x7f125800bef0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.304+0000 7f128b5ddb80 1 -- 172.21.15.131:0/4184792818 shutdown_connections 2024-05-10T05:38:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.304+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/4184792818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 0x564fa082d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.304+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/4184792818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa0834fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.304+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/4184792818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082e190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.304+0000 7f128b5ddb80 1 -- 172.21.15.131:0/4184792818 >> 172.21.15.131:0/4184792818 conn(0x564fa0786a60 msgr2=0x564fa081efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.304+0000 7f128b5ddb80 1 -- 172.21.15.131:0/4184792818 shutdown_connections 2024-05-10T05:38:39.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.304+0000 7f128b5ddb80 1 -- 172.21.15.131:0/4184792818 wait complete. 2024-05-10T05:38:39.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.305+0000 7f128b5ddb80 1 -- start start 2024-05-10T05:38:39.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.306+0000 7f128b5ddb80 1 Processor -- start 2024-05-10T05:38:39.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.306+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 0x564fa0825cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.306+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa0826230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.306+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa082b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.306+0000 7f128b5ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa06e0560 con 0x564fa082ceb0 2024-05-10T05:38:39.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.306+0000 7f128b5ddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa06e0200 con 0x564fa082dd80 2024-05-10T05:38:39.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.306+0000 7f128b5ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa05c76a0 con 0x564fa082e6d0 2024-05-10T05:38:39.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.307+0000 7f128b5ddb80 10 client.0 _ll_register_callbacks cb 0x564fa06e9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:39.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.307+0000 7f1270c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa0826230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.307+0000 7f126be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 0x564fa0825cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.308+0000 7f1270c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa0826230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39262/0 (socket says 172.21.15.131:39262) 2024-05-10T05:38:39.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.308+0000 7f1270c00700 1 -- 172.21.15.131:0/2690136670 learned_addr learned my addr 172.21.15.131:0/2690136670 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:39.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.308+0000 7f126be00700 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa082b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.310+0000 7f126be00700 1 -- 172.21.15.131:0/2690136670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 msgr2=0x564fa082b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:39.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.310+0000 7f126be00700 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa082b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.310+0000 7f126be00700 1 -- 172.21.15.131:0/2690136670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 msgr2=0x564fa0826230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.310+0000 7f126be00700 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa0826230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.310+0000 7f126be00700 1 -- 172.21.15.131:0/2690136670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125800b810 con 0x564fa082ceb0 2024-05-10T05:38:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.312+0000 7f1256000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:38:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.312+0000 7f126be00700 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 0x564fa0825cf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f126c19b270 tx=0x7f125c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a623a176420349a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.313+0000 7f1256000700 1 -- 172.21.15.131:0/2690136670 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c01c070 con 0x564fa082ceb0 2024-05-10T05:38:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.313+0000 7f1256000700 1 -- 172.21.15.131:0/2690136670 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f125c005030 con 0x564fa082ceb0 2024-05-10T05:38:39.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.318+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2690136670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fa07f3780 con 0x564fa082ceb0 2024-05-10T05:38:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.321+0000 7f1256000700 1 -- 172.21.15.131:0/2690136670 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f125c01c070 con 0x564fa082ceb0 2024-05-10T05:38:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.321+0000 7f128b5ddb80 10 client.16164 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.321+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2690136670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564fa0571b60 con 0x564fa082ceb0 2024-05-10T05:38:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.322+0000 7f1256000700 1 -- 172.21.15.131:0/2690136670 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564fa07f3780 con 0x564fa082ceb0 2024-05-10T05:38:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.322+0000 7f1256000700 1 -- 172.21.15.131:0/2690136670 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f125c015020 con 0x564fa082ceb0 2024-05-10T05:38:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.323+0000 7f128b5ddb80 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.323+0000 7f128b5ddb80 10 client.16164 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:38:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.323+0000 7f128b5ddb80 20 client.16164 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.323+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564fa084b330 0x564fa084d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.323+0000 7f128b5ddb80 4 client.16164 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.323+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2690136670 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fa073cec0 con 0x564fa084b330 2024-05-10T05:38:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.324+0000 7f1270c00700 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564fa084b330 0x564fa084d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.325+0000 7f1270c00700 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564fa084b330 0x564fa084d780 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed46f067e38bc3e server_cookie=254d3c59ce35e7ee in_seq=0 out_seq=0 2024-05-10T05:38:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.325+0000 7f1256000700 10 client.16164 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:38:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.326+0000 7f1256000700 1 -- 172.21.15.131:0/2690136670 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564fa073cec0 con 0x564fa084b330 2024-05-10T05:38:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.326+0000 7f1256000700 10 client.16164 handle_command_reply: tid=0 2024-05-10T05:38:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.327+0000 7f128b5ddb80 1 client.16164 shutdown 2024-05-10T05:38:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.327+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2690136670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 msgr2=0x564fa0825cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.327+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 0x564fa0825cf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f126c19b270 tx=0x7f125c009c00 comp rx=0 tx=0).stop 2024-05-10T05:38:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.327+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2690136670 shutdown_connections 2024-05-10T05:38:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.327+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564fa084b330 0x564fa084d780 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.327+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa082ceb0 0x564fa0825cf0 secure :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f126c19b270 tx=0x7f125c009c00 comp rx=0 tx=0).stop 2024-05-10T05:38:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.327+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa082b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.327+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2690136670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa0826230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.327+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2690136670 >> 172.21.15.131:0/2690136670 conn(0x564fa0786a60 msgr2=0x564fa081efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.328+0000 7f1256000700 0 client.16164 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:38:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.330+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2690136670 shutdown_connections 2024-05-10T05:38:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.330+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2690136670 wait complete. 2024-05-10T05:38:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.331+0000 7f128b5ddb80 20 client.16164 trim_cache size 0 max 16384 2024-05-10T05:38:39.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.359+0000 7f128b5ddb80 1 Processor -- start 2024-05-10T05:38:39.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.359+0000 7f128b5ddb80 1 -- start start 2024-05-10T05:38:39.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.360+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.360+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa081ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.360+0000 7f126be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.360+0000 7f126be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39270/0 (socket says 172.21.15.131:39270) 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.360+0000 7f126be00700 1 -- 172.21.15.131:0/2312051148 learned_addr learned my addr 172.21.15.131:0/2312051148 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.360+0000 7f1270c00700 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa081ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.360+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 0x564fa085b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.361+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2312051148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa08269b0 con 0x564fa081f1d0 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.361+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2312051148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa08316c0 con 0x564fa082dd80 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.361+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2312051148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa082bf10 con 0x564fa082e6d0 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.362+0000 7f126be00700 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 0x564fa085b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.362+0000 7f1270c00700 1 -- 172.21.15.131:0/2312051148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 msgr2=0x564fa082b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.362+0000 7f1270c00700 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.362+0000 7f1270c00700 1 -- 172.21.15.131:0/2312051148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 msgr2=0x564fa085b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.362+0000 7f1270c00700 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 0x564fa085b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.362+0000 7f1270c00700 1 -- 172.21.15.131:0/2312051148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fa070a560 con 0x564fa082e6d0 2024-05-10T05:38:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.362+0000 7f126be00700 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 0x564fa085b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.364+0000 7f1270c00700 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa081ec90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x564fa0826d60 tx=0x7f1258017ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1debf70ee5a277ae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.368+0000 7f1262600700 1 -- 172.21.15.131:0/2312051148 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258029070 con 0x564fa082e6d0 2024-05-10T05:38:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.368+0000 7f1262600700 1 -- 172.21.15.131:0/2312051148 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f125800b470 con 0x564fa082e6d0 2024-05-10T05:38:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.368+0000 7f1262600700 1 -- 172.21.15.131:0/2312051148 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125800f4b0 con 0x564fa082e6d0 2024-05-10T05:38:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.369+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2312051148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 msgr2=0x564fa081ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.369+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa081ec90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x564fa0826d60 tx=0x7f1258017ef0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.370+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2312051148 shutdown_connections 2024-05-10T05:38:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.370+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 0x564fa085b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.370+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082dd80 0x564fa082b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.370+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2312051148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082e6d0 0x564fa081ec90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.370+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2312051148 >> 172.21.15.131:0/2312051148 conn(0x564fa0786a60 msgr2=0x564fa0830fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.371+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2312051148 shutdown_connections 2024-05-10T05:38:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.371+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2312051148 wait complete. 2024-05-10T05:38:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.372+0000 7f128b5ddb80 1 -- start start 2024-05-10T05:38:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.372+0000 7f128b5ddb80 1 Processor -- start 2024-05-10T05:38:39.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.374+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 0x564fa085a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.374+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082dd80 0x564fa085ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.374+0000 7f128b5ddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082e6d0 0x564fa08a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.374+0000 7f128b5ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa085b470 con 0x564fa081f1d0 2024-05-10T05:38:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.374+0000 7f128b5ddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa08a07b0 con 0x564fa082e6d0 2024-05-10T05:38:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.374+0000 7f128b5ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa08a08f0 con 0x564fa082dd80 2024-05-10T05:38:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.374+0000 7f128b5ddb80 10 client.0 _ll_register_callbacks cb 0x564fa06e9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.375+0000 7f126be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082e6d0 0x564fa08a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.375+0000 7f126be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082e6d0 0x564fa08a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39272/0 (socket says 172.21.15.131:39272) 2024-05-10T05:38:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.375+0000 7f126be00700 1 -- 172.21.15.131:0/2897214186 learned_addr learned my addr 172.21.15.131:0/2897214186 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.375+0000 7f1270c00700 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082dd80 0x564fa085ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.375+0000 7f126be00700 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 0x564fa085a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.376+0000 7f126be00700 1 -- 172.21.15.131:0/2897214186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082dd80 msgr2=0x564fa085ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.376+0000 7f126be00700 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082dd80 0x564fa085ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.376+0000 7f126be00700 1 -- 172.21.15.131:0/2897214186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 msgr2=0x564fa085a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.376+0000 7f126be00700 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 0x564fa085a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.376+0000 7f126be00700 1 -- 172.21.15.131:0/2897214186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fa05c53b0 con 0x564fa082e6d0 2024-05-10T05:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.376+0000 7f126be00700 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082e6d0 0x564fa08a0270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b690 tx=0x7f125c009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4fa1a08aba19cf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.376+0000 7f1256000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:38:39.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.377+0000 7f1256000700 1 -- 172.21.15.131:0/2897214186 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c01c040 con 0x564fa082e6d0 2024-05-10T05:38:39.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.377+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2897214186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fa069d270 con 0x564fa082e6d0 2024-05-10T05:38:39.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.377+0000 7f1256000700 1 -- 172.21.15.131:0/2897214186 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f125c000e40 con 0x564fa082e6d0 2024-05-10T05:38:39.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.377+0000 7f1256000700 1 -- 172.21.15.131:0/2897214186 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f125c003990 con 0x564fa082e6d0 2024-05-10T05:38:39.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.378+0000 7f128b5ddb80 10 client.26309 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:39.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.378+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2897214186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564fa05c9cd0 con 0x564fa082e6d0 2024-05-10T05:38:39.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.378+0000 7f1256000700 1 -- 172.21.15.131:0/2897214186 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f125c043020 con 0x564fa082e6d0 2024-05-10T05:38:39.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.378+0000 7f1256000700 1 -- 172.21.15.131:0/2897214186 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f125c015070 con 0x564fa082e6d0 2024-05-10T05:38:39.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.378+0000 7f128b5ddb80 10 client.26309 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:39.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.379+0000 7f128b5ddb80 10 client.26309 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:38:39.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.379+0000 7f128b5ddb80 20 client.26309 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:39.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.379+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564fa08a1c90 0x564fa08a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:39.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.379+0000 7f128b5ddb80 4 client.26309 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:38:39.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.379+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2897214186 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564fa0592cf0 con 0x564fa08a1c90 2024-05-10T05:38:39.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.380+0000 7f1270c00700 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564fa08a1c90 0x564fa08a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:39.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.381+0000 7f1270c00700 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564fa08a1c90 0x564fa08a40b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53a66f58d9181721 server_cookie=1f2adcd47164f48c in_seq=0 out_seq=0 2024-05-10T05:38:39.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.381+0000 7f1256000700 10 client.26309 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:38:39.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.381+0000 7f1256000700 1 -- 172.21.15.131:0/2897214186 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564fa0592cf0 con 0x564fa08a1c90 2024-05-10T05:38:39.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.381+0000 7f1256000700 10 client.26309 handle_command_reply: tid=0 2024-05-10T05:38:39.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.381+0000 7f128b5ddb80 1 client.26309 shutdown 2024-05-10T05:38:39.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.382+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2897214186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082e6d0 msgr2=0x564fa08a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:39.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.382+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082e6d0 0x564fa08a0270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b690 tx=0x7f125c009230 comp rx=0 tx=0).stop 2024-05-10T05:38:39.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.382+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2897214186 shutdown_connections 2024-05-10T05:38:39.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.382+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x564fa08a1c90 0x564fa08a40b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.382+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564fa081f1d0 0x564fa085a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.382+0000 7f1256000700 0 client.26309 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:38:39.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.382+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564fa082e6d0 0x564fa08a0270 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.382+0000 7f128b5ddb80 1 --2- 172.21.15.131:0/2897214186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa082dd80 0x564fa085ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.382+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2897214186 >> 172.21.15.131:0/2897214186 conn(0x564fa0786a60 msgr2=0x564fa0830fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:39.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.382+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2897214186 shutdown_connections 2024-05-10T05:38:39.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.383+0000 7f128b5ddb80 1 -- 172.21.15.131:0/2897214186 wait complete. 2024-05-10T05:38:39.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.383+0000 7f128b5ddb80 20 client.26309 trim_cache size 0 max 16384 2024-05-10T05:38:39.385 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:38:39.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.384+0000 7f1273600700 1 -- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12500a1240 msgr2=0x7f12500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:39.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.384+0000 7f1273600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12500a1240 0x7f12500a3750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1264096540 tx=0x7f126400c980 comp rx=0 tx=0).stop 2024-05-10T05:38:39.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.384+0000 7f1273600700 1 -- 172.21.15.131:0/3343644476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c107970 msgr2=0x7f126c1a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:39.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.384+0000 7f1273600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c107970 0x7f126c1a16a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f126400ef90 tx=0x7f126400c450 comp rx=0 tx=0).stop 2024-05-10T05:38:39.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.384+0000 7f1273600700 1 -- 172.21.15.131:0/3343644476 shutdown_connections 2024-05-10T05:38:39.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.384+0000 7f1273600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12500a1240 0x7f12500a3750 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.384+0000 7f1273600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c1026b0 0x7f126c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.384+0000 7f1273600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c107970 0x7f126c1a16a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.384+0000 7f1273600700 1 --2- 172.21.15.131:0/3343644476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c105010 0x7f126c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:39.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.384+0000 7f1273600700 1 -- 172.21.15.131:0/3343644476 >> 172.21.15.131:0/3343644476 conn(0x7f126c0fc0d0 msgr2=0x7f126c1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:39.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.385+0000 7f1273600700 1 -- 172.21.15.131:0/3343644476 shutdown_connections 2024-05-10T05:38:39.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:39.385+0000 7f1273600700 1 -- 172.21.15.131:0/3343644476 wait complete. 2024-05-10T05:38:39.451 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:38:40.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.162+0000 7f0bc9200700 1 -- 172.21.15.131:0/1998971988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc41085e0 msgr2=0x7f0bc41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:40.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.162+0000 7f0bc9200700 1 --2- 172.21.15.131:0/1998971988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc41085e0 0x7f0bc41089f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00099e0 tx=0x7f0bb0009cf0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.163+0000 7f0bc9200700 1 -- 172.21.15.131:0/1998971988 shutdown_connections 2024-05-10T05:38:40.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.163+0000 7f0bc9200700 1 --2- 172.21.15.131:0/1998971988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc41085e0 0x7f0bc41089f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.163+0000 7f0bc9200700 1 --2- 172.21.15.131:0/1998971988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc410a0a0 0x7f0bc41109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.163+0000 7f0bc9200700 1 --2- 172.21.15.131:0/1998971988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc4109750 0x7f0bc4109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.163+0000 7f0bc9200700 1 -- 172.21.15.131:0/1998971988 >> 172.21.15.131:0/1998971988 conn(0x7f0bc406ea00 msgr2=0x7f0bc4070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.163+0000 7f0bc9200700 1 -- 172.21.15.131:0/1998971988 shutdown_connections 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.163+0000 7f0bc9200700 1 -- 172.21.15.131:0/1998971988 wait complete. 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.165+0000 7f0bc9200700 1 Processor -- start 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.165+0000 7f0bc9200700 1 -- start start 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.166+0000 7f0bc9200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc41085e0 0x7f0bc41199d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.166+0000 7f0bc9200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4109750 0x7f0bc4119f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.167+0000 7f0bc9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc410a0a0 0x7f0bc4120ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.167+0000 7f0bc2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc410a0a0 0x7f0bc4120ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.167+0000 7f0bc2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc410a0a0 0x7f0bc4120ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37848/0 (socket says 172.21.15.131:37848) 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.167+0000 7f0bc2800700 1 -- 172.21.15.131:0/1112298104 learned_addr learned my addr 172.21.15.131:0/1112298104 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:40.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.167+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41214b0 con 0x7f0bc4109750 2024-05-10T05:38:40.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.167+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc4121620 con 0x7f0bc41085e0 2024-05-10T05:38:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.168+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41217e0 con 0x7f0bc410a0a0 2024-05-10T05:38:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.168+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc41085e0 0x7f0bc41199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.168+0000 7f0bc2800700 1 -- 172.21.15.131:0/1112298104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc41085e0 msgr2=0x7f0bc41199d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.168+0000 7f0bc2800700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc41085e0 0x7f0bc41199d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.168+0000 7f0bc2800700 1 -- 172.21.15.131:0/1112298104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4109750 msgr2=0x7f0bc4119f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.168+0000 7f0bc2800700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4109750 0x7f0bc4119f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.168+0000 7f0bc2800700 1 -- 172.21.15.131:0/1112298104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00095f0 con 0x7f0bc410a0a0 2024-05-10T05:38:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.169+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc41085e0 0x7f0bc41199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.169+0000 7f0bc2800700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc410a0a0 0x7f0bc4120ea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00ee40 tx=0x7f0bbc00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.169+0000 7f0bbaa00700 1 -- 172.21.15.131:0/1112298104 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc018070 con 0x7f0bc410a0a0 2024-05-10T05:38:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.169+0000 7f0bbaa00700 1 -- 172.21.15.131:0/1112298104 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bbc0092e0 con 0x7f0bc410a0a0 2024-05-10T05:38:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.169+0000 7f0bbaa00700 1 -- 172.21.15.131:0/1112298104 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc010040 con 0x7f0bc410a0a0 2024-05-10T05:38:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.169+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc4121a60 con 0x7f0bc410a0a0 2024-05-10T05:38:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.170+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc41b7c60 con 0x7f0bc410a0a0 2024-05-10T05:38:40.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.181+0000 7f0bbaa00700 1 -- 172.21.15.131:0/1112298104 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0bbc004020 con 0x7f0bc410a0a0 2024-05-10T05:38:40.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.184+0000 7f0bbaa00700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ba80a1240 0x7f0ba80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.186+0000 7f0bbaa00700 1 -- 172.21.15.131:0/1112298104 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0bbc09aae0 con 0x7f0bc410a0a0 2024-05-10T05:38:40.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.189+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ba80a1240 0x7f0ba80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.203+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ba80a1240 0x7f0ba80a3750 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0003820 tx=0x7f0bb0005040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:40.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.205+0000 7f0be11e0b80 1 Processor -- start 2024-05-10T05:38:40.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.205+0000 7f0be11e0b80 1 -- start start 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.205+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd806a0 0x55988bd80af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.205+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 0x55988bd81980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.205+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.206+0000 7f0be11e0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bca7150 con 0x55988bd806a0 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.206+0000 7f0bc1400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 0x55988bd81980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.206+0000 7f0bc1400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 0x55988bd81980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37852/0 (socket says 172.21.15.131:37852) 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.206+0000 7f0bc1400700 1 -- 172.21.15.131:0/4203632192 learned_addr learned my addr 172.21.15.131:0/4203632192 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.206+0000 7f0bc2800700 1 --2- 172.21.15.131:0/4203632192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.207+0000 7f0be11e0b80 1 -- 172.21.15.131:0/4203632192 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bca4b10 con 0x55988bd81ec0 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.207+0000 7f0be11e0b80 1 -- 172.21.15.131:0/4203632192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bca0400 con 0x55988bd81570 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.209+0000 7f0bc2800700 1 -- 172.21.15.131:0/4203632192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 msgr2=0x55988bd81980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.209+0000 7f0bc2800700 1 --2- 172.21.15.131:0/4203632192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 0x55988bd81980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.209+0000 7f0bc2800700 1 -- 172.21.15.131:0/4203632192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd806a0 msgr2=0x55988bd80af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:40.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.209+0000 7f0bc2800700 1 --2- 172.21.15.131:0/4203632192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd806a0 0x55988bd80af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.209+0000 7f0bc2800700 1 -- 172.21.15.131:0/4203632192 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55988bb213b0 con 0x55988bd81ec0 2024-05-10T05:38:40.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.209+0000 7f0bc2800700 1 --2- 172.21.15.131:0/4203632192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd887c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc094f50 tx=0x7f0bbc0a1220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bbc2d368f88b5d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:40.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.210+0000 7f0baf400700 1 -- 172.21.15.131:0/4203632192 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0a0070 con 0x55988bd81ec0 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.211+0000 7f0baf400700 1 -- 172.21.15.131:0/4203632192 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bbc085110 con 0x55988bd81ec0 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.211+0000 7f0baf400700 1 -- 172.21.15.131:0/4203632192 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0cac20 con 0x55988bd81ec0 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.213+0000 7f0be11e0b80 1 -- 172.21.15.131:0/4203632192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 msgr2=0x55988bd887c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.213+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/4203632192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd887c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc094f50 tx=0x7f0bbc0a1220 comp rx=0 tx=0).stop 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.214+0000 7f0be11e0b80 1 -- 172.21.15.131:0/4203632192 shutdown_connections 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.214+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/4203632192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd806a0 0x55988bd80af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.214+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/4203632192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd887c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.214+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/4203632192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 0x55988bd81980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.214+0000 7f0be11e0b80 1 -- 172.21.15.131:0/4203632192 >> 172.21.15.131:0/4203632192 conn(0x55988bce2a60 msgr2=0x55988bd7afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.214+0000 7f0be11e0b80 1 -- 172.21.15.131:0/4203632192 shutdown_connections 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.214+0000 7f0be11e0b80 1 -- 172.21.15.131:0/4203632192 wait complete. 2024-05-10T05:38:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.216+0000 7f0be11e0b80 1 -- start start 2024-05-10T05:38:40.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.221+0000 7f0be11e0b80 1 Processor -- start 2024-05-10T05:38:40.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.229+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd806a0 0x55988bd8a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.229+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81570 0x55988bd8ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.229+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81ec0 0x55988bd91bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.229+0000 7f0be11e0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bc3c560 con 0x55988bd806a0 2024-05-10T05:38:40.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.229+0000 7f0be11e0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bc3c200 con 0x55988bd81570 2024-05-10T05:38:40.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.229+0000 7f0be11e0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bb236a0 con 0x55988bd81ec0 2024-05-10T05:38:40.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.229+0000 7f0be11e0b80 10 client.0 _ll_register_callbacks cb 0x55988bc45b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.230+0000 7f0bc2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81ec0 0x55988bd91bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.230+0000 7f0bc2800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81ec0 0x55988bd91bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37868/0 (socket says 172.21.15.131:37868) 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.230+0000 7f0bc2800700 1 -- 172.21.15.131:0/3063997884 learned_addr learned my addr 172.21.15.131:0/3063997884 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.230+0000 7f0bc1400700 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81570 0x55988bd8ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.230+0000 7f0bc2800700 1 -- 172.21.15.131:0/3063997884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81570 msgr2=0x55988bd8ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.230+0000 7f0bc2800700 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81570 0x55988bd8ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.230+0000 7f0bc2800700 1 -- 172.21.15.131:0/3063997884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd806a0 msgr2=0x55988bd8a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.230+0000 7f0bc2800700 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd806a0 0x55988bd8a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.230+0000 7f0bc2800700 1 -- 172.21.15.131:0/3063997884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc41b7c60 con 0x55988bd81ec0 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.231+0000 7f0bc1400700 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81570 0x55988bd8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.231+0000 7f0bc2800700 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81ec0 0x55988bd91bb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009cb0 tx=0x7f0bbc099d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e5f32793e7e745 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.231+0000 7f0ba3e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.232+0000 7f0ba3e00700 1 -- 172.21.15.131:0/3063997884 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0a0070 con 0x55988bd81ec0 2024-05-10T05:38:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.232+0000 7f0be11e0b80 1 -- 172.21.15.131:0/3063997884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55988bd4f780 con 0x55988bd81ec0 2024-05-10T05:38:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.232+0000 7f0ba3e00700 1 -- 172.21.15.131:0/3063997884 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0bbc0998d0 con 0x55988bd81ec0 2024-05-10T05:38:40.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.251+0000 7f0ba3e00700 1 -- 172.21.15.131:0/3063997884 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bbc0a0070 con 0x55988bd81ec0 2024-05-10T05:38:40.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.252+0000 7f0be11e0b80 10 client.26341 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:40.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.252+0000 7f0be11e0b80 1 -- 172.21.15.131:0/3063997884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55988bacdb60 con 0x55988bd81ec0 2024-05-10T05:38:40.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.253+0000 7f0ba3e00700 1 -- 172.21.15.131:0/3063997884 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55988bd4f780 con 0x55988bd81ec0 2024-05-10T05:38:40.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.255+0000 7f0ba3e00700 1 -- 172.21.15.131:0/3063997884 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0bbc0a2300 con 0x55988bd81ec0 2024-05-10T05:38:40.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.255+0000 7f0be11e0b80 10 client.26341 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:40.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.255+0000 7f0be11e0b80 10 client.26341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:38:40.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.255+0000 7f0be11e0b80 20 client.26341 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:40.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.255+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55988bd95dd0 0x55988bd98220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.256+0000 7f0be11e0b80 4 client.26341 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:40.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.256+0000 7f0be11e0b80 1 -- 172.21.15.131:0/3063997884 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55988bc98ec0 con 0x55988bd95dd0 2024-05-10T05:38:40.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.257+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55988bd95dd0 0x55988bd98220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.262+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55988bd95dd0 0x55988bd98220 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=474318b2ef0a0c70 server_cookie=831676c6e3483ae2 in_seq=0 out_seq=0 2024-05-10T05:38:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.263+0000 7f0ba3e00700 10 client.26341 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:38:40.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.265+0000 7f0ba3e00700 1 -- 172.21.15.131:0/3063997884 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55988bc98ec0 con 0x55988bd95dd0 2024-05-10T05:38:40.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.265+0000 7f0ba3e00700 10 client.26341 handle_command_reply: tid=0 2024-05-10T05:38:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.269+0000 7f0be11e0b80 1 client.26341 shutdown 2024-05-10T05:38:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.273+0000 7f0be11e0b80 1 -- 172.21.15.131:0/3063997884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81ec0 msgr2=0x55988bd91bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.275+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81ec0 0x55988bd91bb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009cb0 tx=0x7f0bbc099d40 comp rx=0 tx=0).stop 2024-05-10T05:38:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.275+0000 7f0be11e0b80 1 -- 172.21.15.131:0/3063997884 shutdown_connections 2024-05-10T05:38:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.275+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55988bd95dd0 0x55988bd98220 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.275+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd806a0 0x55988bd8a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.275+0000 7f0ba3e00700 0 client.26341 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:38:40.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.275+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81ec0 0x55988bd91bb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.275+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/3063997884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81570 0x55988bd8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.275+0000 7f0be11e0b80 1 -- 172.21.15.131:0/3063997884 >> 172.21.15.131:0/3063997884 conn(0x55988bce2a60 msgr2=0x55988bd7afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:40.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.276+0000 7f0be11e0b80 1 -- 172.21.15.131:0/3063997884 shutdown_connections 2024-05-10T05:38:40.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.276+0000 7f0be11e0b80 1 -- 172.21.15.131:0/3063997884 wait complete. 2024-05-10T05:38:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.276+0000 7f0be11e0b80 20 client.26341 trim_cache size 0 max 16384 2024-05-10T05:38:40.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.332+0000 7f0be11e0b80 1 Processor -- start 2024-05-10T05:38:40.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.332+0000 7f0be11e0b80 1 -- start start 2024-05-10T05:38:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.333+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd806a0 0x55988bd98220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.333+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 0x55988bd917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.334+0000 7f0bc1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd806a0 0x55988bd98220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.334+0000 7f0bc1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd806a0 0x55988bd98220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39304/0 (socket says 172.21.15.131:39304) 2024-05-10T05:38:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.334+0000 7f0bc1e00700 1 -- 172.21.15.131:0/2266632038 learned_addr learned my addr 172.21.15.131:0/2266632038 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.334+0000 7f0bc1400700 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 0x55988bd917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.334+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81ec0 0x55988bd89cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.334+0000 7f0be11e0b80 1 -- 172.21.15.131:0/2266632038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bd7b500 con 0x55988bd81ec0 2024-05-10T05:38:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.335+0000 7f0be11e0b80 1 -- 172.21.15.131:0/2266632038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bd757c0 con 0x55988bd806a0 2024-05-10T05:38:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.335+0000 7f0be11e0b80 1 -- 172.21.15.131:0/2266632038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bd86050 con 0x55988bd81570 2024-05-10T05:38:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.335+0000 7f0bc2800700 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81ec0 0x55988bd89cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.335+0000 7f0bc1e00700 1 -- 172.21.15.131:0/2266632038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 msgr2=0x55988bd917e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.336+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 0x55988bd917e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.336+0000 7f0bc1e00700 1 -- 172.21.15.131:0/2266632038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81ec0 msgr2=0x55988bd89cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.336+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81ec0 0x55988bd89cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.336+0000 7f0bc1e00700 1 -- 172.21.15.131:0/2266632038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55988bc66560 con 0x55988bd806a0 2024-05-10T05:38:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.336+0000 7f0bc2800700 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81ec0 0x55988bd89cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.337+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd806a0 0x55988bd98220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00099e0 tx=0x7f0bb0009cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf4dedbcad31410e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.337+0000 7f0bafe00700 1 -- 172.21.15.131:0/2266632038 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb001d070 con 0x55988bd806a0 2024-05-10T05:38:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.337+0000 7f0bafe00700 1 -- 172.21.15.131:0/2266632038 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bb002e7e0 con 0x55988bd806a0 2024-05-10T05:38:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.337+0000 7f0bafe00700 1 -- 172.21.15.131:0/2266632038 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb000ba30 con 0x55988bd806a0 2024-05-10T05:38:40.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.339+0000 7f0be11e0b80 1 -- 172.21.15.131:0/2266632038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd806a0 msgr2=0x55988bd98220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:40.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.339+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd806a0 0x55988bd98220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00099e0 tx=0x7f0bb0009cf0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.339+0000 7f0be11e0b80 1 -- 172.21.15.131:0/2266632038 shutdown_connections 2024-05-10T05:38:40.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.339+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81ec0 0x55988bd89cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.339+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd806a0 0x55988bd98220 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.339+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/2266632038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd81570 0x55988bd917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.339+0000 7f0be11e0b80 1 -- 172.21.15.131:0/2266632038 >> 172.21.15.131:0/2266632038 conn(0x55988bce2a60 msgr2=0x55988bd7bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:40.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.339+0000 7f0be11e0b80 1 -- 172.21.15.131:0/2266632038 shutdown_connections 2024-05-10T05:38:40.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.340+0000 7f0be11e0b80 1 -- 172.21.15.131:0/2266632038 wait complete. 2024-05-10T05:38:40.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.341+0000 7f0be11e0b80 1 -- start start 2024-05-10T05:38:40.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.341+0000 7f0be11e0b80 1 Processor -- start 2024-05-10T05:38:40.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.342+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd806a0 0x55988bd8f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.342+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81570 0x55988bd8f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.342+0000 7f0be11e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd90600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.342+0000 7f0be11e0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bd90b40 con 0x55988bd81570 2024-05-10T05:38:40.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.343+0000 7f0be11e0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bd90cb0 con 0x55988bd81ec0 2024-05-10T05:38:40.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.343+0000 7f0be11e0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988bd90e70 con 0x55988bd806a0 2024-05-10T05:38:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.342+0000 7f0bc2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd90600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.343+0000 7f0bc2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd90600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39314/0 (socket says 172.21.15.131:39314) 2024-05-10T05:38:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.343+0000 7f0bc2800700 1 -- 172.21.15.131:0/126521695 learned_addr learned my addr 172.21.15.131:0/126521695 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.343+0000 7f0be11e0b80 10 client.0 _ll_register_callbacks cb 0x55988bc45b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.343+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd806a0 0x55988bd8f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.344+0000 7f0bc1400700 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81570 0x55988bd8f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.344+0000 7f0bc2800700 1 -- 172.21.15.131:0/126521695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd806a0 msgr2=0x55988bd8f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.344+0000 7f0bc2800700 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd806a0 0x55988bd8f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.344+0000 7f0bc2800700 1 -- 172.21.15.131:0/126521695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81570 msgr2=0x55988bd8f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.344+0000 7f0bc2800700 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81570 0x55988bd8f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.344+0000 7f0bc2800700 1 -- 172.21.15.131:0/126521695 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc4111460 con 0x55988bd81ec0 2024-05-10T05:38:40.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.344+0000 7f0bc1400700 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81570 0x55988bd8f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:38:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.345+0000 7f0ba3e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:38:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.345+0000 7f0bc2800700 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd90600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc096980 tx=0x7f0bbc004d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fd0731ebd2af93 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.345+0000 7f0ba3e00700 1 -- 172.21.15.131:0/126521695 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc09f030 con 0x55988bd81ec0 2024-05-10T05:38:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.346+0000 7f0be11e0b80 1 -- 172.21.15.131:0/126521695 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55988bbf9270 con 0x55988bd81ec0 2024-05-10T05:38:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.346+0000 7f0ba3e00700 1 -- 172.21.15.131:0/126521695 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0bbc0998d0 con 0x55988bd81ec0 2024-05-10T05:38:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.346+0000 7f0ba3e00700 1 -- 172.21.15.131:0/126521695 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bbc09f030 con 0x55988bd81ec0 2024-05-10T05:38:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.346+0000 7f0be11e0b80 10 client.26321 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.346+0000 7f0be11e0b80 1 -- 172.21.15.131:0/126521695 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55988bb25cd0 con 0x55988bd81ec0 2024-05-10T05:38:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.347+0000 7f0ba3e00700 1 -- 172.21.15.131:0/126521695 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bbc0a6020 con 0x55988bd81ec0 2024-05-10T05:38:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.347+0000 7f0ba3e00700 1 -- 172.21.15.131:0/126521695 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0bbc09ad70 con 0x55988bd81ec0 2024-05-10T05:38:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.347+0000 7f0be11e0b80 10 client.26321 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.347+0000 7f0be11e0b80 10 client.26321 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:38:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.347+0000 7f0be11e0b80 20 client.26321 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.347+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55988bd95dd0 0x55988bdee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.347+0000 7f0be11e0b80 4 client.26321 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:38:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.347+0000 7f0be11e0b80 1 -- 172.21.15.131:0/126521695 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55988baeecf0 con 0x55988bd95dd0 2024-05-10T05:38:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.348+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55988bd95dd0 0x55988bdee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.349+0000 7f0bc1e00700 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55988bd95dd0 0x55988bdee3c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea502aa2c1013cf4 server_cookie=ad39237e4fb38f2 in_seq=0 out_seq=0 2024-05-10T05:38:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.350+0000 7f0ba3e00700 10 client.26321 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:38:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.350+0000 7f0ba3e00700 1 -- 172.21.15.131:0/126521695 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f0bb001e040 con 0x55988bd95dd0 2024-05-10T05:38:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.350+0000 7f0ba3e00700 10 client.26321 handle_command_reply: tid=0 2024-05-10T05:38:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.350+0000 7f0be11e0b80 1 client.26321 shutdown 2024-05-10T05:38:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.350+0000 7f0be11e0b80 1 -- 172.21.15.131:0/126521695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 msgr2=0x55988bd90600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.350+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd90600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc096980 tx=0x7f0bbc004d30 comp rx=0 tx=0).stop 2024-05-10T05:38:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.351+0000 7f0be11e0b80 1 -- 172.21.15.131:0/126521695 shutdown_connections 2024-05-10T05:38:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.351+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55988bd95dd0 0x55988bdee3c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.351+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55988bd81ec0 0x55988bd90600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.351+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55988bd806a0 0x55988bd8f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.351+0000 7f0be11e0b80 1 --2- 172.21.15.131:0/126521695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55988bd81570 0x55988bd8f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.351+0000 7f0be11e0b80 1 -- 172.21.15.131:0/126521695 >> 172.21.15.131:0/126521695 conn(0x55988bce2a60 msgr2=0x55988bd853f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:40.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.351+0000 7f0ba3e00700 0 client.26321 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:38:40.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.351+0000 7f0be11e0b80 1 -- 172.21.15.131:0/126521695 shutdown_connections 2024-05-10T05:38:40.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.351+0000 7f0be11e0b80 1 -- 172.21.15.131:0/126521695 wait complete. 2024-05-10T05:38:40.355 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.352+0000 7f0be11e0b80 20 client.26321 trim_cache size 0 max 16384 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.353+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ba80a1240 msgr2=0x7f0ba80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.353+0000 7f0bc9200700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ba80a1240 0x7f0ba80a3750 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0003820 tx=0x7f0bb0005040 comp rx=0 tx=0).stop 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.353+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc410a0a0 msgr2=0x7f0bc4120ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.353+0000 7f0bc9200700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc410a0a0 0x7f0bc4120ea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00ee40 tx=0x7f0bbc00b390 comp rx=0 tx=0).stop 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.353+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 shutdown_connections 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.353+0000 7f0bc9200700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ba80a1240 0x7f0ba80a3750 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.353+0000 7f0bc9200700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bc410a0a0 0x7f0bc4120ea0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.353+0000 7f0bc9200700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bc41085e0 0x7f0bc41199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.353+0000 7f0bc9200700 1 --2- 172.21.15.131:0/1112298104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bc4109750 0x7f0bc4119f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.353+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 >> 172.21.15.131:0/1112298104 conn(0x7f0bc406ea00 msgr2=0x7f0bc410c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.354+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 shutdown_connections 2024-05-10T05:38:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:40.354+0000 7f0bc9200700 1 -- 172.21.15.131:0/1112298104 wait complete. 2024-05-10T05:38:40.415 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:38:40.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:40 smithi157 ceph-mon[78427]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 258 MiB/s wr, 79 op/s 2024-05-10T05:38:40.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:40 smithi131 ceph-mon[37264]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 258 MiB/s wr, 79 op/s 2024-05-10T05:38:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:40 smithi142 ceph-mon[77667]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 258 MiB/s wr, 79 op/s 2024-05-10T05:38:41.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.058+0000 7f49ca600700 1 -- 172.21.15.131:0/1721334603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4102380 msgr2=0x7f49c4108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.058+0000 7f49ca600700 1 --2- 172.21.15.131:0/1721334603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4102380 0x7f49c4108c30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00b0d0 tx=0x7f49bc00b3e0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.059+0000 7f49ca600700 1 -- 172.21.15.131:0/1721334603 shutdown_connections 2024-05-10T05:38:41.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.059+0000 7f49ca600700 1 --2- 172.21.15.131:0/1721334603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c41008c0 0x7f49c4100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.059+0000 7f49ca600700 1 --2- 172.21.15.131:0/1721334603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4102380 0x7f49c4108c30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.059+0000 7f49ca600700 1 --2- 172.21.15.131:0/1721334603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c41019e0 0x7f49c4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.059+0000 7f49ca600700 1 -- 172.21.15.131:0/1721334603 >> 172.21.15.131:0/1721334603 conn(0x7f49c40fbef0 msgr2=0x7f49c40fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:41.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.061+0000 7f49ca600700 1 -- 172.21.15.131:0/1721334603 shutdown_connections 2024-05-10T05:38:41.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.061+0000 7f49ca600700 1 -- 172.21.15.131:0/1721334603 wait complete. 2024-05-10T05:38:41.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.065+0000 7f49ca600700 1 Processor -- start 2024-05-10T05:38:41.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.065+0000 7f49ca600700 1 -- start start 2024-05-10T05:38:41.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.066+0000 7f49ca600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c41008c0 0x7f49c41952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.067+0000 7f49ca600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c41019e0 0x7f49c4195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.068+0000 7f49c3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c41019e0 0x7f49c4195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.068+0000 7f49c3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c41019e0 0x7f49c4195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37892/0 (socket says 172.21.15.131:37892) 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.068+0000 7f49ca600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4102380 0x7f49c4191e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.068+0000 7f49c3e00700 1 -- 172.21.15.131:0/3708274988 learned_addr learned my addr 172.21.15.131:0/3708274988 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.068+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4192550 con 0x7f49c41008c0 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.068+0000 7f49c9200700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c41008c0 0x7f49c41952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.068+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c41926c0 con 0x7f49c4102380 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.068+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4192880 con 0x7f49c41019e0 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.069+0000 7f49c9c00700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4102380 0x7f49c4191e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.069+0000 7f49c3e00700 1 -- 172.21.15.131:0/3708274988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4102380 msgr2=0x7f49c4191e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.069+0000 7f49c3e00700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4102380 0x7f49c4191e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.069+0000 7f49c3e00700 1 -- 172.21.15.131:0/3708274988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c41008c0 msgr2=0x7f49c41952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.069+0000 7f49c3e00700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c41008c0 0x7f49c41952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.069+0000 7f49c3e00700 1 -- 172.21.15.131:0/3708274988 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc009d00 con 0x7f49c41019e0 2024-05-10T05:38:41.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.069+0000 7f49c9c00700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4102380 0x7f49c4191e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:41.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.069+0000 7f49c3e00700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c41019e0 0x7f49c4195800 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49b001c790 tx=0x7f49b001cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:41.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.072+0000 7f49c1600700 1 -- 172.21.15.131:0/3708274988 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0021800 con 0x7f49c41019e0 2024-05-10T05:38:41.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.072+0000 7f49c1600700 1 -- 172.21.15.131:0/3708274988 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b0021e20 con 0x7f49c41019e0 2024-05-10T05:38:41.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.072+0000 7f49c1600700 1 -- 172.21.15.131:0/3708274988 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0020350 con 0x7f49c41019e0 2024-05-10T05:38:41.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.073+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c4199e10 con 0x7f49c41019e0 2024-05-10T05:38:41.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.073+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c419a2c0 con 0x7f49c41019e0 2024-05-10T05:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.093+0000 7f49c1600700 1 -- 172.21.15.131:0/3708274988 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f49b00284f0 con 0x7f49c41019e0 2024-05-10T05:38:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.095+0000 7f49c1600700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49a80a1320 0x7f49a80a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.095+0000 7f49c1600700 1 -- 172.21.15.131:0/3708274988 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f49b00dcfc0 con 0x7f49c41019e0 2024-05-10T05:38:41.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.101+0000 7f49e3bbbb80 1 Processor -- start 2024-05-10T05:38:41.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.102+0000 7f49c9200700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49a80a1320 0x7f49a80a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.102+0000 7f49c9200700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49a80a1320 0x7f49a80a3830 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f49c41018e0 tx=0x7f49b4008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:41.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.107+0000 7f49e3bbbb80 1 -- start start 2024-05-10T05:38:41.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.108+0000 7f49e3bbbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c97c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.108+0000 7f49e3bbbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 0x564c4c97dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.108+0000 7f49e3bbbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97e0f0 0x564c4c9849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.108+0000 7f49e3bbbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c8a3150 con 0x564c4c97e0f0 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.108+0000 7f49e3bbbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c8a0b10 con 0x564c4c97d7a0 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.108+0000 7f49e3bbbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c89c400 con 0x564c4c97c530 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.108+0000 7f49c9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c97c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.108+0000 7f49c9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c97c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37896/0 (socket says 172.21.15.131:37896) 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.109+0000 7f49c9200700 1 -- 172.21.15.131:0/2226752897 learned_addr learned my addr 172.21.15.131:0/2226752897 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.109+0000 7f49c3e00700 1 --2- 172.21.15.131:0/2226752897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 0x564c4c97dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.109+0000 7f49c9200700 1 -- 172.21.15.131:0/2226752897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 msgr2=0x564c4c97dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.109+0000 7f49c9200700 1 --2- 172.21.15.131:0/2226752897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 0x564c4c97dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.109+0000 7f49c9200700 1 -- 172.21.15.131:0/2226752897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97e0f0 msgr2=0x564c4c9849f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.109+0000 7f49c9200700 1 --2- 172.21.15.131:0/2226752897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97e0f0 0x564c4c9849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.109+0000 7f49c9200700 1 -- 172.21.15.131:0/2226752897 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c4c71d3b0 con 0x564c4c97c530 2024-05-10T05:38:41.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.110+0000 7f49c9200700 1 --2- 172.21.15.131:0/2226752897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c97c960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f49b40096f0 tx=0x7f49b4010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa43d7190ee3cc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.110+0000 7f49ba000700 1 -- 172.21.15.131:0/2226752897 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4009e60 con 0x564c4c97c530 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.110+0000 7f49ba000700 1 -- 172.21.15.131:0/2226752897 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b4014b00 con 0x564c4c97c530 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.110+0000 7f49ba000700 1 -- 172.21.15.131:0/2226752897 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b401fb30 con 0x564c4c97c530 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.112+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/2226752897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 msgr2=0x564c4c97c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.112+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/2226752897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c97c960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f49b40096f0 tx=0x7f49b4010000 comp rx=0 tx=0).stop 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.115+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/2226752897 shutdown_connections 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.115+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/2226752897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97e0f0 0x564c4c9849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.115+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/2226752897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c97c960 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.115+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/2226752897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 0x564c4c97dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.115+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/2226752897 >> 172.21.15.131:0/2226752897 conn(0x564c4c8dea60 msgr2=0x564c4c977330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.117+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/2226752897 shutdown_connections 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.117+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/2226752897 wait complete. 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.118+0000 7f49e3bbbb80 1 -- start start 2024-05-10T05:38:41.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.118+0000 7f49e3bbbb80 1 Processor -- start 2024-05-10T05:38:41.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.124+0000 7f49e3bbbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97c530 0x564c4c9868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.124+0000 7f49e3bbbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97d7a0 0x564c4c986e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.124+0000 7f49e3bbbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97e0f0 0x564c4c98dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.124+0000 7f49e3bbbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c838560 con 0x564c4c97c530 2024-05-10T05:38:41.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.124+0000 7f49e3bbbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c838200 con 0x564c4c97e0f0 2024-05-10T05:38:41.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.124+0000 7f49e3bbbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c71f6a0 con 0x564c4c97d7a0 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.125+0000 7f49c3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97d7a0 0x564c4c986e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.125+0000 7f49c3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97d7a0 0x564c4c986e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37910/0 (socket says 172.21.15.131:37910) 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.125+0000 7f49c3e00700 1 -- 172.21.15.131:0/1974810234 learned_addr learned my addr 172.21.15.131:0/1974810234 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.125+0000 7f49e3bbbb80 10 client.0 _ll_register_callbacks cb 0x564c4c841b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.125+0000 7f49c3e00700 1 -- 172.21.15.131:0/1974810234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97e0f0 msgr2=0x564c4c98dd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.125+0000 7f49c3e00700 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97e0f0 0x564c4c98dd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.125+0000 7f49c3e00700 1 -- 172.21.15.131:0/1974810234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97c530 msgr2=0x564c4c9868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.125+0000 7f49c3e00700 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97c530 0x564c4c9868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.125+0000 7f49c3e00700 1 -- 172.21.15.131:0/1974810234 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b400e040 con 0x564c4c97d7a0 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.126+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1974810234 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c4c7f5270 con 0x564c4c97d7a0 2024-05-10T05:38:41.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.126+0000 7f49c3e00700 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97d7a0 0x564c4c986e20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f49b00a68b0 tx=0x7f49b00a9d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69367076c54ad9eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:41.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.126+0000 7f49aea00700 10 client.26356 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:38:41.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.126+0000 7f49aea00700 1 -- 172.21.15.131:0/1974810234 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00a9aa0 con 0x564c4c97d7a0 2024-05-10T05:38:41.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.126+0000 7f49aea00700 1 -- 172.21.15.131:0/1974810234 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f49b00aa3b0 con 0x564c4c97d7a0 2024-05-10T05:38:41.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.127+0000 7f49aea00700 1 -- 172.21.15.131:0/1974810234 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b00ae1e0 con 0x564c4c97d7a0 2024-05-10T05:38:41.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.130+0000 7f49e3bbbb80 10 client.26356 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:41.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.130+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1974810234 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x564c4c6c9b60 con 0x564c4c97d7a0 2024-05-10T05:38:41.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.131+0000 7f49aea00700 1 -- 172.21.15.131:0/1974810234 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f49b00b23a0 con 0x564c4c97d7a0 2024-05-10T05:38:41.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.131+0000 7f49e3bbbb80 10 client.26356 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:41.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.131+0000 7f49e3bbbb80 10 client.26356 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:38:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.131+0000 7f49e3bbbb80 20 client.26356 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.131+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x564c4c9923e0 0x564c4c994850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.131+0000 7f49e3bbbb80 4 client.26356 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:41.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.131+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1974810234 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c4c94b780 con 0x564c4c9923e0 2024-05-10T05:38:41.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.133+0000 7f49c9200700 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x564c4c9923e0 0x564c4c994850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.134+0000 7f49aea00700 10 client.26356 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:38:41.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.134+0000 7f49c9200700 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x564c4c9923e0 0x564c4c994850 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=581e5abe8137178d server_cookie=fe42c6a4aaf7b1a4 in_seq=0 out_seq=0 2024-05-10T05:38:41.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.138+0000 7f49aea00700 1 -- 172.21.15.131:0/1974810234 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c4c94b780 con 0x564c4c9923e0 2024-05-10T05:38:41.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.138+0000 7f49aea00700 10 client.26356 handle_command_reply: tid=0 2024-05-10T05:38:41.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.138+0000 7f49e3bbbb80 1 client.26356 shutdown 2024-05-10T05:38:41.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.138+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1974810234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97d7a0 msgr2=0x564c4c986e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.138+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97d7a0 0x564c4c986e20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f49b00a68b0 tx=0x7f49b00a9d10 comp rx=0 tx=0).stop 2024-05-10T05:38:41.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.139+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1974810234 shutdown_connections 2024-05-10T05:38:41.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.141+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x564c4c9923e0 0x564c4c994850 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.141+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97c530 0x564c4c9868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.141+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97e0f0 0x564c4c98dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.141+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1974810234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97d7a0 0x564c4c986e20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.141+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1974810234 >> 172.21.15.131:0/1974810234 conn(0x564c4c8dea60 msgr2=0x564c4c97b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:41.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.141+0000 7f49aea00700 0 client.26356 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:38:41.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.142+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1974810234 shutdown_connections 2024-05-10T05:38:41.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.143+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1974810234 wait complete. 2024-05-10T05:38:41.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.143+0000 7f49e3bbbb80 20 client.26356 trim_cache size 0 max 16384 2024-05-10T05:38:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.183+0000 7f49e3bbbb80 1 Processor -- start 2024-05-10T05:38:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.184+0000 7f49e3bbbb80 1 -- start start 2024-05-10T05:38:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.184+0000 7f49e3bbbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c98e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.187+0000 7f49e3bbbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 0x564c4c98dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.187+0000 7f49c9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c98e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.187+0000 7f49c9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c98e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37918/0 (socket says 172.21.15.131:37918) 2024-05-10T05:38:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.187+0000 7f49c9200700 1 -- 172.21.15.131:0/3916925183 learned_addr learned my addr 172.21.15.131:0/3916925183 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.188+0000 7f49c3e00700 1 --2- 172.21.15.131:0/3916925183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 0x564c4c98dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.188+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/3916925183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97e0f0 0x564c4c9815d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.188+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/3916925183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c97d610 con 0x564c4c97e0f0 2024-05-10T05:38:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.188+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/3916925183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c984ce0 con 0x564c4c97d7a0 2024-05-10T05:38:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.188+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/3916925183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c984960 con 0x564c4c97c530 2024-05-10T05:38:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.190+0000 7f49c3e00700 1 -- 172.21.15.131:0/3916925183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 msgr2=0x564c4c98e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.190+0000 7f49c3e00700 1 --2- 172.21.15.131:0/3916925183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c98e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.190+0000 7f49c3e00700 1 -- 172.21.15.131:0/3916925183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97e0f0 msgr2=0x564c4c9815d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.190+0000 7f49c3e00700 1 --2- 172.21.15.131:0/3916925183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97e0f0 0x564c4c9815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.190+0000 7f49c3e00700 1 -- 172.21.15.131:0/3916925183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c4c862560 con 0x564c4c97d7a0 2024-05-10T05:38:41.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.193+0000 7f49c3e00700 1 --2- 172.21.15.131:0/3916925183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 0x564c4c98dd40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f49b00a6e20 tx=0x7f49b00ad000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17a1b03b11eb7bf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.193+0000 7f49baa00700 1 -- 172.21.15.131:0/3916925183 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0030910 con 0x564c4c97d7a0 2024-05-10T05:38:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.193+0000 7f49baa00700 1 -- 172.21.15.131:0/3916925183 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c4c7f5270 con 0x564c4c97d7a0 2024-05-10T05:38:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.193+0000 7f49baa00700 1 -- 172.21.15.131:0/3916925183 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00a9b00 con 0x564c4c97d7a0 2024-05-10T05:38:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.195+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/3916925183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 msgr2=0x564c4c98dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.195+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/3916925183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 0x564c4c98dd40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f49b00a6e20 tx=0x7f49b00ad000 comp rx=0 tx=0).stop 2024-05-10T05:38:41.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.195+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/3916925183 shutdown_connections 2024-05-10T05:38:41.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.195+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/3916925183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97e0f0 0x564c4c9815d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.195+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/3916925183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c98e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.195+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/3916925183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97d7a0 0x564c4c98dd40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.196+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/3916925183 >> 172.21.15.131:0/3916925183 conn(0x564c4c8dea60 msgr2=0x564c4c9868e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:41.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.196+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/3916925183 shutdown_connections 2024-05-10T05:38:41.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.196+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/3916925183 wait complete. 2024-05-10T05:38:41.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.197+0000 7f49e3bbbb80 1 -- start start 2024-05-10T05:38:41.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.198+0000 7f49e3bbbb80 1 Processor -- start 2024-05-10T05:38:41.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.199+0000 7f49e3bbbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c98b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.200+0000 7f49e3bbbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97d7a0 0x564c4c98b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.200+0000 7f49c9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c98b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.200+0000 7f49c9200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c98b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37930/0 (socket says 172.21.15.131:37930) 2024-05-10T05:38:41.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.200+0000 7f49c9200700 1 -- 172.21.15.131:0/1246559824 learned_addr learned my addr 172.21.15.131:0/1246559824 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:41.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.200+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97e0f0 0x564c4c98c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.200+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c98c960 con 0x564c4c97d7a0 2024-05-10T05:38:41.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.200+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c98cad0 con 0x564c4c97e0f0 2024-05-10T05:38:41.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.200+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4c98cc90 con 0x564c4c97c530 2024-05-10T05:38:41.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.200+0000 7f49e3bbbb80 10 client.0 _ll_register_callbacks cb 0x564c4c841b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:41.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.201+0000 7f49c3e00700 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97d7a0 0x564c4c98b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.202+0000 7f49c3e00700 1 -- 172.21.15.131:0/1246559824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 msgr2=0x564c4c98b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:41.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.202+0000 7f49c3e00700 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c98b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.202+0000 7f49c3e00700 1 -- 172.21.15.131:0/1246559824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97e0f0 msgr2=0x564c4c98c420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:38:41.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.202+0000 7f49c3e00700 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97e0f0 0x564c4c98c420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.202+0000 7f49c3e00700 1 -- 172.21.15.131:0/1246559824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c4c6c9b60 con 0x564c4c97d7a0 2024-05-10T05:38:41.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.203+0000 7f49aea00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:38:41.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.203+0000 7f49c3e00700 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97d7a0 0x564c4c98b5b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f49b001a040 tx=0x7f49b00ae110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef5fbfb330fac55 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.207+0000 7f49aea00700 1 -- 172.21.15.131:0/1246559824 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0030b60 con 0x564c4c97d7a0 2024-05-10T05:38:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.208+0000 7f49aea00700 1 -- 172.21.15.131:0/1246559824 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f49b0030cc0 con 0x564c4c97d7a0 2024-05-10T05:38:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.208+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49b4014b00 con 0x564c4c97d7a0 2024-05-10T05:38:41.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.211+0000 7f49aea00700 1 -- 172.21.15.131:0/1246559824 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b0030e80 con 0x564c4c97d7a0 2024-05-10T05:38:41.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.211+0000 7f49e3bbbb80 10 client.16179 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:41.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.211+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564c4c721cd0 con 0x564c4c97d7a0 2024-05-10T05:38:41.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.213+0000 7f49aea00700 1 -- 172.21.15.131:0/1246559824 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b008d020 con 0x564c4c97d7a0 2024-05-10T05:38:41.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.213+0000 7f49aea00700 1 -- 172.21.15.131:0/1246559824 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f49b00bb070 con 0x564c4c97d7a0 2024-05-10T05:38:41.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.213+0000 7f49e3bbbb80 10 client.16179 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:41.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.213+0000 7f49e3bbbb80 10 client.16179 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:38:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.213+0000 7f49e3bbbb80 20 client.16179 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.213+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x564c4c9e8910 0x564c4c9ead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.213+0000 7f49e3bbbb80 4 client.16179 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:38:41.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.213+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c4c894ec0 con 0x564c4c9e8910 2024-05-10T05:38:41.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.217+0000 7f49c9200700 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x564c4c9e8910 0x564c4c9ead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.218+0000 7f49aea00700 10 client.16179 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:38:41.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.218+0000 7f49c9200700 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x564c4c9e8910 0x564c4c9ead30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2cd2f42ae817551 server_cookie=98156453e88863d3 in_seq=0 out_seq=0 2024-05-10T05:38:41.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.219+0000 7f49aea00700 1 -- 172.21.15.131:0/1246559824 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c4c894ec0 con 0x564c4c9e8910 2024-05-10T05:38:41.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.219+0000 7f49aea00700 10 client.16179 handle_command_reply: tid=0 2024-05-10T05:38:41.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.219+0000 7f49e3bbbb80 1 client.16179 shutdown 2024-05-10T05:38:41.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.221+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97d7a0 msgr2=0x564c4c98b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.221+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97d7a0 0x564c4c98b5b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f49b001a040 tx=0x7f49b00ae110 comp rx=0 tx=0).stop 2024-05-10T05:38:41.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.221+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 shutdown_connections 2024-05-10T05:38:41.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.221+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x564c4c9e8910 0x564c4c9ead30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.221+0000 7f49aea00700 0 client.16179 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:38:41.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.222+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4c97e0f0 0x564c4c98c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.222+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c4c97c530 0x564c4c98b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.222+0000 7f49e3bbbb80 1 --2- 172.21.15.131:0/1246559824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564c4c97d7a0 0x564c4c98b5b0 secure :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f49b001a040 tx=0x7f49b00ae110 comp rx=0 tx=0).stop 2024-05-10T05:38:41.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.222+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 >> 172.21.15.131:0/1246559824 conn(0x564c4c8dea60 msgr2=0x564c4c9702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:41.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.222+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 shutdown_connections 2024-05-10T05:38:41.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.223+0000 7f49e3bbbb80 1 -- 172.21.15.131:0/1246559824 wait complete. 2024-05-10T05:38:41.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.229+0000 7f49e3bbbb80 20 client.16179 trim_cache size 0 max 16384 2024-05-10T05:38:41.230 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:38:41.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.230+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49a80a1320 msgr2=0x7f49a80a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.230+0000 7f49ca600700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49a80a1320 0x7f49a80a3830 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f49c41018e0 tx=0x7f49b4008040 comp rx=0 tx=0).stop 2024-05-10T05:38:41.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.230+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c41019e0 msgr2=0x7f49c4195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.230+0000 7f49ca600700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c41019e0 0x7f49c4195800 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49b001c790 tx=0x7f49b001cb50 comp rx=0 tx=0).stop 2024-05-10T05:38:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.231+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 shutdown_connections 2024-05-10T05:38:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.231+0000 7f49ca600700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f49a80a1320 0x7f49a80a3830 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.231+0000 7f49ca600700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c41008c0 0x7f49c41952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.231+0000 7f49ca600700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4102380 0x7f49c4191e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.231+0000 7f49ca600700 1 --2- 172.21.15.131:0/3708274988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c41019e0 0x7f49c4195800 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.231+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 >> 172.21.15.131:0/3708274988 conn(0x7f49c40fbef0 msgr2=0x7f49c40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:41.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.232+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 shutdown_connections 2024-05-10T05:38:41.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.232+0000 7f49ca600700 1 -- 172.21.15.131:0/3708274988 wait complete. 2024-05-10T05:38:41.287 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:38:41.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.972+0000 7f9095400700 1 -- 172.21.15.131:0/3055814649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9090109ad0 msgr2=0x7f9090110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.972+0000 7f9095400700 1 --2- 172.21.15.131:0/3055814649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9090109ad0 0x7f9090110380 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f907c01a9d0 tx=0x7f907c01ace0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.976+0000 7f9095400700 1 -- 172.21.15.131:0/3055814649 shutdown_connections 2024-05-10T05:38:41.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.976+0000 7f9095400700 1 --2- 172.21.15.131:0/3055814649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9090109ad0 0x7f9090110380 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.976+0000 7f9095400700 1 --2- 172.21.15.131:0/3055814649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9090108010 0x7f9090108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.976+0000 7f9095400700 1 --2- 172.21.15.131:0/3055814649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9090109130 0x7f9090109540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.976+0000 7f9095400700 1 -- 172.21.15.131:0/3055814649 >> 172.21.15.131:0/3055814649 conn(0x7f9090077040 msgr2=0x7f9090106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:41.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.976+0000 7f9095400700 1 -- 172.21.15.131:0/3055814649 shutdown_connections 2024-05-10T05:38:41.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.976+0000 7f9095400700 1 -- 172.21.15.131:0/3055814649 wait complete. 2024-05-10T05:38:41.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.979+0000 7f9095400700 1 Processor -- start 2024-05-10T05:38:41.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.980+0000 7f9095400700 1 -- start start 2024-05-10T05:38:41.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.982+0000 7f9095400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9090108010 0x7f90901b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.982+0000 7f9095400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9090109130 0x7f90901b8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.982+0000 7f908fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9090108010 0x7f90901b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.982+0000 7f908fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9090108010 0x7f90901b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37942/0 (socket says 172.21.15.131:37942) 2024-05-10T05:38:41.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.982+0000 7f9095400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9090109ad0 0x7f90901bf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:41.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.983+0000 7f908f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9090109130 0x7f90901b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.983+0000 7f9094a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9090109ad0 0x7f90901bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:41.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.983+0000 7f908f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9090109130 0x7f90901b8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39382/0 (socket says 172.21.15.131:39382) 2024-05-10T05:38:41.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.983+0000 7f9094a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9090109ad0 0x7f90901bf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43016/0 (socket says 172.21.15.131:43016) 2024-05-10T05:38:41.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.983+0000 7f908f400700 1 -- 172.21.15.131:0/3014049317 learned_addr learned my addr 172.21.15.131:0/3014049317 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:41.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.983+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901bf750 con 0x7f9090109ad0 2024-05-10T05:38:41.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.983+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901bf890 con 0x7f9090109130 2024-05-10T05:38:41.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.983+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901bfa50 con 0x7f9090108010 2024-05-10T05:38:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.984+0000 7f908f400700 1 -- 172.21.15.131:0/3014049317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9090108010 msgr2=0x7f90901b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.984+0000 7f908f400700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9090108010 0x7f90901b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.984+0000 7f908f400700 1 -- 172.21.15.131:0/3014049317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9090109ad0 msgr2=0x7f90901bf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.984+0000 7f908f400700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9090109ad0 0x7f90901bf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.985+0000 7f908f400700 1 -- 172.21.15.131:0/3014049317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c01a600 con 0x7f9090109130 2024-05-10T05:38:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.985+0000 7f908fe00700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9090108010 0x7f90901b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.985+0000 7f9094a00700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9090109ad0 0x7f90901bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:38:41.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.985+0000 7f908f400700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9090109130 0x7f90901b8520 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f908800ba80 tx=0x7f908800bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:41.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.985+0000 7f908cc00700 1 -- 172.21.15.131:0/3014049317 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800c9b0 con 0x7f9090109130 2024-05-10T05:38:41.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.985+0000 7f908cc00700 1 -- 172.21.15.131:0/3014049317 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9088014440 con 0x7f9090109130 2024-05-10T05:38:41.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.985+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90901bfd30 con 0x7f9090109130 2024-05-10T05:38:41.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.986+0000 7f908cc00700 1 -- 172.21.15.131:0/3014049317 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800c9b0 con 0x7f9090109130 2024-05-10T05:38:41.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.986+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90901c0190 con 0x7f9090109130 2024-05-10T05:38:42.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.992+0000 7f908cc00700 1 -- 172.21.15.131:0/3014049317 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f90880145b0 con 0x7f9090109130 2024-05-10T05:38:42.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.999+0000 7f908cc00700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f90740a1230 0x7f90740a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:41.999+0000 7f908cc00700 1 -- 172.21.15.131:0/3014049317 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f90880cb890 con 0x7f9090109130 2024-05-10T05:38:42.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.001+0000 7f90ae8cfb80 1 Processor -- start 2024-05-10T05:38:42.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.002+0000 7f908fe00700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f90740a1230 0x7f90740a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.004+0000 7f90ae8cfb80 1 -- start start 2024-05-10T05:38:42.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.004+0000 7f908fe00700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f90740a1230 0x7f90740a3740 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9080006fd0 tx=0x7f9080008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:42.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.005+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a3530 0x55ed089a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.006+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a47a0 0x55ed089a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.006+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a50f0 0x55ed089ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.006+0000 7f90ae8cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed088ca150 con 0x55ed089a47a0 2024-05-10T05:38:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.006+0000 7f90ae8cfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed088c7b10 con 0x55ed089a50f0 2024-05-10T05:38:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.006+0000 7f90ae8cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed088c3400 con 0x55ed089a3530 2024-05-10T05:38:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.006+0000 7f908fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a3530 0x55ed089a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.006+0000 7f908fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a3530 0x55ed089a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:37956/0 (socket says 172.21.15.131:37956) 2024-05-10T05:38:42.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.006+0000 7f908fe00700 1 -- 172.21.15.131:0/2856035798 learned_addr learned my addr 172.21.15.131:0/2856035798 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:42.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.007+0000 7f908fe00700 1 -- 172.21.15.131:0/2856035798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a50f0 msgr2=0x55ed089ab9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:38:42.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.007+0000 7f908fe00700 1 --2- 172.21.15.131:0/2856035798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a50f0 0x55ed089ab9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.007+0000 7f908fe00700 1 -- 172.21.15.131:0/2856035798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a47a0 msgr2=0x55ed089a4bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:38:42.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.007+0000 7f908fe00700 1 --2- 172.21.15.131:0/2856035798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a47a0 0x55ed089a4bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.007+0000 7f908fe00700 1 -- 172.21.15.131:0/2856035798 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed087443b0 con 0x55ed089a3530 2024-05-10T05:38:42.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.008+0000 7f908fe00700 1 --2- 172.21.15.131:0/2856035798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a3530 0x55ed089a3960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f908000c8e0 tx=0x7f908000f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f70818d28d0eb3e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:42.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.009+0000 7f9085200700 1 -- 172.21.15.131:0/2856035798 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080004220 con 0x55ed089a3530 2024-05-10T05:38:42.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.009+0000 7f9085200700 1 -- 172.21.15.131:0/2856035798 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9080003c00 con 0x55ed089a3530 2024-05-10T05:38:42.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.009+0000 7f9085200700 1 -- 172.21.15.131:0/2856035798 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080013a70 con 0x55ed089a3530 2024-05-10T05:38:42.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.010+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/2856035798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a3530 msgr2=0x55ed089a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:42.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.010+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/2856035798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a3530 0x55ed089a3960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f908000c8e0 tx=0x7f908000f000 comp rx=0 tx=0).stop 2024-05-10T05:38:42.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.011+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/2856035798 shutdown_connections 2024-05-10T05:38:42.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.011+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/2856035798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a50f0 0x55ed089ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.011+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/2856035798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a3530 0x55ed089a3960 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.011+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/2856035798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a47a0 0x55ed089a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.011+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/2856035798 >> 172.21.15.131:0/2856035798 conn(0x55ed08905a60 msgr2=0x55ed0899e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:42.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.012+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/2856035798 shutdown_connections 2024-05-10T05:38:42.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.012+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/2856035798 wait complete. 2024-05-10T05:38:42.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.014+0000 7f90ae8cfb80 1 -- start start 2024-05-10T05:38:42.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.015+0000 7f90ae8cfb80 1 Processor -- start 2024-05-10T05:38:42.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.017+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a3530 0x55ed089ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.019+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 0x55ed089add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.020+0000 7f908fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a3530 0x55ed089ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.020+0000 7f908fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a3530 0x55ed089ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43022/0 (socket says 172.21.15.131:43022) 2024-05-10T05:38:42.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.020+0000 7f908fe00700 1 -- 172.21.15.131:0/1653271055 learned_addr learned my addr 172.21.15.131:0/1653271055 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:42.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.020+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a50f0 0x55ed089b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.020+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0885f560 con 0x55ed089a3530 2024-05-10T05:38:42.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.020+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0885f200 con 0x55ed089a47a0 2024-05-10T05:38:42.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.020+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed087466a0 con 0x55ed089a50f0 2024-05-10T05:38:42.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.021+0000 7f908fe00700 1 -- 172.21.15.131:0/1653271055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a50f0 msgr2=0x55ed089b4c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:38:42.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.021+0000 7f908fe00700 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a50f0 0x55ed089b4c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.021+0000 7f908fe00700 1 -- 172.21.15.131:0/1653271055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 msgr2=0x55ed089add00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:38:42.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.021+0000 7f908fe00700 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 0x55ed089add00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.021+0000 7f908fe00700 1 -- 172.21.15.131:0/1653271055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908000c650 con 0x55ed089a3530 2024-05-10T05:38:42.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.022+0000 7f90ae8cfb80 10 client.0 _ll_register_callbacks cb 0x55ed08868b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:42.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.022+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed0881c270 con 0x55ed089a3530 2024-05-10T05:38:42.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.023+0000 7f908fe00700 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a3530 0x55ed089ad7c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9080020040 tx=0x7f9080013b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea258c7ce5e5faa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:42.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.023+0000 7f9079600700 10 client.16185 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:38:42.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.023+0000 7f9079600700 1 -- 172.21.15.131:0/1653271055 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080004280 con 0x55ed089a3530 2024-05-10T05:38:42.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.024+0000 7f9079600700 1 -- 172.21.15.131:0/1653271055 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9080028460 con 0x55ed089a3530 2024-05-10T05:38:42.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.026+0000 7f9079600700 1 -- 172.21.15.131:0/1653271055 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90800043e0 con 0x55ed089a3530 2024-05-10T05:38:42.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.027+0000 7f90ae8cfb80 10 client.16185 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:42.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.027+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55ed086f0b60 con 0x55ed089a3530 2024-05-10T05:38:42.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.028+0000 7f9079600700 1 -- 172.21.15.131:0/1653271055 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9080026040 con 0x55ed089a3530 2024-05-10T05:38:42.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.028+0000 7f90ae8cfb80 10 client.16185 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:42.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.028+0000 7f90ae8cfb80 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:42.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.028+0000 7f90ae8cfb80 20 client.16185 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:42.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.028+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ed089ca1c0 0x55ed089cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.028+0000 7f90ae8cfb80 4 client.16185 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.028+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed08972780 con 0x55ed089ca1c0 2024-05-10T05:38:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.029+0000 7f908f400700 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ed089ca1c0 0x55ed089cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.029+0000 7f908f400700 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ed089ca1c0 0x55ed089cc610 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=843d408911bd15ca server_cookie=cfe7ad9726cfd865 in_seq=0 out_seq=0 2024-05-10T05:38:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.030+0000 7f9079600700 10 client.16185 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.031+0000 7f9079600700 1 -- 172.21.15.131:0/1653271055 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed08972780 con 0x55ed089ca1c0 2024-05-10T05:38:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.031+0000 7f9079600700 10 client.16185 handle_command_reply: tid=0 2024-05-10T05:38:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.031+0000 7f90ae8cfb80 1 client.16185 shutdown 2024-05-10T05:38:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.032+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a3530 msgr2=0x55ed089ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.032+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a3530 0x55ed089ad7c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9080020040 tx=0x7f9080013b10 comp rx=0 tx=0).stop 2024-05-10T05:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.032+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 shutdown_connections 2024-05-10T05:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.032+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ed089ca1c0 0x55ed089cc610 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.032+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a3530 0x55ed089ad7c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.032+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a50f0 0x55ed089b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.032+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1653271055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 0x55ed089add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.032+0000 7f9079600700 0 client.16185 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.032+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 >> 172.21.15.131:0/1653271055 conn(0x55ed08905a60 msgr2=0x55ed089a2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:42.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.033+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 shutdown_connections 2024-05-10T05:38:42.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.033+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1653271055 wait complete. 2024-05-10T05:38:42.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.034+0000 7f90ae8cfb80 20 client.16185 trim_cache size 0 max 16384 2024-05-10T05:38:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.080+0000 7f90ae8cfb80 1 Processor -- start 2024-05-10T05:38:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.080+0000 7f90ae8cfb80 1 -- start start 2024-05-10T05:38:42.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.083+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 0x55ed089ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.083+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a50f0 0x55ed089b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.083+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089b3120 0x55ed0899dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.083+0000 7f90ae8cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed089988d0 con 0x55ed089a50f0 2024-05-10T05:38:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.083+0000 7f90ae8cfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed089b5690 con 0x55ed089a47a0 2024-05-10T05:38:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.083+0000 7f90ae8cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed089d1810 con 0x55ed089b3120 2024-05-10T05:38:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.084+0000 7f908fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 0x55ed089ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.084+0000 7f908fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 0x55ed089ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39404/0 (socket says 172.21.15.131:39404) 2024-05-10T05:38:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.084+0000 7f908fe00700 1 -- 172.21.15.131:0/1870345181 learned_addr learned my addr 172.21.15.131:0/1870345181 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.084+0000 7f9094a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089b3120 0x55ed0899dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.085+0000 7f908f400700 1 --2- 172.21.15.131:0/1870345181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a50f0 0x55ed089b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.085+0000 7f908fe00700 1 -- 172.21.15.131:0/1870345181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089b3120 msgr2=0x55ed0899dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.085+0000 7f908fe00700 1 --2- 172.21.15.131:0/1870345181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089b3120 0x55ed0899dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.085+0000 7f908fe00700 1 -- 172.21.15.131:0/1870345181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a50f0 msgr2=0x55ed089b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.085+0000 7f908fe00700 1 --2- 172.21.15.131:0/1870345181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a50f0 0x55ed089b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.085+0000 7f908fe00700 1 -- 172.21.15.131:0/1870345181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed08889560 con 0x55ed089a47a0 2024-05-10T05:38:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.086+0000 7f908fe00700 1 --2- 172.21.15.131:0/1870345181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 0x55ed089ac640 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f908001dd70 tx=0x7f9080028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6472d0cd4a0af189 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:42.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.088+0000 7f9085c00700 1 -- 172.21.15.131:0/1870345181 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080013630 con 0x55ed089a47a0 2024-05-10T05:38:42.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.088+0000 7f9085c00700 1 -- 172.21.15.131:0/1870345181 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9080017030 con 0x55ed089a47a0 2024-05-10T05:38:42.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.088+0000 7f9085c00700 1 -- 172.21.15.131:0/1870345181 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908001db00 con 0x55ed089a47a0 2024-05-10T05:38:42.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.091+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1870345181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 msgr2=0x55ed089ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:42.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.091+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1870345181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 0x55ed089ac640 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f908001dd70 tx=0x7f9080028000 comp rx=0 tx=0).stop 2024-05-10T05:38:42.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.091+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1870345181 shutdown_connections 2024-05-10T05:38:42.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.091+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1870345181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089b3120 0x55ed0899dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.092+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1870345181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a47a0 0x55ed089ac640 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.092+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/1870345181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089a50f0 0x55ed089b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.092+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1870345181 >> 172.21.15.131:0/1870345181 conn(0x55ed08905a60 msgr2=0x55ed089abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:42.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.092+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1870345181 shutdown_connections 2024-05-10T05:38:42.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.092+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/1870345181 wait complete. 2024-05-10T05:38:42.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.093+0000 7f90ae8cfb80 1 -- start start 2024-05-10T05:38:42.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.093+0000 7f90ae8cfb80 1 Processor -- start 2024-05-10T05:38:42.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.095+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a47a0 0x55ed08a1a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.095+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a50f0 0x55ed08a1cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.095+0000 7f90ae8cfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089b3120 0x55ed08a1f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.096+0000 7f90ae8cfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed08a1f790 con 0x55ed089b3120 2024-05-10T05:38:42.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.096+0000 7f90ae8cfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed08a1f900 con 0x55ed089a50f0 2024-05-10T05:38:42.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.096+0000 7f90ae8cfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed08a1fac0 con 0x55ed089a47a0 2024-05-10T05:38:42.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.096+0000 7f908f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a50f0 0x55ed08a1cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.096+0000 7f90ae8cfb80 10 client.0 _ll_register_callbacks cb 0x55ed08868b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:42.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.096+0000 7f908f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a50f0 0x55ed08a1cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39406/0 (socket says 172.21.15.131:39406) 2024-05-10T05:38:42.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.096+0000 7f908f400700 1 -- 172.21.15.131:0/3781420730 learned_addr learned my addr 172.21.15.131:0/3781420730 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:42.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.096+0000 7f9094a00700 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089b3120 0x55ed08a1f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.096+0000 7f908fe00700 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a47a0 0x55ed08a1a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.097+0000 7f9094a00700 1 -- 172.21.15.131:0/3781420730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a47a0 msgr2=0x55ed08a1a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:42.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.097+0000 7f9094a00700 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a47a0 0x55ed08a1a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.097+0000 7f9094a00700 1 -- 172.21.15.131:0/3781420730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a50f0 msgr2=0x55ed08a1cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:42.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.097+0000 7f9094a00700 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a50f0 0x55ed08a1cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.097+0000 7f9094a00700 1 -- 172.21.15.131:0/3781420730 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed086f0b60 con 0x55ed089b3120 2024-05-10T05:38:42.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.097+0000 7f9079600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:38:42.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.098+0000 7f9094a00700 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089b3120 0x55ed08a1f140 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f907c01a9a0 tx=0x7f907c005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc9c682d2e92c85 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:42.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.098+0000 7f9079600700 1 -- 172.21.15.131:0/3781420730 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c003c80 con 0x55ed089b3120 2024-05-10T05:38:42.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.099+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/3781420730 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9080003c00 con 0x55ed089b3120 2024-05-10T05:38:42.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.099+0000 7f9079600700 1 -- 172.21.15.131:0/3781420730 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f907c003de0 con 0x55ed089b3120 2024-05-10T05:38:42.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.100+0000 7f9079600700 1 -- 172.21.15.131:0/3781420730 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f907c003c80 con 0x55ed089b3120 2024-05-10T05:38:42.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.100+0000 7f90ae8cfb80 10 client.16191 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:42.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.100+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/3781420730 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ed08748cd0 con 0x55ed089b3120 2024-05-10T05:38:42.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.101+0000 7f9079600700 1 -- 172.21.15.131:0/3781420730 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9080003c00 con 0x55ed089b3120 2024-05-10T05:38:42.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.104+0000 7f9079600700 1 -- 172.21.15.131:0/3781420730 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f907c0284e0 con 0x55ed089b3120 2024-05-10T05:38:42.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.104+0000 7f90ae8cfb80 10 client.16191 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:42.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.104+0000 7f90ae8cfb80 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:42.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.104+0000 7f90ae8cfb80 20 client.16191 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:42.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.104+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ed08a21450 0x55ed08a23870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:42.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.104+0000 7f90ae8cfb80 4 client.16191 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:38:42.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.104+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/3781420730 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed088bbec0 con 0x55ed08a21450 2024-05-10T05:38:42.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.105+0000 7f908fe00700 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ed08a21450 0x55ed08a23870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:42.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.106+0000 7f908fe00700 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ed08a21450 0x55ed08a23870 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d92a45b96a8400 server_cookie=dba311656f0d699f in_seq=0 out_seq=0 2024-05-10T05:38:42.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.106+0000 7f9079600700 10 client.16191 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:42.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.106+0000 7f9079600700 1 -- 172.21.15.131:0/3781420730 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed088bbec0 con 0x55ed08a21450 2024-05-10T05:38:42.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.106+0000 7f9079600700 10 client.16191 handle_command_reply: tid=0 2024-05-10T05:38:42.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.109+0000 7f90ae8cfb80 1 client.16191 shutdown 2024-05-10T05:38:42.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.110+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/3781420730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089b3120 msgr2=0x55ed08a1f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:42.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.110+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089b3120 0x55ed08a1f140 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f907c01a9a0 tx=0x7f907c005790 comp rx=0 tx=0).stop 2024-05-10T05:38:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.113+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/3781420730 shutdown_connections 2024-05-10T05:38:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.113+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ed08a21450 0x55ed08a23870 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.113+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed089b3120 0x55ed08a1f140 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.113+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed089a47a0 0x55ed08a1a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.113+0000 7f90ae8cfb80 1 --2- 172.21.15.131:0/3781420730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed089a50f0 0x55ed08a1cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.113+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/3781420730 >> 172.21.15.131:0/3781420730 conn(0x55ed08905a60 msgr2=0x55ed089972e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.121+0000 7f9079600700 0 client.16191 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.121+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/3781420730 shutdown_connections 2024-05-10T05:38:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.127+0000 7f90ae8cfb80 1 -- 172.21.15.131:0/3781420730 wait complete. 2024-05-10T05:38:42.131 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:38:42.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.128+0000 7f90ae8cfb80 20 client.16191 trim_cache size 0 max 16384 2024-05-10T05:38:42.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.136+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f90740a1230 msgr2=0x7f90740a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:42.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.136+0000 7f9095400700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f90740a1230 0x7f90740a3740 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9080006fd0 tx=0x7f9080008040 comp rx=0 tx=0).stop 2024-05-10T05:38:42.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.136+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9090109130 msgr2=0x7f90901b8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:42.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.136+0000 7f9095400700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9090109130 0x7f90901b8520 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f908800ba80 tx=0x7f908800bd90 comp rx=0 tx=0).stop 2024-05-10T05:38:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.143+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 shutdown_connections 2024-05-10T05:38:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.143+0000 7f9095400700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f90740a1230 0x7f90740a3740 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.143+0000 7f9095400700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9090109ad0 0x7f90901bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.143+0000 7f9095400700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9090108010 0x7f90901b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.143+0000 7f9095400700 1 --2- 172.21.15.131:0/3014049317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9090109130 0x7f90901b8520 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:42.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.143+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 >> 172.21.15.131:0/3014049317 conn(0x7f9090077040 msgr2=0x7f90900700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:42.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.149+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 shutdown_connections 2024-05-10T05:38:42.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:42.152+0000 7f9095400700 1 -- 172.21.15.131:0/3014049317 wait complete. 2024-05-10T05:38:42.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:42 smithi157 ceph-mon[78427]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 267 MiB/s wr, 84 op/s 2024-05-10T05:38:42.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:42 smithi142 ceph-mon[77667]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 267 MiB/s wr, 84 op/s 2024-05-10T05:38:42.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:42 smithi131 ceph-mon[37264]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 267 MiB/s wr, 84 op/s 2024-05-10T05:38:43.270 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:38:43.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:38:44.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.077+0000 7efc66c00700 1 -- 172.21.15.131:0/1694512064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc60108b60 msgr2=0x7efc6010fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:44.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.077+0000 7efc66c00700 1 --2- 172.21.15.131:0/1694512064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc60108b60 0x7efc6010fde0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efc500099c0 tx=0x7efc50009cd0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.077+0000 7efc66c00700 1 -- 172.21.15.131:0/1694512064 shutdown_connections 2024-05-10T05:38:44.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.077+0000 7efc66c00700 1 --2- 172.21.15.131:0/1694512064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc60110320 0x7efc60112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.077+0000 7efc66c00700 1 --2- 172.21.15.131:0/1694512064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc60108210 0x7efc60108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.077+0000 7efc66c00700 1 --2- 172.21.15.131:0/1694512064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc60108b60 0x7efc6010fde0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.078+0000 7efc66c00700 1 -- 172.21.15.131:0/1694512064 >> 172.21.15.131:0/1694512064 conn(0x7efc6006ea00 msgr2=0x7efc60070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.078+0000 7efc66c00700 1 -- 172.21.15.131:0/1694512064 shutdown_connections 2024-05-10T05:38:44.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.078+0000 7efc66c00700 1 -- 172.21.15.131:0/1694512064 wait complete. 2024-05-10T05:38:44.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.080+0000 7efc66c00700 1 Processor -- start 2024-05-10T05:38:44.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.081+0000 7efc66c00700 1 -- start start 2024-05-10T05:38:44.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.082+0000 7efc66c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc60108210 0x7efc601a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.082+0000 7efc66c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc60108b60 0x7efc601a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.082+0000 7efc66c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc60110320 0x7efc601a7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.084+0000 7efc66c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc601a81d0 con 0x7efc60110320 2024-05-10T05:38:44.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.084+0000 7efc66c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc601a8340 con 0x7efc60108b60 2024-05-10T05:38:44.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.084+0000 7efc66c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc601a8500 con 0x7efc60108210 2024-05-10T05:38:44.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.085+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc60108210 0x7efc601a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.085+0000 7efc5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc60108b60 0x7efc601a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.085+0000 7efc5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc60108b60 0x7efc601a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:49334/0 (socket says 172.21.15.131:49334) 2024-05-10T05:38:44.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.085+0000 7efc5f400700 1 -- 172.21.15.131:0/1750789306 learned_addr learned my addr 172.21.15.131:0/1750789306 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:44.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.085+0000 7efc5f400700 1 -- 172.21.15.131:0/1750789306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc60108210 msgr2=0x7efc601a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:44.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.085+0000 7efc5f400700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc60108210 0x7efc601a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.085+0000 7efc5f400700 1 -- 172.21.15.131:0/1750789306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc60110320 msgr2=0x7efc601a7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:44.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.085+0000 7efc5f400700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc60110320 0x7efc601a7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.085+0000 7efc5f400700 1 -- 172.21.15.131:0/1750789306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc500095f0 con 0x7efc60108b60 2024-05-10T05:38:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.086+0000 7efc5f400700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc60108b60 0x7efc601a0c30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efc50009fd0 tx=0x7efc50009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.086+0000 7efc5cc00700 1 -- 172.21.15.131:0/1750789306 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5001d070 con 0x7efc60108b60 2024-05-10T05:38:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.086+0000 7efc5cc00700 1 -- 172.21.15.131:0/1750789306 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc500054f0 con 0x7efc60108b60 2024-05-10T05:38:44.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.087+0000 7efc66c00700 1 -- 172.21.15.131:0/1750789306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc601a8780 con 0x7efc60108b60 2024-05-10T05:38:44.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.087+0000 7efc66c00700 1 -- 172.21.15.131:0/1750789306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc601a8ba0 con 0x7efc60108b60 2024-05-10T05:38:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.088+0000 7efc5cc00700 1 -- 172.21.15.131:0/1750789306 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5001d070 con 0x7efc60108b60 2024-05-10T05:38:44.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.097+0000 7efc5cc00700 1 -- 172.21.15.131:0/1750789306 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7efc500173f0 con 0x7efc60108b60 2024-05-10T05:38:44.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.099+0000 7efc5cc00700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efc440a1280 0x7efc440a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.100+0000 7efc5cc00700 1 -- 172.21.15.131:0/1750789306 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7efc5009a7c0 con 0x7efc60108b60 2024-05-10T05:38:44.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.100+0000 7efc5fe00700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efc440a1280 0x7efc440a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.105+0000 7efc7eb3ab80 1 Processor -- start 2024-05-10T05:38:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.106+0000 7efc7eb3ab80 1 -- start start 2024-05-10T05:38:44.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.108+0000 7efc5fe00700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efc440a1280 0x7efc440a3790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc601095f0 tx=0x7efc48009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:44.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.108+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4cf0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.108+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4cec480 0x55ceb4cee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.108+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4ceede0 0x55ceb4cef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.108+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4cf0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.109+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4cf0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35778/0 (socket says 172.21.15.131:35778) 2024-05-10T05:38:44.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.109+0000 7efc5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4cec480 0x55ceb4cee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.109+0000 7efc5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4cec480 0x55ceb4cee8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:49340/0 (socket says 172.21.15.131:49340) 2024-05-10T05:38:44.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.109+0000 7efc5fe00700 1 -- 172.21.15.131:0/2534745332 learned_addr learned my addr 172.21.15.131:0/2534745332 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:44.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.109+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/2534745332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4c111d0 con 0x55ceb4ceede0 2024-05-10T05:38:44.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.109+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/2534745332 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4c0eb90 con 0x55ceb4cec480 2024-05-10T05:38:44.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.109+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/2534745332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4c0a480 con 0x55ceb4cefcb0 2024-05-10T05:38:44.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.110+0000 7efc5f400700 1 -- 172.21.15.131:0/2534745332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 msgr2=0x55ceb4cf0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:44.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.110+0000 7efc5f400700 1 --2- 172.21.15.131:0/2534745332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4cf0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.110+0000 7efc5f400700 1 -- 172.21.15.131:0/2534745332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4ceede0 msgr2=0x55ceb4cef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:44.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.110+0000 7efc64c00700 1 --2- 172.21.15.131:0/2534745332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4ceede0 0x55ceb4cef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.110+0000 7efc5f400700 1 --2- 172.21.15.131:0/2534745332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4ceede0 0x55ceb4cef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.110+0000 7efc5f400700 1 -- 172.21.15.131:0/2534745332 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceb4a8b430 con 0x55ceb4cec480 2024-05-10T05:38:44.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.111+0000 7efc5f400700 1 --2- 172.21.15.131:0/2534745332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4cec480 0x55ceb4cee8a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efc50094c30 tx=0x7efc500a0f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a745f214251aee8d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:44.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.111+0000 7efc64c00700 1 --2- 172.21.15.131:0/2534745332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4ceede0 0x55ceb4cef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:44.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.113+0000 7efc55200700 1 -- 172.21.15.131:0/2534745332 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5001d070 con 0x55ceb4cec480 2024-05-10T05:38:44.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.114+0000 7efc55200700 1 -- 172.21.15.131:0/2534745332 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc500054f0 con 0x55ceb4cec480 2024-05-10T05:38:44.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.114+0000 7efc55200700 1 -- 172.21.15.131:0/2534745332 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500056f0 con 0x55ceb4cec480 2024-05-10T05:38:44.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.115+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/2534745332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4cec480 msgr2=0x55ceb4cee8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:44.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.115+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/2534745332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4cec480 0x55ceb4cee8a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efc50094c30 tx=0x7efc500a0f00 comp rx=0 tx=0).stop 2024-05-10T05:38:44.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.116+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/2534745332 shutdown_connections 2024-05-10T05:38:44.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.116+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/2534745332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4ceede0 0x55ceb4cef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.116+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/2534745332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4cf0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.116+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/2534745332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4cec480 0x55ceb4cee8a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.116+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/2534745332 >> 172.21.15.131:0/2534745332 conn(0x55ceb4c4cae0 msgr2=0x55ceb4ce5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:44.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.116+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/2534745332 shutdown_connections 2024-05-10T05:38:44.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.116+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/2534745332 wait complete. 2024-05-10T05:38:44.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.118+0000 7efc7eb3ab80 1 -- start start 2024-05-10T05:38:44.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.119+0000 7efc7eb3ab80 1 Processor -- start 2024-05-10T05:38:44.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.120+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4cfcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.120+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4cfd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.121+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4cfcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.121+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4d04040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.121+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4cfcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48682/0 (socket says 172.21.15.131:48682) 2024-05-10T05:38:44.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.121+0000 7efc7eb3ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4ba65e0 con 0x55ceb4cec480 2024-05-10T05:38:44.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.121+0000 7efc7eb3ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4ba6280 con 0x55ceb4ceede0 2024-05-10T05:38:44.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.121+0000 7efc5fe00700 1 -- 172.21.15.131:0/1741381169 learned_addr learned my addr 172.21.15.131:0/1741381169 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:44.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.121+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/1741381169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4a8d720 con 0x55ceb4cefcb0 2024-05-10T05:38:44.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.121+0000 7efc7eb3ab80 10 client.0 _ll_register_callbacks cb 0x55ceb4bafbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:44.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.121+0000 7efc5f400700 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4cfd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.123+0000 7efc5f400700 1 -- 172.21.15.131:0/1741381169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 msgr2=0x55ceb4d04040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:44.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.123+0000 7efc5f400700 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4d04040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.123+0000 7efc5f400700 1 -- 172.21.15.131:0/1741381169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 msgr2=0x55ceb4cfcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:44.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.124+0000 7efc5f400700 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4cfcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.124+0000 7efc5f400700 1 -- 172.21.15.131:0/1741381169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc601a8ba0 con 0x55ceb4ceede0 2024-05-10T05:38:44.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.126+0000 7efc5f400700 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4cfd0e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc5001f040 tx=0x7efc500a24b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ffdd4e3c350ea6f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:44.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.126+0000 7efc4d600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:38:44.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.127+0000 7efc4d600700 1 -- 172.21.15.131:0/1741381169 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5009d070 con 0x55ceb4ceede0 2024-05-10T05:38:44.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.128+0000 7efc4d600700 1 -- 172.21.15.131:0/1741381169 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7efc500a22d0 con 0x55ceb4ceede0 2024-05-10T05:38:44.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.128+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/1741381169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ceb4b632f0 con 0x55ceb4ceede0 2024-05-10T05:38:44.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.129+0000 7efc4d600700 1 -- 172.21.15.131:0/1741381169 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc5001d070 con 0x55ceb4ceede0 2024-05-10T05:38:44.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.130+0000 7efc7eb3ab80 10 client.26351 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:44.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.130+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/1741381169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ceb4a37be0 con 0x55ceb4ceede0 2024-05-10T05:38:44.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.345+0000 7efc4d600700 1 -- 172.21.15.131:0/1741381169 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ceb4b632f0 con 0x55ceb4ceede0 2024-05-10T05:38:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.347+0000 7efc4d600700 1 -- 172.21.15.131:0/1741381169 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efc500995b0 con 0x55ceb4ceede0 2024-05-10T05:38:44.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.347+0000 7efc7eb3ab80 10 client.26351 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:44.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.347+0000 7efc7eb3ab80 10 client.26351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:44.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.349+0000 7efc7eb3ab80 20 client.26351 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.349+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ceb4d08330 0x55ceb4d0a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.350+0000 7efc7eb3ab80 4 client.26351 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:44.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.350+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/1741381169 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ceb4cb9800 con 0x55ceb4d08330 2024-05-10T05:38:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.352+0000 7efc5fe00700 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ceb4d08330 0x55ceb4d0a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.353+0000 7efc5fe00700 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ceb4d08330 0x55ceb4d0a750 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7258268fa9385183 server_cookie=6044a116f24a562 in_seq=0 out_seq=0 2024-05-10T05:38:44.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.354+0000 7efc4d600700 10 client.26351 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:44.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.355+0000 7efc4d600700 1 -- 172.21.15.131:0/1741381169 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ceb4cb9800 con 0x55ceb4d08330 2024-05-10T05:38:44.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.355+0000 7efc4d600700 10 client.26351 handle_command_reply: tid=0 2024-05-10T05:38:44.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.355+0000 7efc7eb3ab80 1 client.26351 shutdown 2024-05-10T05:38:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.356+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/1741381169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 msgr2=0x55ceb4cfd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.356+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4cfd0e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc5001f040 tx=0x7efc500a24b0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.360+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/1741381169 shutdown_connections 2024-05-10T05:38:44.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.360+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ceb4d08330 0x55ceb4d0a750 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.360+0000 7efc4d600700 0 client.26351 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:44.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.361+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4cfcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.361+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4d04040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.361+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/1741381169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4cfd0e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.361+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/1741381169 >> 172.21.15.131:0/1741381169 conn(0x55ceb4c4cae0 msgr2=0x55ceb4ce5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:44.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.361+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/1741381169 shutdown_connections 2024-05-10T05:38:44.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.362+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/1741381169 wait complete. 2024-05-10T05:38:44.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.364+0000 7efc7eb3ab80 20 client.26351 trim_cache size 0 max 16384 2024-05-10T05:38:44.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.419+0000 7efc7eb3ab80 1 Processor -- start 2024-05-10T05:38:44.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.420+0000 7efc7eb3ab80 1 -- start start 2024-05-10T05:38:44.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.423+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cec480 0x55ceb4ce5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.423+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d04000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.424+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cefcb0 0x55ceb4ceb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.424+0000 7efc5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d04000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.424+0000 7efc5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d04000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:49360/0 (socket says 172.21.15.131:49360) 2024-05-10T05:38:44.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.424+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cec480 0x55ceb4ce5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.424+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cec480 0x55ceb4ce5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35794/0 (socket says 172.21.15.131:35794) 2024-05-10T05:38:44.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.424+0000 7efc5f400700 1 -- 172.21.15.131:0/777456512 learned_addr learned my addr 172.21.15.131:0/777456512 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:44.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.424+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/777456512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4ce1db0 con 0x55ceb4cefcb0 2024-05-10T05:38:44.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.424+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/777456512 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4ce7320 con 0x55ceb4ceede0 2024-05-10T05:38:44.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.424+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/777456512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4cdf840 con 0x55ceb4cec480 2024-05-10T05:38:44.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.425+0000 7efc5f400700 1 -- 172.21.15.131:0/777456512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cec480 msgr2=0x55ceb4ce5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:44.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.425+0000 7efc5f400700 1 --2- 172.21.15.131:0/777456512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cec480 0x55ceb4ce5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.425+0000 7efc5f400700 1 -- 172.21.15.131:0/777456512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cefcb0 msgr2=0x55ceb4ceb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:44.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.425+0000 7efc5f400700 1 --2- 172.21.15.131:0/777456512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cefcb0 0x55ceb4ceb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.425+0000 7efc5f400700 1 -- 172.21.15.131:0/777456512 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceb4ce14d0 con 0x55ceb4ceede0 2024-05-10T05:38:44.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.425+0000 7efc5f400700 1 --2- 172.21.15.131:0/777456512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d04000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efc50099b90 tx=0x7efc50098670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49c2b31b73f02ab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:44.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.426+0000 7efc55c00700 1 -- 172.21.15.131:0/777456512 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5009d070 con 0x55ceb4ceede0 2024-05-10T05:38:44.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.426+0000 7efc55c00700 1 -- 172.21.15.131:0/777456512 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc500b2be0 con 0x55ceb4ceede0 2024-05-10T05:38:44.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.426+0000 7efc55c00700 1 -- 172.21.15.131:0/777456512 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5001d020 con 0x55ceb4ceede0 2024-05-10T05:38:44.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.428+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/777456512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 msgr2=0x55ceb4d04000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:44.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.428+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/777456512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d04000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efc50099b90 tx=0x7efc50098670 comp rx=0 tx=0).stop 2024-05-10T05:38:44.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.434+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/777456512 shutdown_connections 2024-05-10T05:38:44.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.434+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/777456512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cefcb0 0x55ceb4ceb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.434+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/777456512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cec480 0x55ceb4ce5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.434+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/777456512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d04000 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.434+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/777456512 >> 172.21.15.131:0/777456512 conn(0x55ceb4c4cae0 msgr2=0x55ceb4d0a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:44.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.434+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/777456512 shutdown_connections 2024-05-10T05:38:44.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.435+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/777456512 wait complete. 2024-05-10T05:38:44.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.436+0000 7efc7eb3ab80 1 -- start start 2024-05-10T05:38:44.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.439+0000 7efc7eb3ab80 1 Processor -- start 2024-05-10T05:38:44.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.441+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4d01fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.442+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d024f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.442+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4d01fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.443+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4d01fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48698/0 (socket says 172.21.15.131:48698) 2024-05-10T05:38:44.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.443+0000 7efc7eb3ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4d03420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.443+0000 7efc5fe00700 1 -- 172.21.15.131:0/3144689395 learned_addr learned my addr 172.21.15.131:0/3144689395 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:44.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.443+0000 7efc64c00700 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4d03420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.443+0000 7efc5f400700 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.443+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4d03960 con 0x55ceb4cec480 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.444+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4d03ad0 con 0x55ceb4ceede0 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.444+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb4d03c90 con 0x55ceb4cefcb0 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.444+0000 7efc5f400700 1 -- 172.21.15.131:0/3144689395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 msgr2=0x55ceb4d03420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.444+0000 7efc5f400700 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4d03420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.444+0000 7efc5f400700 1 -- 172.21.15.131:0/3144689395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 msgr2=0x55ceb4d01fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.444+0000 7efc5f400700 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4d01fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.444+0000 7efc5f400700 1 -- 172.21.15.131:0/3144689395 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ceb4a37be0 con 0x55ceb4ceede0 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.445+0000 7efc5fe00700 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4d01fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.445+0000 7efc5f400700 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d024f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efc5000bf40 tx=0x7efc50099ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e97a9866b97042c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.445+0000 7efc64c00700 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4d03420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:38:44.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.445+0000 7efc4d600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:38:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.445+0000 7efc4d600700 1 -- 172.21.15.131:0/3144689395 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5009d070 con 0x55ceb4ceede0 2024-05-10T05:38:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.446+0000 7efc4d600700 1 -- 172.21.15.131:0/3144689395 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7efc500050d0 con 0x55ceb4ceede0 2024-05-10T05:38:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.446+0000 7efc4d600700 10 client.0 inactive, discarding osd_map(82..82 src has 1..82) v4 2024-05-10T05:38:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.446+0000 7efc7eb3ab80 10 client.0 _ll_register_callbacks cb 0x55ceb4bafbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.446+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc500054f0 con 0x55ceb4ceede0 2024-05-10T05:38:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.447+0000 7efc4d600700 1 -- 172.21.15.131:0/3144689395 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc5001d070 con 0x55ceb4ceede0 2024-05-10T05:38:44.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.451+0000 7efc7eb3ab80 10 client.26357 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:44.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.452+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55ceb4bd05e0 con 0x55ceb4ceede0 2024-05-10T05:38:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.452+0000 7efc4d600700 1 -- 172.21.15.131:0/3144689395 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efc5000fbc0 con 0x55ceb4ceede0 2024-05-10T05:38:44.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.452+0000 7efc7eb3ab80 10 client.26357 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:44.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.452+0000 7efc7eb3ab80 10 client.26357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:44.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.452+0000 7efc7eb3ab80 20 client.26357 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:44.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.453+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ceb4d08330 0x55ceb4d61050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:44.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.453+0000 7efc7eb3ab80 4 client.26357 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:38:44.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.453+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7efc500054f0 con 0x55ceb4d08330 2024-05-10T05:38:44.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.453+0000 7efc5fe00700 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ceb4d08330 0x55ceb4d61050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:44.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.457+0000 7efc5fe00700 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ceb4d08330 0x55ceb4d61050 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66fb1f2645c569 server_cookie=f2d0bb0a000ffa18 in_seq=0 out_seq=0 2024-05-10T05:38:44.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.458+0000 7efc4d600700 10 client.26357 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:44.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.458+0000 7efc4d600700 1 -- 172.21.15.131:0/3144689395 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7efc500054f0 con 0x55ceb4d08330 2024-05-10T05:38:44.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.458+0000 7efc4d600700 10 client.26357 handle_command_reply: tid=0 2024-05-10T05:38:44.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.459+0000 7efc7eb3ab80 1 client.26357 shutdown 2024-05-10T05:38:44.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.460+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 msgr2=0x55ceb4d024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:44.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.460+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d024f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efc5000bf40 tx=0x7efc50099ac0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.461+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 shutdown_connections 2024-05-10T05:38:44.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.461+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ceb4d08330 0x55ceb4d61050 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.461+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ceb4cec480 0x55ceb4d01fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.461+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ceb4cefcb0 0x55ceb4d03420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.461+0000 7efc7eb3ab80 1 --2- 172.21.15.131:0/3144689395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ceb4ceede0 0x55ceb4d024f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.461+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 >> 172.21.15.131:0/3144689395 conn(0x55ceb4c4cae0 msgr2=0x55ceb4ce5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:44.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.461+0000 7efc4d600700 0 client.26357 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:44.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.461+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 shutdown_connections 2024-05-10T05:38:44.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.461+0000 7efc7eb3ab80 1 -- 172.21.15.131:0/3144689395 wait complete. 2024-05-10T05:38:44.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.462+0000 7efc7eb3ab80 20 client.26357 trim_cache size 0 max 16384 2024-05-10T05:38:44.466 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:38:44.466 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:38:44.466 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "61e3953a-e5e1-4bdb-af5d-c9949509b2bd", 2024-05-10T05:38:44.466 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:38:44.466 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:38:44.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.465+0000 7efc66c00700 1 -- 172.21.15.131:0/1750789306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efc440a1280 msgr2=0x7efc440a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:44.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.466+0000 7efc66c00700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efc440a1280 0x7efc440a3790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc601095f0 tx=0x7efc48009260 comp rx=0 tx=0).stop 2024-05-10T05:38:44.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.466+0000 7efc66c00700 1 -- 172.21.15.131:0/1750789306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc60108b60 msgr2=0x7efc601a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:44.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.466+0000 7efc66c00700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc60108b60 0x7efc601a0c30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efc50009fd0 tx=0x7efc50009580 comp rx=0 tx=0).stop 2024-05-10T05:38:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.467+0000 7efc66c00700 1 -- 172.21.15.131:0/1750789306 shutdown_connections 2024-05-10T05:38:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.467+0000 7efc66c00700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7efc440a1280 0x7efc440a3790 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.467+0000 7efc66c00700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc60110320 0x7efc601a7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.467+0000 7efc66c00700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc60108210 0x7efc601a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.467+0000 7efc66c00700 1 --2- 172.21.15.131:0/1750789306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc60108b60 0x7efc601a0c30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.467+0000 7efc66c00700 1 -- 172.21.15.131:0/1750789306 >> 172.21.15.131:0/1750789306 conn(0x7efc6006ea00 msgr2=0x7efc6006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.468+0000 7efc66c00700 1 -- 172.21.15.131:0/1750789306 shutdown_connections 2024-05-10T05:38:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:44.468+0000 7efc66c00700 1 -- 172.21.15.131:0/1750789306 wait complete. 2024-05-10T05:38:44.543 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:44 smithi131 ceph-mon[37264]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 212 MiB/s wr, 67 op/s 2024-05-10T05:38:44.553 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:38:44.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:44 smithi157 ceph-mon[78427]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 212 MiB/s wr, 67 op/s 2024-05-10T05:38:44.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:44 smithi142 ceph-mon[77667]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 212 MiB/s wr, 67 op/s 2024-05-10T05:38:45.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.394+0000 7fc2b1e00700 1 -- 172.21.15.131:0/1854722970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ac108b60 msgr2=0x7fc2ac10fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:45.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.394+0000 7fc2b1e00700 1 --2- 172.21.15.131:0/1854722970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ac108b60 0x7fc2ac10fde0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009fc0 tx=0x7fc29c00b330 comp rx=0 tx=0).stop 2024-05-10T05:38:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.397+0000 7fc2b1e00700 1 -- 172.21.15.131:0/1854722970 shutdown_connections 2024-05-10T05:38:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.398+0000 7fc2b1e00700 1 --2- 172.21.15.131:0/1854722970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2ac110320 0x7fc2ac112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.398+0000 7fc2b1e00700 1 --2- 172.21.15.131:0/1854722970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac108210 0x7fc2ac108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.398+0000 7fc2b1e00700 1 --2- 172.21.15.131:0/1854722970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ac108b60 0x7fc2ac10fde0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.398+0000 7fc2b1e00700 1 -- 172.21.15.131:0/1854722970 >> 172.21.15.131:0/1854722970 conn(0x7fc2ac06ea00 msgr2=0x7fc2ac070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.400+0000 7fc2b1e00700 1 -- 172.21.15.131:0/1854722970 shutdown_connections 2024-05-10T05:38:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.400+0000 7fc2b1e00700 1 -- 172.21.15.131:0/1854722970 wait complete. 2024-05-10T05:38:45.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.409+0000 7fc2b1e00700 1 Processor -- start 2024-05-10T05:38:45.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.411+0000 7fc2b1e00700 1 -- start start 2024-05-10T05:38:45.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.415+0000 7fc2b1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ac108210 0x7fc2ac10b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.415+0000 7fc2b1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac108b60 0x7fc2ac109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.415+0000 7fc2b1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2ac110320 0x7fc2ac10a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.415+0000 7fc2b1e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac10a870 con 0x7fc2ac110320 2024-05-10T05:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.415+0000 7fc2b1e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac10a9e0 con 0x7fc2ac108210 2024-05-10T05:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.415+0000 7fc2b1e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac10aba0 con 0x7fc2ac108b60 2024-05-10T05:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.415+0000 7fc2a3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac108b60 0x7fc2ac109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.415+0000 7fc2a3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac108b60 0x7fc2ac109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35826/0 (socket says 172.21.15.131:35826) 2024-05-10T05:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.416+0000 7fc2a3e00700 1 -- 172.21.15.131:0/2185537932 learned_addr learned my addr 172.21.15.131:0/2185537932 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.416+0000 7fc2a3e00700 1 -- 172.21.15.131:0/2185537932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ac108210 msgr2=0x7fc2ac10b630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:38:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.416+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ac108210 0x7fc2ac10b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.416+0000 7fc2a3e00700 1 -- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2ac110320 msgr2=0x7fc2ac10a210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:38:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.418+0000 7fc2ab400700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2ac110320 0x7fc2ac10a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.418+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2ac110320 0x7fc2ac10a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.418+0000 7fc2a3e00700 1 -- 172.21.15.131:0/2185537932 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2940095f0 con 0x7fc2ac108b60 2024-05-10T05:38:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.418+0000 7fc2ab400700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2ac110320 0x7fc2ac10a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:45.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.420+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac108b60 0x7fc2ac109cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0060b0 tx=0x7fc29c003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:45.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.421+0000 7fc2b0a00700 1 -- 172.21.15.131:0/2185537932 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00e070 con 0x7fc2ac108b60 2024-05-10T05:38:45.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.421+0000 7fc2b0a00700 1 -- 172.21.15.131:0/2185537932 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc29c007e80 con 0x7fc2ac108b60 2024-05-10T05:38:45.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.421+0000 7fc2b0a00700 1 -- 172.21.15.131:0/2185537932 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c01b7d0 con 0x7fc2ac108b60 2024-05-10T05:38:45.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.421+0000 7fc2b1e00700 1 -- 172.21.15.131:0/2185537932 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c009c40 con 0x7fc2ac108b60 2024-05-10T05:38:45.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.422+0000 7fc2b1e00700 1 -- 172.21.15.131:0/2185537932 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac1aad70 con 0x7fc2ac108b60 2024-05-10T05:38:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.428+0000 7fc2b0a00700 1 -- 172.21.15.131:0/2185537932 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc29c01b930 con 0x7fc2ac108b60 2024-05-10T05:38:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.430+0000 7fc2c9c8fb80 1 Processor -- start 2024-05-10T05:38:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.430+0000 7fc2b0a00700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc2900a1240 0x7fc2900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.431+0000 7fc2b0a00700 1 -- 172.21.15.131:0/2185537932 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc29c0cd1e0 con 0x7fc2ac108b60 2024-05-10T05:38:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.431+0000 7fc2c9c8fb80 1 -- start start 2024-05-10T05:38:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.431+0000 7fc2ab400700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc2900a1240 0x7fc2900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.431+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e36684300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.431+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36684d80 0x564e36685190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.432+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e366856d0 0x564e3668bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.432+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e365a2150 con 0x564e36684d80 2024-05-10T05:38:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.432+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3659fb10 con 0x564e366856d0 2024-05-10T05:38:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.432+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3659b400 con 0x564e36683eb0 2024-05-10T05:38:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.432+0000 7fc2aaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e36684300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.432+0000 7fc2aaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e36684300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35842/0 (socket says 172.21.15.131:35842) 2024-05-10T05:38:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.432+0000 7fc2aaa00700 1 -- 172.21.15.131:0/2671469608 learned_addr learned my addr 172.21.15.131:0/2671469608 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.432+0000 7fc2ab400700 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e366856d0 0x564e3668bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.432+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36684d80 0x564e36685190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.433+0000 7fc2aaa00700 1 -- 172.21.15.131:0/2671469608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e366856d0 msgr2=0x564e3668bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.433+0000 7fc2aaa00700 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e366856d0 0x564e3668bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.433+0000 7fc2aaa00700 1 -- 172.21.15.131:0/2671469608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36684d80 msgr2=0x564e36685190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:45.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.433+0000 7fc2aaa00700 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36684d80 0x564e36685190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.433+0000 7fc2aaa00700 1 -- 172.21.15.131:0/2671469608 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e3641c3b0 con 0x564e36683eb0 2024-05-10T05:38:45.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.434+0000 7fc2ab400700 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e366856d0 0x564e3668bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:38:45.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.434+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36684d80 0x564e36685190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:38:45.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.434+0000 7fc2ab400700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc2900a1240 0x7fc2900a3750 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4008fd0 tx=0x7fc2a4008a30 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:45.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.434+0000 7fc2aaa00700 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e36684300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc294009e60 tx=0x7fc29400e630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8439f03ddeed3f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:45.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.434+0000 7fc29be00700 1 -- 172.21.15.131:0/2671469608 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29400ed90 con 0x564e36683eb0 2024-05-10T05:38:45.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.434+0000 7fc29be00700 1 -- 172.21.15.131:0/2671469608 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2940044f0 con 0x564e36683eb0 2024-05-10T05:38:45.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.435+0000 7fc29be00700 1 -- 172.21.15.131:0/2671469608 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294010430 con 0x564e36683eb0 2024-05-10T05:38:45.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.436+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2671469608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 msgr2=0x564e36684300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:45.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.436+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e36684300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc294009e60 tx=0x7fc29400e630 comp rx=0 tx=0).stop 2024-05-10T05:38:45.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.436+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2671469608 shutdown_connections 2024-05-10T05:38:45.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.436+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e366856d0 0x564e3668bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.436+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e36684300 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.436+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/2671469608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36684d80 0x564e36685190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.436+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2671469608 >> 172.21.15.131:0/2671469608 conn(0x564e365dda60 msgr2=0x564e36675fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:45.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.437+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2671469608 shutdown_connections 2024-05-10T05:38:45.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.437+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2671469608 wait complete. 2024-05-10T05:38:45.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.438+0000 7fc2c9c8fb80 1 -- start start 2024-05-10T05:38:45.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.438+0000 7fc2c9c8fb80 1 Processor -- start 2024-05-10T05:38:45.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.439+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e3667cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.439+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3667d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.439+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e366856d0 0x564e36681f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.439+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e36537560 con 0x564e366856d0 2024-05-10T05:38:45.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.439+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e36537200 con 0x564e36684d80 2024-05-10T05:38:45.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.439+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3641e6a0 con 0x564e36683eb0 2024-05-10T05:38:45.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.439+0000 7fc2c9c8fb80 10 client.0 _ll_register_callbacks cb 0x564e36540b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:45.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.440+0000 7fc2aaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e3667cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.440+0000 7fc2a3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3667d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.440+0000 7fc2a3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3667d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:49396/0 (socket says 172.21.15.131:49396) 2024-05-10T05:38:45.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.440+0000 7fc2a3e00700 1 -- 172.21.15.131:0/3062288748 learned_addr learned my addr 172.21.15.131:0/3062288748 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:45.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.440+0000 7fc2ab400700 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e366856d0 0x564e36681f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc2a3e00700 1 -- 172.21.15.131:0/3062288748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 msgr2=0x564e3667cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:45.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e3667cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc2a3e00700 1 -- 172.21.15.131:0/3062288748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e366856d0 msgr2=0x564e36681f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e366856d0 0x564e36681f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc2a3e00700 1 -- 172.21.15.131:0/3062288748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a400f520 con 0x564e36684d80 2024-05-10T05:38:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc2ab400700 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e366856d0 0x564e36681f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3667d000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc29c096bc0 tx=0x7fc29c09adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc1edc42cbdcba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc298c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:38:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc298c00700 1 -- 172.21.15.131:0/3062288748 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c09a6c0 con 0x564e36684d80 2024-05-10T05:38:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc298c00700 1 -- 172.21.15.131:0/3062288748 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc29c09e4f0 con 0x564e36684d80 2024-05-10T05:38:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.441+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/3062288748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e364f4270 con 0x564e36684d80 2024-05-10T05:38:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.442+0000 7fc298c00700 1 -- 172.21.15.131:0/3062288748 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc29c0a2ab0 con 0x564e36684d80 2024-05-10T05:38:45.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.442+0000 7fc2c9c8fb80 10 client.26366 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:45.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.442+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/3062288748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fc294009b10 con 0x564e36684d80 2024-05-10T05:38:45.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.442+0000 7fc298c00700 1 -- 172.21.15.131:0/3062288748 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e364f4270 con 0x564e36684d80 2024-05-10T05:38:45.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.443+0000 7fc298c00700 1 -- 172.21.15.131:0/3062288748 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc29c04f020 con 0x564e36684d80 2024-05-10T05:38:45.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.443+0000 7fc2c9c8fb80 10 client.26366 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:45.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.443+0000 7fc2c9c8fb80 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:45.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.443+0000 7fc2c9c8fb80 20 client.26366 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:45.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.443+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564e366a1f60 0x564e366a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.443+0000 7fc2c9c8fb80 4 client.26366 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:38:45.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.443+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/3062288748 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e3664a780 con 0x564e366a1f60 2024-05-10T05:38:45.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.443+0000 7fc2aaa00700 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564e366a1f60 0x564e366a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.444+0000 7fc2aaa00700 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564e366a1f60 0x564e366a43b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b84c20243cc13d server_cookie=502a74bd6604101b in_seq=0 out_seq=0 2024-05-10T05:38:45.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.444+0000 7fc298c00700 10 client.26366 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:45.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc298c00700 1 -- 172.21.15.131:0/3062288748 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e3664a780 con 0x564e366a1f60 2024-05-10T05:38:45.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc298c00700 10 client.26366 handle_command_reply: tid=0 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc2c9c8fb80 1 client.26366 shutdown 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/3062288748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 msgr2=0x564e3667d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3667d000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc29c096bc0 tx=0x7fc29c09adc0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/3062288748 shutdown_connections 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564e366a1f60 0x564e366a43b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e366856d0 0x564e36681f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e3667cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/3062288748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3667d000 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/3062288748 >> 172.21.15.131:0/3062288748 conn(0x564e365dda60 msgr2=0x564e36675fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.445+0000 7fc298c00700 0 client.26366 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.446+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/3062288748 shutdown_connections 2024-05-10T05:38:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.446+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/3062288748 wait complete. 2024-05-10T05:38:45.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.446+0000 7fc2c9c8fb80 20 client.26366 trim_cache size 0 max 16384 2024-05-10T05:38:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.482+0000 7fc2c9c8fb80 1 Processor -- start 2024-05-10T05:38:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.483+0000 7fc2c9c8fb80 1 -- start start 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.484+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e366a8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.484+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3668b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.484+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e366856d0 0x564e3668bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.484+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e366707c0 con 0x564e366856d0 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.484+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e36675c90 con 0x564e36684d80 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.484+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3667acf0 con 0x564e36683eb0 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.484+0000 7fc2aaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e366a8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.484+0000 7fc2aaa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e366a8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:35856/0 (socket says 172.21.15.131:35856) 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.484+0000 7fc2aaa00700 1 -- 172.21.15.131:0/1187844035 learned_addr learned my addr 172.21.15.131:0/1187844035 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.484+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/1187844035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3668b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.485+0000 7fc2a3e00700 1 -- 172.21.15.131:0/1187844035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 msgr2=0x564e366a8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:45.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.485+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/1187844035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e366a8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.485+0000 7fc2a3e00700 1 -- 172.21.15.131:0/1187844035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e366856d0 msgr2=0x564e3668bee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:38:45.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.485+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/1187844035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e366856d0 0x564e3668bee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.485+0000 7fc2a3e00700 1 -- 172.21.15.131:0/1187844035 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e363c8b60 con 0x564e36684d80 2024-05-10T05:38:45.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.485+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/1187844035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3668b9a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc29c097130 tx=0x7fc29c0cad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8f762ecaf4b322 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:45.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.485+0000 7fc2a1200700 1 -- 172.21.15.131:0/1187844035 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c09e4f0 con 0x564e36684d80 2024-05-10T05:38:45.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.485+0000 7fc2a1200700 1 -- 172.21.15.131:0/1187844035 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc29c09a6c0 con 0x564e36684d80 2024-05-10T05:38:45.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.485+0000 7fc2a1200700 1 -- 172.21.15.131:0/1187844035 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c09e170 con 0x564e36684d80 2024-05-10T05:38:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.487+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/1187844035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 msgr2=0x564e3668b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.487+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/1187844035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3668b9a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc29c097130 tx=0x7fc29c0cad30 comp rx=0 tx=0).stop 2024-05-10T05:38:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.487+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/1187844035 shutdown_connections 2024-05-10T05:38:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.487+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/1187844035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e366856d0 0x564e3668bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.487+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/1187844035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e36683eb0 0x564e366a8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.487+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/1187844035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3668b9a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.487+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/1187844035 >> 172.21.15.131:0/1187844035 conn(0x564e365dda60 msgr2=0x564e36676c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.488+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/1187844035 shutdown_connections 2024-05-10T05:38:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.488+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/1187844035 wait complete. 2024-05-10T05:38:45.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.488+0000 7fc2c9c8fb80 1 -- start start 2024-05-10T05:38:45.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.489+0000 7fc2c9c8fb80 1 Processor -- start 2024-05-10T05:38:45.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.493+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36683eb0 0x564e36689ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.494+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3668a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.494+0000 7fc2c9c8fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e366856d0 0x564e366f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.494+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3668a980 con 0x564e36683eb0 2024-05-10T05:38:45.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.494+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3668aaf0 con 0x564e36684d80 2024-05-10T05:38:45.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.494+0000 7fc2c9c8fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e366f6900 con 0x564e366856d0 2024-05-10T05:38:45.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.494+0000 7fc2c9c8fb80 10 client.0 _ll_register_callbacks cb 0x564e36682430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:38:45.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.494+0000 7fc2aaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36683eb0 0x564e36689ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.494+0000 7fc2aaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36683eb0 0x564e36689ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48742/0 (socket says 172.21.15.131:48742) 2024-05-10T05:38:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.494+0000 7fc2aaa00700 1 -- 172.21.15.131:0/2684336084 learned_addr learned my addr 172.21.15.131:0/2684336084 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:38:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.495+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3668a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.495+0000 7fc2ab400700 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e366856d0 0x564e366f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.495+0000 7fc2aaa00700 1 -- 172.21.15.131:0/2684336084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e366856d0 msgr2=0x564e366f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.495+0000 7fc2aaa00700 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e366856d0 0x564e366f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.495+0000 7fc2aaa00700 1 -- 172.21.15.131:0/2684336084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 msgr2=0x564e3668a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:38:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.495+0000 7fc2aaa00700 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3668a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.495+0000 7fc2aaa00700 1 -- 172.21.15.131:0/2684336084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc294009b10 con 0x564e36683eb0 2024-05-10T05:38:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.495+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3668a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:38:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.495+0000 7fc2ab400700 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e366856d0 0x564e366f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:38:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.496+0000 7fc2aaa00700 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36683eb0 0x564e36689ce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc294009e30 tx=0x7fc294003810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4b7c548272e74e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:38:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.496+0000 7fc298c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:38:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.496+0000 7fc298c00700 1 -- 172.21.15.131:0/2684336084 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294003cb0 con 0x564e36683eb0 2024-05-10T05:38:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.496+0000 7fc298c00700 1 -- 172.21.15.131:0/2684336084 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc294014070 con 0x564e36683eb0 2024-05-10T05:38:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.497+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2684336084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2940044f0 con 0x564e36683eb0 2024-05-10T05:38:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.497+0000 7fc298c00700 1 -- 172.21.15.131:0/2684336084 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc294003e10 con 0x564e36683eb0 2024-05-10T05:38:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.497+0000 7fc2c9c8fb80 10 client.16206 fetch_fsmap learned FSMap version 29 2024-05-10T05:38:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.498+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2684336084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564e36561560 con 0x564e36683eb0 2024-05-10T05:38:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.498+0000 7fc298c00700 1 -- 172.21.15.131:0/2684336084 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2940044f0 con 0x564e36683eb0 2024-05-10T05:38:45.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.498+0000 7fc298c00700 1 -- 172.21.15.131:0/2684336084 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc2940046f0 con 0x564e36683eb0 2024-05-10T05:38:45.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.499+0000 7fc2c9c8fb80 10 client.16206 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:38:45.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.499+0000 7fc2c9c8fb80 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:38:45.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.499+0000 7fc2c9c8fb80 20 client.16206 populate_metadata read hostname 'smithi131' 2024-05-10T05:38:45.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.499+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564e366f8230 0x564e366fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:38:45.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.499+0000 7fc2c9c8fb80 4 client.16206 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:38:45.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.499+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2684336084 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e36593ec0 con 0x564e366f8230 2024-05-10T05:38:45.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.499+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564e366f8230 0x564e366fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:38:45.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.500+0000 7fc298c00700 10 client.16206 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:45.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.500+0000 7fc2a3e00700 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564e366f8230 0x564e366fa650 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1981d594cfd357e3 server_cookie=fbe500cfffa080a4 in_seq=0 out_seq=0 2024-05-10T05:38:45.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.500+0000 7fc298c00700 1 -- 172.21.15.131:0/2684336084 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564e36593ec0 con 0x564e366f8230 2024-05-10T05:38:45.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.500+0000 7fc298c00700 10 client.16206 handle_command_reply: tid=0 2024-05-10T05:38:45.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.500+0000 7fc2c9c8fb80 1 client.16206 shutdown 2024-05-10T05:38:45.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.501+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2684336084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36683eb0 msgr2=0x564e36689ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:45.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.501+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36683eb0 0x564e36689ce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc294009e30 tx=0x7fc294003810 comp rx=0 tx=0).stop 2024-05-10T05:38:45.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.501+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2684336084 shutdown_connections 2024-05-10T05:38:45.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.501+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564e366f8230 0x564e366fa650 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.501+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e36683eb0 0x564e36689ce0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.501+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564e366856d0 0x564e366f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.501+0000 7fc2c9c8fb80 1 --2- 172.21.15.131:0/2684336084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e36684d80 0x564e3668a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.501+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2684336084 >> 172.21.15.131:0/2684336084 conn(0x564e365dda60 msgr2=0x564e366842c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:45.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.501+0000 7fc298c00700 0 client.16206 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:38:45.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.501+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2684336084 shutdown_connections 2024-05-10T05:38:45.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.502+0000 7fc2c9c8fb80 1 -- 172.21.15.131:0/2684336084 wait complete. 2024-05-10T05:38:45.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.502+0000 7fc2c9c8fb80 20 client.16206 trim_cache size 0 max 16384 2024-05-10T05:38:45.503 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:38:45.504 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:38:45.504 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:38:45.504 INFO:teuthology.orchestra.run.smithi131.stdout: "61e3953a-e5e1-4bdb-af5d-c9949509b2bd": { 2024-05-10T05:38:45.504 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:38:45.504 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "61e3953a-e5e1-4bdb-af5d-c9949509b2bd", 2024-05-10T05:38:45.504 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:38:45.504 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:38:45.504 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:38:45.504 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:38:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.503+0000 7fc2b1e00700 1 -- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc2900a1240 msgr2=0x7fc2900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.503+0000 7fc2b1e00700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc2900a1240 0x7fc2900a3750 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4008fd0 tx=0x7fc2a4008a30 comp rx=0 tx=0).stop 2024-05-10T05:38:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.503+0000 7fc2b1e00700 1 -- 172.21.15.131:0/2185537932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac108b60 msgr2=0x7fc2ac109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:38:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.503+0000 7fc2b1e00700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac108b60 0x7fc2ac109cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0060b0 tx=0x7fc29c003d60 comp rx=0 tx=0).stop 2024-05-10T05:38:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.503+0000 7fc2b1e00700 1 -- 172.21.15.131:0/2185537932 shutdown_connections 2024-05-10T05:38:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.503+0000 7fc2b1e00700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc2900a1240 0x7fc2900a3750 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.503+0000 7fc2b1e00700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2ac110320 0x7fc2ac10a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.504+0000 7fc2b1e00700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ac108210 0x7fc2ac10b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.504+0000 7fc2b1e00700 1 --2- 172.21.15.131:0/2185537932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2ac108b60 0x7fc2ac109cd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:38:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.504+0000 7fc2b1e00700 1 -- 172.21.15.131:0/2185537932 >> 172.21.15.131:0/2185537932 conn(0x7fc2ac06ea00 msgr2=0x7fc2ac06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:38:45.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.504+0000 7fc2b1e00700 1 -- 172.21.15.131:0/2185537932 shutdown_connections 2024-05-10T05:38:45.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:38:45.504+0000 7fc2b1e00700 1 -- 172.21.15.131:0/2185537932 wait complete. 2024-05-10T05:38:45.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:45 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:38:45.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:45 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:45.554 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:45 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:38:45.554 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:45 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:45.558 INFO:tasks.cephfs.filesystem:scrub status for tag:61e3953a-e5e1-4bdb-af5d-c9949509b2bd - {'path': '/', 'tag': '61e3953a-e5e1-4bdb-af5d-c9949509b2bd', 'options': 'recursive,force'} 2024-05-10T05:38:45.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:45 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:38:45.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:45 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:46.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:46 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:38:46.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:46 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:46.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:46 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:38:46.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:46 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:46.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:46 smithi157 ceph-mon[78427]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 84 op/s 2024-05-10T05:38:46.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:46 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:38:46.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:46 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:46.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:46 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:38:46.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:46 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:46.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:46 smithi142 ceph-mon[77667]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 84 op/s 2024-05-10T05:38:46.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:46 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:38:46.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:46 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:46.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:46 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:38:46.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:46 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:38:46.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:46 smithi131 ceph-mon[37264]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 84 op/s 2024-05-10T05:38:47.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:47 smithi142 ceph-mon[77667]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 225 MiB/s wr, 74 op/s 2024-05-10T05:38:47.717 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:47 smithi131 ceph-mon[37264]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 225 MiB/s wr, 74 op/s 2024-05-10T05:38:47.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:47 smithi157 ceph-mon[78427]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 225 MiB/s wr, 74 op/s 2024-05-10T05:38:50.465 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:50 smithi131 ceph-mon[37264]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 252 MiB/s wr, 79 op/s 2024-05-10T05:38:50.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:50 smithi157 ceph-mon[78427]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 252 MiB/s wr, 79 op/s 2024-05-10T05:38:50.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:50 smithi142 ceph-mon[77667]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 252 MiB/s wr, 79 op/s 2024-05-10T05:38:51.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:51 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:38:52.901 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:51 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:38:52.902 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:52 smithi157 ceph-mon[78427]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 270 MiB/s wr, 87 op/s 2024-05-10T05:38:52.902 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:51 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:38:52.903 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:51 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:38:52.903 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:52 smithi131 ceph-mon[37264]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 270 MiB/s wr, 87 op/s 2024-05-10T05:38:52.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:51 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:38:52.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:51 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:38:52.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:52 smithi142 ceph-mon[77667]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 270 MiB/s wr, 87 op/s 2024-05-10T05:38:54.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:54 smithi157 ceph-mon[78427]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 226 MiB/s wr, 71 op/s 2024-05-10T05:38:54.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:54 smithi142 ceph-mon[77667]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 226 MiB/s wr, 71 op/s 2024-05-10T05:38:54.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:54 smithi131 ceph-mon[37264]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 226 MiB/s wr, 71 op/s 2024-05-10T05:38:56.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:56 smithi157 ceph-mon[78427]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 290 MiB/s wr, 89 op/s 2024-05-10T05:38:56.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:56 smithi142 ceph-mon[77667]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 290 MiB/s wr, 89 op/s 2024-05-10T05:38:56.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:56 smithi131 ceph-mon[37264]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 290 MiB/s wr, 89 op/s 2024-05-10T05:38:57.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:38:57 smithi142 ceph-mon[77667]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 241 MiB/s wr, 76 op/s 2024-05-10T05:38:57.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:38:57 smithi131 ceph-mon[37264]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 241 MiB/s wr, 76 op/s 2024-05-10T05:38:57.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:38:57 smithi157 ceph-mon[78427]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 241 MiB/s wr, 76 op/s 2024-05-10T05:39:00.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:00 smithi131 ceph-mon[37264]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 276 MiB/s wr, 84 op/s 2024-05-10T05:39:00.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:00 smithi157 ceph-mon[78427]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 276 MiB/s wr, 84 op/s 2024-05-10T05:39:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:00 smithi142 ceph-mon[77667]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 276 MiB/s wr, 84 op/s 2024-05-10T05:39:02.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:02 smithi157 ceph-mon[78427]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 86 op/s 2024-05-10T05:39:02.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:02 smithi142 ceph-mon[77667]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 86 op/s 2024-05-10T05:39:02.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:02 smithi131 ceph-mon[37264]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 86 op/s 2024-05-10T05:39:04.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:04 smithi157 ceph-mon[78427]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 70 op/s 2024-05-10T05:39:04.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:04 smithi142 ceph-mon[77667]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 70 op/s 2024-05-10T05:39:04.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:04 smithi131 ceph-mon[37264]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 70 op/s 2024-05-10T05:39:06.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:06 smithi142 ceph-mon[77667]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 296 MiB/s wr, 89 op/s 2024-05-10T05:39:06.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:06 smithi157 ceph-mon[78427]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 296 MiB/s wr, 89 op/s 2024-05-10T05:39:06.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:06 smithi131 ceph-mon[37264]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 296 MiB/s wr, 89 op/s 2024-05-10T05:39:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:07 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:39:07.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:07 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:39:07.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:07 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:39:08.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:08 smithi157 ceph-mon[78427]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 244 MiB/s wr, 75 op/s 2024-05-10T05:39:08.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:08 smithi142 ceph-mon[77667]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 244 MiB/s wr, 75 op/s 2024-05-10T05:39:08.720 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:08 smithi131 ceph-mon[37264]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 244 MiB/s wr, 75 op/s 2024-05-10T05:39:09.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:39:09.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:39:09.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:39:09.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:39:09.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:39:09.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:39:09.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:39:09.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:39:09.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:39:10.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:10 smithi157 ceph-mon[78427]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 268 MiB/s wr, 81 op/s 2024-05-10T05:39:10.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:10 smithi142 ceph-mon[77667]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 268 MiB/s wr, 81 op/s 2024-05-10T05:39:10.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:10 smithi131 ceph-mon[37264]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 268 MiB/s wr, 81 op/s 2024-05-10T05:39:12.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:12 smithi142 ceph-mon[77667]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 273 MiB/s wr, 83 op/s 2024-05-10T05:39:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:12 smithi131 ceph-mon[37264]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 273 MiB/s wr, 83 op/s 2024-05-10T05:39:12.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:12 smithi157 ceph-mon[78427]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 273 MiB/s wr, 83 op/s 2024-05-10T05:39:14.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:14 smithi142 ceph-mon[77667]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 233 MiB/s wr, 70 op/s 2024-05-10T05:39:14.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:14 smithi131 ceph-mon[37264]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 233 MiB/s wr, 70 op/s 2024-05-10T05:39:14.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:14 smithi157 ceph-mon[78427]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 233 MiB/s wr, 70 op/s 2024-05-10T05:39:15.560 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:39:15.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:15 smithi142 ceph-mon[77667]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 85 op/s 2024-05-10T05:39:15.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:15 smithi131 ceph-mon[37264]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 85 op/s 2024-05-10T05:39:15.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:15 smithi157 ceph-mon[78427]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 85 op/s 2024-05-10T05:39:16.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.238+0000 7fe379e00700 1 -- 172.21.15.131:0/3919872020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe374109c50 msgr2=0x7fe37410e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:16.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.238+0000 7fe379e00700 1 --2- 172.21.15.131:0/3919872020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe374109c50 0x7fe37410e560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe3600098f0 tx=0x7fe360009c00 comp rx=0 tx=0).stop 2024-05-10T05:39:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.241+0000 7fe379e00700 1 -- 172.21.15.131:0/3919872020 shutdown_connections 2024-05-10T05:39:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.241+0000 7fe379e00700 1 --2- 172.21.15.131:0/3919872020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe374108630 0x7fe374108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.241+0000 7fe379e00700 1 --2- 172.21.15.131:0/3919872020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe374109c50 0x7fe37410e560 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.241+0000 7fe379e00700 1 --2- 172.21.15.131:0/3919872020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe374109300 0x7fe374109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.241+0000 7fe379e00700 1 -- 172.21.15.131:0/3919872020 >> 172.21.15.131:0/3919872020 conn(0x7fe37406ea00 msgr2=0x7fe374070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:16.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.243+0000 7fe379e00700 1 -- 172.21.15.131:0/3919872020 shutdown_connections 2024-05-10T05:39:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.243+0000 7fe379e00700 1 -- 172.21.15.131:0/3919872020 wait complete. 2024-05-10T05:39:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.257+0000 7fe379e00700 1 Processor -- start 2024-05-10T05:39:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.258+0000 7fe379e00700 1 -- start start 2024-05-10T05:39:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.259+0000 7fe379e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe374108630 0x7fe374110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.261+0000 7fe379e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe374109300 0x7fe374110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.261+0000 7fe379e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe374109c50 0x7fe374117a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.261+0000 7fe379e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe374118030 con 0x7fe374109300 2024-05-10T05:39:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.261+0000 7fe379e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3741181a0 con 0x7fe374108630 2024-05-10T05:39:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.261+0000 7fe379e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe374118360 con 0x7fe374109c50 2024-05-10T05:39:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.262+0000 7fe373400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe374109c50 0x7fe374117a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.262+0000 7fe373400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe374109c50 0x7fe374117a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41106/0 (socket says 172.21.15.131:41106) 2024-05-10T05:39:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.262+0000 7fe373400700 1 -- 172.21.15.131:0/266685442 learned_addr learned my addr 172.21.15.131:0/266685442 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.262+0000 7fe372a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe374108630 0x7fe374110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.264+0000 7fe373400700 1 -- 172.21.15.131:0/266685442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe374108630 msgr2=0x7fe374110550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.264+0000 7fe373400700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe374108630 0x7fe374110550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.264+0000 7fe373400700 1 -- 172.21.15.131:0/266685442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe374109300 msgr2=0x7fe374110a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:39:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.266+0000 7fe372000700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe374109300 0x7fe374110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.266+0000 7fe373400700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe374109300 0x7fe374110a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.266+0000 7fe373400700 1 -- 172.21.15.131:0/266685442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c009d00 con 0x7fe374109c50 2024-05-10T05:39:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.267+0000 7fe373400700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe374109c50 0x7fe374117a20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe360009230 tx=0x7fe36000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:16.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.267+0000 7fe378a00700 1 -- 172.21.15.131:0/266685442 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36003e070 con 0x7fe374109c50 2024-05-10T05:39:16.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.267+0000 7fe378a00700 1 -- 172.21.15.131:0/266685442 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36000f7d0 con 0x7fe374109c50 2024-05-10T05:39:16.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.267+0000 7fe378a00700 1 -- 172.21.15.131:0/266685442 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360004af0 con 0x7fe374109c50 2024-05-10T05:39:16.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.268+0000 7fe379e00700 1 -- 172.21.15.131:0/266685442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe360009520 con 0x7fe374109c50 2024-05-10T05:39:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.272+0000 7fe379e00700 1 -- 172.21.15.131:0/266685442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3741187c0 con 0x7fe374109c50 2024-05-10T05:39:16.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.289+0000 7fe378a00700 1 -- 172.21.15.131:0/266685442 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe360042070 con 0x7fe374109c50 2024-05-10T05:39:16.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.300+0000 7fe391d3db80 1 Processor -- start 2024-05-10T05:39:16.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.300+0000 7fe378a00700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3580a1240 0x7fe3580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.301+0000 7fe378a00700 1 -- 172.21.15.131:0/266685442 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe3600ef520 con 0x7fe374109c50 2024-05-10T05:39:16.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.307+0000 7fe391d3db80 1 -- start start 2024-05-10T05:39:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.308+0000 7fe372000700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3580a1240 0x7fe3580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.309+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db70af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.309+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db71980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.309+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db787c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.310+0000 7fe372000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db71980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.310+0000 7fe372000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db71980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37170/0 (socket says 172.21.15.131:37170) 2024-05-10T05:39:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.310+0000 7fe391d3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9da97150 con 0x556d9db71ec0 2024-05-10T05:39:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.310+0000 7fe391d3db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9da94b10 con 0x556d9db71570 2024-05-10T05:39:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.310+0000 7fe391d3db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9da90400 con 0x556d9db706a0 2024-05-10T05:39:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.310+0000 7fe373400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.311+0000 7fe373400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db787c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32774/0 (socket says 172.21.15.131:32774) 2024-05-10T05:39:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.311+0000 7fe373400700 1 -- 172.21.15.131:0/2650988183 learned_addr learned my addr 172.21.15.131:0/2650988183 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.311+0000 7fe372a00700 1 --2- 172.21.15.131:0/2650988183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db70af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.312+0000 7fe373400700 1 -- 172.21.15.131:0/2650988183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 msgr2=0x556d9db70af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.312+0000 7fe373400700 1 --2- 172.21.15.131:0/2650988183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db70af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.312+0000 7fe373400700 1 -- 172.21.15.131:0/2650988183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 msgr2=0x556d9db71980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.312+0000 7fe373400700 1 --2- 172.21.15.131:0/2650988183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db71980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.312+0000 7fe373400700 1 -- 172.21.15.131:0/2650988183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d9d9113b0 con 0x556d9db71ec0 2024-05-10T05:39:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.313+0000 7fe372000700 1 --2- 172.21.15.131:0/2650988183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db71980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-10T05:39:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.314+0000 7fe372000700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3580a1240 0x7fe3580a3750 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe364005950 tx=0x7fe3640099c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:16.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.318+0000 7fe373400700 1 --2- 172.21.15.131:0/2650988183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db787c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe3600b8f00 tx=0x7fe3600056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c098807db5ebee8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.323+0000 7fe35fe00700 1 -- 172.21.15.131:0/2650988183 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600bca00 con 0x556d9db71ec0 2024-05-10T05:39:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.323+0000 7fe35fe00700 1 -- 172.21.15.131:0/2650988183 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3600c0830 con 0x556d9db71ec0 2024-05-10T05:39:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.323+0000 7fe35fe00700 1 -- 172.21.15.131:0/2650988183 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600bca00 con 0x556d9db71ec0 2024-05-10T05:39:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.324+0000 7fe391d3db80 1 -- 172.21.15.131:0/2650988183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 msgr2=0x556d9db787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.324+0000 7fe391d3db80 1 --2- 172.21.15.131:0/2650988183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db787c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe3600b8f00 tx=0x7fe3600056e0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.328+0000 7fe391d3db80 1 -- 172.21.15.131:0/2650988183 shutdown_connections 2024-05-10T05:39:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.328+0000 7fe391d3db80 1 --2- 172.21.15.131:0/2650988183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db787c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.328+0000 7fe391d3db80 1 --2- 172.21.15.131:0/2650988183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db70af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.328+0000 7fe391d3db80 1 --2- 172.21.15.131:0/2650988183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db71980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.328+0000 7fe391d3db80 1 -- 172.21.15.131:0/2650988183 >> 172.21.15.131:0/2650988183 conn(0x556d9dad2a60 msgr2=0x556d9db6afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:16.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.331+0000 7fe391d3db80 1 -- 172.21.15.131:0/2650988183 shutdown_connections 2024-05-10T05:39:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.332+0000 7fe391d3db80 1 -- 172.21.15.131:0/2650988183 wait complete. 2024-05-10T05:39:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.332+0000 7fe391d3db80 1 -- start start 2024-05-10T05:39:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.333+0000 7fe391d3db80 1 Processor -- start 2024-05-10T05:39:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.338+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.338+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db7a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.339+0000 7fe372a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.339+0000 7fe372a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41120/0 (socket says 172.21.15.131:41120) 2024-05-10T05:39:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.339+0000 7fe372a00700 1 -- 172.21.15.131:0/1754195370 learned_addr learned my addr 172.21.15.131:0/1754195370 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.339+0000 7fe391d3db80 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.339+0000 7fe372000700 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db7a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.340+0000 7fe373400700 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.340+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9da2c560 con 0x556d9db71ec0 2024-05-10T05:39:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.340+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9da2c200 con 0x556d9db71570 2024-05-10T05:39:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.340+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9d9136a0 con 0x556d9db706a0 2024-05-10T05:39:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.340+0000 7fe391d3db80 10 client.0 _ll_register_callbacks cb 0x556d9da35b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:16.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.341+0000 7fe373400700 1 -- 172.21.15.131:0/1754195370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 msgr2=0x556d9db7a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:16.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.341+0000 7fe373400700 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.341+0000 7fe373400700 1 -- 172.21.15.131:0/1754195370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 msgr2=0x556d9db7a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:16.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.341+0000 7fe373400700 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db7a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.341+0000 7fe373400700 1 -- 172.21.15.131:0/1754195370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3741187c0 con 0x556d9db71ec0 2024-05-10T05:39:16.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.342+0000 7fe373400700 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db817b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe3600baff0 tx=0x7fe360009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2cf3fab9d85829 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:16.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.342+0000 7fe35cc00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:39:16.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.345+0000 7fe35cc00700 1 -- 172.21.15.131:0/1754195370 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600bca00 con 0x556d9db71ec0 2024-05-10T05:39:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.346+0000 7fe35cc00700 1 -- 172.21.15.131:0/1754195370 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe3600c4df0 con 0x556d9db71ec0 2024-05-10T05:39:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.349+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d9d9e9270 con 0x556d9db71ec0 2024-05-10T05:39:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.349+0000 7fe35cc00700 1 -- 172.21.15.131:0/1754195370 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe36000b9e0 con 0x556d9db71ec0 2024-05-10T05:39:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.349+0000 7fe391d3db80 10 client.16215 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.349+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556d9d8bdb60 con 0x556d9db71ec0 2024-05-10T05:39:16.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.353+0000 7fe35cc00700 1 -- 172.21.15.131:0/1754195370 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d9d9e9270 con 0x556d9db71ec0 2024-05-10T05:39:16.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.353+0000 7fe35cc00700 1 -- 172.21.15.131:0/1754195370 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe360005040 con 0x556d9db71ec0 2024-05-10T05:39:16.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.353+0000 7fe391d3db80 10 client.16215 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:16.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.353+0000 7fe391d3db80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:39:16.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.354+0000 7fe391d3db80 20 client.16215 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:16.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.354+0000 7fe391d3db80 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556d9db85990 0x556d9db87db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.354+0000 7fe391d3db80 4 client.16215 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:16.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.354+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d9db3f780 con 0x556d9db85990 2024-05-10T05:39:16.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.366+0000 7fe372a00700 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556d9db85990 0x556d9db87db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.369+0000 7fe35cc00700 10 client.16215 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:16.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.369+0000 7fe372a00700 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556d9db85990 0x556d9db87db0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1855b208e6af26 server_cookie=c8a0dd22cfabf1c1 in_seq=0 out_seq=0 2024-05-10T05:39:16.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.370+0000 7fe35cc00700 1 -- 172.21.15.131:0/1754195370 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d9db3f780 con 0x556d9db85990 2024-05-10T05:39:16.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.370+0000 7fe35cc00700 10 client.16215 handle_command_reply: tid=0 2024-05-10T05:39:16.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.371+0000 7fe391d3db80 1 client.16215 shutdown 2024-05-10T05:39:16.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.371+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 msgr2=0x556d9db817b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:16.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.371+0000 7fe391d3db80 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db817b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe3600baff0 tx=0x7fe360009260 comp rx=0 tx=0).stop 2024-05-10T05:39:16.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.372+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 shutdown_connections 2024-05-10T05:39:16.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.372+0000 7fe391d3db80 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556d9db85990 0x556d9db87db0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.372+0000 7fe391d3db80 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db817b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.372+0000 7fe391d3db80 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.372+0000 7fe391d3db80 1 --2- 172.21.15.131:0/1754195370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db7a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.372+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 >> 172.21.15.131:0/1754195370 conn(0x556d9dad2a60 msgr2=0x556d9db6afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:16.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.372+0000 7fe35cc00700 0 client.16215 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:16.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.373+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 shutdown_connections 2024-05-10T05:39:16.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.373+0000 7fe391d3db80 1 -- 172.21.15.131:0/1754195370 wait complete. 2024-05-10T05:39:16.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.375+0000 7fe391d3db80 20 client.16215 trim_cache size 0 max 16384 2024-05-10T05:39:16.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.433+0000 7fe391d3db80 1 Processor -- start 2024-05-10T05:39:16.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.433+0000 7fe391d3db80 1 -- start start 2024-05-10T05:39:16.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.434+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db706a0 0x556d9db79880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.434+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71570 0x556d9db81770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.434+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db71ec0 0x556d9db751b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.434+0000 7fe391d3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9d9e9270 con 0x556d9db71570 2024-05-10T05:39:16.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.434+0000 7fe391d3db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9db67d30 con 0x556d9db706a0 2024-05-10T05:39:16.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.434+0000 7fe391d3db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9db713c0 con 0x556d9db71ec0 2024-05-10T05:39:16.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe372a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db706a0 0x556d9db79880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe372a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db706a0 0x556d9db79880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37180/0 (socket says 172.21.15.131:37180) 2024-05-10T05:39:16.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe372a00700 1 -- 172.21.15.131:0/2342010354 learned_addr learned my addr 172.21.15.131:0/2342010354 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:16.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe372000700 1 --2- 172.21.15.131:0/2342010354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71570 0x556d9db81770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe373400700 1 --2- 172.21.15.131:0/2342010354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db71ec0 0x556d9db751b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe372000700 1 -- 172.21.15.131:0/2342010354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db71ec0 msgr2=0x556d9db751b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:16.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe372000700 1 --2- 172.21.15.131:0/2342010354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db71ec0 0x556d9db751b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe372000700 1 -- 172.21.15.131:0/2342010354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db706a0 msgr2=0x556d9db79880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:16.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe372000700 1 --2- 172.21.15.131:0/2342010354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db706a0 0x556d9db79880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe372000700 1 -- 172.21.15.131:0/2342010354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d9da56560 con 0x556d9db71570 2024-05-10T05:39:16.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.435+0000 7fe372a00700 1 --2- 172.21.15.131:0/2342010354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db706a0 0x556d9db79880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:16.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.437+0000 7fe372000700 1 --2- 172.21.15.131:0/2342010354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71570 0x556d9db81770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe36400edf0 tx=0x7fe364011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0beb9e890476f60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:16.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.437+0000 7fe369200700 1 -- 172.21.15.131:0/2342010354 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364011b60 con 0x556d9db71570 2024-05-10T05:39:16.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.437+0000 7fe369200700 1 -- 172.21.15.131:0/2342010354 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe364020940 con 0x556d9db71570 2024-05-10T05:39:16.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.437+0000 7fe369200700 1 -- 172.21.15.131:0/2342010354 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3640139a0 con 0x556d9db71570 2024-05-10T05:39:16.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.438+0000 7fe391d3db80 1 -- 172.21.15.131:0/2342010354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71570 msgr2=0x556d9db81770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:16.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.438+0000 7fe391d3db80 1 --2- 172.21.15.131:0/2342010354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71570 0x556d9db81770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe36400edf0 tx=0x7fe364011000 comp rx=0 tx=0).stop 2024-05-10T05:39:16.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.438+0000 7fe391d3db80 1 -- 172.21.15.131:0/2342010354 shutdown_connections 2024-05-10T05:39:16.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.438+0000 7fe391d3db80 1 --2- 172.21.15.131:0/2342010354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db71ec0 0x556d9db751b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.438+0000 7fe391d3db80 1 --2- 172.21.15.131:0/2342010354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db706a0 0x556d9db79880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.438+0000 7fe391d3db80 1 --2- 172.21.15.131:0/2342010354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71570 0x556d9db81770 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.438+0000 7fe391d3db80 1 -- 172.21.15.131:0/2342010354 >> 172.21.15.131:0/2342010354 conn(0x556d9dad2a60 msgr2=0x556d9db747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:16.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.439+0000 7fe391d3db80 1 -- 172.21.15.131:0/2342010354 shutdown_connections 2024-05-10T05:39:16.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.439+0000 7fe391d3db80 1 -- 172.21.15.131:0/2342010354 wait complete. 2024-05-10T05:39:16.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.439+0000 7fe391d3db80 1 -- start start 2024-05-10T05:39:16.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.440+0000 7fe391d3db80 1 Processor -- start 2024-05-10T05:39:16.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.441+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.441+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db7f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.441+0000 7fe391d3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.441+0000 7fe372a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe372a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41142/0 (socket says 172.21.15.131:41142) 2024-05-10T05:39:16.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe372a00700 1 -- 172.21.15.131:0/3927826139 learned_addr learned my addr 172.21.15.131:0/3927826139 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:16.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9db80d10 con 0x556d9db71ec0 2024-05-10T05:39:16.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9db80e80 con 0x556d9db71570 2024-05-10T05:39:16.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9db81040 con 0x556d9db706a0 2024-05-10T05:39:16.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.442+0000 7fe373400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe391d3db80 10 client.0 _ll_register_callbacks cb 0x556d9da35b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:16.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.442+0000 7fe372000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db7f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe372a00700 1 -- 172.21.15.131:0/3927826139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 msgr2=0x556d9db7f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:16.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe372a00700 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db7f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe372a00700 1 -- 172.21.15.131:0/3927826139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 msgr2=0x556d9db807d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:16.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe372a00700 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db807d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.443+0000 7fe372a00700 1 -- 172.21.15.131:0/3927826139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36400eb60 con 0x556d9db706a0 2024-05-10T05:39:16.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.444+0000 7fe373400700 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:39:16.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.444+0000 7fe372a00700 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7f360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe36c003f30 tx=0x7fe36c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26e87d49b7517ef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:16.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.444+0000 7fe35cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:39:16.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.446+0000 7fe35cc00700 1 -- 172.21.15.131:0/3927826139 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c00e040 con 0x556d9db706a0 2024-05-10T05:39:16.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.446+0000 7fe35cc00700 1 -- 172.21.15.131:0/3927826139 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe36c004db0 con 0x556d9db706a0 2024-05-10T05:39:16.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.446+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3600c0830 con 0x556d9db706a0 2024-05-10T05:39:16.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.448+0000 7fe35cc00700 1 -- 172.21.15.131:0/3927826139 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe36c01b4b0 con 0x556d9db706a0 2024-05-10T05:39:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.448+0000 7fe391d3db80 10 client.26419 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.448+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556d9d915cd0 con 0x556d9db706a0 2024-05-10T05:39:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.449+0000 7fe35cc00700 1 -- 172.21.15.131:0/3927826139 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3600c0830 con 0x556d9db706a0 2024-05-10T05:39:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.449+0000 7fe35cc00700 1 -- 172.21.15.131:0/3927826139 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe36c014050 con 0x556d9db706a0 2024-05-10T05:39:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.450+0000 7fe391d3db80 10 client.26419 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.450+0000 7fe391d3db80 10 client.26419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:39:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.450+0000 7fe391d3db80 20 client.26419 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.450+0000 7fe391d3db80 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556d9dbdcb90 0x556d9dbdefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.450+0000 7fe391d3db80 4 client.26419 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:39:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.450+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556d9da88ec0 con 0x556d9dbdcb90 2024-05-10T05:39:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.451+0000 7fe372000700 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556d9dbdcb90 0x556d9dbdefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.452+0000 7fe35cc00700 10 client.26419 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.454+0000 7fe372000700 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556d9dbdcb90 0x556d9dbdefb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0e2ad7b006b3fc server_cookie=23c70b41b24c6d6e in_seq=0 out_seq=0 2024-05-10T05:39:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.456+0000 7fe35cc00700 1 -- 172.21.15.131:0/3927826139 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556d9da88ec0 con 0x556d9dbdcb90 2024-05-10T05:39:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.457+0000 7fe35cc00700 10 client.26419 handle_command_reply: tid=0 2024-05-10T05:39:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.457+0000 7fe391d3db80 1 client.26419 shutdown 2024-05-10T05:39:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.458+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 msgr2=0x556d9db7f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.458+0000 7fe391d3db80 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7f360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe36c003f30 tx=0x7fe36c0099e0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.459+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 shutdown_connections 2024-05-10T05:39:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.459+0000 7fe391d3db80 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556d9dbdcb90 0x556d9dbdefb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.462+0000 7fe391d3db80 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556d9db71ec0 0x556d9db807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.462+0000 7fe391d3db80 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d9db706a0 0x556d9db7f360 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.462+0000 7fe35cc00700 0 client.26419 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.462+0000 7fe391d3db80 1 --2- 172.21.15.131:0/3927826139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d9db71570 0x556d9db7f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.462+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 >> 172.21.15.131:0/3927826139 conn(0x556d9dad2a60 msgr2=0x556d9db747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.462+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 shutdown_connections 2024-05-10T05:39:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.462+0000 7fe391d3db80 1 -- 172.21.15.131:0/3927826139 wait complete. 2024-05-10T05:39:16.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.463+0000 7fe391d3db80 20 client.26419 trim_cache size 0 max 16384 2024-05-10T05:39:16.464 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:39:16.464 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:39:16.464 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:39:16.464 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:39:16.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.464+0000 7fe379e00700 1 -- 172.21.15.131:0/266685442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3580a1240 msgr2=0x7fe3580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:16.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.464+0000 7fe379e00700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3580a1240 0x7fe3580a3750 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe364005950 tx=0x7fe3640099c0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.466+0000 7fe379e00700 1 -- 172.21.15.131:0/266685442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe374109c50 msgr2=0x7fe374117a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:16.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.466+0000 7fe379e00700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe374109c50 0x7fe374117a20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe360009230 tx=0x7fe36000b840 comp rx=0 tx=0).stop 2024-05-10T05:39:16.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.467+0000 7fe379e00700 1 -- 172.21.15.131:0/266685442 shutdown_connections 2024-05-10T05:39:16.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.467+0000 7fe379e00700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe3580a1240 0x7fe3580a3750 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.467+0000 7fe379e00700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe374109c50 0x7fe374117a20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.467+0000 7fe379e00700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe374108630 0x7fe374110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.467+0000 7fe379e00700 1 --2- 172.21.15.131:0/266685442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe374109300 0x7fe374110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:16.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.467+0000 7fe379e00700 1 -- 172.21.15.131:0/266685442 >> 172.21.15.131:0/266685442 conn(0x7fe37406ea00 msgr2=0x7fe37406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:16.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.468+0000 7fe379e00700 1 -- 172.21.15.131:0/266685442 shutdown_connections 2024-05-10T05:39:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:16.469+0000 7fe379e00700 1 -- 172.21.15.131:0/266685442 wait complete. 2024-05-10T05:39:16.526 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:39:16.526 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:39:17.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.051+0000 7fdb5a800700 1 -- 172.21.15.131:0/459658592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb4c094f00 msgr2=0x7fdb4c097320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.052+0000 7fdb5a800700 1 --2- 172.21.15.131:0/459658592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb4c094f00 0x7fdb4c097320 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009990 tx=0x7fdb48009ca0 comp rx=0 tx=0).stop 2024-05-10T05:39:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.052+0000 7fdb5a800700 1 -- 172.21.15.131:0/459658592 shutdown_connections 2024-05-10T05:39:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.052+0000 7fdb5a800700 1 --2- 172.21.15.131:0/459658592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb4c09a1c0 0x7fdb4c09a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.052+0000 7fdb5a800700 1 --2- 172.21.15.131:0/459658592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb4c094f00 0x7fdb4c097320 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.052+0000 7fdb5a800700 1 --2- 172.21.15.131:0/459658592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb4c097860 0x7fdb4c099c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.052+0000 7fdb5a800700 1 -- 172.21.15.131:0/459658592 >> 172.21.15.131:0/459658592 conn(0x7fdb4c090b10 msgr2=0x7fdb4c092fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.053+0000 7fdb5a800700 1 -- 172.21.15.131:0/459658592 shutdown_connections 2024-05-10T05:39:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.053+0000 7fdb5a800700 1 -- 172.21.15.131:0/459658592 wait complete. 2024-05-10T05:39:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.054+0000 7fdb5a800700 1 Processor -- start 2024-05-10T05:39:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.056+0000 7fdb5a800700 1 -- start start 2024-05-10T05:39:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.058+0000 7fdb5a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb4c094f00 0x7fdb4c1440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.058+0000 7fdb5a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb4c097860 0x7fdb4c144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.058+0000 7fdb5a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb4c09a1c0 0x7fdb4c14b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.058+0000 7fdb5a800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c14bb60 con 0x7fdb4c094f00 2024-05-10T05:39:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.058+0000 7fdb5a800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c14bca0 con 0x7fdb4c097860 2024-05-10T05:39:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.058+0000 7fdb5a800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c14be30 con 0x7fdb4c09a1c0 2024-05-10T05:39:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.059+0000 7fdb52a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb4c097860 0x7fdb4c144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.059+0000 7fdb52a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb4c097860 0x7fdb4c144600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37210/0 (socket says 172.21.15.131:37210) 2024-05-10T05:39:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.059+0000 7fdb53e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb4c09a1c0 0x7fdb4c14b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.059+0000 7fdb52a00700 1 -- 172.21.15.131:0/1567338174 learned_addr learned my addr 172.21.15.131:0/1567338174 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.059+0000 7fdb53400700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb4c094f00 0x7fdb4c1440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:17.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.059+0000 7fdb52a00700 1 -- 172.21.15.131:0/1567338174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb4c09a1c0 msgr2=0x7fdb4c14b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.059+0000 7fdb52a00700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb4c09a1c0 0x7fdb4c14b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.059+0000 7fdb52a00700 1 -- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb4c094f00 msgr2=0x7fdb4c1440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:17.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.060+0000 7fdb52a00700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb4c094f00 0x7fdb4c1440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:17.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.060+0000 7fdb52a00700 1 -- 172.21.15.131:0/1567338174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480095f0 con 0x7fdb4c097860 2024-05-10T05:39:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.060+0000 7fdb53400700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb4c094f00 0x7fdb4c1440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.063+0000 7fdb52a00700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb4c097860 0x7fdb4c144600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000b530 tx=0x7fdb4000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.063+0000 7fdb47e00700 1 -- 172.21.15.131:0/1567338174 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb400118f0 con 0x7fdb4c097860 2024-05-10T05:39:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.063+0000 7fdb47e00700 1 -- 172.21.15.131:0/1567338174 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb40010380 con 0x7fdb4c097860 2024-05-10T05:39:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.063+0000 7fdb47e00700 1 -- 172.21.15.131:0/1567338174 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000f540 con 0x7fdb4c097860 2024-05-10T05:39:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.063+0000 7fdb5a800700 1 -- 172.21.15.131:0/1567338174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c14c110 con 0x7fdb4c097860 2024-05-10T05:39:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.064+0000 7fdb5a800700 1 -- 172.21.15.131:0/1567338174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c14c650 con 0x7fdb4c097860 2024-05-10T05:39:17.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.072+0000 7fdb47e00700 1 -- 172.21.15.131:0/1567338174 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fdb40011a50 con 0x7fdb4c097860 2024-05-10T05:39:17.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.073+0000 7fdb5a800700 1 -- 172.21.15.131:0/1567338174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb34005320 con 0x7fdb4c097860 2024-05-10T05:39:17.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.076+0000 7fdb47e00700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdb3c0a12a0 0x7fdb3c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:17.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.076+0000 7fdb47e00700 1 -- 172.21.15.131:0/1567338174 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fdb400cc2e0 con 0x7fdb4c097860 2024-05-10T05:39:17.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.080+0000 7fdb53400700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdb3c0a12a0 0x7fdb3c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:17.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.081+0000 7fdb53400700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdb3c0a12a0 0x7fdb3c0a37b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdb48006010 tx=0x7fdb48005750 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:17.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.084+0000 7fdb47e00700 1 -- 172.21.15.131:0/1567338174 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdb40095a70 con 0x7fdb4c097860 2024-05-10T05:39:17.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.774+0000 7fdb5a800700 1 -- 172.21.15.131:0/1567338174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb34005b50 con 0x7fdb4c097860 2024-05-10T05:39:17.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.779+0000 7fdb47e00700 1 -- 172.21.15.131:0/1567338174 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fdb40099090 con 0x7fdb4c097860 2024-05-10T05:39:17.781 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:39:17.781 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:39:17.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.783+0000 7fdb5a800700 1 -- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdb3c0a12a0 msgr2=0x7fdb3c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:17.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.783+0000 7fdb5a800700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdb3c0a12a0 0x7fdb3c0a37b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdb48006010 tx=0x7fdb48005750 comp rx=0 tx=0).stop 2024-05-10T05:39:17.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.783+0000 7fdb5a800700 1 -- 172.21.15.131:0/1567338174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb4c097860 msgr2=0x7fdb4c144600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:17.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.783+0000 7fdb5a800700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb4c097860 0x7fdb4c144600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000b530 tx=0x7fdb4000b840 comp rx=0 tx=0).stop 2024-05-10T05:39:17.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.783+0000 7fdb5a800700 1 -- 172.21.15.131:0/1567338174 shutdown_connections 2024-05-10T05:39:17.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.783+0000 7fdb5a800700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdb3c0a12a0 0x7fdb3c0a37b0 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdb48006010 tx=0x7fdb48005750 comp rx=0 tx=0).stop 2024-05-10T05:39:17.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.783+0000 7fdb5a800700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb4c094f00 0x7fdb4c1440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:17.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.783+0000 7fdb5a800700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb4c09a1c0 0x7fdb4c14b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:17.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.783+0000 7fdb5a800700 1 --2- 172.21.15.131:0/1567338174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb4c097860 0x7fdb4c144600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:17.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.783+0000 7fdb5a800700 1 -- 172.21.15.131:0/1567338174 >> 172.21.15.131:0/1567338174 conn(0x7fdb4c090b10 msgr2=0x7fdb4c095e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:17.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.785+0000 7fdb5a800700 1 -- 172.21.15.131:0/1567338174 shutdown_connections 2024-05-10T05:39:17.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:17.785+0000 7fdb5a800700 1 -- 172.21.15.131:0/1567338174 wait complete. 2024-05-10T05:39:17.788 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:39:17.837 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:39:18.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.413+0000 7ffb6f200700 1 -- 172.21.15.131:0/2978788551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb681036a0 msgr2=0x7ffb68109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.413+0000 7ffb6f200700 1 --2- 172.21.15.131:0/2978788551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb681036a0 0x7ffb68109f50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009fc0 tx=0x7ffb6000b330 comp rx=0 tx=0).stop 2024-05-10T05:39:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.413+0000 7ffb6f200700 1 -- 172.21.15.131:0/2978788551 shutdown_connections 2024-05-10T05:39:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.413+0000 7ffb6f200700 1 --2- 172.21.15.131:0/2978788551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb681036a0 0x7ffb68109f50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.413+0000 7ffb6f200700 1 --2- 172.21.15.131:0/2978788551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb68101be0 0x7ffb68101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.413+0000 7ffb6f200700 1 --2- 172.21.15.131:0/2978788551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb68102d00 0x7ffb68103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.413+0000 7ffb6f200700 1 -- 172.21.15.131:0/2978788551 >> 172.21.15.131:0/2978788551 conn(0x7ffb680fd1d0 msgr2=0x7ffb680ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.414+0000 7ffb6f200700 1 -- 172.21.15.131:0/2978788551 shutdown_connections 2024-05-10T05:39:18.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.414+0000 7ffb6f200700 1 -- 172.21.15.131:0/2978788551 wait complete. 2024-05-10T05:39:18.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.415+0000 7ffb6f200700 1 Processor -- start 2024-05-10T05:39:18.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.415+0000 7ffb6f200700 1 -- start start 2024-05-10T05:39:18.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.416+0000 7ffb6f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb68101be0 0x7ffb68074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.416+0000 7ffb6f200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb68102d00 0x7ffb68073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.416+0000 7ffb6f200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb681036a0 0x7ffb68073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.416+0000 7ffb6f200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68074180 con 0x7ffb681036a0 2024-05-10T05:39:18.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.416+0000 7ffb6f200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680742f0 con 0x7ffb68102d00 2024-05-10T05:39:18.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.416+0000 7ffb6f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681a20d0 con 0x7ffb68101be0 2024-05-10T05:39:18.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.416+0000 7ffb67400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb68102d00 0x7ffb68073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.416+0000 7ffb67400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb68102d00 0x7ffb68073560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37224/0 (socket says 172.21.15.131:37224) 2024-05-10T05:39:18.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.416+0000 7ffb67400700 1 -- 172.21.15.131:0/1732611031 learned_addr learned my addr 172.21.15.131:0/1732611031 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:18.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.417+0000 7ffb67e00700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb68101be0 0x7ffb68074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.417+0000 7ffb6d200700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb681036a0 0x7ffb68073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.417+0000 7ffb67400700 1 -- 172.21.15.131:0/1732611031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb68101be0 msgr2=0x7ffb68074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:18.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.417+0000 7ffb67400700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb68101be0 0x7ffb68074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.417+0000 7ffb67400700 1 -- 172.21.15.131:0/1732611031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb681036a0 msgr2=0x7ffb68073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:18.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.417+0000 7ffb67400700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb681036a0 0x7ffb68073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.417+0000 7ffb67400700 1 -- 172.21.15.131:0/1732611031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60009c40 con 0x7ffb68102d00 2024-05-10T05:39:18.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.417+0000 7ffb67e00700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb68101be0 0x7ffb68074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:39:18.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.417+0000 7ffb67400700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb68102d00 0x7ffb68073560 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e9c0 tx=0x7ffb5400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:18.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.418+0000 7ffb64c00700 1 -- 172.21.15.131:0/1732611031 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb540097b0 con 0x7ffb68102d00 2024-05-10T05:39:18.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.418+0000 7ffb64c00700 1 -- 172.21.15.131:0/1732611031 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb54004cf0 con 0x7ffb68102d00 2024-05-10T05:39:18.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.418+0000 7ffb64c00700 1 -- 172.21.15.131:0/1732611031 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54005700 con 0x7ffb68102d00 2024-05-10T05:39:18.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.418+0000 7ffb6f200700 1 -- 172.21.15.131:0/1732611031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb681a2330 con 0x7ffb68102d00 2024-05-10T05:39:18.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.418+0000 7ffb6f200700 1 -- 172.21.15.131:0/1732611031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb681a27c0 con 0x7ffb68102d00 2024-05-10T05:39:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.424+0000 7ffb64c00700 1 -- 172.21.15.131:0/1732611031 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ffb5400cbd0 con 0x7ffb68102d00 2024-05-10T05:39:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.426+0000 7ffb64c00700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffb4c0a12c0 0x7ffb4c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.427+0000 7ffb64c00700 1 -- 172.21.15.131:0/1732611031 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ffb54014070 con 0x7ffb68102d00 2024-05-10T05:39:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.428+0000 7ffb87103b80 1 Processor -- start 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.428+0000 7ffb87103b80 1 -- start start 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.428+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205e800 0x55ea4205ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.428+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205d380 0x55ea4205d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.428+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205dcd0 0x55ea42079ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.428+0000 7ffb87103b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea41f87150 con 0x55ea4205d380 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.428+0000 7ffb87103b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea41f84b10 con 0x55ea4205e800 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.428+0000 7ffb87103b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea41f80400 con 0x55ea4205dcd0 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.429+0000 7ffb6d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205dcd0 0x55ea42079ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.429+0000 7ffb6d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205dcd0 0x55ea42079ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41208/0 (socket says 172.21.15.131:41208) 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.429+0000 7ffb6d200700 1 -- 172.21.15.131:0/2200909809 learned_addr learned my addr 172.21.15.131:0/2200909809 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.430+0000 7ffb67e00700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffb4c0a12c0 0x7ffb4c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.430+0000 7ffb67400700 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205d380 0x55ea4205d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.430+0000 7ffb67e00700 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205e800 0x55ea4205ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.431+0000 7ffb6d200700 1 -- 172.21.15.131:0/2200909809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205e800 msgr2=0x55ea4205ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:18.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.431+0000 7ffb6d200700 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205e800 0x55ea4205ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.431+0000 7ffb6d200700 1 -- 172.21.15.131:0/2200909809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205d380 msgr2=0x55ea4205d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:18.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.431+0000 7ffb6d200700 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205d380 0x55ea4205d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.431+0000 7ffb6d200700 1 -- 172.21.15.131:0/2200909809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea41e013b0 con 0x55ea4205dcd0 2024-05-10T05:39:18.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.432+0000 7ffb67400700 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205d380 0x55ea4205d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:39:18.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.433+0000 7ffb67e00700 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205e800 0x55ea4205ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:39:18.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.435+0000 7ffb6d200700 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205dcd0 0x55ea42079ae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009bc0 tx=0x7ffb600096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31390cd3ec0ee712 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:18.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.436+0000 7ffb67e00700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffb4c0a12c0 0x7ffb4c0a37d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffb580046e0 tx=0x7ffb580097f0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:18.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.436+0000 7ffb5d600700 1 -- 172.21.15.131:0/2200909809 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000e040 con 0x55ea4205dcd0 2024-05-10T05:39:18.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.436+0000 7ffb5d600700 1 -- 172.21.15.131:0/2200909809 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb60009360 con 0x55ea4205dcd0 2024-05-10T05:39:18.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.436+0000 7ffb5d600700 1 -- 172.21.15.131:0/2200909809 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000e040 con 0x55ea4205dcd0 2024-05-10T05:39:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.439+0000 7ffb87103b80 1 -- 172.21.15.131:0/2200909809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205dcd0 msgr2=0x55ea42079ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.439+0000 7ffb87103b80 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205dcd0 0x55ea42079ae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009bc0 tx=0x7ffb600096c0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.441+0000 7ffb87103b80 1 -- 172.21.15.131:0/2200909809 shutdown_connections 2024-05-10T05:39:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.441+0000 7ffb87103b80 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205dcd0 0x55ea42079ae0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.441+0000 7ffb87103b80 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205e800 0x55ea4205ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.441+0000 7ffb87103b80 1 --2- 172.21.15.131:0/2200909809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205d380 0x55ea4205d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.441+0000 7ffb87103b80 1 -- 172.21.15.131:0/2200909809 >> 172.21.15.131:0/2200909809 conn(0x55ea41fc2a60 msgr2=0x55ea4205afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.441+0000 7ffb87103b80 1 -- 172.21.15.131:0/2200909809 shutdown_connections 2024-05-10T05:39:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.442+0000 7ffb87103b80 1 -- 172.21.15.131:0/2200909809 wait complete. 2024-05-10T05:39:18.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.443+0000 7ffb87103b80 1 -- start start 2024-05-10T05:39:18.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.444+0000 7ffb87103b80 1 Processor -- start 2024-05-10T05:39:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.445+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205d380 0x55ea4206a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.445+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4206ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.445+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205e800 0x55ea42071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.445+0000 7ffb87103b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea41f1c560 con 0x55ea4205e800 2024-05-10T05:39:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.445+0000 7ffb87103b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea41f1c200 con 0x55ea4205dcd0 2024-05-10T05:39:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.445+0000 7ffb87103b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea41e036a0 con 0x55ea4205d380 2024-05-10T05:39:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.446+0000 7ffb67400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4206ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.446+0000 7ffb67400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4206ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37250/0 (socket says 172.21.15.131:37250) 2024-05-10T05:39:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.446+0000 7ffb67400700 1 -- 172.21.15.131:0/658872602 learned_addr learned my addr 172.21.15.131:0/658872602 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.446+0000 7ffb67e00700 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205d380 0x55ea4206a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.446+0000 7ffb87103b80 10 client.0 _ll_register_callbacks cb 0x55ea41f25b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.446+0000 7ffb67400700 1 -- 172.21.15.131:0/658872602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205d380 msgr2=0x55ea4206a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.446+0000 7ffb67400700 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205d380 0x55ea4206a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.446+0000 7ffb67400700 1 -- 172.21.15.131:0/658872602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205e800 msgr2=0x55ea42071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.446+0000 7ffb67400700 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205e800 0x55ea42071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.446+0000 7ffb67400700 1 -- 172.21.15.131:0/658872602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6810a9e0 con 0x55ea4205dcd0 2024-05-10T05:39:18.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.447+0000 7ffb67400700 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4206ab50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffb54096b10 tx=0x7ffb5409a850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10ed57235832adc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:18.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.448+0000 7ffb52000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:39:18.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.448+0000 7ffb52000700 1 -- 172.21.15.131:0/658872602 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5409ded0 con 0x55ea4205dcd0 2024-05-10T05:39:18.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.448+0000 7ffb52000700 1 -- 172.21.15.131:0/658872602 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ffb540a2520 con 0x55ea4205dcd0 2024-05-10T05:39:18.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.453+0000 7ffb87103b80 1 -- 172.21.15.131:0/658872602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea41ed9270 con 0x55ea4205dcd0 2024-05-10T05:39:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.454+0000 7ffb52000700 1 -- 172.21.15.131:0/658872602 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb540ca860 con 0x55ea4205dcd0 2024-05-10T05:39:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.454+0000 7ffb87103b80 10 client.26405 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.454+0000 7ffb87103b80 1 -- 172.21.15.131:0/658872602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ea41dadb60 con 0x55ea4205dcd0 2024-05-10T05:39:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.455+0000 7ffb52000700 1 -- 172.21.15.131:0/658872602 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea41ed9270 con 0x55ea4205dcd0 2024-05-10T05:39:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.463+0000 7ffb52000700 1 -- 172.21.15.131:0/658872602 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffb540a2090 con 0x55ea4205dcd0 2024-05-10T05:39:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.463+0000 7ffb87103b80 10 client.26405 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.463+0000 7ffb87103b80 10 client.26405 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:39:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.463+0000 7ffb87103b80 20 client.26405 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.463+0000 7ffb87103b80 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ea42076220 0x55ea42078670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.463+0000 7ffb87103b80 4 client.26405 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.463+0000 7ffb87103b80 1 -- 172.21.15.131:0/658872602 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea4202f780 con 0x55ea42076220 2024-05-10T05:39:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.463+0000 7ffb67e00700 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ea42076220 0x55ea42078670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.480+0000 7ffb67e00700 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ea42076220 0x55ea42078670 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cc523dd6256dc3ee server_cookie=c656f76c55c32527 in_seq=0 out_seq=0 2024-05-10T05:39:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.480+0000 7ffb52000700 10 client.26405 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:39:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.481+0000 7ffb52000700 1 -- 172.21.15.131:0/658872602 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea4202f780 con 0x55ea42076220 2024-05-10T05:39:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.481+0000 7ffb52000700 10 client.26405 handle_command_reply: tid=0 2024-05-10T05:39:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.481+0000 7ffb87103b80 1 client.26405 shutdown 2024-05-10T05:39:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.485+0000 7ffb87103b80 1 -- 172.21.15.131:0/658872602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 msgr2=0x55ea4206ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.485+0000 7ffb87103b80 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4206ab50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffb54096b10 tx=0x7ffb5409a850 comp rx=0 tx=0).stop 2024-05-10T05:39:18.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.488+0000 7ffb87103b80 1 -- 172.21.15.131:0/658872602 shutdown_connections 2024-05-10T05:39:18.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.488+0000 7ffb87103b80 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ea42076220 0x55ea42078670 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.488+0000 7ffb87103b80 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205e800 0x55ea42071ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.488+0000 7ffb87103b80 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205d380 0x55ea4206a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.488+0000 7ffb87103b80 1 --2- 172.21.15.131:0/658872602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4206ab50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.488+0000 7ffb87103b80 1 -- 172.21.15.131:0/658872602 >> 172.21.15.131:0/658872602 conn(0x55ea41fc2a60 msgr2=0x55ea4205afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:18.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.488+0000 7ffb52000700 0 client.26405 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:39:18.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.491+0000 7ffb87103b80 1 -- 172.21.15.131:0/658872602 shutdown_connections 2024-05-10T05:39:18.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.493+0000 7ffb87103b80 1 -- 172.21.15.131:0/658872602 wait complete. 2024-05-10T05:39:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.505+0000 7ffb87103b80 20 client.26405 trim_cache size 0 max 16384 2024-05-10T05:39:18.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:18 smithi157 ceph-mon[78427]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 219 MiB/s wr, 69 op/s 2024-05-10T05:39:18.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:18 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1567338174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:39:18.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.567+0000 7ffb87103b80 1 Processor -- start 2024-05-10T05:39:18.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.570+0000 7ffb87103b80 1 -- start start 2024-05-10T05:39:18.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.571+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205d380 0x55ea420720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.571+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4205af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.571+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205e800 0x55ea42068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.571+0000 7ffb87103b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4205c770 con 0x55ea4205e800 2024-05-10T05:39:18.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.571+0000 7ffb87103b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea420a0900 con 0x55ea4205dcd0 2024-05-10T05:39:18.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.571+0000 7ffb87103b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4205b4d0 con 0x55ea4205d380 2024-05-10T05:39:18.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.571+0000 7ffb67400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4205af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.571+0000 7ffb67400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4205af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37264/0 (socket says 172.21.15.131:37264) 2024-05-10T05:39:18.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.571+0000 7ffb67400700 1 -- 172.21.15.131:0/1951146109 learned_addr learned my addr 172.21.15.131:0/1951146109 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:18.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.572+0000 7ffb6d200700 1 --2- 172.21.15.131:0/1951146109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205e800 0x55ea42068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.573+0000 7ffb67400700 1 -- 172.21.15.131:0/1951146109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205d380 msgr2=0x55ea420720c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:39:18.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.573+0000 7ffb67400700 1 --2- 172.21.15.131:0/1951146109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205d380 0x55ea420720c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.573+0000 7ffb67400700 1 -- 172.21.15.131:0/1951146109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205e800 msgr2=0x55ea42068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:18.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.575+0000 7ffb67400700 1 --2- 172.21.15.131:0/1951146109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205e800 0x55ea42068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.576+0000 7ffb67400700 1 -- 172.21.15.131:0/1951146109 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea41f46560 con 0x55ea4205dcd0 2024-05-10T05:39:18.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.577+0000 7ffb6d200700 1 --2- 172.21.15.131:0/1951146109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205e800 0x55ea42068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:39:18.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.577+0000 7ffb67400700 1 --2- 172.21.15.131:0/1951146109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4205af90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb54099b70 tx=0x7ffb54098650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=142d8f3e25787845 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:18.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.580+0000 7ffb5e000700 1 -- 172.21.15.131:0/1951146109 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb540a2520 con 0x55ea4205dcd0 2024-05-10T05:39:18.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.580+0000 7ffb5e000700 1 -- 172.21.15.131:0/1951146109 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb5409ded0 con 0x55ea4205dcd0 2024-05-10T05:39:18.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.580+0000 7ffb5e000700 1 -- 172.21.15.131:0/1951146109 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb540b8380 con 0x55ea4205dcd0 2024-05-10T05:39:18.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.583+0000 7ffb87103b80 1 -- 172.21.15.131:0/1951146109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 msgr2=0x55ea4205af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:18.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.583+0000 7ffb87103b80 1 --2- 172.21.15.131:0/1951146109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4205af90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb54099b70 tx=0x7ffb54098650 comp rx=0 tx=0).stop 2024-05-10T05:39:18.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.585+0000 7ffb87103b80 1 -- 172.21.15.131:0/1951146109 shutdown_connections 2024-05-10T05:39:18.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.585+0000 7ffb87103b80 1 --2- 172.21.15.131:0/1951146109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205e800 0x55ea42068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.585+0000 7ffb87103b80 1 --2- 172.21.15.131:0/1951146109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205d380 0x55ea420720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.585+0000 7ffb87103b80 1 --2- 172.21.15.131:0/1951146109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205dcd0 0x55ea4205af90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.585+0000 7ffb87103b80 1 -- 172.21.15.131:0/1951146109 >> 172.21.15.131:0/1951146109 conn(0x55ea41fc2a60 msgr2=0x55ea4205ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:18.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.585+0000 7ffb87103b80 1 -- 172.21.15.131:0/1951146109 shutdown_connections 2024-05-10T05:39:18.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.585+0000 7ffb87103b80 1 -- 172.21.15.131:0/1951146109 wait complete. 2024-05-10T05:39:18.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.586+0000 7ffb87103b80 1 -- start start 2024-05-10T05:39:18.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.586+0000 7ffb87103b80 1 Processor -- start 2024-05-10T05:39:18.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.588+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205d380 0x55ea42064870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.592+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205dcd0 0x55ea42062ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.592+0000 7ffb87103b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205e800 0x55ea42063420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.592+0000 7ffb87103b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea420639f0 con 0x55ea4205dcd0 2024-05-10T05:39:18.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.592+0000 7ffb87103b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea42063b60 con 0x55ea4205d380 2024-05-10T05:39:18.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.592+0000 7ffb87103b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea42063d20 con 0x55ea4205e800 2024-05-10T05:39:18.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.592+0000 7ffb87103b80 10 client.0 _ll_register_callbacks cb 0x55ea41f25b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:18.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.592+0000 7ffb6d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205e800 0x55ea42063420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.593+0000 7ffb6d200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205e800 0x55ea42063420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41236/0 (socket says 172.21.15.131:41236) 2024-05-10T05:39:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.593+0000 7ffb6d200700 1 -- 172.21.15.131:0/2257634894 learned_addr learned my addr 172.21.15.131:0/2257634894 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.593+0000 7ffb67400700 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205dcd0 0x55ea42062ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.593+0000 7ffb67e00700 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205d380 0x55ea42064870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.593+0000 7ffb6d200700 1 -- 172.21.15.131:0/2257634894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205d380 msgr2=0x55ea42064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.593+0000 7ffb6d200700 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205d380 0x55ea42064870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.593+0000 7ffb6d200700 1 -- 172.21.15.131:0/2257634894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205dcd0 msgr2=0x55ea42062ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.593+0000 7ffb6d200700 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205dcd0 0x55ea42062ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.593+0000 7ffb6d200700 1 -- 172.21.15.131:0/2257634894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea41dadb60 con 0x55ea4205e800 2024-05-10T05:39:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.594+0000 7ffb6d200700 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205e800 0x55ea42063420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009700 tx=0x7ffb60004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed4485fe26637aef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.594+0000 7ffb52000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:39:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.595+0000 7ffb52000700 1 -- 172.21.15.131:0/2257634894 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000e040 con 0x55ea4205e800 2024-05-10T05:39:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.596+0000 7ffb52000700 1 -- 172.21.15.131:0/2257634894 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ffb60007ce0 con 0x55ea4205e800 2024-05-10T05:39:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.596+0000 7ffb87103b80 1 -- 172.21.15.131:0/2257634894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb60009360 con 0x55ea4205e800 2024-05-10T05:39:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.596+0000 7ffb52000700 1 -- 172.21.15.131:0/2257634894 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb60007ea0 con 0x55ea4205e800 2024-05-10T05:39:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.596+0000 7ffb87103b80 10 client.26440 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.596+0000 7ffb87103b80 1 -- 172.21.15.131:0/2257634894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ea41e05cd0 con 0x55ea4205e800 2024-05-10T05:39:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.597+0000 7ffb52000700 1 -- 172.21.15.131:0/2257634894 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb60009360 con 0x55ea4205e800 2024-05-10T05:39:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.598+0000 7ffb52000700 1 -- 172.21.15.131:0/2257634894 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffb60019040 con 0x55ea4205e800 2024-05-10T05:39:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.602+0000 7ffb87103b80 10 client.26440 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.602+0000 7ffb87103b80 10 client.26440 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:39:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.602+0000 7ffb87103b80 20 client.26440 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.602+0000 7ffb87103b80 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ea420eec80 0x55ea420f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.602+0000 7ffb87103b80 4 client.26440 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:39:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.602+0000 7ffb87103b80 1 -- 172.21.15.131:0/2257634894 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ea41f78ec0 con 0x55ea420eec80 2024-05-10T05:39:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.602+0000 7ffb67e00700 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ea420eec80 0x55ea420f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.603+0000 7ffb67e00700 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ea420eec80 0x55ea420f10a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7bc103788a937184 server_cookie=e59c80827503aae2 in_seq=0 out_seq=0 2024-05-10T05:39:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.605+0000 7ffb52000700 10 client.26440 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:39:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.605+0000 7ffb52000700 1 -- 172.21.15.131:0/2257634894 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ea41f78ec0 con 0x55ea420eec80 2024-05-10T05:39:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.605+0000 7ffb52000700 10 client.26440 handle_command_reply: tid=0 2024-05-10T05:39:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.605+0000 7ffb87103b80 1 client.26440 shutdown 2024-05-10T05:39:18.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.609+0000 7ffb87103b80 1 -- 172.21.15.131:0/2257634894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205e800 msgr2=0x55ea42063420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:18.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.609+0000 7ffb87103b80 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205e800 0x55ea42063420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009700 tx=0x7ffb60004d30 comp rx=0 tx=0).stop 2024-05-10T05:39:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.609+0000 7ffb87103b80 1 -- 172.21.15.131:0/2257634894 shutdown_connections 2024-05-10T05:39:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.609+0000 7ffb87103b80 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ea420eec80 0x55ea420f10a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.609+0000 7ffb87103b80 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea4205e800 0x55ea42063420 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.609+0000 7ffb87103b80 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea4205d380 0x55ea42064870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.609+0000 7ffb87103b80 1 --2- 172.21.15.131:0/2257634894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea4205dcd0 0x55ea42062ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.609+0000 7ffb87103b80 1 -- 172.21.15.131:0/2257634894 >> 172.21.15.131:0/2257634894 conn(0x55ea41fc2a60 msgr2=0x55ea4205ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.609+0000 7ffb52000700 0 client.26440 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:39:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.610+0000 7ffb87103b80 1 -- 172.21.15.131:0/2257634894 shutdown_connections 2024-05-10T05:39:18.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.610+0000 7ffb87103b80 1 -- 172.21.15.131:0/2257634894 wait complete. 2024-05-10T05:39:18.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.613+0000 7ffb87103b80 20 client.26440 trim_cache size 0 max 16384 2024-05-10T05:39:18.617 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:39:18.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.623+0000 7ffb6f200700 1 -- 172.21.15.131:0/1732611031 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffb4c0a12c0 msgr2=0x7ffb4c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:18.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.623+0000 7ffb6f200700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffb4c0a12c0 0x7ffb4c0a37d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffb580046e0 tx=0x7ffb580097f0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.623+0000 7ffb6f200700 1 -- 172.21.15.131:0/1732611031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb68102d00 msgr2=0x7ffb68073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.623+0000 7ffb6f200700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb68102d00 0x7ffb68073560 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e9c0 tx=0x7ffb5400ed80 comp rx=0 tx=0).stop 2024-05-10T05:39:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.623+0000 7ffb6f200700 1 -- 172.21.15.131:0/1732611031 shutdown_connections 2024-05-10T05:39:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.623+0000 7ffb6f200700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ffb4c0a12c0 0x7ffb4c0a37d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.623+0000 7ffb6f200700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb681036a0 0x7ffb68073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.623+0000 7ffb6f200700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb68101be0 0x7ffb68074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.623+0000 7ffb6f200700 1 --2- 172.21.15.131:0/1732611031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb68102d00 0x7ffb68073560 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.623+0000 7ffb6f200700 1 -- 172.21.15.131:0/1732611031 >> 172.21.15.131:0/1732611031 conn(0x7ffb680fd1d0 msgr2=0x7ffb68105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:18.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.630+0000 7ffb6f200700 1 -- 172.21.15.131:0/1732611031 shutdown_connections 2024-05-10T05:39:18.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:18.630+0000 7ffb6f200700 1 -- 172.21.15.131:0/1732611031 wait complete. 2024-05-10T05:39:18.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:18 smithi142 ceph-mon[77667]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 219 MiB/s wr, 69 op/s 2024-05-10T05:39:18.649 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:18 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1567338174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:39:18.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:18 smithi131 ceph-mon[37264]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 219 MiB/s wr, 69 op/s 2024-05-10T05:39:18.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:18 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1567338174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:39:18.742 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:39:19.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.495+0000 7f127a000700 1 -- 172.21.15.131:0/673044814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c0a6550 msgr2=0x7f126c0a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:19.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.495+0000 7f127a000700 1 --2- 172.21.15.131:0/673044814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c0a6550 0x7f126c0a6960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f12740672f0 tx=0x7f1274069d30 comp rx=0 tx=0).stop 2024-05-10T05:39:19.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.498+0000 7f127a000700 1 -- 172.21.15.131:0/673044814 shutdown_connections 2024-05-10T05:39:19.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.498+0000 7f127a000700 1 --2- 172.21.15.131:0/673044814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c0a6ea0 0x7f126c0ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.498+0000 7f127a000700 1 --2- 172.21.15.131:0/673044814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c0a5570 0x7f126c0a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.498+0000 7f127a000700 1 --2- 172.21.15.131:0/673044814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c0a6550 0x7f126c0a6960 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.498+0000 7f127a000700 1 -- 172.21.15.131:0/673044814 >> 172.21.15.131:0/673044814 conn(0x7f126c0a0a00 msgr2=0x7f126c0a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:19.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.504+0000 7f127a000700 1 -- 172.21.15.131:0/673044814 shutdown_connections 2024-05-10T05:39:19.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.505+0000 7f127a000700 1 -- 172.21.15.131:0/673044814 wait complete. 2024-05-10T05:39:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.506+0000 7f127a000700 1 Processor -- start 2024-05-10T05:39:19.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.508+0000 7f127a000700 1 -- start start 2024-05-10T05:39:19.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.509+0000 7f127a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c0a5570 0x7f126c144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.509+0000 7f127a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c0a6550 0x7f126c144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.510+0000 7f127a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c0a6ea0 0x7f126c14b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.510+0000 7f1273e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c0a6550 0x7f126c144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.511+0000 7f1273e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c0a6550 0x7f126c144580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41250/0 (socket says 172.21.15.131:41250) 2024-05-10T05:39:19.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.511+0000 7f1273e00700 1 -- 172.21.15.131:0/890050668 learned_addr learned my addr 172.21.15.131:0/890050668 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.511+0000 7f1279600700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c0a6ea0 0x7f126c14b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.511+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c14ba50 con 0x7f126c0a5570 2024-05-10T05:39:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.511+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c14bb90 con 0x7f126c0a6ea0 2024-05-10T05:39:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.511+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c14bd20 con 0x7f126c0a6550 2024-05-10T05:39:19.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.513+0000 7f1278c00700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c0a5570 0x7f126c144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.513+0000 7f1273e00700 1 -- 172.21.15.131:0/890050668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c0a6ea0 msgr2=0x7f126c14b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.513+0000 7f1273e00700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c0a6ea0 0x7f126c14b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.514+0000 7f1273e00700 1 -- 172.21.15.131:0/890050668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c0a5570 msgr2=0x7f126c144040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.514+0000 7f1273e00700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c0a5570 0x7f126c144040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.514+0000 7f1273e00700 1 -- 172.21.15.131:0/890050668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1274069700 con 0x7f126c0a6550 2024-05-10T05:39:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.514+0000 7f1279600700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c0a6ea0 0x7f126c14b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.514+0000 7f1273e00700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c0a6550 0x7f126c144580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1274066e70 tx=0x7f1274067280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:19.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.515+0000 7f1271600700 1 -- 172.21.15.131:0/890050668 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127406fb70 con 0x7f126c0a6550 2024-05-10T05:39:19.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.516+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c14bfa0 con 0x7f126c0a6550 2024-05-10T05:39:19.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.516+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c14c3c0 con 0x7f126c0a6550 2024-05-10T05:39:19.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.516+0000 7f1271600700 1 -- 172.21.15.131:0/890050668 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127406fcd0 con 0x7f126c0a6550 2024-05-10T05:39:19.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.518+0000 7f1271600700 1 -- 172.21.15.131:0/890050668 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274073480 con 0x7f126c0a6550 2024-05-10T05:39:19.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.525+0000 7f1271600700 1 -- 172.21.15.131:0/890050668 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f127407b040 con 0x7f126c0a6550 2024-05-10T05:39:19.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.541+0000 7f12935f4b80 1 Processor -- start 2024-05-10T05:39:19.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.541+0000 7f12935f4b80 1 -- start start 2024-05-10T05:39:19.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.542+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0a530 0x55d5a8f0a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.542+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f0bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.542+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.542+0000 7f12935f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8e31150 con 0x55d5a8f0a530 2024-05-10T05:39:19.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.542+0000 7f12935f4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8e2eb10 con 0x55d5a8f0b7a0 2024-05-10T05:39:19.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.542+0000 7f12935f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8e2a400 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.543+0000 7f1273e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f0bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.543+0000 7f1273e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f0bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37296/0 (socket says 172.21.15.131:37296) 2024-05-10T05:39:19.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.543+0000 7f1273e00700 1 -- 172.21.15.131:0/1036809587 learned_addr learned my addr 172.21.15.131:0/1036809587 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:19.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.543+0000 7f1279600700 1 --2- 172.21.15.131:0/1036809587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.544+0000 7f1271600700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f125c0a1270 0x7f125c0a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.544+0000 7f1271600700 1 -- 172.21.15.131:0/890050668 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1274130fd0 con 0x7f126c0a6550 2024-05-10T05:39:19.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.546+0000 7f1273e00700 1 -- 172.21.15.131:0/1036809587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0c0f0 msgr2=0x55d5a8f129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:19.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.546+0000 7f1273e00700 1 --2- 172.21.15.131:0/1036809587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.546+0000 7f1273e00700 1 -- 172.21.15.131:0/1036809587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0a530 msgr2=0x55d5a8f0a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:19.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.546+0000 7f1273e00700 1 --2- 172.21.15.131:0/1036809587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0a530 0x55d5a8f0a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.548+0000 7f1278c00700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f125c0a1270 0x7f125c0a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.550+0000 7f1273e00700 1 -- 172.21.15.131:0/1036809587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5a8cab3b0 con 0x55d5a8f0b7a0 2024-05-10T05:39:19.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.552+0000 7f1273e00700 1 --2- 172.21.15.131:0/1036809587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f0bbb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f12740fe440 tx=0x7f1274069d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fac6f3f64c70f97 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:19.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.554+0000 7f1265c00700 1 -- 172.21.15.131:0/1036809587 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740fde10 con 0x55d5a8f0b7a0 2024-05-10T05:39:19.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.554+0000 7f1265c00700 1 -- 172.21.15.131:0/1036809587 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12740febb0 con 0x55d5a8f0b7a0 2024-05-10T05:39:19.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.554+0000 7f1265c00700 1 -- 172.21.15.131:0/1036809587 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740fde10 con 0x55d5a8f0b7a0 2024-05-10T05:39:19.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.556+0000 7f12935f4b80 1 -- 172.21.15.131:0/1036809587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 msgr2=0x55d5a8f0bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:19.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.556+0000 7f12935f4b80 1 --2- 172.21.15.131:0/1036809587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f0bbb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f12740fe440 tx=0x7f1274069d30 comp rx=0 tx=0).stop 2024-05-10T05:39:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.561+0000 7f1278c00700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f125c0a1270 0x7f125c0a3780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55d5a8f0a2d0 tx=0x7f12600092a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.563+0000 7f12935f4b80 1 -- 172.21.15.131:0/1036809587 shutdown_connections 2024-05-10T05:39:19.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.563+0000 7f12935f4b80 1 --2- 172.21.15.131:0/1036809587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0a530 0x55d5a8f0a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.563+0000 7f12935f4b80 1 --2- 172.21.15.131:0/1036809587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.563+0000 7f12935f4b80 1 --2- 172.21.15.131:0/1036809587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f0bbb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.563+0000 7f12935f4b80 1 -- 172.21.15.131:0/1036809587 >> 172.21.15.131:0/1036809587 conn(0x55d5a8e6ca60 msgr2=0x55d5a8f05330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.572+0000 7f12935f4b80 1 -- 172.21.15.131:0/1036809587 shutdown_connections 2024-05-10T05:39:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.575+0000 7f12935f4b80 1 -- 172.21.15.131:0/1036809587 wait complete. 2024-05-10T05:39:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.583+0000 7f12935f4b80 1 -- start start 2024-05-10T05:39:19.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.586+0000 7f12935f4b80 1 Processor -- start 2024-05-10T05:39:19.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.608+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f14480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.608+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f149c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.609+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f1b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.609+0000 7f12935f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8dc6560 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.609+0000 7f12935f4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8dc6200 con 0x55d5a8f0b7a0 2024-05-10T05:39:19.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.609+0000 7f12935f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8cad6a0 con 0x55d5a8f0a530 2024-05-10T05:39:19.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.609+0000 7f12935f4b80 10 client.0 _ll_register_callbacks cb 0x55d5a8dcfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:19.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.609+0000 7f1278c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f14480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.609+0000 7f1278c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f14480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41260/0 (socket says 172.21.15.131:41260) 2024-05-10T05:39:19.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.609+0000 7f1278c00700 1 -- 172.21.15.131:0/3186825186 learned_addr learned my addr 172.21.15.131:0/3186825186 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:19.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.610+0000 7f1273e00700 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.612+0000 7f1278c00700 1 -- 172.21.15.131:0/3186825186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 msgr2=0x55d5a8f149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:19.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.612+0000 7f1278c00700 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f149c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.612+0000 7f1278c00700 1 -- 172.21.15.131:0/3186825186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0c0f0 msgr2=0x55d5a8f1b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:19.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.612+0000 7f1278c00700 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f1b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.614+0000 7f1278c00700 1 -- 172.21.15.131:0/3186825186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c14c3c0 con 0x55d5a8f0a530 2024-05-10T05:39:19.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.614+0000 7f1273e00700 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:19.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.624+0000 7f1278c00700 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f14480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f126000fea0 tx=0x7f12600047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82564cf15653e1e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:19.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.624+0000 7f125a000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:39:19.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.628+0000 7f125a000700 1 -- 172.21.15.131:0/3186825186 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260011ce0 con 0x55d5a8f0a530 2024-05-10T05:39:19.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.628+0000 7f125a000700 1 -- 172.21.15.131:0/3186825186 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1260011e40 con 0x55d5a8f0a530 2024-05-10T05:39:19.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.629+0000 7f12935f4b80 1 -- 172.21.15.131:0/3186825186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5a8d83270 con 0x55d5a8f0a530 2024-05-10T05:39:19.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.629+0000 7f125a000700 1 -- 172.21.15.131:0/3186825186 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1260026440 con 0x55d5a8f0a530 2024-05-10T05:39:19.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.629+0000 7f12935f4b80 10 client.26452 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:19.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.629+0000 7f12935f4b80 1 -- 172.21.15.131:0/3186825186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d5a8c57b60 con 0x55d5a8f0a530 2024-05-10T05:39:19.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.630+0000 7f125a000700 1 -- 172.21.15.131:0/3186825186 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5a8d83270 con 0x55d5a8f0a530 2024-05-10T05:39:19.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.632+0000 7f125a000700 1 -- 172.21.15.131:0/3186825186 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f126001d7c0 con 0x55d5a8f0a530 2024-05-10T05:39:19.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.632+0000 7f12935f4b80 10 client.26452 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:19.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.632+0000 7f12935f4b80 10 client.26452 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:39:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.632+0000 7f12935f4b80 20 client.26452 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.632+0000 7f12935f4b80 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d5a8f201f0 0x55d5a8f22640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.633+0000 7f12935f4b80 4 client.26452 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.633+0000 7f12935f4b80 1 -- 172.21.15.131:0/3186825186 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5a8ed9780 con 0x55d5a8f201f0 2024-05-10T05:39:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.633+0000 7f1273e00700 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d5a8f201f0 0x55d5a8f22640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.634+0000 7f125a000700 10 client.26452 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:39:19.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.639+0000 7f1273e00700 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d5a8f201f0 0x55d5a8f22640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=448a3d320d47935b server_cookie=6100478faeb4c9b9 in_seq=0 out_seq=0 2024-05-10T05:39:19.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.641+0000 7f125a000700 1 -- 172.21.15.131:0/3186825186 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5a8ed9780 con 0x55d5a8f201f0 2024-05-10T05:39:19.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.641+0000 7f125a000700 10 client.26452 handle_command_reply: tid=0 2024-05-10T05:39:19.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.644+0000 7f12935f4b80 1 client.26452 shutdown 2024-05-10T05:39:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.647+0000 7f12935f4b80 1 -- 172.21.15.131:0/3186825186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 msgr2=0x55d5a8f14480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.647+0000 7f12935f4b80 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f14480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f126000fea0 tx=0x7f12600047c0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.648+0000 7f12935f4b80 1 -- 172.21.15.131:0/3186825186 shutdown_connections 2024-05-10T05:39:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.648+0000 7f12935f4b80 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d5a8f201f0 0x55d5a8f22640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.648+0000 7f12935f4b80 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f1b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.648+0000 7f125a000700 0 client.26452 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:39:19.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.648+0000 7f12935f4b80 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f14480 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.648+0000 7f12935f4b80 1 --2- 172.21.15.131:0/3186825186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.648+0000 7f12935f4b80 1 -- 172.21.15.131:0/3186825186 >> 172.21.15.131:0/3186825186 conn(0x55d5a8e6ca60 msgr2=0x55d5a8f09060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:19.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.648+0000 7f12935f4b80 1 -- 172.21.15.131:0/3186825186 shutdown_connections 2024-05-10T05:39:19.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.649+0000 7f12935f4b80 1 -- 172.21.15.131:0/3186825186 wait complete. 2024-05-10T05:39:19.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.650+0000 7f12935f4b80 20 client.26452 trim_cache size 0 max 16384 2024-05-10T05:39:19.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.680+0000 7f12935f4b80 1 Processor -- start 2024-05-10T05:39:19.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.681+0000 7f12935f4b80 1 -- start start 2024-05-10T05:39:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.682+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f13710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.682+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f1b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.683+0000 7f1278c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f13710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.683+0000 7f1278c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f13710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41264/0 (socket says 172.21.15.131:41264) 2024-05-10T05:39:19.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.683+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f12aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.684+0000 7f12935f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8f155f0 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.684+0000 7f12935f4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8f06bd0 con 0x55d5a8f0b7a0 2024-05-10T05:39:19.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.684+0000 7f12935f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8f0faa0 con 0x55d5a8f0a530 2024-05-10T05:39:19.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.684+0000 7f1279600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f12aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.684+0000 7f1278c00700 1 -- 172.21.15.131:0/2914331899 learned_addr learned my addr 172.21.15.131:0/2914331899 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:19.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.685+0000 7f1273e00700 1 --2- 172.21.15.131:0/2914331899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f1b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.686+0000 7f1279600700 1 -- 172.21.15.131:0/2914331899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 msgr2=0x55d5a8f13710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:19.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.687+0000 7f1279600700 1 --2- 172.21.15.131:0/2914331899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f13710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.687+0000 7f1279600700 1 -- 172.21.15.131:0/2914331899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 msgr2=0x55d5a8f1b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:19.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.687+0000 7f1279600700 1 --2- 172.21.15.131:0/2914331899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f1b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.687+0000 7f1279600700 1 -- 172.21.15.131:0/2914331899 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5a8df0560 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.687+0000 7f1273e00700 1 --2- 172.21.15.131:0/2914331899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f1b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.687+0000 7f1278c00700 1 --2- 172.21.15.131:0/2914331899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f13710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.688+0000 7f1279600700 1 --2- 172.21.15.131:0/2914331899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f12aa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f12680127e0 tx=0x7f1268012af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98abb372fe782063 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.689+0000 7f1266600700 1 -- 172.21.15.131:0/2914331899 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268004250 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.689+0000 7f1266600700 1 -- 172.21.15.131:0/2914331899 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12680043b0 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.689+0000 7f1266600700 1 -- 172.21.15.131:0/2914331899 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126800f9b0 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.698+0000 7f12935f4b80 1 -- 172.21.15.131:0/2914331899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0c0f0 msgr2=0x55d5a8f12aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:19.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.698+0000 7f12935f4b80 1 --2- 172.21.15.131:0/2914331899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f12aa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f12680127e0 tx=0x7f1268012af0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.701+0000 7f12935f4b80 1 -- 172.21.15.131:0/2914331899 shutdown_connections 2024-05-10T05:39:19.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.701+0000 7f12935f4b80 1 --2- 172.21.15.131:0/2914331899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f12aa0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.701+0000 7f12935f4b80 1 --2- 172.21.15.131:0/2914331899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f13710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.702+0000 7f12935f4b80 1 --2- 172.21.15.131:0/2914331899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f1b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.702+0000 7f12935f4b80 1 -- 172.21.15.131:0/2914331899 >> 172.21.15.131:0/2914331899 conn(0x55d5a8e6ca60 msgr2=0x55d5a8f0f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:19.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.705+0000 7f12935f4b80 1 -- 172.21.15.131:0/2914331899 shutdown_connections 2024-05-10T05:39:19.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.705+0000 7f12935f4b80 1 -- 172.21.15.131:0/2914331899 wait complete. 2024-05-10T05:39:19.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.706+0000 7f12935f4b80 1 -- start start 2024-05-10T05:39:19.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.707+0000 7f12935f4b80 1 Processor -- start 2024-05-10T05:39:19.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.711+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f190e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.716+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f19620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.717+0000 7f12935f4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f1a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.717+0000 7f12935f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8f1aa90 con 0x55d5a8f0b7a0 2024-05-10T05:39:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.717+0000 7f12935f4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8f1ac00 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.717+0000 7f12935f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a8f1adc0 con 0x55d5a8f0a530 2024-05-10T05:39:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.717+0000 7f12935f4b80 10 client.0 _ll_register_callbacks cb 0x55d5a8dcfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.717+0000 7f1279600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f1a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.718+0000 7f1279600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f1a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37320/0 (socket says 172.21.15.131:37320) 2024-05-10T05:39:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.718+0000 7f1279600700 1 -- 172.21.15.131:0/414632284 learned_addr learned my addr 172.21.15.131:0/414632284 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.718+0000 7f1279600700 1 -- 172.21.15.131:0/414632284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 msgr2=0x55d5a8f190e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:39:19.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.719+0000 7f1278c00700 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f190e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.720+0000 7f1279600700 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f190e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.720+0000 7f1279600700 1 -- 172.21.15.131:0/414632284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0b7a0 msgr2=0x55d5a8f19620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:19.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.720+0000 7f1279600700 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f19620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.720+0000 7f1279600700 1 -- 172.21.15.131:0/414632284 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5a8c57b60 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.720+0000 7f1278c00700 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f190e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.721+0000 7f1279600700 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f1a550 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1268005950 tx=0x7f1268003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c92bd9cf172541ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.721+0000 7f125a000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:39:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.721+0000 7f125a000700 1 -- 172.21.15.131:0/414632284 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268026070 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.721+0000 7f125a000700 1 -- 172.21.15.131:0/414632284 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1268004520 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.724+0000 7f12935f4b80 1 -- 172.21.15.131:0/414632284 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12740febb0 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.725+0000 7f125a000700 1 -- 172.21.15.131:0/414632284 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1268005030 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.725+0000 7f12935f4b80 10 client.26429 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:19.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.725+0000 7f12935f4b80 1 -- 172.21.15.131:0/414632284 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d5a8cab3b0 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.727+0000 7f125a000700 1 -- 172.21.15.131:0/414632284 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12740febb0 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.727+0000 7f125a000700 1 -- 172.21.15.131:0/414632284 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1268029020 con 0x55d5a8f0c0f0 2024-05-10T05:39:19.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.729+0000 7f12935f4b80 10 client.26429 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:19.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.729+0000 7f12935f4b80 10 client.26429 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:39:19.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.729+0000 7f12935f4b80 20 client.26429 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:19.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.729+0000 7f12935f4b80 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d5a8f201f0 0x55d5a8f78ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:19.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.729+0000 7f12935f4b80 4 client.26429 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:39:19.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.729+0000 7f12935f4b80 1 -- 172.21.15.131:0/414632284 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d5a8e22ec0 con 0x55d5a8f201f0 2024-05-10T05:39:19.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.730+0000 7f1278c00700 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d5a8f201f0 0x55d5a8f78ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:19.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.734+0000 7f1278c00700 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d5a8f201f0 0x55d5a8f78ec0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79392fa84decc4cc server_cookie=9bfc29ce204b2cbd in_seq=0 out_seq=0 2024-05-10T05:39:19.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.735+0000 7f125a000700 10 client.26429 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:39:19.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.735+0000 7f125a000700 1 -- 172.21.15.131:0/414632284 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d5a8e22ec0 con 0x55d5a8f201f0 2024-05-10T05:39:19.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.735+0000 7f125a000700 10 client.26429 handle_command_reply: tid=0 2024-05-10T05:39:19.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.737+0000 7f12935f4b80 1 client.26429 shutdown 2024-05-10T05:39:19.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.743+0000 7f12935f4b80 1 -- 172.21.15.131:0/414632284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0c0f0 msgr2=0x55d5a8f1a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:19.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.743+0000 7f12935f4b80 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f1a550 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1268005950 tx=0x7f1268003960 comp rx=0 tx=0).stop 2024-05-10T05:39:19.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.744+0000 7f12935f4b80 1 -- 172.21.15.131:0/414632284 shutdown_connections 2024-05-10T05:39:19.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.744+0000 7f12935f4b80 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d5a8f201f0 0x55d5a8f78ec0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.744+0000 7f12935f4b80 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5a8f0c0f0 0x55d5a8f1a550 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.744+0000 7f12935f4b80 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5a8f0a530 0x55d5a8f190e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.745+0000 7f12935f4b80 1 --2- 172.21.15.131:0/414632284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a8f0b7a0 0x55d5a8f19620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.745+0000 7f12935f4b80 1 -- 172.21.15.131:0/414632284 >> 172.21.15.131:0/414632284 conn(0x55d5a8e6ca60 msgr2=0x55d5a8f0f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:19.754 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:39:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.749+0000 7f125a000700 0 client.26429 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:39:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.750+0000 7f12935f4b80 1 -- 172.21.15.131:0/414632284 shutdown_connections 2024-05-10T05:39:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.751+0000 7f12935f4b80 1 -- 172.21.15.131:0/414632284 wait complete. 2024-05-10T05:39:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.751+0000 7f12935f4b80 20 client.26429 trim_cache size 0 max 16384 2024-05-10T05:39:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.753+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f125c0a1270 msgr2=0x7f125c0a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.753+0000 7f127a000700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f125c0a1270 0x7f125c0a3780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55d5a8f0a2d0 tx=0x7f12600092a0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.753+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c0a6550 msgr2=0x7f126c144580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.753+0000 7f127a000700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c0a6550 0x7f126c144580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1274066e70 tx=0x7f1274067280 comp rx=0 tx=0).stop 2024-05-10T05:39:19.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.760+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 shutdown_connections 2024-05-10T05:39:19.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.762+0000 7f127a000700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f125c0a1270 0x7f125c0a3780 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.762+0000 7f127a000700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f126c0a5570 0x7f126c144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.762+0000 7f127a000700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f126c0a6ea0 0x7f126c14b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.762+0000 7f127a000700 1 --2- 172.21.15.131:0/890050668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f126c0a6550 0x7f126c144580 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:19.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.762+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 >> 172.21.15.131:0/890050668 conn(0x7f126c0a0a00 msgr2=0x7f126c0a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:19.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.763+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 shutdown_connections 2024-05-10T05:39:19.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:19.763+0000 7f127a000700 1 -- 172.21.15.131:0/890050668 wait complete. 2024-05-10T05:39:19.836 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:39:20.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:20 smithi157 ceph-mon[78427]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:39:20.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:20 smithi142 ceph-mon[77667]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:39:20.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.688+0000 7f73b6000700 1 -- 172.21.15.131:0/1503397876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b0109cd0 msgr2=0x7f73b0110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:20.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.688+0000 7f73b6000700 1 --2- 172.21.15.131:0/1503397876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b0109cd0 0x7f73b0110580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009920 tx=0x7f73a0009c30 comp rx=0 tx=0).stop 2024-05-10T05:39:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.689+0000 7f73b6000700 1 -- 172.21.15.131:0/1503397876 shutdown_connections 2024-05-10T05:39:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.689+0000 7f73b6000700 1 --2- 172.21.15.131:0/1503397876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73b0108210 0x7f73b0108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.689+0000 7f73b6000700 1 --2- 172.21.15.131:0/1503397876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b0109cd0 0x7f73b0110580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.689+0000 7f73b6000700 1 --2- 172.21.15.131:0/1503397876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b0109330 0x7f73b0109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.689+0000 7f73b6000700 1 -- 172.21.15.131:0/1503397876 >> 172.21.15.131:0/1503397876 conn(0x7f73b0077240 msgr2=0x7f73b0106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.690+0000 7f73b6000700 1 -- 172.21.15.131:0/1503397876 shutdown_connections 2024-05-10T05:39:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.690+0000 7f73b6000700 1 -- 172.21.15.131:0/1503397876 wait complete. 2024-05-10T05:39:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.696+0000 7f73b6000700 1 Processor -- start 2024-05-10T05:39:20.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.698+0000 7f73b6000700 1 -- start start 2024-05-10T05:39:20.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.698+0000 7f73b6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73b0108210 0x7f73b019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.698+0000 7f73b6000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b0109330 0x7f73b019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.698+0000 7f73b6000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b0109cd0 0x7f73b01a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.699+0000 7f73aea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b0109330 0x7f73b019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.699+0000 7f73aea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b0109330 0x7f73b019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41300/0 (socket says 172.21.15.131:41300) 2024-05-10T05:39:20.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.700+0000 7f73af400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73b0108210 0x7f73b019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.700+0000 7f73af400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73b0108210 0x7f73b019e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32962/0 (socket says 172.21.15.131:32962) 2024-05-10T05:39:20.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.701+0000 7f73aea00700 1 -- 172.21.15.131:0/318318880 learned_addr learned my addr 172.21.15.131:0/318318880 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:20.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.701+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b01a6010 con 0x7f73b0108210 2024-05-10T05:39:20.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.701+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b01a6180 con 0x7f73b0109cd0 2024-05-10T05:39:20.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.701+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b01a6340 con 0x7f73b0109330 2024-05-10T05:39:20.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.702+0000 7f73aea00700 1 -- 172.21.15.131:0/318318880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b0109cd0 msgr2=0x7f73b01a5a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:39:20.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.702+0000 7f73aea00700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b0109cd0 0x7f73b01a5a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.702+0000 7f73aea00700 1 -- 172.21.15.131:0/318318880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73b0108210 msgr2=0x7f73b019e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:20.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.702+0000 7f73aea00700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73b0108210 0x7f73b019e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.702+0000 7f73aea00700 1 -- 172.21.15.131:0/318318880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a0009520 con 0x7f73b0109330 2024-05-10T05:39:20.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.703+0000 7f73aea00700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b0109330 0x7f73b019ea70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f739c00d980 tx=0x7f739c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:20.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.704+0000 7f73a7e00700 1 -- 172.21.15.131:0/318318880 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c009930 con 0x7f73b0109330 2024-05-10T05:39:20.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.704+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b01a6620 con 0x7f73b0109330 2024-05-10T05:39:20.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.704+0000 7f73a7e00700 1 -- 172.21.15.131:0/318318880 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f739c00b6a0 con 0x7f73b0109330 2024-05-10T05:39:20.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.706+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b01a6b80 con 0x7f73b0109330 2024-05-10T05:39:20.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.706+0000 7f73a7e00700 1 -- 172.21.15.131:0/318318880 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c009930 con 0x7f73b0109330 2024-05-10T05:39:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.714+0000 7f73cdf73b80 1 Processor -- start 2024-05-10T05:39:20.716 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:20 smithi131 ceph-mon[37264]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 72 op/s 2024-05-10T05:39:20.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.715+0000 7f73cdf73b80 1 -- start start 2024-05-10T05:39:20.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.716+0000 7f73cdf73b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 0x55bc2425aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.717+0000 7f73afe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 0x55bc2425aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.717+0000 7f73afe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 0x55bc2425aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37348/0 (socket says 172.21.15.131:37348) 2024-05-10T05:39:20.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.717+0000 7f73afe00700 1 -- 172.21.15.131:0/1102268472 learned_addr learned my addr 172.21.15.131:0/1102268472 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:20.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.717+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/1102268472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 0x55bc2425b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.718+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/1102268472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 0x55bc242627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.718+0000 7f73cdf73b80 1 -- 172.21.15.131:0/1102268472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc24181150 con 0x55bc2425b570 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.718+0000 7f73cdf73b80 1 -- 172.21.15.131:0/1102268472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2417eb10 con 0x55bc2425a6a0 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.718+0000 7f73cdf73b80 1 -- 172.21.15.131:0/1102268472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2417a400 con 0x55bc2425bec0 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.718+0000 7f73af400700 1 --2- 172.21.15.131:0/1102268472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 0x55bc2425b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.718+0000 7f73afe00700 1 -- 172.21.15.131:0/1102268472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 msgr2=0x55bc242627c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.718+0000 7f73afe00700 1 --2- 172.21.15.131:0/1102268472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 0x55bc242627c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.718+0000 7f73afe00700 1 -- 172.21.15.131:0/1102268472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 msgr2=0x55bc2425b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.718+0000 7f73afe00700 1 --2- 172.21.15.131:0/1102268472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 0x55bc2425b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.718+0000 7f73afe00700 1 -- 172.21.15.131:0/1102268472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc23ffb3b0 con 0x55bc2425a6a0 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.719+0000 7f73afe00700 1 --2- 172.21.15.131:0/1102268472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 0x55bc2425aaf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009810 tx=0x7f73a0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c075208a91859d70 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.719+0000 7f739be00700 1 -- 172.21.15.131:0/1102268472 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000f6e0 con 0x55bc2425a6a0 2024-05-10T05:39:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.719+0000 7f739be00700 1 -- 172.21.15.131:0/1102268472 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a000f840 con 0x55bc2425a6a0 2024-05-10T05:39:20.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.719+0000 7f739be00700 1 -- 172.21.15.131:0/1102268472 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000be60 con 0x55bc2425a6a0 2024-05-10T05:39:20.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.720+0000 7f73a7e00700 1 -- 172.21.15.131:0/318318880 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f739c0414e0 con 0x7f73b0109330 2024-05-10T05:39:20.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.721+0000 7f73cdf73b80 1 -- 172.21.15.131:0/1102268472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 msgr2=0x55bc2425aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:20.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.721+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/1102268472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 0x55bc2425aaf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009810 tx=0x7f73a0005e80 comp rx=0 tx=0).stop 2024-05-10T05:39:20.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.722+0000 7f73a7e00700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73940a1380 0x7f73940a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.722+0000 7f73cdf73b80 1 -- 172.21.15.131:0/1102268472 shutdown_connections 2024-05-10T05:39:20.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.722+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/1102268472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 0x55bc242627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.722+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/1102268472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 0x55bc2425aaf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.722+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/1102268472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 0x55bc2425b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.722+0000 7f73cdf73b80 1 -- 172.21.15.131:0/1102268472 >> 172.21.15.131:0/1102268472 conn(0x55bc241bca60 msgr2=0x55bc24254fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:20.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.723+0000 7f73a7e00700 1 -- 172.21.15.131:0/318318880 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f739c0cba00 con 0x7f73b0109330 2024-05-10T05:39:20.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.724+0000 7f73cdf73b80 1 -- 172.21.15.131:0/1102268472 shutdown_connections 2024-05-10T05:39:20.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.725+0000 7f73afe00700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73940a1380 0x7f73940a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.725+0000 7f73cdf73b80 1 -- 172.21.15.131:0/1102268472 wait complete. 2024-05-10T05:39:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.726+0000 7f73cdf73b80 1 -- start start 2024-05-10T05:39:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.726+0000 7f73cdf73b80 1 Processor -- start 2024-05-10T05:39:20.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.728+0000 7f73afe00700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73940a1380 0x7f73940a3890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f73a000fad0 tx=0x7f73a0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:20.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.729+0000 7f73cdf73b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 0x55bc242646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.729+0000 7f73cdf73b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 0x55bc24264c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.729+0000 7f73cdf73b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 0x55bc2426bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.730+0000 7f73cdf73b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc24116560 con 0x55bc2425b570 2024-05-10T05:39:20.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.730+0000 7f73cdf73b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc24116200 con 0x55bc2425a6a0 2024-05-10T05:39:20.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.730+0000 7f73cdf73b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc23ffd6a0 con 0x55bc2425bec0 2024-05-10T05:39:20.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.730+0000 7f73afe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 0x55bc2426bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.730+0000 7f73afe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 0x55bc2426bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:41304/0 (socket says 172.21.15.131:41304) 2024-05-10T05:39:20.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.731+0000 7f73afe00700 1 -- 172.21.15.131:0/853313434 learned_addr learned my addr 172.21.15.131:0/853313434 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:20.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.731+0000 7f73cdf73b80 10 client.0 _ll_register_callbacks cb 0x55bc2411fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.731+0000 7f73afe00700 1 -- 172.21.15.131:0/853313434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 msgr2=0x55bc242646d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:39:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.732+0000 7f73afe00700 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 0x55bc242646d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.732+0000 7f73afe00700 1 -- 172.21.15.131:0/853313434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 msgr2=0x55bc24264c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:39:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.732+0000 7f73afe00700 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 0x55bc24264c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.732+0000 7f73afe00700 1 -- 172.21.15.131:0/853313434 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b0111010 con 0x55bc2425bec0 2024-05-10T05:39:20.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.733+0000 7f73afe00700 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 0x55bc2426bb70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73a00052a0 tx=0x7f73a0005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5fec4fd5a9b96e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.733+0000 7f7398c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.734+0000 7f7398c00700 1 -- 172.21.15.131:0/853313434 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0003880 con 0x55bc2425bec0 2024-05-10T05:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.734+0000 7f7398c00700 1 -- 172.21.15.131:0/853313434 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f73a00039e0 con 0x55bc2425bec0 2024-05-10T05:39:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.735+0000 7f73cdf73b80 1 -- 172.21.15.131:0/853313434 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc240d3270 con 0x55bc2425bec0 2024-05-10T05:39:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.737+0000 7f7398c00700 1 -- 172.21.15.131:0/853313434 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73a0003ba0 con 0x55bc2425bec0 2024-05-10T05:39:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.737+0000 7f73cdf73b80 10 client.26464 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.737+0000 7f73cdf73b80 1 -- 172.21.15.131:0/853313434 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bc23fa7b60 con 0x55bc2425bec0 2024-05-10T05:39:20.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.738+0000 7f7398c00700 1 -- 172.21.15.131:0/853313434 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc240d3270 con 0x55bc2425bec0 2024-05-10T05:39:20.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.742+0000 7f7398c00700 1 -- 172.21.15.131:0/853313434 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f73a0036410 con 0x55bc2425bec0 2024-05-10T05:39:20.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.744+0000 7f73cdf73b80 10 client.26464 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:20.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.745+0000 7f73cdf73b80 10 client.26464 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:39:20.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.745+0000 7f73cdf73b80 20 client.26464 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:20.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.745+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bc2426ff40 0x55bc24272390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.745+0000 7f73cdf73b80 4 client.26464 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:20.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.745+0000 7f73cdf73b80 1 -- 172.21.15.131:0/853313434 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc24229780 con 0x55bc2426ff40 2024-05-10T05:39:20.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.750+0000 7f73af400700 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bc2426ff40 0x55bc24272390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.755+0000 7f73af400700 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bc2426ff40 0x55bc24272390 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2da9ad2b06abced server_cookie=92269ea4025a28dd in_seq=0 out_seq=0 2024-05-10T05:39:20.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.755+0000 7f7398c00700 10 client.26464 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:39:20.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.757+0000 7f7398c00700 1 -- 172.21.15.131:0/853313434 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc24229780 con 0x55bc2426ff40 2024-05-10T05:39:20.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.757+0000 7f7398c00700 10 client.26464 handle_command_reply: tid=0 2024-05-10T05:39:20.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.757+0000 7f73cdf73b80 1 client.26464 shutdown 2024-05-10T05:39:20.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.760+0000 7f73cdf73b80 1 -- 172.21.15.131:0/853313434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 msgr2=0x55bc2426bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:20.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.760+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 0x55bc2426bb70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73a00052a0 tx=0x7f73a0005e80 comp rx=0 tx=0).stop 2024-05-10T05:39:20.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.761+0000 7f73cdf73b80 1 -- 172.21.15.131:0/853313434 shutdown_connections 2024-05-10T05:39:20.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.761+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bc2426ff40 0x55bc24272390 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.762+0000 7f7398c00700 0 client.26464 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:39:20.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.762+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425bec0 0x55bc2426bb70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.762+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc2425a6a0 0x55bc242646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.762+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/853313434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 0x55bc24264c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.762+0000 7f73cdf73b80 1 -- 172.21.15.131:0/853313434 >> 172.21.15.131:0/853313434 conn(0x55bc241bca60 msgr2=0x55bc24254fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:20.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.765+0000 7f73cdf73b80 1 -- 172.21.15.131:0/853313434 shutdown_connections 2024-05-10T05:39:20.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.765+0000 7f73cdf73b80 1 -- 172.21.15.131:0/853313434 wait complete. 2024-05-10T05:39:20.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.766+0000 7f73cdf73b80 20 client.26464 trim_cache size 0 max 16384 2024-05-10T05:39:20.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.791+0000 7f73cdf73b80 1 Processor -- start 2024-05-10T05:39:20.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.791+0000 7f73cdf73b80 1 -- start start 2024-05-10T05:39:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.792+0000 7f73cdf73b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc2425e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.792+0000 7f73cdf73b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425a6a0 0x55bc2426b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.792+0000 7f73af400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc2425e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.792+0000 7f73af400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc2425e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37372/0 (socket says 172.21.15.131:37372) 2024-05-10T05:39:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.792+0000 7f73cdf73b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425b570 0x55bc24264690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.792+0000 7f73cdf73b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2426c3d0 con 0x55bc2425a6a0 2024-05-10T05:39:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.792+0000 7f73cdf73b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2426c540 con 0x55bc241bca60 2024-05-10T05:39:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.792+0000 7f73cdf73b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2425ece0 con 0x55bc2425b570 2024-05-10T05:39:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.793+0000 7f73af400700 1 -- 172.21.15.131:0/3071915933 learned_addr learned my addr 172.21.15.131:0/3071915933 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.793+0000 7f73afe00700 1 --2- 172.21.15.131:0/3071915933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425b570 0x55bc24264690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.794+0000 7f73aea00700 1 --2- 172.21.15.131:0/3071915933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425a6a0 0x55bc2426b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.795+0000 7f73afe00700 1 -- 172.21.15.131:0/3071915933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 msgr2=0x55bc2425e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.795+0000 7f73afe00700 1 --2- 172.21.15.131:0/3071915933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc2425e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.795+0000 7f73afe00700 1 -- 172.21.15.131:0/3071915933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425a6a0 msgr2=0x55bc2426b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.796+0000 7f73afe00700 1 --2- 172.21.15.131:0/3071915933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425a6a0 0x55bc2426b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.796+0000 7f73afe00700 1 -- 172.21.15.131:0/3071915933 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc24140560 con 0x55bc2425b570 2024-05-10T05:39:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.798+0000 7f73aea00700 1 --2- 172.21.15.131:0/3071915933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425a6a0 0x55bc2426b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:20.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.798+0000 7f73afe00700 1 --2- 172.21.15.131:0/3071915933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425b570 0x55bc24264690 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f73a000fca0 tx=0x7f73a0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d63a08c7a0bc25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:20.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.802+0000 7f73a5200700 1 -- 172.21.15.131:0/3071915933 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0046070 con 0x55bc2425b570 2024-05-10T05:39:20.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.803+0000 7f73a5200700 1 -- 172.21.15.131:0/3071915933 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a003c030 con 0x55bc2425b570 2024-05-10T05:39:20.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.803+0000 7f73a5200700 1 -- 172.21.15.131:0/3071915933 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0046070 con 0x55bc2425b570 2024-05-10T05:39:20.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.807+0000 7f73cdf73b80 1 -- 172.21.15.131:0/3071915933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425b570 msgr2=0x55bc24264690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:20.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.807+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/3071915933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425b570 0x55bc24264690 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f73a000fca0 tx=0x7f73a0009c30 comp rx=0 tx=0).stop 2024-05-10T05:39:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.811+0000 7f73cdf73b80 1 -- 172.21.15.131:0/3071915933 shutdown_connections 2024-05-10T05:39:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.811+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/3071915933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425b570 0x55bc24264690 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.811+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/3071915933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc2425e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.811+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/3071915933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425a6a0 0x55bc2426b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.811+0000 7f73cdf73b80 1 -- 172.21.15.131:0/3071915933 >> 172.21.15.131:0/3071915933 conn(0x55bc24251450 msgr2=0x55bc24272390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:20.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.815+0000 7f73cdf73b80 1 -- 172.21.15.131:0/3071915933 shutdown_connections 2024-05-10T05:39:20.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.816+0000 7f73cdf73b80 1 -- 172.21.15.131:0/3071915933 wait complete. 2024-05-10T05:39:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.817+0000 7f73cdf73b80 1 -- start start 2024-05-10T05:39:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.817+0000 7f73cdf73b80 1 Processor -- start 2024-05-10T05:39:20.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.818+0000 7f73cdf73b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc24268dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.818+0000 7f73cdf73b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425a6a0 0x55bc24269310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.818+0000 7f73cdf73b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 0x55bc2426a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.818+0000 7f73af400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc24268dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.819+0000 7f73af400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc24268dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37384/0 (socket says 172.21.15.131:37384) 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.820+0000 7f73cdf73b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2426a780 con 0x55bc2425b570 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.820+0000 7f73cdf73b80 1 -- 172.21.15.131:0/2621725238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2426a8c0 con 0x55bc241bca60 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.820+0000 7f73af400700 1 -- 172.21.15.131:0/2621725238 learned_addr learned my addr 172.21.15.131:0/2621725238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.820+0000 7f73cdf73b80 1 -- 172.21.15.131:0/2621725238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2426aa80 con 0x55bc2425a6a0 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.820+0000 7f73cdf73b80 10 client.0 _ll_register_callbacks cb 0x55bc2411fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.820+0000 7f73aea00700 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425a6a0 0x55bc24269310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.821+0000 7f73af400700 1 -- 172.21.15.131:0/2621725238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425a6a0 msgr2=0x55bc24269310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.821+0000 7f73af400700 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425a6a0 0x55bc24269310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.821+0000 7f73af400700 1 -- 172.21.15.131:0/2621725238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 msgr2=0x55bc2426a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.821+0000 7f73af400700 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 0x55bc2426a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.821+0000 7f73af400700 1 -- 172.21.15.131:0/2621725238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc23fa7b60 con 0x55bc241bca60 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.821+0000 7f73aea00700 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425a6a0 0x55bc24269310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.821+0000 7f73af400700 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc24268dd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f73a8000f80 tx=0x7f73a80040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbdc8e46aa934a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.821+0000 7f7398c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.821+0000 7f7398c00700 1 -- 172.21.15.131:0/2621725238 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a801a330 con 0x55bc241bca60 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.822+0000 7f73cdf73b80 1 -- 172.21.15.131:0/2621725238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73a000f840 con 0x55bc241bca60 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.822+0000 7f7398c00700 1 -- 172.21.15.131:0/2621725238 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f73a801a490 con 0x55bc241bca60 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.822+0000 7f7398c00700 1 -- 172.21.15.131:0/2621725238 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73a801a330 con 0x55bc241bca60 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.822+0000 7f73cdf73b80 10 client.26441 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.822+0000 7f73cdf73b80 1 -- 172.21.15.131:0/2621725238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bc23fffcd0 con 0x55bc241bca60 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.823+0000 7f7398c00700 1 -- 172.21.15.131:0/2621725238 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a000f840 con 0x55bc241bca60 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.824+0000 7f7398c00700 1 -- 172.21.15.131:0/2621725238 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f73a8007cf0 con 0x55bc241bca60 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.824+0000 7f73cdf73b80 10 client.26441 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.824+0000 7f73cdf73b80 10 client.26441 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.824+0000 7f73cdf73b80 20 client.26441 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.824+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bc2425bec0 0x55bc242d97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.824+0000 7f73cdf73b80 4 client.26441 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.824+0000 7f73cdf73b80 1 -- 172.21.15.131:0/2621725238 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc24172ec0 con 0x55bc2425bec0 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.825+0000 7f73aea00700 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bc2425bec0 0x55bc242d97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.826+0000 7f7398c00700 10 client.26441 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.826+0000 7f73aea00700 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bc2425bec0 0x55bc242d97f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d94f3f4a5e4f1c2 server_cookie=3d80358ae2765d90 in_seq=0 out_seq=0 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.827+0000 7f7398c00700 1 -- 172.21.15.131:0/2621725238 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc24172ec0 con 0x55bc2425bec0 2024-05-10T05:39:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.827+0000 7f7398c00700 10 client.26441 handle_command_reply: tid=0 2024-05-10T05:39:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.827+0000 7f73cdf73b80 1 client.26441 shutdown 2024-05-10T05:39:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.841+0000 7f73cdf73b80 1 -- 172.21.15.131:0/2621725238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 msgr2=0x55bc24268dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.841+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc24268dd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f73a8000f80 tx=0x7f73a80040f0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.841+0000 7f73cdf73b80 1 -- 172.21.15.131:0/2621725238 shutdown_connections 2024-05-10T05:39:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.841+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bc2425bec0 0x55bc242d97f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.841+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc2425b570 0x55bc2426a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.841+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc241bca60 0x55bc24268dd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.841+0000 7f73cdf73b80 1 --2- 172.21.15.131:0/2621725238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc2425a6a0 0x55bc24269310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.841+0000 7f73cdf73b80 1 -- 172.21.15.131:0/2621725238 >> 172.21.15.131:0/2621725238 conn(0x55bc24251450 msgr2=0x55bc24272390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.842+0000 7f7398c00700 0 client.26441 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:39:20.849 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:39:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.842+0000 7f73cdf73b80 1 -- 172.21.15.131:0/2621725238 shutdown_connections 2024-05-10T05:39:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.842+0000 7f73cdf73b80 1 -- 172.21.15.131:0/2621725238 wait complete. 2024-05-10T05:39:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.843+0000 7f73cdf73b80 20 client.26441 trim_cache size 0 max 16384 2024-05-10T05:39:20.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.850+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73940a1380 msgr2=0x7f73940a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:20.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.850+0000 7f73b6000700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73940a1380 0x7f73940a3890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f73a000fad0 tx=0x7f73a0009260 comp rx=0 tx=0).stop 2024-05-10T05:39:20.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.850+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b0109330 msgr2=0x7f73b019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:20.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.850+0000 7f73b6000700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b0109330 0x7f73b019ea70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f739c00d980 tx=0x7f739c00dc90 comp rx=0 tx=0).stop 2024-05-10T05:39:20.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.851+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 shutdown_connections 2024-05-10T05:39:20.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.851+0000 7f73b6000700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73940a1380 0x7f73940a3890 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.851+0000 7f73b6000700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73b0108210 0x7f73b019e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.851+0000 7f73b6000700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73b0109cd0 0x7f73b01a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.851+0000 7f73b6000700 1 --2- 172.21.15.131:0/318318880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73b0109330 0x7f73b019ea70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:20.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.851+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 >> 172.21.15.131:0/318318880 conn(0x7f73b0077240 msgr2=0x7f73b010c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:20.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.851+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 shutdown_connections 2024-05-10T05:39:20.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:20.851+0000 7f73b6000700 1 -- 172.21.15.131:0/318318880 wait complete. 2024-05-10T05:39:20.944 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:39:21.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.552+0000 7ff973a00700 1 -- 172.21.15.131:0/1517015954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c06a380 msgr2=0x7ff96c108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:21.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.552+0000 7ff973a00700 1 --2- 172.21.15.131:0/1517015954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c06a380 0x7ff96c108630 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0098f0 tx=0x7ff95c009c00 comp rx=0 tx=0).stop 2024-05-10T05:39:21.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.552+0000 7ff973a00700 1 -- 172.21.15.131:0/1517015954 shutdown_connections 2024-05-10T05:39:21.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.552+0000 7ff973a00700 1 --2- 172.21.15.131:0/1517015954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c109c60 0x7ff96c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.552+0000 7ff973a00700 1 --2- 172.21.15.131:0/1517015954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c06a380 0x7ff96c108630 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.552+0000 7ff973a00700 1 --2- 172.21.15.131:0/1517015954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff96c108c00 0x7ff96c109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.552+0000 7ff973a00700 1 -- 172.21.15.131:0/1517015954 >> 172.21.15.131:0/1517015954 conn(0x7ff96c06ee40 msgr2=0x7ff96c0712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:21.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.553+0000 7ff973a00700 1 -- 172.21.15.131:0/1517015954 shutdown_connections 2024-05-10T05:39:21.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.553+0000 7ff973a00700 1 -- 172.21.15.131:0/1517015954 wait complete. 2024-05-10T05:39:21.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.554+0000 7ff973a00700 1 Processor -- start 2024-05-10T05:39:21.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.555+0000 7ff973a00700 1 -- start start 2024-05-10T05:39:21.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.556+0000 7ff973a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff96c06a380 0x7ff96c1a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.556+0000 7ff973a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108c00 0x7ff96c1a7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.557+0000 7ff973a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c109c60 0x7ff96c1ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.557+0000 7ff973a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c1ae800 con 0x7ff96c06a380 2024-05-10T05:39:21.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.557+0000 7ff973a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c1ae970 con 0x7ff96c108c00 2024-05-10T05:39:21.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.557+0000 7ff973a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c1aeb30 con 0x7ff96c109c60 2024-05-10T05:39:21.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.557+0000 7ff971000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff96c06a380 0x7ff96c1a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.557+0000 7ff971000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff96c06a380 0x7ff96c1a6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32992/0 (socket says 172.21.15.131:32992) 2024-05-10T05:39:21.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.557+0000 7ff971000700 1 -- 172.21.15.131:0/1973857652 learned_addr learned my addr 172.21.15.131:0/1973857652 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:21.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.558+0000 7ff971a00700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c109c60 0x7ff96c1ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.558+0000 7ff971000700 1 -- 172.21.15.131:0/1973857652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c109c60 msgr2=0x7ff96c1ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:21.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.558+0000 7ff971000700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c109c60 0x7ff96c1ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.558+0000 7ff96be00700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108c00 0x7ff96c1a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.558+0000 7ff971000700 1 -- 172.21.15.131:0/1973857652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108c00 msgr2=0x7ff96c1a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:21.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.558+0000 7ff971000700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108c00 0x7ff96c1a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.558+0000 7ff971000700 1 -- 172.21.15.131:0/1973857652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9600095f0 con 0x7ff96c06a380 2024-05-10T05:39:21.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.558+0000 7ff971a00700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c109c60 0x7ff96c1ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:39:21.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.558+0000 7ff96be00700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108c00 0x7ff96c1a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:21.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.559+0000 7ff971000700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff96c06a380 0x7ff96c1a6d30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff95c000c00 tx=0x7ff95c00bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:21.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.559+0000 7ff969600700 1 -- 172.21.15.131:0/1973857652 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c01c070 con 0x7ff96c06a380 2024-05-10T05:39:21.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.559+0000 7ff969600700 1 -- 172.21.15.131:0/1973857652 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff95c005040 con 0x7ff96c06a380 2024-05-10T05:39:21.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.559+0000 7ff969600700 1 -- 172.21.15.131:0/1973857652 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c004a70 con 0x7ff96c06a380 2024-05-10T05:39:21.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.559+0000 7ff973a00700 1 -- 172.21.15.131:0/1973857652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95c009520 con 0x7ff96c06a380 2024-05-10T05:39:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.560+0000 7ff973a00700 1 -- 172.21.15.131:0/1973857652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c1aeff0 con 0x7ff96c06a380 2024-05-10T05:39:21.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.567+0000 7ff98b982b80 1 Processor -- start 2024-05-10T05:39:21.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.568+0000 7ff98b982b80 1 -- start start 2024-05-10T05:39:21.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.568+0000 7ff969600700 1 -- 172.21.15.131:0/1973857652 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff95c020070 con 0x7ff96c06a380 2024-05-10T05:39:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.568+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 0x555a49e03af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.568+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04570 0x555a49e04980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.568+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e04ec0 0x555a49e0b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.568+0000 7ff98b982b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49d2a150 con 0x555a49e036a0 2024-05-10T05:39:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.568+0000 7ff98b982b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49d27b10 con 0x555a49e04570 2024-05-10T05:39:21.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.568+0000 7ff98b982b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49d23400 con 0x555a49e04ec0 2024-05-10T05:39:21.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.569+0000 7ff971a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 0x555a49e03af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.569+0000 7ff971a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 0x555a49e03af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32998/0 (socket says 172.21.15.131:32998) 2024-05-10T05:39:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.569+0000 7ff971a00700 1 -- 172.21.15.131:0/1085485909 learned_addr learned my addr 172.21.15.131:0/1085485909 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.569+0000 7ff971000700 1 --2- 172.21.15.131:0/1085485909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04570 0x555a49e04980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.569+0000 7ff96be00700 1 --2- 172.21.15.131:0/1085485909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e04ec0 0x555a49e0b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.570+0000 7ff971a00700 1 -- 172.21.15.131:0/1085485909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e04ec0 msgr2=0x555a49e0b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.570+0000 7ff971a00700 1 --2- 172.21.15.131:0/1085485909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e04ec0 0x555a49e0b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.570+0000 7ff971a00700 1 -- 172.21.15.131:0/1085485909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04570 msgr2=0x555a49e04980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.570+0000 7ff971a00700 1 --2- 172.21.15.131:0/1085485909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04570 0x555a49e04980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.570+0000 7ff971a00700 1 -- 172.21.15.131:0/1085485909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a49ba43b0 con 0x555a49e036a0 2024-05-10T05:39:21.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.570+0000 7ff971a00700 1 --2- 172.21.15.131:0/1085485909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 0x555a49e03af0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff9640093a0 tx=0x7ff964009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947ff58de7a708ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:21.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.571+0000 7ff969600700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9540a13b0 0x7ff9540a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.571+0000 7ff95a000700 1 -- 172.21.15.131:0/1085485909 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96400e030 con 0x555a49e036a0 2024-05-10T05:39:21.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.571+0000 7ff969600700 1 -- 172.21.15.131:0/1973857652 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff95c0cd480 con 0x7ff96c06a380 2024-05-10T05:39:21.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.571+0000 7ff95a000700 1 -- 172.21.15.131:0/1085485909 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff96400f040 con 0x555a49e036a0 2024-05-10T05:39:21.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.571+0000 7ff95a000700 1 -- 172.21.15.131:0/1085485909 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964015b60 con 0x555a49e036a0 2024-05-10T05:39:21.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.572+0000 7ff971000700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9540a13b0 0x7ff9540a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.573+0000 7ff98b982b80 1 -- 172.21.15.131:0/1085485909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 msgr2=0x555a49e03af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:21.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.573+0000 7ff971000700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9540a13b0 0x7ff9540a38c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00bd50 tx=0x7ff95c009c00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:21.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.573+0000 7ff98b982b80 1 --2- 172.21.15.131:0/1085485909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 0x555a49e03af0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff9640093a0 tx=0x7ff964009920 comp rx=0 tx=0).stop 2024-05-10T05:39:21.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.574+0000 7ff98b982b80 1 -- 172.21.15.131:0/1085485909 shutdown_connections 2024-05-10T05:39:21.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.574+0000 7ff98b982b80 1 --2- 172.21.15.131:0/1085485909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 0x555a49e03af0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.574+0000 7ff98b982b80 1 --2- 172.21.15.131:0/1085485909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e04ec0 0x555a49e0b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.574+0000 7ff98b982b80 1 --2- 172.21.15.131:0/1085485909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04570 0x555a49e04980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.574+0000 7ff98b982b80 1 -- 172.21.15.131:0/1085485909 >> 172.21.15.131:0/1085485909 conn(0x555a49d65a60 msgr2=0x555a49dfdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:21.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.574+0000 7ff98b982b80 1 -- 172.21.15.131:0/1085485909 shutdown_connections 2024-05-10T05:39:21.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.574+0000 7ff98b982b80 1 -- 172.21.15.131:0/1085485909 wait complete. 2024-05-10T05:39:21.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.575+0000 7ff98b982b80 1 -- start start 2024-05-10T05:39:21.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.576+0000 7ff98b982b80 1 Processor -- start 2024-05-10T05:39:21.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.576+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 0x555a49e0d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.576+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e04570 0x555a49e0da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.576+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e149a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.576+0000 7ff98b982b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49cbf560 con 0x555a49e036a0 2024-05-10T05:39:21.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.576+0000 7ff98b982b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49cbf200 con 0x555a49e04ec0 2024-05-10T05:39:21.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.576+0000 7ff98b982b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49ba66a0 con 0x555a49e04570 2024-05-10T05:39:21.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.576+0000 7ff98b982b80 10 client.0 _ll_register_callbacks cb 0x555a49cc8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:21.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.577+0000 7ff96be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e04570 0x555a49e0da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.577+0000 7ff971a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.577+0000 7ff971a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e149a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37424/0 (socket says 172.21.15.131:37424) 2024-05-10T05:39:21.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.577+0000 7ff971a00700 1 -- 172.21.15.131:0/3070126403 learned_addr learned my addr 172.21.15.131:0/3070126403 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:21.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.577+0000 7ff971a00700 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 0x555a49e0d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.578+0000 7ff971a00700 1 -- 172.21.15.131:0/3070126403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e04570 msgr2=0x555a49e0da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:21.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.578+0000 7ff971a00700 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e04570 0x555a49e0da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.578+0000 7ff971a00700 1 -- 172.21.15.131:0/3070126403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 msgr2=0x555a49e0d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:21.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.578+0000 7ff971a00700 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 0x555a49e0d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.578+0000 7ff971a00700 1 -- 172.21.15.131:0/3070126403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff964009050 con 0x555a49e04ec0 2024-05-10T05:39:21.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.578+0000 7ff971a00700 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e149a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff964015640 tx=0x7ff9640060a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70440ccc463a1404 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:21.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.578+0000 7ff94ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:39:21.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.579+0000 7ff94ea00700 1 -- 172.21.15.131:0/3070126403 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96400e040 con 0x555a49e04ec0 2024-05-10T05:39:21.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.579+0000 7ff94ea00700 1 -- 172.21.15.131:0/3070126403 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff9640158e0 con 0x555a49e04ec0 2024-05-10T05:39:21.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.579+0000 7ff98b982b80 1 -- 172.21.15.131:0/3070126403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a49c7c270 con 0x555a49e04ec0 2024-05-10T05:39:21.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.579+0000 7ff94ea00700 1 -- 172.21.15.131:0/3070126403 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff964021420 con 0x555a49e04ec0 2024-05-10T05:39:21.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.579+0000 7ff98b982b80 10 client.26450 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:21.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.580+0000 7ff98b982b80 1 -- 172.21.15.131:0/3070126403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555a49b50b60 con 0x555a49e04ec0 2024-05-10T05:39:21.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.580+0000 7ff94ea00700 1 -- 172.21.15.131:0/3070126403 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a49c7c270 con 0x555a49e04ec0 2024-05-10T05:39:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.580+0000 7ff94ea00700 1 -- 172.21.15.131:0/3070126403 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff96401e5b0 con 0x555a49e04ec0 2024-05-10T05:39:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.580+0000 7ff98b982b80 10 client.26450 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.580+0000 7ff98b982b80 10 client.26450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:39:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.580+0000 7ff98b982b80 20 client.26450 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.580+0000 7ff98b982b80 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x555a49e19070 0x555a49e1b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.581+0000 7ff98b982b80 4 client.26450 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.581+0000 7ff98b982b80 1 -- 172.21.15.131:0/3070126403 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a49dd2780 con 0x555a49e19070 2024-05-10T05:39:21.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.581+0000 7ff96be00700 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x555a49e19070 0x555a49e1b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.582+0000 7ff96be00700 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x555a49e19070 0x555a49e1b4c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a429a572c3b6202d server_cookie=c7039f5be19b3f93 in_seq=0 out_seq=0 2024-05-10T05:39:21.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.582+0000 7ff94ea00700 10 client.26450 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:39:21.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.583+0000 7ff94ea00700 1 -- 172.21.15.131:0/3070126403 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a49dd2780 con 0x555a49e19070 2024-05-10T05:39:21.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.583+0000 7ff94ea00700 10 client.26450 handle_command_reply: tid=0 2024-05-10T05:39:21.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.584+0000 7ff98b982b80 1 client.26450 shutdown 2024-05-10T05:39:21.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.584+0000 7ff98b982b80 1 -- 172.21.15.131:0/3070126403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 msgr2=0x555a49e149a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:21.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.584+0000 7ff98b982b80 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e149a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff964015640 tx=0x7ff9640060a0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.584+0000 7ff98b982b80 1 -- 172.21.15.131:0/3070126403 shutdown_connections 2024-05-10T05:39:21.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.584+0000 7ff98b982b80 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x555a49e19070 0x555a49e1b4c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.584+0000 7ff98b982b80 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e036a0 0x555a49e0d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.584+0000 7ff98b982b80 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e149a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.584+0000 7ff98b982b80 1 --2- 172.21.15.131:0/3070126403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e04570 0x555a49e0da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.584+0000 7ff98b982b80 1 -- 172.21.15.131:0/3070126403 >> 172.21.15.131:0/3070126403 conn(0x555a49d65a60 msgr2=0x555a49dfdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:21.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.584+0000 7ff94ea00700 0 client.26450 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:39:21.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.585+0000 7ff98b982b80 1 -- 172.21.15.131:0/3070126403 shutdown_connections 2024-05-10T05:39:21.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.585+0000 7ff98b982b80 1 -- 172.21.15.131:0/3070126403 wait complete. 2024-05-10T05:39:21.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.585+0000 7ff98b982b80 20 client.26450 trim_cache size 0 max 16384 2024-05-10T05:39:21.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.616+0000 7ff98b982b80 1 Processor -- start 2024-05-10T05:39:21.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.616+0000 7ff98b982b80 1 -- start start 2024-05-10T05:39:21.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.616+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e036a0 0x555a49dfee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.616+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e04570 0x555a49e0ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.616+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.616+0000 7ff98b982b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49e18ea0 con 0x555a49e04570 2024-05-10T05:39:21.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.616+0000 7ff98b982b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49e20660 con 0x555a49e04ec0 2024-05-10T05:39:21.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.616+0000 7ff98b982b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49e08b40 con 0x555a49e036a0 2024-05-10T05:39:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.617+0000 7ff971a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e036a0 0x555a49dfee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.617+0000 7ff96be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e04570 0x555a49e0ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.618+0000 7ff96be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e04570 0x555a49e0ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33024/0 (socket says 172.21.15.131:33024) 2024-05-10T05:39:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.618+0000 7ff96be00700 1 -- 172.21.15.131:0/3809053582 learned_addr learned my addr 172.21.15.131:0/3809053582 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.619+0000 7ff96be00700 1 -- 172.21.15.131:0/3809053582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e036a0 msgr2=0x555a49dfee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.619+0000 7ff96be00700 1 --2- 172.21.15.131:0/3809053582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e036a0 0x555a49dfee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.619+0000 7ff96be00700 1 -- 172.21.15.131:0/3809053582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 msgr2=0x555a49e077a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:39:21.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.619+0000 7ff96be00700 1 --2- 172.21.15.131:0/3809053582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e077a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.623+0000 7ff96be00700 1 -- 172.21.15.131:0/3809053582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a49ce9560 con 0x555a49e04570 2024-05-10T05:39:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.623+0000 7ff971a00700 1 --2- 172.21.15.131:0/3809053582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e036a0 0x555a49dfee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.623+0000 7ff96be00700 1 --2- 172.21.15.131:0/3809053582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e04570 0x555a49e0ca70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x555a49e1bd30 tx=0x7ff9600147a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b229144c054585e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.624+0000 7ff95aa00700 1 -- 172.21.15.131:0/3809053582 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960017040 con 0x555a49e04570 2024-05-10T05:39:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.625+0000 7ff95aa00700 1 -- 172.21.15.131:0/3809053582 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff96001d8d0 con 0x555a49e04570 2024-05-10T05:39:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.625+0000 7ff95aa00700 1 -- 172.21.15.131:0/3809053582 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9600038e0 con 0x555a49e04570 2024-05-10T05:39:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.626+0000 7ff98b982b80 1 -- 172.21.15.131:0/3809053582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e04570 msgr2=0x555a49e0ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.626+0000 7ff98b982b80 1 --2- 172.21.15.131:0/3809053582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e04570 0x555a49e0ca70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x555a49e1bd30 tx=0x7ff9600147a0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.627+0000 7ff98b982b80 1 -- 172.21.15.131:0/3809053582 shutdown_connections 2024-05-10T05:39:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.627+0000 7ff98b982b80 1 --2- 172.21.15.131:0/3809053582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.627+0000 7ff98b982b80 1 --2- 172.21.15.131:0/3809053582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e036a0 0x555a49dfee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.627+0000 7ff98b982b80 1 --2- 172.21.15.131:0/3809053582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e04570 0x555a49e0ca70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.627+0000 7ff98b982b80 1 -- 172.21.15.131:0/3809053582 >> 172.21.15.131:0/3809053582 conn(0x555a49d65a60 msgr2=0x555a49dfdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:21.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.628+0000 7ff98b982b80 1 -- 172.21.15.131:0/3809053582 shutdown_connections 2024-05-10T05:39:21.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.628+0000 7ff98b982b80 1 -- 172.21.15.131:0/3809053582 wait complete. 2024-05-10T05:39:21.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.629+0000 7ff98b982b80 1 -- start start 2024-05-10T05:39:21.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.630+0000 7ff98b982b80 1 Processor -- start 2024-05-10T05:39:21.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.635+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e036a0 0x555a49e12da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.635+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e132e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.635+0000 7ff98b982b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e14210 0x555a49e14620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.635+0000 7ff98b982b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49e14b60 con 0x555a49e14210 2024-05-10T05:39:21.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.635+0000 7ff98b982b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49e14cd0 con 0x555a49e04ec0 2024-05-10T05:39:21.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.635+0000 7ff98b982b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a49e14e90 con 0x555a49e036a0 2024-05-10T05:39:21.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.635+0000 7ff98b982b80 10 client.0 _ll_register_callbacks cb 0x555a49cc8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:21.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.636+0000 7ff96be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.636+0000 7ff96be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e132e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37448/0 (socket says 172.21.15.131:37448) 2024-05-10T05:39:21.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.636+0000 7ff96be00700 1 -- 172.21.15.131:0/1757585338 learned_addr learned my addr 172.21.15.131:0/1757585338 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:21.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.637+0000 7ff971a00700 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e14210 0x555a49e14620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.637+0000 7ff96be00700 1 -- 172.21.15.131:0/1757585338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e036a0 msgr2=0x555a49e12da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:39:21.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.637+0000 7ff96be00700 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e036a0 0x555a49e12da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.637+0000 7ff96be00700 1 -- 172.21.15.131:0/1757585338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e14210 msgr2=0x555a49e14620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:21.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.637+0000 7ff96be00700 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e14210 0x555a49e14620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.637+0000 7ff96be00700 1 -- 172.21.15.131:0/1757585338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a49ba8cd0 con 0x555a49e04ec0 2024-05-10T05:39:21.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.638+0000 7ff971a00700 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e14210 0x555a49e14620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:21.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.638+0000 7ff96be00700 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e132e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff960014040 tx=0x7ff960005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c468560a2a4443b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:21.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.639+0000 7ff94ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:39:21.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.639+0000 7ff94ea00700 1 -- 172.21.15.131:0/1757585338 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96002b070 con 0x555a49e04ec0 2024-05-10T05:39:21.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.639+0000 7ff98b982b80 1 -- 172.21.15.131:0/1757585338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff96400f040 con 0x555a49e04ec0 2024-05-10T05:39:21.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.640+0000 7ff94ea00700 1 -- 172.21.15.131:0/1757585338 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff9600140d0 con 0x555a49e04ec0 2024-05-10T05:39:21.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.640+0000 7ff94ea00700 1 -- 172.21.15.131:0/1757585338 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff960017040 con 0x555a49e04ec0 2024-05-10T05:39:21.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.640+0000 7ff98b982b80 10 client.26453 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:21.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.640+0000 7ff98b982b80 1 -- 172.21.15.131:0/1757585338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555a49b4f030 con 0x555a49e04ec0 2024-05-10T05:39:21.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.644+0000 7ff94ea00700 1 -- 172.21.15.131:0/1757585338 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff96400f040 con 0x555a49e04ec0 2024-05-10T05:39:21.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.644+0000 7ff94ea00700 1 -- 172.21.15.131:0/1757585338 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff960026470 con 0x555a49e04ec0 2024-05-10T05:39:21.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.644+0000 7ff98b982b80 10 client.26453 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:21.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.645+0000 7ff98b982b80 10 client.26453 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:39:21.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.645+0000 7ff98b982b80 20 client.26453 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:21.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.645+0000 7ff98b982b80 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x555a49e813d0 0x555a49e837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:21.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.645+0000 7ff98b982b80 4 client.26453 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:39:21.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.645+0000 7ff98b982b80 1 -- 172.21.15.131:0/1757585338 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555a49d1bec0 con 0x555a49e813d0 2024-05-10T05:39:21.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.646+0000 7ff971a00700 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x555a49e813d0 0x555a49e837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:21.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.647+0000 7ff971a00700 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x555a49e813d0 0x555a49e837f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f1711f30adc957f server_cookie=1582ba419d19b326 in_seq=0 out_seq=0 2024-05-10T05:39:21.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.647+0000 7ff94ea00700 10 client.26453 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:39:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.648+0000 7ff94ea00700 1 -- 172.21.15.131:0/1757585338 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555a49d1bec0 con 0x555a49e813d0 2024-05-10T05:39:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.648+0000 7ff94ea00700 10 client.26453 handle_command_reply: tid=0 2024-05-10T05:39:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.648+0000 7ff98b982b80 1 client.26453 shutdown 2024-05-10T05:39:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.649+0000 7ff98b982b80 1 -- 172.21.15.131:0/1757585338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 msgr2=0x555a49e132e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.649+0000 7ff98b982b80 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e132e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff960014040 tx=0x7ff960005830 comp rx=0 tx=0).stop 2024-05-10T05:39:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.649+0000 7ff98b982b80 1 -- 172.21.15.131:0/1757585338 shutdown_connections 2024-05-10T05:39:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.649+0000 7ff98b982b80 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x555a49e813d0 0x555a49e837f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.650+0000 7ff98b982b80 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a49e14210 0x555a49e14620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.650+0000 7ff98b982b80 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555a49e036a0 0x555a49e12da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.650+0000 7ff98b982b80 1 --2- 172.21.15.131:0/1757585338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555a49e04ec0 0x555a49e132e0 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff960014040 tx=0x7ff960005830 comp rx=0 tx=0).stop 2024-05-10T05:39:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.650+0000 7ff94ea00700 0 client.26453 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:39:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.650+0000 7ff98b982b80 1 -- 172.21.15.131:0/1757585338 >> 172.21.15.131:0/1757585338 conn(0x555a49d65a60 msgr2=0x555a49e0b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.651+0000 7ff98b982b80 1 -- 172.21.15.131:0/1757585338 shutdown_connections 2024-05-10T05:39:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.656+0000 7ff98b982b80 1 -- 172.21.15.131:0/1757585338 wait complete. 2024-05-10T05:39:21.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.656+0000 7ff98b982b80 20 client.26453 trim_cache size 0 max 16384 2024-05-10T05:39:21.657 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:39:21.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.659+0000 7ff973a00700 1 -- 172.21.15.131:0/1973857652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9540a13b0 msgr2=0x7ff9540a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:21.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.659+0000 7ff973a00700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9540a13b0 0x7ff9540a38c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00bd50 tx=0x7ff95c009c00 comp rx=0 tx=0).stop 2024-05-10T05:39:21.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.659+0000 7ff973a00700 1 -- 172.21.15.131:0/1973857652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff96c06a380 msgr2=0x7ff96c1a6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:21.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.659+0000 7ff973a00700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff96c06a380 0x7ff96c1a6d30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff95c000c00 tx=0x7ff95c00bfa0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.659+0000 7ff973a00700 1 -- 172.21.15.131:0/1973857652 shutdown_connections 2024-05-10T05:39:21.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.659+0000 7ff973a00700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff9540a13b0 0x7ff9540a38c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.659+0000 7ff973a00700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff96c06a380 0x7ff96c1a6d30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.659+0000 7ff973a00700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff96c109c60 0x7ff96c1ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.659+0000 7ff973a00700 1 --2- 172.21.15.131:0/1973857652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108c00 0x7ff96c1a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:21.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.660+0000 7ff973a00700 1 -- 172.21.15.131:0/1973857652 >> 172.21.15.131:0/1973857652 conn(0x7ff96c06ee40 msgr2=0x7ff96c10ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:21.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.669+0000 7ff973a00700 1 -- 172.21.15.131:0/1973857652 shutdown_connections 2024-05-10T05:39:21.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:21.669+0000 7ff973a00700 1 -- 172.21.15.131:0/1973857652 wait complete. 2024-05-10T05:39:21.873 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:39:22.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:22 smithi157 ceph-mon[78427]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 78 op/s 2024-05-10T05:39:22.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.604+0000 7f6dbf400700 1 -- 172.21.15.131:0/2923257470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac017400 con 0x7f6db80aaa60 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.605+0000 7f6dc6000700 1 -- 172.21.15.131:0/2923257470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db80aaa60 msgr2=0x7f6db80ace80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.605+0000 7f6dc6000700 1 --2- 172.21.15.131:0/2923257470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db80aaa60 0x7f6db80ace80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009940 tx=0x7f6dac009c50 comp rx=0 tx=0).stop 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.606+0000 7f6dc6000700 1 -- 172.21.15.131:0/2923257470 shutdown_connections 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.606+0000 7f6dc6000700 1 --2- 172.21.15.131:0/2923257470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80ad3c0 0x7f6db80af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.606+0000 7f6dc6000700 1 --2- 172.21.15.131:0/2923257470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80a8100 0x7f6db80aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.606+0000 7f6dc6000700 1 --2- 172.21.15.131:0/2923257470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db80aaa60 0x7f6db80ace80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.606+0000 7f6dc6000700 1 -- 172.21.15.131:0/2923257470 >> 172.21.15.131:0/2923257470 conn(0x7f6db80a1b20 msgr2=0x7f6db80a3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.606+0000 7f6dc6000700 1 -- 172.21.15.131:0/2923257470 shutdown_connections 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.606+0000 7f6dc6000700 1 -- 172.21.15.131:0/2923257470 wait complete. 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.608+0000 7f6dc6000700 1 Processor -- start 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.608+0000 7f6dc6000700 1 -- start start 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.611+0000 7f6dc6000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80a8100 0x7f6db813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.613+0000 7f6dc6000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80aaa60 0x7f6db813ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.613+0000 7f6dc6000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db80ad3c0 0x7f6db8146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.613+0000 7f6dc6000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db8147510 con 0x7f6db80ad3c0 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.613+0000 7f6dc6000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db8147650 con 0x7f6db80a8100 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.613+0000 7f6dc6000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81477e0 con 0x7f6db80aaa60 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.614+0000 7f6dc4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80a8100 0x7f6db813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.614+0000 7f6dc4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80a8100 0x7f6db813fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37466/0 (socket says 172.21.15.131:37466) 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.614+0000 7f6dc4c00700 1 -- 172.21.15.131:0/4128181820 learned_addr learned my addr 172.21.15.131:0/4128181820 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.614+0000 7f6dbfe00700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80aaa60 0x7f6db813ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.614+0000 7f6dc4c00700 1 -- 172.21.15.131:0/4128181820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80aaa60 msgr2=0x7f6db813ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.614+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80aaa60 0x7f6db813ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.614+0000 7f6dc4c00700 1 -- 172.21.15.131:0/4128181820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db80ad3c0 msgr2=0x7f6db8146f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.614+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db80ad3c0 0x7f6db8146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.614+0000 7f6dc4c00700 1 -- 172.21.15.131:0/4128181820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac009520 con 0x7f6db80a8100 2024-05-10T05:39:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.615+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80a8100 0x7f6db813fa70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e9f0 tx=0x7f6db400edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.615+0000 7f6dbd600700 1 -- 172.21.15.131:0/4128181820 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4009690 con 0x7f6db80a8100 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.615+0000 7f6dbd600700 1 -- 172.21.15.131:0/4128181820 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db400cbd0 con 0x7f6db80a8100 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.615+0000 7f6dbd600700 1 -- 172.21.15.131:0/4128181820 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4018680 con 0x7f6db80a8100 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.615+0000 7f6dc6000700 1 -- 172.21.15.131:0/4128181820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db8147ac0 con 0x7f6db80a8100 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.616+0000 7f6dc6000700 1 -- 172.21.15.131:0/4128181820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db8147f80 con 0x7f6db80a8100 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.624+0000 7f6ddf610b80 1 Processor -- start 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.626+0000 7f6ddf610b80 1 -- start start 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.626+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cbf2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.626+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cbf3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.626+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf40f0 0x556f2cbfa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.626+0000 7f6ddf610b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2cb19150 con 0x556f2cbf2530 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.626+0000 7f6ddf610b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2cb16b10 con 0x556f2cbf40f0 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.626+0000 7f6ddf610b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2cb12400 con 0x556f2cbf37a0 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.628+0000 7f6dc5600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cbf2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.628+0000 7f6dc5600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cbf2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33060/0 (socket says 172.21.15.131:33060) 2024-05-10T05:39:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.628+0000 7f6dc5600700 1 -- 172.21.15.131:0/4139414897 learned_addr learned my addr 172.21.15.131:0/4139414897 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.628+0000 7f6dbd600700 1 -- 172.21.15.131:0/4128181820 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6db400cd40 con 0x7f6db80a8100 2024-05-10T05:39:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.629+0000 7f6dc5600700 1 -- 172.21.15.131:0/4139414897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 msgr2=0x556f2cbf3bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:39:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.629+0000 7f6dc5600700 1 --2- 172.21.15.131:0/4139414897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cbf3bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.629+0000 7f6dc5600700 1 -- 172.21.15.131:0/4139414897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf40f0 msgr2=0x556f2cbfa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.629+0000 7f6dc5600700 1 --2- 172.21.15.131:0/4139414897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf40f0 0x556f2cbfa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.629+0000 7f6dc5600700 1 -- 172.21.15.131:0/4139414897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f2c9933b0 con 0x556f2cbf2530 2024-05-10T05:39:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.632+0000 7f6dc5600700 1 --2- 172.21.15.131:0/4139414897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cbf2960 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0067600 tx=0x7f6dc0068ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ed3868627337bc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.634+0000 7f6db1c00700 1 -- 172.21.15.131:0/4139414897 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0072ab0 con 0x556f2cbf2530 2024-05-10T05:39:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.634+0000 7f6db1c00700 1 -- 172.21.15.131:0/4139414897 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc0072c10 con 0x556f2cbf2530 2024-05-10T05:39:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.634+0000 7f6db1c00700 1 -- 172.21.15.131:0/4139414897 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc007c660 con 0x556f2cbf2530 2024-05-10T05:39:22.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:22 smithi142 ceph-mon[77667]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 78 op/s 2024-05-10T05:39:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.639+0000 7f6ddf610b80 1 -- 172.21.15.131:0/4139414897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 msgr2=0x556f2cbf2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.639+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/4139414897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cbf2960 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0067600 tx=0x7f6dc0068ee0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.640+0000 7f6dbd600700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6da80a12f0 0x7f6da80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.641+0000 7f6dbd600700 1 -- 172.21.15.131:0/4128181820 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6db4014070 con 0x7f6db80a8100 2024-05-10T05:39:22.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.641+0000 7f6dc5600700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6da80a12f0 0x7f6da80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.642+0000 7f6dc5600700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6da80a12f0 0x7f6da80a3800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0064180 tx=0x7f6dc0074f00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.646+0000 7f6ddf610b80 1 -- 172.21.15.131:0/4139414897 shutdown_connections 2024-05-10T05:39:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.646+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/4139414897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cbf2960 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.646+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/4139414897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf40f0 0x556f2cbfa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.646+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/4139414897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cbf3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.646+0000 7f6ddf610b80 1 -- 172.21.15.131:0/4139414897 >> 172.21.15.131:0/4139414897 conn(0x556f2cb54a60 msgr2=0x556f2cbed330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.646+0000 7f6ddf610b80 1 -- 172.21.15.131:0/4139414897 shutdown_connections 2024-05-10T05:39:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.646+0000 7f6ddf610b80 1 -- 172.21.15.131:0/4139414897 wait complete. 2024-05-10T05:39:22.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.648+0000 7f6ddf610b80 1 -- start start 2024-05-10T05:39:22.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.649+0000 7f6ddf610b80 1 Processor -- start 2024-05-10T05:39:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.656+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cc05170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.656+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf37a0 0x556f2cc056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.656+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf40f0 0x556f2cc0c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.656+0000 7f6ddf610b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2caae560 con 0x556f2cbf2530 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.656+0000 7f6ddf610b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2caae200 con 0x556f2cbf37a0 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.656+0000 7f6ddf610b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2c9956a0 con 0x556f2cbf40f0 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.657+0000 7f6ddf610b80 10 client.0 _ll_register_callbacks cb 0x556f2cab7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.657+0000 7f6dbfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf37a0 0x556f2cc056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.657+0000 7f6dbfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf37a0 0x556f2cc056b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37480/0 (socket says 172.21.15.131:37480) 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.657+0000 7f6dbfe00700 1 -- 172.21.15.131:0/1363537734 learned_addr learned my addr 172.21.15.131:0/1363537734 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.657+0000 7f6dc5600700 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf40f0 0x556f2cc0c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.658+0000 7f6dc5600700 1 -- 172.21.15.131:0/1363537734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf37a0 msgr2=0x556f2cc056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.658+0000 7f6dc5600700 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf37a0 0x556f2cc056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.658+0000 7f6dc5600700 1 -- 172.21.15.131:0/1363537734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 msgr2=0x556f2cc05170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.658+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cc05170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.659+0000 7f6dc5600700 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cc05170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.659+0000 7f6dc5600700 1 -- 172.21.15.131:0/1363537734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc00688d0 con 0x556f2cbf40f0 2024-05-10T05:39:22.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.659+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cc05170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:22.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.659+0000 7f6dc5600700 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf40f0 0x556f2cc0c610 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x556f2cbf22d0 tx=0x7f6dc0068ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fa9e47bd7094b0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:22.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.660+0000 7f6da6000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:39:22.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.660+0000 7f6da6000700 1 -- 172.21.15.131:0/1363537734 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0074430 con 0x556f2cbf40f0 2024-05-10T05:39:22.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.660+0000 7f6da6000700 1 -- 172.21.15.131:0/1363537734 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6dc0074590 con 0x556f2cbf40f0 2024-05-10T05:39:22.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.661+0000 7f6ddf610b80 1 -- 172.21.15.131:0/1363537734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f2ca6b270 con 0x556f2cbf40f0 2024-05-10T05:39:22.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.661+0000 7f6da6000700 1 -- 172.21.15.131:0/1363537734 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc0074750 con 0x556f2cbf40f0 2024-05-10T05:39:22.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.661+0000 7f6ddf610b80 10 client.26485 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:22.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.662+0000 7f6ddf610b80 1 -- 172.21.15.131:0/1363537734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556f2c93fb60 con 0x556f2cbf40f0 2024-05-10T05:39:22.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.662+0000 7f6da6000700 1 -- 172.21.15.131:0/1363537734 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f2ca6b270 con 0x556f2cbf40f0 2024-05-10T05:39:22.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.664+0000 7f6da6000700 1 -- 172.21.15.131:0/1363537734 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6dc00bd070 con 0x556f2cbf40f0 2024-05-10T05:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.664+0000 7f6ddf610b80 10 client.26485 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.664+0000 7f6ddf610b80 10 client.26485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.664+0000 7f6ddf610b80 20 client.26485 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.664+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556f2cc00670 0x556f2cc02a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.664+0000 7f6ddf610b80 4 client.26485 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.664+0000 7f6ddf610b80 1 -- 172.21.15.131:0/1363537734 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f2cbc1780 con 0x556f2cc00670 2024-05-10T05:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.665+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556f2cc00670 0x556f2cc02a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.665+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556f2cc00670 0x556f2cc02a90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b816e0af6b998b40 server_cookie=a6db58ad79cb22ff in_seq=0 out_seq=0 2024-05-10T05:39:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.665+0000 7f6da6000700 10 client.26485 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.666+0000 7f6da6000700 1 -- 172.21.15.131:0/1363537734 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556f2cbc1780 con 0x556f2cc00670 2024-05-10T05:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.666+0000 7f6da6000700 10 client.26485 handle_command_reply: tid=0 2024-05-10T05:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.666+0000 7f6ddf610b80 1 client.26485 shutdown 2024-05-10T05:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.667+0000 7f6ddf610b80 1 -- 172.21.15.131:0/1363537734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf40f0 msgr2=0x556f2cc0c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.667+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf40f0 0x556f2cc0c610 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x556f2cbf22d0 tx=0x7f6dc0068ee0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.667+0000 7f6ddf610b80 1 -- 172.21.15.131:0/1363537734 shutdown_connections 2024-05-10T05:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.667+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556f2cc00670 0x556f2cc02a90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.667+0000 7f6da6000700 0 client.26485 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.667+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cc05170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.667+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf40f0 0x556f2cc0c610 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.667+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/1363537734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf37a0 0x556f2cc056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.667+0000 7f6ddf610b80 1 -- 172.21.15.131:0/1363537734 >> 172.21.15.131:0/1363537734 conn(0x556f2cb54a60 msgr2=0x556f2cbf1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.668+0000 7f6ddf610b80 1 -- 172.21.15.131:0/1363537734 shutdown_connections 2024-05-10T05:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.668+0000 7f6ddf610b80 1 -- 172.21.15.131:0/1363537734 wait complete. 2024-05-10T05:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.669+0000 7f6ddf610b80 20 client.26485 trim_cache size 0 max 16384 2024-05-10T05:39:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.701+0000 7f6ddf610b80 1 Processor -- start 2024-05-10T05:39:22.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.701+0000 7f6ddf610b80 1 -- start start 2024-05-10T05:39:22.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.702+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf2530 0x556f2cbfc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.703+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cbfbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.703+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf40f0 0x556f2cc0a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.703+0000 7f6ddf610b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2cbeeba0 con 0x556f2cbf40f0 2024-05-10T05:39:22.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.703+0000 7f6ddf610b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2cbe72f0 con 0x556f2cbf2530 2024-05-10T05:39:22.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.703+0000 7f6ddf610b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2cbfaa20 con 0x556f2cbf37a0 2024-05-10T05:39:22.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.704+0000 7f6dc5600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf40f0 0x556f2cc0a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.704+0000 7f6dc4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf2530 0x556f2cbfc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.704+0000 7f6dc4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf2530 0x556f2cbfc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37482/0 (socket says 172.21.15.131:37482) 2024-05-10T05:39:22.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.704+0000 7f6dc4c00700 1 -- 172.21.15.131:0/2819527644 learned_addr learned my addr 172.21.15.131:0/2819527644 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:22.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.704+0000 7f6dbfe00700 1 --2- 172.21.15.131:0/2819527644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cbfbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.705+0000 7f6dc5600700 1 -- 172.21.15.131:0/2819527644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 msgr2=0x556f2cbfbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:22.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.705+0000 7f6dc5600700 1 --2- 172.21.15.131:0/2819527644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cbfbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.705+0000 7f6dc5600700 1 -- 172.21.15.131:0/2819527644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf2530 msgr2=0x556f2cbfc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:22.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.705+0000 7f6dc5600700 1 --2- 172.21.15.131:0/2819527644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf2530 0x556f2cbfc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.705+0000 7f6dc5600700 1 -- 172.21.15.131:0/2819527644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f2cad8560 con 0x556f2cbf40f0 2024-05-10T05:39:22.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.706+0000 7f6dc5600700 1 --2- 172.21.15.131:0/2819527644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf40f0 0x556f2cc0a5c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0067600 tx=0x7f6dc0066ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6393bf5f63cff37 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:22.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.706+0000 7f6db2600700 1 -- 172.21.15.131:0/2819527644 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0074750 con 0x556f2cbf40f0 2024-05-10T05:39:22.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.707+0000 7f6db2600700 1 -- 172.21.15.131:0/2819527644 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc00af070 con 0x556f2cbf40f0 2024-05-10T05:39:22.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.707+0000 7f6db2600700 1 -- 172.21.15.131:0/2819527644 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00748b0 con 0x556f2cbf40f0 2024-05-10T05:39:22.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.708+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2819527644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf40f0 msgr2=0x556f2cc0a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:22.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.708+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/2819527644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf40f0 0x556f2cc0a5c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0067600 tx=0x7f6dc0066ca0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.708+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2819527644 shutdown_connections 2024-05-10T05:39:22.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.708+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/2819527644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf40f0 0x556f2cc0a5c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.708+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/2819527644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf2530 0x556f2cbfc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.708+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/2819527644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cbfbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.708+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2819527644 >> 172.21.15.131:0/2819527644 conn(0x556f2cb54a60 msgr2=0x556f2cbf7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:22.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.709+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2819527644 shutdown_connections 2024-05-10T05:39:22.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.709+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2819527644 wait complete. 2024-05-10T05:39:22.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:22 smithi131 ceph-mon[37264]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 78 op/s 2024-05-10T05:39:22.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.710+0000 7f6ddf610b80 1 -- start start 2024-05-10T05:39:22.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.710+0000 7f6ddf610b80 1 Processor -- start 2024-05-10T05:39:22.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.711+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cc08560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.711+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cc08aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.711+0000 7f6ddf610b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf40f0 0x556f2cc099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.713+0000 7f6ddf610b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2cc09f10 con 0x556f2cbf2530 2024-05-10T05:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.712+0000 7f6dbfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cc08aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.712+0000 7f6dc5600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf40f0 0x556f2cc099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.713+0000 7f6dc5600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf40f0 0x556f2cc099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:37488/0 (socket says 172.21.15.131:37488) 2024-05-10T05:39:22.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.713+0000 7f6dc5600700 1 -- 172.21.15.131:0/2848357298 learned_addr learned my addr 172.21.15.131:0/2848357298 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:22.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.713+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cc08560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.713+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2848357298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2cc0a080 con 0x556f2cbf40f0 2024-05-10T05:39:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.713+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2848357298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2cc0a240 con 0x556f2cbf37a0 2024-05-10T05:39:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.713+0000 7f6ddf610b80 10 client.0 _ll_register_callbacks cb 0x556f2cab7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.714+0000 7f6dc4c00700 1 -- 172.21.15.131:0/2848357298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 msgr2=0x556f2cc08aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.714+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cc08aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.714+0000 7f6dc4c00700 1 -- 172.21.15.131:0/2848357298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf40f0 msgr2=0x556f2cc099d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.714+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf40f0 0x556f2cc099d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.714+0000 7f6dc4c00700 1 -- 172.21.15.131:0/2848357298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f2c93fb60 con 0x556f2cbf2530 2024-05-10T05:39:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.715+0000 7f6dc4c00700 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cc08560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6db4099c30 tx=0x7f6db409a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14863ffd218d4cd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.715+0000 7f6da6000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:39:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.720+0000 7f6da6000700 1 -- 172.21.15.131:0/2848357298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db40a23b0 con 0x556f2cbf2530 2024-05-10T05:39:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.720+0000 7f6da6000700 1 -- 172.21.15.131:0/2848357298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6db40ca6f0 con 0x556f2cbf2530 2024-05-10T05:39:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.721+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2848357298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dc0072c10 con 0x556f2cbf2530 2024-05-10T05:39:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.721+0000 7f6da6000700 1 -- 172.21.15.131:0/2848357298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6db4099420 con 0x556f2cbf2530 2024-05-10T05:39:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.723+0000 7f6ddf610b80 10 client.16278 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.723+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2848357298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556f2c997cd0 con 0x556f2cbf2530 2024-05-10T05:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.724+0000 7f6da6000700 1 -- 172.21.15.131:0/2848357298 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db40b6070 con 0x556f2cbf2530 2024-05-10T05:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.724+0000 7f6da6000700 1 -- 172.21.15.131:0/2848357298 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6db4095d20 con 0x556f2cbf2530 2024-05-10T05:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.724+0000 7f6ddf610b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.724+0000 7f6ddf610b80 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.724+0000 7f6ddf610b80 20 client.16278 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.724+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556f2cc5edc0 0x556f2cc611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.724+0000 7f6ddf610b80 4 client.16278 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.724+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2848357298 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556f2cb0aec0 con 0x556f2cc5edc0 2024-05-10T05:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.725+0000 7f6dbfe00700 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556f2cc5edc0 0x556f2cc611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:22.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.725+0000 7f6da6000700 10 client.16278 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:22.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.726+0000 7f6dbfe00700 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556f2cc5edc0 0x556f2cc611e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d843b9732df1ad server_cookie=447be1e3b068031b in_seq=0 out_seq=0 2024-05-10T05:39:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6da6000700 1 -- 172.21.15.131:0/2848357298 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556f2cb0aec0 con 0x556f2cc5edc0 2024-05-10T05:39:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6da6000700 10 client.16278 handle_command_reply: tid=0 2024-05-10T05:39:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6ddf610b80 1 client.16278 shutdown 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2848357298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 msgr2=0x556f2cc08560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cc08560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6db4099c30 tx=0x7f6db409a050 comp rx=0 tx=0).stop 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2848357298 shutdown_connections 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556f2cc5edc0 0x556f2cc611e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f2cbf2530 0x556f2cc08560 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f2cbf40f0 0x556f2cc099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6da6000700 0 client.16278 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6ddf610b80 1 --2- 172.21.15.131:0/2848357298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f2cbf37a0 0x556f2cc08aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.729+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2848357298 >> 172.21.15.131:0/2848357298 conn(0x556f2cb54a60 msgr2=0x556f2cbfabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.730+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2848357298 shutdown_connections 2024-05-10T05:39:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.730+0000 7f6ddf610b80 1 -- 172.21.15.131:0/2848357298 wait complete. 2024-05-10T05:39:22.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.730+0000 7f6ddf610b80 20 client.16278 trim_cache size 0 max 16384 2024-05-10T05:39:22.732 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:39:22.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.731+0000 7f6dc6000700 1 -- 172.21.15.131:0/4128181820 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6da80a12f0 msgr2=0x7f6da80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:22.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.731+0000 7f6dc6000700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6da80a12f0 0x7f6da80a3800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0064180 tx=0x7f6dc0074f00 comp rx=0 tx=0).stop 2024-05-10T05:39:22.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.731+0000 7f6dc6000700 1 -- 172.21.15.131:0/4128181820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80a8100 msgr2=0x7f6db813fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:22.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.731+0000 7f6dc6000700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80a8100 0x7f6db813fa70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e9f0 tx=0x7f6db400edb0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.733+0000 7f6dc6000700 1 -- 172.21.15.131:0/4128181820 shutdown_connections 2024-05-10T05:39:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.733+0000 7f6dc6000700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f6da80a12f0 0x7f6da80a3800 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.733+0000 7f6dc6000700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db80ad3c0 0x7f6db8146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.733+0000 7f6dc6000700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db80a8100 0x7f6db813fa70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.733+0000 7f6dc6000700 1 --2- 172.21.15.131:0/4128181820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80aaa60 0x7f6db813ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.733+0000 7f6dc6000700 1 -- 172.21.15.131:0/4128181820 >> 172.21.15.131:0/4128181820 conn(0x7f6db80a1b20 msgr2=0x7f6db80abf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.733+0000 7f6dc6000700 1 -- 172.21.15.131:0/4128181820 shutdown_connections 2024-05-10T05:39:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:22.734+0000 7f6dc6000700 1 -- 172.21.15.131:0/4128181820 wait complete. 2024-05-10T05:39:23.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:23 smithi131 ceph-mon[37264]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 202 MiB/s wr, 62 op/s 2024-05-10T05:39:23.788 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:39:23.788 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:39:23.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:23 smithi157 ceph-mon[78427]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 202 MiB/s wr, 62 op/s 2024-05-10T05:39:23.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:23 smithi142 ceph-mon[77667]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 202 MiB/s wr, 62 op/s 2024-05-10T05:39:24.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.484+0000 7fb5d0600700 1 -- 172.21.15.131:0/953953964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8109330 msgr2=0x7fb5c8109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:24.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.484+0000 7fb5d0600700 1 --2- 172.21.15.131:0/953953964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8109330 0x7fb5c8109740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40099c0 tx=0x7fb5b4009cd0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.486+0000 7fb5d0600700 1 -- 172.21.15.131:0/953953964 shutdown_connections 2024-05-10T05:39:24.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.486+0000 7fb5d0600700 1 --2- 172.21.15.131:0/953953964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c8108210 0x7fb5c8108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.486+0000 7fb5d0600700 1 --2- 172.21.15.131:0/953953964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c8109cd0 0x7fb5c8110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.486+0000 7fb5d0600700 1 --2- 172.21.15.131:0/953953964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8109330 0x7fb5c8109740 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.487+0000 7fb5d0600700 1 -- 172.21.15.131:0/953953964 >> 172.21.15.131:0/953953964 conn(0x7fb5c8077240 msgr2=0x7fb5c8106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:24.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.488+0000 7fb5d0600700 1 -- 172.21.15.131:0/953953964 shutdown_connections 2024-05-10T05:39:24.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.488+0000 7fb5d0600700 1 -- 172.21.15.131:0/953953964 wait complete. 2024-05-10T05:39:24.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.490+0000 7fb5d0600700 1 Processor -- start 2024-05-10T05:39:24.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.490+0000 7fb5d0600700 1 -- start start 2024-05-10T05:39:24.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.491+0000 7fb5d0600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c8108210 0x7fb5c8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.491+0000 7fb5d0600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c8109330 0x7fb5c8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.491+0000 7fb5d0600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8109cd0 0x7fb5c8071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.491+0000 7fb5d0600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8071770 con 0x7fb5c8108210 2024-05-10T05:39:24.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.491+0000 7fb5d0600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80718e0 con 0x7fb5c8109330 2024-05-10T05:39:24.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.491+0000 7fb5d0600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8071aa0 con 0x7fb5c8109cd0 2024-05-10T05:39:24.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.492+0000 7fb5cd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c8109330 0x7fb5c8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.492+0000 7fb5cd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c8109330 0x7fb5c8072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56516/0 (socket says 172.21.15.131:56516) 2024-05-10T05:39:24.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.492+0000 7fb5cd200700 1 -- 172.21.15.131:0/1656560108 learned_addr learned my addr 172.21.15.131:0/1656560108 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:24.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.492+0000 7fb5ce600700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8109cd0 0x7fb5c8071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.493+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c8108210 0x7fb5c8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.493+0000 7fb5cd200700 1 -- 172.21.15.131:0/1656560108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8109cd0 msgr2=0x7fb5c8071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:24.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.493+0000 7fb5cd200700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8109cd0 0x7fb5c8071070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.493+0000 7fb5cd200700 1 -- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c8108210 msgr2=0x7fb5c8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:24.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.493+0000 7fb5cd200700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c8108210 0x7fb5c8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.493+0000 7fb5cd200700 1 -- 172.21.15.131:0/1656560108 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b40095f0 con 0x7fb5c8109330 2024-05-10T05:39:24.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.493+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c8108210 0x7fb5c8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:24.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.493+0000 7fb5cd200700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c8109330 0x7fb5c8072e40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:24.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.494+0000 7fb5c6000700 1 -- 172.21.15.131:0/1656560108 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b401d070 con 0x7fb5c8109330 2024-05-10T05:39:24.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.494+0000 7fb5c6000700 1 -- 172.21.15.131:0/1656560108 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5b40040f0 con 0x7fb5c8109330 2024-05-10T05:39:24.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.494+0000 7fb5c6000700 1 -- 172.21.15.131:0/1656560108 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4022e40 con 0x7fb5c8109330 2024-05-10T05:39:24.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.494+0000 7fb5d0600700 1 -- 172.21.15.131:0/1656560108 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c8071d20 con 0x7fb5c8109330 2024-05-10T05:39:24.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.494+0000 7fb5d0600700 1 -- 172.21.15.131:0/1656560108 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c81af210 con 0x7fb5c8109330 2024-05-10T05:39:24.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.500+0000 7fb5c6000700 1 -- 172.21.15.131:0/1656560108 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb5b400f460 con 0x7fb5c8109330 2024-05-10T05:39:24.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.504+0000 7fb5c6000700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb5b00a13b0 0x7fb5b00a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.505+0000 7fb5e845eb80 1 Processor -- start 2024-05-10T05:39:24.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.505+0000 7fb5c6000700 1 -- 172.21.15.131:0/1656560108 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb5b40ce8f0 con 0x7fb5c8109330 2024-05-10T05:39:24.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.506+0000 7fb5e845eb80 1 -- start start 2024-05-10T05:39:24.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.506+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419233720 0x55d419233b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.506+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4192345f0 0x55d419234a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.506+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d41923b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.506+0000 7fb5e845eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d41915a1d0 con 0x55d419234f40 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.506+0000 7fb5e845eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d419157b90 con 0x55d4192345f0 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.506+0000 7fb5e845eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d419153480 con 0x55d419233720 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.507+0000 7fb5cd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4192345f0 0x55d419234a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.507+0000 7fb5cd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4192345f0 0x55d419234a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56518/0 (socket says 172.21.15.131:56518) 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.507+0000 7fb5cd200700 1 -- 172.21.15.131:0/1874648865 learned_addr learned my addr 172.21.15.131:0/1874648865 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.507+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/1874648865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419233720 0x55d419233b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.507+0000 7fb5ce600700 1 --2- 172.21.15.131:0/1874648865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d41923b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.507+0000 7fb5cd200700 1 -- 172.21.15.131:0/1874648865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419233720 msgr2=0x55d419233b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.507+0000 7fb5cd200700 1 --2- 172.21.15.131:0/1874648865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419233720 0x55d419233b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.507+0000 7fb5cd200700 1 -- 172.21.15.131:0/1874648865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 msgr2=0x55d41923b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.507+0000 7fb5cd200700 1 --2- 172.21.15.131:0/1874648865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d41923b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.507+0000 7fb5cd200700 1 -- 172.21.15.131:0/1874648865 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d418fd4430 con 0x55d4192345f0 2024-05-10T05:39:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.508+0000 7fb5cd200700 1 --2- 172.21.15.131:0/1874648865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4192345f0 0x55d419234a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40a3c30 tx=0x7fb5b409b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cbbaf5658fb5839 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.510+0000 7fb5baa00700 1 -- 172.21.15.131:0/1874648865 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b409fba0 con 0x55d4192345f0 2024-05-10T05:39:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.510+0000 7fb5baa00700 1 -- 172.21.15.131:0/1874648865 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5b40a4160 con 0x55d4192345f0 2024-05-10T05:39:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.510+0000 7fb5baa00700 1 -- 172.21.15.131:0/1874648865 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40a3f30 con 0x55d4192345f0 2024-05-10T05:39:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.510+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb5b00a13b0 0x7fb5b00a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.514+0000 7fb5e845eb80 1 -- 172.21.15.131:0/1874648865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4192345f0 msgr2=0x55d419234a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.514+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/1874648865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4192345f0 0x55d419234a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40a3c30 tx=0x7fb5b409b6d0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.514+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb5b00a13b0 0x7fb5b00a38c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55d419233400 tx=0x7fb5bc0047c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.515+0000 7fb5e845eb80 1 -- 172.21.15.131:0/1874648865 shutdown_connections 2024-05-10T05:39:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.515+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/1874648865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d41923b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.515+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/1874648865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419233720 0x55d419233b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.515+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/1874648865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4192345f0 0x55d419234a00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.515+0000 7fb5e845eb80 1 -- 172.21.15.131:0/1874648865 >> 172.21.15.131:0/1874648865 conn(0x55d419195ae0 msgr2=0x55d41922e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.516+0000 7fb5e845eb80 1 -- 172.21.15.131:0/1874648865 shutdown_connections 2024-05-10T05:39:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.516+0000 7fb5e845eb80 1 -- 172.21.15.131:0/1874648865 wait complete. 2024-05-10T05:39:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.517+0000 7fb5e845eb80 1 -- start start 2024-05-10T05:39:24.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.518+0000 7fb5e845eb80 1 Processor -- start 2024-05-10T05:39:24.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.520+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419233720 0x55d41923d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.520+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d41923d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.520+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d419244680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.520+0000 7fb5e845eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4190ef5e0 con 0x55d419234f40 2024-05-10T05:39:24.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.520+0000 7fb5e845eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4190ef280 con 0x55d419233720 2024-05-10T05:39:24.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.520+0000 7fb5cd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d41923d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.521+0000 7fb5cd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d41923d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43952/0 (socket says 172.21.15.131:43952) 2024-05-10T05:39:24.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.521+0000 7fb5cd200700 1 -- 172.21.15.131:0/755143505 learned_addr learned my addr 172.21.15.131:0/755143505 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:24.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.521+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419233720 0x55d41923d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.521+0000 7fb5e845eb80 1 -- 172.21.15.131:0/755143505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d418fd6720 con 0x55d4192345f0 2024-05-10T05:39:24.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.522+0000 7fb5e845eb80 10 client.0 _ll_register_callbacks cb 0x55d4190f8bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.522+0000 7fb5cdc00700 1 -- 172.21.15.131:0/755143505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 msgr2=0x55d41923d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.522+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d41923d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.522+0000 7fb5cdc00700 1 -- 172.21.15.131:0/755143505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 msgr2=0x55d419244680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.522+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d419244680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.522+0000 7fb5cdc00700 1 -- 172.21.15.131:0/755143505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c81af210 con 0x55d419233720 2024-05-10T05:39:24.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.523+0000 7fb5af400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:39:24.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.523+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419233720 0x55d41923d1e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8109230 tx=0x7fb5bc005ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9859c809fb5358b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:24.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.524+0000 7fb5af400700 1 -- 172.21.15.131:0/755143505 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc011dd0 con 0x55d419233720 2024-05-10T05:39:24.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.524+0000 7fb5af400700 1 -- 172.21.15.131:0/755143505 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb5bc01c420 con 0x55d419233720 2024-05-10T05:39:24.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.524+0000 7fb5e845eb80 1 -- 172.21.15.131:0/755143505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4190ac2f0 con 0x55d419233720 2024-05-10T05:39:24.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.525+0000 7fb5af400700 1 -- 172.21.15.131:0/755143505 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5bc01c5e0 con 0x55d419233720 2024-05-10T05:39:24.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.525+0000 7fb5e845eb80 10 client.26477 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:24.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.525+0000 7fb5e845eb80 1 -- 172.21.15.131:0/755143505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d418f80be0 con 0x55d419233720 2024-05-10T05:39:24.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.525+0000 7fb5af400700 1 -- 172.21.15.131:0/755143505 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4190ac2f0 con 0x55d419233720 2024-05-10T05:39:24.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.526+0000 7fb5af400700 1 -- 172.21.15.131:0/755143505 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb5bc0268a0 con 0x55d419233720 2024-05-10T05:39:24.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.526+0000 7fb5e845eb80 10 client.26477 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:24.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.526+0000 7fb5e845eb80 10 client.26477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:39:24.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.526+0000 7fb5e845eb80 20 client.26477 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:24.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.526+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d419248ff0 0x55d41924b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.526+0000 7fb5e845eb80 4 client.26477 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:24.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.526+0000 7fb5e845eb80 1 -- 172.21.15.131:0/755143505 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d419202800 con 0x55d419248ff0 2024-05-10T05:39:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.526+0000 7fb5cd200700 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d419248ff0 0x55d41924b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.527+0000 7fb5af400700 10 client.26477 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:24.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.527+0000 7fb5cd200700 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d419248ff0 0x55d41924b440 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2107034c22e001a server_cookie=7965c8d22d33131 in_seq=0 out_seq=0 2024-05-10T05:39:24.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.528+0000 7fb5af400700 1 -- 172.21.15.131:0/755143505 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d419202800 con 0x55d419248ff0 2024-05-10T05:39:24.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.528+0000 7fb5af400700 10 client.26477 handle_command_reply: tid=0 2024-05-10T05:39:24.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.528+0000 7fb5e845eb80 1 client.26477 shutdown 2024-05-10T05:39:24.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5e845eb80 1 -- 172.21.15.131:0/755143505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419233720 msgr2=0x55d41923d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419233720 0x55d41923d1e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8109230 tx=0x7fb5bc005ca0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5e845eb80 1 -- 172.21.15.131:0/755143505 shutdown_connections 2024-05-10T05:39:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d419248ff0 0x55d41924b440 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d419244680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419233720 0x55d41923d1e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/755143505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d41923d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5e845eb80 1 -- 172.21.15.131:0/755143505 >> 172.21.15.131:0/755143505 conn(0x55d419195ae0 msgr2=0x55d41922e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5af400700 0 client.26477 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5e845eb80 1 -- 172.21.15.131:0/755143505 shutdown_connections 2024-05-10T05:39:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.529+0000 7fb5e845eb80 1 -- 172.21.15.131:0/755143505 wait complete. 2024-05-10T05:39:24.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.530+0000 7fb5e845eb80 20 client.26477 trim_cache size 0 max 16384 2024-05-10T05:39:24.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.566+0000 7fb5e845eb80 1 Processor -- start 2024-05-10T05:39:24.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.567+0000 7fb5e845eb80 1 -- start start 2024-05-10T05:39:24.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.567+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419195ae0 0x55d41924b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.567+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d419244640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.567+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d41923c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.567+0000 7fb5e845eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d419228840 con 0x55d419234f40 2024-05-10T05:39:24.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.567+0000 7fb5e845eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d41923c1c0 con 0x55d419195ae0 2024-05-10T05:39:24.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.567+0000 7fb5e845eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d41923b410 con 0x55d4192345f0 2024-05-10T05:39:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.567+0000 7fb5cdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419195ae0 0x55d41924b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.568+0000 7fb5cd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d419244640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.568+0000 7fb5cd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d419244640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43956/0 (socket says 172.21.15.131:43956) 2024-05-10T05:39:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.568+0000 7fb5cd200700 1 -- 172.21.15.131:0/3432889597 learned_addr learned my addr 172.21.15.131:0/3432889597 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.568+0000 7fb5ce600700 1 --2- 172.21.15.131:0/3432889597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d41923c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.568+0000 7fb5cd200700 1 -- 172.21.15.131:0/3432889597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419195ae0 msgr2=0x55d41924b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:24.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.568+0000 7fb5cd200700 1 --2- 172.21.15.131:0/3432889597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419195ae0 0x55d41924b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.568+0000 7fb5cd200700 1 -- 172.21.15.131:0/3432889597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 msgr2=0x55d41923c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:24.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.568+0000 7fb5cd200700 1 --2- 172.21.15.131:0/3432889597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d41923c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.568+0000 7fb5cd200700 1 -- 172.21.15.131:0/3432889597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4191195e0 con 0x55d4192345f0 2024-05-10T05:39:24.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.569+0000 7fb5cd200700 1 --2- 172.21.15.131:0/3432889597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d419244640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4098270 tx=0x7fb5b401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd0ddfe0bfce972 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:24.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.569+0000 7fb5bb400700 1 -- 172.21.15.131:0/3432889597 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40b3040 con 0x55d4192345f0 2024-05-10T05:39:24.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.569+0000 7fb5bb400700 1 -- 172.21.15.131:0/3432889597 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5b409fc20 con 0x55d4192345f0 2024-05-10T05:39:24.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.569+0000 7fb5bb400700 1 -- 172.21.15.131:0/3432889597 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40afdd0 con 0x55d4192345f0 2024-05-10T05:39:24.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.570+0000 7fb5e845eb80 1 -- 172.21.15.131:0/3432889597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 msgr2=0x55d419244640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.570+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/3432889597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d419244640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4098270 tx=0x7fb5b401a040 comp rx=0 tx=0).stop 2024-05-10T05:39:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.571+0000 7fb5e845eb80 1 -- 172.21.15.131:0/3432889597 shutdown_connections 2024-05-10T05:39:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.571+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/3432889597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d419234f40 0x55d41923c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.571+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/3432889597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419195ae0 0x55d41924b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.571+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/3432889597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4192345f0 0x55d419244640 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.571+0000 7fb5e845eb80 1 -- 172.21.15.131:0/3432889597 >> 172.21.15.131:0/3432889597 conn(0x55d41922a4d0 msgr2=0x55d41924b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.571+0000 7fb5e845eb80 1 -- 172.21.15.131:0/3432889597 shutdown_connections 2024-05-10T05:39:24.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.571+0000 7fb5e845eb80 1 -- 172.21.15.131:0/3432889597 wait complete. 2024-05-10T05:39:24.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.572+0000 7fb5e845eb80 1 -- start start 2024-05-10T05:39:24.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.572+0000 7fb5e845eb80 1 Processor -- start 2024-05-10T05:39:24.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.573+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419195ae0 0x55d419244030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.573+0000 7fb5e845eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d4192345f0 0x55d419244570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.573+0000 7fb5cdc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419195ae0 0x55d419244030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.573+0000 7fb5cdc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419195ae0 0x55d419244030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43958/0 (socket says 172.21.15.131:43958) 2024-05-10T05:39:24.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.573+0000 7fb5cdc00700 1 -- 172.21.15.131:0/955865479 learned_addr learned my addr 172.21.15.131:0/955865479 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:24.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.573+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419234f40 0x55d4192426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.573+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d419242d10 con 0x55d4192345f0 2024-05-10T05:39:24.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.573+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d419242e80 con 0x55d419234f40 2024-05-10T05:39:24.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.573+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d419243040 con 0x55d419195ae0 2024-05-10T05:39:24.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.573+0000 7fb5e845eb80 10 client.0 _ll_register_callbacks cb 0x55d4190f8bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:24.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.574+0000 7fb5cd200700 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d4192345f0 0x55d419244570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.574+0000 7fb5ce600700 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419234f40 0x55d4192426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.574+0000 7fb5cdc00700 1 -- 172.21.15.131:0/955865479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419234f40 msgr2=0x55d4192426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419234f40 0x55d4192426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5cdc00700 1 -- 172.21.15.131:0/955865479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d4192345f0 msgr2=0x55d419244570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d4192345f0 0x55d419244570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5cdc00700 1 -- 172.21.15.131:0/955865479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d418fd4430 con 0x55d419195ae0 2024-05-10T05:39:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5ce600700 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419234f40 0x55d4192426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5cd200700 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d4192345f0 0x55d419244570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5cdc00700 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419195ae0 0x55d419244030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc026d00 tx=0x7fb5bc026720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=759d65d00a9a9f03 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5af400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:39:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5af400700 1 -- 172.21.15.131:0/955865479 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc004c80 con 0x55d419195ae0 2024-05-10T05:39:24.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5b40a4160 con 0x55d419195ae0 2024-05-10T05:39:24.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.575+0000 7fb5af400700 1 -- 172.21.15.131:0/955865479 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb5bc0269c0 con 0x55d419195ae0 2024-05-10T05:39:24.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5af400700 1 -- 172.21.15.131:0/955865479 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5bc030420 con 0x55d419195ae0 2024-05-10T05:39:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5e845eb80 10 client.26509 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d418fd8d50 con 0x55d419195ae0 2024-05-10T05:39:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5af400700 1 -- 172.21.15.131:0/955865479 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5b40a4160 con 0x55d419195ae0 2024-05-10T05:39:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5af400700 1 -- 172.21.15.131:0/955865479 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb5bc030750 con 0x55d419195ae0 2024-05-10T05:39:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5e845eb80 10 client.26509 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5e845eb80 10 client.26509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:39:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5e845eb80 20 client.26509 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d4192a07a0 0x55d4192a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5e845eb80 4 client.26509 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:39:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.576+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d41914bf40 con 0x55d4192a07a0 2024-05-10T05:39:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.579+0000 7fb5cd200700 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d4192a07a0 0x55d4192a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:24.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.579+0000 7fb5cd200700 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d4192a07a0 0x55d4192a2bc0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37423238d8d66b75 server_cookie=f385ae8b4cf9a2c1 in_seq=0 out_seq=0 2024-05-10T05:39:24.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.579+0000 7fb5af400700 10 client.26509 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:24.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.580+0000 7fb5af400700 1 -- 172.21.15.131:0/955865479 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d41914bf40 con 0x55d4192a07a0 2024-05-10T05:39:24.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.580+0000 7fb5af400700 10 client.26509 handle_command_reply: tid=0 2024-05-10T05:39:24.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.581+0000 7fb5e845eb80 1 client.26509 shutdown 2024-05-10T05:39:24.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.581+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419195ae0 msgr2=0x55d419244030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:24.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.581+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419195ae0 0x55d419244030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc026d00 tx=0x7fb5bc026720 comp rx=0 tx=0).stop 2024-05-10T05:39:24.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.581+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 shutdown_connections 2024-05-10T05:39:24.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.581+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d4192a07a0 0x55d4192a2bc0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.581+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d419234f40 0x55d4192426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.581+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d419195ae0 0x55d419244030 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.581+0000 7fb5e845eb80 1 --2- 172.21.15.131:0/955865479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d4192345f0 0x55d419244570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.581+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 >> 172.21.15.131:0/955865479 conn(0x55d41922a4d0 msgr2=0x55d419232330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:24.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.581+0000 7fb5af400700 0 client.26509 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:24.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.582+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 shutdown_connections 2024-05-10T05:39:24.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.582+0000 7fb5e845eb80 1 -- 172.21.15.131:0/955865479 wait complete. 2024-05-10T05:39:24.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.582+0000 7fb5e845eb80 20 client.26509 trim_cache size 0 max 16384 2024-05-10T05:39:24.584 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:39:24.584 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:39:24.584 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "010aad8c-10a6-468a-9998-99fc3faa422d", 2024-05-10T05:39:24.584 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:39:24.584 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:39:24.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.583+0000 7fb5d0600700 1 -- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb5b00a13b0 msgr2=0x7fb5b00a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:24.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.583+0000 7fb5d0600700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb5b00a13b0 0x7fb5b00a38c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55d419233400 tx=0x7fb5bc0047c0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.583+0000 7fb5d0600700 1 -- 172.21.15.131:0/1656560108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c8109330 msgr2=0x7fb5c8072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:24.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.583+0000 7fb5d0600700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c8109330 0x7fb5c8072e40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b4005740 comp rx=0 tx=0).stop 2024-05-10T05:39:24.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.583+0000 7fb5d0600700 1 -- 172.21.15.131:0/1656560108 shutdown_connections 2024-05-10T05:39:24.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.583+0000 7fb5d0600700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb5b00a13b0 0x7fb5b00a38c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.583+0000 7fb5d0600700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5c8108210 0x7fb5c8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.583+0000 7fb5d0600700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8109cd0 0x7fb5c8071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.583+0000 7fb5d0600700 1 --2- 172.21.15.131:0/1656560108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c8109330 0x7fb5c8072e40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:24.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.583+0000 7fb5d0600700 1 -- 172.21.15.131:0/1656560108 >> 172.21.15.131:0/1656560108 conn(0x7fb5c8077240 msgr2=0x7fb5c810c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:24.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.584+0000 7fb5d0600700 1 -- 172.21.15.131:0/1656560108 shutdown_connections 2024-05-10T05:39:24.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:24.584+0000 7fb5d0600700 1 -- 172.21.15.131:0/1656560108 wait complete. 2024-05-10T05:39:24.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:39:24.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:24 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:39:24.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:24 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:39:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:24 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:39:25.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:24 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:39:25.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:24 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:39:25.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:24 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:39:25.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.413+0000 7f2a40c00700 1 -- 172.21.15.131:0/2822366607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c109bb0 msgr2=0x7f2a3c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:25.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.413+0000 7f2a40c00700 1 --2- 172.21.15.131:0/2822366607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c109bb0 0x7f2a3c10bfd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00bd80 tx=0x7f2a2c00d330 comp rx=0 tx=0).stop 2024-05-10T05:39:25.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.427+0000 7f2a40c00700 1 -- 172.21.15.131:0/2822366607 shutdown_connections 2024-05-10T05:39:25.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.427+0000 7f2a40c00700 1 --2- 172.21.15.131:0/2822366607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c109bb0 0x7f2a3c10bfd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.427+0000 7f2a40c00700 1 --2- 172.21.15.131:0/2822366607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3c10ee70 0x7f2a3c111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.427+0000 7f2a40c00700 1 --2- 172.21.15.131:0/2822366607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c10c510 0x7f2a3c10e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.427+0000 7f2a40c00700 1 -- 172.21.15.131:0/2822366607 >> 172.21.15.131:0/2822366607 conn(0x7f2a3c06e800 msgr2=0x7f2a3c070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:25.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.428+0000 7f2a40c00700 1 -- 172.21.15.131:0/2822366607 shutdown_connections 2024-05-10T05:39:25.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.428+0000 7f2a40c00700 1 -- 172.21.15.131:0/2822366607 wait complete. 2024-05-10T05:39:25.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.437+0000 7f2a40c00700 1 Processor -- start 2024-05-10T05:39:25.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.440+0000 7f2a40c00700 1 -- start start 2024-05-10T05:39:25.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.445+0000 7f2a40c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c109bb0 0x7f2a3c1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.445+0000 7f2a40c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c10c510 0x7f2a3c1a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.446+0000 7f2a40c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3c10ee70 0x7f2a3c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.446+0000 7f2a40c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c19eb70 con 0x7f2a3c10c510 2024-05-10T05:39:25.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.446+0000 7f2a40c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c19ece0 con 0x7f2a3c109bb0 2024-05-10T05:39:25.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.446+0000 7f2a40c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c19eea0 con 0x7f2a3c10ee70 2024-05-10T05:39:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.447+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c109bb0 0x7f2a3c1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.447+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c10c510 0x7f2a3c1a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.447+0000 7f2a3be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3c10ee70 0x7f2a3c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.447+0000 7f2a3be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3c10ee70 0x7f2a3c19e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43982/0 (socket says 172.21.15.131:43982) 2024-05-10T05:39:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.447+0000 7f2a3be00700 1 -- 172.21.15.131:0/1539759061 learned_addr learned my addr 172.21.15.131:0/1539759061 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.448+0000 7f2a3be00700 1 -- 172.21.15.131:0/1539759061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c109bb0 msgr2=0x7f2a3c1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.448+0000 7f2a3be00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c109bb0 0x7f2a3c1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.448+0000 7f2a3be00700 1 -- 172.21.15.131:0/1539759061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c10c510 msgr2=0x7f2a3c1a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:25.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.448+0000 7f2a3be00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c10c510 0x7f2a3c1a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.448+0000 7f2a3be00700 1 -- 172.21.15.131:0/1539759061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a28009520 con 0x7f2a3c10ee70 2024-05-10T05:39:25.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.448+0000 7f2a3aa00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c10c510 0x7f2a3c1a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:25.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.452+0000 7f2a3be00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3c10ee70 0x7f2a3c19e470 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2a34007e90 tx=0x7f2a3400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:25.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.453+0000 7f2a33e00700 1 -- 172.21.15.131:0/1539759061 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400a0d0 con 0x7f2a3c10ee70 2024-05-10T05:39:25.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.453+0000 7f2a33e00700 1 -- 172.21.15.131:0/1539759061 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a34020070 con 0x7f2a3c10ee70 2024-05-10T05:39:25.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.453+0000 7f2a33e00700 1 -- 172.21.15.131:0/1539759061 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34005280 con 0x7f2a3c10ee70 2024-05-10T05:39:25.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.456+0000 7f2a40c00700 1 -- 172.21.15.131:0/1539759061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c00b800 con 0x7f2a3c10ee70 2024-05-10T05:39:25.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.456+0000 7f2a40c00700 1 -- 172.21.15.131:0/1539759061 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c1a2760 con 0x7f2a3c10ee70 2024-05-10T05:39:25.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.472+0000 7f2a33e00700 1 -- 172.21.15.131:0/1539759061 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2a3400c8d0 con 0x7f2a3c10ee70 2024-05-10T05:39:25.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.474+0000 7f2a33e00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a200a1320 0x7f2a200a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.475+0000 7f2a33e00700 1 -- 172.21.15.131:0/1539759061 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2a340dc8d0 con 0x7f2a3c10ee70 2024-05-10T05:39:25.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.478+0000 7f2a3b400700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a200a1320 0x7f2a200a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.480+0000 7f2a3b400700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a200a1320 0x7f2a200a3830 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c006010 tx=0x7f2a2c00de80 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:25.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.520+0000 7f2a5a09eb80 1 Processor -- start 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.521+0000 7f2a5a09eb80 1 -- start start 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.521+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc602530 0x561cbc602960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.521+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6037a0 0x561cbc603bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.521+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc6040f0 0x561cbc60a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.521+0000 7f2a5a09eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc529150 con 0x561cbc6040f0 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.521+0000 7f2a5a09eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc526b10 con 0x561cbc6037a0 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.521+0000 7f2a5a09eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc522400 con 0x561cbc602530 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.522+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc602530 0x561cbc602960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.522+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6037a0 0x561cbc603bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.522+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6037a0 0x561cbc603bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56550/0 (socket says 172.21.15.131:56550) 2024-05-10T05:39:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.522+0000 7f2a3aa00700 1 -- 172.21.15.131:0/2936845151 learned_addr learned my addr 172.21.15.131:0/2936845151 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.523+0000 7f2a3aa00700 1 -- 172.21.15.131:0/2936845151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc602530 msgr2=0x561cbc602960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.523+0000 7f2a3aa00700 1 --2- 172.21.15.131:0/2936845151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc602530 0x561cbc602960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.523+0000 7f2a3aa00700 1 -- 172.21.15.131:0/2936845151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc6040f0 msgr2=0x561cbc60a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.524+0000 7f2a3aa00700 1 --2- 172.21.15.131:0/2936845151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc6040f0 0x561cbc60a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.526+0000 7f2a3aa00700 1 -- 172.21.15.131:0/2936845151 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cbc3a33b0 con 0x561cbc6037a0 2024-05-10T05:39:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.526+0000 7f2a3aa00700 1 --2- 172.21.15.131:0/2936845151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6037a0 0x561cbc603bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a28011810 tx=0x7f2a28011b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a4194482611d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:25.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.526+0000 7f2a27e00700 1 -- 172.21.15.131:0/2936845151 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28004830 con 0x561cbc6037a0 2024-05-10T05:39:25.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.527+0000 7f2a27e00700 1 -- 172.21.15.131:0/2936845151 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a28004e50 con 0x561cbc6037a0 2024-05-10T05:39:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.534+0000 7f2a27e00700 1 -- 172.21.15.131:0/2936845151 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28004830 con 0x561cbc6037a0 2024-05-10T05:39:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.534+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2936845151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6037a0 msgr2=0x561cbc603bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.534+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/2936845151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6037a0 0x561cbc603bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a28011810 tx=0x7f2a28011b20 comp rx=0 tx=0).stop 2024-05-10T05:39:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.535+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2936845151 shutdown_connections 2024-05-10T05:39:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.535+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/2936845151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc6040f0 0x561cbc60a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.535+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/2936845151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc602530 0x561cbc602960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.535+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/2936845151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6037a0 0x561cbc603bb0 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a28011810 tx=0x7f2a28011b20 comp rx=0 tx=0).stop 2024-05-10T05:39:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.535+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2936845151 >> 172.21.15.131:0/2936845151 conn(0x561cbc564a60 msgr2=0x561cbc5fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.535+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2936845151 shutdown_connections 2024-05-10T05:39:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.535+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2936845151 wait complete. 2024-05-10T05:39:25.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.538+0000 7f2a5a09eb80 1 -- start start 2024-05-10T05:39:25.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.539+0000 7f2a5a09eb80 1 Processor -- start 2024-05-10T05:39:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.540+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc60c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.540+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 0x561cbc60cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.540+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc611e80 0x561cbc6142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.540+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 0x561cbc60cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.540+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 0x561cbc60cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:43986/0 (socket says 172.21.15.131:43986) 2024-05-10T05:39:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.541+0000 7f2a3aa00700 1 -- 172.21.15.131:0/2628941273 learned_addr learned my addr 172.21.15.131:0/2628941273 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.541+0000 7f2a3be00700 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc611e80 0x561cbc6142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.541+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc4be560 con 0x561cbc602530 2024-05-10T05:39:25.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.546+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc4be200 con 0x561cbc611e80 2024-05-10T05:39:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.546+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc3a56a0 con 0x561cbc6040f0 2024-05-10T05:39:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.546+0000 7f2a3b400700 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc60c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.546+0000 7f2a5a09eb80 10 client.0 _ll_register_callbacks cb 0x561cbc4c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.547+0000 7f2a3b400700 1 -- 172.21.15.131:0/2628941273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 msgr2=0x561cbc60cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.547+0000 7f2a3b400700 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 0x561cbc60cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.547+0000 7f2a3b400700 1 -- 172.21.15.131:0/2628941273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc611e80 msgr2=0x561cbc6142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.547+0000 7f2a3b400700 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc611e80 0x561cbc6142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.547+0000 7f2a3b400700 1 -- 172.21.15.131:0/2628941273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a28011440 con 0x561cbc602530 2024-05-10T05:39:25.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.547+0000 7f2a24c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:39:25.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.547+0000 7f2a3b400700 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc60c9f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00def0 tx=0x7f2a2c00b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db62d5eafe0c888 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.548+0000 7f2a3be00700 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc611e80 0x561cbc6142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.548+0000 7f2a3aa00700 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 0x561cbc60cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.548+0000 7f2a24c00700 1 -- 172.21.15.131:0/2628941273 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c025040 con 0x561cbc602530 2024-05-10T05:39:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.548+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cbc47b270 con 0x561cbc602530 2024-05-10T05:39:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.548+0000 7f2a24c00700 1 -- 172.21.15.131:0/2628941273 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2a2c014640 con 0x561cbc602530 2024-05-10T05:39:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.549+0000 7f2a24c00700 1 -- 172.21.15.131:0/2628941273 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a2c025040 con 0x561cbc602530 2024-05-10T05:39:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.549+0000 7f2a5a09eb80 10 client.16296 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.549+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561cbc34fb60 con 0x561cbc602530 2024-05-10T05:39:25.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.549+0000 7f2a24c00700 1 -- 172.21.15.131:0/2628941273 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561cbc47b270 con 0x561cbc602530 2024-05-10T05:39:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.550+0000 7f2a24c00700 1 -- 172.21.15.131:0/2628941273 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2a2c0356b0 con 0x561cbc602530 2024-05-10T05:39:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.550+0000 7f2a5a09eb80 10 client.16296 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.550+0000 7f2a5a09eb80 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:39:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.550+0000 7f2a5a09eb80 20 client.16296 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.550+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561cbc6188e0 0x561cbc61ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.550+0000 7f2a5a09eb80 4 client.16296 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.550+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cbc5d1780 con 0x561cbc6188e0 2024-05-10T05:39:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.551+0000 7f2a3aa00700 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561cbc6188e0 0x561cbc61ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.553+0000 7f2a3aa00700 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561cbc6188e0 0x561cbc61ad30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11877ecee70ee878 server_cookie=df4319c85f9a82f5 in_seq=0 out_seq=0 2024-05-10T05:39:25.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.554+0000 7f2a24c00700 10 client.16296 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:25.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.556+0000 7f2a24c00700 1 -- 172.21.15.131:0/2628941273 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cbc5d1780 con 0x561cbc6188e0 2024-05-10T05:39:25.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.556+0000 7f2a24c00700 10 client.16296 handle_command_reply: tid=0 2024-05-10T05:39:25.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.556+0000 7f2a5a09eb80 1 client.16296 shutdown 2024-05-10T05:39:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.557+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 msgr2=0x561cbc60c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.557+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc60c9f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00def0 tx=0x7f2a2c00b520 comp rx=0 tx=0).stop 2024-05-10T05:39:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.565+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 shutdown_connections 2024-05-10T05:39:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.565+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561cbc6188e0 0x561cbc61ad30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.565+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc60c9f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.565+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc611e80 0x561cbc6142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.565+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/2628941273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 0x561cbc60cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.565+0000 7f2a24c00700 0 client.16296 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:25.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.565+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 >> 172.21.15.131:0/2628941273 conn(0x561cbc564a60 msgr2=0x561cbc601060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:25.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.566+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 shutdown_connections 2024-05-10T05:39:25.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.566+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/2628941273 wait complete. 2024-05-10T05:39:25.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.566+0000 7f2a5a09eb80 20 client.16296 trim_cache size 0 max 16384 2024-05-10T05:39:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.593+0000 7f2a5a09eb80 1 Processor -- start 2024-05-10T05:39:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.593+0000 7f2a5a09eb80 1 -- start start 2024-05-10T05:39:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.593+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc628c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.594+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6040f0 0x561cbc629160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.594+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc611e80 0x561cbc5fcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.594+0000 7f2a5a09eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc611d00 con 0x561cbc602530 2024-05-10T05:39:25.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.594+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc628c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.594+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc628c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49132/0 (socket says 172.21.15.131:49132) 2024-05-10T05:39:25.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.594+0000 7f2a3b400700 1 -- 172.21.15.131:0/751984011 learned_addr learned my addr 172.21.15.131:0/751984011 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:25.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.594+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/751984011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc5f72f0 con 0x561cbc6040f0 2024-05-10T05:39:25.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.594+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/751984011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc60a960 con 0x561cbc611e80 2024-05-10T05:39:25.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.595+0000 7f2a3be00700 1 --2- 172.21.15.131:0/751984011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc611e80 0x561cbc5fcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.595+0000 7f2a3b400700 1 -- 172.21.15.131:0/751984011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc611e80 msgr2=0x561cbc5fcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:25.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.595+0000 7f2a3b400700 1 --2- 172.21.15.131:0/751984011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc611e80 0x561cbc5fcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.595+0000 7f2a3b400700 1 -- 172.21.15.131:0/751984011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6040f0 msgr2=0x561cbc629160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:39:25.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.595+0000 7f2a3b400700 1 --2- 172.21.15.131:0/751984011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6040f0 0x561cbc629160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.595+0000 7f2a3b400700 1 -- 172.21.15.131:0/751984011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cbc4e8560 con 0x561cbc602530 2024-05-10T05:39:25.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.595+0000 7f2a3b400700 1 --2- 172.21.15.131:0/751984011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc628c20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c000c00 tx=0x7f2a2c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9030ee1a37fe80f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:25.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.597+0000 7f2a31200700 1 -- 172.21.15.131:0/751984011 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c010040 con 0x561cbc602530 2024-05-10T05:39:25.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.597+0000 7f2a31200700 1 -- 172.21.15.131:0/751984011 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a2c00b300 con 0x561cbc602530 2024-05-10T05:39:25.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.597+0000 7f2a31200700 1 -- 172.21.15.131:0/751984011 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c03f6a0 con 0x561cbc602530 2024-05-10T05:39:25.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.598+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/751984011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 msgr2=0x561cbc628c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:25.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.598+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/751984011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc628c20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c000c00 tx=0x7f2a2c00bf50 comp rx=0 tx=0).stop 2024-05-10T05:39:25.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.599+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/751984011 shutdown_connections 2024-05-10T05:39:25.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.599+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/751984011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc602530 0x561cbc628c20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.599+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/751984011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc611e80 0x561cbc5fcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.599+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/751984011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc6040f0 0x561cbc629160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.599+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/751984011 >> 172.21.15.131:0/751984011 conn(0x561cbc564a60 msgr2=0x561cbc6069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:25.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.599+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/751984011 shutdown_connections 2024-05-10T05:39:25.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.599+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/751984011 wait complete. 2024-05-10T05:39:25.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.602+0000 7f2a5a09eb80 1 -- start start 2024-05-10T05:39:25.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.602+0000 7f2a5a09eb80 1 Processor -- start 2024-05-10T05:39:25.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.606+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 0x561cbc628320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.606+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc611e80 0x561cbc628880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.606+0000 7f2a5a09eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc66b960 0x561cbc66dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.607+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc611e80 0x561cbc628880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.607+0000 7f2a5a09eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc66e2c0 con 0x561cbc611e80 2024-05-10T05:39:25.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.607+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc611e80 0x561cbc628880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49140/0 (socket says 172.21.15.131:49140) 2024-05-10T05:39:25.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.607+0000 7f2a3aa00700 1 -- 172.21.15.131:0/1979955984 learned_addr learned my addr 172.21.15.131:0/1979955984 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.607+0000 7f2a3be00700 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc66b960 0x561cbc66dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.607+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/1979955984 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc66e400 con 0x561cbc66b960 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.607+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/1979955984 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbc66e5c0 con 0x561cbc6040f0 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.607+0000 7f2a5a09eb80 10 client.0 _ll_register_callbacks cb 0x561cbc4c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.607+0000 7f2a3b400700 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 0x561cbc628320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.608+0000 7f2a3aa00700 1 -- 172.21.15.131:0/1979955984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 msgr2=0x561cbc628320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.608+0000 7f2a3aa00700 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 0x561cbc628320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.608+0000 7f2a3aa00700 1 -- 172.21.15.131:0/1979955984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc66b960 msgr2=0x561cbc66dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.608+0000 7f2a3aa00700 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc66b960 0x561cbc66dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.608+0000 7f2a3aa00700 1 -- 172.21.15.131:0/1979955984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cbc34fb60 con 0x561cbc611e80 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.608+0000 7f2a3be00700 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc66b960 0x561cbc66dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.609+0000 7f2a3aa00700 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc611e80 0x561cbc628880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2a2801b040 tx=0x7f2a28005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28b59230ae80529 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.610+0000 7f2a24c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:39:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.610+0000 7f2a24c00700 1 -- 172.21.15.131:0/1979955984 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a280055b0 con 0x561cbc611e80 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.610+0000 7f2a24c00700 1 -- 172.21.15.131:0/1979955984 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2a28029150 con 0x561cbc611e80 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.610+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/1979955984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a28004e50 con 0x561cbc611e80 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.610+0000 7f2a24c00700 1 -- 172.21.15.131:0/1979955984 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a28009260 con 0x561cbc611e80 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.612+0000 7f2a5a09eb80 10 client.16302 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.612+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/1979955984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561cbc3a7cd0 con 0x561cbc611e80 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.613+0000 7f2a24c00700 1 -- 172.21.15.131:0/1979955984 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a28004e50 con 0x561cbc611e80 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.613+0000 7f2a24c00700 1 -- 172.21.15.131:0/1979955984 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2a28011840 con 0x561cbc611e80 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.613+0000 7f2a5a09eb80 10 client.16302 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.613+0000 7f2a5a09eb80 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.613+0000 7f2a5a09eb80 20 client.16302 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.613+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561cbc66ff70 0x561cbc672390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.613+0000 7f2a5a09eb80 4 client.16302 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:39:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.613+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/1979955984 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561cbc51aec0 con 0x561cbc66ff70 2024-05-10T05:39:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.617+0000 7f2a3b400700 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561cbc66ff70 0x561cbc672390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.618+0000 7f2a3b400700 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561cbc66ff70 0x561cbc672390 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4292dada685c46 server_cookie=ffd023f0f4928f33 in_seq=0 out_seq=0 2024-05-10T05:39:25.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.618+0000 7f2a24c00700 10 client.16302 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.618+0000 7f2a24c00700 1 -- 172.21.15.131:0/1979955984 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561cbc51aec0 con 0x561cbc66ff70 2024-05-10T05:39:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.618+0000 7f2a24c00700 10 client.16302 handle_command_reply: tid=0 2024-05-10T05:39:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.618+0000 7f2a5a09eb80 1 client.16302 shutdown 2024-05-10T05:39:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.619+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/1979955984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc611e80 msgr2=0x561cbc628880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.619+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc611e80 0x561cbc628880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2a2801b040 tx=0x7f2a28005e50 comp rx=0 tx=0).stop 2024-05-10T05:39:25.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.619+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/1979955984 shutdown_connections 2024-05-10T05:39:25.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.619+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x561cbc66ff70 0x561cbc672390 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.619+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cbc66b960 0x561cbc66dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.619+0000 7f2a24c00700 0 client.16302 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:25.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.619+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561cbc6040f0 0x561cbc628320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.619+0000 7f2a5a09eb80 1 --2- 172.21.15.131:0/1979955984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561cbc611e80 0x561cbc628880 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.619+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/1979955984 >> 172.21.15.131:0/1979955984 conn(0x561cbc564a60 msgr2=0x561cbc5fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:25.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.619+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/1979955984 shutdown_connections 2024-05-10T05:39:25.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.620+0000 7f2a5a09eb80 1 -- 172.21.15.131:0/1979955984 wait complete. 2024-05-10T05:39:25.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.623+0000 7f2a5a09eb80 20 client.16302 trim_cache size 0 max 16384 2024-05-10T05:39:25.625 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:39:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:39:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:39:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "010aad8c-10a6-468a-9998-99fc3faa422d": { 2024-05-10T05:39:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:39:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "010aad8c-10a6-468a-9998-99fc3faa422d", 2024-05-10T05:39:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:39:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:39:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:39:25.626 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:39:25.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.627+0000 7f2a40c00700 1 -- 172.21.15.131:0/1539759061 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a200a1320 msgr2=0x7f2a200a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:25.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.627+0000 7f2a40c00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a200a1320 0x7f2a200a3830 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c006010 tx=0x7f2a2c00de80 comp rx=0 tx=0).stop 2024-05-10T05:39:25.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.627+0000 7f2a40c00700 1 -- 172.21.15.131:0/1539759061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3c10ee70 msgr2=0x7f2a3c19e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:25.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.627+0000 7f2a40c00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3c10ee70 0x7f2a3c19e470 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2a34007e90 tx=0x7f2a3400c330 comp rx=0 tx=0).stop 2024-05-10T05:39:25.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.628+0000 7f2a40c00700 1 -- 172.21.15.131:0/1539759061 shutdown_connections 2024-05-10T05:39:25.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.628+0000 7f2a40c00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a200a1320 0x7f2a200a3830 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.628+0000 7f2a40c00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a3c10ee70 0x7f2a3c19e470 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.629+0000 7f2a40c00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c109bb0 0x7f2a3c1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.629+0000 7f2a40c00700 1 --2- 172.21.15.131:0/1539759061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c10c510 0x7f2a3c1a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:25.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.629+0000 7f2a40c00700 1 -- 172.21.15.131:0/1539759061 >> 172.21.15.131:0/1539759061 conn(0x7f2a3c06e800 msgr2=0x7f2a3c10aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:25.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.629+0000 7f2a40c00700 1 -- 172.21.15.131:0/1539759061 shutdown_connections 2024-05-10T05:39:25.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:25.629+0000 7f2a40c00700 1 -- 172.21.15.131:0/1539759061 wait complete. 2024-05-10T05:39:25.701 INFO:tasks.cephfs.filesystem:scrub status for tag:010aad8c-10a6-468a-9998-99fc3faa422d - {'path': '/', 'tag': '010aad8c-10a6-468a-9998-99fc3faa422d', 'options': 'recursive,force'} 2024-05-10T05:39:25.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:25 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:39:25.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:25 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:39:25.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:25 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:39:25.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:25 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:39:25.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:25 smithi131 ceph-mon[37264]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 259 MiB/s wr, 79 op/s 2024-05-10T05:39:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:25 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:39:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:25 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:39:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:25 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:39:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:25 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:39:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:25 smithi142 ceph-mon[77667]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 259 MiB/s wr, 79 op/s 2024-05-10T05:39:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:25 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:39:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:25 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:39:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:25 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:39:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:25 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:39:26.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:25 smithi157 ceph-mon[78427]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 259 MiB/s wr, 79 op/s 2024-05-10T05:39:28.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:28 smithi157 ceph-mon[78427]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 218 MiB/s wr, 68 op/s 2024-05-10T05:39:28.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:28 smithi142 ceph-mon[77667]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 218 MiB/s wr, 68 op/s 2024-05-10T05:39:28.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:28 smithi131 ceph-mon[37264]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 218 MiB/s wr, 68 op/s 2024-05-10T05:39:30.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:30 smithi157 ceph-mon[78427]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:39:30.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:30 smithi142 ceph-mon[77667]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:39:30.716 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:30 smithi131 ceph-mon[37264]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:39:31.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:31 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:39:31.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:31 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:39:31.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:31 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:39:31.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:31 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:39:31.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:31 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:39:31.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:31 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:39:32.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:32 smithi142 ceph-mon[77667]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 260 MiB/s wr, 81 op/s 2024-05-10T05:39:32.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:32 smithi131 ceph-mon[37264]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 260 MiB/s wr, 81 op/s 2024-05-10T05:39:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:32 smithi157 ceph-mon[78427]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 260 MiB/s wr, 81 op/s 2024-05-10T05:39:33.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:33 smithi131 ceph-mon[37264]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 213 MiB/s wr, 66 op/s 2024-05-10T05:39:33.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:33 smithi157 ceph-mon[78427]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 213 MiB/s wr, 66 op/s 2024-05-10T05:39:33.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:33 smithi142 ceph-mon[77667]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 213 MiB/s wr, 66 op/s 2024-05-10T05:39:36.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:36 smithi157 ceph-mon[78427]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 272 MiB/s wr, 82 op/s 2024-05-10T05:39:36.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:36 smithi142 ceph-mon[77667]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 272 MiB/s wr, 82 op/s 2024-05-10T05:39:36.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:36 smithi131 ceph-mon[37264]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 272 MiB/s wr, 82 op/s 2024-05-10T05:39:38.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:38 smithi157 ceph-mon[78427]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 222 MiB/s wr, 70 op/s 2024-05-10T05:39:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:38 smithi142 ceph-mon[77667]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 222 MiB/s wr, 70 op/s 2024-05-10T05:39:38.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:38 smithi131 ceph-mon[37264]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 222 MiB/s wr, 70 op/s 2024-05-10T05:39:40.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:40 smithi157 ceph-mon[78427]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 253 MiB/s wr, 77 op/s 2024-05-10T05:39:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:40 smithi142 ceph-mon[77667]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 253 MiB/s wr, 77 op/s 2024-05-10T05:39:40.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:40 smithi131 ceph-mon[37264]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 253 MiB/s wr, 77 op/s 2024-05-10T05:39:42.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:42 smithi157 ceph-mon[78427]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 259 MiB/s wr, 79 op/s 2024-05-10T05:39:42.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:42 smithi142 ceph-mon[77667]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 259 MiB/s wr, 79 op/s 2024-05-10T05:39:42.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:42 smithi131 ceph-mon[37264]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 259 MiB/s wr, 79 op/s 2024-05-10T05:39:44.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:44 smithi142 ceph-mon[77667]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 219 MiB/s wr, 66 op/s 2024-05-10T05:39:44.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:44 smithi131 ceph-mon[37264]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 219 MiB/s wr, 66 op/s 2024-05-10T05:39:44.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:44 smithi157 ceph-mon[78427]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 219 MiB/s wr, 66 op/s 2024-05-10T05:39:46.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:46 smithi142 ceph-mon[77667]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 85 op/s 2024-05-10T05:39:46.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:46 smithi131 ceph-mon[37264]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 85 op/s 2024-05-10T05:39:46.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:46 smithi157 ceph-mon[78427]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 85 op/s 2024-05-10T05:39:48.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:48 smithi131 ceph-mon[37264]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 237 MiB/s wr, 72 op/s 2024-05-10T05:39:48.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:48 smithi157 ceph-mon[78427]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 237 MiB/s wr, 72 op/s 2024-05-10T05:39:48.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:48 smithi142 ceph-mon[77667]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 237 MiB/s wr, 72 op/s 2024-05-10T05:39:50.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:50 smithi131 ceph-mon[37264]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 262 MiB/s wr, 77 op/s 2024-05-10T05:39:50.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:50 smithi157 ceph-mon[78427]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 262 MiB/s wr, 77 op/s 2024-05-10T05:39:50.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:50 smithi142 ceph-mon[77667]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 262 MiB/s wr, 77 op/s 2024-05-10T05:39:51.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:51 smithi157 ceph-mon[78427]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 267 MiB/s wr, 81 op/s 2024-05-10T05:39:51.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:51 smithi142 ceph-mon[77667]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 267 MiB/s wr, 81 op/s 2024-05-10T05:39:51.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:51 smithi131 ceph-mon[37264]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 267 MiB/s wr, 81 op/s 2024-05-10T05:39:54.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:54 smithi157 ceph-mon[78427]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 224 MiB/s wr, 67 op/s 2024-05-10T05:39:54.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:54 smithi142 ceph-mon[77667]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 224 MiB/s wr, 67 op/s 2024-05-10T05:39:54.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:54 smithi131 ceph-mon[37264]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 224 MiB/s wr, 67 op/s 2024-05-10T05:39:55.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:39:56.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.457+0000 7feb3d600700 1 -- 172.21.15.131:0/1597732132 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30003a80 con 0x7feb401097a0 2024-05-10T05:39:56.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.462+0000 7feb3fe00700 1 -- 172.21.15.131:0/1597732132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb401097a0 msgr2=0x7feb40109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:56.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.462+0000 7feb3fe00700 1 --2- 172.21.15.131:0/1597732132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb401097a0 0x7feb40109bf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb3000b370 tx=0x7feb3000b680 comp rx=0 tx=0).stop 2024-05-10T05:39:56.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.465+0000 7feb3fe00700 1 -- 172.21.15.131:0/1597732132 shutdown_connections 2024-05-10T05:39:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.465+0000 7feb3fe00700 1 --2- 172.21.15.131:0/1597732132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb40108c90 0x7feb401107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.465+0000 7feb3fe00700 1 --2- 172.21.15.131:0/1597732132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb401097a0 0x7feb40109bf0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.466+0000 7feb3fe00700 1 --2- 172.21.15.131:0/1597732132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb40108340 0x7feb40108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.466+0000 7feb3fe00700 1 -- 172.21.15.131:0/1597732132 >> 172.21.15.131:0/1597732132 conn(0x7feb4006e800 msgr2=0x7feb40070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:56.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.469+0000 7feb3fe00700 1 -- 172.21.15.131:0/1597732132 shutdown_connections 2024-05-10T05:39:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.470+0000 7feb3fe00700 1 -- 172.21.15.131:0/1597732132 wait complete. 2024-05-10T05:39:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.471+0000 7feb3fe00700 1 Processor -- start 2024-05-10T05:39:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.473+0000 7feb3fe00700 1 -- start start 2024-05-10T05:39:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.473+0000 7feb3fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb40108340 0x7feb4019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.474+0000 7feb3fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb40108c90 0x7feb4019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.474+0000 7feb3fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb401097a0 0x7feb40079470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.474+0000 7feb3fe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40079a70 con 0x7feb40108c90 2024-05-10T05:39:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.474+0000 7feb3fe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40079be0 con 0x7feb401097a0 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.475+0000 7feb3fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40079da0 con 0x7feb40108340 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.475+0000 7feb3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb401097a0 0x7feb40079470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.475+0000 7feb3ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb40108340 0x7feb4019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.475+0000 7feb3ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb40108340 0x7feb4019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34416/0 (socket says 172.21.15.131:34416) 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.475+0000 7feb3ea00700 1 -- 172.21.15.131:0/3967659501 learned_addr learned my addr 172.21.15.131:0/3967659501 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.475+0000 7feb3e000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb40108c90 0x7feb4019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.476+0000 7feb3ea00700 1 -- 172.21.15.131:0/3967659501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb401097a0 msgr2=0x7feb40079470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.476+0000 7feb3ea00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb401097a0 0x7feb40079470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.476+0000 7feb3ea00700 1 -- 172.21.15.131:0/3967659501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb40108c90 msgr2=0x7feb4019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.476+0000 7feb3ea00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb40108c90 0x7feb4019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.476+0000 7feb3ea00700 1 -- 172.21.15.131:0/3967659501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3000b050 con 0x7feb40108340 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.476+0000 7feb3e000700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb40108c90 0x7feb4019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.476+0000 7feb3ea00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb40108340 0x7feb4019e400 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feb30009d70 tx=0x7feb30003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:56.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.477+0000 7feb37400700 1 -- 172.21.15.131:0/3967659501 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3000e040 con 0x7feb40108340 2024-05-10T05:39:56.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.477+0000 7feb37400700 1 -- 172.21.15.131:0/3967659501 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb3001fe30 con 0x7feb40108340 2024-05-10T05:39:56.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.477+0000 7feb37400700 1 -- 172.21.15.131:0/3967659501 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30004da0 con 0x7feb40108340 2024-05-10T05:39:56.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.477+0000 7feb3fe00700 1 -- 172.21.15.131:0/3967659501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4007a020 con 0x7feb40108340 2024-05-10T05:39:56.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.477+0000 7feb3fe00700 1 -- 172.21.15.131:0/3967659501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4007a4e0 con 0x7feb40108340 2024-05-10T05:39:56.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.488+0000 7feb37400700 1 -- 172.21.15.131:0/3967659501 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7feb30032430 con 0x7feb40108340 2024-05-10T05:39:56.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.490+0000 7feb5d5e4b80 1 Processor -- start 2024-05-10T05:39:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.490+0000 7feb5d5e4b80 1 -- start start 2024-05-10T05:39:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.491+0000 7feb5d5e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd593a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.492+0000 7feb5d5e4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd593bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.492+0000 7feb5d5e4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd59429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.492+0000 7feb5d5e4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd5861150 con 0x55fdd593c0f0 2024-05-10T05:39:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.492+0000 7feb5d5e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd585eb10 con 0x55fdd593a530 2024-05-10T05:39:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.492+0000 7feb3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd593a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.492+0000 7feb3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd593a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57742/0 (socket says 172.21.15.131:57742) 2024-05-10T05:39:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.492+0000 7feb3f400700 1 -- 172.21.15.131:0/1165473480 learned_addr learned my addr 172.21.15.131:0/1165473480 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.492+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/1165473480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd585a400 con 0x55fdd593b7a0 2024-05-10T05:39:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.492+0000 7feb37400700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feb240a12d0 0x7feb240a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.493+0000 7feb3e000700 1 --2- 172.21.15.131:0/1165473480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd593bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.493+0000 7feb37400700 1 -- 172.21.15.131:0/3967659501 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7feb300d5790 con 0x7feb40108340 2024-05-10T05:39:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.493+0000 7feb3e000700 1 -- 172.21.15.131:0/1165473480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 msgr2=0x55fdd593a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.493+0000 7feb3e000700 1 --2- 172.21.15.131:0/1165473480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd593a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.493+0000 7feb3e000700 1 -- 172.21.15.131:0/1165473480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 msgr2=0x55fdd59429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.493+0000 7feb3f400700 1 --2- 172.21.15.131:0/1165473480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd59429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.493+0000 7feb3e000700 1 --2- 172.21.15.131:0/1165473480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd59429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.493+0000 7feb3e000700 1 -- 172.21.15.131:0/1165473480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdd56db3b0 con 0x55fdd593b7a0 2024-05-10T05:39:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.494+0000 7feb3f400700 1 --2- 172.21.15.131:0/1165473480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd59429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.494+0000 7feb3e000700 1 --2- 172.21.15.131:0/1165473480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd593bbb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009ca0 tx=0x7feb2c010370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4731aa6ffe93e400 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.495+0000 7feb2b400700 1 -- 172.21.15.131:0/1165473480 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c010bf0 con 0x55fdd593b7a0 2024-05-10T05:39:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.495+0000 7feb2b400700 1 -- 172.21.15.131:0/1165473480 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb2c0044f0 con 0x55fdd593b7a0 2024-05-10T05:39:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.495+0000 7feb2b400700 1 -- 172.21.15.131:0/1165473480 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c003ea0 con 0x55fdd593b7a0 2024-05-10T05:39:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.496+0000 7feb3ea00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feb240a12d0 0x7feb240a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.497+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/1165473480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 msgr2=0x55fdd593bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.497+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/1165473480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd593bbb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009ca0 tx=0x7feb2c010370 comp rx=0 tx=0).stop 2024-05-10T05:39:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.497+0000 7feb3ea00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feb240a12d0 0x7feb240a37e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb30020fd0 tx=0x7feb30009ba0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.497+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/1165473480 shutdown_connections 2024-05-10T05:39:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.497+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/1165473480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd59429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.497+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/1165473480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd593a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.497+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/1165473480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd593bbb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.497+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/1165473480 >> 172.21.15.131:0/1165473480 conn(0x55fdd589ca60 msgr2=0x55fdd5935330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.498+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/1165473480 shutdown_connections 2024-05-10T05:39:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.498+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/1165473480 wait complete. 2024-05-10T05:39:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.499+0000 7feb5d5e4b80 1 -- start start 2024-05-10T05:39:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.499+0000 7feb5d5e4b80 1 Processor -- start 2024-05-10T05:39:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.500+0000 7feb5d5e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5944910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.501+0000 7feb3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5944910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.501+0000 7feb3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5944910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57748/0 (socket says 172.21.15.131:57748) 2024-05-10T05:39:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.501+0000 7feb3f400700 1 -- 172.21.15.131:0/3885041436 learned_addr learned my addr 172.21.15.131:0/3885041436 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.501+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd5944e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.501+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd594bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.501+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd57f6560 con 0x55fdd593c0f0 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.501+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd57f6200 con 0x55fdd593a530 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.501+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd56dd6a0 con 0x55fdd593b7a0 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.501+0000 7feb5d5e4b80 10 client.0 _ll_register_callbacks cb 0x55fdd57ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.502+0000 7feb3e000700 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd5944e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.502+0000 7feb3f400700 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd594bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.502+0000 7feb3f400700 1 -- 172.21.15.131:0/3885041436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 msgr2=0x55fdd5944e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.502+0000 7feb3f400700 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd5944e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.502+0000 7feb3f400700 1 -- 172.21.15.131:0/3885041436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 msgr2=0x55fdd594bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.502+0000 7feb3f400700 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd594bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.502+0000 7feb3f400700 1 -- 172.21.15.131:0/3885041436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c009900 con 0x55fdd593a530 2024-05-10T05:39:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.502+0000 7feb3e000700 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd5944e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:56.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.503+0000 7feb1fe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:39:56.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.503+0000 7feb3f400700 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5944910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feb38007e20 tx=0x7feb38009100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c8c1e5df40d4d4b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:56.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.504+0000 7feb1fe00700 1 -- 172.21.15.131:0/3885041436 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800e050 con 0x55fdd593a530 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.504+0000 7feb1fe00700 1 -- 172.21.15.131:0/3885041436 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7feb3801e430 con 0x55fdd593a530 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.504+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdd57b3270 con 0x55fdd593a530 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.505+0000 7feb1fe00700 1 -- 172.21.15.131:0/3885041436 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb3800e050 con 0x55fdd593a530 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.505+0000 7feb5d5e4b80 10 client.26504 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.505+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fdd5687b60 con 0x55fdd593a530 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.506+0000 7feb1fe00700 1 -- 172.21.15.131:0/3885041436 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fdd57b3270 con 0x55fdd593a530 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.506+0000 7feb1fe00700 1 -- 172.21.15.131:0/3885041436 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feb3801d420 con 0x55fdd593a530 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.507+0000 7feb5d5e4b80 10 client.26504 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.507+0000 7feb5d5e4b80 10 client.26504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.507+0000 7feb5d5e4b80 20 client.26504 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.507+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fdd59503c0 0x55fdd5952810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.507+0000 7feb5d5e4b80 4 client.26504 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.507+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdd5909780 con 0x55fdd59503c0 2024-05-10T05:39:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.507+0000 7feb3e000700 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fdd59503c0 0x55fdd5952810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.508+0000 7feb3e000700 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fdd59503c0 0x55fdd5952810 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e77b96df2b1777 server_cookie=5618c9ef12810aff in_seq=0 out_seq=0 2024-05-10T05:39:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.508+0000 7feb1fe00700 10 client.26504 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:56.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.509+0000 7feb1fe00700 1 -- 172.21.15.131:0/3885041436 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fdd5909780 con 0x55fdd59503c0 2024-05-10T05:39:56.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.509+0000 7feb1fe00700 10 client.26504 handle_command_reply: tid=0 2024-05-10T05:39:56.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.509+0000 7feb5d5e4b80 1 client.26504 shutdown 2024-05-10T05:39:56.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.509+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 msgr2=0x55fdd5944910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:56.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.509+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5944910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feb38007e20 tx=0x7feb38009100 comp rx=0 tx=0).stop 2024-05-10T05:39:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.510+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 shutdown_connections 2024-05-10T05:39:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.510+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fdd59503c0 0x55fdd5952810 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.510+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd594bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.510+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5944910 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.510+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3885041436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd5944e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.510+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 >> 172.21.15.131:0/3885041436 conn(0x55fdd589ca60 msgr2=0x55fdd5939060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.510+0000 7feb1fe00700 0 client.26504 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.511+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 shutdown_connections 2024-05-10T05:39:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.511+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3885041436 wait complete. 2024-05-10T05:39:56.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.513+0000 7feb5d5e4b80 20 client.26504 trim_cache size 0 max 16384 2024-05-10T05:39:56.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:56 smithi157 ceph-mon[78427]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 83 op/s 2024-05-10T05:39:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.552+0000 7feb5d5e4b80 1 Processor -- start 2024-05-10T05:39:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.552+0000 7feb5d5e4b80 1 -- start start 2024-05-10T05:39:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.553+0000 7feb5d5e4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593a530 0x55fdd59523d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.553+0000 7feb5d5e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593b7a0 0x55fdd594bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.553+0000 7feb5d5e4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd5952910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.553+0000 7feb5d5e4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd5952e50 con 0x55fdd593c0f0 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.553+0000 7feb5d5e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd592cd60 con 0x55fdd593b7a0 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.553+0000 7feb5d5e4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd593fc60 con 0x55fdd593a530 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.553+0000 7feb3f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593a530 0x55fdd59523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.553+0000 7feb3f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593a530 0x55fdd59523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34450/0 (socket says 172.21.15.131:34450) 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.553+0000 7feb3f400700 1 -- 172.21.15.131:0/3603855567 learned_addr learned my addr 172.21.15.131:0/3603855567 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.554+0000 7feb3e000700 1 --2- 172.21.15.131:0/3603855567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593b7a0 0x55fdd594bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.554+0000 7feb3f400700 1 -- 172.21.15.131:0/3603855567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593b7a0 msgr2=0x55fdd594bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.554+0000 7feb3f400700 1 --2- 172.21.15.131:0/3603855567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593b7a0 0x55fdd594bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.554+0000 7feb3f400700 1 -- 172.21.15.131:0/3603855567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 msgr2=0x55fdd5952910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:56.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.554+0000 7feb3f400700 1 --2- 172.21.15.131:0/3603855567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd5952910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.554+0000 7feb3f400700 1 -- 172.21.15.131:0/3603855567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdd5820560 con 0x55fdd593a530 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.555+0000 7feb3e000700 1 --2- 172.21.15.131:0/3603855567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593b7a0 0x55fdd594bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.555+0000 7feb3f400700 1 --2- 172.21.15.131:0/3603855567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593a530 0x55fdd59523d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb38002230 tx=0x7feb38009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e6cbef6715f9f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.555+0000 7feb2be00700 1 -- 172.21.15.131:0/3603855567 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800e070 con 0x55fdd593a530 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.556+0000 7feb2be00700 1 -- 172.21.15.131:0/3603855567 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb38012e90 con 0x55fdd593a530 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.556+0000 7feb2be00700 1 -- 172.21.15.131:0/3603855567 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800e070 con 0x55fdd593a530 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.557+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3603855567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593a530 msgr2=0x55fdd59523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.557+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3603855567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593a530 0x55fdd59523d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb38002230 tx=0x7feb38009250 comp rx=0 tx=0).stop 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.557+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3603855567 shutdown_connections 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.557+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3603855567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd5952910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.557+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3603855567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593a530 0x55fdd59523d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.557+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/3603855567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593b7a0 0x55fdd594bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.557+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3603855567 >> 172.21.15.131:0/3603855567 conn(0x55fdd589ca60 msgr2=0x55fdd593f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.562+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3603855567 shutdown_connections 2024-05-10T05:39:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.562+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/3603855567 wait complete. 2024-05-10T05:39:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.563+0000 7feb5d5e4b80 1 -- start start 2024-05-10T05:39:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.564+0000 7feb5d5e4b80 1 Processor -- start 2024-05-10T05:39:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.564+0000 7feb5d5e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5949820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.568+0000 7feb5d5e4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd5949d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.568+0000 7feb5d5e4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd594ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.568+0000 7feb5d5e4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd594b180 con 0x55fdd593c0f0 2024-05-10T05:39:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.568+0000 7feb5d5e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd594b2f0 con 0x55fdd593a530 2024-05-10T05:39:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.568+0000 7feb5d5e4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd594b4b0 con 0x55fdd593b7a0 2024-05-10T05:39:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.568+0000 7feb5d5e4b80 10 client.0 _ll_register_callbacks cb 0x55fdd57ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.568+0000 7feb3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5949820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.568+0000 7feb3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5949820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57778/0 (socket says 172.21.15.131:57778) 2024-05-10T05:39:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.568+0000 7feb3f400700 1 -- 172.21.15.131:0/577511278 learned_addr learned my addr 172.21.15.131:0/577511278 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:56.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.569+0000 7feb3e000700 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd5949d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.571+0000 7feb3f400700 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd594ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.572+0000 7feb3f400700 1 -- 172.21.15.131:0/577511278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 msgr2=0x55fdd5949d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:56.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.572+0000 7feb3f400700 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd5949d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.572+0000 7feb3f400700 1 -- 172.21.15.131:0/577511278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 msgr2=0x55fdd594ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:56.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.572+0000 7feb3f400700 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd594ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.572+0000 7feb3f400700 1 -- 172.21.15.131:0/577511278 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdd56db3b0 con 0x55fdd593a530 2024-05-10T05:39:56.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.572+0000 7feb3e000700 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd5949d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:56.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.573+0000 7feb3f400700 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5949820 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feb3801d830 tx=0x7feb38006160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd2f2189cdcd25bb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:56.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.573+0000 7feb1fe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:39:56.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.574+0000 7feb1fe00700 1 -- 172.21.15.131:0/577511278 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800e040 con 0x55fdd593a530 2024-05-10T05:39:56.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.574+0000 7feb1fe00700 1 -- 172.21.15.131:0/577511278 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7feb3801e430 con 0x55fdd593a530 2024-05-10T05:39:56.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.574+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/577511278 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb2c0044f0 con 0x55fdd593a530 2024-05-10T05:39:56.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.574+0000 7feb1fe00700 1 -- 172.21.15.131:0/577511278 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb38016040 con 0x55fdd593a530 2024-05-10T05:39:56.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.574+0000 7feb5d5e4b80 10 client.26510 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:56.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.574+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/577511278 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fdd5687b60 con 0x55fdd593a530 2024-05-10T05:39:56.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.575+0000 7feb1fe00700 1 -- 172.21.15.131:0/577511278 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb38042020 con 0x55fdd593a530 2024-05-10T05:39:56.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.575+0000 7feb1fe00700 1 -- 172.21.15.131:0/577511278 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feb38031560 con 0x55fdd593a530 2024-05-10T05:39:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.575+0000 7feb5d5e4b80 10 client.26510 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.575+0000 7feb5d5e4b80 10 client.26510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:39:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.575+0000 7feb5d5e4b80 20 client.26510 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.575+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fdd59a7160 0x55fdd59a9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.575+0000 7feb5d5e4b80 4 client.26510 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:39:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.575+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/577511278 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fdd5852ec0 con 0x55fdd59a7160 2024-05-10T05:39:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.576+0000 7feb3e000700 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fdd59a7160 0x55fdd59a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.576+0000 7feb1fe00700 10 client.26510 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.580+0000 7feb3e000700 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fdd59a7160 0x55fdd59a9580 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936182bd8018a2c server_cookie=5f063c6f28bbeecd in_seq=0 out_seq=0 2024-05-10T05:39:56.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.584+0000 7feb1fe00700 1 -- 172.21.15.131:0/577511278 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fdd5852ec0 con 0x55fdd59a7160 2024-05-10T05:39:56.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.584+0000 7feb1fe00700 10 client.26510 handle_command_reply: tid=0 2024-05-10T05:39:56.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.584+0000 7feb5d5e4b80 1 client.26510 shutdown 2024-05-10T05:39:56.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.584+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/577511278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 msgr2=0x55fdd5949820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:56.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.584+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5949820 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feb3801d830 tx=0x7feb38006160 comp rx=0 tx=0).stop 2024-05-10T05:39:56.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.585+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/577511278 shutdown_connections 2024-05-10T05:39:56.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.585+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fdd59a7160 0x55fdd59a9580 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.585+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdd593c0f0 0x55fdd594ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.585+0000 7feb1fe00700 0 client.26510 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:39:56.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.585+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdd593a530 0x55fdd5949820 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.585+0000 7feb5d5e4b80 1 --2- 172.21.15.131:0/577511278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fdd593b7a0 0x55fdd5949d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.585+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/577511278 >> 172.21.15.131:0/577511278 conn(0x55fdd589ca60 msgr2=0x55fdd593f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:56.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.591+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/577511278 shutdown_connections 2024-05-10T05:39:56.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.593+0000 7feb5d5e4b80 1 -- 172.21.15.131:0/577511278 wait complete. 2024-05-10T05:39:56.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.596+0000 7feb5d5e4b80 20 client.26510 trim_cache size 0 max 16384 2024-05-10T05:39:56.606 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:39:56.607 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:39:56.607 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:39:56.607 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:39:56.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.606+0000 7feb3fe00700 1 -- 172.21.15.131:0/3967659501 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feb240a12d0 msgr2=0x7feb240a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:56.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.606+0000 7feb3fe00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feb240a12d0 0x7feb240a37e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb30020fd0 tx=0x7feb30009ba0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.606+0000 7feb3fe00700 1 -- 172.21.15.131:0/3967659501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb40108340 msgr2=0x7feb4019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:56.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.606+0000 7feb3fe00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb40108340 0x7feb4019e400 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feb30009d70 tx=0x7feb30003c00 comp rx=0 tx=0).stop 2024-05-10T05:39:56.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.608+0000 7feb3fe00700 1 -- 172.21.15.131:0/3967659501 shutdown_connections 2024-05-10T05:39:56.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.608+0000 7feb3fe00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feb240a12d0 0x7feb240a37e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.608+0000 7feb3fe00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb401097a0 0x7feb40079470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.608+0000 7feb3fe00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb40108340 0x7feb4019e400 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.608+0000 7feb3fe00700 1 --2- 172.21.15.131:0/3967659501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb40108c90 0x7feb4019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:56.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.608+0000 7feb3fe00700 1 -- 172.21.15.131:0/3967659501 >> 172.21.15.131:0/3967659501 conn(0x7feb4006e800 msgr2=0x7feb4010c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:56.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.609+0000 7feb3fe00700 1 -- 172.21.15.131:0/3967659501 shutdown_connections 2024-05-10T05:39:56.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:56.609+0000 7feb3fe00700 1 -- 172.21.15.131:0/3967659501 wait complete. 2024-05-10T05:39:56.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:56 smithi142 ceph-mon[77667]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 83 op/s 2024-05-10T05:39:56.719 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:56 smithi131 ceph-mon[37264]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 83 op/s 2024-05-10T05:39:56.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:39:56.736 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:39:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.328+0000 7fe422800700 1 -- 172.21.15.131:0/2380425198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe41c1034e0 msgr2=0x7fe41c109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.328+0000 7fe422800700 1 --2- 172.21.15.131:0/2380425198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe41c1034e0 0x7fe41c109d90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe410009990 tx=0x7fe410009ca0 comp rx=0 tx=0).stop 2024-05-10T05:39:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.330+0000 7fe422800700 1 -- 172.21.15.131:0/2380425198 shutdown_connections 2024-05-10T05:39:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.330+0000 7fe422800700 1 --2- 172.21.15.131:0/2380425198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe41c1034e0 0x7fe41c109d90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.330+0000 7fe422800700 1 --2- 172.21.15.131:0/2380425198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c101a20 0x7fe41c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.330+0000 7fe422800700 1 --2- 172.21.15.131:0/2380425198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c102b40 0x7fe41c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:57.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.330+0000 7fe422800700 1 -- 172.21.15.131:0/2380425198 >> 172.21.15.131:0/2380425198 conn(0x7fe41c0fd050 msgr2=0x7fe41c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.331+0000 7fe422800700 1 -- 172.21.15.131:0/2380425198 shutdown_connections 2024-05-10T05:39:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.331+0000 7fe422800700 1 -- 172.21.15.131:0/2380425198 wait complete. 2024-05-10T05:39:57.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.336+0000 7fe422800700 1 Processor -- start 2024-05-10T05:39:57.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.336+0000 7fe422800700 1 -- start start 2024-05-10T05:39:57.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.336+0000 7fe422800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c101a20 0x7fe41c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:57.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.336+0000 7fe422800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe41c102b40 0x7fe41c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:57.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.337+0000 7fe422800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c1034e0 0x7fe41c1a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:57.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.337+0000 7fe421400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c101a20 0x7fe41c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:57.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.338+0000 7fe421400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c101a20 0x7fe41c19e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34478/0 (socket says 172.21.15.131:34478) 2024-05-10T05:39:57.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.338+0000 7fe421400700 1 -- 172.21.15.131:0/2972777125 learned_addr learned my addr 172.21.15.131:0/2972777125 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:57.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.338+0000 7fe420a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe41c102b40 0x7fe41c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:57.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.338+0000 7fe421e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c1034e0 0x7fe41c1a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:57.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.338+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41c1a5c70 con 0x7fe41c102b40 2024-05-10T05:39:57.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.338+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41c1a5db0 con 0x7fe41c1034e0 2024-05-10T05:39:57.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.338+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41c1a5f40 con 0x7fe41c101a20 2024-05-10T05:39:57.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.339+0000 7fe421e00700 1 -- 172.21.15.131:0/2972777125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c101a20 msgr2=0x7fe41c19e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.340+0000 7fe421e00700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c101a20 0x7fe41c19e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.340+0000 7fe421e00700 1 -- 172.21.15.131:0/2972777125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe41c102b40 msgr2=0x7fe41c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.340+0000 7fe421e00700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe41c102b40 0x7fe41c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.341+0000 7fe421e00700 1 -- 172.21.15.131:0/2972777125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100095f0 con 0x7fe41c1034e0 2024-05-10T05:39:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.343+0000 7fe421e00700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c1034e0 0x7fe41c1a5730 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe410009990 tx=0x7fe410003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.344+0000 7fe41a000700 1 -- 172.21.15.131:0/2972777125 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41001d070 con 0x7fe41c1034e0 2024-05-10T05:39:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.345+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41c1a61c0 con 0x7fe41c1034e0 2024-05-10T05:39:57.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.345+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41c1a6640 con 0x7fe41c1034e0 2024-05-10T05:39:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.348+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41c068110 con 0x7fe41c1034e0 2024-05-10T05:39:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.348+0000 7fe41a000700 1 -- 172.21.15.131:0/2972777125 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe410003e60 con 0x7fe41c1034e0 2024-05-10T05:39:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.348+0000 7fe41a000700 1 -- 172.21.15.131:0/2972777125 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41000fe90 con 0x7fe41c1034e0 2024-05-10T05:39:57.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.355+0000 7fe41a000700 1 -- 172.21.15.131:0/2972777125 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe41000f5f0 con 0x7fe41c1034e0 2024-05-10T05:39:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.358+0000 7fe41a000700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe4040a1210 0x7fe4040a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.358+0000 7fe41a000700 1 -- 172.21.15.131:0/2972777125 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe4100cdde0 con 0x7fe41c1034e0 2024-05-10T05:39:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.359+0000 7fe421400700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe4040a1210 0x7fe4040a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.360+0000 7fe421400700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe4040a1210 0x7fe4040a3720 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe41400e350 tx=0x7fe414009040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:57.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.386+0000 7fe41a000700 1 -- 172.21.15.131:0/2972777125 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe41005ed60 con 0x7fe41c1034e0 2024-05-10T05:39:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.917+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe41c075050 con 0x7fe41c1034e0 2024-05-10T05:39:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.918+0000 7fe41a000700 1 -- 172.21.15.131:0/2972777125 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fe410097600 con 0x7fe41c1034e0 2024-05-10T05:39:57.918 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:39:57.918 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:39:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe4040a1210 msgr2=0x7fe4040a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe4040a1210 0x7fe4040a3720 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe41400e350 tx=0x7fe414009040 comp rx=0 tx=0).stop 2024-05-10T05:39:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c1034e0 msgr2=0x7fe41c1a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c1034e0 0x7fe41c1a5730 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe410009990 tx=0x7fe410003c00 comp rx=0 tx=0).stop 2024-05-10T05:39:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 shutdown_connections 2024-05-10T05:39:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe4040a1210 0x7fe4040a3720 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe41c1034e0 0x7fe41c1a5730 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c101a20 0x7fe41c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 --2- 172.21.15.131:0/2972777125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe41c102b40 0x7fe41c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 >> 172.21.15.131:0/2972777125 conn(0x7fe41c0fd050 msgr2=0x7fe41c10dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 shutdown_connections 2024-05-10T05:39:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:57.921+0000 7fe422800700 1 -- 172.21.15.131:0/2972777125 wait complete. 2024-05-10T05:39:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:39:57.959 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:39:58.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:58 smithi157 ceph-mon[78427]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 69 op/s 2024-05-10T05:39:58.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:39:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2972777125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:39:58.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.632+0000 7f5d93000700 1 -- 172.21.15.131:0/84137810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c108750 msgr2=0x7f5d8c108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:58.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.632+0000 7f5d93000700 1 --2- 172.21.15.131:0/84137810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c108750 0x7f5d8c108b60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009990 tx=0x7f5d78009ca0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.633+0000 7f5d93000700 1 -- 172.21.15.131:0/84137810 shutdown_connections 2024-05-10T05:39:58.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.633+0000 7f5d93000700 1 --2- 172.21.15.131:0/84137810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d8c109b60 0x7f5d8c108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.633+0000 7f5d93000700 1 --2- 172.21.15.131:0/84137810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c077460 0x7f5d8c079880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.633+0000 7f5d93000700 1 --2- 172.21.15.131:0/84137810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c108750 0x7f5d8c108b60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.633+0000 7f5d93000700 1 -- 172.21.15.131:0/84137810 >> 172.21.15.131:0/84137810 conn(0x7f5d8c06ea00 msgr2=0x7f5d8c070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:58.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.633+0000 7f5d93000700 1 -- 172.21.15.131:0/84137810 shutdown_connections 2024-05-10T05:39:58.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.633+0000 7f5d93000700 1 -- 172.21.15.131:0/84137810 wait complete. 2024-05-10T05:39:58.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.636+0000 7f5d93000700 1 Processor -- start 2024-05-10T05:39:58.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.637+0000 7f5d93000700 1 -- start start 2024-05-10T05:39:58.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.638+0000 7f5d93000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c077460 0x7f5d8c083470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.639+0000 7f5d93000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c108750 0x7f5d8c081b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.639+0000 7f5d8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c077460 0x7f5d8c083470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.639+0000 7f5d8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c077460 0x7f5d8c083470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34492/0 (socket says 172.21.15.131:34492) 2024-05-10T05:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.639+0000 7f5d93000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d8c109b60 0x7f5d8c082190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.640+0000 7f5d93000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c082890 con 0x7f5d8c109b60 2024-05-10T05:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.640+0000 7f5d93000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1bfa90 con 0x7f5d8c108750 2024-05-10T05:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.640+0000 7f5d93000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1bfbd0 con 0x7f5d8c077460 2024-05-10T05:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.640+0000 7f5d8be00700 1 -- 172.21.15.131:0/2600528193 learned_addr learned my addr 172.21.15.131:0/2600528193 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.640+0000 7f5d8b400700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c108750 0x7f5d8c081b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.640+0000 7f5d91000700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d8c109b60 0x7f5d8c082190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.641+0000 7f5d8be00700 1 -- 172.21.15.131:0/2600528193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c108750 msgr2=0x7f5d8c081b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.641+0000 7f5d8be00700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c108750 0x7f5d8c081b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.641+0000 7f5d8be00700 1 -- 172.21.15.131:0/2600528193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d8c109b60 msgr2=0x7f5d8c082190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.641+0000 7f5d8be00700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d8c109b60 0x7f5d8c082190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.641+0000 7f5d8be00700 1 -- 172.21.15.131:0/2600528193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d780095f0 con 0x7f5d8c077460 2024-05-10T05:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.641+0000 7f5d8b400700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c108750 0x7f5d8c081b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:39:58.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:58 smithi142 ceph-mon[77667]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 69 op/s 2024-05-10T05:39:58.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:39:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2972777125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:39:58.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.647+0000 7f5d8be00700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c077460 0x7f5d8c083470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00e970 tx=0x7f5d7c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:58.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.648+0000 7f5d88c00700 1 -- 172.21.15.131:0/2600528193 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c009790 con 0x7f5d8c077460 2024-05-10T05:39:58.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.648+0000 7f5d88c00700 1 -- 172.21.15.131:0/2600528193 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d7c004cf0 con 0x7f5d8c077460 2024-05-10T05:39:58.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.648+0000 7f5d88c00700 1 -- 172.21.15.131:0/2600528193 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c005680 con 0x7f5d8c077460 2024-05-10T05:39:58.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.648+0000 7f5d93000700 1 -- 172.21.15.131:0/2600528193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c1bfe30 con 0x7f5d8c077460 2024-05-10T05:39:58.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.650+0000 7f5d93000700 1 -- 172.21.15.131:0/2600528193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c1c0200 con 0x7f5d8c077460 2024-05-10T05:39:58.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.673+0000 7f5d88c00700 1 -- 172.21.15.131:0/2600528193 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5d7c004830 con 0x7f5d8c077460 2024-05-10T05:39:58.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.681+0000 7f5d88c00700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d700a12c0 0x7f5d700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.681+0000 7f5d88c00700 1 -- 172.21.15.131:0/2600528193 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5d7c014070 con 0x7f5d8c077460 2024-05-10T05:39:58.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.682+0000 7f5d8b400700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d700a12c0 0x7f5d700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.686+0000 7f5d8b400700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d700a12c0 0x7f5d700a37d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800b2e0 tx=0x7f5d7801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:58.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.686+0000 7f5daaeceb80 1 Processor -- start 2024-05-10T05:39:58.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.686+0000 7f5daaeceb80 1 -- start start 2024-05-10T05:39:58.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.686+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a666a0 0x5558b4a66af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a67570 0x5558b4a67980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 0x5558b4a6e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5daaeceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b498d150 con 0x5558b4a67570 2024-05-10T05:39:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5daaeceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b498ab10 con 0x5558b4a67ec0 2024-05-10T05:39:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5daaeceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b4986400 con 0x5558b4a666a0 2024-05-10T05:39:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5d8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a666a0 0x5558b4a66af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5d8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a666a0 0x5558b4a66af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34494/0 (socket says 172.21.15.131:34494) 2024-05-10T05:39:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5d8be00700 1 -- 172.21.15.131:0/3941829148 learned_addr learned my addr 172.21.15.131:0/3941829148 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5d91000700 1 --2- 172.21.15.131:0/3941829148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 0x5558b4a6e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5d8b400700 1 --2- 172.21.15.131:0/3941829148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a67570 0x5558b4a67980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5d8be00700 1 -- 172.21.15.131:0/3941829148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 msgr2=0x5558b4a6e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5d8be00700 1 --2- 172.21.15.131:0/3941829148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 0x5558b4a6e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.687+0000 7f5d8be00700 1 -- 172.21.15.131:0/3941829148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a67570 msgr2=0x5558b4a67980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.688+0000 7f5d8be00700 1 --2- 172.21.15.131:0/3941829148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a67570 0x5558b4a67980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.688+0000 7f5d8be00700 1 -- 172.21.15.131:0/3941829148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558b48073b0 con 0x5558b4a666a0 2024-05-10T05:39:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.688+0000 7f5d8b400700 1 --2- 172.21.15.131:0/3941829148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a67570 0x5558b4a67980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:39:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.688+0000 7f5d8be00700 1 --2- 172.21.15.131:0/3941829148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a666a0 0x5558b4a66af0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c099ee0 tx=0x7f5d7c099b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e5a354baf5b15d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.688+0000 7f5d81600700 1 -- 172.21.15.131:0/3941829148 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b1070 con 0x5558b4a666a0 2024-05-10T05:39:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.688+0000 7f5d81600700 1 -- 172.21.15.131:0/3941829148 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d7c09a650 con 0x5558b4a666a0 2024-05-10T05:39:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.688+0000 7f5d81600700 1 -- 172.21.15.131:0/3941829148 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c004e60 con 0x5558b4a666a0 2024-05-10T05:39:58.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.691+0000 7f5daaeceb80 1 -- 172.21.15.131:0/3941829148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a666a0 msgr2=0x5558b4a66af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.695+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/3941829148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a666a0 0x5558b4a66af0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c099ee0 tx=0x7f5d7c099b90 comp rx=0 tx=0).stop 2024-05-10T05:39:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.695+0000 7f5daaeceb80 1 -- 172.21.15.131:0/3941829148 shutdown_connections 2024-05-10T05:39:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.695+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/3941829148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 0x5558b4a6e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.695+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/3941829148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a666a0 0x5558b4a66af0 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c099ee0 tx=0x7f5d7c099b90 comp rx=0 tx=0).stop 2024-05-10T05:39:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.695+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/3941829148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a67570 0x5558b4a67980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.695+0000 7f5daaeceb80 1 -- 172.21.15.131:0/3941829148 >> 172.21.15.131:0/3941829148 conn(0x5558b49c8a60 msgr2=0x5558b4a60fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:58.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.696+0000 7f5daaeceb80 1 -- 172.21.15.131:0/3941829148 shutdown_connections 2024-05-10T05:39:58.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.696+0000 7f5daaeceb80 1 -- 172.21.15.131:0/3941829148 wait complete. 2024-05-10T05:39:58.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.697+0000 7f5daaeceb80 1 -- start start 2024-05-10T05:39:58.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.698+0000 7f5daaeceb80 1 Processor -- start 2024-05-10T05:39:58.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.703+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67570 0x5558b4a707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.703+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67ec0 0x5558b4a70dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.703+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 0x5558b4a78080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.704+0000 7f5d8be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67570 0x5558b4a707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.704+0000 7f5d8be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67570 0x5558b4a707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57822/0 (socket says 172.21.15.131:57822) 2024-05-10T05:39:58.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.704+0000 7f5d8be00700 1 -- 172.21.15.131:0/2724269470 learned_addr learned my addr 172.21.15.131:0/2724269470 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:58.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.704+0000 7f5d91000700 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 0x5558b4a78080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.704+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b4922560 con 0x5558b4a75c60 2024-05-10T05:39:58.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.704+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b4922200 con 0x5558b4a67570 2024-05-10T05:39:58.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.704+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b48096a0 con 0x5558b4a67ec0 2024-05-10T05:39:58.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.704+0000 7f5daaeceb80 10 client.0 _ll_register_callbacks cb 0x5558b492bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:58.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.707+0000 7f5d8be00700 1 -- 172.21.15.131:0/2724269470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67ec0 msgr2=0x5558b4a70dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.705+0000 7f5d8b400700 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67ec0 0x5558b4a70dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.707+0000 7f5d8be00700 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67ec0 0x5558b4a70dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.707+0000 7f5d8be00700 1 -- 172.21.15.131:0/2724269470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 msgr2=0x5558b4a78080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.707+0000 7f5d8be00700 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 0x5558b4a78080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.707+0000 7f5d8be00700 1 -- 172.21.15.131:0/2724269470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c1c0200 con 0x5558b4a67570 2024-05-10T05:39:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.707+0000 7f5d91000700 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 0x5558b4a78080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:39:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.708+0000 7f5d8be00700 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67570 0x5558b4a707d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00c5b0 tx=0x7f5d7c09a7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d5fd971a9fba7be server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.708+0000 7f5d76000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:39:58.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.708+0000 7f5d8b400700 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67ec0 0x5558b4a70dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:39:58.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.708+0000 7f5d76000700 1 -- 172.21.15.131:0/2724269470 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b1070 con 0x5558b4a67570 2024-05-10T05:39:58.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.708+0000 7f5d76000700 1 -- 172.21.15.131:0/2724269470 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5d7c09dd80 con 0x5558b4a67570 2024-05-10T05:39:58.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.711+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558b48df270 con 0x5558b4a67570 2024-05-10T05:39:58.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:58 smithi131 ceph-mon[37264]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 69 op/s 2024-05-10T05:39:58.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:39:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2972777125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:39:58.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.711+0000 7f5d76000700 1 -- 172.21.15.131:0/2724269470 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7c0b8250 con 0x5558b4a67570 2024-05-10T05:39:58.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.712+0000 7f5daaeceb80 10 client.26525 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:58.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.712+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5558b47b3b60 con 0x5558b4a67570 2024-05-10T05:39:58.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.712+0000 7f5d76000700 1 -- 172.21.15.131:0/2724269470 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558b48df270 con 0x5558b4a67570 2024-05-10T05:39:58.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.719+0000 7f5d76000700 1 -- 172.21.15.131:0/2724269470 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5d7c0a1f40 con 0x5558b4a67570 2024-05-10T05:39:58.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.720+0000 7f5daaeceb80 10 client.26525 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:58.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.720+0000 7f5daaeceb80 10 client.26525 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:39:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.720+0000 7f5daaeceb80 20 client.26525 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.720+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5558b4a7c500 0x5558b4a7e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.720+0000 7f5daaeceb80 4 client.26525 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.720+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558b4a35780 con 0x5558b4a7c500 2024-05-10T05:39:58.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.721+0000 7f5d8b400700 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5558b4a7c500 0x5558b4a7e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.730+0000 7f5d8b400700 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5558b4a7c500 0x5558b4a7e920 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=357383120f76d354 server_cookie=d6bb2f77db698d70 in_seq=0 out_seq=0 2024-05-10T05:39:58.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.730+0000 7f5d76000700 10 client.26525 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:39:58.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.734+0000 7f5d76000700 1 -- 172.21.15.131:0/2724269470 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558b4a35780 con 0x5558b4a7c500 2024-05-10T05:39:58.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.734+0000 7f5d76000700 10 client.26525 handle_command_reply: tid=0 2024-05-10T05:39:58.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.735+0000 7f5daaeceb80 1 client.26525 shutdown 2024-05-10T05:39:58.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.735+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67570 msgr2=0x5558b4a707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:58.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.735+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67570 0x5558b4a707d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00c5b0 tx=0x7f5d7c09a7c0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.735+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 shutdown_connections 2024-05-10T05:39:58.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.735+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5558b4a7c500 0x5558b4a7e920 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.735+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 0x5558b4a78080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.736+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67570 0x5558b4a707d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.736+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/2724269470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67ec0 0x5558b4a70dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.736+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 >> 172.21.15.131:0/2724269470 conn(0x5558b49c8a60 msgr2=0x5558b4a60fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:58.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.736+0000 7f5d76000700 0 client.26525 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:39:58.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.736+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 shutdown_connections 2024-05-10T05:39:58.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.736+0000 7f5daaeceb80 1 -- 172.21.15.131:0/2724269470 wait complete. 2024-05-10T05:39:58.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.744+0000 7f5daaeceb80 20 client.26525 trim_cache size 0 max 16384 2024-05-10T05:39:58.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.794+0000 7f5daaeceb80 1 Processor -- start 2024-05-10T05:39:58.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.794+0000 7f5daaeceb80 1 -- start start 2024-05-10T05:39:58.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.794+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a67570 0x5558b4a78080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.795+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 0x5558b4a61e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.795+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a75c60 0x5558b4a78760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.795+0000 7f5daaeceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b4a5dd30 con 0x5558b4a67570 2024-05-10T05:39:58.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.795+0000 7f5daaeceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b4a79230 con 0x5558b4a67ec0 2024-05-10T05:39:58.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.795+0000 7f5daaeceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b4a66d00 con 0x5558b4a75c60 2024-05-10T05:39:58.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.795+0000 7f5d91000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a75c60 0x5558b4a78760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.795+0000 7f5d91000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a75c60 0x5558b4a78760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34506/0 (socket says 172.21.15.131:34506) 2024-05-10T05:39:58.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.795+0000 7f5d91000700 1 -- 172.21.15.131:0/149012140 learned_addr learned my addr 172.21.15.131:0/149012140 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:58.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.796+0000 7f5d91000700 1 -- 172.21.15.131:0/149012140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 msgr2=0x5558b4a61e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:39:58.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.796+0000 7f5d91000700 1 --2- 172.21.15.131:0/149012140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 0x5558b4a61e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.796+0000 7f5d91000700 1 -- 172.21.15.131:0/149012140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a67570 msgr2=0x5558b4a78080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:58.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.796+0000 7f5d91000700 1 --2- 172.21.15.131:0/149012140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a67570 0x5558b4a78080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.796+0000 7f5d91000700 1 -- 172.21.15.131:0/149012140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558b494c560 con 0x5558b4a75c60 2024-05-10T05:39:58.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.797+0000 7f5d91000700 1 --2- 172.21.15.131:0/149012140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a75c60 0x5558b4a78760 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5d84007910 tx=0x7f5d8400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ae70afca180eb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:58.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.797+0000 7f5d82000700 1 -- 172.21.15.131:0/149012140 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400e040 con 0x5558b4a75c60 2024-05-10T05:39:58.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.797+0000 7f5d82000700 1 -- 172.21.15.131:0/149012140 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d8400f040 con 0x5558b4a75c60 2024-05-10T05:39:58.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.799+0000 7f5daaeceb80 1 -- 172.21.15.131:0/149012140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a75c60 msgr2=0x5558b4a78760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:58.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.799+0000 7f5d82000700 1 -- 172.21.15.131:0/149012140 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400e040 con 0x5558b4a75c60 2024-05-10T05:39:58.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.799+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/149012140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a75c60 0x5558b4a78760 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5d84007910 tx=0x7f5d8400b040 comp rx=0 tx=0).stop 2024-05-10T05:39:58.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.799+0000 7f5daaeceb80 1 -- 172.21.15.131:0/149012140 shutdown_connections 2024-05-10T05:39:58.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.799+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/149012140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a67570 0x5558b4a78080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.799+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/149012140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a75c60 0x5558b4a78760 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.799+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/149012140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 0x5558b4a61e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.799+0000 7f5daaeceb80 1 -- 172.21.15.131:0/149012140 >> 172.21.15.131:0/149012140 conn(0x5558b49c8a60 msgr2=0x5558b4a5d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:58.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.800+0000 7f5daaeceb80 1 -- 172.21.15.131:0/149012140 shutdown_connections 2024-05-10T05:39:58.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.800+0000 7f5daaeceb80 1 -- 172.21.15.131:0/149012140 wait complete. 2024-05-10T05:39:58.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.802+0000 7f5daaeceb80 1 -- start start 2024-05-10T05:39:58.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.802+0000 7f5daaeceb80 1 Processor -- start 2024-05-10T05:39:58.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.807+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67570 0x5558b4a8c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.807+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 0x5558b4a8af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.807+0000 7f5daaeceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 0x5558b4a8b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.808+0000 7f5daaeceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b4a8d0c0 con 0x5558b4a75c60 2024-05-10T05:39:58.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.808+0000 7f5daaeceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b4a8b9e0 con 0x5558b4a67ec0 2024-05-10T05:39:58.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.808+0000 7f5daaeceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b4a8bba0 con 0x5558b4a67570 2024-05-10T05:39:58.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.808+0000 7f5daaeceb80 10 client.0 _ll_register_callbacks cb 0x5558b492bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:58.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.808+0000 7f5d8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67570 0x5558b4a8c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.808+0000 7f5d8be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67570 0x5558b4a8c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34514/0 (socket says 172.21.15.131:34514) 2024-05-10T05:39:58.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.808+0000 7f5d8be00700 1 -- 172.21.15.131:0/1912451584 learned_addr learned my addr 172.21.15.131:0/1912451584 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:58.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.808+0000 7f5d91000700 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 0x5558b4a8b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.809+0000 7f5d8be00700 1 -- 172.21.15.131:0/1912451584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 msgr2=0x5558b4a8af60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:39:58.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.809+0000 7f5d8be00700 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 0x5558b4a8af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.809+0000 7f5d8be00700 1 -- 172.21.15.131:0/1912451584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 msgr2=0x5558b4a8b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:58.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.809+0000 7f5d8be00700 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 0x5558b4a8b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.809+0000 7f5d8be00700 1 -- 172.21.15.131:0/1912451584 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d84007620 con 0x5558b4a67570 2024-05-10T05:39:58.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.813+0000 7f5d8be00700 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67570 0x5558b4a8c8f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c09a110 tx=0x7f5d7c09a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eac2f1301dc90ad1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:58.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.814+0000 7f5d76000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:39:58.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.814+0000 7f5d76000700 1 -- 172.21.15.131:0/1912451584 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0998b0 con 0x5558b4a67570 2024-05-10T05:39:58.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.815+0000 7f5daaeceb80 1 -- 172.21.15.131:0/1912451584 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d7c09a650 con 0x5558b4a67570 2024-05-10T05:39:58.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.815+0000 7f5d76000700 1 -- 172.21.15.131:0/1912451584 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5d7c0a23d0 con 0x5558b4a67570 2024-05-10T05:39:58.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.816+0000 7f5d76000700 1 -- 172.21.15.131:0/1912451584 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7c09dd80 con 0x5558b4a67570 2024-05-10T05:39:58.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.816+0000 7f5daaeceb80 10 client.26554 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:58.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.816+0000 7f5daaeceb80 1 -- 172.21.15.131:0/1912451584 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5558b480bcd0 con 0x5558b4a67570 2024-05-10T05:39:58.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.816+0000 7f5d76000700 1 -- 172.21.15.131:0/1912451584 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d7c0a1f40 con 0x5558b4a67570 2024-05-10T05:39:58.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.817+0000 7f5d76000700 1 -- 172.21.15.131:0/1912451584 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5d7c0994d0 con 0x5558b4a67570 2024-05-10T05:39:58.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.817+0000 7f5daaeceb80 10 client.26554 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:58.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.817+0000 7f5daaeceb80 10 client.26554 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:39:58.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.817+0000 7f5daaeceb80 20 client.26554 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:58.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.817+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5558b4ad23d0 0x5558b4ad47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:58.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.818+0000 7f5daaeceb80 4 client.26554 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:39:58.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.818+0000 7f5daaeceb80 1 -- 172.21.15.131:0/1912451584 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f5d7c09a650 con 0x5558b4ad23d0 2024-05-10T05:39:58.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.818+0000 7f5d8b400700 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5558b4ad23d0 0x5558b4ad47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:58.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.822+0000 7f5d76000700 10 client.26554 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:39:58.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.825+0000 7f5d8b400700 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5558b4ad23d0 0x5558b4ad47f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25f334bb6c49daed server_cookie=fd2ec2219cdf59ab in_seq=0 out_seq=0 2024-05-10T05:39:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.827+0000 7f5d76000700 1 -- 172.21.15.131:0/1912451584 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f5d7c09a650 con 0x5558b4ad23d0 2024-05-10T05:39:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.828+0000 7f5d76000700 10 client.26554 handle_command_reply: tid=0 2024-05-10T05:39:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.828+0000 7f5daaeceb80 1 client.26554 shutdown 2024-05-10T05:39:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.830+0000 7f5daaeceb80 1 -- 172.21.15.131:0/1912451584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67570 msgr2=0x5558b4a8c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.830+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67570 0x5558b4a8c8f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c09a110 tx=0x7f5d7c09a000 comp rx=0 tx=0).stop 2024-05-10T05:39:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.830+0000 7f5daaeceb80 1 -- 172.21.15.131:0/1912451584 shutdown_connections 2024-05-10T05:39:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.830+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5558b4ad23d0 0x5558b4ad47f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.830+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5558b4a75c60 0x5558b4a8b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.830+0000 7f5d76000700 0 client.26554 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:39:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.830+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b4a67570 0x5558b4a8c8f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.830+0000 7f5daaeceb80 1 --2- 172.21.15.131:0/1912451584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558b4a67ec0 0x5558b4a8af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.830+0000 7f5daaeceb80 1 -- 172.21.15.131:0/1912451584 >> 172.21.15.131:0/1912451584 conn(0x5558b49c8a60 msgr2=0x5558b4a5d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:58.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.831+0000 7f5daaeceb80 1 -- 172.21.15.131:0/1912451584 shutdown_connections 2024-05-10T05:39:58.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.831+0000 7f5daaeceb80 1 -- 172.21.15.131:0/1912451584 wait complete. 2024-05-10T05:39:58.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.831+0000 7f5daaeceb80 20 client.26554 trim_cache size 0 max 16384 2024-05-10T05:39:58.833 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:39:58.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.833+0000 7f5d93000700 1 -- 172.21.15.131:0/2600528193 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d700a12c0 msgr2=0x7f5d700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:58.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.833+0000 7f5d93000700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d700a12c0 0x7f5d700a37d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800b2e0 tx=0x7f5d7801a040 comp rx=0 tx=0).stop 2024-05-10T05:39:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.833+0000 7f5d93000700 1 -- 172.21.15.131:0/2600528193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c077460 msgr2=0x7f5d8c083470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.833+0000 7f5d93000700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c077460 0x7f5d8c083470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00e970 tx=0x7f5d7c00ed30 comp rx=0 tx=0).stop 2024-05-10T05:39:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.833+0000 7f5d93000700 1 -- 172.21.15.131:0/2600528193 shutdown_connections 2024-05-10T05:39:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.833+0000 7f5d93000700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5d700a12c0 0x7f5d700a37d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.833+0000 7f5d93000700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d8c109b60 0x7f5d8c082190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.833+0000 7f5d93000700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d8c077460 0x7f5d8c083470 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.833+0000 7f5d93000700 1 --2- 172.21.15.131:0/2600528193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c108750 0x7f5d8c081b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.833+0000 7f5d93000700 1 -- 172.21.15.131:0/2600528193 >> 172.21.15.131:0/2600528193 conn(0x7f5d8c06ea00 msgr2=0x7f5d8c083aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.834+0000 7f5d93000700 1 -- 172.21.15.131:0/2600528193 shutdown_connections 2024-05-10T05:39:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:58.834+0000 7f5d93000700 1 -- 172.21.15.131:0/2600528193 wait complete. 2024-05-10T05:39:58.902 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:39:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.576+0000 7f5715c00700 1 -- 172.21.15.131:0/373680776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5710105790 msgr2=0x7f5710105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.576+0000 7f5715c00700 1 --2- 172.21.15.131:0/373680776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5710105790 0x7f5710105ba0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f570800b0a0 tx=0x7f570800b3b0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.576+0000 7f5715c00700 1 -- 172.21.15.131:0/373680776 shutdown_connections 2024-05-10T05:39:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.576+0000 7f5715c00700 1 --2- 172.21.15.131:0/373680776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57101004d0 0x7f57101028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.576+0000 7f5715c00700 1 --2- 172.21.15.131:0/373680776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5710105790 0x7f5710105ba0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.576+0000 7f5715c00700 1 --2- 172.21.15.131:0/373680776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5710102e30 0x7f5710105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.576+0000 7f5715c00700 1 -- 172.21.15.131:0/373680776 >> 172.21.15.131:0/373680776 conn(0x7f57100fc0e0 msgr2=0x7f57100fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.577+0000 7f5715c00700 1 -- 172.21.15.131:0/373680776 shutdown_connections 2024-05-10T05:39:59.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.577+0000 7f5715c00700 1 -- 172.21.15.131:0/373680776 wait complete. 2024-05-10T05:39:59.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.578+0000 7f5715c00700 1 Processor -- start 2024-05-10T05:39:59.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.579+0000 7f5715c00700 1 -- start start 2024-05-10T05:39:59.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.579+0000 7f5715c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57101004d0 0x7f571019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.579+0000 7f5715c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5710105790 0x7f571019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.579+0000 7f5715c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57101a3ae0 0x7f57101a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.579+0000 7f5715c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57101a6510 con 0x7f5710105790 2024-05-10T05:39:59.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.579+0000 7f5715c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57101a6680 con 0x7f57101004d0 2024-05-10T05:39:59.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.579+0000 7f5715c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57101a6840 con 0x7f57101a3ae0 2024-05-10T05:39:59.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57101004d0 0x7f571019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57101004d0 0x7f571019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57848/0 (socket says 172.21.15.131:57848) 2024-05-10T05:39:59.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570ea00700 1 -- 172.21.15.131:0/745439124 learned_addr learned my addr 172.21.15.131:0/745439124 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:59.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570f400700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57101a3ae0 0x7f57101a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570e000700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5710105790 0x7f571019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570ea00700 1 -- 172.21.15.131:0/745439124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57101a3ae0 msgr2=0x7f57101a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:59.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570ea00700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57101a3ae0 0x7f57101a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570ea00700 1 -- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5710105790 msgr2=0x7f571019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:59.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570ea00700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5710105790 0x7f571019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570ea00700 1 -- 172.21.15.131:0/745439124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5708009d00 con 0x7f57101004d0 2024-05-10T05:39:59.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.580+0000 7f570e000700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5710105790 0x7f571019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:39:59.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.581+0000 7f570ea00700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57101004d0 0x7f571019e620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f570000d950 tx=0x7f570000dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:59.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.581+0000 7f5707400700 1 -- 172.21.15.131:0/745439124 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570000bb60 con 0x7f57101004d0 2024-05-10T05:39:59.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.581+0000 7f5707400700 1 -- 172.21.15.131:0/745439124 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5700007620 con 0x7f57101004d0 2024-05-10T05:39:59.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.581+0000 7f5707400700 1 -- 172.21.15.131:0/745439124 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700005560 con 0x7f57101004d0 2024-05-10T05:39:59.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.582+0000 7f5715c00700 1 -- 172.21.15.131:0/745439124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57101a6b20 con 0x7f57101004d0 2024-05-10T05:39:59.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.582+0000 7f5715c00700 1 -- 172.21.15.131:0/745439124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57101a6fe0 con 0x7f57101004d0 2024-05-10T05:39:59.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.588+0000 7f5707400700 1 -- 172.21.15.131:0/745439124 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5700004750 con 0x7f57101004d0 2024-05-10T05:39:59.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.590+0000 7f572dadcb80 1 Processor -- start 2024-05-10T05:39:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.591+0000 7f572dadcb80 1 -- start start 2024-05-10T05:39:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.592+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae26a0 0x55ed22ae2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.592+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae3570 0x55ed22ae3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.592+0000 7f5707400700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56f40a1240 0x7f56f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.592+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 0x55ed22aea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.592+0000 7f572dadcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed22a09150 con 0x55ed22ae3ec0 2024-05-10T05:39:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.592+0000 7f572dadcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed22a06b10 con 0x55ed22ae26a0 2024-05-10T05:39:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.592+0000 7f572dadcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed22a02400 con 0x55ed22ae3570 2024-05-10T05:39:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.593+0000 7f5707400700 1 -- 172.21.15.131:0/745439124 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f57000cc540 con 0x7f57101004d0 2024-05-10T05:39:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.593+0000 7f570ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae3570 0x55ed22ae3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.593+0000 7f570ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae3570 0x55ed22ae3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34546/0 (socket says 172.21.15.131:34546) 2024-05-10T05:39:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.593+0000 7f570ea00700 1 -- 172.21.15.131:0/1543128803 learned_addr learned my addr 172.21.15.131:0/1543128803 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.593+0000 7f570f400700 1 --2- 172.21.15.131:0/1543128803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 0x55ed22aea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.593+0000 7f570e000700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56f40a1240 0x7f56f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.594+0000 7f570f400700 1 -- 172.21.15.131:0/1543128803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae3570 msgr2=0x55ed22ae3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.594+0000 7f570f400700 1 --2- 172.21.15.131:0/1543128803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae3570 0x55ed22ae3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.594+0000 7f570f400700 1 -- 172.21.15.131:0/1543128803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae26a0 msgr2=0x55ed22ae2af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:39:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.594+0000 7f570f400700 1 --2- 172.21.15.131:0/1543128803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae26a0 0x55ed22ae2af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.594+0000 7f570f400700 1 -- 172.21.15.131:0/1543128803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed228833b0 con 0x55ed22ae3ec0 2024-05-10T05:39:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.594+0000 7f570e000700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56f40a1240 0x7f56f40a3750 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f571019f6c0 tx=0x7f56fc008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.594+0000 7f570f400700 1 --2- 172.21.15.131:0/1543128803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 0x55ed22aea7c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f570800b070 tx=0x7f5708009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18260e74749e0d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.595+0000 7f56fbe00700 1 -- 172.21.15.131:0/1543128803 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570800e050 con 0x55ed22ae3ec0 2024-05-10T05:39:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.595+0000 7f56fbe00700 1 -- 172.21.15.131:0/1543128803 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57080078c0 con 0x55ed22ae3ec0 2024-05-10T05:39:59.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.595+0000 7f56fbe00700 1 -- 172.21.15.131:0/1543128803 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708005460 con 0x55ed22ae3ec0 2024-05-10T05:39:59.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.597+0000 7f572dadcb80 1 -- 172.21.15.131:0/1543128803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 msgr2=0x55ed22aea7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:59.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.597+0000 7f572dadcb80 1 --2- 172.21.15.131:0/1543128803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 0x55ed22aea7c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f570800b070 tx=0x7f5708009810 comp rx=0 tx=0).stop 2024-05-10T05:39:59.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.597+0000 7f572dadcb80 1 -- 172.21.15.131:0/1543128803 shutdown_connections 2024-05-10T05:39:59.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.597+0000 7f572dadcb80 1 --2- 172.21.15.131:0/1543128803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 0x55ed22aea7c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.597+0000 7f572dadcb80 1 --2- 172.21.15.131:0/1543128803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae26a0 0x55ed22ae2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.597+0000 7f572dadcb80 1 --2- 172.21.15.131:0/1543128803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae3570 0x55ed22ae3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.597+0000 7f572dadcb80 1 -- 172.21.15.131:0/1543128803 >> 172.21.15.131:0/1543128803 conn(0x55ed22a44a60 msgr2=0x55ed22adcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:59.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.597+0000 7f572dadcb80 1 -- 172.21.15.131:0/1543128803 shutdown_connections 2024-05-10T05:39:59.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.597+0000 7f572dadcb80 1 -- 172.21.15.131:0/1543128803 wait complete. 2024-05-10T05:39:59.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.598+0000 7f572dadcb80 1 -- start start 2024-05-10T05:39:59.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.599+0000 7f572dadcb80 1 Processor -- start 2024-05-10T05:39:59.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.599+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae26a0 0x55ed22afd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.599+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae3570 0x55ed22afdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.600+0000 7f570e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae3570 0x55ed22afdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.600+0000 7f570e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae3570 0x55ed22afdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57858/0 (socket says 172.21.15.131:57858) 2024-05-10T05:39:59.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.600+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 0x55ed22b04ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.600+0000 7f570e000700 1 -- 172.21.15.131:0/319101244 learned_addr learned my addr 172.21.15.131:0/319101244 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:59.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.600+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2299e560 con 0x55ed22ae3ec0 2024-05-10T05:39:59.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.600+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2299e200 con 0x55ed22ae3570 2024-05-10T05:39:59.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.600+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed228856a0 con 0x55ed22ae26a0 2024-05-10T05:39:59.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.600+0000 7f572dadcb80 10 client.0 _ll_register_callbacks cb 0x55ed229a7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:59.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.601+0000 7f570ea00700 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae26a0 0x55ed22afd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.601+0000 7f570f400700 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 0x55ed22b04ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.601+0000 7f570e000700 1 -- 172.21.15.131:0/319101244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae26a0 msgr2=0x55ed22afd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.601+0000 7f570e000700 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae26a0 0x55ed22afd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.601+0000 7f570e000700 1 -- 172.21.15.131:0/319101244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 msgr2=0x55ed22b04ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.601+0000 7f570e000700 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 0x55ed22b04ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.601+0000 7f570e000700 1 -- 172.21.15.131:0/319101244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57101067c0 con 0x55ed22ae3570 2024-05-10T05:39:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.602+0000 7f570e000700 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae3570 0x55ed22afdc40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00dc80 tx=0x7f56fc004800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1351d07476bba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.602+0000 7f56f8c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:39:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.602+0000 7f56f8c00700 1 -- 172.21.15.131:0/319101244 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc014a50 con 0x55ed22ae3570 2024-05-10T05:39:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.602+0000 7f56f8c00700 1 -- 172.21.15.131:0/319101244 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f56fc014bb0 con 0x55ed22ae3570 2024-05-10T05:39:59.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.602+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed2295b270 con 0x55ed22ae3570 2024-05-10T05:39:59.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.603+0000 7f56f8c00700 1 -- 172.21.15.131:0/319101244 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56fc014a50 con 0x55ed22ae3570 2024-05-10T05:39:59.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.603+0000 7f572dadcb80 10 client.26534 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:59.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.603+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ed2282fb60 con 0x55ed22ae3570 2024-05-10T05:39:59.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.603+0000 7f56f8c00700 1 -- 172.21.15.131:0/319101244 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed2295b270 con 0x55ed22ae3570 2024-05-10T05:39:59.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.605+0000 7f56f8c00700 1 -- 172.21.15.131:0/319101244 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f56fc027070 con 0x55ed22ae3570 2024-05-10T05:39:59.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.605+0000 7f572dadcb80 10 client.26534 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:59.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.605+0000 7f572dadcb80 10 client.26534 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:39:59.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.605+0000 7f572dadcb80 20 client.26534 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:59.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.605+0000 7f572dadcb80 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55ed22b09380 0x55ed22b0b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.605+0000 7f572dadcb80 4 client.26534 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:39:59.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.605+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed22ab1780 con 0x55ed22b09380 2024-05-10T05:39:59.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.605+0000 7f570ea00700 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55ed22b09380 0x55ed22b0b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.606+0000 7f570ea00700 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55ed22b09380 0x55ed22b0b7d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6bf7cda283dd4f77 server_cookie=bfd37f81ad653389 in_seq=0 out_seq=0 2024-05-10T05:39:59.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.606+0000 7f56f8c00700 10 client.26534 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:39:59.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f56f8c00700 1 -- 172.21.15.131:0/319101244 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed22ab1780 con 0x55ed22b09380 2024-05-10T05:39:59.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f56f8c00700 10 client.26534 handle_command_reply: tid=0 2024-05-10T05:39:59.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f572dadcb80 1 client.26534 shutdown 2024-05-10T05:39:59.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae3570 msgr2=0x55ed22afdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:59.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f572dadcb80 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae3570 0x55ed22afdc40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00dc80 tx=0x7f56fc004800 comp rx=0 tx=0).stop 2024-05-10T05:39:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 shutdown_connections 2024-05-10T05:39:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f572dadcb80 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55ed22b09380 0x55ed22b0b7d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f572dadcb80 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3ec0 0x55ed22b04ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f572dadcb80 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae26a0 0x55ed22afd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f572dadcb80 1 --2- 172.21.15.131:0/319101244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae3570 0x55ed22afdc40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.607+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 >> 172.21.15.131:0/319101244 conn(0x55ed22a44a60 msgr2=0x55ed22adcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.608+0000 7f56f8c00700 0 client.26534 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:39:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.608+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 shutdown_connections 2024-05-10T05:39:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.608+0000 7f572dadcb80 1 -- 172.21.15.131:0/319101244 wait complete. 2024-05-10T05:39:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.608+0000 7f572dadcb80 20 client.26534 trim_cache size 0 max 16384 2024-05-10T05:39:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.636+0000 7f572dadcb80 1 Processor -- start 2024-05-10T05:39:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.637+0000 7f572dadcb80 1 -- start start 2024-05-10T05:39:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.637+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22a44a60 0x55ed22afcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.637+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae26a0 0x55ed22b04b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3570 0x55ed22afd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f572dadcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed22afe7a0 con 0x55ed22ae3570 2024-05-10T05:39:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f572dadcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed22adcfb0 con 0x55ed22a44a60 2024-05-10T05:39:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f572dadcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed22add170 con 0x55ed22ae26a0 2024-05-10T05:39:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f570ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22a44a60 0x55ed22afcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f570ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22a44a60 0x55ed22afcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57874/0 (socket says 172.21.15.131:57874) 2024-05-10T05:39:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f570ea00700 1 -- 172.21.15.131:0/3810162717 learned_addr learned my addr 172.21.15.131:0/3810162717 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f570e000700 1 --2- 172.21.15.131:0/3810162717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae26a0 0x55ed22b04b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f570ea00700 1 -- 172.21.15.131:0/3810162717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae26a0 msgr2=0x55ed22b04b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f570ea00700 1 --2- 172.21.15.131:0/3810162717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae26a0 0x55ed22b04b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f570ea00700 1 -- 172.21.15.131:0/3810162717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3570 msgr2=0x55ed22afd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f570ea00700 1 --2- 172.21.15.131:0/3810162717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3570 0x55ed22afd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f570ea00700 1 -- 172.21.15.131:0/3810162717 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed229c8560 con 0x55ed22a44a60 2024-05-10T05:39:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.638+0000 7f570ea00700 1 --2- 172.21.15.131:0/3810162717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22a44a60 0x55ed22afcc70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f570000bec0 tx=0x7f57000b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9439280818040de5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.639+0000 7f5704c00700 1 -- 172.21.15.131:0/3810162717 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700099990 con 0x55ed22a44a60 2024-05-10T05:39:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.639+0000 7f5704c00700 1 -- 172.21.15.131:0/3810162717 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f570009d7c0 con 0x55ed22a44a60 2024-05-10T05:39:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.639+0000 7f5704c00700 1 -- 172.21.15.131:0/3810162717 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700097d30 con 0x55ed22a44a60 2024-05-10T05:39:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.640+0000 7f572dadcb80 1 -- 172.21.15.131:0/3810162717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22a44a60 msgr2=0x55ed22afcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.640+0000 7f572dadcb80 1 --2- 172.21.15.131:0/3810162717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22a44a60 0x55ed22afcc70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f570000bec0 tx=0x7f57000b9040 comp rx=0 tx=0).stop 2024-05-10T05:39:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.640+0000 7f572dadcb80 1 -- 172.21.15.131:0/3810162717 shutdown_connections 2024-05-10T05:39:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.640+0000 7f572dadcb80 1 --2- 172.21.15.131:0/3810162717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3570 0x55ed22afd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.640+0000 7f572dadcb80 1 --2- 172.21.15.131:0/3810162717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22a44a60 0x55ed22afcc70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.640+0000 7f572dadcb80 1 --2- 172.21.15.131:0/3810162717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22ae26a0 0x55ed22b04b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.640+0000 7f572dadcb80 1 -- 172.21.15.131:0/3810162717 >> 172.21.15.131:0/3810162717 conn(0x55ed22ad9450 msgr2=0x55ed22ae2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.640+0000 7f572dadcb80 1 -- 172.21.15.131:0/3810162717 shutdown_connections 2024-05-10T05:39:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.641+0000 7f572dadcb80 1 -- 172.21.15.131:0/3810162717 wait complete. 2024-05-10T05:39:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.641+0000 7f572dadcb80 1 -- start start 2024-05-10T05:39:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.642+0000 7f572dadcb80 1 Processor -- start 2024-05-10T05:39:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.642+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22a44a60 0x55ed22b02370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.642+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae26a0 0x55ed22b028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.642+0000 7f572dadcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3570 0x55ed22b037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.642+0000 7f572dadcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed22b03d20 con 0x55ed22ae3570 2024-05-10T05:39:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.642+0000 7f572dadcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed22b03e90 con 0x55ed22ae26a0 2024-05-10T05:39:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.642+0000 7f572dadcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed22b04050 con 0x55ed22a44a60 2024-05-10T05:39:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.642+0000 7f572dadcb80 10 client.0 _ll_register_callbacks cb 0x55ed229a7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:39:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22a44a60 0x55ed22b02370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22a44a60 0x55ed22b02370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34568/0 (socket says 172.21.15.131:34568) 2024-05-10T05:39:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570ea00700 1 -- 172.21.15.131:0/654130082 learned_addr learned my addr 172.21.15.131:0/654130082 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:39:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570e000700 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae26a0 0x55ed22b028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570f400700 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3570 0x55ed22b037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570ea00700 1 -- 172.21.15.131:0/654130082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae26a0 msgr2=0x55ed22b028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570ea00700 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae26a0 0x55ed22b028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570ea00700 1 -- 172.21.15.131:0/654130082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3570 msgr2=0x55ed22b037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:39:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570ea00700 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3570 0x55ed22b037e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570ea00700 1 -- 172.21.15.131:0/654130082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57101a6fe0 con 0x55ed22a44a60 2024-05-10T05:39:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f570ea00700 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22a44a60 0x55ed22b02370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f57000ca090 tx=0x7f570009a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb7a84bf035b7d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:39:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.643+0000 7f56f8c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:39:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.644+0000 7f56f8c00700 1 -- 172.21.15.131:0/654130082 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700099990 con 0x55ed22a44a60 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.644+0000 7f56f8c00700 1 -- 172.21.15.131:0/654130082 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5700082070 con 0x55ed22a44a60 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.644+0000 7f572dadcb80 1 -- 172.21.15.131:0/654130082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57080078c0 con 0x55ed22a44a60 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.644+0000 7f56f8c00700 1 -- 172.21.15.131:0/654130082 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57000b2270 con 0x55ed22a44a60 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.644+0000 7f572dadcb80 10 client.26569 fetch_fsmap learned FSMap version 29 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.644+0000 7f572dadcb80 1 -- 172.21.15.131:0/654130082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ed22887cd0 con 0x55ed22a44a60 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.644+0000 7f56f8c00700 1 -- 172.21.15.131:0/654130082 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57080078c0 con 0x55ed22a44a60 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.645+0000 7f56f8c00700 1 -- 172.21.15.131:0/654130082 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5700098e80 con 0x55ed22a44a60 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.646+0000 7f572dadcb80 10 client.26569 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.646+0000 7f572dadcb80 10 client.26569 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.646+0000 7f572dadcb80 20 client.26569 populate_metadata read hostname 'smithi131' 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.646+0000 7f572dadcb80 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55ed22ae3ec0 0x55ed22b61cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.646+0000 7f572dadcb80 4 client.26569 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:39:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.646+0000 7f572dadcb80 1 -- 172.21.15.131:0/654130082 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed229faec0 con 0x55ed22ae3ec0 2024-05-10T05:39:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.646+0000 7f570e000700 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55ed22ae3ec0 0x55ed22b61cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:39:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.647+0000 7f56f8c00700 10 client.26569 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:39:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.647+0000 7f570e000700 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55ed22ae3ec0 0x55ed22b61cf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4faf8f33df45e50 server_cookie=6db103a1ca08ab97 in_seq=0 out_seq=0 2024-05-10T05:39:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.647+0000 7f56f8c00700 1 -- 172.21.15.131:0/654130082 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed229faec0 con 0x55ed22ae3ec0 2024-05-10T05:39:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.647+0000 7f56f8c00700 10 client.26569 handle_command_reply: tid=0 2024-05-10T05:39:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.647+0000 7f572dadcb80 1 client.26569 shutdown 2024-05-10T05:39:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.647+0000 7f572dadcb80 1 -- 172.21.15.131:0/654130082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22a44a60 msgr2=0x55ed22b02370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.647+0000 7f572dadcb80 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22a44a60 0x55ed22b02370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f57000ca090 tx=0x7f570009a1f0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.648+0000 7f572dadcb80 1 -- 172.21.15.131:0/654130082 shutdown_connections 2024-05-10T05:39:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.648+0000 7f572dadcb80 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55ed22ae3ec0 0x55ed22b61cf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.648+0000 7f572dadcb80 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed22ae3570 0x55ed22b037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.648+0000 7f572dadcb80 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed22a44a60 0x55ed22b02370 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.648+0000 7f572dadcb80 1 --2- 172.21.15.131:0/654130082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed22ae26a0 0x55ed22b028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.648+0000 7f572dadcb80 1 -- 172.21.15.131:0/654130082 >> 172.21.15.131:0/654130082 conn(0x55ed22ad9450 msgr2=0x55ed22ae7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.648+0000 7f56f8c00700 0 client.26569 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:39:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.648+0000 7f572dadcb80 1 -- 172.21.15.131:0/654130082 shutdown_connections 2024-05-10T05:39:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.648+0000 7f572dadcb80 1 -- 172.21.15.131:0/654130082 wait complete. 2024-05-10T05:39:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.648+0000 7f572dadcb80 20 client.26569 trim_cache size 0 max 16384 2024-05-10T05:39:59.653 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:39:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.649+0000 7f5715c00700 1 -- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56f40a1240 msgr2=0x7f56f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.649+0000 7f5715c00700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56f40a1240 0x7f56f40a3750 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f571019f6c0 tx=0x7f56fc008040 comp rx=0 tx=0).stop 2024-05-10T05:39:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.649+0000 7f5715c00700 1 -- 172.21.15.131:0/745439124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57101004d0 msgr2=0x7f571019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:39:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.649+0000 7f5715c00700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57101004d0 0x7f571019e620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f570000d950 tx=0x7f570000dd10 comp rx=0 tx=0).stop 2024-05-10T05:39:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.650+0000 7f5715c00700 1 -- 172.21.15.131:0/745439124 shutdown_connections 2024-05-10T05:39:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.650+0000 7f5715c00700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56f40a1240 0x7f56f40a3750 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.650+0000 7f5715c00700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57101a3ae0 0x7f57101a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.650+0000 7f5715c00700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57101004d0 0x7f571019e620 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.650+0000 7f5715c00700 1 --2- 172.21.15.131:0/745439124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5710105790 0x7f571019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:39:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.650+0000 7f5715c00700 1 -- 172.21.15.131:0/745439124 >> 172.21.15.131:0/745439124 conn(0x7f57100fc0e0 msgr2=0x7f5710103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:39:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.650+0000 7f5715c00700 1 -- 172.21.15.131:0/745439124 shutdown_connections 2024-05-10T05:39:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:39:59.650+0000 7f5715c00700 1 -- 172.21.15.131:0/745439124 wait complete. 2024-05-10T05:39:59.706 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:40:00.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.454+0000 7fa036000700 1 -- 172.21.15.131:0/1300821428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0280a6550 msgr2=0x7fa0280a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:00.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.454+0000 7fa036000700 1 --2- 172.21.15.131:0/1300821428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0280a6550 0x7fa0280a6960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa0240098f0 tx=0x7fa024009c00 comp rx=0 tx=0).stop 2024-05-10T05:40:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.457+0000 7fa036000700 1 -- 172.21.15.131:0/1300821428 shutdown_connections 2024-05-10T05:40:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.457+0000 7fa036000700 1 --2- 172.21.15.131:0/1300821428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0280a6ea0 0x7fa0280ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.457+0000 7fa036000700 1 --2- 172.21.15.131:0/1300821428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0280a5570 0x7fa0280a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.457+0000 7fa036000700 1 --2- 172.21.15.131:0/1300821428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0280a6550 0x7fa0280a6960 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.457+0000 7fa036000700 1 -- 172.21.15.131:0/1300821428 >> 172.21.15.131:0/1300821428 conn(0x7fa0280a0a00 msgr2=0x7fa0280a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.463+0000 7fa036000700 1 -- 172.21.15.131:0/1300821428 shutdown_connections 2024-05-10T05:40:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.465+0000 7fa036000700 1 -- 172.21.15.131:0/1300821428 wait complete. 2024-05-10T05:40:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.467+0000 7fa036000700 1 Processor -- start 2024-05-10T05:40:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.470+0000 7fa036000700 1 -- start start 2024-05-10T05:40:00.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.472+0000 7fa036000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0280a5570 0x7fa028144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.472+0000 7fa036000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0280a6550 0x7fa0281445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.472+0000 7fa036000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0280a6ea0 0x7fa02814b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.472+0000 7fa02ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0280a6550 0x7fa0281445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.474+0000 7fa02f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0280a5570 0x7fa028144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.474+0000 7fa02f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0280a5570 0x7fa028144090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34580/0 (socket says 172.21.15.131:34580) 2024-05-10T05:40:00.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.475+0000 7fa02f400700 1 -- 172.21.15.131:0/4102734596 learned_addr learned my addr 172.21.15.131:0/4102734596 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.475+0000 7fa02fe00700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0280a6ea0 0x7fa02814b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.475+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02814bb30 con 0x7fa0280a6ea0 2024-05-10T05:40:00.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.475+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02814bc70 con 0x7fa0280a6550 2024-05-10T05:40:00.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.475+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02814be00 con 0x7fa0280a5570 2024-05-10T05:40:00.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.476+0000 7fa02ea00700 1 -- 172.21.15.131:0/4102734596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0280a5570 msgr2=0x7fa028144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:00.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.476+0000 7fa02ea00700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0280a5570 0x7fa028144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.477+0000 7fa02ea00700 1 -- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0280a6ea0 msgr2=0x7fa02814b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.477+0000 7fa02ea00700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0280a6ea0 0x7fa02814b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.477+0000 7fa02ea00700 1 -- 172.21.15.131:0/4102734596 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024009520 con 0x7fa0280a6550 2024-05-10T05:40:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.477+0000 7fa02fe00700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0280a6ea0 0x7fa02814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.478+0000 7fa02ea00700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0280a6550 0x7fa0281445d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa024000c00 tx=0x7fa02400bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.478+0000 7fa01fe00700 1 -- 172.21.15.131:0/4102734596 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02400b990 con 0x7fa0280a6550 2024-05-10T05:40:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.479+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0280ad160 con 0x7fa0280a6550 2024-05-10T05:40:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.479+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0280ad630 con 0x7fa0280a6550 2024-05-10T05:40:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.479+0000 7fa01fe00700 1 -- 172.21.15.131:0/4102734596 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa02402d070 con 0x7fa0280a6550 2024-05-10T05:40:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.479+0000 7fa01fe00700 1 -- 172.21.15.131:0/4102734596 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024003ae0 con 0x7fa0280a6550 2024-05-10T05:40:00.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.487+0000 7fa04df41b80 1 Processor -- start 2024-05-10T05:40:00.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.488+0000 7fa04df41b80 1 -- start start 2024-05-10T05:40:00.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.489+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd24960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.489+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 0x557d2fd25bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.489+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd2c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.489+0000 7fa04df41b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fc4b150 con 0x557d2fd260f0 2024-05-10T05:40:00.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.489+0000 7fa04df41b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fc48b10 con 0x557d2fd257a0 2024-05-10T05:40:00.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.489+0000 7fa04df41b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fc44400 con 0x557d2fd24530 2024-05-10T05:40:00.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.490+0000 7fa02f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 0x557d2fd25bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.490+0000 7fa02fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd24960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.490+0000 7fa02fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd24960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34588/0 (socket says 172.21.15.131:34588) 2024-05-10T05:40:00.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.490+0000 7fa02fe00700 1 -- 172.21.15.131:0/1548713023 learned_addr learned my addr 172.21.15.131:0/1548713023 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:00.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.491+0000 7fa02fe00700 1 -- 172.21.15.131:0/1548713023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 msgr2=0x557d2fd25bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:00.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.491+0000 7fa02fe00700 1 --2- 172.21.15.131:0/1548713023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 0x557d2fd25bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.491+0000 7fa02fe00700 1 -- 172.21.15.131:0/1548713023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 msgr2=0x557d2fd2c9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:40:00.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.492+0000 7fa02fe00700 1 --2- 172.21.15.131:0/1548713023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd2c9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.492+0000 7fa02fe00700 1 -- 172.21.15.131:0/1548713023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d2fac53b0 con 0x557d2fd24530 2024-05-10T05:40:00.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.495+0000 7fa02fe00700 1 --2- 172.21.15.131:0/1548713023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd24960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa03004fb30 tx=0x7fa0300746b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11f8bb93f9e348de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:00.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.495+0000 7fa013e00700 1 -- 172.21.15.131:0/1548713023 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0300690d0 con 0x557d2fd24530 2024-05-10T05:40:00.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.495+0000 7fa013e00700 1 -- 172.21.15.131:0/1548713023 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0300676e0 con 0x557d2fd24530 2024-05-10T05:40:00.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.495+0000 7fa013e00700 1 -- 172.21.15.131:0/1548713023 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030075680 con 0x557d2fd24530 2024-05-10T05:40:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.497+0000 7fa04df41b80 1 -- 172.21.15.131:0/1548713023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 msgr2=0x557d2fd24960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.497+0000 7fa04df41b80 1 --2- 172.21.15.131:0/1548713023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd24960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa03004fb30 tx=0x7fa0300746b0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.502+0000 7fa01fe00700 1 -- 172.21.15.131:0/4102734596 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa02402f070 con 0x7fa0280a6550 2024-05-10T05:40:00.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.502+0000 7fa04df41b80 1 -- 172.21.15.131:0/1548713023 shutdown_connections 2024-05-10T05:40:00.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.502+0000 7fa04df41b80 1 --2- 172.21.15.131:0/1548713023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd2c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.502+0000 7fa04df41b80 1 --2- 172.21.15.131:0/1548713023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd24960 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.502+0000 7fa04df41b80 1 --2- 172.21.15.131:0/1548713023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 0x557d2fd25bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.502+0000 7fa04df41b80 1 -- 172.21.15.131:0/1548713023 >> 172.21.15.131:0/1548713023 conn(0x557d2fc86a60 msgr2=0x557d2fd1f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:00.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.502+0000 7fa04df41b80 1 -- 172.21.15.131:0/1548713023 shutdown_connections 2024-05-10T05:40:00.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.502+0000 7fa04df41b80 1 -- 172.21.15.131:0/1548713023 wait complete. 2024-05-10T05:40:00.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.503+0000 7fa04df41b80 1 -- start start 2024-05-10T05:40:00.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.504+0000 7fa04df41b80 1 Processor -- start 2024-05-10T05:40:00.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.509+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd24530 0x557d2fd2e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.509+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 0x557d2fd2ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.509+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd260f0 0x557d2fd35dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.509+0000 7fa02fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd24530 0x557d2fd2e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.509+0000 7fa04df41b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fbe0560 con 0x557d2fd24530 2024-05-10T05:40:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.510+0000 7fa02fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd24530 0x557d2fd2e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47838/0 (socket says 172.21.15.131:47838) 2024-05-10T05:40:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.510+0000 7fa02fe00700 1 -- 172.21.15.131:0/853579810 learned_addr learned my addr 172.21.15.131:0/853579810 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.510+0000 7fa02ea00700 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd260f0 0x557d2fd35dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.510+0000 7fa04df41b80 1 -- 172.21.15.131:0/853579810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fbe0200 con 0x557d2fd260f0 2024-05-10T05:40:00.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.510+0000 7fa04df41b80 1 -- 172.21.15.131:0/853579810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fac76a0 con 0x557d2fd257a0 2024-05-10T05:40:00.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.510+0000 7fa04df41b80 10 client.0 _ll_register_callbacks cb 0x557d2fbe9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:00.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.510+0000 7fa02f400700 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 0x557d2fd2ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.511+0000 7fa02f400700 1 -- 172.21.15.131:0/853579810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd260f0 msgr2=0x557d2fd35dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:00.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.511+0000 7fa02f400700 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd260f0 0x557d2fd35dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.511+0000 7fa02f400700 1 -- 172.21.15.131:0/853579810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd24530 msgr2=0x557d2fd2e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:00.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.511+0000 7fa02f400700 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd24530 0x557d2fd2e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.511+0000 7fa02f400700 1 -- 172.21.15.131:0/853579810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa030069ca0 con 0x557d2fd257a0 2024-05-10T05:40:00.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.511+0000 7fa02f400700 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 0x557d2fd2ee60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa020012740 tx=0x7fa020012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36a699660ca844d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:00.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.512+0000 7fa02fe00700 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd24530 0x557d2fd2e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:00.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.512+0000 7fa02ea00700 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd260f0 0x557d2fd35dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:00.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.512+0000 7fa010c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:40:00.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.512+0000 7fa010c00700 1 -- 172.21.15.131:0/853579810 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020017070 con 0x557d2fd257a0 2024-05-10T05:40:00.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.512+0000 7fa010c00700 1 -- 172.21.15.131:0/853579810 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa020004050 con 0x557d2fd257a0 2024-05-10T05:40:00.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.513+0000 7fa04df41b80 1 -- 172.21.15.131:0/853579810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d2fb9d270 con 0x557d2fd257a0 2024-05-10T05:40:00.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.514+0000 7fa010c00700 1 -- 172.21.15.131:0/853579810 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa020004210 con 0x557d2fd257a0 2024-05-10T05:40:00.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.514+0000 7fa04df41b80 10 client.26581 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:00.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.514+0000 7fa04df41b80 1 -- 172.21.15.131:0/853579810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557d2fa71b60 con 0x557d2fd257a0 2024-05-10T05:40:00.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.516+0000 7fa01fe00700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0180a1130 0x7fa0180a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.516+0000 7fa010c00700 1 -- 172.21.15.131:0/853579810 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d2fb9d270 con 0x557d2fd257a0 2024-05-10T05:40:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.517+0000 7fa010c00700 1 -- 172.21.15.131:0/853579810 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa02000f6d0 con 0x557d2fd257a0 2024-05-10T05:40:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.517+0000 7fa04df41b80 10 client.26581 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.517+0000 7fa04df41b80 10 client.26581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:40:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.517+0000 7fa04df41b80 20 client.26581 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.517+0000 7fa04df41b80 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x557d2fd3a670 0x557d2fd3cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.517+0000 7fa04df41b80 4 client.26581 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:00.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.517+0000 7fa04df41b80 1 -- 172.21.15.131:0/853579810 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d2fcf3780 con 0x557d2fd3a670 2024-05-10T05:40:00.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.517+0000 7fa02fe00700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0180a1130 0x7fa0180a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.517+0000 7fa02ea00700 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x557d2fd3a670 0x557d2fd3cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.517+0000 7fa01fe00700 1 -- 172.21.15.131:0/4102734596 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa0240de2f0 con 0x7fa0280a6550 2024-05-10T05:40:00.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.518+0000 7fa02fe00700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0180a1130 0x7fa0180a3640 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa030050560 tx=0x7fa030077040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:00.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.519+0000 7fa010c00700 10 client.26581 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:40:00.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.519+0000 7fa02ea00700 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x557d2fd3a670 0x557d2fd3cac0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b65033c4bdaf8e25 server_cookie=31f244c9693bb547 in_seq=0 out_seq=0 2024-05-10T05:40:00.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.520+0000 7fa010c00700 1 -- 172.21.15.131:0/853579810 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d2fcf3780 con 0x557d2fd3a670 2024-05-10T05:40:00.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.520+0000 7fa010c00700 10 client.26581 handle_command_reply: tid=0 2024-05-10T05:40:00.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.520+0000 7fa04df41b80 1 client.26581 shutdown 2024-05-10T05:40:00.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.521+0000 7fa04df41b80 1 -- 172.21.15.131:0/853579810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 msgr2=0x557d2fd2ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:00.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.521+0000 7fa04df41b80 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 0x557d2fd2ee60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa020012740 tx=0x7fa020012b00 comp rx=0 tx=0).stop 2024-05-10T05:40:00.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.522+0000 7fa04df41b80 1 -- 172.21.15.131:0/853579810 shutdown_connections 2024-05-10T05:40:00.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.522+0000 7fa04df41b80 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x557d2fd3a670 0x557d2fd3cac0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.522+0000 7fa010c00700 0 client.26581 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:40:00.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.522+0000 7fa04df41b80 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd24530 0x557d2fd2e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.522+0000 7fa04df41b80 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd260f0 0x557d2fd35dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.522+0000 7fa04df41b80 1 --2- 172.21.15.131:0/853579810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 0x557d2fd2ee60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.522+0000 7fa04df41b80 1 -- 172.21.15.131:0/853579810 >> 172.21.15.131:0/853579810 conn(0x557d2fc86a60 msgr2=0x557d2fd23060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:00.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.523+0000 7fa04df41b80 1 -- 172.21.15.131:0/853579810 shutdown_connections 2024-05-10T05:40:00.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.523+0000 7fa04df41b80 1 -- 172.21.15.131:0/853579810 wait complete. 2024-05-10T05:40:00.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.524+0000 7fa04df41b80 20 client.26581 trim_cache size 0 max 16384 2024-05-10T05:40:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.552+0000 7fa04df41b80 1 Processor -- start 2024-05-10T05:40:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.552+0000 7fa04df41b80 1 -- start start 2024-05-10T05:40:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.552+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd24530 0x557d2fd36350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:00 smithi157 ceph-mon[78427]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 253 MiB/s wr, 75 op/s 2024-05-10T05:40:00.556 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:00 smithi157 ceph-mon[78427]: overall HEALTH_OK 2024-05-10T05:40:00.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.553+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 0x557d2fd35c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.553+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd1f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.553+0000 7fa04df41b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fd198d0 con 0x557d2fd260f0 2024-05-10T05:40:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.553+0000 7fa04df41b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fd2f9c0 con 0x557d2fd24530 2024-05-10T05:40:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.553+0000 7fa04df41b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fd41c60 con 0x557d2fd257a0 2024-05-10T05:40:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.553+0000 7fa02f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd24530 0x557d2fd36350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.554+0000 7fa02f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd24530 0x557d2fd36350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57922/0 (socket says 172.21.15.131:57922) 2024-05-10T05:40:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.554+0000 7fa02f400700 1 -- 172.21.15.131:0/187710007 learned_addr learned my addr 172.21.15.131:0/187710007 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:00.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.555+0000 7fa02ea00700 1 --2- 172.21.15.131:0/187710007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 0x557d2fd35c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.555+0000 7fa02fe00700 1 --2- 172.21.15.131:0/187710007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd1f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.555+0000 7fa02f400700 1 -- 172.21.15.131:0/187710007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 msgr2=0x557d2fd35c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:00.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.555+0000 7fa02f400700 1 --2- 172.21.15.131:0/187710007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 0x557d2fd35c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.555+0000 7fa02f400700 1 -- 172.21.15.131:0/187710007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 msgr2=0x557d2fd1f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:00.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.555+0000 7fa02f400700 1 --2- 172.21.15.131:0/187710007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd1f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.555+0000 7fa02f400700 1 -- 172.21.15.131:0/187710007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d2fc0a560 con 0x557d2fd24530 2024-05-10T05:40:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.557+0000 7fa02f400700 1 --2- 172.21.15.131:0/187710007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd24530 0x557d2fd36350 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa020012f00 tx=0x7fa020013fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f92aed377dcfca43 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.558+0000 7fa01d200700 1 -- 172.21.15.131:0/187710007 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020017070 con 0x557d2fd24530 2024-05-10T05:40:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.558+0000 7fa01d200700 1 -- 172.21.15.131:0/187710007 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa020018040 con 0x557d2fd24530 2024-05-10T05:40:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.558+0000 7fa01d200700 1 -- 172.21.15.131:0/187710007 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020017070 con 0x557d2fd24530 2024-05-10T05:40:00.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.559+0000 7fa04df41b80 1 -- 172.21.15.131:0/187710007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd24530 msgr2=0x557d2fd36350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:00.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.559+0000 7fa04df41b80 1 --2- 172.21.15.131:0/187710007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd24530 0x557d2fd36350 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa020012f00 tx=0x7fa020013fc0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.560+0000 7fa04df41b80 1 -- 172.21.15.131:0/187710007 shutdown_connections 2024-05-10T05:40:00.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.560+0000 7fa04df41b80 1 --2- 172.21.15.131:0/187710007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd1f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.560+0000 7fa04df41b80 1 --2- 172.21.15.131:0/187710007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd24530 0x557d2fd36350 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.560+0000 7fa04df41b80 1 --2- 172.21.15.131:0/187710007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd257a0 0x557d2fd35c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.561+0000 7fa04df41b80 1 -- 172.21.15.131:0/187710007 >> 172.21.15.131:0/187710007 conn(0x557d2fc86a60 msgr2=0x557d2fd1ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.561+0000 7fa04df41b80 1 -- 172.21.15.131:0/187710007 shutdown_connections 2024-05-10T05:40:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.561+0000 7fa04df41b80 1 -- 172.21.15.131:0/187710007 wait complete. 2024-05-10T05:40:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.562+0000 7fa04df41b80 1 -- start start 2024-05-10T05:40:00.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.562+0000 7fa04df41b80 1 Processor -- start 2024-05-10T05:40:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.565+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd33840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.565+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 0x557d2fd33d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.565+0000 7fa04df41b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd34c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.567+0000 7fa02ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 0x557d2fd33d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.567+0000 7fa02f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd33840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.567+0000 7fa02f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd33840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34614/0 (socket says 172.21.15.131:34614) 2024-05-10T05:40:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.567+0000 7fa02ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 0x557d2fd33d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57938/0 (socket says 172.21.15.131:57938) 2024-05-10T05:40:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.568+0000 7fa04df41b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fd351a0 con 0x557d2fd260f0 2024-05-10T05:40:00.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.568+0000 7fa04df41b80 1 -- 172.21.15.131:0/2162104574 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fd35310 con 0x557d2fd257a0 2024-05-10T05:40:00.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.568+0000 7fa02ea00700 1 -- 172.21.15.131:0/2162104574 learned_addr learned my addr 172.21.15.131:0/2162104574 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:00.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.568+0000 7fa04df41b80 1 -- 172.21.15.131:0/2162104574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d2fd354d0 con 0x557d2fd24530 2024-05-10T05:40:00.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.569+0000 7fa02fe00700 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd34c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.570+0000 7fa02ea00700 1 -- 172.21.15.131:0/2162104574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 msgr2=0x557d2fd33840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:00.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.570+0000 7fa02ea00700 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd33840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.570+0000 7fa02ea00700 1 -- 172.21.15.131:0/2162104574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 msgr2=0x557d2fd34c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.570+0000 7fa02ea00700 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd34c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.570+0000 7fa02ea00700 1 -- 172.21.15.131:0/2162104574 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d2fa71b60 con 0x557d2fd257a0 2024-05-10T05:40:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.570+0000 7fa02f400700 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd33840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.571+0000 7fa04df41b80 10 client.0 _ll_register_callbacks cb 0x557d2fbe9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.571+0000 7fa04df41b80 1 -- 172.21.15.131:0/2162104574 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0300676e0 con 0x557d2fd257a0 2024-05-10T05:40:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.572+0000 7fa02ea00700 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 0x557d2fd33d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa02400f9a0 tx=0x7fa02402d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f68eef8c6b3232 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.572+0000 7fa010c00700 10 client.26555 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.572+0000 7fa010c00700 1 -- 172.21.15.131:0/2162104574 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0240ac350 con 0x557d2fd257a0 2024-05-10T05:40:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.572+0000 7fa010c00700 1 -- 172.21.15.131:0/2162104574 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa0240abc50 con 0x557d2fd257a0 2024-05-10T05:40:00.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.572+0000 7fa010c00700 1 -- 172.21.15.131:0/2162104574 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0240afa80 con 0x557d2fd257a0 2024-05-10T05:40:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.575+0000 7fa04df41b80 10 client.26555 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.575+0000 7fa04df41b80 1 -- 172.21.15.131:0/2162104574 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x557d2fac9cd0 con 0x557d2fd257a0 2024-05-10T05:40:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.575+0000 7fa010c00700 1 -- 172.21.15.131:0/2162104574 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa024003c40 con 0x557d2fd257a0 2024-05-10T05:40:00.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.575+0000 7fa04df41b80 10 client.26555 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:00.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.575+0000 7fa04df41b80 10 client.26555 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:40:00.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.575+0000 7fa04df41b80 20 client.26555 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:00.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.575+0000 7fa04df41b80 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x557d2fd911d0 0x557d2fd93640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:00.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.575+0000 7fa04df41b80 4 client.26555 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:40:00.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.575+0000 7fa04df41b80 1 -- 172.21.15.131:0/2162104574 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d2fc3cec0 con 0x557d2fd911d0 2024-05-10T05:40:00.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.584+0000 7fa02f400700 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x557d2fd911d0 0x557d2fd93640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:00.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.585+0000 7fa010c00700 10 client.26555 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:40:00.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.585+0000 7fa02f400700 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x557d2fd911d0 0x557d2fd93640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fa8d277ca8209e1 server_cookie=5f133d0c53ce733c in_seq=0 out_seq=0 2024-05-10T05:40:00.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.586+0000 7fa010c00700 1 -- 172.21.15.131:0/2162104574 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d2fc3cec0 con 0x557d2fd911d0 2024-05-10T05:40:00.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.586+0000 7fa010c00700 10 client.26555 handle_command_reply: tid=0 2024-05-10T05:40:00.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.586+0000 7fa04df41b80 1 client.26555 shutdown 2024-05-10T05:40:00.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.586+0000 7fa04df41b80 1 -- 172.21.15.131:0/2162104574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 msgr2=0x557d2fd33d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:00.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.586+0000 7fa04df41b80 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 0x557d2fd33d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa02400f9a0 tx=0x7fa02402d000 comp rx=0 tx=0).stop 2024-05-10T05:40:00.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.588+0000 7fa04df41b80 1 -- 172.21.15.131:0/2162104574 shutdown_connections 2024-05-10T05:40:00.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.588+0000 7fa04df41b80 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x557d2fd911d0 0x557d2fd93640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.588+0000 7fa04df41b80 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d2fd260f0 0x557d2fd34c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.588+0000 7fa04df41b80 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d2fd24530 0x557d2fd33840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.588+0000 7fa04df41b80 1 --2- 172.21.15.131:0/2162104574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d2fd257a0 0x557d2fd33d80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.588+0000 7fa04df41b80 1 -- 172.21.15.131:0/2162104574 >> 172.21.15.131:0/2162104574 conn(0x557d2fc86a60 msgr2=0x557d2fd182e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:00.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.588+0000 7fa010c00700 0 client.26555 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:40:00.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.590+0000 7fa04df41b80 1 -- 172.21.15.131:0/2162104574 shutdown_connections 2024-05-10T05:40:00.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.590+0000 7fa04df41b80 1 -- 172.21.15.131:0/2162104574 wait complete. 2024-05-10T05:40:00.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.590+0000 7fa04df41b80 20 client.26555 trim_cache size 0 max 16384 2024-05-10T05:40:00.591 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:40:00.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.603+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0180a1130 msgr2=0x7fa0180a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:00.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.603+0000 7fa036000700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0180a1130 0x7fa0180a3640 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa030050560 tx=0x7fa030077040 comp rx=0 tx=0).stop 2024-05-10T05:40:00.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.603+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0280a6550 msgr2=0x7fa0281445d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:00.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.603+0000 7fa036000700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0280a6550 0x7fa0281445d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa024000c00 tx=0x7fa02400bfc0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.604+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 shutdown_connections 2024-05-10T05:40:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.604+0000 7fa036000700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa0180a1130 0x7fa0180a3640 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.604+0000 7fa036000700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0280a6ea0 0x7fa02814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.604+0000 7fa036000700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0280a5570 0x7fa028144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.604+0000 7fa036000700 1 --2- 172.21.15.131:0/4102734596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0280a6550 0x7fa0281445d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:00.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.604+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 >> 172.21.15.131:0/4102734596 conn(0x7fa0280a0a00 msgr2=0x7fa0280a7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:00.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.605+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 shutdown_connections 2024-05-10T05:40:00.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:00.610+0000 7fa036000700 1 -- 172.21.15.131:0/4102734596 wait complete. 2024-05-10T05:40:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:00 smithi142 ceph-mon[77667]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 253 MiB/s wr, 75 op/s 2024-05-10T05:40:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:00 smithi142 ceph-mon[77667]: overall HEALTH_OK 2024-05-10T05:40:00.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:00 smithi131 ceph-mon[37264]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 253 MiB/s wr, 75 op/s 2024-05-10T05:40:00.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:00 smithi131 ceph-mon[37264]: overall HEALTH_OK 2024-05-10T05:40:00.714 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:40:01.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.495+0000 7f75dea00700 1 -- 172.21.15.131:0/2426319634 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000fe90 con 0x7f75e0104f20 2024-05-10T05:40:01.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.496+0000 7f75e6e00700 1 -- 172.21.15.131:0/2426319634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e0104f20 msgr2=0x7f75e0105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:01.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.496+0000 7f75e6e00700 1 --2- 172.21.15.131:0/2426319634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e0104f20 0x7f75e0105330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009920 tx=0x7f75d0009c30 comp rx=0 tx=0).stop 2024-05-10T05:40:01.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.497+0000 7f75e6e00700 1 -- 172.21.15.131:0/2426319634 shutdown_connections 2024-05-10T05:40:01.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.497+0000 7f75e6e00700 1 --2- 172.21.15.131:0/2426319634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e0103e00 0x7f75e0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.497+0000 7f75e6e00700 1 --2- 172.21.15.131:0/2426319634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e01058c0 0x7f75e010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.497+0000 7f75e6e00700 1 --2- 172.21.15.131:0/2426319634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e0104f20 0x7f75e0105330 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.497+0000 7f75e6e00700 1 -- 172.21.15.131:0/2426319634 >> 172.21.15.131:0/2426319634 conn(0x7f75e00ff410 msgr2=0x7f75e01018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:01.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.499+0000 7f75e6e00700 1 -- 172.21.15.131:0/2426319634 shutdown_connections 2024-05-10T05:40:01.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.499+0000 7f75e6e00700 1 -- 172.21.15.131:0/2426319634 wait complete. 2024-05-10T05:40:01.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.501+0000 7f75e6e00700 1 Processor -- start 2024-05-10T05:40:01.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.501+0000 7f75e6e00700 1 -- start start 2024-05-10T05:40:01.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.501+0000 7f75e6e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e0103e00 0x7f75e0195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.501+0000 7f75e6e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e0104f20 0x7f75e01962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.502+0000 7f75e6e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e01058c0 0x7f75e019d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.502+0000 7f75e6e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e019d8a0 con 0x7f75e0103e00 2024-05-10T05:40:01.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.502+0000 7f75e6e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e019da10 con 0x7f75e01058c0 2024-05-10T05:40:01.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.502+0000 7f75e6e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e019dbd0 con 0x7f75e0104f20 2024-05-10T05:40:01.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.502+0000 7f75e4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e01058c0 0x7f75e019d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.502+0000 7f75e4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e01058c0 0x7f75e019d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57958/0 (socket says 172.21.15.131:57958) 2024-05-10T05:40:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.502+0000 7f75e4e00700 1 -- 172.21.15.131:0/1059948480 learned_addr learned my addr 172.21.15.131:0/1059948480 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.503+0000 7f75df400700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e0104f20 0x7f75e01962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.503+0000 7f75df400700 1 -- 172.21.15.131:0/1059948480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e01058c0 msgr2=0x7f75e019d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.503+0000 7f75df400700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e01058c0 0x7f75e019d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.503+0000 7f75df400700 1 -- 172.21.15.131:0/1059948480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e0103e00 msgr2=0x7f75e0195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.503+0000 7f75df400700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e0103e00 0x7f75e0195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.503+0000 7f75df400700 1 -- 172.21.15.131:0/1059948480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d0009520 con 0x7f75e0104f20 2024-05-10T05:40:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.504+0000 7f75df400700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e0104f20 0x7f75e01962b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f75d0005950 tx=0x7f75d000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.504+0000 7f75dcc00700 1 -- 172.21.15.131:0/1059948480 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d001c070 con 0x7f75e0104f20 2024-05-10T05:40:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.504+0000 7f75e6e00700 1 -- 172.21.15.131:0/1059948480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e019de50 con 0x7f75e0104f20 2024-05-10T05:40:01.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.505+0000 7f75e6e00700 1 -- 172.21.15.131:0/1059948480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e019e280 con 0x7f75e0104f20 2024-05-10T05:40:01.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.507+0000 7f75dcc00700 1 -- 172.21.15.131:0/1059948480 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75d000f7f0 con 0x7f75e0104f20 2024-05-10T05:40:01.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.507+0000 7f75dcc00700 1 -- 172.21.15.131:0/1059948480 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d00038e0 con 0x7f75e0104f20 2024-05-10T05:40:01.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.512+0000 7f75dcc00700 1 -- 172.21.15.131:0/1059948480 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f75d0020030 con 0x7f75e0104f20 2024-05-10T05:40:01.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.513+0000 7f75feceab80 1 Processor -- start 2024-05-10T05:40:01.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.514+0000 7f75feceab80 1 -- start start 2024-05-10T05:40:01.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.514+0000 7f75dcc00700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75c40a1290 0x7f75c40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.515+0000 7f75dcc00700 1 -- 172.21.15.131:0/1059948480 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f75d00cd740 con 0x7f75e0104f20 2024-05-10T05:40:01.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.515+0000 7f75e4e00700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75c40a1290 0x7f75c40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.516+0000 7f75feceab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a916a0 0x55c409a91af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.516+0000 7f75feceab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92570 0x55c409a92980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.516+0000 7f75feceab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.516+0000 7f75feceab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4099b8150 con 0x55c409a916a0 2024-05-10T05:40:01.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.516+0000 7f75df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92570 0x55c409a92980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.516+0000 7f75feceab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4099b5b10 con 0x55c409a92570 2024-05-10T05:40:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.516+0000 7f75feceab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4099b1400 con 0x55c409a92ec0 2024-05-10T05:40:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.516+0000 7f75df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92570 0x55c409a92980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57962/0 (socket says 172.21.15.131:57962) 2024-05-10T05:40:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.516+0000 7f75df400700 1 -- 172.21.15.131:0/3097122338 learned_addr learned my addr 172.21.15.131:0/3097122338 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.517+0000 7f75e4e00700 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.517+0000 7f75dfe00700 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a916a0 0x55c409a91af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.520+0000 7f75e4e00700 1 -- 172.21.15.131:0/3097122338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92570 msgr2=0x55c409a92980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.520+0000 7f75e4e00700 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92570 0x55c409a92980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.520+0000 7f75e4e00700 1 -- 172.21.15.131:0/3097122338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a916a0 msgr2=0x55c409a91af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.520+0000 7f75e4e00700 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a916a0 0x55c409a91af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.520+0000 7f75e4e00700 1 -- 172.21.15.131:0/3097122338 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4098323b0 con 0x55c409a92ec0 2024-05-10T05:40:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.520+0000 7f75e4e00700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75c40a1290 0x7f75c40a37a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f75cc013570 tx=0x7f75cc009800 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.520+0000 7f75dfe00700 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a916a0 0x55c409a91af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:40:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.520+0000 7f75df400700 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92570 0x55c409a92980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.521+0000 7f75e4e00700 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a997c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f75cc013cc0 tx=0x7f75cc020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a159fcd9b07fe8d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.521+0000 7f75d5600700 1 -- 172.21.15.131:0/3097122338 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc020c80 con 0x55c409a92ec0 2024-05-10T05:40:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.521+0000 7f75d5600700 1 -- 172.21.15.131:0/3097122338 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75cc020de0 con 0x55c409a92ec0 2024-05-10T05:40:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.521+0000 7f75d5600700 1 -- 172.21.15.131:0/3097122338 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00f620 con 0x55c409a92ec0 2024-05-10T05:40:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.523+0000 7f75feceab80 1 -- 172.21.15.131:0/3097122338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 msgr2=0x55c409a997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.523+0000 7f75feceab80 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a997c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f75cc013cc0 tx=0x7f75cc020000 comp rx=0 tx=0).stop 2024-05-10T05:40:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.523+0000 7f75feceab80 1 -- 172.21.15.131:0/3097122338 shutdown_connections 2024-05-10T05:40:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.523+0000 7f75feceab80 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a916a0 0x55c409a91af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.523+0000 7f75feceab80 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a997c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.523+0000 7f75feceab80 1 --2- 172.21.15.131:0/3097122338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92570 0x55c409a92980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.523+0000 7f75feceab80 1 -- 172.21.15.131:0/3097122338 >> 172.21.15.131:0/3097122338 conn(0x55c4099f3a60 msgr2=0x55c409a8bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.523+0000 7f75feceab80 1 -- 172.21.15.131:0/3097122338 shutdown_connections 2024-05-10T05:40:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.523+0000 7f75feceab80 1 -- 172.21.15.131:0/3097122338 wait complete. 2024-05-10T05:40:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.524+0000 7f75feceab80 1 -- start start 2024-05-10T05:40:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.524+0000 7f75feceab80 1 Processor -- start 2024-05-10T05:40:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.525+0000 7f75feceab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 0x55c409a9b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.525+0000 7f75feceab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409a9bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.525+0000 7f75feceab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409aa2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.526+0000 7f75feceab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c40994d560 con 0x55c409a92570 2024-05-10T05:40:01.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.526+0000 7f75dfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 0x55c409a9b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.526+0000 7f75feceab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c40994d200 con 0x55c409a92ec0 2024-05-10T05:40:01.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.526+0000 7f75feceab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4098346a0 con 0x55c409a916a0 2024-05-10T05:40:01.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.526+0000 7f75e4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409aa2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.526+0000 7f75e4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409aa2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57976/0 (socket says 172.21.15.131:57976) 2024-05-10T05:40:01.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.526+0000 7f75e4e00700 1 -- 172.21.15.131:0/1702283120 learned_addr learned my addr 172.21.15.131:0/1702283120 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:01.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.526+0000 7f75df400700 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409a9bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.527+0000 7f75feceab80 10 client.0 _ll_register_callbacks cb 0x55c409956b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:01.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.526+0000 7f75dfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 0x55c409a9b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34656/0 (socket says 172.21.15.131:34656) 2024-05-10T05:40:01.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.528+0000 7f75e4e00700 1 -- 172.21.15.131:0/1702283120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 msgr2=0x55c409a9b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:01.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.528+0000 7f75e4e00700 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 0x55c409a9b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.528+0000 7f75e4e00700 1 -- 172.21.15.131:0/1702283120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 msgr2=0x55c409a9bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:01.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.528+0000 7f75e4e00700 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409a9bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.528+0000 7f75e4e00700 1 -- 172.21.15.131:0/1702283120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e019e280 con 0x55c409a92ec0 2024-05-10T05:40:01.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.528+0000 7f75df400700 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409a9bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:01.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.529+0000 7f75e4e00700 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409aa2b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00f5f0 tx=0x7f75cc00f6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=364a9976de18fc63 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:01.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.529+0000 7f75ca000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:01.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.529+0000 7f75ca000700 1 -- 172.21.15.131:0/1702283120 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc02e070 con 0x55c409a92ec0 2024-05-10T05:40:01.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.529+0000 7f75ca000700 1 -- 172.21.15.131:0/1702283120 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f75cc010640 con 0x55c409a92ec0 2024-05-10T05:40:01.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.529+0000 7f75feceab80 1 -- 172.21.15.131:0/1702283120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c40990a270 con 0x55c409a92ec0 2024-05-10T05:40:01.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75ca000700 1 -- 172.21.15.131:0/1702283120 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75cc020c80 con 0x55c409a92ec0 2024-05-10T05:40:01.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75feceab80 10 client.26567 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:01.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75feceab80 1 -- 172.21.15.131:0/1702283120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f75cc020430 con 0x55c409a92ec0 2024-05-10T05:40:01.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75ca000700 1 -- 172.21.15.131:0/1702283120 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c40990a270 con 0x55c409a92ec0 2024-05-10T05:40:01.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75ca000700 1 -- 172.21.15.131:0/1702283120 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f75cc005340 con 0x55c409a92ec0 2024-05-10T05:40:01.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75feceab80 10 client.26567 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:01.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75feceab80 10 client.26567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:40:01.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75feceab80 20 client.26567 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:01.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75feceab80 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55c409aa7290 0x55c409aa96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75feceab80 4 client.26567 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:01.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.530+0000 7f75feceab80 1 -- 172.21.15.131:0/1702283120 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c409a60780 con 0x55c409aa7290 2024-05-10T05:40:01.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.532+0000 7f75dfe00700 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55c409aa7290 0x55c409aa96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.532+0000 7f75dfe00700 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55c409aa7290 0x55c409aa96e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=106d987854b40ed6 server_cookie=599f91568f80a908 in_seq=0 out_seq=0 2024-05-10T05:40:01.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.532+0000 7f75ca000700 10 client.26567 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:40:01.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.534+0000 7f75ca000700 1 -- 172.21.15.131:0/1702283120 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c409a60780 con 0x55c409aa7290 2024-05-10T05:40:01.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.534+0000 7f75ca000700 10 client.26567 handle_command_reply: tid=0 2024-05-10T05:40:01.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.534+0000 7f75feceab80 1 client.26567 shutdown 2024-05-10T05:40:01.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.534+0000 7f75feceab80 1 -- 172.21.15.131:0/1702283120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 msgr2=0x55c409aa2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:01.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.534+0000 7f75feceab80 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409aa2b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00f5f0 tx=0x7f75cc00f6d0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.535+0000 7f75feceab80 1 -- 172.21.15.131:0/1702283120 shutdown_connections 2024-05-10T05:40:01.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.535+0000 7f75feceab80 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55c409aa7290 0x55c409aa96e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.535+0000 7f75feceab80 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409aa2b50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.535+0000 7f75feceab80 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 0x55c409a9b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.535+0000 7f75feceab80 1 --2- 172.21.15.131:0/1702283120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409a9bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.535+0000 7f75feceab80 1 -- 172.21.15.131:0/1702283120 >> 172.21.15.131:0/1702283120 conn(0x55c4099f3a60 msgr2=0x55c409a8bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:01.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.535+0000 7f75ca000700 0 client.26567 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:40:01.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.535+0000 7f75feceab80 1 -- 172.21.15.131:0/1702283120 shutdown_connections 2024-05-10T05:40:01.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.535+0000 7f75feceab80 1 -- 172.21.15.131:0/1702283120 wait complete. 2024-05-10T05:40:01.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.536+0000 7f75feceab80 20 client.26567 trim_cache size 0 max 16384 2024-05-10T05:40:01.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.563+0000 7f75feceab80 1 Processor -- start 2024-05-10T05:40:01.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.563+0000 7f75feceab80 1 -- start start 2024-05-10T05:40:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.564+0000 7f75feceab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 0x55c409a957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.564+0000 7f75feceab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409aa26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.564+0000 7f75feceab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409a962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.564+0000 7f75feceab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c409a867e0 con 0x55c409a92570 2024-05-10T05:40:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.564+0000 7f75feceab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c409a9c0e0 con 0x55c409a92ec0 2024-05-10T05:40:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.564+0000 7f75feceab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c409aae880 con 0x55c409a916a0 2024-05-10T05:40:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75dfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 0x55c409a957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75df400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409aa26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75dfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 0x55c409a957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34664/0 (socket says 172.21.15.131:34664) 2024-05-10T05:40:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75dfe00700 1 -- 172.21.15.131:0/1190389946 learned_addr learned my addr 172.21.15.131:0/1190389946 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75e4e00700 1 --2- 172.21.15.131:0/1190389946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409a962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75df400700 1 -- 172.21.15.131:0/1190389946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 msgr2=0x55c409a957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75df400700 1 --2- 172.21.15.131:0/1190389946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 0x55c409a957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75df400700 1 -- 172.21.15.131:0/1190389946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 msgr2=0x55c409a962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75df400700 1 --2- 172.21.15.131:0/1190389946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409a962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75df400700 1 -- 172.21.15.131:0/1190389946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4097deb60 con 0x55c409a92570 2024-05-10T05:40:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.565+0000 7f75e4e00700 1 --2- 172.21.15.131:0/1190389946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409a962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:01.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.566+0000 7f75df400700 1 --2- 172.21.15.131:0/1190389946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409aa26f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f75d009abb0 tx=0x7f75d0019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6186783c006721a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:01.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.567+0000 7f75d6000700 1 -- 172.21.15.131:0/1190389946 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d009ac20 con 0x55c409a92570 2024-05-10T05:40:01.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.567+0000 7f75d6000700 1 -- 172.21.15.131:0/1190389946 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75d009ea50 con 0x55c409a92570 2024-05-10T05:40:01.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.568+0000 7f75feceab80 1 -- 172.21.15.131:0/1190389946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 msgr2=0x55c409aa26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:01.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.568+0000 7f75feceab80 1 --2- 172.21.15.131:0/1190389946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409aa26f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f75d009abb0 tx=0x7f75d0019040 comp rx=0 tx=0).stop 2024-05-10T05:40:01.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.579+0000 7f75feceab80 1 -- 172.21.15.131:0/1190389946 shutdown_connections 2024-05-10T05:40:01.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.579+0000 7f75feceab80 1 --2- 172.21.15.131:0/1190389946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a92ec0 0x55c409a962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.579+0000 7f75feceab80 1 --2- 172.21.15.131:0/1190389946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a916a0 0x55c409a957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.579+0000 7f75feceab80 1 --2- 172.21.15.131:0/1190389946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409aa26f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.579+0000 7f75feceab80 1 -- 172.21.15.131:0/1190389946 >> 172.21.15.131:0/1190389946 conn(0x55c4099f3a60 msgr2=0x55c409a8bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:01.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.579+0000 7f75feceab80 1 -- 172.21.15.131:0/1190389946 shutdown_connections 2024-05-10T05:40:01.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.579+0000 7f75feceab80 1 -- 172.21.15.131:0/1190389946 wait complete. 2024-05-10T05:40:01.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.580+0000 7f75feceab80 1 -- start start 2024-05-10T05:40:01.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.580+0000 7f75feceab80 1 Processor -- start 2024-05-10T05:40:01.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.581+0000 7f75feceab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a916a0 0x55c409aa2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.581+0000 7f75feceab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409a9f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.581+0000 7f75feceab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a9fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.581+0000 7f75dfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a916a0 0x55c409aa2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.581+0000 7f75dfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a916a0 0x55c409aa2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:57984/0 (socket says 172.21.15.131:57984) 2024-05-10T05:40:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.582+0000 7f75e4e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a9fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.582+0000 7f75e4e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a9fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34680/0 (socket says 172.21.15.131:34680) 2024-05-10T05:40:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.582+0000 7f75e4e00700 1 -- 172.21.15.131:0/1899686169 learned_addr learned my addr 172.21.15.131:0/1899686169 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.582+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c409aa0310 con 0x55c409a92570 2024-05-10T05:40:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.582+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c409aa0480 con 0x55c409a916a0 2024-05-10T05:40:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.582+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c409afb760 con 0x55c409a92ec0 2024-05-10T05:40:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.582+0000 7f75feceab80 10 client.0 _ll_register_callbacks cb 0x55c409956b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.582+0000 7f75df400700 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409a9f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.583+0000 7f75dfe00700 1 -- 172.21.15.131:0/1899686169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 msgr2=0x55c409a9fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.583+0000 7f75dfe00700 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a9fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.583+0000 7f75dfe00700 1 -- 172.21.15.131:0/1899686169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 msgr2=0x55c409a9f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.583+0000 7f75dfe00700 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409a9f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.583+0000 7f75dfe00700 1 -- 172.21.15.131:0/1899686169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e019de50 con 0x55c409a916a0 2024-05-10T05:40:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.584+0000 7f75e4e00700 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a9fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.584+0000 7f75df400700 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409a9f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.584+0000 7f75dfe00700 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a916a0 0x55c409aa2cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f75d8004b10 tx=0x7f75d801a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d4d3a9d7d53f6ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.584+0000 7f75ca000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.584+0000 7f75ca000700 1 -- 172.21.15.131:0/1899686169 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8003c00 con 0x55c409a916a0 2024-05-10T05:40:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.585+0000 7f75ca000700 1 -- 172.21.15.131:0/1899686169 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f75d8003d60 con 0x55c409a916a0 2024-05-10T05:40:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.585+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75cc020de0 con 0x55c409a916a0 2024-05-10T05:40:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.585+0000 7f75ca000700 1 -- 172.21.15.131:0/1899686169 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75d8007890 con 0x55c409a916a0 2024-05-10T05:40:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.585+0000 7f75feceab80 10 client.26570 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.585+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c409977560 con 0x55c409a916a0 2024-05-10T05:40:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.586+0000 7f75ca000700 1 -- 172.21.15.131:0/1899686169 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75cc020de0 con 0x55c409a916a0 2024-05-10T05:40:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.587+0000 7f75ca000700 1 -- 172.21.15.131:0/1899686169 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f75d8023430 con 0x55c409a916a0 2024-05-10T05:40:01.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.587+0000 7f75feceab80 10 client.26570 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:01.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.587+0000 7f75feceab80 10 client.26570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:40:01.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.587+0000 7f75feceab80 20 client.26570 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:01.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.587+0000 7f75feceab80 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55c409afd000 0x55c409aff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:01.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.587+0000 7f75feceab80 4 client.26570 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:40:01.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.587+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c4099a9ec0 con 0x55c409afd000 2024-05-10T05:40:01.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.588+0000 7f75df400700 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55c409afd000 0x55c409aff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:01.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.589+0000 7f75df400700 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55c409afd000 0x55c409aff420 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a005202360efe472 server_cookie=87e5722b31c1585e in_seq=0 out_seq=0 2024-05-10T05:40:01.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.589+0000 7f75ca000700 10 client.26570 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:40:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.590+0000 7f75ca000700 1 -- 172.21.15.131:0/1899686169 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c4099a9ec0 con 0x55c409afd000 2024-05-10T05:40:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.590+0000 7f75ca000700 10 client.26570 handle_command_reply: tid=0 2024-05-10T05:40:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.590+0000 7f75feceab80 1 client.26570 shutdown 2024-05-10T05:40:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.591+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a916a0 msgr2=0x55c409aa2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.591+0000 7f75feceab80 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a916a0 0x55c409aa2cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f75d8004b10 tx=0x7f75d801a5f0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.591+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 shutdown_connections 2024-05-10T05:40:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.591+0000 7f75feceab80 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55c409afd000 0x55c409aff420 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.591+0000 7f75feceab80 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c409a92ec0 0x55c409a9fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.591+0000 7f75feceab80 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c409a916a0 0x55c409aa2cc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.591+0000 7f75feceab80 1 --2- 172.21.15.131:0/1899686169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c409a92570 0x55c409a9f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.591+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 >> 172.21.15.131:0/1899686169 conn(0x55c4099f3a60 msgr2=0x55c409a957a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.591+0000 7f75ca000700 0 client.26570 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:40:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.592+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 shutdown_connections 2024-05-10T05:40:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.592+0000 7f75feceab80 1 -- 172.21.15.131:0/1899686169 wait complete. 2024-05-10T05:40:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.593+0000 7f75feceab80 20 client.26570 trim_cache size 0 max 16384 2024-05-10T05:40:01.595 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:40:01.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.597+0000 7f75e6e00700 1 -- 172.21.15.131:0/1059948480 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75c40a1290 msgr2=0x7f75c40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:01.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.597+0000 7f75e6e00700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75c40a1290 0x7f75c40a37a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f75cc013570 tx=0x7f75cc009800 comp rx=0 tx=0).stop 2024-05-10T05:40:01.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.597+0000 7f75e6e00700 1 -- 172.21.15.131:0/1059948480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e0104f20 msgr2=0x7f75e01962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:01.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.597+0000 7f75e6e00700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e0104f20 0x7f75e01962b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f75d0005950 tx=0x7f75d000b790 comp rx=0 tx=0).stop 2024-05-10T05:40:01.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.597+0000 7f75e6e00700 1 -- 172.21.15.131:0/1059948480 shutdown_connections 2024-05-10T05:40:01.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.598+0000 7f75e6e00700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75c40a1290 0x7f75c40a37a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.598+0000 7f75e6e00700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75e0103e00 0x7f75e0195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.598+0000 7f75e6e00700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e01058c0 0x7f75e019d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.598+0000 7f75e6e00700 1 --2- 172.21.15.131:0/1059948480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75e0104f20 0x7f75e01962b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:01.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.598+0000 7f75e6e00700 1 -- 172.21.15.131:0/1059948480 >> 172.21.15.131:0/1059948480 conn(0x7f75e00ff410 msgr2=0x7f75e0077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:01.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.599+0000 7f75e6e00700 1 -- 172.21.15.131:0/1059948480 shutdown_connections 2024-05-10T05:40:01.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:01.599+0000 7f75e6e00700 1 -- 172.21.15.131:0/1059948480 wait complete. 2024-05-10T05:40:01.676 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:40:02.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.454+0000 7ff7e3000700 1 -- 172.21.15.131:0/4151614692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7dc1036a0 msgr2=0x7ff7dc109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:02.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.454+0000 7ff7e3000700 1 --2- 172.21.15.131:0/4151614692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7dc1036a0 0x7ff7dc109f50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009f90 tx=0x7ff7d400b330 comp rx=0 tx=0).stop 2024-05-10T05:40:02.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.455+0000 7ff7e3000700 1 -- 172.21.15.131:0/4151614692 shutdown_connections 2024-05-10T05:40:02.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.455+0000 7ff7e3000700 1 --2- 172.21.15.131:0/4151614692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7dc1036a0 0x7ff7dc109f50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.455+0000 7ff7e3000700 1 --2- 172.21.15.131:0/4151614692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7dc101be0 0x7ff7dc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.455+0000 7ff7e3000700 1 --2- 172.21.15.131:0/4151614692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7dc102d00 0x7ff7dc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.455+0000 7ff7e3000700 1 -- 172.21.15.131:0/4151614692 >> 172.21.15.131:0/4151614692 conn(0x7ff7dc0fd1d0 msgr2=0x7ff7dc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:02.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.455+0000 7ff7e3000700 1 -- 172.21.15.131:0/4151614692 shutdown_connections 2024-05-10T05:40:02.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.455+0000 7ff7e3000700 1 -- 172.21.15.131:0/4151614692 wait complete. 2024-05-10T05:40:02.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.457+0000 7ff7e3000700 1 Processor -- start 2024-05-10T05:40:02.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.457+0000 7ff7e3000700 1 -- start start 2024-05-10T05:40:02.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.457+0000 7ff7e3000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7dc101be0 0x7ff7dc197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.457+0000 7ff7e3000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7dc102d00 0x7ff7dc198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.457+0000 7ff7e3000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7dc1036a0 0x7ff7dc19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.457+0000 7ff7e3000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc19fad0 con 0x7ff7dc101be0 2024-05-10T05:40:02.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.457+0000 7ff7e3000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc19fc40 con 0x7ff7dc1036a0 2024-05-10T05:40:02.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.457+0000 7ff7e3000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc19fe00 con 0x7ff7dc102d00 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7e1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7dc1036a0 0x7ff7dc19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7db400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7dc102d00 0x7ff7dc198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7e1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7dc1036a0 0x7ff7dc19f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:58000/0 (socket says 172.21.15.131:58000) 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7db400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7dc102d00 0x7ff7dc198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34704/0 (socket says 172.21.15.131:34704) 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7e1000700 1 -- 172.21.15.131:0/3627220970 learned_addr learned my addr 172.21.15.131:0/3627220970 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7dc101be0 0x7ff7dc197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7e1000700 1 -- 172.21.15.131:0/3627220970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7dc102d00 msgr2=0x7ff7dc198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7e1000700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7dc102d00 0x7ff7dc198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7e1000700 1 -- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7dc101be0 msgr2=0x7ff7dc197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7e1000700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7dc101be0 0x7ff7dc197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7e1000700 1 -- 172.21.15.131:0/3627220970 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4009c40 con 0x7ff7dc1036a0 2024-05-10T05:40:02.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7dc101be0 0x7ff7dc197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:02.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.458+0000 7ff7db400700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7dc102d00 0x7ff7dc198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:40:02.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.459+0000 7ff7e1000700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7dc1036a0 0x7ff7dc19f4c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400b6c0 tx=0x7ff7d4004d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:02.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.459+0000 7ff7d8c00700 1 -- 172.21.15.131:0/3627220970 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d400e040 con 0x7ff7dc1036a0 2024-05-10T05:40:02.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.459+0000 7ff7d8c00700 1 -- 172.21.15.131:0/3627220970 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7d4008e80 con 0x7ff7dc1036a0 2024-05-10T05:40:02.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.459+0000 7ff7d8c00700 1 -- 172.21.15.131:0/3627220970 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40052a0 con 0x7ff7dc1036a0 2024-05-10T05:40:02.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.460+0000 7ff7e3000700 1 -- 172.21.15.131:0/3627220970 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc1a0080 con 0x7ff7dc1036a0 2024-05-10T05:40:02.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.461+0000 7ff7e3000700 1 -- 172.21.15.131:0/3627220970 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc1a04b0 con 0x7ff7dc1036a0 2024-05-10T05:40:02.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.466+0000 7ff7d8c00700 1 -- 172.21.15.131:0/3627220970 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff7d401d470 con 0x7ff7dc1036a0 2024-05-10T05:40:02.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.468+0000 7ff7faf35b80 1 Processor -- start 2024-05-10T05:40:02.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.468+0000 7ff7faf35b80 1 -- start start 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.468+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d55fd6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.468+0000 7ff7d8c00700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7c00a12c0 0x7ff7c00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.468+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d55fd7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.468+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fdebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.468+0000 7ff7faf35b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55ef9150 con 0x557d55fd6ab0 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.468+0000 7ff7faf35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55ef6b10 con 0x557d55fd7980 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.468+0000 7ff7faf35b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55ef2400 con 0x557d55fd82d0 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.469+0000 7ff7d8c00700 1 -- 172.21.15.131:0/3627220970 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff7d40cd600 con 0x7ff7dc1036a0 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.469+0000 7ff7e1000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fdebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.469+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7c00a12c0 0x7ff7c00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.469+0000 7ff7e1000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fdebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34714/0 (socket says 172.21.15.131:34714) 2024-05-10T05:40:02.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.469+0000 7ff7e1000700 1 -- 172.21.15.131:0/4086355942 learned_addr learned my addr 172.21.15.131:0/4086355942 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:02.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.470+0000 7ff7db400700 1 --2- 172.21.15.131:0/4086355942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d55fd6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.470+0000 7ff7e1000700 1 -- 172.21.15.131:0/4086355942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 msgr2=0x557d55fd7d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:40:02.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.470+0000 7ff7e1000700 1 --2- 172.21.15.131:0/4086355942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d55fd7d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.470+0000 7ff7e1000700 1 -- 172.21.15.131:0/4086355942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 msgr2=0x557d55fd6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:02.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.470+0000 7ff7e1000700 1 --2- 172.21.15.131:0/4086355942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d55fd6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.470+0000 7ff7e1000700 1 -- 172.21.15.131:0/4086355942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d55d733b0 con 0x557d55fd82d0 2024-05-10T05:40:02.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.470+0000 7ff7db400700 1 --2- 172.21.15.131:0/4086355942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d55fd6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:02.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.470+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7c00a12c0 0x7ff7c00a37d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc102c00 tx=0x7ff7cc009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:02.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.471+0000 7ff7e1000700 1 --2- 172.21.15.131:0/4086355942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fdebd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4097580 tx=0x7ff7d4007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=228961dbc8ca42ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:02.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.471+0000 7ff7d1600700 1 -- 172.21.15.131:0/4086355942 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d409b210 con 0x557d55fd82d0 2024-05-10T05:40:02.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.471+0000 7ff7d1600700 1 -- 172.21.15.131:0/4086355942 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7d409ab10 con 0x557d55fd82d0 2024-05-10T05:40:02.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.471+0000 7ff7d1600700 1 -- 172.21.15.131:0/4086355942 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d409ad70 con 0x557d55fd82d0 2024-05-10T05:40:02.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.473+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4086355942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 msgr2=0x557d55fdebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:02.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.473+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/4086355942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fdebd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4097580 tx=0x7ff7d4007930 comp rx=0 tx=0).stop 2024-05-10T05:40:02.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.473+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4086355942 shutdown_connections 2024-05-10T05:40:02.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.473+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/4086355942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d55fd6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.473+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/4086355942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fdebd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.473+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/4086355942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d55fd7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.473+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4086355942 >> 172.21.15.131:0/4086355942 conn(0x557d55f34a60 msgr2=0x557d55fccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:02.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.473+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4086355942 shutdown_connections 2024-05-10T05:40:02.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.473+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4086355942 wait complete. 2024-05-10T05:40:02.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.474+0000 7ff7faf35b80 1 -- start start 2024-05-10T05:40:02.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.475+0000 7ff7faf35b80 1 Processor -- start 2024-05-10T05:40:02.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.476+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd6ab0 0x557d55fd5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.476+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd7980 0x557d55fd5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.476+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fd2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.476+0000 7ff7faf35b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55e8e560 con 0x557d55fd7980 2024-05-10T05:40:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.476+0000 7ff7faf35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55e8e200 con 0x557d55fd6ab0 2024-05-10T05:40:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.476+0000 7ff7faf35b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55d756a0 con 0x557d55fd82d0 2024-05-10T05:40:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.476+0000 7ff7faf35b80 10 client.0 _ll_register_callbacks cb 0x557d55e97b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.476+0000 7ff7dbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd6ab0 0x557d55fd5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.476+0000 7ff7dbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd6ab0 0x557d55fd5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:58008/0 (socket says 172.21.15.131:58008) 2024-05-10T05:40:02.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.476+0000 7ff7dbe00700 1 -- 172.21.15.131:0/147775562 learned_addr learned my addr 172.21.15.131:0/147775562 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.477+0000 7ff7dbe00700 1 -- 172.21.15.131:0/147775562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 msgr2=0x557d55fd2630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:40:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.477+0000 7ff7e1000700 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fd2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.477+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fd2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.477+0000 7ff7dbe00700 1 -- 172.21.15.131:0/147775562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd7980 msgr2=0x557d55fd5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.477+0000 7ff7db400700 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd7980 0x557d55fd5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.477+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd7980 0x557d55fd5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.477+0000 7ff7dbe00700 1 -- 172.21.15.131:0/147775562 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc1a04b0 con 0x557d55fd6ab0 2024-05-10T05:40:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.477+0000 7ff7e1000700 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fd2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.477+0000 7ff7db400700 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd7980 0x557d55fd5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.478+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd6ab0 0x557d55fd5ab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc010e40 tx=0x7ff7cc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ef007586560dca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.478+0000 7ff7c6000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.478+0000 7ff7c6000700 1 -- 172.21.15.131:0/147775562 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc0098a0 con 0x557d55fd6ab0 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.478+0000 7ff7c6000700 1 -- 172.21.15.131:0/147775562 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff7cc01f450 con 0x557d55fd6ab0 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.478+0000 7ff7faf35b80 1 -- 172.21.15.131:0/147775562 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d55e4b270 con 0x557d55fd6ab0 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.478+0000 7ff7c6000700 1 -- 172.21.15.131:0/147775562 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7cc01f610 con 0x557d55fd6ab0 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.479+0000 7ff7faf35b80 10 client.26579 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.479+0000 7ff7faf35b80 1 -- 172.21.15.131:0/147775562 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557d55d1fb60 con 0x557d55fd6ab0 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.479+0000 7ff7c6000700 1 -- 172.21.15.131:0/147775562 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d55e4b270 con 0x557d55fd6ab0 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.479+0000 7ff7c6000700 1 -- 172.21.15.131:0/147775562 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff7cc027a60 con 0x557d55fd6ab0 2024-05-10T05:40:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.479+0000 7ff7faf35b80 10 client.26579 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:02.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.479+0000 7ff7faf35b80 10 client.26579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:02.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.479+0000 7ff7faf35b80 20 client.26579 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:02.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.479+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557d55ff0600 0x557d55ff2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.479+0000 7ff7faf35b80 4 client.26579 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:02.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.479+0000 7ff7faf35b80 1 -- 172.21.15.131:0/147775562 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d55fa1780 con 0x557d55ff0600 2024-05-10T05:40:02.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.480+0000 7ff7db400700 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557d55ff0600 0x557d55ff2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.480+0000 7ff7c6000700 10 client.26579 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:02.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.482+0000 7ff7db400700 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557d55ff0600 0x557d55ff2a50 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd30786b7da4a0d3 server_cookie=9d506c65a0bd49e in_seq=0 out_seq=0 2024-05-10T05:40:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.483+0000 7ff7c6000700 1 -- 172.21.15.131:0/147775562 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d55fa1780 con 0x557d55ff0600 2024-05-10T05:40:02.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.483+0000 7ff7c6000700 10 client.26579 handle_command_reply: tid=0 2024-05-10T05:40:02.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.483+0000 7ff7faf35b80 1 client.26579 shutdown 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.483+0000 7ff7faf35b80 1 -- 172.21.15.131:0/147775562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd6ab0 msgr2=0x557d55fd5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.483+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd6ab0 0x557d55fd5ab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc010e40 tx=0x7ff7cc004750 comp rx=0 tx=0).stop 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.483+0000 7ff7faf35b80 1 -- 172.21.15.131:0/147775562 shutdown_connections 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.483+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557d55ff0600 0x557d55ff2a50 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.484+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55fd2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.484+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd6ab0 0x557d55fd5ab0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.484+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/147775562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd7980 0x557d55fd5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.484+0000 7ff7faf35b80 1 -- 172.21.15.131:0/147775562 >> 172.21.15.131:0/147775562 conn(0x557d55f34a60 msgr2=0x557d55fccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.484+0000 7ff7c6000700 0 client.26579 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.484+0000 7ff7faf35b80 1 -- 172.21.15.131:0/147775562 shutdown_connections 2024-05-10T05:40:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.484+0000 7ff7faf35b80 1 -- 172.21.15.131:0/147775562 wait complete. 2024-05-10T05:40:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.484+0000 7ff7faf35b80 20 client.26579 trim_cache size 0 max 16384 2024-05-10T05:40:02.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.512+0000 7ff7faf35b80 1 Processor -- start 2024-05-10T05:40:02.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.512+0000 7ff7faf35b80 1 -- start start 2024-05-10T05:40:02.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.513+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d560007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.513+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d56000d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.513+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55ff2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.513+0000 7ff7faf35b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55fed030 con 0x557d55fd6ab0 2024-05-10T05:40:02.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.513+0000 7ff7faf35b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55ff7b50 con 0x557d55fd7980 2024-05-10T05:40:02.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.513+0000 7ff7faf35b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55fccc90 con 0x557d55fd82d0 2024-05-10T05:40:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.514+0000 7ff7db400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d56000d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.514+0000 7ff7db400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d56000d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:58012/0 (socket says 172.21.15.131:58012) 2024-05-10T05:40:02.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.514+0000 7ff7db400700 1 -- 172.21.15.131:0/1561579904 learned_addr learned my addr 172.21.15.131:0/1561579904 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:02.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.514+0000 7ff7e1000700 1 --2- 172.21.15.131:0/1561579904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55ff2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.514+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/1561579904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d560007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.515+0000 7ff7db400700 1 -- 172.21.15.131:0/1561579904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 msgr2=0x557d55ff2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:02.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.515+0000 7ff7db400700 1 --2- 172.21.15.131:0/1561579904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55ff2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.515+0000 7ff7db400700 1 -- 172.21.15.131:0/1561579904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 msgr2=0x557d560007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:02.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.515+0000 7ff7db400700 1 --2- 172.21.15.131:0/1561579904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d560007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.515+0000 7ff7db400700 1 -- 172.21.15.131:0/1561579904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d55eb8560 con 0x557d55fd7980 2024-05-10T05:40:02.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.515+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/1561579904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d560007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:02.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.515+0000 7ff7db400700 1 --2- 172.21.15.131:0/1561579904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d56000d20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc199090 tx=0x7ff7c8009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=276c617fdb3cf1e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:02.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.516+0000 7ff7d2000700 1 -- 172.21.15.131:0/1561579904 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8016040 con 0x557d55fd7980 2024-05-10T05:40:02.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.516+0000 7ff7d2000700 1 -- 172.21.15.131:0/1561579904 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c800fba0 con 0x557d55fd7980 2024-05-10T05:40:02.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.516+0000 7ff7d2000700 1 -- 172.21.15.131:0/1561579904 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c800ec40 con 0x557d55fd7980 2024-05-10T05:40:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.517+0000 7ff7faf35b80 1 -- 172.21.15.131:0/1561579904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 msgr2=0x557d56000d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.517+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/1561579904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d56000d20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc199090 tx=0x7ff7c8009fd0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.518+0000 7ff7faf35b80 1 -- 172.21.15.131:0/1561579904 shutdown_connections 2024-05-10T05:40:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.518+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/1561579904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d560007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.518+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/1561579904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d55ff2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.518+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/1561579904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d56000d20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.518+0000 7ff7faf35b80 1 -- 172.21.15.131:0/1561579904 >> 172.21.15.131:0/1561579904 conn(0x557d55f34a60 msgr2=0x557d55fccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.518+0000 7ff7faf35b80 1 -- 172.21.15.131:0/1561579904 shutdown_connections 2024-05-10T05:40:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.518+0000 7ff7faf35b80 1 -- 172.21.15.131:0/1561579904 wait complete. 2024-05-10T05:40:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.519+0000 7ff7faf35b80 1 -- start start 2024-05-10T05:40:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.519+0000 7ff7faf35b80 1 Processor -- start 2024-05-10T05:40:02.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.523+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d55ffdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.523+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d55ffe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7faf35b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d560564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7db400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d55ffe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7db400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d55ffe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:58020/0 (socket says 172.21.15.131:58020) 2024-05-10T05:40:02.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7db400700 1 -- 172.21.15.131:0/4260604149 learned_addr learned my addr 172.21.15.131:0/4260604149 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:02.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55ffe670 con 0x557d55fd6ab0 2024-05-10T05:40:02.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55ffe7e0 con 0x557d55fd7980 2024-05-10T05:40:02.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d56056a70 con 0x557d55fd82d0 2024-05-10T05:40:02.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7e1000700 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d560564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7faf35b80 10 client.0 _ll_register_callbacks cb 0x557d55e97b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:02.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7db400700 1 -- 172.21.15.131:0/4260604149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 msgr2=0x557d560564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:02.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.524+0000 7ff7db400700 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d560564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.525+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d55ffdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.525+0000 7ff7db400700 1 -- 172.21.15.131:0/4260604149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 msgr2=0x557d55ffdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:02.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.525+0000 7ff7db400700 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d55ffdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.525+0000 7ff7db400700 1 -- 172.21.15.131:0/4260604149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c8009e40 con 0x557d55fd7980 2024-05-10T05:40:02.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.525+0000 7ff7db400700 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d55ffe0e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8020000 tx=0x7ff7c800ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6498bd3dbcfedbb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:02.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.526+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7d409ab10 con 0x557d55fd7980 2024-05-10T05:40:02.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.526+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d55ffdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:02.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.527+0000 7ff7c6000700 10 client.26585 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:02.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.527+0000 7ff7c6000700 1 -- 172.21.15.131:0/4260604149 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8016050 con 0x557d55fd7980 2024-05-10T05:40:02.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.527+0000 7ff7c6000700 1 -- 172.21.15.131:0/4260604149 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff7c800edd0 con 0x557d55fd7980 2024-05-10T05:40:02.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.527+0000 7ff7c6000700 1 -- 172.21.15.131:0/4260604149 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7c800f4e0 con 0x557d55fd7980 2024-05-10T05:40:02.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.527+0000 7ff7faf35b80 10 client.26585 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:02.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.527+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x557d55d77cd0 con 0x557d55fd7980 2024-05-10T05:40:02.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.531+0000 7ff7c6000700 1 -- 172.21.15.131:0/4260604149 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff7c8019070 con 0x557d55fd7980 2024-05-10T05:40:02.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.531+0000 7ff7faf35b80 10 client.26585 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:02.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.531+0000 7ff7faf35b80 10 client.26585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:02.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.531+0000 7ff7faf35b80 20 client.26585 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:02.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.531+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557d56058090 0x557d5605a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:02.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.531+0000 7ff7faf35b80 4 client.26585 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:40:02.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.532+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d55eeaec0 con 0x557d56058090 2024-05-10T05:40:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.532+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557d56058090 0x557d5605a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.534+0000 7ff7c6000700 10 client.26585 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:02.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.534+0000 7ff7dbe00700 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557d56058090 0x557d5605a4d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7066e65d25a1e96c server_cookie=b5cf5395a5b67165 in_seq=0 out_seq=0 2024-05-10T05:40:02.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.535+0000 7ff7c6000700 1 -- 172.21.15.131:0/4260604149 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d55eeaec0 con 0x557d56058090 2024-05-10T05:40:02.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.535+0000 7ff7c6000700 10 client.26585 handle_command_reply: tid=0 2024-05-10T05:40:02.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.535+0000 7ff7faf35b80 1 client.26585 shutdown 2024-05-10T05:40:02.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.535+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 msgr2=0x557d55ffe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:02.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.535+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d55ffe0e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8020000 tx=0x7ff7c800ff00 comp rx=0 tx=0).stop 2024-05-10T05:40:02.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.536+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 shutdown_connections 2024-05-10T05:40:02.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.536+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x557d56058090 0x557d5605a4d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.536+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d55fd6ab0 0x557d55ffdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.536+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d55fd82d0 0x557d560564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.536+0000 7ff7faf35b80 1 --2- 172.21.15.131:0/4260604149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55fd7980 0x557d55ffe0e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.536+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 >> 172.21.15.131:0/4260604149 conn(0x557d55f34a60 msgr2=0x557d55fdb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:02.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.536+0000 7ff7c6000700 0 client.26585 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:02.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.536+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 shutdown_connections 2024-05-10T05:40:02.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.537+0000 7ff7faf35b80 1 -- 172.21.15.131:0/4260604149 wait complete. 2024-05-10T05:40:02.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.537+0000 7ff7faf35b80 20 client.26585 trim_cache size 0 max 16384 2024-05-10T05:40:02.539 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:40:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.539+0000 7ff7e3000700 1 -- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7c00a12c0 msgr2=0x7ff7c00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.539+0000 7ff7e3000700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7c00a12c0 0x7ff7c00a37d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc102c00 tx=0x7ff7cc009310 comp rx=0 tx=0).stop 2024-05-10T05:40:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.540+0000 7ff7e3000700 1 -- 172.21.15.131:0/3627220970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7dc1036a0 msgr2=0x7ff7dc19f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.540+0000 7ff7e3000700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7dc1036a0 0x7ff7dc19f4c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400b6c0 tx=0x7ff7d4004d30 comp rx=0 tx=0).stop 2024-05-10T05:40:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.540+0000 7ff7e3000700 1 -- 172.21.15.131:0/3627220970 shutdown_connections 2024-05-10T05:40:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.540+0000 7ff7e3000700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7c00a12c0 0x7ff7c00a37d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.540+0000 7ff7e3000700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7dc101be0 0x7ff7dc197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.540+0000 7ff7e3000700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7dc1036a0 0x7ff7dc19f4c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.540+0000 7ff7e3000700 1 --2- 172.21.15.131:0/3627220970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7dc102d00 0x7ff7dc198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.540+0000 7ff7e3000700 1 -- 172.21.15.131:0/3627220970 >> 172.21.15.131:0/3627220970 conn(0x7ff7dc0fd1d0 msgr2=0x7ff7dc105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:02.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.542+0000 7ff7e3000700 1 -- 172.21.15.131:0/3627220970 shutdown_connections 2024-05-10T05:40:02.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:02.542+0000 7ff7e3000700 1 -- 172.21.15.131:0/3627220970 wait complete. 2024-05-10T05:40:02.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:02 smithi157 ceph-mon[78427]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 262 MiB/s wr, 80 op/s 2024-05-10T05:40:02.622 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:02 smithi131 ceph-mon[37264]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 262 MiB/s wr, 80 op/s 2024-05-10T05:40:02.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:02 smithi142 ceph-mon[77667]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 262 MiB/s wr, 80 op/s 2024-05-10T05:40:03.628 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:40:03.628 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:40:04.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.346+0000 7f08a9800700 1 -- 172.21.15.131:0/2648142220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08a4075140 msgr2=0x7f08a4073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:04.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.347+0000 7f08a9800700 1 --2- 172.21.15.131:0/2648142220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08a4075140 0x7f08a4073600 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f08940098f0 tx=0x7f0894009c00 comp rx=0 tx=0).stop 2024-05-10T05:40:04.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.347+0000 7f08a9800700 1 -- 172.21.15.131:0/2648142220 shutdown_connections 2024-05-10T05:40:04.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.347+0000 7f08a9800700 1 --2- 172.21.15.131:0/2648142220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08a4075140 0x7f08a4073600 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.348+0000 7f08a9800700 1 --2- 172.21.15.131:0/2648142220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a410e220 0x7f08a4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.348+0000 7f08a9800700 1 --2- 172.21.15.131:0/2648142220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a4073b40 0x7f08a4074010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.348+0000 7f08a9800700 1 -- 172.21.15.131:0/2648142220 >> 172.21.15.131:0/2648142220 conn(0x7f08a40fd210 msgr2=0x7f08a40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:04.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.348+0000 7f08a9800700 1 -- 172.21.15.131:0/2648142220 shutdown_connections 2024-05-10T05:40:04.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.348+0000 7f08a9800700 1 -- 172.21.15.131:0/2648142220 wait complete. 2024-05-10T05:40:04.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.352+0000 7f08a9800700 1 Processor -- start 2024-05-10T05:40:04.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.353+0000 7f08a9800700 1 -- start start 2024-05-10T05:40:04.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.355+0000 7f08a9800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a4073b40 0x7f08a419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.355+0000 7f08a9800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a410e220 0x7f08a419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.356+0000 7f08a2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a4073b40 0x7f08a419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.356+0000 7f08a2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a4073b40 0x7f08a419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34888/0 (socket says 172.21.15.131:34888) 2024-05-10T05:40:04.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.356+0000 7f08a2a00700 1 -- 172.21.15.131:0/727888034 learned_addr learned my addr 172.21.15.131:0/727888034 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:04.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.356+0000 7f08a2000700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a410e220 0x7f08a419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.356+0000 7f08a9800700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08a41a3b50 0x7f08a41a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.357+0000 7f08a3400700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08a41a3b50 0x7f08a41a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.357+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a41a6650 con 0x7f08a41a3b50 2024-05-10T05:40:04.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.357+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a41a67c0 con 0x7f08a4073b40 2024-05-10T05:40:04.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.358+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a41a6980 con 0x7f08a410e220 2024-05-10T05:40:04.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.359+0000 7f08a2000700 1 -- 172.21.15.131:0/727888034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a4073b40 msgr2=0x7f08a419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:04.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.359+0000 7f08a2000700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a4073b40 0x7f08a419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.359+0000 7f08a2000700 1 -- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08a41a3b50 msgr2=0x7f08a41a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:04.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.359+0000 7f08a2000700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08a41a3b50 0x7f08a41a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.359+0000 7f08a2000700 1 -- 172.21.15.131:0/727888034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0894009520 con 0x7f08a410e220 2024-05-10T05:40:04.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.359+0000 7f08a2000700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a410e220 0x7f08a419ec90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f089000e9c0 tx=0x7f089000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:04.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.360+0000 7f089b400700 1 -- 172.21.15.131:0/727888034 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089000c9c0 con 0x7f08a410e220 2024-05-10T05:40:04.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.360+0000 7f089b400700 1 -- 172.21.15.131:0/727888034 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0890010430 con 0x7f08a410e220 2024-05-10T05:40:04.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.360+0000 7f089b400700 1 -- 172.21.15.131:0/727888034 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890018680 con 0x7f08a410e220 2024-05-10T05:40:04.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.361+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a41a6c60 con 0x7f08a410e220 2024-05-10T05:40:04.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.362+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a41a7060 con 0x7f08a410e220 2024-05-10T05:40:04.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.366+0000 7f089b400700 1 -- 172.21.15.131:0/727888034 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f08900187e0 con 0x7f08a410e220 2024-05-10T05:40:04.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.369+0000 7f089b400700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08880a1330 0x7f08880a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.370+0000 7f08a2a00700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08880a1330 0x7f08880a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.373+0000 7f089b400700 1 -- 172.21.15.131:0/727888034 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0890014070 con 0x7f08a410e220 2024-05-10T05:40:04.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.374+0000 7f08a2a00700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08880a1330 0x7f08880a3840 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f08940098c0 tx=0x7f0894019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:04.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.389+0000 7f08c1627b80 1 Processor -- start 2024-05-10T05:40:04.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.389+0000 7f08c1627b80 1 -- start start 2024-05-10T05:40:04.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.390+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802b720 0x56493802bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.390+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802c5f0 0x56493802ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.390+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802cf40 0x564938033840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.390+0000 7f08c1627b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564937f521d0 con 0x56493802cf40 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.390+0000 7f08c1627b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564937f4fb90 con 0x56493802b720 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.390+0000 7f08c1627b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564937f4b480 con 0x56493802c5f0 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.391+0000 7f08a3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802cf40 0x564938033840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.391+0000 7f08a2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802c5f0 0x56493802ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.391+0000 7f08a2000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802c5f0 0x56493802ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53898/0 (socket says 172.21.15.131:53898) 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.391+0000 7f08a2000700 1 -- 172.21.15.131:0/3114704083 learned_addr learned my addr 172.21.15.131:0/3114704083 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.392+0000 7f08a3400700 1 -- 172.21.15.131:0/3114704083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802c5f0 msgr2=0x56493802ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.392+0000 7f08a3400700 1 --2- 172.21.15.131:0/3114704083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802c5f0 0x56493802ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.392+0000 7f08a3400700 1 -- 172.21.15.131:0/3114704083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802b720 msgr2=0x56493802bb70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:40:04.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.392+0000 7f08a3400700 1 --2- 172.21.15.131:0/3114704083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802b720 0x56493802bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.392+0000 7f08a3400700 1 -- 172.21.15.131:0/3114704083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564937dcc430 con 0x56493802cf40 2024-05-10T05:40:04.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.393+0000 7f08a3400700 1 --2- 172.21.15.131:0/3114704083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802cf40 0x564938033840 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f089c011580 tx=0x7f089c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebcb70ff43a7bce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:04.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.394+0000 7f088fe00700 1 -- 172.21.15.131:0/3114704083 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c00e040 con 0x56493802cf40 2024-05-10T05:40:04.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.394+0000 7f088fe00700 1 -- 172.21.15.131:0/3114704083 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f089c00f040 con 0x56493802cf40 2024-05-10T05:40:04.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.394+0000 7f088fe00700 1 -- 172.21.15.131:0/3114704083 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c0044f0 con 0x56493802cf40 2024-05-10T05:40:04.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.396+0000 7f08c1627b80 1 -- 172.21.15.131:0/3114704083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802cf40 msgr2=0x564938033840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:04.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.396+0000 7f08c1627b80 1 --2- 172.21.15.131:0/3114704083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802cf40 0x564938033840 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f089c011580 tx=0x7f089c009920 comp rx=0 tx=0).stop 2024-05-10T05:40:04.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.403+0000 7f08c1627b80 1 -- 172.21.15.131:0/3114704083 shutdown_connections 2024-05-10T05:40:04.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.404+0000 7f08c1627b80 1 --2- 172.21.15.131:0/3114704083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802cf40 0x564938033840 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.404+0000 7f08c1627b80 1 --2- 172.21.15.131:0/3114704083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802b720 0x56493802bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.404+0000 7f08c1627b80 1 --2- 172.21.15.131:0/3114704083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802c5f0 0x56493802ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.404+0000 7f08c1627b80 1 -- 172.21.15.131:0/3114704083 >> 172.21.15.131:0/3114704083 conn(0x564937f8dae0 msgr2=0x564938026060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:04.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.405+0000 7f08c1627b80 1 -- 172.21.15.131:0/3114704083 shutdown_connections 2024-05-10T05:40:04.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.406+0000 7f08c1627b80 1 -- 172.21.15.131:0/3114704083 wait complete. 2024-05-10T05:40:04.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.412+0000 7f08c1627b80 1 -- start start 2024-05-10T05:40:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.414+0000 7f08c1627b80 1 Processor -- start 2024-05-10T05:40:04.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.415+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x564938035780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.416+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802c5f0 0x564938035cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.416+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802cf40 0x56493803cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.416+0000 7f08c1627b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564937ee75e0 con 0x56493802b720 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.416+0000 7f08c1627b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564937ee7280 con 0x56493802cf40 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.416+0000 7f08c1627b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564937dce720 con 0x56493802c5f0 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.416+0000 7f08c1627b80 10 client.0 _ll_register_callbacks cb 0x564937ef0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.416+0000 7f08a2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x564938035780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.416+0000 7f08a2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x564938035780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38790/0 (socket says 172.21.15.131:38790) 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.416+0000 7f08a2a00700 1 -- 172.21.15.131:0/1859166727 learned_addr learned my addr 172.21.15.131:0/1859166727 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.417+0000 7f08a3400700 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802cf40 0x56493803cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.417+0000 7f08a2a00700 1 -- 172.21.15.131:0/1859166727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802c5f0 msgr2=0x564938035cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.417+0000 7f08a2a00700 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802c5f0 0x564938035cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.417+0000 7f08a2a00700 1 -- 172.21.15.131:0/1859166727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802cf40 msgr2=0x56493803cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:04.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.417+0000 7f08a2a00700 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802cf40 0x56493803cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.417+0000 7f08a2a00700 1 -- 172.21.15.131:0/1859166727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c011230 con 0x56493802b720 2024-05-10T05:40:04.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.417+0000 7f08a3400700 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802cf40 0x56493803cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.419+0000 7f08a2a00700 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x564938035780 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0894009230 tx=0x7f0894009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaee096737cbbaf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.419+0000 7f088cc00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:40:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.419+0000 7f088cc00700 1 -- 172.21.15.131:0/1859166727 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894028070 con 0x56493802b720 2024-05-10T05:40:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.419+0000 7f08c1627b80 1 -- 172.21.15.131:0/1859166727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564937ea42f0 con 0x56493802b720 2024-05-10T05:40:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.419+0000 7f088cc00700 1 -- 172.21.15.131:0/1859166727 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0894005250 con 0x56493802b720 2024-05-10T05:40:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.420+0000 7f088cc00700 1 -- 172.21.15.131:0/1859166727 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f089401c040 con 0x56493802b720 2024-05-10T05:40:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.420+0000 7f08c1627b80 10 client.16383 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.420+0000 7f08c1627b80 1 -- 172.21.15.131:0/1859166727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564937d78be0 con 0x56493802b720 2024-05-10T05:40:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.421+0000 7f088cc00700 1 -- 172.21.15.131:0/1859166727 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564937ea42f0 con 0x56493802b720 2024-05-10T05:40:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.422+0000 7f088cc00700 1 -- 172.21.15.131:0/1859166727 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0894035070 con 0x56493802b720 2024-05-10T05:40:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.422+0000 7f08c1627b80 10 client.16383 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.422+0000 7f08c1627b80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.423+0000 7f08c1627b80 20 client.16383 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.423+0000 7f08c1627b80 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564938041340 0x564938043790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.423+0000 7f08c1627b80 4 client.16383 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.423+0000 7f08c1627b80 1 -- 172.21.15.131:0/1859166727 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564937ffa800 con 0x564938041340 2024-05-10T05:40:04.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.424+0000 7f08a2000700 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564938041340 0x564938043790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.425+0000 7f088cc00700 10 client.16383 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:04.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.425+0000 7f08a2000700 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564938041340 0x564938043790 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b19c7ca86e5d2c server_cookie=2f4970e13ffe77d7 in_seq=0 out_seq=0 2024-05-10T05:40:04.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.426+0000 7f088cc00700 1 -- 172.21.15.131:0/1859166727 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564937ffa800 con 0x564938041340 2024-05-10T05:40:04.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.426+0000 7f088cc00700 10 client.16383 handle_command_reply: tid=0 2024-05-10T05:40:04.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.426+0000 7f08c1627b80 1 client.16383 shutdown 2024-05-10T05:40:04.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.428+0000 7f08c1627b80 1 -- 172.21.15.131:0/1859166727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 msgr2=0x564938035780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:04.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.428+0000 7f08c1627b80 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x564938035780 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0894009230 tx=0x7f0894009260 comp rx=0 tx=0).stop 2024-05-10T05:40:04.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.428+0000 7f08c1627b80 1 -- 172.21.15.131:0/1859166727 shutdown_connections 2024-05-10T05:40:04.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.428+0000 7f08c1627b80 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564938041340 0x564938043790 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.428+0000 7f08c1627b80 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x564938035780 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.428+0000 7f08c1627b80 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802cf40 0x56493803cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.428+0000 7f08c1627b80 1 --2- 172.21.15.131:0/1859166727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802c5f0 0x564938035cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.428+0000 7f08c1627b80 1 -- 172.21.15.131:0/1859166727 >> 172.21.15.131:0/1859166727 conn(0x564937f8dae0 msgr2=0x564938026030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:04.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.429+0000 7f088cc00700 0 client.16383 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:04.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.429+0000 7f08c1627b80 1 -- 172.21.15.131:0/1859166727 shutdown_connections 2024-05-10T05:40:04.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.430+0000 7f08c1627b80 1 -- 172.21.15.131:0/1859166727 wait complete. 2024-05-10T05:40:04.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.431+0000 7f08c1627b80 20 client.16383 trim_cache size 0 max 16384 2024-05-10T05:40:04.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.466+0000 7f08c1627b80 1 Processor -- start 2024-05-10T05:40:04.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.466+0000 7f08c1627b80 1 -- start start 2024-05-10T05:40:04.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.467+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802b720 0x5649380347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.467+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802c5f0 0x56493802aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.467+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 0x56493803c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.467+0000 7f08c1627b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564938020840 con 0x56493802c5f0 2024-05-10T05:40:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.467+0000 7f08c1627b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564938026b50 con 0x56493802b720 2024-05-10T05:40:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.467+0000 7f08c1627b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564938041060 con 0x56493802cf40 2024-05-10T05:40:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.467+0000 7f08a3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 0x56493803c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.467+0000 7f08a2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802b720 0x5649380347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.467+0000 7f08a3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 0x56493803c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53928/0 (socket says 172.21.15.131:53928) 2024-05-10T05:40:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.467+0000 7f08a3400700 1 -- 172.21.15.131:0/553856947 learned_addr learned my addr 172.21.15.131:0/553856947 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.468+0000 7f08a2000700 1 --2- 172.21.15.131:0/553856947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802c5f0 0x56493802aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.468+0000 7f08a2a00700 1 -- 172.21.15.131:0/553856947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 msgr2=0x56493803c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.468+0000 7f08a2a00700 1 --2- 172.21.15.131:0/553856947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 0x56493803c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.468+0000 7f08a2a00700 1 -- 172.21.15.131:0/553856947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802c5f0 msgr2=0x56493802aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.468+0000 7f08a2a00700 1 --2- 172.21.15.131:0/553856947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802c5f0 0x56493802aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.468+0000 7f08a2a00700 1 -- 172.21.15.131:0/553856947 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564937f115e0 con 0x56493802b720 2024-05-10T05:40:04.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.468+0000 7f08a3400700 1 --2- 172.21.15.131:0/553856947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 0x56493803c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:04.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.469+0000 7f08a2a00700 1 --2- 172.21.15.131:0/553856947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802b720 0x5649380347b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0894000c00 tx=0x7f0894005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f585220e1905aff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:04.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.469+0000 7f0898c00700 1 -- 172.21.15.131:0/553856947 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894028040 con 0x56493802b720 2024-05-10T05:40:04.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.469+0000 7f0898c00700 1 -- 172.21.15.131:0/553856947 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f089402d4c0 con 0x56493802b720 2024-05-10T05:40:04.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.469+0000 7f0898c00700 1 -- 172.21.15.131:0/553856947 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089401c040 con 0x56493802b720 2024-05-10T05:40:04.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.470+0000 7f08c1627b80 1 -- 172.21.15.131:0/553856947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802b720 msgr2=0x5649380347b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.470+0000 7f08c1627b80 1 --2- 172.21.15.131:0/553856947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802b720 0x5649380347b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0894000c00 tx=0x7f0894005c20 comp rx=0 tx=0).stop 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.470+0000 7f08c1627b80 1 -- 172.21.15.131:0/553856947 shutdown_connections 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.470+0000 7f08c1627b80 1 --2- 172.21.15.131:0/553856947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 0x56493803c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.470+0000 7f08c1627b80 1 --2- 172.21.15.131:0/553856947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802b720 0x5649380347b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.470+0000 7f08c1627b80 1 --2- 172.21.15.131:0/553856947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802c5f0 0x56493802aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.470+0000 7f08c1627b80 1 -- 172.21.15.131:0/553856947 >> 172.21.15.131:0/553856947 conn(0x564937f8dae0 msgr2=0x56493802a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.470+0000 7f08c1627b80 1 -- 172.21.15.131:0/553856947 shutdown_connections 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.470+0000 7f08c1627b80 1 -- 172.21.15.131:0/553856947 wait complete. 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.471+0000 7f08c1627b80 1 -- start start 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.471+0000 7f08c1627b80 1 Processor -- start 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x56493803c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802c5f0 0x56493803cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08c1627b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 0x56493803ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08c1627b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493803b420 con 0x56493802b720 2024-05-10T05:40:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08c1627b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493803b590 con 0x56493802c5f0 2024-05-10T05:40:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08c1627b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493803b750 con 0x56493802cf40 2024-05-10T05:40:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08c1627b80 10 client.0 _ll_register_callbacks cb 0x564937ef0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08a2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802c5f0 0x56493803cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08a2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x56493803c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08a2a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x56493803c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38808/0 (socket says 172.21.15.131:38808) 2024-05-10T05:40:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.472+0000 7f08a2a00700 1 -- 172.21.15.131:0/2908252311 learned_addr learned my addr 172.21.15.131:0/2908252311 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.473+0000 7f08a3400700 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 0x56493803ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.473+0000 7f08a2a00700 1 -- 172.21.15.131:0/2908252311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 msgr2=0x56493803ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.473+0000 7f08a2a00700 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 0x56493803ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.473+0000 7f08a2a00700 1 -- 172.21.15.131:0/2908252311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802c5f0 msgr2=0x56493803cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.473+0000 7f08a2a00700 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802c5f0 0x56493803cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.473+0000 7f08a2a00700 1 -- 172.21.15.131:0/2908252311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564937dcc430 con 0x56493802b720 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.474+0000 7f08a2a00700 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x56493803c610 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0894021700 tx=0x7f0894009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a28af256164346b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.474+0000 7f088cc00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.474+0000 7f088cc00700 1 -- 172.21.15.131:0/2908252311 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089401c070 con 0x56493802b720 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.474+0000 7f088cc00700 1 -- 172.21.15.131:0/2908252311 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0894005250 con 0x56493802b720 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.474+0000 7f08c1627b80 1 -- 172.21.15.131:0/2908252311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f089c00f040 con 0x56493802b720 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.475+0000 7f088cc00700 1 -- 172.21.15.131:0/2908252311 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0894028040 con 0x56493802b720 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.475+0000 7f08c1627b80 10 client.16389 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.475+0000 7f08c1627b80 1 -- 172.21.15.131:0/2908252311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564937d78be0 con 0x56493802b720 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.475+0000 7f088cc00700 1 -- 172.21.15.131:0/2908252311 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f089c00f040 con 0x56493802b720 2024-05-10T05:40:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.475+0000 7f088cc00700 1 -- 172.21.15.131:0/2908252311 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0894035070 con 0x56493802b720 2024-05-10T05:40:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.475+0000 7f08c1627b80 10 client.16389 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.475+0000 7f08c1627b80 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.475+0000 7f08c1627b80 20 client.16389 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.475+0000 7f08c1627b80 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564938098860 0x56493809ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.476+0000 7f08c1627b80 4 client.16389 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:40:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.476+0000 7f08c1627b80 1 -- 172.21.15.131:0/2908252311 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564937f43f40 con 0x564938098860 2024-05-10T05:40:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.476+0000 7f08a2000700 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564938098860 0x56493809ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:04.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.479+0000 7f088cc00700 10 client.16389 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.479+0000 7f08a2000700 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564938098860 0x56493809ac80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35dbd7fb11129a19 server_cookie=c261770f901cb0db in_seq=0 out_seq=0 2024-05-10T05:40:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.480+0000 7f088cc00700 1 -- 172.21.15.131:0/2908252311 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564937f43f40 con 0x564938098860 2024-05-10T05:40:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.480+0000 7f088cc00700 10 client.16389 handle_command_reply: tid=0 2024-05-10T05:40:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.480+0000 7f08c1627b80 1 client.16389 shutdown 2024-05-10T05:40:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.481+0000 7f08c1627b80 1 -- 172.21.15.131:0/2908252311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 msgr2=0x56493803c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.481+0000 7f08c1627b80 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x56493803c610 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0894021700 tx=0x7f0894009260 comp rx=0 tx=0).stop 2024-05-10T05:40:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.481+0000 7f08c1627b80 1 -- 172.21.15.131:0/2908252311 shutdown_connections 2024-05-10T05:40:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.481+0000 7f08c1627b80 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x564938098860 0x56493809ac80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.481+0000 7f08c1627b80 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56493802b720 0x56493803c610 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.481+0000 7f08c1627b80 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56493802cf40 0x56493803ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.481+0000 7f08c1627b80 1 --2- 172.21.15.131:0/2908252311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56493802c5f0 0x56493803cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.481+0000 7f08c1627b80 1 -- 172.21.15.131:0/2908252311 >> 172.21.15.131:0/2908252311 conn(0x564937f8dae0 msgr2=0x564938034500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:04.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.481+0000 7f088cc00700 0 client.16389 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:04.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.482+0000 7f08c1627b80 1 -- 172.21.15.131:0/2908252311 shutdown_connections 2024-05-10T05:40:04.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.482+0000 7f08c1627b80 1 -- 172.21.15.131:0/2908252311 wait complete. 2024-05-10T05:40:04.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.482+0000 7f08c1627b80 20 client.16389 trim_cache size 0 max 16384 2024-05-10T05:40:04.483 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:40:04.483 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:40:04.483 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "0f88dc7c-f48c-4d46-ada5-4a56f842e196", 2024-05-10T05:40:04.483 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:40:04.483 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.483+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08880a1330 msgr2=0x7f08880a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.483+0000 7f08a9800700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08880a1330 0x7f08880a3840 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f08940098c0 tx=0x7f0894019040 comp rx=0 tx=0).stop 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.483+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a410e220 msgr2=0x7f08a419ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.483+0000 7f08a9800700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a410e220 0x7f08a419ec90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f089000e9c0 tx=0x7f089000ecd0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.483+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 shutdown_connections 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.483+0000 7f08a9800700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f08880a1330 0x7f08880a3840 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.483+0000 7f08a9800700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08a41a3b50 0x7f08a41a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.483+0000 7f08a9800700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a4073b40 0x7f08a419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.483+0000 7f08a9800700 1 --2- 172.21.15.131:0/727888034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a410e220 0x7f08a419ec90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.483+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 >> 172.21.15.131:0/727888034 conn(0x7f08a40fd210 msgr2=0x7f08a41015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.484+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 shutdown_connections 2024-05-10T05:40:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:04.484+0000 7f08a9800700 1 -- 172.21.15.131:0/727888034 wait complete. 2024-05-10T05:40:04.544 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:40:04.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:04 smithi142 ceph-mon[77667]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 218 MiB/s wr, 66 op/s 2024-05-10T05:40:04.652 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:04 smithi131 ceph-mon[37264]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 218 MiB/s wr, 66 op/s 2024-05-10T05:40:04.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:04 smithi157 ceph-mon[78427]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 218 MiB/s wr, 66 op/s 2024-05-10T05:40:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.235+0000 7f89b5000700 1 -- 172.21.15.131:0/1611611127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b0108b60 msgr2=0x7f89b010fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.235+0000 7f89b5000700 1 --2- 172.21.15.131:0/1611611127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b0108b60 0x7f89b010fde0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f89a00099c0 tx=0x7f89a0009cd0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.235+0000 7f89b5000700 1 -- 172.21.15.131:0/1611611127 shutdown_connections 2024-05-10T05:40:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.235+0000 7f89b5000700 1 --2- 172.21.15.131:0/1611611127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89b0110320 0x7f89b0112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.235+0000 7f89b5000700 1 --2- 172.21.15.131:0/1611611127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0108210 0x7f89b0108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.235+0000 7f89b5000700 1 --2- 172.21.15.131:0/1611611127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b0108b60 0x7f89b010fde0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.235+0000 7f89b5000700 1 -- 172.21.15.131:0/1611611127 >> 172.21.15.131:0/1611611127 conn(0x7f89b006ea00 msgr2=0x7f89b0070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:05.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.244+0000 7f89b5000700 1 -- 172.21.15.131:0/1611611127 shutdown_connections 2024-05-10T05:40:05.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.244+0000 7f89b5000700 1 -- 172.21.15.131:0/1611611127 wait complete. 2024-05-10T05:40:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.245+0000 7f89b5000700 1 Processor -- start 2024-05-10T05:40:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.246+0000 7f89b5000700 1 -- start start 2024-05-10T05:40:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.246+0000 7f89b5000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89b0108210 0x7f89b019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.246+0000 7f89b5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0108b60 0x7f89b019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.247+0000 7f89b5000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b0110320 0x7f89b01a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.247+0000 7f89b5000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b01a6050 con 0x7f89b0108210 2024-05-10T05:40:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.247+0000 7f89b5000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b01a61c0 con 0x7f89b0108b60 2024-05-10T05:40:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.247+0000 7f89b5000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b01a6380 con 0x7f89b0110320 2024-05-10T05:40:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.247+0000 7f89ade00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89b0108210 0x7f89b019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.247+0000 7f89ae800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b0110320 0x7f89b01a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.247+0000 7f89ae800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b0110320 0x7f89b01a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53950/0 (socket says 172.21.15.131:53950) 2024-05-10T05:40:05.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.247+0000 7f89ae800700 1 -- 172.21.15.131:0/3298225179 learned_addr learned my addr 172.21.15.131:0/3298225179 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.250+0000 7f89ae800700 1 -- 172.21.15.131:0/3298225179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0108b60 msgr2=0x7f89b019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.248+0000 7f89ad400700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0108b60 0x7f89b019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.252+0000 7f89ae800700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0108b60 0x7f89b019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.252+0000 7f89ae800700 1 -- 172.21.15.131:0/3298225179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89b0108210 msgr2=0x7f89b019e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.252+0000 7f89ae800700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89b0108210 0x7f89b019e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.252+0000 7f89ae800700 1 -- 172.21.15.131:0/3298225179 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00095f0 con 0x7f89b0110320 2024-05-10T05:40:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.252+0000 7f89ade00700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89b0108210 0x7f89b019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.252+0000 7f89ad400700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0108b60 0x7f89b019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:05.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.252+0000 7f89ae800700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b0110320 0x7f89b01a5a10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f89a800b970 tx=0x7f89a800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.253+0000 7f89a6a00700 1 -- 172.21.15.131:0/3298225179 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a800c930 con 0x7f89b0110320 2024-05-10T05:40:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.253+0000 7f89b5000700 1 -- 172.21.15.131:0/3298225179 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b01a6660 con 0x7f89b0110320 2024-05-10T05:40:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.253+0000 7f89b5000700 1 -- 172.21.15.131:0/3298225179 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b01a6b70 con 0x7f89b0110320 2024-05-10T05:40:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.253+0000 7f89a6a00700 1 -- 172.21.15.131:0/3298225179 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89a8014440 con 0x7f89b0110320 2024-05-10T05:40:05.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.253+0000 7f89a6a00700 1 -- 172.21.15.131:0/3298225179 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a80075d0 con 0x7f89b0110320 2024-05-10T05:40:05.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.266+0000 7f89a6a00700 1 -- 172.21.15.131:0/3298225179 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f89a8007730 con 0x7f89b0110320 2024-05-10T05:40:05.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.268+0000 7f89a6a00700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89940a1330 0x7f89940a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.270+0000 7f89ade00700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89940a1330 0x7f89940a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.271+0000 7f89ade00700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89940a1330 0x7f89940a3840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f89b019f970 tx=0x7f899c00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:05.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.272+0000 7f89ccea6b80 1 Processor -- start 2024-05-10T05:40:05.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.272+0000 7f89ccea6b80 1 -- start start 2024-05-10T05:40:05.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.272+0000 7f89a6a00700 1 -- 172.21.15.131:0/3298225179 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f89a8006e40 con 0x7f89b0110320 2024-05-10T05:40:05.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.273+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b2f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.274+0000 7f89ade00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b2f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.274+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b2f7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.274+0000 7f89ade00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b2f6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38828/0 (socket says 172.21.15.131:38828) 2024-05-10T05:40:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.274+0000 7f89ade00700 1 -- 172.21.15.131:0/1805040035 learned_addr learned my addr 172.21.15.131:0/1805040035 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.275+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b2fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.275+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1805040035 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b21d150 con 0x56128b2f66a0 2024-05-10T05:40:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.275+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1805040035 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b21ab10 con 0x56128b2f7ec0 2024-05-10T05:40:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.275+0000 7f89ae800700 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b2fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.275+0000 7f89ad400700 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b2f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.275+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1805040035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b216400 con 0x56128b2f7570 2024-05-10T05:40:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.276+0000 7f89ad400700 1 -- 172.21.15.131:0/1805040035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 msgr2=0x56128b2fe7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.276+0000 7f89ad400700 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b2fe7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.276+0000 7f89ad400700 1 -- 172.21.15.131:0/1805040035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 msgr2=0x56128b2f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.276+0000 7f89ad400700 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b2f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.277+0000 7f89ad400700 1 -- 172.21.15.131:0/1805040035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56128b0973b0 con 0x56128b2f7570 2024-05-10T05:40:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.278+0000 7f89ae800700 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b2fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.278+0000 7f89ade00700 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b2f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:40:05.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.279+0000 7f89ad400700 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b2f7980 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f89a0006010 tx=0x7f89a0005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b744fb2dc67bee7a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:05.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.280+0000 7f899b400700 1 -- 172.21.15.131:0/1805040035 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0004230 con 0x56128b2f7570 2024-05-10T05:40:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.280+0000 7f899b400700 1 -- 172.21.15.131:0/1805040035 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89a0010430 con 0x56128b2f7570 2024-05-10T05:40:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.282+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1805040035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 msgr2=0x56128b2f7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.282+0000 7f899b400700 1 -- 172.21.15.131:0/1805040035 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a000b790 con 0x56128b2f7570 2024-05-10T05:40:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.282+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b2f7980 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f89a0006010 tx=0x7f89a0005a90 comp rx=0 tx=0).stop 2024-05-10T05:40:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.282+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1805040035 shutdown_connections 2024-05-10T05:40:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.282+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b2f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.282+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b2fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.283+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1805040035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b2f7980 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.283+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1805040035 >> 172.21.15.131:0/1805040035 conn(0x56128b258a60 msgr2=0x56128b2f0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.283+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1805040035 shutdown_connections 2024-05-10T05:40:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.283+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1805040035 wait complete. 2024-05-10T05:40:05.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.285+0000 7f89ccea6b80 1 -- start start 2024-05-10T05:40:05.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.286+0000 7f89ccea6b80 1 Processor -- start 2024-05-10T05:40:05.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.286+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b300640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.287+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b300b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.287+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b307ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.287+0000 7f89ccea6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b1b2560 con 0x56128b2f66a0 2024-05-10T05:40:05.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.287+0000 7f89ccea6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b1b2200 con 0x56128b2f7ec0 2024-05-10T05:40:05.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.287+0000 7f89ccea6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b0996a0 con 0x56128b2f7570 2024-05-10T05:40:05.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.287+0000 7f89ccea6b80 10 client.0 _ll_register_callbacks cb 0x56128b1bbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:05.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.287+0000 7f89ade00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b300640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.288+0000 7f89ade00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b300640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38840/0 (socket says 172.21.15.131:38840) 2024-05-10T05:40:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.288+0000 7f89ade00700 1 -- 172.21.15.131:0/1904880748 learned_addr learned my addr 172.21.15.131:0/1904880748 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.288+0000 7f89ad400700 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b300b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.288+0000 7f89ae800700 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b307ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.288+0000 7f89ade00700 1 -- 172.21.15.131:0/1904880748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 msgr2=0x56128b300b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.288+0000 7f89ade00700 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b300b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.288+0000 7f89ade00700 1 -- 172.21.15.131:0/1904880748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 msgr2=0x56128b307ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.288+0000 7f89ade00700 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b307ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.288+0000 7f89ade00700 1 -- 172.21.15.131:0/1904880748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b01a6b70 con 0x56128b2f66a0 2024-05-10T05:40:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.289+0000 7f89ae800700 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b307ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.289+0000 7f89ade00700 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b300640 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f899c011a10 tx=0x7f899c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248d428259de685e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.289+0000 7f898fe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:40:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.289+0000 7f898fe00700 1 -- 172.21.15.131:0/1904880748 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c004020 con 0x56128b2f66a0 2024-05-10T05:40:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.289+0000 7f898fe00700 1 -- 172.21.15.131:0/1904880748 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f899c017070 con 0x56128b2f66a0 2024-05-10T05:40:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.289+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1904880748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56128b16f270 con 0x56128b2f66a0 2024-05-10T05:40:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.290+0000 7f898fe00700 1 -- 172.21.15.131:0/1904880748 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f899c004180 con 0x56128b2f66a0 2024-05-10T05:40:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.290+0000 7f89ccea6b80 10 client.16398 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.290+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1904880748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f89b01131d0 con 0x56128b2f66a0 2024-05-10T05:40:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.290+0000 7f898fe00700 1 -- 172.21.15.131:0/1904880748 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56128b16f270 con 0x56128b2f66a0 2024-05-10T05:40:05.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.292+0000 7f898fe00700 1 -- 172.21.15.131:0/1904880748 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f899c01eaf0 con 0x56128b2f66a0 2024-05-10T05:40:05.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.292+0000 7f89ccea6b80 10 client.16398 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:05.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.292+0000 7f89ccea6b80 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:05.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.292+0000 7f89ccea6b80 20 client.16398 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:05.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.292+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56128b30c3c0 0x56128b30e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.292+0000 7f89ccea6b80 4 client.16398 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:05.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.292+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1904880748 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56128b2c5780 con 0x56128b30c3c0 2024-05-10T05:40:05.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.292+0000 7f89ad400700 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56128b30c3c0 0x56128b30e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.294+0000 7f89ad400700 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56128b30c3c0 0x56128b30e810 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af19ba7947ded46c server_cookie=8619e72484c6d326 in_seq=0 out_seq=0 2024-05-10T05:40:05.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.294+0000 7f898fe00700 10 client.16398 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:05.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.296+0000 7f898fe00700 1 -- 172.21.15.131:0/1904880748 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56128b2c5780 con 0x56128b30c3c0 2024-05-10T05:40:05.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.296+0000 7f898fe00700 10 client.16398 handle_command_reply: tid=0 2024-05-10T05:40:05.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.296+0000 7f89ccea6b80 1 client.16398 shutdown 2024-05-10T05:40:05.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.296+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1904880748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 msgr2=0x56128b300640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:05.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.296+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b300640 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f899c011a10 tx=0x7f899c0047c0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.298+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1904880748 shutdown_connections 2024-05-10T05:40:05.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.298+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56128b30c3c0 0x56128b30e810 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.298+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f66a0 0x56128b300640 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.298+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b307ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.298+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1904880748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b300b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.298+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1904880748 >> 172.21.15.131:0/1904880748 conn(0x56128b258a60 msgr2=0x56128b2f0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:05.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.298+0000 7f898fe00700 0 client.16398 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:05.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.300+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1904880748 shutdown_connections 2024-05-10T05:40:05.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.300+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1904880748 wait complete. 2024-05-10T05:40:05.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.304+0000 7f89ccea6b80 20 client.16398 trim_cache size 0 max 16384 2024-05-10T05:40:05.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.353+0000 7f89ccea6b80 1 Processor -- start 2024-05-10T05:40:05.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.353+0000 7f89ccea6b80 1 -- start start 2024-05-10T05:40:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.353+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f66a0 0x56128b30e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.353+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f7570 0x56128b307aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.353+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b30e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.354+0000 7f89ade00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f66a0 0x56128b30e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.354+0000 7f89ade00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f66a0 0x56128b30e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53990/0 (socket says 172.21.15.131:53990) 2024-05-10T05:40:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.354+0000 7f89ade00700 1 -- 172.21.15.131:0/2390760867 learned_addr learned my addr 172.21.15.131:0/2390760867 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.354+0000 7f89ccea6b80 1 -- 172.21.15.131:0/2390760867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b308be0 con 0x56128b2f7570 2024-05-10T05:40:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.354+0000 7f89ccea6b80 1 -- 172.21.15.131:0/2390760867 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b2f1ad0 con 0x56128b2f7ec0 2024-05-10T05:40:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.354+0000 7f89ccea6b80 1 -- 172.21.15.131:0/2390760867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b313940 con 0x56128b2f66a0 2024-05-10T05:40:05.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.354+0000 7f89ae800700 1 --2- 172.21.15.131:0/2390760867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b30e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.358+0000 7f89ade00700 1 -- 172.21.15.131:0/2390760867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 msgr2=0x56128b30e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:05.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.358+0000 7f89ade00700 1 --2- 172.21.15.131:0/2390760867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b30e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.358+0000 7f89ade00700 1 -- 172.21.15.131:0/2390760867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f7570 msgr2=0x56128b307aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:40:05.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.367+0000 7f89ad400700 1 --2- 172.21.15.131:0/2390760867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f7570 0x56128b307aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.368+0000 7f89ade00700 1 --2- 172.21.15.131:0/2390760867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f7570 0x56128b307aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.368+0000 7f89ade00700 1 -- 172.21.15.131:0/2390760867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56128b043b60 con 0x56128b2f66a0 2024-05-10T05:40:05.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.369+0000 7f89ad400700 1 --2- 172.21.15.131:0/2390760867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f7570 0x56128b307aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:05.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.369+0000 7f89ade00700 1 --2- 172.21.15.131:0/2390760867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f66a0 0x56128b30e3d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f899c01eea0 tx=0x7f899c011e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=557de93b24bffa58 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:05.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.369+0000 7f899be00700 1 -- 172.21.15.131:0/2390760867 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c01e920 con 0x56128b2f66a0 2024-05-10T05:40:05.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.369+0000 7f899be00700 1 -- 172.21.15.131:0/2390760867 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f899c023070 con 0x56128b2f66a0 2024-05-10T05:40:05.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.369+0000 7f899be00700 1 -- 172.21.15.131:0/2390760867 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c0053f0 con 0x56128b2f66a0 2024-05-10T05:40:05.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.374+0000 7f89ccea6b80 1 -- 172.21.15.131:0/2390760867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f66a0 msgr2=0x56128b30e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:05.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.374+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/2390760867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f66a0 0x56128b30e3d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f899c01eea0 tx=0x7f899c011e50 comp rx=0 tx=0).stop 2024-05-10T05:40:05.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.378+0000 7f89ccea6b80 1 -- 172.21.15.131:0/2390760867 shutdown_connections 2024-05-10T05:40:05.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.378+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/2390760867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f7ec0 0x56128b30e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.378+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/2390760867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f66a0 0x56128b30e3d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.378+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/2390760867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f7570 0x56128b307aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.378+0000 7f89ccea6b80 1 -- 172.21.15.131:0/2390760867 >> 172.21.15.131:0/2390760867 conn(0x56128b258a60 msgr2=0x56128b2fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:05.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.384+0000 7f89ccea6b80 1 -- 172.21.15.131:0/2390760867 shutdown_connections 2024-05-10T05:40:05.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.384+0000 7f89ccea6b80 1 -- 172.21.15.131:0/2390760867 wait complete. 2024-05-10T05:40:05.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.384+0000 7f89ccea6b80 1 -- start start 2024-05-10T05:40:05.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.387+0000 7f89ccea6b80 1 Processor -- start 2024-05-10T05:40:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.388+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f66a0 0x56128b305610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.389+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b305b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.390+0000 7f89ad400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b305b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.390+0000 7f89ad400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b305b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54004/0 (socket says 172.21.15.131:54004) 2024-05-10T05:40:05.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.393+0000 7f89ade00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f66a0 0x56128b305610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.393+0000 7f89ade00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f66a0 0x56128b305610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34996/0 (socket says 172.21.15.131:34996) 2024-05-10T05:40:05.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.393+0000 7f89ccea6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f7ec0 0x56128b306a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.393+0000 7f89ccea6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b306f70 con 0x56128b2f7ec0 2024-05-10T05:40:05.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.393+0000 7f89ccea6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b3070e0 con 0x56128b2f66a0 2024-05-10T05:40:05.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.393+0000 7f89ccea6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b3072a0 con 0x56128b2f7570 2024-05-10T05:40:05.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.393+0000 7f89ccea6b80 10 client.0 _ll_register_callbacks cb 0x56128b1bbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:05.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.393+0000 7f89ad400700 1 -- 172.21.15.131:0/1336459547 learned_addr learned my addr 172.21.15.131:0/1336459547 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:05.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.394+0000 7f89ade00700 1 -- 172.21.15.131:0/1336459547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 msgr2=0x56128b305b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:05.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.394+0000 7f89ade00700 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b305b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.394+0000 7f89ade00700 1 -- 172.21.15.131:0/1336459547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f7ec0 msgr2=0x56128b306a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:05.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.394+0000 7f89ade00700 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f7ec0 0x56128b306a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.394+0000 7f89ade00700 1 -- 172.21.15.131:0/1336459547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b01131d0 con 0x56128b2f66a0 2024-05-10T05:40:05.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.397+0000 7f89ade00700 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f66a0 0x56128b305610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x56128b2f82f0 tx=0x7f899c01ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f92248ad0f1d5ab1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:05.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.397+0000 7f898fe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:05.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.398+0000 7f898fe00700 1 -- 172.21.15.131:0/1336459547 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c005600 con 0x56128b2f66a0 2024-05-10T05:40:05.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.398+0000 7f898fe00700 1 -- 172.21.15.131:0/1336459547 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f899c017040 con 0x56128b2f66a0 2024-05-10T05:40:05.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.398+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1336459547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89a0010430 con 0x56128b2f66a0 2024-05-10T05:40:05.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.399+0000 7f898fe00700 1 -- 172.21.15.131:0/1336459547 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f899c005760 con 0x56128b2f66a0 2024-05-10T05:40:05.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.399+0000 7f89ccea6b80 10 client.26612 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:05.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.399+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1336459547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56128b1dc560 con 0x56128b2f66a0 2024-05-10T05:40:05.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.402+0000 7f898fe00700 1 -- 172.21.15.131:0/1336459547 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89a0010430 con 0x56128b2f66a0 2024-05-10T05:40:05.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.402+0000 7f898fe00700 1 -- 172.21.15.131:0/1336459547 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f899c003680 con 0x56128b2f66a0 2024-05-10T05:40:05.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.402+0000 7f89ccea6b80 10 client.26612 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:05.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.402+0000 7f89ccea6b80 10 client.26612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:05.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.402+0000 7f89ccea6b80 20 client.26612 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:05.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.402+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56128b363000 0x56128b365420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:05.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.405+0000 7f89ccea6b80 4 client.26612 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:40:05.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.405+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1336459547 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56128b20eec0 con 0x56128b363000 2024-05-10T05:40:05.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.407+0000 7f89ad400700 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56128b363000 0x56128b365420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:05.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.408+0000 7f89ad400700 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56128b363000 0x56128b365420 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95378d008f38683f server_cookie=f62fac5e0effbbd5 in_seq=0 out_seq=0 2024-05-10T05:40:05.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.408+0000 7f898fe00700 10 client.26612 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.409+0000 7f898fe00700 1 -- 172.21.15.131:0/1336459547 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56128b20eec0 con 0x56128b363000 2024-05-10T05:40:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.409+0000 7f898fe00700 10 client.26612 handle_command_reply: tid=0 2024-05-10T05:40:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.409+0000 7f89ccea6b80 1 client.26612 shutdown 2024-05-10T05:40:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.409+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1336459547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f66a0 msgr2=0x56128b305610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:05.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.410+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f66a0 0x56128b305610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x56128b2f82f0 tx=0x7f899c01ea90 comp rx=0 tx=0).stop 2024-05-10T05:40:05.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.410+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1336459547 shutdown_connections 2024-05-10T05:40:05.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.410+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56128b363000 0x56128b365420 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.411+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56128b2f7ec0 0x56128b306a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.411+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128b2f66a0 0x56128b305610 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.411+0000 7f89ccea6b80 1 --2- 172.21.15.131:0/1336459547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56128b2f7570 0x56128b305b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.411+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1336459547 >> 172.21.15.131:0/1336459547 conn(0x56128b258a60 msgr2=0x56128b30e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:05.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.411+0000 7f898fe00700 0 client.26612 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:05.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.411+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1336459547 shutdown_connections 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.411+0000 7f89ccea6b80 1 -- 172.21.15.131:0/1336459547 wait complete. 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.412+0000 7f89ccea6b80 20 client.26612 trim_cache size 0 max 16384 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stdout: "0f88dc7c-f48c-4d46-ada5-4a56f842e196": { 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "0f88dc7c-f48c-4d46-ada5-4a56f842e196", 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:40:05.412 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:40:05.413 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:40:05.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.413+0000 7f89b5000700 1 -- 172.21.15.131:0/3298225179 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89940a1330 msgr2=0x7f89940a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:05.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.413+0000 7f89b5000700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89940a1330 0x7f89940a3840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f89b019f970 tx=0x7f899c00b410 comp rx=0 tx=0).stop 2024-05-10T05:40:05.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.413+0000 7f89b5000700 1 -- 172.21.15.131:0/3298225179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b0110320 msgr2=0x7f89b01a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:05.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.413+0000 7f89b5000700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b0110320 0x7f89b01a5a10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f89a800b970 tx=0x7f89a800bc80 comp rx=0 tx=0).stop 2024-05-10T05:40:05.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.414+0000 7f89b5000700 1 -- 172.21.15.131:0/3298225179 shutdown_connections 2024-05-10T05:40:05.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.414+0000 7f89b5000700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f89940a1330 0x7f89940a3840 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.414+0000 7f89b5000700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89b0108210 0x7f89b019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.414+0000 7f89b5000700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b0110320 0x7f89b01a5a10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.414+0000 7f89b5000700 1 --2- 172.21.15.131:0/3298225179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0108b60 0x7f89b019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:05.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.414+0000 7f89b5000700 1 -- 172.21.15.131:0/3298225179 >> 172.21.15.131:0/3298225179 conn(0x7f89b006ea00 msgr2=0x7f89b010bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:05.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.415+0000 7f89b5000700 1 -- 172.21.15.131:0/3298225179 shutdown_connections 2024-05-10T05:40:05.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:05.415+0000 7f89b5000700 1 -- 172.21.15.131:0/3298225179 wait complete. 2024-05-10T05:40:05.493 INFO:tasks.cephfs.filesystem:scrub status for tag:0f88dc7c-f48c-4d46-ada5-4a56f842e196 - {'path': '/', 'tag': '0f88dc7c-f48c-4d46-ada5-4a56f842e196', 'options': 'recursive,force'} 2024-05-10T05:40:05.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:05 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:40:05.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:05 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:05.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:05 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:40:05.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:05 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:05.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:05 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:40:05.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:05 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:06.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:06 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:40:06.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:06 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:06.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:06 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:40:06.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:06 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:06.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:06 smithi142 ceph-mon[77667]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 81 op/s 2024-05-10T05:40:06.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:06 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:40:06.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:06 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:06.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:06 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:40:06.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:06 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:06.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:06 smithi131 ceph-mon[37264]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 81 op/s 2024-05-10T05:40:06.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:06 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:40:06.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:06 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:06.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:06 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:40:06.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:06 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:06.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:06 smithi157 ceph-mon[78427]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 81 op/s 2024-05-10T05:40:08.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:08 smithi142 ceph-mon[77667]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 226 MiB/s wr, 70 op/s 2024-05-10T05:40:08.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:08 smithi131 ceph-mon[37264]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 226 MiB/s wr, 70 op/s 2024-05-10T05:40:08.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:08 smithi157 ceph-mon[78427]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 226 MiB/s wr, 70 op/s 2024-05-10T05:40:09.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:09 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:40:09.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:09 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:40:09.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:09 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:40:10.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:10 smithi142 ceph-mon[77667]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 253 MiB/s wr, 75 op/s 2024-05-10T05:40:10.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:10 smithi131 ceph-mon[37264]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 253 MiB/s wr, 75 op/s 2024-05-10T05:40:10.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:10 smithi157 ceph-mon[78427]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 253 MiB/s wr, 75 op/s 2024-05-10T05:40:11.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:11 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:40:11.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:11 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:40:11.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:11 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:40:11.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:11 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:40:11.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:11 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:40:11.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:11 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:40:12.719 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:12 smithi131 ceph-mon[37264]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-05-10T05:40:12.719 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:40:12.719 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:40:12.719 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:40:12.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:12 smithi157 ceph-mon[78427]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-05-10T05:40:12.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:40:12.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:40:12.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:40:12.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:12 smithi142 ceph-mon[77667]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-05-10T05:40:12.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:40:12.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:40:12.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:40:14.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:14 smithi131 ceph-mon[37264]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 218 MiB/s wr, 65 op/s 2024-05-10T05:40:14.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:14 smithi157 ceph-mon[78427]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 218 MiB/s wr, 65 op/s 2024-05-10T05:40:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:14 smithi142 ceph-mon[77667]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 218 MiB/s wr, 65 op/s 2024-05-10T05:40:15.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:15 smithi157 ceph-mon[78427]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 279 MiB/s wr, 82 op/s 2024-05-10T05:40:15.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:15 smithi142 ceph-mon[77667]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 279 MiB/s wr, 82 op/s 2024-05-10T05:40:15.964 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:15 smithi131 ceph-mon[37264]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 279 MiB/s wr, 82 op/s 2024-05-10T05:40:18.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:18 smithi157 ceph-mon[78427]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 71 op/s 2024-05-10T05:40:18.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:18 smithi142 ceph-mon[77667]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 71 op/s 2024-05-10T05:40:18.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:18 smithi131 ceph-mon[37264]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 71 op/s 2024-05-10T05:40:20.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:20 smithi142 ceph-mon[77667]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 260 MiB/s wr, 77 op/s 2024-05-10T05:40:20.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:20 smithi131 ceph-mon[37264]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 260 MiB/s wr, 77 op/s 2024-05-10T05:40:20.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:20 smithi157 ceph-mon[78427]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 260 MiB/s wr, 77 op/s 2024-05-10T05:40:22.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:22 smithi142 ceph-mon[77667]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 265 MiB/s wr, 81 op/s 2024-05-10T05:40:22.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:22 smithi131 ceph-mon[37264]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 265 MiB/s wr, 81 op/s 2024-05-10T05:40:22.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:22 smithi157 ceph-mon[78427]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 265 MiB/s wr, 81 op/s 2024-05-10T05:40:24.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:24 smithi142 ceph-mon[77667]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 222 MiB/s wr, 66 op/s 2024-05-10T05:40:25.478 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:24 smithi157 ceph-mon[78427]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 222 MiB/s wr, 66 op/s 2024-05-10T05:40:25.478 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:24 smithi131 ceph-mon[37264]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 222 MiB/s wr, 66 op/s 2024-05-10T05:40:26.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:26 smithi142 ceph-mon[77667]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 287 MiB/s wr, 85 op/s 2024-05-10T05:40:26.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:26 smithi131 ceph-mon[37264]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 287 MiB/s wr, 85 op/s 2024-05-10T05:40:26.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:26 smithi157 ceph-mon[78427]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 287 MiB/s wr, 85 op/s 2024-05-10T05:40:28.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:28 smithi131 ceph-mon[37264]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 72 op/s 2024-05-10T05:40:28.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:28 smithi157 ceph-mon[78427]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 72 op/s 2024-05-10T05:40:28.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:28 smithi142 ceph-mon[77667]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 72 op/s 2024-05-10T05:40:30.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:30 smithi131 ceph-mon[37264]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 262 MiB/s wr, 79 op/s 2024-05-10T05:40:30.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:30 smithi157 ceph-mon[78427]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 262 MiB/s wr, 79 op/s 2024-05-10T05:40:30.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:30 smithi142 ceph-mon[77667]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 262 MiB/s wr, 79 op/s 2024-05-10T05:40:32.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:32 smithi131 ceph-mon[37264]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 278 MiB/s wr, 85 op/s 2024-05-10T05:40:32.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:32 smithi157 ceph-mon[78427]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 278 MiB/s wr, 85 op/s 2024-05-10T05:40:32.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:32 smithi142 ceph-mon[77667]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 278 MiB/s wr, 85 op/s 2024-05-10T05:40:34.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:34 smithi131 ceph-mon[37264]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 232 MiB/s wr, 70 op/s 2024-05-10T05:40:34.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:34 smithi157 ceph-mon[78427]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 232 MiB/s wr, 70 op/s 2024-05-10T05:40:34.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:34 smithi142 ceph-mon[77667]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 232 MiB/s wr, 70 op/s 2024-05-10T05:40:35.494 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:40:35.717 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:35 smithi131 ceph-mon[37264]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 83 op/s 2024-05-10T05:40:35.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:35 smithi157 ceph-mon[78427]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 83 op/s 2024-05-10T05:40:35.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:35 smithi142 ceph-mon[77667]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 83 op/s 2024-05-10T05:40:36.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.195+0000 7fb486000700 1 -- 172.21.15.131:0/2744554526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb480076c70 msgr2=0x7fb480077080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:36.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.196+0000 7fb486000700 1 --2- 172.21.15.131:0/2744554526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb480076c70 0x7fb480077080 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb47001a9a0 tx=0x7fb47001acb0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.196+0000 7fb486000700 1 -- 172.21.15.131:0/2744554526 shutdown_connections 2024-05-10T05:40:36.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.196+0000 7fb486000700 1 --2- 172.21.15.131:0/2744554526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb480078730 0x7fb480079020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.196+0000 7fb486000700 1 --2- 172.21.15.131:0/2744554526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb480076c70 0x7fb480077080 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.196+0000 7fb486000700 1 --2- 172.21.15.131:0/2744554526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb480077de0 0x7fb4800781f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.196+0000 7fb486000700 1 -- 172.21.15.131:0/2744554526 >> 172.21.15.131:0/2744554526 conn(0x7fb4800ff450 msgr2=0x7fb4801018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.197+0000 7fb486000700 1 -- 172.21.15.131:0/2744554526 shutdown_connections 2024-05-10T05:40:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.198+0000 7fb486000700 1 -- 172.21.15.131:0/2744554526 wait complete. 2024-05-10T05:40:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.199+0000 7fb486000700 1 Processor -- start 2024-05-10T05:40:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.200+0000 7fb486000700 1 -- start start 2024-05-10T05:40:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.200+0000 7fb486000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb480076c70 0x7fb48019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.200+0000 7fb486000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb480077de0 0x7fb48019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.200+0000 7fb486000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb480078730 0x7fb4801a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.200+0000 7fb486000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4801a6090 con 0x7fb480076c70 2024-05-10T05:40:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.200+0000 7fb486000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4801a61d0 con 0x7fb480078730 2024-05-10T05:40:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.200+0000 7fb486000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4801a6360 con 0x7fb480077de0 2024-05-10T05:40:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.201+0000 7fb47fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb480078730 0x7fb4801a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.201+0000 7fb47fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb480078730 0x7fb4801a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:44998/0 (socket says 172.21.15.131:44998) 2024-05-10T05:40:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.201+0000 7fb47fe00700 1 -- 172.21.15.131:0/3362931192 learned_addr learned my addr 172.21.15.131:0/3362931192 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.201+0000 7fb476a00700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb480077de0 0x7fb48019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.201+0000 7fb47f400700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb480076c70 0x7fb48019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.202+0000 7fb47fe00700 1 -- 172.21.15.131:0/3362931192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb480077de0 msgr2=0x7fb48019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.202+0000 7fb47fe00700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb480077de0 0x7fb48019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.202+0000 7fb47fe00700 1 -- 172.21.15.131:0/3362931192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb480076c70 msgr2=0x7fb48019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.202+0000 7fb47fe00700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb480076c70 0x7fb48019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.202+0000 7fb47fe00700 1 -- 172.21.15.131:0/3362931192 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47001a600 con 0x7fb480078730 2024-05-10T05:40:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.202+0000 7fb476a00700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb480077de0 0x7fb48019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:40:36.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.202+0000 7fb47fe00700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb480078730 0x7fb4801a5ac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb478007d70 tx=0x7fb47800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:36.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.203+0000 7fb47cc00700 1 -- 172.21.15.131:0/3362931192 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47800ad40 con 0x7fb480078730 2024-05-10T05:40:36.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.203+0000 7fb47cc00700 1 -- 172.21.15.131:0/3362931192 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4780128e0 con 0x7fb480078730 2024-05-10T05:40:36.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.203+0000 7fb47cc00700 1 -- 172.21.15.131:0/3362931192 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47800ad40 con 0x7fb480078730 2024-05-10T05:40:36.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.203+0000 7fb486000700 1 -- 172.21.15.131:0/3362931192 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4801a6640 con 0x7fb480078730 2024-05-10T05:40:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.203+0000 7fb486000700 1 -- 172.21.15.131:0/3362931192 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4801a6aa0 con 0x7fb480078730 2024-05-10T05:40:36.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.210+0000 7fb49df51b80 1 Processor -- start 2024-05-10T05:40:36.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.219+0000 7fb47cc00700 1 -- 172.21.15.131:0/3362931192 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb478012a50 con 0x7fb480078730 2024-05-10T05:40:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.222+0000 7fb47cc00700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb4640a12b0 0x7fb4640a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.223+0000 7fb49df51b80 1 -- start start 2024-05-10T05:40:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.223+0000 7fb476a00700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb4640a12b0 0x7fb4640a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.223+0000 7fb49df51b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874c96af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.223+0000 7fb49df51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 0x559874c97980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.223+0000 7fb49df51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874c9e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.224+0000 7fb49df51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874bbd150 con 0x559874c97ec0 2024-05-10T05:40:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.224+0000 7fb49df51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874bbab10 con 0x559874c97570 2024-05-10T05:40:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.224+0000 7fb49df51b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874bb6400 con 0x559874c966a0 2024-05-10T05:40:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.224+0000 7fb47cc00700 1 -- 172.21.15.131:0/3362931192 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb47809a1c0 con 0x7fb480078730 2024-05-10T05:40:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.224+0000 7fb47fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874c9e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.224+0000 7fb47fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874c9e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50560/0 (socket says 172.21.15.131:50560) 2024-05-10T05:40:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.224+0000 7fb47fe00700 1 -- 172.21.15.131:0/2736465476 learned_addr learned my addr 172.21.15.131:0/2736465476 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:36.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.224+0000 7fb476a00700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb4640a12b0 0x7fb4640a37c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb48019f690 tx=0x7fb468008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:36.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.224+0000 7fb47f400700 1 --2- 172.21.15.131:0/2736465476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874c96af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.224+0000 7fb476a00700 1 --2- 172.21.15.131:0/2736465476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 0x559874c97980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.225+0000 7fb47f400700 1 -- 172.21.15.131:0/2736465476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 msgr2=0x559874c97980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.225+0000 7fb47f400700 1 --2- 172.21.15.131:0/2736465476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 0x559874c97980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.225+0000 7fb47f400700 1 -- 172.21.15.131:0/2736465476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 msgr2=0x559874c9e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.225+0000 7fb47f400700 1 --2- 172.21.15.131:0/2736465476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874c9e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.225+0000 7fb47f400700 1 -- 172.21.15.131:0/2736465476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559874a373b0 con 0x559874c966a0 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.225+0000 7fb476a00700 1 --2- 172.21.15.131:0/2736465476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 0x559874c97980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.225+0000 7fb47f400700 1 --2- 172.21.15.131:0/2736465476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874c96af0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb470006010 tx=0x7fb470005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a1255e78b42f9df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.226+0000 7fb46fe00700 1 -- 172.21.15.131:0/2736465476 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47001c9f0 con 0x559874c966a0 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.226+0000 7fb46fe00700 1 -- 172.21.15.131:0/2736465476 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb47002e040 con 0x559874c966a0 2024-05-10T05:40:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.226+0000 7fb46fe00700 1 -- 172.21.15.131:0/2736465476 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb470004950 con 0x559874c966a0 2024-05-10T05:40:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.227+0000 7fb49df51b80 1 -- 172.21.15.131:0/2736465476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 msgr2=0x559874c96af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.227+0000 7fb49df51b80 1 --2- 172.21.15.131:0/2736465476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874c96af0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb470006010 tx=0x7fb470005dc0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.227+0000 7fb49df51b80 1 -- 172.21.15.131:0/2736465476 shutdown_connections 2024-05-10T05:40:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.227+0000 7fb49df51b80 1 --2- 172.21.15.131:0/2736465476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874c9e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.227+0000 7fb49df51b80 1 --2- 172.21.15.131:0/2736465476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874c96af0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.227+0000 7fb49df51b80 1 --2- 172.21.15.131:0/2736465476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 0x559874c97980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.227+0000 7fb49df51b80 1 -- 172.21.15.131:0/2736465476 >> 172.21.15.131:0/2736465476 conn(0x559874bf8a60 msgr2=0x559874c90fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:36.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.228+0000 7fb49df51b80 1 -- 172.21.15.131:0/2736465476 shutdown_connections 2024-05-10T05:40:36.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.228+0000 7fb49df51b80 1 -- 172.21.15.131:0/2736465476 wait complete. 2024-05-10T05:40:36.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.229+0000 7fb49df51b80 1 -- start start 2024-05-10T05:40:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.230+0000 7fb49df51b80 1 Processor -- start 2024-05-10T05:40:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.232+0000 7fb49df51b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874ca06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.232+0000 7fb49df51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 0x559874ca0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.232+0000 7fb49df51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874ca7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.232+0000 7fb49df51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874b52560 con 0x559874c97ec0 2024-05-10T05:40:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.232+0000 7fb49df51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874b52200 con 0x559874c97570 2024-05-10T05:40:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.232+0000 7fb49df51b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874a396a0 con 0x559874c966a0 2024-05-10T05:40:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.232+0000 7fb49df51b80 10 client.0 _ll_register_callbacks cb 0x559874b5bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:36.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.232+0000 7fb47f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874ca06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.233+0000 7fb476a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 0x559874ca0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.233+0000 7fb47f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874ca06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59250/0 (socket says 172.21.15.131:59250) 2024-05-10T05:40:36.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.233+0000 7fb47f400700 1 -- 172.21.15.131:0/995477506 learned_addr learned my addr 172.21.15.131:0/995477506 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:36.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.233+0000 7fb47fe00700 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874ca7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.234+0000 7fb47f400700 1 -- 172.21.15.131:0/995477506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 msgr2=0x559874ca0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.234+0000 7fb47f400700 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 0x559874ca0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.234+0000 7fb47f400700 1 -- 172.21.15.131:0/995477506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 msgr2=0x559874ca7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.234+0000 7fb47f400700 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874ca7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.234+0000 7fb47f400700 1 -- 172.21.15.131:0/995477506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb480079b10 con 0x559874c966a0 2024-05-10T05:40:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.235+0000 7fb47fe00700 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874ca7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.237+0000 7fb47f400700 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874ca06a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb470005f50 tx=0x7fb47001ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36245e3242b7ccef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.237+0000 7fb46cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:40:36.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.238+0000 7fb46cc00700 1 -- 172.21.15.131:0/995477506 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb470005520 con 0x559874c966a0 2024-05-10T05:40:36.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.238+0000 7fb46cc00700 1 -- 172.21.15.131:0/995477506 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb4700046d0 con 0x559874c966a0 2024-05-10T05:40:36.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.239+0000 7fb49df51b80 1 -- 172.21.15.131:0/995477506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559874b0f270 con 0x559874c966a0 2024-05-10T05:40:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.241+0000 7fb46cc00700 1 -- 172.21.15.131:0/995477506 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb470004890 con 0x559874c966a0 2024-05-10T05:40:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.246+0000 7fb49df51b80 10 client.26656 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:36.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.246+0000 7fb49df51b80 1 -- 172.21.15.131:0/995477506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5598749e3b60 con 0x559874c966a0 2024-05-10T05:40:36.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.247+0000 7fb46cc00700 1 -- 172.21.15.131:0/995477506 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559874b0f270 con 0x559874c966a0 2024-05-10T05:40:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.247+0000 7fb46cc00700 1 -- 172.21.15.131:0/995477506 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb470043070 con 0x559874c966a0 2024-05-10T05:40:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.248+0000 7fb49df51b80 10 client.26656 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.248+0000 7fb49df51b80 10 client.26656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.248+0000 7fb49df51b80 20 client.26656 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.248+0000 7fb49df51b80 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559874cabdb0 0x559874cae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.248+0000 7fb49df51b80 4 client.26656 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.248+0000 7fb49df51b80 1 -- 172.21.15.131:0/995477506 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559874c65780 con 0x559874cabdb0 2024-05-10T05:40:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.249+0000 7fb476a00700 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559874cabdb0 0x559874cae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.250+0000 7fb46cc00700 10 client.26656 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.250+0000 7fb476a00700 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559874cabdb0 0x559874cae200 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d82f4864b3e6f server_cookie=3b61803c0ece858d in_seq=0 out_seq=0 2024-05-10T05:40:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.251+0000 7fb46cc00700 1 -- 172.21.15.131:0/995477506 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559874c65780 con 0x559874cabdb0 2024-05-10T05:40:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.251+0000 7fb46cc00700 10 client.26656 handle_command_reply: tid=0 2024-05-10T05:40:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.251+0000 7fb49df51b80 1 client.26656 shutdown 2024-05-10T05:40:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.252+0000 7fb49df51b80 1 -- 172.21.15.131:0/995477506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 msgr2=0x559874ca06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.252+0000 7fb49df51b80 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874ca06a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb470005f50 tx=0x7fb47001ccd0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.252+0000 7fb49df51b80 1 -- 172.21.15.131:0/995477506 shutdown_connections 2024-05-10T05:40:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.252+0000 7fb49df51b80 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559874cabdb0 0x559874cae200 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.252+0000 7fb49df51b80 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874ca7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.252+0000 7fb46cc00700 0 client.26656 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.252+0000 7fb49df51b80 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c966a0 0x559874ca06a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.252+0000 7fb49df51b80 1 --2- 172.21.15.131:0/995477506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97570 0x559874ca0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.252+0000 7fb49df51b80 1 -- 172.21.15.131:0/995477506 >> 172.21.15.131:0/995477506 conn(0x559874bf8a60 msgr2=0x559874c90fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:36.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.253+0000 7fb49df51b80 1 -- 172.21.15.131:0/995477506 shutdown_connections 2024-05-10T05:40:36.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.253+0000 7fb49df51b80 1 -- 172.21.15.131:0/995477506 wait complete. 2024-05-10T05:40:36.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.253+0000 7fb49df51b80 20 client.26656 trim_cache size 0 max 16384 2024-05-10T05:40:36.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.307+0000 7fb49df51b80 1 Processor -- start 2024-05-10T05:40:36.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.307+0000 7fb49df51b80 1 -- start start 2024-05-10T05:40:36.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.308+0000 7fb49df51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c966a0 0x559874c91ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.308+0000 7fb49df51b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 0x559874ca7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.308+0000 7fb49df51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874c9ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.309+0000 7fb49df51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874c65780 con 0x559874c97ec0 2024-05-10T05:40:36.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.309+0000 7fb49df51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874b0f270 con 0x559874c966a0 2024-05-10T05:40:36.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.309+0000 7fb49df51b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874c92770 con 0x559874c97570 2024-05-10T05:40:36.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.309+0000 7fb47fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874c9ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.309+0000 7fb47fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874c9ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50566/0 (socket says 172.21.15.131:50566) 2024-05-10T05:40:36.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.309+0000 7fb47fe00700 1 -- 172.21.15.131:0/2320492696 learned_addr learned my addr 172.21.15.131:0/2320492696 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.311+0000 7fb476a00700 1 --2- 172.21.15.131:0/2320492696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 0x559874ca7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.313+0000 7fb476a00700 1 -- 172.21.15.131:0/2320492696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c966a0 msgr2=0x559874c91ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:40:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.313+0000 7fb476a00700 1 --2- 172.21.15.131:0/2320492696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c966a0 0x559874c91ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.313+0000 7fb476a00700 1 -- 172.21.15.131:0/2320492696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 msgr2=0x559874c9ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.313+0000 7fb476a00700 1 --2- 172.21.15.131:0/2320492696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874c9ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.313+0000 7fb476a00700 1 -- 172.21.15.131:0/2320492696 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559874b7c560 con 0x559874c97570 2024-05-10T05:40:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.314+0000 7fb476a00700 1 --2- 172.21.15.131:0/2320492696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 0x559874ca7b00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x559874caea50 tx=0x7fb468011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bf2a86440517f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.316+0000 7fb475200700 1 -- 172.21.15.131:0/2320492696 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468026300 con 0x559874c97570 2024-05-10T05:40:36.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.316+0000 7fb475200700 1 -- 172.21.15.131:0/2320492696 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb468026920 con 0x559874c97570 2024-05-10T05:40:36.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.316+0000 7fb475200700 1 -- 172.21.15.131:0/2320492696 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468025430 con 0x559874c97570 2024-05-10T05:40:36.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.317+0000 7fb49df51b80 1 -- 172.21.15.131:0/2320492696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 msgr2=0x559874ca7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:36.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.317+0000 7fb49df51b80 1 --2- 172.21.15.131:0/2320492696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 0x559874ca7b00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x559874caea50 tx=0x7fb468011000 comp rx=0 tx=0).stop 2024-05-10T05:40:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.318+0000 7fb49df51b80 1 -- 172.21.15.131:0/2320492696 shutdown_connections 2024-05-10T05:40:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.318+0000 7fb49df51b80 1 --2- 172.21.15.131:0/2320492696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874c97ec0 0x559874c9ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.318+0000 7fb49df51b80 1 --2- 172.21.15.131:0/2320492696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c966a0 0x559874c91ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.318+0000 7fb49df51b80 1 --2- 172.21.15.131:0/2320492696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 0x559874ca7b00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.318+0000 7fb49df51b80 1 -- 172.21.15.131:0/2320492696 >> 172.21.15.131:0/2320492696 conn(0x559874bf8a60 msgr2=0x559874cae200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:36.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.323+0000 7fb49df51b80 1 -- 172.21.15.131:0/2320492696 shutdown_connections 2024-05-10T05:40:36.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.324+0000 7fb49df51b80 1 -- 172.21.15.131:0/2320492696 wait complete. 2024-05-10T05:40:36.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.324+0000 7fb49df51b80 1 -- start start 2024-05-10T05:40:36.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.325+0000 7fb49df51b80 1 Processor -- start 2024-05-10T05:40:36.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.326+0000 7fb49df51b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 0x559874ca59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.327+0000 7fb47f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 0x559874ca59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.327+0000 7fb47f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 0x559874ca59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59270/0 (socket says 172.21.15.131:59270) 2024-05-10T05:40:36.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.329+0000 7fb47f400700 1 -- 172.21.15.131:0/1409282007 learned_addr learned my addr 172.21.15.131:0/1409282007 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:36.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.329+0000 7fb49df51b80 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97ec0 0x559874ca5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.329+0000 7fb49df51b80 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874cabdb0 0x559874ca6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.329+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874ca7300 con 0x559874cabdb0 2024-05-10T05:40:36.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.329+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874ca7470 con 0x559874c97ec0 2024-05-10T05:40:36.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.329+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559874ca7630 con 0x559874c97570 2024-05-10T05:40:36.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.329+0000 7fb49df51b80 10 client.0 _ll_register_callbacks cb 0x559874b5bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:36.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.329+0000 7fb47fe00700 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874cabdb0 0x559874ca6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.329+0000 7fb476a00700 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97ec0 0x559874ca5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.330+0000 7fb476a00700 1 -- 172.21.15.131:0/1409282007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 msgr2=0x559874ca59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:36.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.330+0000 7fb476a00700 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 0x559874ca59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.331+0000 7fb476a00700 1 -- 172.21.15.131:0/1409282007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874cabdb0 msgr2=0x559874ca6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:36.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.331+0000 7fb476a00700 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874cabdb0 0x559874ca6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.331+0000 7fb476a00700 1 -- 172.21.15.131:0/1409282007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46801d040 con 0x559874c97ec0 2024-05-10T05:40:36.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.331+0000 7fb47fe00700 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874cabdb0 0x559874ca6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:36.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.332+0000 7fb476a00700 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97ec0 0x559874ca5ee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb468015c00 tx=0x7fb468004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7583da6ff0e63529 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:36.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.332+0000 7fb46cc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:36.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.333+0000 7fb46cc00700 1 -- 172.21.15.131:0/1409282007 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468033040 con 0x559874c97ec0 2024-05-10T05:40:36.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.333+0000 7fb46cc00700 1 -- 172.21.15.131:0/1409282007 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb468015780 con 0x559874c97ec0 2024-05-10T05:40:36.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.333+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb47002e040 con 0x559874c97ec0 2024-05-10T05:40:36.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.334+0000 7fb46cc00700 1 -- 172.21.15.131:0/1409282007 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb468015940 con 0x559874c97ec0 2024-05-10T05:40:36.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.334+0000 7fb49df51b80 10 client.26621 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:36.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.334+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559874a3bcd0 con 0x559874c97ec0 2024-05-10T05:40:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.334+0000 7fb46cc00700 1 -- 172.21.15.131:0/1409282007 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb47002e040 con 0x559874c97ec0 2024-05-10T05:40:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.336+0000 7fb46cc00700 1 -- 172.21.15.131:0/1409282007 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb46802fa70 con 0x559874c97ec0 2024-05-10T05:40:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.336+0000 7fb49df51b80 10 client.26621 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.336+0000 7fb49df51b80 10 client.26621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.336+0000 7fb49df51b80 20 client.26621 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.336+0000 7fb49df51b80 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559874d030b0 0x559874d054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.336+0000 7fb49df51b80 4 client.26621 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:40:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.336+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559874baeec0 con 0x559874d030b0 2024-05-10T05:40:36.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.336+0000 7fb47f400700 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559874d030b0 0x559874d054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:36.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.338+0000 7fb47f400700 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559874d030b0 0x559874d054d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97d9aaec479c394 server_cookie=104805778f9e0d65 in_seq=0 out_seq=0 2024-05-10T05:40:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.338+0000 7fb46cc00700 10 client.26621 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb46cc00700 1 -- 172.21.15.131:0/1409282007 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559874baeec0 con 0x559874d030b0 2024-05-10T05:40:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb46cc00700 10 client.26621 handle_command_reply: tid=0 2024-05-10T05:40:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb49df51b80 1 client.26621 shutdown 2024-05-10T05:40:36.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97ec0 msgr2=0x559874ca5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:36.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb49df51b80 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97ec0 0x559874ca5ee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb468015c00 tx=0x7fb468004860 comp rx=0 tx=0).stop 2024-05-10T05:40:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 shutdown_connections 2024-05-10T05:40:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb49df51b80 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559874d030b0 0x559874d054d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb46cc00700 0 client.26621 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb49df51b80 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559874cabdb0 0x559874ca6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb49df51b80 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559874c97570 0x559874ca59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb49df51b80 1 --2- 172.21.15.131:0/1409282007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559874c97ec0 0x559874ca5ee0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.339+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 >> 172.21.15.131:0/1409282007 conn(0x559874bf8a60 msgr2=0x559874cae200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.340+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 shutdown_connections 2024-05-10T05:40:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.340+0000 7fb49df51b80 1 -- 172.21.15.131:0/1409282007 wait complete. 2024-05-10T05:40:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.343+0000 7fb49df51b80 20 client.26621 trim_cache size 0 max 16384 2024-05-10T05:40:36.346 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:40:36.346 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:40:36.346 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:40:36.346 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:40:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.350+0000 7fb486000700 1 -- 172.21.15.131:0/3362931192 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb4640a12b0 msgr2=0x7fb4640a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.350+0000 7fb486000700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb4640a12b0 0x7fb4640a37c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb48019f690 tx=0x7fb468008040 comp rx=0 tx=0).stop 2024-05-10T05:40:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.350+0000 7fb486000700 1 -- 172.21.15.131:0/3362931192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb480078730 msgr2=0x7fb4801a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.350+0000 7fb486000700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb480078730 0x7fb4801a5ac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb478007d70 tx=0x7fb47800a340 comp rx=0 tx=0).stop 2024-05-10T05:40:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.350+0000 7fb486000700 1 -- 172.21.15.131:0/3362931192 shutdown_connections 2024-05-10T05:40:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.350+0000 7fb486000700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb4640a12b0 0x7fb4640a37c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.350+0000 7fb486000700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb480076c70 0x7fb48019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.350+0000 7fb486000700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb480078730 0x7fb4801a5ac0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.350+0000 7fb486000700 1 --2- 172.21.15.131:0/3362931192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb480077de0 0x7fb48019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.350+0000 7fb486000700 1 -- 172.21.15.131:0/3362931192 >> 172.21.15.131:0/3362931192 conn(0x7fb4800ff450 msgr2=0x7fb48006cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.351+0000 7fb486000700 1 -- 172.21.15.131:0/3362931192 shutdown_connections 2024-05-10T05:40:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:36.351+0000 7fb486000700 1 -- 172.21.15.131:0/3362931192 wait complete. 2024-05-10T05:40:36.410 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:40:36.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:40:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.193+0000 7f9810e00700 1 -- 172.21.15.131:0/4016615859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f980c108210 msgr2=0x7f980c108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:37.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.193+0000 7f9810e00700 1 --2- 172.21.15.131:0/4016615859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f980c108210 0x7f980c108620 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0099c0 tx=0x7f97fc009cd0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.198+0000 7f9810e00700 1 -- 172.21.15.131:0/4016615859 shutdown_connections 2024-05-10T05:40:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.198+0000 7f9810e00700 1 --2- 172.21.15.131:0/4016615859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f980c109cd0 0x7f980c110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.198+0000 7f9810e00700 1 --2- 172.21.15.131:0/4016615859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f980c108210 0x7f980c108620 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.198+0000 7f9810e00700 1 --2- 172.21.15.131:0/4016615859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c109330 0x7f980c109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.198+0000 7f9810e00700 1 -- 172.21.15.131:0/4016615859 >> 172.21.15.131:0/4016615859 conn(0x7f980c077240 msgr2=0x7f980c106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.198+0000 7f9810e00700 1 -- 172.21.15.131:0/4016615859 shutdown_connections 2024-05-10T05:40:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.198+0000 7f9810e00700 1 -- 172.21.15.131:0/4016615859 wait complete. 2024-05-10T05:40:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.200+0000 7f9810e00700 1 Processor -- start 2024-05-10T05:40:37.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.202+0000 7f9810e00700 1 -- start start 2024-05-10T05:40:37.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.202+0000 7f9810e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f980c108210 0x7f980c07eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.202+0000 7f9810e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c109330 0x7f980c07f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.203+0000 7f9809e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f980c108210 0x7f980c07eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.203+0000 7f9809e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f980c108210 0x7f980c07eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50610/0 (socket says 172.21.15.131:50610) 2024-05-10T05:40:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.209+0000 7f9809e00700 1 -- 172.21.15.131:0/4164604991 learned_addr learned my addr 172.21.15.131:0/4164604991 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.209+0000 7f9810e00700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f980c109cd0 0x7f980c1ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.210+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1ae960 con 0x7f980c108210 2024-05-10T05:40:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.210+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1aead0 con 0x7f980c109cd0 2024-05-10T05:40:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.210+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1aec90 con 0x7f980c109330 2024-05-10T05:40:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.210+0000 7f9809400700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c109330 0x7f980c07f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.211+0000 7f9809400700 1 -- 172.21.15.131:0/4164604991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f980c109cd0 msgr2=0x7f980c1ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.211+0000 7f9809400700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f980c109cd0 0x7f980c1ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.211+0000 7f9809400700 1 -- 172.21.15.131:0/4164604991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f980c108210 msgr2=0x7f980c07eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.211+0000 7f9809400700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f980c108210 0x7f980c07eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.211+0000 7f9809400700 1 -- 172.21.15.131:0/4164604991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0095f0 con 0x7f980c109330 2024-05-10T05:40:37.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.211+0000 7f9809400700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c109330 0x7f980c07f2e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f97f800e9f0 tx=0x7f97f800edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:37.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.212+0000 7f9802a00700 1 -- 172.21.15.131:0/4164604991 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f800cbd0 con 0x7f980c109330 2024-05-10T05:40:37.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.212+0000 7f9802a00700 1 -- 172.21.15.131:0/4164604991 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97f800cd30 con 0x7f980c109330 2024-05-10T05:40:37.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.212+0000 7f9802a00700 1 -- 172.21.15.131:0/4164604991 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8018680 con 0x7f980c109330 2024-05-10T05:40:37.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.213+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c1aef70 con 0x7f980c109330 2024-05-10T05:40:37.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.214+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c1af2e0 con 0x7f980c109330 2024-05-10T05:40:37.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.217+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980c10f420 con 0x7f980c109330 2024-05-10T05:40:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.226+0000 7f9802a00700 1 -- 172.21.15.131:0/4164604991 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f97f8010430 con 0x7f980c109330 2024-05-10T05:40:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.228+0000 7f9802a00700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97f00a1220 0x7f97f00a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.229+0000 7f9802a00700 1 -- 172.21.15.131:0/4164604991 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f97f8014070 con 0x7f980c109330 2024-05-10T05:40:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.230+0000 7f9809e00700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97f00a1220 0x7f97f00a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:37.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.231+0000 7f9809e00700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97f00a1220 0x7f97f00a3730 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f97fc000c00 tx=0x7f97fc01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:37.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.232+0000 7f9802a00700 1 -- 172.21.15.131:0/4164604991 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97f80964a0 con 0x7f980c109330 2024-05-10T05:40:37.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.832+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f980c0694d0 con 0x7f980c109330 2024-05-10T05:40:37.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.833+0000 7f9802a00700 1 -- 172.21.15.131:0/4164604991 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f97f8099ac0 con 0x7f980c109330 2024-05-10T05:40:37.834 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:40:37.834 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:40:37.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.836+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97f00a1220 msgr2=0x7f97f00a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:37.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.836+0000 7f9810e00700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97f00a1220 0x7f97f00a3730 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f97fc000c00 tx=0x7f97fc01a040 comp rx=0 tx=0).stop 2024-05-10T05:40:37.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.836+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c109330 msgr2=0x7f980c07f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:37.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.836+0000 7f9810e00700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c109330 0x7f980c07f2e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f97f800e9f0 tx=0x7f97f800edb0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.837+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 shutdown_connections 2024-05-10T05:40:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.837+0000 7f9810e00700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97f00a1220 0x7f97f00a3730 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.837+0000 7f9810e00700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f980c108210 0x7f980c07eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.837+0000 7f9810e00700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f980c109cd0 0x7f980c1ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.837+0000 7f9810e00700 1 --2- 172.21.15.131:0/4164604991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f980c109330 0x7f980c07f2e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:37.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.837+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 >> 172.21.15.131:0/4164604991 conn(0x7f980c077240 msgr2=0x7f980c10c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:37.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.837+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 shutdown_connections 2024-05-10T05:40:37.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:37.837+0000 7f9810e00700 1 -- 172.21.15.131:0/4164604991 wait complete. 2024-05-10T05:40:37.843 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:40:37.883 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:40:38.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.426+0000 7eff8dc00700 1 -- 172.21.15.131:0/324841174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff88109330 msgr2=0x7eff88109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.426+0000 7eff8dc00700 1 --2- 172.21.15.131:0/324841174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff88109330 0x7eff88109740 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7eff740099c0 tx=0x7eff74009cd0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.427+0000 7eff8dc00700 1 -- 172.21.15.131:0/324841174 shutdown_connections 2024-05-10T05:40:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.427+0000 7eff8dc00700 1 --2- 172.21.15.131:0/324841174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff88109cd0 0x7eff88110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.427+0000 7eff8dc00700 1 --2- 172.21.15.131:0/324841174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff88108210 0x7eff88108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.427+0000 7eff8dc00700 1 --2- 172.21.15.131:0/324841174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff88109330 0x7eff88109740 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.427+0000 7eff8dc00700 1 -- 172.21.15.131:0/324841174 >> 172.21.15.131:0/324841174 conn(0x7eff88077240 msgr2=0x7eff88106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.427+0000 7eff8dc00700 1 -- 172.21.15.131:0/324841174 shutdown_connections 2024-05-10T05:40:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.428+0000 7eff8dc00700 1 -- 172.21.15.131:0/324841174 wait complete. 2024-05-10T05:40:38.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.444+0000 7eff8dc00700 1 Processor -- start 2024-05-10T05:40:38.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.447+0000 7eff8dc00700 1 -- start start 2024-05-10T05:40:38.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.451+0000 7eff8dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff88108210 0x7eff8819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.451+0000 7eff8dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff88109330 0x7eff8819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.452+0000 7eff86a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff88108210 0x7eff8819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.452+0000 7eff86a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff88108210 0x7eff8819e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59292/0 (socket says 172.21.15.131:59292) 2024-05-10T05:40:38.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.459+0000 7eff8dc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff88109cd0 0x7eff881a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.462+0000 7eff87400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff88109cd0 0x7eff881a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.466+0000 7eff87400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff88109cd0 0x7eff881a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50632/0 (socket says 172.21.15.131:50632) 2024-05-10T05:40:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.462+0000 7eff86000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff88109330 0x7eff8819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.466+0000 7eff8dc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff881a5ff0 con 0x7eff88109cd0 2024-05-10T05:40:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.466+0000 7eff8dc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff881a6130 con 0x7eff88109330 2024-05-10T05:40:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.466+0000 7eff8dc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff881a62c0 con 0x7eff88108210 2024-05-10T05:40:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.466+0000 7eff86a00700 1 -- 172.21.15.131:0/1726944724 learned_addr learned my addr 172.21.15.131:0/1726944724 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.468+0000 7eff87400700 1 -- 172.21.15.131:0/1726944724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff88108210 msgr2=0x7eff8819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.468+0000 7eff87400700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff88108210 0x7eff8819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.468+0000 7eff87400700 1 -- 172.21.15.131:0/1726944724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff88109330 msgr2=0x7eff8819ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.468+0000 7eff87400700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff88109330 0x7eff8819ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.468+0000 7eff87400700 1 -- 172.21.15.131:0/1726944724 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff740095f0 con 0x7eff88109cd0 2024-05-10T05:40:38.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.469+0000 7eff87400700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff88109cd0 0x7eff881a5a20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c9a0 tx=0x7eff7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:38.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.469+0000 7eff7f400700 1 -- 172.21.15.131:0/1726944724 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7800e880 con 0x7eff88109cd0 2024-05-10T05:40:38.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.470+0000 7eff8dc00700 1 -- 172.21.15.131:0/1726944724 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff881a65a0 con 0x7eff88109cd0 2024-05-10T05:40:38.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.470+0000 7eff8dc00700 1 -- 172.21.15.131:0/1726944724 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff881a6ab0 con 0x7eff88109cd0 2024-05-10T05:40:38.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.470+0000 7eff7f400700 1 -- 172.21.15.131:0/1726944724 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff7800f450 con 0x7eff88109cd0 2024-05-10T05:40:38.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.472+0000 7eff7f400700 1 -- 172.21.15.131:0/1726944724 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7800e880 con 0x7eff88109cd0 2024-05-10T05:40:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.489+0000 7eff7f400700 1 -- 172.21.15.131:0/1726944724 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7eff78004830 con 0x7eff88109cd0 2024-05-10T05:40:38.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.491+0000 7effa5bb3b80 1 Processor -- start 2024-05-10T05:40:38.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.493+0000 7effa5bb3b80 1 -- start start 2024-05-10T05:40:38.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.494+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 0x561a02500af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.494+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02501980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.494+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501ec0 0x561a025087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.494+0000 7effa5bb3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a02427150 con 0x561a02501ec0 2024-05-10T05:40:38.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.494+0000 7effa5bb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a02424b10 con 0x561a025006a0 2024-05-10T05:40:38.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.494+0000 7effa5bb3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a02420400 con 0x561a02501570 2024-05-10T05:40:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.498+0000 7eff7f400700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff6c0a12d0 0x7eff6c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.499+0000 7eff86a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02501980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.499+0000 7eff86a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02501980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59296/0 (socket says 172.21.15.131:59296) 2024-05-10T05:40:38.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.499+0000 7eff86a00700 1 -- 172.21.15.131:0/4052024145 learned_addr learned my addr 172.21.15.131:0/4052024145 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:38.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.500+0000 7eff7f400700 1 -- 172.21.15.131:0/1726944724 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7eff7800ea40 con 0x7eff88109cd0 2024-05-10T05:40:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.501+0000 7eff86000700 1 --2- 172.21.15.131:0/4052024145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501ec0 0x561a025087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.501+0000 7eff86000700 1 --2- 172.21.15.131:0/4052024145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 0x561a02500af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.501+0000 7eff86a00700 1 -- 172.21.15.131:0/4052024145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 msgr2=0x561a02500af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:38.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.501+0000 7eff86a00700 1 --2- 172.21.15.131:0/4052024145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 0x561a02500af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.501+0000 7eff86a00700 1 -- 172.21.15.131:0/4052024145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501ec0 msgr2=0x561a025087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:38.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.501+0000 7eff86a00700 1 --2- 172.21.15.131:0/4052024145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501ec0 0x561a025087c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.501+0000 7eff86a00700 1 -- 172.21.15.131:0/4052024145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a022a13b0 con 0x561a02501570 2024-05-10T05:40:38.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.506+0000 7eff87400700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff6c0a12d0 0x7eff6c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.507+0000 7eff86a00700 1 --2- 172.21.15.131:0/4052024145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02501980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7eff80009fa0 tx=0x7eff80009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62024d1d82220fce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:38.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.510+0000 7eff73e00700 1 -- 172.21.15.131:0/4052024145 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80009a70 con 0x561a02501570 2024-05-10T05:40:38.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.510+0000 7eff73e00700 1 -- 172.21.15.131:0/4052024145 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8000f040 con 0x561a02501570 2024-05-10T05:40:38.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.510+0000 7eff73e00700 1 -- 172.21.15.131:0/4052024145 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80015c00 con 0x561a02501570 2024-05-10T05:40:38.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.511+0000 7effa5bb3b80 1 -- 172.21.15.131:0/4052024145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 msgr2=0x561a02501980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:38.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.511+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/4052024145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02501980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7eff80009fa0 tx=0x7eff80009200 comp rx=0 tx=0).stop 2024-05-10T05:40:38.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.513+0000 7effa5bb3b80 1 -- 172.21.15.131:0/4052024145 shutdown_connections 2024-05-10T05:40:38.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.513+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/4052024145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501ec0 0x561a025087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.513+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/4052024145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 0x561a02500af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.513+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/4052024145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02501980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.513+0000 7effa5bb3b80 1 -- 172.21.15.131:0/4052024145 >> 172.21.15.131:0/4052024145 conn(0x561a02462a60 msgr2=0x561a024fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:38.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.515+0000 7eff87400700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff6c0a12d0 0x7eff6c0a37e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7eff78096370 tx=0x7eff78005b00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:38.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.517+0000 7effa5bb3b80 1 -- 172.21.15.131:0/4052024145 shutdown_connections 2024-05-10T05:40:38.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.522+0000 7effa5bb3b80 1 -- 172.21.15.131:0/4052024145 wait complete. 2024-05-10T05:40:38.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.523+0000 7effa5bb3b80 1 -- start start 2024-05-10T05:40:38.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.524+0000 7effa5bb3b80 1 Processor -- start 2024-05-10T05:40:38.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.531+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 0x561a0250a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.531+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501570 0x561a0250ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.531+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501ec0 0x561a02511b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.531+0000 7effa5bb3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a023bc560 con 0x561a02501570 2024-05-10T05:40:38.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.531+0000 7effa5bb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a023bc200 con 0x561a025006a0 2024-05-10T05:40:38.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.531+0000 7effa5bb3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a022a36a0 con 0x561a02501ec0 2024-05-10T05:40:38.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.531+0000 7effa5bb3b80 10 client.0 _ll_register_callbacks cb 0x561a023c5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.533+0000 7eff86000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 0x561a0250a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.534+0000 7eff86000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 0x561a0250a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45096/0 (socket says 172.21.15.131:45096) 2024-05-10T05:40:38.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.534+0000 7eff86000700 1 -- 172.21.15.131:0/1197254710 learned_addr learned my addr 172.21.15.131:0/1197254710 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:38.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.534+0000 7eff86000700 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501ec0 0x561a02511b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.535+0000 7eff86a00700 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501570 0x561a0250ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.535+0000 7eff86000700 1 -- 172.21.15.131:0/1197254710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501ec0 msgr2=0x561a02511b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.535+0000 7eff86000700 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501ec0 0x561a02511b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.535+0000 7eff86000700 1 -- 172.21.15.131:0/1197254710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501570 msgr2=0x561a0250ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:38.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.536+0000 7eff86000700 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501570 0x561a0250ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.536+0000 7eff86000700 1 -- 172.21.15.131:0/1197254710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff80009bd0 con 0x561a025006a0 2024-05-10T05:40:38.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.536+0000 7eff86000700 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 0x561a0250a6d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7eff740099c0 tx=0x7eff74009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20512d68e1b43e27 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:38.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.536+0000 7eff70c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:38.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.537+0000 7eff86a00700 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501570 0x561a0250ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:38.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.537+0000 7eff70c00700 1 -- 172.21.15.131:0/1197254710 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7401d070 con 0x561a025006a0 2024-05-10T05:40:38.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.537+0000 7effa5bb3b80 1 -- 172.21.15.131:0/1197254710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a02379270 con 0x561a025006a0 2024-05-10T05:40:38.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.537+0000 7eff70c00700 1 -- 172.21.15.131:0/1197254710 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7eff74003ad0 con 0x561a025006a0 2024-05-10T05:40:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.538+0000 7eff70c00700 1 -- 172.21.15.131:0/1197254710 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff74003c90 con 0x561a025006a0 2024-05-10T05:40:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.538+0000 7effa5bb3b80 10 client.26636 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.538+0000 7effa5bb3b80 1 -- 172.21.15.131:0/1197254710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561a0224db60 con 0x561a025006a0 2024-05-10T05:40:38.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.542+0000 7eff70c00700 1 -- 172.21.15.131:0/1197254710 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a02379270 con 0x561a025006a0 2024-05-10T05:40:38.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.542+0000 7eff70c00700 1 -- 172.21.15.131:0/1197254710 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7eff7400f9a0 con 0x561a025006a0 2024-05-10T05:40:38.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.542+0000 7effa5bb3b80 10 client.26636 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:38.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.542+0000 7effa5bb3b80 10 client.26636 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:40:38.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.542+0000 7effa5bb3b80 20 client.26636 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:38.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.542+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561a025163b0 0x561a02518800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.542+0000 7effa5bb3b80 4 client.26636 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:38.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.542+0000 7effa5bb3b80 1 -- 172.21.15.131:0/1197254710 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a024cf780 con 0x561a025163b0 2024-05-10T05:40:38.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.544+0000 7eff86a00700 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561a025163b0 0x561a02518800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.548+0000 7eff86a00700 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561a025163b0 0x561a02518800 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4bb33d999a56af1e server_cookie=609ffdd54d2509ed in_seq=0 out_seq=0 2024-05-10T05:40:38.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.548+0000 7eff70c00700 10 client.26636 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:40:38.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.549+0000 7eff70c00700 1 -- 172.21.15.131:0/1197254710 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a024cf780 con 0x561a025163b0 2024-05-10T05:40:38.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.549+0000 7eff70c00700 10 client.26636 handle_command_reply: tid=0 2024-05-10T05:40:38.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.549+0000 7effa5bb3b80 1 client.26636 shutdown 2024-05-10T05:40:38.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:38 smithi157 ceph-mon[78427]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 220 MiB/s wr, 69 op/s 2024-05-10T05:40:38.552 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:38 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4164604991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:40:38.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.552+0000 7effa5bb3b80 1 -- 172.21.15.131:0/1197254710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 msgr2=0x561a0250a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:38.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.552+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 0x561a0250a6d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7eff740099c0 tx=0x7eff74009310 comp rx=0 tx=0).stop 2024-05-10T05:40:38.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.554+0000 7effa5bb3b80 1 -- 172.21.15.131:0/1197254710 shutdown_connections 2024-05-10T05:40:38.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.554+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561a025163b0 0x561a02518800 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.554+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501ec0 0x561a02511b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.554+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a025006a0 0x561a0250a6d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.554+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/1197254710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a02501570 0x561a0250ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.554+0000 7effa5bb3b80 1 -- 172.21.15.131:0/1197254710 >> 172.21.15.131:0/1197254710 conn(0x561a02462a60 msgr2=0x561a024fafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:38.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.554+0000 7eff70c00700 0 client.26636 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:40:38.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.555+0000 7effa5bb3b80 1 -- 172.21.15.131:0/1197254710 shutdown_connections 2024-05-10T05:40:38.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.557+0000 7effa5bb3b80 1 -- 172.21.15.131:0/1197254710 wait complete. 2024-05-10T05:40:38.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.557+0000 7effa5bb3b80 20 client.26636 trim_cache size 0 max 16384 2024-05-10T05:40:38.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.610+0000 7effa5bb3b80 1 Processor -- start 2024-05-10T05:40:38.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.611+0000 7effa5bb3b80 1 -- start start 2024-05-10T05:40:38.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.611+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 0x561a02518ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.611+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02511b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7eff86a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02511b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7eff86a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02511b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59318/0 (socket says 172.21.15.131:59318) 2024-05-10T05:40:38.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7eff86a00700 1 -- 172.21.15.131:0/3849666324 learned_addr learned my addr 172.21.15.131:0/3849666324 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:38.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/3849666324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501ec0 0x561a02508a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7effa5bb3b80 1 -- 172.21.15.131:0/3849666324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0250b7b0 con 0x561a025006a0 2024-05-10T05:40:38.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7effa5bb3b80 1 -- 172.21.15.131:0/3849666324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a024fc770 con 0x561a02501ec0 2024-05-10T05:40:38.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7effa5bb3b80 1 -- 172.21.15.131:0/3849666324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a024f57c0 con 0x561a02501570 2024-05-10T05:40:38.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7eff86000700 1 --2- 172.21.15.131:0/3849666324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 0x561a02518ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7eff86a00700 1 -- 172.21.15.131:0/3849666324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501ec0 msgr2=0x561a02508a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:40:38.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7eff86a00700 1 --2- 172.21.15.131:0/3849666324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501ec0 0x561a02508a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7eff86a00700 1 -- 172.21.15.131:0/3849666324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 msgr2=0x561a02518ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:38.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7eff86a00700 1 --2- 172.21.15.131:0/3849666324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 0x561a02518ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.612+0000 7eff86a00700 1 -- 172.21.15.131:0/3849666324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a02500d80 con 0x561a02501570 2024-05-10T05:40:38.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.613+0000 7eff86a00700 1 --2- 172.21.15.131:0/3849666324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02511b00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7eff80009f70 tx=0x7eff80009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab6cfbe25279a28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:38.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.613+0000 7eff7cc00700 1 -- 172.21.15.131:0/3849666324 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80025d70 con 0x561a02501570 2024-05-10T05:40:38.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.613+0000 7eff7cc00700 1 -- 172.21.15.131:0/3849666324 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff80025370 con 0x561a02501570 2024-05-10T05:40:38.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.614+0000 7effa5bb3b80 1 -- 172.21.15.131:0/3849666324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 msgr2=0x561a02511b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:38.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.614+0000 7eff7cc00700 1 -- 172.21.15.131:0/3849666324 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80025d70 con 0x561a02501570 2024-05-10T05:40:38.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.615+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/3849666324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02511b00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7eff80009f70 tx=0x7eff80009040 comp rx=0 tx=0).stop 2024-05-10T05:40:38.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.615+0000 7effa5bb3b80 1 -- 172.21.15.131:0/3849666324 shutdown_connections 2024-05-10T05:40:38.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.615+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/3849666324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 0x561a02518ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.615+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/3849666324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501ec0 0x561a02508a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.615+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/3849666324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501570 0x561a02511b00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.615+0000 7effa5bb3b80 1 -- 172.21.15.131:0/3849666324 >> 172.21.15.131:0/3849666324 conn(0x561a02462a60 msgr2=0x561a025183c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:38.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.615+0000 7effa5bb3b80 1 -- 172.21.15.131:0/3849666324 shutdown_connections 2024-05-10T05:40:38.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.615+0000 7effa5bb3b80 1 -- 172.21.15.131:0/3849666324 wait complete. 2024-05-10T05:40:38.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.616+0000 7effa5bb3b80 1 -- start start 2024-05-10T05:40:38.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.617+0000 7effa5bb3b80 1 Processor -- start 2024-05-10T05:40:38.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.618+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 0x561a0250f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.618+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501570 0x561a0250f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.619+0000 7effa5bb3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501ec0 0x561a02510640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.619+0000 7effa5bb3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a02510b80 con 0x561a025006a0 2024-05-10T05:40:38.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.619+0000 7effa5bb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a02510cf0 con 0x561a02501570 2024-05-10T05:40:38.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.619+0000 7effa5bb3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a02510eb0 con 0x561a02501ec0 2024-05-10T05:40:38.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.619+0000 7eff86a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501570 0x561a0250f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.619+0000 7eff86a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501570 0x561a0250f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45114/0 (socket says 172.21.15.131:45114) 2024-05-10T05:40:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.619+0000 7eff86a00700 1 -- 172.21.15.131:0/95663095 learned_addr learned my addr 172.21.15.131:0/95663095 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.619+0000 7eff86000700 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 0x561a0250f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.619+0000 7effa5bb3b80 10 client.0 _ll_register_callbacks cb 0x561a023c5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.620+0000 7eff86a00700 1 -- 172.21.15.131:0/95663095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501ec0 msgr2=0x561a02510640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.620+0000 7eff86a00700 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501ec0 0x561a02510640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.620+0000 7eff86a00700 1 -- 172.21.15.131:0/95663095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 msgr2=0x561a0250f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.620+0000 7eff86a00700 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 0x561a0250f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.620+0000 7eff86a00700 1 -- 172.21.15.131:0/95663095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a022a13b0 con 0x561a02501570 2024-05-10T05:40:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.620+0000 7eff86000700 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 0x561a0250f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:40:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.621+0000 7eff86a00700 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501570 0x561a0250f710 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7eff80025e00 tx=0x7eff80015540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182864f92759370e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.621+0000 7eff70c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.621+0000 7eff70c00700 1 -- 172.21.15.131:0/95663095 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80004e40 con 0x561a02501570 2024-05-10T05:40:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.621+0000 7eff70c00700 1 -- 172.21.15.131:0/95663095 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7eff80023070 con 0x561a02501570 2024-05-10T05:40:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.622+0000 7effa5bb3b80 1 -- 172.21.15.131:0/95663095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff8000f040 con 0x561a02501570 2024-05-10T05:40:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.622+0000 7eff70c00700 1 -- 172.21.15.131:0/95663095 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff800256e0 con 0x561a02501570 2024-05-10T05:40:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.622+0000 7effa5bb3b80 10 client.26639 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.622+0000 7effa5bb3b80 1 -- 172.21.15.131:0/95663095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561a023e6560 con 0x561a02501570 2024-05-10T05:40:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.623+0000 7eff70c00700 1 -- 172.21.15.131:0/95663095 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8000f040 con 0x561a02501570 2024-05-10T05:40:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.623+0000 7eff70c00700 1 -- 172.21.15.131:0/95663095 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7eff800146a0 con 0x561a02501570 2024-05-10T05:40:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.623+0000 7effa5bb3b80 10 client.26639 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.623+0000 7effa5bb3b80 10 client.26639 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:40:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.623+0000 7effa5bb3b80 20 client.26639 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:38.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.623+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561a0256ccb0 0x561a0256f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:38.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.623+0000 7effa5bb3b80 4 client.26639 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:40:38.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.623+0000 7effa5bb3b80 1 -- 172.21.15.131:0/95663095 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561a02418ec0 con 0x561a0256ccb0 2024-05-10T05:40:38.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.625+0000 7eff86000700 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561a0256ccb0 0x561a0256f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:38.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.628+0000 7eff86000700 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561a0256ccb0 0x561a0256f0d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e28222b21183eca server_cookie=f43a1b3dd4c06693 in_seq=0 out_seq=0 2024-05-10T05:40:38.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.628+0000 7eff70c00700 10 client.26639 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:40:38.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.628+0000 7eff70c00700 1 -- 172.21.15.131:0/95663095 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561a02418ec0 con 0x561a0256ccb0 2024-05-10T05:40:38.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.629+0000 7eff70c00700 10 client.26639 handle_command_reply: tid=0 2024-05-10T05:40:38.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.630+0000 7effa5bb3b80 1 client.26639 shutdown 2024-05-10T05:40:38.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.632+0000 7effa5bb3b80 1 -- 172.21.15.131:0/95663095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501570 msgr2=0x561a0250f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:38.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.632+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501570 0x561a0250f710 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7eff80025e00 tx=0x7eff80015540 comp rx=0 tx=0).stop 2024-05-10T05:40:38.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.634+0000 7effa5bb3b80 1 -- 172.21.15.131:0/95663095 shutdown_connections 2024-05-10T05:40:38.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.634+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x561a0256ccb0 0x561a0256f0d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.634+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a025006a0 0x561a0250f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.634+0000 7eff70c00700 0 client.26639 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:40:38.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.637+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a02501ec0 0x561a02510640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.637+0000 7effa5bb3b80 1 --2- 172.21.15.131:0/95663095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a02501570 0x561a0250f710 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.637+0000 7effa5bb3b80 1 -- 172.21.15.131:0/95663095 >> 172.21.15.131:0/95663095 conn(0x561a02462a60 msgr2=0x561a025089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.637+0000 7effa5bb3b80 1 -- 172.21.15.131:0/95663095 shutdown_connections 2024-05-10T05:40:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.637+0000 7effa5bb3b80 1 -- 172.21.15.131:0/95663095 wait complete. 2024-05-10T05:40:38.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.638+0000 7effa5bb3b80 20 client.26639 trim_cache size 0 max 16384 2024-05-10T05:40:38.639 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:40:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:38 smithi142 ceph-mon[77667]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 220 MiB/s wr, 69 op/s 2024-05-10T05:40:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:38 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4164604991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:40:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.643+0000 7eff8dc00700 1 -- 172.21.15.131:0/1726944724 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff6c0a12d0 msgr2=0x7eff6c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.643+0000 7eff8dc00700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff6c0a12d0 0x7eff6c0a37e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7eff78096370 tx=0x7eff78005b00 comp rx=0 tx=0).stop 2024-05-10T05:40:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.643+0000 7eff8dc00700 1 -- 172.21.15.131:0/1726944724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff88109cd0 msgr2=0x7eff881a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.643+0000 7eff8dc00700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff88109cd0 0x7eff881a5a20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c9a0 tx=0x7eff7800ccb0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.644+0000 7eff8dc00700 1 -- 172.21.15.131:0/1726944724 shutdown_connections 2024-05-10T05:40:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.644+0000 7eff8dc00700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff6c0a12d0 0x7eff6c0a37e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.644+0000 7eff8dc00700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff88109cd0 0x7eff881a5a20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.644+0000 7eff8dc00700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff88108210 0x7eff8819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.644+0000 7eff8dc00700 1 --2- 172.21.15.131:0/1726944724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff88109330 0x7eff8819ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:38.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.644+0000 7eff8dc00700 1 -- 172.21.15.131:0/1726944724 >> 172.21.15.131:0/1726944724 conn(0x7eff88077240 msgr2=0x7eff8810c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:38.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.647+0000 7eff8dc00700 1 -- 172.21.15.131:0/1726944724 shutdown_connections 2024-05-10T05:40:38.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:38.647+0000 7eff8dc00700 1 -- 172.21.15.131:0/1726944724 wait complete. 2024-05-10T05:40:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:38 smithi131 ceph-mon[37264]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 220 MiB/s wr, 69 op/s 2024-05-10T05:40:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:38 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4164604991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:40:38.717 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:40:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.324+0000 7f5687200700 1 -- 172.21.15.131:0/880486696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5680108210 msgr2=0x7f5680108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.324+0000 7f5687200700 1 --2- 172.21.15.131:0/880486696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5680108210 0x7f5680108620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f567800b0d0 tx=0x7f567800b3e0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.325+0000 7f5687200700 1 -- 172.21.15.131:0/880486696 shutdown_connections 2024-05-10T05:40:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.325+0000 7f5687200700 1 --2- 172.21.15.131:0/880486696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680110320 0x7f5680112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.325+0000 7f5687200700 1 --2- 172.21.15.131:0/880486696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5680108210 0x7f5680108620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.325+0000 7f5687200700 1 --2- 172.21.15.131:0/880486696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5680108b60 0x7f568010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.325+0000 7f5687200700 1 -- 172.21.15.131:0/880486696 >> 172.21.15.131:0/880486696 conn(0x7f568006ea00 msgr2=0x7f5680070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.326+0000 7f5687200700 1 -- 172.21.15.131:0/880486696 shutdown_connections 2024-05-10T05:40:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.326+0000 7f5687200700 1 -- 172.21.15.131:0/880486696 wait complete. 2024-05-10T05:40:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.327+0000 7f5687200700 1 Processor -- start 2024-05-10T05:40:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.327+0000 7f5687200700 1 -- start start 2024-05-10T05:40:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.329+0000 7f5687200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5680108210 0x7f56801a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.329+0000 7f5687200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680108b60 0x7f56801a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.329+0000 7f5687200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5680110320 0x7f56801a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.329+0000 7f5687200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801a81e0 con 0x7f5680110320 2024-05-10T05:40:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.329+0000 7f5687200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801a8320 con 0x7f5680108210 2024-05-10T05:40:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.329+0000 7f5687200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5680109c30 con 0x7f5680108b60 2024-05-10T05:40:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.330+0000 7f567fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5680108210 0x7f56801a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.330+0000 7f567fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5680108210 0x7f56801a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45128/0 (socket says 172.21.15.131:45128) 2024-05-10T05:40:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.330+0000 7f567fe00700 1 -- 172.21.15.131:0/3954577530 learned_addr learned my addr 172.21.15.131:0/3954577530 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:39.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.330+0000 7f567fe00700 1 -- 172.21.15.131:0/3954577530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680108b60 msgr2=0x7f56801a0c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:40:39.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.330+0000 7f567fe00700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680108b60 0x7f56801a0c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.330+0000 7f567fe00700 1 -- 172.21.15.131:0/3954577530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5680110320 msgr2=0x7f56801a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:39.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.330+0000 7f567fe00700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5680110320 0x7f56801a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.330+0000 7f567fe00700 1 -- 172.21.15.131:0/3954577530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678009d00 con 0x7f5680108210 2024-05-10T05:40:39.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.331+0000 7f567fe00700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5680108210 0x7f56801a0740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f56780093b0 tx=0x7f5678009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:39.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.331+0000 7f567cc00700 1 -- 172.21.15.131:0/3954577530 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567800e040 con 0x7f5680108210 2024-05-10T05:40:39.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.331+0000 7f567cc00700 1 -- 172.21.15.131:0/3954577530 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f567801b920 con 0x7f5680108210 2024-05-10T05:40:39.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.331+0000 7f5687200700 1 -- 172.21.15.131:0/3954577530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5680109eb0 con 0x7f5680108210 2024-05-10T05:40:39.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.332+0000 7f567cc00700 1 -- 172.21.15.131:0/3954577530 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678005540 con 0x7f5680108210 2024-05-10T05:40:39.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.332+0000 7f5687200700 1 -- 172.21.15.131:0/3954577530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568010a3b0 con 0x7f5680108210 2024-05-10T05:40:39.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.339+0000 7f567cc00700 1 -- 172.21.15.131:0/3954577530 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f56780056a0 con 0x7f5680108210 2024-05-10T05:40:39.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.342+0000 7f567cc00700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56640a12c0 0x7f56640a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.342+0000 7f567cc00700 1 -- 172.21.15.131:0/3954577530 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f56780ce8b0 con 0x7f5680108210 2024-05-10T05:40:39.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.343+0000 7f567f400700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56640a12c0 0x7f56640a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.343+0000 7f567f400700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56640a12c0 0x7f56640a37d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f56801a17e0 tx=0x7f5670005c90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:39.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.347+0000 7f569f04cb80 1 Processor -- start 2024-05-10T05:40:39.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.348+0000 7f569f04cb80 1 -- start start 2024-05-10T05:40:39.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.349+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 0x55d901874300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.349+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 0x55d901875190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.349+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 0x55d90187bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.349+0000 7f569f04cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d901792150 con 0x55d901874d80 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.349+0000 7f569f04cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90178fb10 con 0x55d901873eb0 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.349+0000 7f569f04cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90178b400 con 0x55d9018756d0 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.350+0000 7f567fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 0x55d901874300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.350+0000 7f567fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 0x55d901874300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45138/0 (socket says 172.21.15.131:45138) 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.350+0000 7f567fe00700 1 -- 172.21.15.131:0/3939057883 learned_addr learned my addr 172.21.15.131:0/3939057883 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.350+0000 7f5685200700 1 --2- 172.21.15.131:0/3939057883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 0x55d90187bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.350+0000 7f567fe00700 1 -- 172.21.15.131:0/3939057883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 msgr2=0x55d90187bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.350+0000 7f567fe00700 1 --2- 172.21.15.131:0/3939057883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 0x55d90187bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.350+0000 7f567fe00700 1 -- 172.21.15.131:0/3939057883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 msgr2=0x55d901875190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.350+0000 7f567f400700 1 --2- 172.21.15.131:0/3939057883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 0x55d901875190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.351+0000 7f567fe00700 1 --2- 172.21.15.131:0/3939057883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 0x55d901875190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.351+0000 7f567fe00700 1 -- 172.21.15.131:0/3939057883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d90160c3b0 con 0x55d901873eb0 2024-05-10T05:40:39.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.351+0000 7f567f400700 1 --2- 172.21.15.131:0/3939057883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 0x55d901875190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:39.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.351+0000 7f567fe00700 1 --2- 172.21.15.131:0/3939057883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 0x55d901874300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f567800b820 tx=0x7f567800b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962698d827611bff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:39.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.351+0000 7f5675600700 1 -- 172.21.15.131:0/3939057883 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567809bd30 con 0x55d901873eb0 2024-05-10T05:40:39.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.351+0000 7f5675600700 1 -- 172.21.15.131:0/3939057883 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f567809fb60 con 0x55d901873eb0 2024-05-10T05:40:39.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.351+0000 7f5675600700 1 -- 172.21.15.131:0/3939057883 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567809bf90 con 0x55d901873eb0 2024-05-10T05:40:39.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.353+0000 7f569f04cb80 1 -- 172.21.15.131:0/3939057883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 msgr2=0x55d901874300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:39.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.353+0000 7f569f04cb80 1 --2- 172.21.15.131:0/3939057883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 0x55d901874300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f567800b820 tx=0x7f567800b3e0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.353+0000 7f569f04cb80 1 -- 172.21.15.131:0/3939057883 shutdown_connections 2024-05-10T05:40:39.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.353+0000 7f569f04cb80 1 --2- 172.21.15.131:0/3939057883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 0x55d90187bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.353+0000 7f569f04cb80 1 --2- 172.21.15.131:0/3939057883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 0x55d901874300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.353+0000 7f569f04cb80 1 --2- 172.21.15.131:0/3939057883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 0x55d901875190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.353+0000 7f569f04cb80 1 -- 172.21.15.131:0/3939057883 >> 172.21.15.131:0/3939057883 conn(0x55d9017cda60 msgr2=0x55d901865fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:39.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.354+0000 7f569f04cb80 1 -- 172.21.15.131:0/3939057883 shutdown_connections 2024-05-10T05:40:39.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.354+0000 7f569f04cb80 1 -- 172.21.15.131:0/3939057883 wait complete. 2024-05-10T05:40:39.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.358+0000 7f569f04cb80 1 -- start start 2024-05-10T05:40:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.361+0000 7f569f04cb80 1 Processor -- start 2024-05-10T05:40:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.362+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 0x55d90187dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.362+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d90187e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.362+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9018756d0 0x55d9018851d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.362+0000 7f569f04cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d901727560 con 0x55d9018756d0 2024-05-10T05:40:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.362+0000 7f569f04cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d901727200 con 0x55d901874d80 2024-05-10T05:40:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.362+0000 7f569f04cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90160e6a0 con 0x55d901873eb0 2024-05-10T05:40:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.362+0000 7f569f04cb80 10 client.0 _ll_register_callbacks cb 0x55d901730b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.363+0000 7f567fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 0x55d90187dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.363+0000 7f567f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d90187e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.363+0000 7f567f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d90187e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45144/0 (socket says 172.21.15.131:45144) 2024-05-10T05:40:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.363+0000 7f567f400700 1 -- 172.21.15.131:0/3735366017 learned_addr learned my addr 172.21.15.131:0/3735366017 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.363+0000 7f5685200700 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9018756d0 0x55d9018851d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.364+0000 7f5685200700 1 -- 172.21.15.131:0/3735366017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 msgr2=0x55d90187dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.364+0000 7f5685200700 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 0x55d90187dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.364+0000 7f5685200700 1 -- 172.21.15.131:0/3735366017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 msgr2=0x55d90187e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.364+0000 7f5685200700 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d90187e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.364+0000 7f5685200700 1 -- 172.21.15.131:0/3735366017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568010a3b0 con 0x55d9018756d0 2024-05-10T05:40:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.365+0000 7f567f400700 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d90187e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:40:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.365+0000 7f5685200700 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9018756d0 0x55d9018851d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f566c003900 tx=0x7f566c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ca12a77bf7d645 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.365+0000 7f566a000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:40:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.366+0000 7f566a000700 1 -- 172.21.15.131:0/3735366017 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c00bbf0 con 0x55d9018756d0 2024-05-10T05:40:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.366+0000 7f566a000700 1 -- 172.21.15.131:0/3735366017 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f566c00bd50 con 0x55d9018756d0 2024-05-10T05:40:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.366+0000 7f569f04cb80 1 -- 172.21.15.131:0/3735366017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9016e4270 con 0x55d9018756d0 2024-05-10T05:40:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.366+0000 7f566a000700 1 -- 172.21.15.131:0/3735366017 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f566c01a470 con 0x55d9018756d0 2024-05-10T05:40:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.367+0000 7f569f04cb80 10 client.16425 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.368+0000 7f569f04cb80 1 -- 172.21.15.131:0/3735366017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d9015b8b60 con 0x55d9018756d0 2024-05-10T05:40:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.370+0000 7f566a000700 1 -- 172.21.15.131:0/3735366017 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9016e4270 con 0x55d9018756d0 2024-05-10T05:40:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.371+0000 7f566a000700 1 -- 172.21.15.131:0/3735366017 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f566c011740 con 0x55d9018756d0 2024-05-10T05:40:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.374+0000 7f569f04cb80 10 client.16425 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.374+0000 7f569f04cb80 10 client.16425 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:40:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.374+0000 7f569f04cb80 20 client.16425 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.374+0000 7f569f04cb80 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d90186f500 0x55d901871950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.374+0000 7f569f04cb80 4 client.16425 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.374+0000 7f569f04cb80 1 -- 172.21.15.131:0/3735366017 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d90183a780 con 0x55d90186f500 2024-05-10T05:40:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.376+0000 7f567fe00700 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d90186f500 0x55d901871950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.379+0000 7f567fe00700 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d90186f500 0x55d901871950 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=34f37cc1e73a0fb1 server_cookie=6f8465f0fc73d43 in_seq=0 out_seq=0 2024-05-10T05:40:39.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.379+0000 7f566a000700 10 client.16425 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:40:39.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.380+0000 7f566a000700 1 -- 172.21.15.131:0/3735366017 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d90183a780 con 0x55d90186f500 2024-05-10T05:40:39.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.380+0000 7f566a000700 10 client.16425 handle_command_reply: tid=0 2024-05-10T05:40:39.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.380+0000 7f569f04cb80 1 client.16425 shutdown 2024-05-10T05:40:39.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.381+0000 7f569f04cb80 1 -- 172.21.15.131:0/3735366017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9018756d0 msgr2=0x55d9018851d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:39.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.381+0000 7f569f04cb80 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9018756d0 0x55d9018851d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f566c003900 tx=0x7f566c003c10 comp rx=0 tx=0).stop 2024-05-10T05:40:39.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.382+0000 7f569f04cb80 1 -- 172.21.15.131:0/3735366017 shutdown_connections 2024-05-10T05:40:39.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.383+0000 7f569f04cb80 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d90186f500 0x55d901871950 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.386+0000 7f569f04cb80 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9018756d0 0x55d9018851d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.386+0000 7f569f04cb80 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 0x55d90187dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.387+0000 7f569f04cb80 1 --2- 172.21.15.131:0/3735366017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d90187e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.387+0000 7f569f04cb80 1 -- 172.21.15.131:0/3735366017 >> 172.21.15.131:0/3735366017 conn(0x55d9017cda60 msgr2=0x55d901865fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:39.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.383+0000 7f566a000700 0 client.16425 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:40:39.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.388+0000 7f569f04cb80 1 -- 172.21.15.131:0/3735366017 shutdown_connections 2024-05-10T05:40:39.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.390+0000 7f569f04cb80 1 -- 172.21.15.131:0/3735366017 wait complete. 2024-05-10T05:40:39.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.395+0000 7f569f04cb80 20 client.16425 trim_cache size 0 max 16384 2024-05-10T05:40:39.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.423+0000 7f569f04cb80 1 Processor -- start 2024-05-10T05:40:39.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.425+0000 7f569f04cb80 1 -- start start 2024-05-10T05:40:39.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.425+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 0x55d901862180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.425+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 0x55d901885080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.425+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 0x55d901878c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.425+0000 7f569f04cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9016e4270 con 0x55d901874d80 2024-05-10T05:40:39.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.425+0000 7f569f04cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d901867240 con 0x55d901873eb0 2024-05-10T05:40:39.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.425+0000 7f569f04cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d901865c90 con 0x55d9018756d0 2024-05-10T05:40:39.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.426+0000 7f567f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 0x55d901885080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.426+0000 7f567f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 0x55d901885080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50700/0 (socket says 172.21.15.131:50700) 2024-05-10T05:40:39.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.426+0000 7f567f400700 1 -- 172.21.15.131:0/1307558495 learned_addr learned my addr 172.21.15.131:0/1307558495 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:39.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.426+0000 7f5685200700 1 --2- 172.21.15.131:0/1307558495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 0x55d901878c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.427+0000 7f5685200700 1 -- 172.21.15.131:0/1307558495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 msgr2=0x55d901862180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:40:39.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.427+0000 7f5685200700 1 --2- 172.21.15.131:0/1307558495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 0x55d901862180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.427+0000 7f5685200700 1 -- 172.21.15.131:0/1307558495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 msgr2=0x55d901885080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:39.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.427+0000 7f5685200700 1 --2- 172.21.15.131:0/1307558495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 0x55d901885080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.427+0000 7f5685200700 1 -- 172.21.15.131:0/1307558495 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d901751560 con 0x55d9018756d0 2024-05-10T05:40:39.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.429+0000 7f5685200700 1 --2- 172.21.15.131:0/1307558495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 0x55d901878c40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f566c010020 tx=0x7f566c004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0f6ecb94286c0f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:39.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.431+0000 7f5676000700 1 -- 172.21.15.131:0/1307558495 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c01a470 con 0x55d9018756d0 2024-05-10T05:40:39.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.431+0000 7f5676000700 1 -- 172.21.15.131:0/1307558495 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f566c00beb0 con 0x55d9018756d0 2024-05-10T05:40:39.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.431+0000 7f5676000700 1 -- 172.21.15.131:0/1307558495 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c01ac60 con 0x55d9018756d0 2024-05-10T05:40:39.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.432+0000 7f569f04cb80 1 -- 172.21.15.131:0/1307558495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 msgr2=0x55d901878c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:39.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.432+0000 7f569f04cb80 1 --2- 172.21.15.131:0/1307558495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 0x55d901878c40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f566c010020 tx=0x7f566c004740 comp rx=0 tx=0).stop 2024-05-10T05:40:39.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.432+0000 7f569f04cb80 1 -- 172.21.15.131:0/1307558495 shutdown_connections 2024-05-10T05:40:39.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.433+0000 7f569f04cb80 1 --2- 172.21.15.131:0/1307558495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d9018756d0 0x55d901878c40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.433+0000 7f569f04cb80 1 --2- 172.21.15.131:0/1307558495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901873eb0 0x55d901862180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.433+0000 7f569f04cb80 1 --2- 172.21.15.131:0/1307558495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d901874d80 0x55d901885080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.433+0000 7f569f04cb80 1 -- 172.21.15.131:0/1307558495 >> 172.21.15.131:0/1307558495 conn(0x55d9017cda60 msgr2=0x55d901865f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:39.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.433+0000 7f569f04cb80 1 -- 172.21.15.131:0/1307558495 shutdown_connections 2024-05-10T05:40:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.433+0000 7f569f04cb80 1 -- 172.21.15.131:0/1307558495 wait complete. 2024-05-10T05:40:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.434+0000 7f569f04cb80 1 -- start start 2024-05-10T05:40:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.434+0000 7f569f04cb80 1 Processor -- start 2024-05-10T05:40:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.435+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 0x55d901880b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.435+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d901881080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.435+0000 7f569f04cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9018756d0 0x55d901883fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.435+0000 7f569f04cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d901884500 con 0x55d9018756d0 2024-05-10T05:40:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.435+0000 7f569f04cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d901884670 con 0x55d901874d80 2024-05-10T05:40:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.435+0000 7f569f04cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d901884830 con 0x55d901873eb0 2024-05-10T05:40:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.435+0000 7f569f04cb80 10 client.0 _ll_register_callbacks cb 0x55d901730b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.437+0000 7f567fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 0x55d901880b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.437+0000 7f567fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 0x55d901880b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59380/0 (socket says 172.21.15.131:59380) 2024-05-10T05:40:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.437+0000 7f567fe00700 1 -- 172.21.15.131:0/745766361 learned_addr learned my addr 172.21.15.131:0/745766361 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.437+0000 7f567f400700 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d901881080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.438+0000 7f567f400700 1 -- 172.21.15.131:0/745766361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 msgr2=0x55d901880b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.438+0000 7f567f400700 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 0x55d901880b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.438+0000 7f567f400700 1 -- 172.21.15.131:0/745766361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9018756d0 msgr2=0x55d901883fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.438+0000 7f567f400700 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9018756d0 0x55d901883fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.438+0000 7f567f400700 1 -- 172.21.15.131:0/745766361 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9015b8b60 con 0x55d901874d80 2024-05-10T05:40:39.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.438+0000 7f567f400700 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d901881080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55d90187ee90 tx=0x7f56700047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42658374a39de7e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:39.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.438+0000 7f566a000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:39.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.439+0000 7f566a000700 1 -- 172.21.15.131:0/745766361 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5670004220 con 0x55d901874d80 2024-05-10T05:40:39.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.439+0000 7f569f04cb80 1 -- 172.21.15.131:0/745766361 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f567809fb60 con 0x55d901874d80 2024-05-10T05:40:39.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.439+0000 7f566a000700 1 -- 172.21.15.131:0/745766361 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5670004380 con 0x55d901874d80 2024-05-10T05:40:39.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.440+0000 7f566a000700 1 -- 172.21.15.131:0/745766361 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5670004540 con 0x55d901874d80 2024-05-10T05:40:39.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.440+0000 7f569f04cb80 10 client.26654 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:39.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.440+0000 7f569f04cb80 1 -- 172.21.15.131:0/745766361 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d901610cd0 con 0x55d901874d80 2024-05-10T05:40:39.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.441+0000 7f566a000700 1 -- 172.21.15.131:0/745766361 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f567809fb60 con 0x55d901874d80 2024-05-10T05:40:39.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.442+0000 7f566a000700 1 -- 172.21.15.131:0/745766361 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f567001ec90 con 0x55d901874d80 2024-05-10T05:40:39.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.443+0000 7f569f04cb80 10 client.26654 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:39.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.443+0000 7f569f04cb80 10 client.26654 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:40:39.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.443+0000 7f569f04cb80 20 client.26654 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:39.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.443+0000 7f569f04cb80 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d9018e8a40 0x55d9018eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:39.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.443+0000 7f569f04cb80 4 client.26654 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:40:39.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.443+0000 7f569f04cb80 1 -- 172.21.15.131:0/745766361 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d901783ec0 con 0x55d9018e8a40 2024-05-10T05:40:39.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.443+0000 7f567fe00700 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d9018e8a40 0x55d9018eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:39.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.444+0000 7f566a000700 10 client.26654 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:40:39.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.444+0000 7f567fe00700 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d9018e8a40 0x55d9018eae60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1959fc0bdffd5b20 server_cookie=9b448614726daa3c in_seq=0 out_seq=0 2024-05-10T05:40:39.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.444+0000 7f566a000700 1 -- 172.21.15.131:0/745766361 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d901783ec0 con 0x55d9018e8a40 2024-05-10T05:40:39.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.444+0000 7f566a000700 10 client.26654 handle_command_reply: tid=0 2024-05-10T05:40:39.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.444+0000 7f569f04cb80 1 client.26654 shutdown 2024-05-10T05:40:39.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.445+0000 7f569f04cb80 1 -- 172.21.15.131:0/745766361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 msgr2=0x55d901881080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:39.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.445+0000 7f569f04cb80 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d901881080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55d90187ee90 tx=0x7f56700047c0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.445+0000 7f569f04cb80 1 -- 172.21.15.131:0/745766361 shutdown_connections 2024-05-10T05:40:39.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.445+0000 7f569f04cb80 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55d9018e8a40 0x55d9018eae60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.445+0000 7f569f04cb80 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9018756d0 0x55d901883fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.445+0000 7f569f04cb80 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d901873eb0 0x55d901880b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.445+0000 7f569f04cb80 1 --2- 172.21.15.131:0/745766361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d901874d80 0x55d901881080 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.445+0000 7f569f04cb80 1 -- 172.21.15.131:0/745766361 >> 172.21.15.131:0/745766361 conn(0x55d9017cda60 msgr2=0x55d901878c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:39.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.445+0000 7f566a000700 0 client.26654 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:40:39.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.445+0000 7f569f04cb80 1 -- 172.21.15.131:0/745766361 shutdown_connections 2024-05-10T05:40:39.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.446+0000 7f569f04cb80 1 -- 172.21.15.131:0/745766361 wait complete. 2024-05-10T05:40:39.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.446+0000 7f569f04cb80 20 client.26654 trim_cache size 0 max 16384 2024-05-10T05:40:39.447 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:40:39.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.447+0000 7f5687200700 1 -- 172.21.15.131:0/3954577530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56640a12c0 msgr2=0x7f56640a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:39.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.447+0000 7f5687200700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56640a12c0 0x7f56640a37d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f56801a17e0 tx=0x7f5670005c90 comp rx=0 tx=0).stop 2024-05-10T05:40:39.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.447+0000 7f5687200700 1 -- 172.21.15.131:0/3954577530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5680108210 msgr2=0x7f56801a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:39.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.447+0000 7f5687200700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5680108210 0x7f56801a0740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f56780093b0 tx=0x7f5678009490 comp rx=0 tx=0).stop 2024-05-10T05:40:39.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.448+0000 7f5687200700 1 -- 172.21.15.131:0/3954577530 shutdown_connections 2024-05-10T05:40:39.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.448+0000 7f5687200700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f56640a12c0 0x7f56640a37d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.448+0000 7f5687200700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5680110320 0x7f56801a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.448+0000 7f5687200700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5680108210 0x7f56801a0740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.448+0000 7f5687200700 1 --2- 172.21.15.131:0/3954577530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680108b60 0x7f56801a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.448+0000 7f5687200700 1 -- 172.21.15.131:0/3954577530 >> 172.21.15.131:0/3954577530 conn(0x7f568006ea00 msgr2=0x7f568006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.449+0000 7f5687200700 1 -- 172.21.15.131:0/3954577530 shutdown_connections 2024-05-10T05:40:39.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:39.449+0000 7f5687200700 1 -- 172.21.15.131:0/3954577530 wait complete. 2024-05-10T05:40:39.500 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:40:40.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.095+0000 7fed76800700 1 -- 172.21.15.131:0/3420770028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed7010fec0 msgr2=0x7fed701123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.095+0000 7fed76800700 1 --2- 172.21.15.131:0/3420770028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed7010fec0 0x7fed701123d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fed6800b3e0 tx=0x7fed6800b6f0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.099+0000 7fed76800700 1 -- 172.21.15.131:0/3420770028 shutdown_connections 2024-05-10T05:40:40.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.100+0000 7fed76800700 1 --2- 172.21.15.131:0/3420770028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed7010fec0 0x7fed701123d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.101+0000 7fed76800700 1 --2- 172.21.15.131:0/3420770028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed70109730 0x7fed70108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.101+0000 7fed76800700 1 --2- 172.21.15.131:0/3420770028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed701087e0 0x7fed70108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.101+0000 7fed76800700 1 -- 172.21.15.131:0/3420770028 >> 172.21.15.131:0/3420770028 conn(0x7fed7006e830 msgr2=0x7fed70070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.102+0000 7fed76800700 1 -- 172.21.15.131:0/3420770028 shutdown_connections 2024-05-10T05:40:40.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.103+0000 7fed76800700 1 -- 172.21.15.131:0/3420770028 wait complete. 2024-05-10T05:40:40.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.107+0000 7fed76800700 1 Processor -- start 2024-05-10T05:40:40.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.107+0000 7fed76800700 1 -- start start 2024-05-10T05:40:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.108+0000 7fed76800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed701087e0 0x7fed701a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.108+0000 7fed76800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed70109730 0x7fed701a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.109+0000 7fed76800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed7010fec0 0x7fed701a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.109+0000 7fed6ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed70109730 0x7fed701a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.109+0000 7fed6ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed70109730 0x7fed701a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45190/0 (socket says 172.21.15.131:45190) 2024-05-10T05:40:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.109+0000 7fed6ea00700 1 -- 172.21.15.131:0/1778058489 learned_addr learned my addr 172.21.15.131:0/1778058489 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.109+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed701a8070 con 0x7fed701087e0 2024-05-10T05:40:40.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.109+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed701a81b0 con 0x7fed70109730 2024-05-10T05:40:40.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.109+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed701a8340 con 0x7fed7010fec0 2024-05-10T05:40:40.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.111+0000 7fed6fe00700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed7010fec0 0x7fed701a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.111+0000 7fed6ea00700 1 -- 172.21.15.131:0/1778058489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed7010fec0 msgr2=0x7fed701a7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.111+0000 7fed6ea00700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed7010fec0 0x7fed701a7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.111+0000 7fed6ea00700 1 -- 172.21.15.131:0/1778058489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed701087e0 msgr2=0x7fed701a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.111+0000 7fed6ea00700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed701087e0 0x7fed701a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.112+0000 7fed6ea00700 1 -- 172.21.15.131:0/1778058489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed68009c40 con 0x7fed70109730 2024-05-10T05:40:40.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.112+0000 7fed6fe00700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed7010fec0 0x7fed701a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:40.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.113+0000 7fed6ea00700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed70109730 0x7fed701a0b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fed60009f00 tx=0x7fed6000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:40.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.114+0000 7fed67e00700 1 -- 172.21.15.131:0/1778058489 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6000cbd0 con 0x7fed70109730 2024-05-10T05:40:40.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.114+0000 7fed67e00700 1 -- 172.21.15.131:0/1778058489 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6000ee80 con 0x7fed70109730 2024-05-10T05:40:40.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.114+0000 7fed67e00700 1 -- 172.21.15.131:0/1778058489 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60005610 con 0x7fed70109730 2024-05-10T05:40:40.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.116+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed70073440 con 0x7fed70109730 2024-05-10T05:40:40.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.116+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed700738a0 con 0x7fed70109730 2024-05-10T05:40:40.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.127+0000 7fed67e00700 1 -- 172.21.15.131:0/1778058489 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fed60004750 con 0x7fed70109730 2024-05-10T05:40:40.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.129+0000 7fed67e00700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fed540a1220 0x7fed540a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.130+0000 7fed8e70bb80 1 Processor -- start 2024-05-10T05:40:40.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.130+0000 7fed6f400700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fed540a1220 0x7fed540a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.131+0000 7fed67e00700 1 -- 172.21.15.131:0/1778058489 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fed60036070 con 0x7fed70109730 2024-05-10T05:40:40.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.131+0000 7fed6f400700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fed540a1220 0x7fed540a3730 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed70073410 tx=0x7fed5c0096c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:40.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.133+0000 7fed8e70bb80 1 -- start start 2024-05-10T05:40:40.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.134+0000 7fed8e70bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e8c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.134+0000 7fed8e70bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9e96370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.134+0000 7fed8e70bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9e9d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.134+0000 7fed6f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e8c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.134+0000 7fed6f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e8c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45200/0 (socket says 172.21.15.131:45200) 2024-05-10T05:40:40.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.134+0000 7fed6ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9e96370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.134+0000 7fed6ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9e96370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59418/0 (socket says 172.21.15.131:59418) 2024-05-10T05:40:40.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.135+0000 7fed6f400700 1 -- 172.21.15.131:0/3752416772 learned_addr learned my addr 172.21.15.131:0/3752416772 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:40.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.135+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3752416772 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9db3150 con 0x5615f9e968b0 2024-05-10T05:40:40.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.135+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3752416772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9db0b10 con 0x5615f9e8c580 2024-05-10T05:40:40.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.135+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3752416772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9dac400 con 0x5615f9e95f60 2024-05-10T05:40:40.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.136+0000 7fed6fe00700 1 --2- 172.21.15.131:0/3752416772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9e9d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.136+0000 7fed6f400700 1 -- 172.21.15.131:0/3752416772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 msgr2=0x5615f9e96370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.136+0000 7fed6f400700 1 --2- 172.21.15.131:0/3752416772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9e96370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.136+0000 7fed6f400700 1 -- 172.21.15.131:0/3752416772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 msgr2=0x5615f9e9d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.136+0000 7fed6f400700 1 --2- 172.21.15.131:0/3752416772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9e9d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.136+0000 7fed6f400700 1 -- 172.21.15.131:0/3752416772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615f9c2d3b0 con 0x5615f9e8c580 2024-05-10T05:40:40.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.136+0000 7fed6fe00700 1 --2- 172.21.15.131:0/3752416772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9e9d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:40:40.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.136+0000 7fed6ea00700 1 --2- 172.21.15.131:0/3752416772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9e96370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:40.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.137+0000 7fed6f400700 1 --2- 172.21.15.131:0/3752416772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e8c9b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00ebd0 tx=0x7fed5c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d6a77109bb594bf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:40.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.138+0000 7fed64c00700 1 -- 172.21.15.131:0/3752416772 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00eee0 con 0x5615f9e8c580 2024-05-10T05:40:40.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.138+0000 7fed64c00700 1 -- 172.21.15.131:0/3752416772 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c025460 con 0x5615f9e8c580 2024-05-10T05:40:40.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.138+0000 7fed64c00700 1 -- 172.21.15.131:0/3752416772 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c01c620 con 0x5615f9e8c580 2024-05-10T05:40:40.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.139+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3752416772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 msgr2=0x5615f9e8c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.139+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/3752416772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e8c9b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00ebd0 tx=0x7fed5c011000 comp rx=0 tx=0).stop 2024-05-10T05:40:40.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.140+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3752416772 shutdown_connections 2024-05-10T05:40:40.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.140+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/3752416772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9e9d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.140+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/3752416772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e8c9b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.140+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/3752416772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9e96370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.140+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3752416772 >> 172.21.15.131:0/3752416772 conn(0x5615f9deea60 msgr2=0x5615f9e87330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:40.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.140+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3752416772 shutdown_connections 2024-05-10T05:40:40.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.141+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3752416772 wait complete. 2024-05-10T05:40:40.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.142+0000 7fed8e70bb80 1 -- start start 2024-05-10T05:40:40.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.142+0000 7fed8e70bb80 1 Processor -- start 2024-05-10T05:40:40.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.143+0000 7fed8e70bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e8c580 0x5615f9e9f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.143+0000 7fed8e70bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e95f60 0x5615f9e9f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.143+0000 7fed8e70bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e968b0 0x5615f9ea65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.144+0000 7fed6ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e95f60 0x5615f9e9f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.144+0000 7fed6ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e95f60 0x5615f9e9f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45206/0 (socket says 172.21.15.131:45206) 2024-05-10T05:40:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.145+0000 7fed6fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e968b0 0x5615f9ea65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.145+0000 7fed6fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e968b0 0x5615f9ea65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59434/0 (socket says 172.21.15.131:59434) 2024-05-10T05:40:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.145+0000 7fed6fe00700 1 -- 172.21.15.131:0/3662421677 learned_addr learned my addr 172.21.15.131:0/3662421677 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.145+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9d48560 con 0x5615f9e8c580 2024-05-10T05:40:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.145+0000 7fed6f400700 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e8c580 0x5615f9e9f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.145+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9d48200 con 0x5615f9e95f60 2024-05-10T05:40:40.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.145+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9c2f6a0 con 0x5615f9e968b0 2024-05-10T05:40:40.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.146+0000 7fed6fe00700 1 -- 172.21.15.131:0/3662421677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e95f60 msgr2=0x5615f9e9f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.146+0000 7fed6fe00700 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e95f60 0x5615f9e9f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.146+0000 7fed8e70bb80 10 client.0 _ll_register_callbacks cb 0x5615f9d51b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:40.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.146+0000 7fed6fe00700 1 -- 172.21.15.131:0/3662421677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e8c580 msgr2=0x5615f9e9f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.146+0000 7fed6fe00700 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e8c580 0x5615f9e9f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.146+0000 7fed6fe00700 1 -- 172.21.15.131:0/3662421677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed700738a0 con 0x5615f9e968b0 2024-05-10T05:40:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.147+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615f9d05270 con 0x5615f9e968b0 2024-05-10T05:40:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.148+0000 7fed59600700 10 client.26710 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:40:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.148+0000 7fed6fe00700 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e968b0 0x5615f9ea65f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed68007760 tx=0x7fed68007790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ace62371bf5cc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:40.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.150+0000 7fed59600700 1 -- 172.21.15.131:0/3662421677 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800e070 con 0x5615f9e968b0 2024-05-10T05:40:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.151+0000 7fed59600700 1 -- 172.21.15.131:0/3662421677 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fed6800be40 con 0x5615f9e968b0 2024-05-10T05:40:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.152+0000 7fed59600700 1 -- 172.21.15.131:0/3662421677 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed68003c00 con 0x5615f9e968b0 2024-05-10T05:40:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.152+0000 7fed8e70bb80 10 client.26710 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.152+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x7fed5c00e8b0 con 0x5615f9e968b0 2024-05-10T05:40:40.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.157+0000 7fed59600700 1 -- 172.21.15.131:0/3662421677 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fed6800b410 con 0x5615f9e968b0 2024-05-10T05:40:40.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.157+0000 7fed8e70bb80 10 client.26710 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:40.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.157+0000 7fed8e70bb80 10 client.26710 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:40:40.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.157+0000 7fed8e70bb80 20 client.26710 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:40.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.157+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5615f9eaaac0 0x5615f9eacf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.158+0000 7fed8e70bb80 4 client.26710 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:40.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.158+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615f9e5b780 con 0x5615f9eaaac0 2024-05-10T05:40:40.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.163+0000 7fed6f400700 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5615f9eaaac0 0x5615f9eacf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.164+0000 7fed6f400700 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5615f9eaaac0 0x5615f9eacf10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe1820d74bcb9b91 server_cookie=a188daf5399d5cb4 in_seq=0 out_seq=0 2024-05-10T05:40:40.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.165+0000 7fed59600700 10 client.26710 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:40:40.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.166+0000 7fed59600700 1 -- 172.21.15.131:0/3662421677 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615f9e5b780 con 0x5615f9eaaac0 2024-05-10T05:40:40.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.166+0000 7fed59600700 10 client.26710 handle_command_reply: tid=0 2024-05-10T05:40:40.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.170+0000 7fed8e70bb80 1 client.26710 shutdown 2024-05-10T05:40:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.171+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e968b0 msgr2=0x5615f9ea65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.171+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e968b0 0x5615f9ea65f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed68007760 tx=0x7fed68007790 comp rx=0 tx=0).stop 2024-05-10T05:40:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.172+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 shutdown_connections 2024-05-10T05:40:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.172+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5615f9eaaac0 0x5615f9eacf10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.172+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e8c580 0x5615f9e9f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.172+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e968b0 0x5615f9ea65f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.172+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/3662421677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e95f60 0x5615f9e9f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.172+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 >> 172.21.15.131:0/3662421677 conn(0x5615f9deea60 msgr2=0x5615f9e8b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.172+0000 7fed59600700 0 client.26710 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:40:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.173+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 shutdown_connections 2024-05-10T05:40:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.174+0000 7fed8e70bb80 1 -- 172.21.15.131:0/3662421677 wait complete. 2024-05-10T05:40:40.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.190+0000 7fed8e70bb80 20 client.26710 trim_cache size 0 max 16384 2024-05-10T05:40:40.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.261+0000 7fed8e70bb80 1 Processor -- start 2024-05-10T05:40:40.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.262+0000 7fed8e70bb80 1 -- start start 2024-05-10T05:40:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.263+0000 7fed8e70bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e9f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.264+0000 7fed8e70bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9ea65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.264+0000 7fed8e70bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9eacad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.264+0000 7fed6ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9ea65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.264+0000 7fed6f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e9f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.264+0000 7fed6ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9ea65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59448/0 (socket says 172.21.15.131:59448) 2024-05-10T05:40:40.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.265+0000 7fed6f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e9f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45214/0 (socket says 172.21.15.131:45214) 2024-05-10T05:40:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.265+0000 7fed6fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9eacad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed6fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9eacad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50778/0 (socket says 172.21.15.131:50778) 2024-05-10T05:40:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed8e70bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9e8ce10 con 0x5615f9e968b0 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed6ea00700 1 -- 172.21.15.131:0/1615870362 learned_addr learned my addr 172.21.15.131:0/1615870362 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed8e70bb80 1 -- 172.21.15.131:0/1615870362 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9e7ed60 con 0x5615f9e8c580 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed8e70bb80 1 -- 172.21.15.131:0/1615870362 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9e9d120 con 0x5615f9e95f60 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed6ea00700 1 -- 172.21.15.131:0/1615870362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 msgr2=0x5615f9e9f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed6ea00700 1 --2- 172.21.15.131:0/1615870362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e9f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed6ea00700 1 -- 172.21.15.131:0/1615870362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 msgr2=0x5615f9eacad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed6ea00700 1 --2- 172.21.15.131:0/1615870362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9eacad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed6ea00700 1 -- 172.21.15.131:0/1615870362 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615f9bd9b60 con 0x5615f9e95f60 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.266+0000 7fed6fe00700 1 --2- 172.21.15.131:0/1615870362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9eacad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.267+0000 7fed6ea00700 1 --2- 172.21.15.131:0/1615870362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9ea65b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed6000cfd0 tx=0x7fed600ee040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38db0fb8885eb273 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.267+0000 7fed6f400700 1 --2- 172.21.15.131:0/1615870362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e9f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.267+0000 7fed65600700 1 -- 172.21.15.131:0/1615870362 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed600bc680 con 0x5615f9e95f60 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.267+0000 7fed65600700 1 -- 172.21.15.131:0/1615870362 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed600bbf80 con 0x5615f9e95f60 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.267+0000 7fed65600700 1 -- 172.21.15.131:0/1615870362 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed600bc440 con 0x5615f9e95f60 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.268+0000 7fed8e70bb80 1 -- 172.21.15.131:0/1615870362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 msgr2=0x5615f9ea65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.268+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/1615870362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9ea65b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed6000cfd0 tx=0x7fed600ee040 comp rx=0 tx=0).stop 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.269+0000 7fed8e70bb80 1 -- 172.21.15.131:0/1615870362 shutdown_connections 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.269+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/1615870362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9eacad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.269+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/1615870362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9e9f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.269+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/1615870362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9ea65b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.269+0000 7fed8e70bb80 1 -- 172.21.15.131:0/1615870362 >> 172.21.15.131:0/1615870362 conn(0x5615f9deea60 msgr2=0x5615f9e99d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:40.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.270+0000 7fed8e70bb80 1 -- 172.21.15.131:0/1615870362 shutdown_connections 2024-05-10T05:40:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.270+0000 7fed8e70bb80 1 -- 172.21.15.131:0/1615870362 wait complete. 2024-05-10T05:40:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.271+0000 7fed8e70bb80 1 -- start start 2024-05-10T05:40:40.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.271+0000 7fed8e70bb80 1 Processor -- start 2024-05-10T05:40:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.278+0000 7fed8e70bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9ea45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.280+0000 7fed6f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9ea45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.280+0000 7fed6f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9ea45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45226/0 (socket says 172.21.15.131:45226) 2024-05-10T05:40:40.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.281+0000 7fed6f400700 1 -- 172.21.15.131:0/2036591372 learned_addr learned my addr 172.21.15.131:0/2036591372 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.281+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9ea4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.283+0000 7fed6ea00700 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9ea4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.284+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9ea5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.285+0000 7fed6fe00700 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9ea5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.285+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9ea5f70 con 0x5615f9e968b0 2024-05-10T05:40:40.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.285+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9ea60e0 con 0x5615f9e8c580 2024-05-10T05:40:40.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.285+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615f9ea62a0 con 0x5615f9e95f60 2024-05-10T05:40:40.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.286+0000 7fed8e70bb80 10 client.0 _ll_register_callbacks cb 0x5615f9d51b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:40.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.287+0000 7fed6f400700 1 -- 172.21.15.131:0/2036591372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 msgr2=0x5615f9ea4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.287+0000 7fed6f400700 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9ea4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.287+0000 7fed6f400700 1 -- 172.21.15.131:0/2036591372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 msgr2=0x5615f9ea5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.287+0000 7fed6f400700 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9ea5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.287+0000 7fed6f400700 1 -- 172.21.15.131:0/2036591372 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5c00e8b0 con 0x5615f9e8c580 2024-05-10T05:40:40.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.287+0000 7fed6fe00700 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9ea5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:40.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.288+0000 7fed6f400700 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9ea45c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00e760 tx=0x7fed5c005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77a6c144eb1befec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:40.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.288+0000 7fed59600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:40.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.290+0000 7fed59600700 1 -- 172.21.15.131:0/2036591372 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c025a60 con 0x5615f9e8c580 2024-05-10T05:40:40.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.290+0000 7fed59600700 1 -- 172.21.15.131:0/2036591372 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fed5c0108c0 con 0x5615f9e8c580 2024-05-10T05:40:40.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.292+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed5c025460 con 0x5615f9e8c580 2024-05-10T05:40:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.297+0000 7fed59600700 1 -- 172.21.15.131:0/2036591372 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed5c010a80 con 0x5615f9e8c580 2024-05-10T05:40:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.297+0000 7fed8e70bb80 10 client.26669 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:40.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.297+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fed70073440 con 0x5615f9e8c580 2024-05-10T05:40:40.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.300+0000 7fed59600700 1 -- 172.21.15.131:0/2036591372 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c025460 con 0x5615f9e8c580 2024-05-10T05:40:40.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.300+0000 7fed59600700 1 -- 172.21.15.131:0/2036591372 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fed5c018070 con 0x5615f9e8c580 2024-05-10T05:40:40.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.301+0000 7fed8e70bb80 10 client.26669 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:40.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.301+0000 7fed8e70bb80 10 client.26669 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:40:40.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.301+0000 7fed8e70bb80 20 client.26669 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:40.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.301+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5615f9f01f70 0x5615f9f04390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.301+0000 7fed8e70bb80 4 client.26669 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:40:40.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.301+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615f9da4ec0 con 0x5615f9f01f70 2024-05-10T05:40:40.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.306+0000 7fed6ea00700 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5615f9f01f70 0x5615f9f04390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.309+0000 7fed6ea00700 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5615f9f01f70 0x5615f9f04390 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab5436ca1e63f152 server_cookie=22e591758fcc989 in_seq=0 out_seq=0 2024-05-10T05:40:40.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.309+0000 7fed59600700 10 client.26669 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:40:40.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.310+0000 7fed59600700 1 -- 172.21.15.131:0/2036591372 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615f9da4ec0 con 0x5615f9f01f70 2024-05-10T05:40:40.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.310+0000 7fed59600700 10 client.26669 handle_command_reply: tid=0 2024-05-10T05:40:40.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.310+0000 7fed8e70bb80 1 client.26669 shutdown 2024-05-10T05:40:40.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.310+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 msgr2=0x5615f9ea45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.310+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9ea45c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00e760 tx=0x7fed5c005df0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.313+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 shutdown_connections 2024-05-10T05:40:40.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.313+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x5615f9f01f70 0x5615f9f04390 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.313+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615f9e968b0 0x5615f9ea5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.314+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615f9e8c580 0x5615f9ea45c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.314+0000 7fed8e70bb80 1 --2- 172.21.15.131:0/2036591372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615f9e95f60 0x5615f9ea4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.314+0000 7fed59600700 0 client.26669 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:40:40.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.314+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 >> 172.21.15.131:0/2036591372 conn(0x5615f9deea60 msgr2=0x5615f9e99d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:40.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.317+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 shutdown_connections 2024-05-10T05:40:40.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.320+0000 7fed8e70bb80 1 -- 172.21.15.131:0/2036591372 wait complete. 2024-05-10T05:40:40.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.321+0000 7fed8e70bb80 20 client.26669 trim_cache size 0 max 16384 2024-05-10T05:40:40.321 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:40:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.322+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fed540a1220 msgr2=0x7fed540a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.322+0000 7fed76800700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fed540a1220 0x7fed540a3730 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed70073410 tx=0x7fed5c0096c0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.322+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed70109730 msgr2=0x7fed701a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.322+0000 7fed76800700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed70109730 0x7fed701a0b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fed60009f00 tx=0x7fed6000eca0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.324+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 shutdown_connections 2024-05-10T05:40:40.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.324+0000 7fed76800700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fed540a1220 0x7fed540a3730 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.324+0000 7fed76800700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed701087e0 0x7fed701a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.324+0000 7fed76800700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed7010fec0 0x7fed701a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.324+0000 7fed76800700 1 --2- 172.21.15.131:0/1778058489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed70109730 0x7fed701a0b10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.325+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 >> 172.21.15.131:0/1778058489 conn(0x7fed7006e830 msgr2=0x7fed7010bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:40.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.325+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 shutdown_connections 2024-05-10T05:40:40.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.325+0000 7fed76800700 1 -- 172.21.15.131:0/1778058489 wait complete. 2024-05-10T05:40:40.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:40:40.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:40 smithi157 ceph-mon[78427]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 242 MiB/s wr, 73 op/s 2024-05-10T05:40:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:40 smithi142 ceph-mon[77667]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 242 MiB/s wr, 73 op/s 2024-05-10T05:40:40.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:40 smithi131 ceph-mon[37264]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 242 MiB/s wr, 73 op/s 2024-05-10T05:40:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.945+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4213401920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c10e220 msgr2=0x7ff69c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.945+0000 7ff6a1e00700 1 --2- 172.21.15.131:0/4213401920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c10e220 0x7ff69c110640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff69400b0a0 tx=0x7ff69400b3b0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.945+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4213401920 shutdown_connections 2024-05-10T05:40:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.945+0000 7ff6a1e00700 1 --2- 172.21.15.131:0/4213401920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c075140 0x7ff69c073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.945+0000 7ff6a1e00700 1 --2- 172.21.15.131:0/4213401920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c10e220 0x7ff69c110640 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.945+0000 7ff6a1e00700 1 --2- 172.21.15.131:0/4213401920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c073b40 0x7ff69c074010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.945+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4213401920 >> 172.21.15.131:0/4213401920 conn(0x7ff69c0fd1d0 msgr2=0x7ff69c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.945+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4213401920 shutdown_connections 2024-05-10T05:40:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.945+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4213401920 wait complete. 2024-05-10T05:40:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.947+0000 7ff6a1e00700 1 Processor -- start 2024-05-10T05:40:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.947+0000 7ff6a1e00700 1 -- start start 2024-05-10T05:40:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.947+0000 7ff6a1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c073b40 0x7ff69c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.947+0000 7ff6a1e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c075140 0x7ff69c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.947+0000 7ff6a1e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c10e220 0x7ff69c1a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.947+0000 7ff6a1e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c1a5ff0 con 0x7ff69c10e220 2024-05-10T05:40:40.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.947+0000 7ff6a1e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c1a6160 con 0x7ff69c073b40 2024-05-10T05:40:40.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.947+0000 7ff6a1e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c1a6320 con 0x7ff69c075140 2024-05-10T05:40:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.948+0000 7ff69aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c073b40 0x7ff69c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.948+0000 7ff69aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c073b40 0x7ff69c19e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45236/0 (socket says 172.21.15.131:45236) 2024-05-10T05:40:40.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.948+0000 7ff69aa00700 1 -- 172.21.15.131:0/4219143482 learned_addr learned my addr 172.21.15.131:0/4219143482 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.948+0000 7ff69a000700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c075140 0x7ff69c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.948+0000 7ff69aa00700 1 -- 172.21.15.131:0/4219143482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c075140 msgr2=0x7ff69c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.948+0000 7ff69aa00700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c075140 0x7ff69c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.948+0000 7ff69aa00700 1 -- 172.21.15.131:0/4219143482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c10e220 msgr2=0x7ff69c1a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.949+0000 7ff69aa00700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c10e220 0x7ff69c1a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.949+0000 7ff69aa00700 1 -- 172.21.15.131:0/4219143482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff694009d00 con 0x7ff69c073b40 2024-05-10T05:40:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.949+0000 7ff69aa00700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c073b40 0x7ff69c19e510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff68800e970 tx=0x7ff68800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.949+0000 7ff6a0a00700 1 -- 172.21.15.131:0/4219143482 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688009790 con 0x7ff69c073b40 2024-05-10T05:40:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.949+0000 7ff6a0a00700 1 -- 172.21.15.131:0/4219143482 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff688004cf0 con 0x7ff69c073b40 2024-05-10T05:40:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.949+0000 7ff6a0a00700 1 -- 172.21.15.131:0/4219143482 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688005690 con 0x7ff69c073b40 2024-05-10T05:40:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.949+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4219143482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c1a6600 con 0x7ff69c073b40 2024-05-10T05:40:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.950+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4219143482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c1a6ac0 con 0x7ff69c073b40 2024-05-10T05:40:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.954+0000 7ff6a0a00700 1 -- 172.21.15.131:0/4219143482 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff68800ccd0 con 0x7ff69c073b40 2024-05-10T05:40:40.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.956+0000 7ff6b9da1b80 1 Processor -- start 2024-05-10T05:40:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.956+0000 7ff6b9da1b80 1 -- start start 2024-05-10T05:40:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 0x55e10108eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108f570 0x55e10108f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 0x55e1010967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff6b9da1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e100fb5150 con 0x55e10108e6a0 2024-05-10T05:40:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff6b9da1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e100fb2b10 con 0x55e10108f570 2024-05-10T05:40:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff6b9da1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e100fae400 con 0x55e10108fec0 2024-05-10T05:40:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff69a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 0x55e1010967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff6a0a00700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6800a1240 0x7ff6800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff69a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 0x55e1010967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59490/0 (socket says 172.21.15.131:59490) 2024-05-10T05:40:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff69a000700 1 -- 172.21.15.131:0/1596012906 learned_addr learned my addr 172.21.15.131:0/1596012906 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff69aa00700 1 --2- 172.21.15.131:0/1596012906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108f570 0x55e10108f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.957+0000 7ff69b400700 1 --2- 172.21.15.131:0/1596012906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 0x55e10108eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.958+0000 7ff6a0a00700 1 -- 172.21.15.131:0/4219143482 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff688014070 con 0x7ff69c073b40 2024-05-10T05:40:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.958+0000 7ff69a000700 1 -- 172.21.15.131:0/1596012906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108f570 msgr2=0x55e10108f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.958+0000 7ff69a000700 1 --2- 172.21.15.131:0/1596012906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108f570 0x55e10108f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.958+0000 7ff69a000700 1 -- 172.21.15.131:0/1596012906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 msgr2=0x55e10108eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.958+0000 7ff69a000700 1 --2- 172.21.15.131:0/1596012906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 0x55e10108eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.958+0000 7ff69a000700 1 -- 172.21.15.131:0/1596012906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e100e2f3b0 con 0x55e10108fec0 2024-05-10T05:40:40.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.958+0000 7ff69b400700 1 --2- 172.21.15.131:0/1596012906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 0x55e10108eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:40.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.958+0000 7ff69a000700 1 --2- 172.21.15.131:0/1596012906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 0x55e1010967c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00bd00 tx=0x7ff68c00f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b23224612c8880 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:40.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.959+0000 7ff69b400700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6800a1240 0x7ff6800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.959+0000 7ff687e00700 1 -- 172.21.15.131:0/1596012906 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c00fda0 con 0x55e10108fec0 2024-05-10T05:40:40.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.959+0000 7ff687e00700 1 -- 172.21.15.131:0/1596012906 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff68c00be50 con 0x55e10108fec0 2024-05-10T05:40:40.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.959+0000 7ff687e00700 1 -- 172.21.15.131:0/1596012906 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c010730 con 0x55e10108fec0 2024-05-10T05:40:40.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.960+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1596012906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 msgr2=0x55e1010967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.960+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1596012906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 0x55e1010967c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00bd00 tx=0x7ff68c00f640 comp rx=0 tx=0).stop 2024-05-10T05:40:40.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.960+0000 7ff69b400700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6800a1240 0x7ff6800a3750 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff69400b780 tx=0x7ff694009c00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:40.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.960+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1596012906 shutdown_connections 2024-05-10T05:40:40.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.960+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1596012906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 0x55e10108eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.960+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1596012906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 0x55e1010967c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.960+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1596012906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108f570 0x55e10108f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.960+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1596012906 >> 172.21.15.131:0/1596012906 conn(0x55e100ff0a60 msgr2=0x55e101088fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:40.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.960+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1596012906 shutdown_connections 2024-05-10T05:40:40.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.960+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1596012906 wait complete. 2024-05-10T05:40:40.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.961+0000 7ff6b9da1b80 1 -- start start 2024-05-10T05:40:40.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.961+0000 7ff6b9da1b80 1 Processor -- start 2024-05-10T05:40:40.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.962+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108e6a0 0x55e101098690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108f570 0x55e101098bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 0x55e10109fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff69aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108e6a0 0x55e101098690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff69aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108e6a0 0x55e101098690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45260/0 (socket says 172.21.15.131:45260) 2024-05-10T05:40:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff69aa00700 1 -- 172.21.15.131:0/1667143153 learned_addr learned my addr 172.21.15.131:0/1667143153 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e100f4a560 con 0x55e10108f570 2024-05-10T05:40:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e100f4a200 con 0x55e10108e6a0 2024-05-10T05:40:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e100e316a0 con 0x55e10108fec0 2024-05-10T05:40:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff69a000700 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108f570 0x55e101098bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff6b9da1b80 10 client.0 _ll_register_callbacks cb 0x55e100f53b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff69aa00700 1 -- 172.21.15.131:0/1667143153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 msgr2=0x55e10109fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff69aa00700 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 0x55e10109fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.963+0000 7ff69aa00700 1 -- 172.21.15.131:0/1667143153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108f570 msgr2=0x55e101098bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:40.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.964+0000 7ff69aa00700 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108f570 0x55e101098bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.964+0000 7ff69aa00700 1 -- 172.21.15.131:0/1667143153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c110fc0 con 0x55e10108e6a0 2024-05-10T05:40:40.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.964+0000 7ff69a000700 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108f570 0x55e101098bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:40:40.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.964+0000 7ff69aa00700 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108e6a0 0x55e101098690 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff6880ca7a0 tx=0x7ff688099b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31e55d16caf0643 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:40.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.964+0000 7ff684c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:40.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.964+0000 7ff684c00700 1 -- 172.21.15.131:0/1667143153 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68809df20 con 0x55e10108e6a0 2024-05-10T05:40:40.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.965+0000 7ff684c00700 1 -- 172.21.15.131:0/1667143153 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff6880a2570 con 0x55e10108e6a0 2024-05-10T05:40:40.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.965+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e100f07270 con 0x55e10108e6a0 2024-05-10T05:40:40.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.965+0000 7ff684c00700 1 -- 172.21.15.131:0/1667143153 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff688016e10 con 0x55e10108e6a0 2024-05-10T05:40:40.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.965+0000 7ff6b9da1b80 10 client.26681 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:40.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.965+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7ff68c00b960 con 0x55e10108e6a0 2024-05-10T05:40:40.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.965+0000 7ff684c00700 1 -- 172.21.15.131:0/1667143153 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e100f07270 con 0x55e10108e6a0 2024-05-10T05:40:40.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.967+0000 7ff684c00700 1 -- 172.21.15.131:0/1667143153 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff688099700 con 0x55e10108e6a0 2024-05-10T05:40:40.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.967+0000 7ff6b9da1b80 10 client.26681 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:40.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.967+0000 7ff6b9da1b80 10 client.26681 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:40:40.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.967+0000 7ff6b9da1b80 20 client.26681 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:40.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.967+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55e1010a4370 0x55e1010a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:40.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.967+0000 7ff6b9da1b80 4 client.26681 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:40.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.967+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e10105d780 con 0x55e1010a4370 2024-05-10T05:40:40.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.968+0000 7ff69a000700 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55e1010a4370 0x55e1010a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:40.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.969+0000 7ff69a000700 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55e1010a4370 0x55e1010a67e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3481a007825cd1d3 server_cookie=50219e975de0b716 in_seq=0 out_seq=0 2024-05-10T05:40:40.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.969+0000 7ff684c00700 10 client.26681 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:40:40.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.970+0000 7ff684c00700 1 -- 172.21.15.131:0/1667143153 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e10105d780 con 0x55e1010a4370 2024-05-10T05:40:40.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.970+0000 7ff684c00700 10 client.26681 handle_command_reply: tid=0 2024-05-10T05:40:40.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.970+0000 7ff6b9da1b80 1 client.26681 shutdown 2024-05-10T05:40:40.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.970+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108e6a0 msgr2=0x55e101098690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:40.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.970+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108e6a0 0x55e101098690 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff6880ca7a0 tx=0x7ff688099b80 comp rx=0 tx=0).stop 2024-05-10T05:40:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.971+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 shutdown_connections 2024-05-10T05:40:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.971+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55e1010a4370 0x55e1010a67e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.971+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108fec0 0x55e10109fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.971+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108e6a0 0x55e101098690 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.971+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1667143153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108f570 0x55e101098bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.971+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 >> 172.21.15.131:0/1667143153 conn(0x55e100ff0a60 msgr2=0x55e101088fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.971+0000 7ff684c00700 0 client.26681 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:40:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.971+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 shutdown_connections 2024-05-10T05:40:40.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.971+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1667143153 wait complete. 2024-05-10T05:40:40.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:40.974+0000 7ff6b9da1b80 20 client.26681 trim_cache size 0 max 16384 2024-05-10T05:40:41.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.003+0000 7ff6b9da1b80 1 Processor -- start 2024-05-10T05:40:41.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.003+0000 7ff6b9da1b80 1 -- start start 2024-05-10T05:40:41.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.004+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108e6a0 0x55e101097c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.004+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108f570 0x55e10109fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.004+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108fec0 0x55e101096d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.004+0000 7ff69aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108e6a0 0x55e101097c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.004+0000 7ff6b9da1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10108a1e0 con 0x55e10108fec0 2024-05-10T05:40:41.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.004+0000 7ff6b9da1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10108ef50 con 0x55e10108f570 2024-05-10T05:40:41.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.004+0000 7ff6b9da1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1010966d0 con 0x55e10108e6a0 2024-05-10T05:40:41.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.004+0000 7ff69aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108e6a0 0x55e101097c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59504/0 (socket says 172.21.15.131:59504) 2024-05-10T05:40:41.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.004+0000 7ff69aa00700 1 -- 172.21.15.131:0/1576632343 learned_addr learned my addr 172.21.15.131:0/1576632343 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:41.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.005+0000 7ff69a000700 1 --2- 172.21.15.131:0/1576632343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108f570 0x55e10109fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.005+0000 7ff69b400700 1 --2- 172.21.15.131:0/1576632343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108fec0 0x55e101096d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.005+0000 7ff69aa00700 1 -- 172.21.15.131:0/1576632343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108f570 msgr2=0x55e10109fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:41.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.005+0000 7ff69aa00700 1 --2- 172.21.15.131:0/1576632343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108f570 0x55e10109fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.005+0000 7ff69aa00700 1 -- 172.21.15.131:0/1576632343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108fec0 msgr2=0x55e101096d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:41.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.005+0000 7ff69aa00700 1 --2- 172.21.15.131:0/1576632343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108fec0 0x55e101096d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.005+0000 7ff69aa00700 1 -- 172.21.15.131:0/1576632343 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e100ddbb60 con 0x55e10108e6a0 2024-05-10T05:40:41.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.005+0000 7ff69b400700 1 --2- 172.21.15.131:0/1576632343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108fec0 0x55e101096d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:41.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.005+0000 7ff69aa00700 1 --2- 172.21.15.131:0/1576632343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108e6a0 0x55e101097c70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff688099df0 tx=0x7ff68809a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725f402ecd625826 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:41.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.006+0000 7ff691200700 1 -- 172.21.15.131:0/1576632343 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6880a21d0 con 0x55e10108e6a0 2024-05-10T05:40:41.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.006+0000 7ff691200700 1 -- 172.21.15.131:0/1576632343 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6880a2570 con 0x55e10108e6a0 2024-05-10T05:40:41.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.006+0000 7ff691200700 1 -- 172.21.15.131:0/1576632343 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688016ea0 con 0x55e10108e6a0 2024-05-10T05:40:41.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.007+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1576632343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108e6a0 msgr2=0x55e101097c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:41.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.007+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1576632343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108e6a0 0x55e101097c70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff688099df0 tx=0x7ff68809a270 comp rx=0 tx=0).stop 2024-05-10T05:40:41.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.007+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1576632343 shutdown_connections 2024-05-10T05:40:41.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.007+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1576632343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108fec0 0x55e101096d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.007+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1576632343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108e6a0 0x55e101097c70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.007+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/1576632343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108f570 0x55e10109fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.007+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1576632343 >> 172.21.15.131:0/1576632343 conn(0x55e100ff0a60 msgr2=0x55e101089ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:41.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.008+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1576632343 shutdown_connections 2024-05-10T05:40:41.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.008+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/1576632343 wait complete. 2024-05-10T05:40:41.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.008+0000 7ff6b9da1b80 1 -- start start 2024-05-10T05:40:41.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.009+0000 7ff6b9da1b80 1 Processor -- start 2024-05-10T05:40:41.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 0x55e10109cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108f570 0x55e10109d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff6b9da1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108fec0 0x55e1010f8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff6b9da1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1010f8dd0 con 0x55e10108e6a0 2024-05-10T05:40:41.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff6b9da1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1010f8f10 con 0x55e10108fec0 2024-05-10T05:40:41.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff6b9da1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1010f9080 con 0x55e10108f570 2024-05-10T05:40:41.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff6b9da1b80 10 client.0 _ll_register_callbacks cb 0x55e100f53b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:41.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff69aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 0x55e10109cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff69b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108fec0 0x55e1010f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff69b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108fec0 0x55e1010f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45272/0 (socket says 172.21.15.131:45272) 2024-05-10T05:40:41.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.010+0000 7ff69b400700 1 -- 172.21.15.131:0/2060665013 learned_addr learned my addr 172.21.15.131:0/2060665013 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:41.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.011+0000 7ff69b400700 1 -- 172.21.15.131:0/2060665013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108f570 msgr2=0x55e10109d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:41.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.011+0000 7ff69b400700 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108f570 0x55e10109d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.011+0000 7ff69b400700 1 -- 172.21.15.131:0/2060665013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 msgr2=0x55e10109cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:41.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.011+0000 7ff69b400700 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 0x55e10109cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.011+0000 7ff69b400700 1 -- 172.21.15.131:0/2060665013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c00b960 con 0x55e10108fec0 2024-05-10T05:40:41.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.011+0000 7ff69aa00700 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 0x55e10109cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:41.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.012+0000 7ff69b400700 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108fec0 0x55e1010f8890 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff69400b0a0 tx=0x7ff69400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ca7893003cd70d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:41.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.012+0000 7ff684c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:41.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.012+0000 7ff684c00700 1 -- 172.21.15.131:0/2060665013 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69400e040 con 0x55e10108fec0 2024-05-10T05:40:41.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.013+0000 7ff684c00700 1 -- 172.21.15.131:0/2060665013 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff694005450 con 0x55e10108fec0 2024-05-10T05:40:41.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.013+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/2060665013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff68c00be50 con 0x55e10108fec0 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.013+0000 7ff684c00700 1 -- 172.21.15.131:0/2060665013 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff694005610 con 0x55e10108fec0 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.013+0000 7ff6b9da1b80 10 client.26684 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.013+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/2060665013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e100f74560 con 0x55e10108fec0 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.013+0000 7ff684c00700 1 -- 172.21.15.131:0/2060665013 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff694035080 con 0x55e10108fec0 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.014+0000 7ff684c00700 1 -- 172.21.15.131:0/2060665013 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff694031cc0 con 0x55e10108fec0 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.014+0000 7ff6b9da1b80 10 client.26684 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.014+0000 7ff6b9da1b80 10 client.26684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.014+0000 7ff6b9da1b80 20 client.26684 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.014+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55e1010faa10 0x55e1010fce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.014+0000 7ff6b9da1b80 4 client.26684 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:40:41.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.014+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/2060665013 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e100fa6ec0 con 0x55e1010faa10 2024-05-10T05:40:41.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.015+0000 7ff69aa00700 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55e1010faa10 0x55e1010fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.016+0000 7ff69aa00700 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55e1010faa10 0x55e1010fce30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b4843a71d840b9f server_cookie=2f55455551217f11 in_seq=0 out_seq=0 2024-05-10T05:40:41.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.016+0000 7ff684c00700 10 client.26684 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:40:41.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.016+0000 7ff684c00700 1 -- 172.21.15.131:0/2060665013 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e100fa6ec0 con 0x55e1010faa10 2024-05-10T05:40:41.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.017+0000 7ff684c00700 10 client.26684 handle_command_reply: tid=0 2024-05-10T05:40:41.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.017+0000 7ff6b9da1b80 1 client.26684 shutdown 2024-05-10T05:40:41.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.017+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/2060665013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108fec0 msgr2=0x55e1010f8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:41.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.017+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108fec0 0x55e1010f8890 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff69400b0a0 tx=0x7ff69400b8b0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.017+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/2060665013 shutdown_connections 2024-05-10T05:40:41.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.017+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55e1010faa10 0x55e1010fce30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.017+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e10108e6a0 0x55e10109cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.017+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e10108fec0 0x55e1010f8890 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.017+0000 7ff6b9da1b80 1 --2- 172.21.15.131:0/2060665013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e10108f570 0x55e10109d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.017+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/2060665013 >> 172.21.15.131:0/2060665013 conn(0x55e100ff0a60 msgr2=0x55e101089ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:41.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.018+0000 7ff684c00700 0 client.26684 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:40:41.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.018+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/2060665013 shutdown_connections 2024-05-10T05:40:41.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.018+0000 7ff6b9da1b80 1 -- 172.21.15.131:0/2060665013 wait complete. 2024-05-10T05:40:41.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.019+0000 7ff6b9da1b80 20 client.26684 trim_cache size 0 max 16384 2024-05-10T05:40:41.020 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:40:41.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.020+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4219143482 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6800a1240 msgr2=0x7ff6800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:41.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.020+0000 7ff6a1e00700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6800a1240 0x7ff6800a3750 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff69400b780 tx=0x7ff694009c00 comp rx=0 tx=0).stop 2024-05-10T05:40:41.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.020+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4219143482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c073b40 msgr2=0x7ff69c19e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:41.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.020+0000 7ff6a1e00700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c073b40 0x7ff69c19e510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff68800e970 tx=0x7ff68800ed30 comp rx=0 tx=0).stop 2024-05-10T05:40:41.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.020+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4219143482 shutdown_connections 2024-05-10T05:40:41.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.020+0000 7ff6a1e00700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff6800a1240 0x7ff6800a3750 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.020+0000 7ff6a1e00700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69c10e220 0x7ff69c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.020+0000 7ff6a1e00700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c073b40 0x7ff69c19e510 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.020+0000 7ff6a1e00700 1 --2- 172.21.15.131:0/4219143482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c075140 0x7ff69c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.021+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4219143482 >> 172.21.15.131:0/4219143482 conn(0x7ff69c0fd1d0 msgr2=0x7ff69c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:41.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.021+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4219143482 shutdown_connections 2024-05-10T05:40:41.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.021+0000 7ff6a1e00700 1 -- 172.21.15.131:0/4219143482 wait complete. 2024-05-10T05:40:41.080 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:40:41.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.816+0000 7fb171600700 1 -- 172.21.15.131:0/2000925981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c109cd0 msgr2=0x7fb16c110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:41.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.816+0000 7fb171600700 1 --2- 172.21.15.131:0/2000925981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c109cd0 0x7fb16c110580 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b0d0 tx=0x7fb16400b3e0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.816+0000 7fb171600700 1 -- 172.21.15.131:0/2000925981 shutdown_connections 2024-05-10T05:40:41.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.817+0000 7fb171600700 1 --2- 172.21.15.131:0/2000925981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c109cd0 0x7fb16c110580 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.817+0000 7fb171600700 1 --2- 172.21.15.131:0/2000925981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c108210 0x7fb16c108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.817+0000 7fb171600700 1 --2- 172.21.15.131:0/2000925981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c109330 0x7fb16c109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.817+0000 7fb171600700 1 -- 172.21.15.131:0/2000925981 >> 172.21.15.131:0/2000925981 conn(0x7fb16c077240 msgr2=0x7fb16c106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:41.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.820+0000 7fb171600700 1 -- 172.21.15.131:0/2000925981 shutdown_connections 2024-05-10T05:40:41.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.820+0000 7fb171600700 1 -- 172.21.15.131:0/2000925981 wait complete. 2024-05-10T05:40:41.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.822+0000 7fb171600700 1 Processor -- start 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.828+0000 7fb171600700 1 -- start start 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.828+0000 7fb171600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c108210 0x7fb16c06ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.828+0000 7fb171600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c109330 0x7fb16c0704f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.828+0000 7fb171600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c109cd0 0x7fb16c071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.828+0000 7fb171600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c071a60 con 0x7fb16c109cd0 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb171600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c071bd0 con 0x7fb16c109330 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb171600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c071d90 con 0x7fb16c108210 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb16aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c108210 0x7fb16c06ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb16aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c108210 0x7fb16c06ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59512/0 (socket says 172.21.15.131:59512) 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb16aa00700 1 -- 172.21.15.131:0/2733060958 learned_addr learned my addr 172.21.15.131:0/2733060958 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb16aa00700 1 -- 172.21.15.131:0/2733060958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c109330 msgr2=0x7fb16c0704f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:40:41.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb16aa00700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c109330 0x7fb16c0704f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb16aa00700 1 -- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c109cd0 msgr2=0x7fb16c071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:41.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb16b400700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c109cd0 0x7fb16c071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb16aa00700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c109cd0 0x7fb16c071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.829+0000 7fb16aa00700 1 -- 172.21.15.131:0/2733060958 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb164009d00 con 0x7fb16c108210 2024-05-10T05:40:41.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.830+0000 7fb16aa00700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c108210 0x7fb16c06ffb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb15800e940 tx=0x7fb15800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:41.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.830+0000 7fb16b400700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c109cd0 0x7fb16c071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:41.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.830+0000 7fb163400700 1 -- 172.21.15.131:0/2733060958 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15800cbd0 con 0x7fb16c108210 2024-05-10T05:40:41.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.830+0000 7fb163400700 1 -- 172.21.15.131:0/2733060958 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1580044e0 con 0x7fb16c108210 2024-05-10T05:40:41.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.830+0000 7fb163400700 1 -- 172.21.15.131:0/2733060958 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158003e00 con 0x7fb16c108210 2024-05-10T05:40:41.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.830+0000 7fb171600700 1 -- 172.21.15.131:0/2733060958 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c072070 con 0x7fb16c108210 2024-05-10T05:40:41.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.830+0000 7fb171600700 1 -- 172.21.15.131:0/2733060958 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c072530 con 0x7fb16c108210 2024-05-10T05:40:41.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.842+0000 7fb163400700 1 -- 172.21.15.131:0/2733060958 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb158003f60 con 0x7fb16c108210 2024-05-10T05:40:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.845+0000 7fb163400700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb1500a1290 0x7fb1500a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.846+0000 7fb16a000700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb1500a1290 0x7fb1500a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.850+0000 7fb16a000700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb1500a1290 0x7fb1500a37a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb16c071050 tx=0x7fb15c005c10 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:41.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.851+0000 7fb18954fb80 1 Processor -- start 2024-05-10T05:40:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.852+0000 7fb18954fb80 1 -- start start 2024-05-10T05:40:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.852+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.854+0000 7fb163400700 1 -- 172.21.15.131:0/2733060958 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb158014070 con 0x7fb16c108210 2024-05-10T05:40:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.856+0000 7fb16aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.856+0000 7fb16aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1ecaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50868/0 (socket says 172.21.15.131:50868) 2024-05-10T05:40:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.856+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1ed980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.856+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1edec0 0x55c0dd1f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.857+0000 7fb16aa00700 1 -- 172.21.15.131:0/3847669868 learned_addr learned my addr 172.21.15.131:0/3847669868 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.857+0000 7fb18954fb80 1 -- 172.21.15.131:0/3847669868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd113150 con 0x55c0dd1ec6a0 2024-05-10T05:40:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.857+0000 7fb18954fb80 1 -- 172.21.15.131:0/3847669868 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd110b10 con 0x55c0dd1edec0 2024-05-10T05:40:41.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.857+0000 7fb18954fb80 1 -- 172.21.15.131:0/3847669868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd10c400 con 0x55c0dd1ed570 2024-05-10T05:40:41.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.857+0000 7fb16a000700 1 --2- 172.21.15.131:0/3847669868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1ed980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.857+0000 7fb16b400700 1 --2- 172.21.15.131:0/3847669868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1edec0 0x55c0dd1f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.857+0000 7fb16a000700 1 -- 172.21.15.131:0/3847669868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1edec0 msgr2=0x55c0dd1f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:41.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.857+0000 7fb16a000700 1 --2- 172.21.15.131:0/3847669868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1edec0 0x55c0dd1f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.857+0000 7fb16a000700 1 -- 172.21.15.131:0/3847669868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 msgr2=0x55c0dd1ecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:41.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.857+0000 7fb16a000700 1 --2- 172.21.15.131:0/3847669868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1ecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.858+0000 7fb16a000700 1 -- 172.21.15.131:0/3847669868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0dcf8d3b0 con 0x55c0dd1ed570 2024-05-10T05:40:41.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.858+0000 7fb16aa00700 1 --2- 172.21.15.131:0/3847669868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.859+0000 7fb16a000700 1 --2- 172.21.15.131:0/3847669868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1ed980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00c5b0 tx=0x7fb15c00f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c37a28ba042ff11 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.860+0000 7fb157400700 1 -- 172.21.15.131:0/3847669868 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c004a30 con 0x55c0dd1ed570 2024-05-10T05:40:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.860+0000 7fb157400700 1 -- 172.21.15.131:0/3847669868 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb15c0054e0 con 0x55c0dd1ed570 2024-05-10T05:40:41.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.860+0000 7fb157400700 1 -- 172.21.15.131:0/3847669868 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00fc20 con 0x55c0dd1ed570 2024-05-10T05:40:41.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.861+0000 7fb18954fb80 1 -- 172.21.15.131:0/3847669868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 msgr2=0x55c0dd1ed980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:41.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.861+0000 7fb18954fb80 1 --2- 172.21.15.131:0/3847669868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1ed980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00c5b0 tx=0x7fb15c00f000 comp rx=0 tx=0).stop 2024-05-10T05:40:41.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.862+0000 7fb18954fb80 1 -- 172.21.15.131:0/3847669868 shutdown_connections 2024-05-10T05:40:41.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.862+0000 7fb18954fb80 1 --2- 172.21.15.131:0/3847669868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.862+0000 7fb18954fb80 1 --2- 172.21.15.131:0/3847669868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1edec0 0x55c0dd1f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.862+0000 7fb18954fb80 1 --2- 172.21.15.131:0/3847669868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1ed980 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.862+0000 7fb18954fb80 1 -- 172.21.15.131:0/3847669868 >> 172.21.15.131:0/3847669868 conn(0x55c0dd14ea60 msgr2=0x55c0dd1e6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.863+0000 7fb18954fb80 1 -- 172.21.15.131:0/3847669868 shutdown_connections 2024-05-10T05:40:41.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.863+0000 7fb18954fb80 1 -- 172.21.15.131:0/3847669868 wait complete. 2024-05-10T05:40:41.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.867+0000 7fb18954fb80 1 -- start start 2024-05-10T05:40:41.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.868+0000 7fb18954fb80 1 Processor -- start 2024-05-10T05:40:41.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.870+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.871+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ed570 0x55c0dd1f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.871+0000 7fb16aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.871+0000 7fb16a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ed570 0x55c0dd1f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.872+0000 7fb16a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ed570 0x55c0dd1f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:45284/0 (socket says 172.21.15.131:45284) 2024-05-10T05:40:41.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.872+0000 7fb16aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50876/0 (socket says 172.21.15.131:50876) 2024-05-10T05:40:41.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.872+0000 7fb16a000700 1 -- 172.21.15.131:0/2192631727 learned_addr learned my addr 172.21.15.131:0/2192631727 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:41.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.872+0000 7fb18954fb80 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 0x55c0dd1fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.872+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd0a8560 con 0x55c0dd1ec6a0 2024-05-10T05:40:41.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.872+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd0a8200 con 0x55c0dd1ed570 2024-05-10T05:40:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.872+0000 7fb16b400700 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 0x55c0dd1fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.872+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dcf8f6a0 con 0x55c0dd1edec0 2024-05-10T05:40:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.873+0000 7fb16aa00700 1 -- 172.21.15.131:0/2192631727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 msgr2=0x55c0dd1fd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.873+0000 7fb16aa00700 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 0x55c0dd1fd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.873+0000 7fb16aa00700 1 -- 172.21.15.131:0/2192631727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ed570 msgr2=0x55c0dd1f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.873+0000 7fb16aa00700 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ed570 0x55c0dd1f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.873+0000 7fb16aa00700 1 -- 172.21.15.131:0/2192631727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c072530 con 0x55c0dd1ec6a0 2024-05-10T05:40:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.875+0000 7fb18954fb80 10 client.0 _ll_register_callbacks cb 0x55c0dd0b1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.875+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0dd065270 con 0x55c0dd1ec6a0 2024-05-10T05:40:41.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.875+0000 7fb16aa00700 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1f64d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb1580a1ec0 tx=0x7fb158003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d97c2b696bf043 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:41.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.875+0000 7fb14be00700 10 client.16461 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:40:41.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.879+0000 7fb14be00700 1 -- 172.21.15.131:0/2192631727 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15809dda0 con 0x55c0dd1ec6a0 2024-05-10T05:40:41.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.879+0000 7fb14be00700 1 -- 172.21.15.131:0/2192631727 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb1580ade40 con 0x55c0dd1ec6a0 2024-05-10T05:40:41.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.879+0000 7fb14be00700 1 -- 172.21.15.131:0/2192631727 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1580a23f0 con 0x55c0dd1ec6a0 2024-05-10T05:40:41.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.879+0000 7fb18954fb80 10 client.16461 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.879+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x7fb15c00a560 con 0x55c0dd1ec6a0 2024-05-10T05:40:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.880+0000 7fb14be00700 1 -- 172.21.15.131:0/2192631727 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb15809a1d0 con 0x55c0dd1ec6a0 2024-05-10T05:40:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.880+0000 7fb18954fb80 10 client.16461 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.880+0000 7fb18954fb80 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.880+0000 7fb18954fb80 20 client.16461 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.881+0000 7fb18954fb80 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c0dd201e40 0x55c0dd204350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.881+0000 7fb18954fb80 4 client.16461 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.881+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0dd1bb780 con 0x55c0dd201e40 2024-05-10T05:40:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.881+0000 7fb16a000700 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c0dd201e40 0x55c0dd204350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.883+0000 7fb16a000700 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c0dd201e40 0x55c0dd204350 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f5b51264aed910a server_cookie=b52024bbce199953 in_seq=0 out_seq=0 2024-05-10T05:40:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.883+0000 7fb14be00700 10 client.16461 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.884+0000 7fb14be00700 1 -- 172.21.15.131:0/2192631727 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0dd1bb780 con 0x55c0dd201e40 2024-05-10T05:40:41.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.884+0000 7fb14be00700 10 client.16461 handle_command_reply: tid=0 2024-05-10T05:40:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.884+0000 7fb18954fb80 1 client.16461 shutdown 2024-05-10T05:40:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.884+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 msgr2=0x55c0dd1f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.884+0000 7fb18954fb80 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1f64d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb1580a1ec0 tx=0x7fb158003700 comp rx=0 tx=0).stop 2024-05-10T05:40:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.885+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 shutdown_connections 2024-05-10T05:40:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.885+0000 7fb18954fb80 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c0dd201e40 0x55c0dd204350 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.886+0000 7fb18954fb80 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1f64d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.886+0000 7fb14be00700 0 client.16461 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.886+0000 7fb18954fb80 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 0x55c0dd1fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.886+0000 7fb18954fb80 1 --2- 172.21.15.131:0/2192631727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ed570 0x55c0dd1f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.886+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 >> 172.21.15.131:0/2192631727 conn(0x55c0dd14ea60 msgr2=0x55c0dd1e6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.887+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 shutdown_connections 2024-05-10T05:40:41.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.898+0000 7fb18954fb80 1 -- 172.21.15.131:0/2192631727 wait complete. 2024-05-10T05:40:41.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.898+0000 7fb18954fb80 20 client.16461 trim_cache size 0 max 16384 2024-05-10T05:40:41.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.981+0000 7fb18954fb80 1 Processor -- start 2024-05-10T05:40:41.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.981+0000 7fb18954fb80 1 -- start start 2024-05-10T05:40:41.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.981+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1ed980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.981+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.982+0000 7fb16a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.982+0000 7fb16a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50886/0 (socket says 172.21.15.131:50886) 2024-05-10T05:40:41.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.982+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 0x55c0dd1e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:41.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.982+0000 7fb18954fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd201bc0 con 0x55c0dd1ed570 2024-05-10T05:40:41.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.982+0000 7fb18954fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd1f07a0 con 0x55c0dd1ec6a0 2024-05-10T05:40:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.982+0000 7fb18954fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd1f0960 con 0x55c0dd1edec0 2024-05-10T05:40:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.982+0000 7fb16a000700 1 -- 172.21.15.131:0/1697648390 learned_addr learned my addr 172.21.15.131:0/1697648390 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.983+0000 7fb16b400700 1 --2- 172.21.15.131:0/1697648390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 0x55c0dd1e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.983+0000 7fb16b400700 1 -- 172.21.15.131:0/1697648390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ec6a0 msgr2=0x55c0dd1ed980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:40:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.983+0000 7fb16b400700 1 --2- 172.21.15.131:0/1697648390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1ed980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.983+0000 7fb16b400700 1 -- 172.21.15.131:0/1697648390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ed570 msgr2=0x55c0dd1fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.983+0000 7fb16b400700 1 --2- 172.21.15.131:0/1697648390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.983+0000 7fb16b400700 1 -- 172.21.15.131:0/1697648390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0dcf39b60 con 0x55c0dd1edec0 2024-05-10T05:40:41.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.985+0000 7fb16b400700 1 --2- 172.21.15.131:0/1697648390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 0x55c0dd1e7d80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb164009900 tx=0x7fb164009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=130d17d734fc7ffd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:41.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.989+0000 7fb157e00700 1 -- 172.21.15.131:0/1697648390 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400e040 con 0x55c0dd1edec0 2024-05-10T05:40:41.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.989+0000 7fb157e00700 1 -- 172.21.15.131:0/1697648390 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb164020070 con 0x55c0dd1edec0 2024-05-10T05:40:41.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:41.989+0000 7fb157e00700 1 -- 172.21.15.131:0/1697648390 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1640052a0 con 0x55c0dd1edec0 2024-05-10T05:40:42.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.000+0000 7fb18954fb80 1 -- 172.21.15.131:0/1697648390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 msgr2=0x55c0dd1e7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:42.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.000+0000 7fb18954fb80 1 --2- 172.21.15.131:0/1697648390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 0x55c0dd1e7d80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb164009900 tx=0x7fb164009390 comp rx=0 tx=0).stop 2024-05-10T05:40:42.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.001+0000 7fb18954fb80 1 -- 172.21.15.131:0/1697648390 shutdown_connections 2024-05-10T05:40:42.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.001+0000 7fb18954fb80 1 --2- 172.21.15.131:0/1697648390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1edec0 0x55c0dd1e7d80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.001+0000 7fb18954fb80 1 --2- 172.21.15.131:0/1697648390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1ed980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.002+0000 7fb18954fb80 1 --2- 172.21.15.131:0/1697648390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.002+0000 7fb18954fb80 1 -- 172.21.15.131:0/1697648390 >> 172.21.15.131:0/1697648390 conn(0x55c0dd14ea60 msgr2=0x55c0dd1e3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:42.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.009+0000 7fb18954fb80 1 -- 172.21.15.131:0/1697648390 shutdown_connections 2024-05-10T05:40:42.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.009+0000 7fb18954fb80 1 -- 172.21.15.131:0/1697648390 wait complete. 2024-05-10T05:40:42.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.011+0000 7fb18954fb80 1 -- start start 2024-05-10T05:40:42.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.012+0000 7fb18954fb80 1 Processor -- start 2024-05-10T05:40:42.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.016+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1fb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:42.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.018+0000 7fb16aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:42.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.018+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:42.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.018+0000 7fb18954fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1edec0 0x55c0dd1fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:42.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.018+0000 7fb16a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:42.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.018+0000 7fb16a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59550/0 (socket says 172.21.15.131:59550) 2024-05-10T05:40:42.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.020+0000 7fb16a000700 1 -- 172.21.15.131:0/405055751 learned_addr learned my addr 172.21.15.131:0/405055751 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:42.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.020+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd1fcd00 con 0x55c0dd1edec0 2024-05-10T05:40:42.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.020+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd1fce70 con 0x55c0dd1ec6a0 2024-05-10T05:40:42.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.020+0000 7fb16b400700 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1edec0 0x55c0dd1fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:42.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.020+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0dd1fd030 con 0x55c0dd1ed570 2024-05-10T05:40:42.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.020+0000 7fb18954fb80 10 client.0 _ll_register_callbacks cb 0x55c0dd0b1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:42.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.021+0000 7fb16a000700 1 -- 172.21.15.131:0/405055751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ec6a0 msgr2=0x55c0dd1fb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:42.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.021+0000 7fb16a000700 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1fb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.021+0000 7fb16a000700 1 -- 172.21.15.131:0/405055751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1edec0 msgr2=0x55c0dd1fc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:42.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.021+0000 7fb16a000700 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1edec0 0x55c0dd1fc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.021+0000 7fb16a000700 1 -- 172.21.15.131:0/405055751 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c111010 con 0x55c0dd1ed570 2024-05-10T05:40:42.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.023+0000 7fb16a000700 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fb890 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb15c011040 tx=0x7fb15c005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e04b231948b78e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:42.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.023+0000 7fb14be00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:40:42.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.024+0000 7fb14be00700 1 -- 172.21.15.131:0/405055751 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c04b040 con 0x55c0dd1ed570 2024-05-10T05:40:42.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.024+0000 7fb14be00700 1 -- 172.21.15.131:0/405055751 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb15c017040 con 0x55c0dd1ed570 2024-05-10T05:40:42.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.024+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb15c0054e0 con 0x55c0dd1ed570 2024-05-10T05:40:42.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.024+0000 7fb14be00700 1 -- 172.21.15.131:0/405055751 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb15c00fc80 con 0x55c0dd1ed570 2024-05-10T05:40:42.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.024+0000 7fb18954fb80 10 client.26746 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:42.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.024+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c0dd0d2560 con 0x55c0dd1ed570 2024-05-10T05:40:42.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.025+0000 7fb14be00700 1 -- 172.21.15.131:0/405055751 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb15c068080 con 0x55c0dd1ed570 2024-05-10T05:40:42.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.025+0000 7fb14be00700 1 -- 172.21.15.131:0/405055751 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb15c004b50 con 0x55c0dd1ed570 2024-05-10T05:40:42.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.025+0000 7fb18954fb80 10 client.26746 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:42.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.025+0000 7fb18954fb80 10 client.26746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:42.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.025+0000 7fb18954fb80 20 client.26746 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:42.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.025+0000 7fb18954fb80 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c0dd201e40 0x55c0dd25ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:42.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.025+0000 7fb18954fb80 4 client.26746 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:40:42.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.025+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c0dd104ec0 con 0x55c0dd201e40 2024-05-10T05:40:42.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.026+0000 7fb16aa00700 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c0dd201e40 0x55c0dd25ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:42.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.026+0000 7fb14be00700 10 client.26746 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:42.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.027+0000 7fb16aa00700 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c0dd201e40 0x55c0dd25ae40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578d394d132657a9 server_cookie=af0c2ad65372e640 in_seq=0 out_seq=0 2024-05-10T05:40:42.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.027+0000 7fb14be00700 1 -- 172.21.15.131:0/405055751 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c0dd104ec0 con 0x55c0dd201e40 2024-05-10T05:40:42.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.027+0000 7fb14be00700 10 client.26746 handle_command_reply: tid=0 2024-05-10T05:40:42.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.027+0000 7fb18954fb80 1 client.26746 shutdown 2024-05-10T05:40:42.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.028+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 msgr2=0x55c0dd1fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:42.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.028+0000 7fb18954fb80 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fb890 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb15c011040 tx=0x7fb15c005280 comp rx=0 tx=0).stop 2024-05-10T05:40:42.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.028+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 shutdown_connections 2024-05-10T05:40:42.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.028+0000 7fb18954fb80 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c0dd201e40 0x55c0dd25ae40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.028+0000 7fb18954fb80 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c0dd1edec0 0x55c0dd1fc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.028+0000 7fb14be00700 0 client.26746 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:42.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.028+0000 7fb18954fb80 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0dd1ec6a0 0x55c0dd1fb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.029+0000 7fb18954fb80 1 --2- 172.21.15.131:0/405055751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0dd1ed570 0x55c0dd1fb890 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.029+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 >> 172.21.15.131:0/405055751 conn(0x55c0dd14ea60 msgr2=0x55c0dd1e3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.029+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 shutdown_connections 2024-05-10T05:40:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.029+0000 7fb18954fb80 1 -- 172.21.15.131:0/405055751 wait complete. 2024-05-10T05:40:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.030+0000 7fb18954fb80 20 client.26746 trim_cache size 0 max 16384 2024-05-10T05:40:42.031 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:40:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.031+0000 7fb171600700 1 -- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb1500a1290 msgr2=0x7fb1500a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.031+0000 7fb171600700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb1500a1290 0x7fb1500a37a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb16c071050 tx=0x7fb15c005c10 comp rx=0 tx=0).stop 2024-05-10T05:40:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.031+0000 7fb171600700 1 -- 172.21.15.131:0/2733060958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c108210 msgr2=0x7fb16c06ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.031+0000 7fb171600700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c108210 0x7fb16c06ffb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb15800e940 tx=0x7fb15800ed00 comp rx=0 tx=0).stop 2024-05-10T05:40:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.032+0000 7fb171600700 1 -- 172.21.15.131:0/2733060958 shutdown_connections 2024-05-10T05:40:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.032+0000 7fb171600700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb1500a1290 0x7fb1500a37a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.032+0000 7fb171600700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c109cd0 0x7fb16c071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.032+0000 7fb171600700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c108210 0x7fb16c06ffb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.032+0000 7fb171600700 1 --2- 172.21.15.131:0/2733060958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c109330 0x7fb16c0704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.032+0000 7fb171600700 1 -- 172.21.15.131:0/2733060958 >> 172.21.15.131:0/2733060958 conn(0x7fb16c077240 msgr2=0x7fb16c10c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.033+0000 7fb171600700 1 -- 172.21.15.131:0/2733060958 shutdown_connections 2024-05-10T05:40:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:42.033+0000 7fb171600700 1 -- 172.21.15.131:0/2733060958 wait complete. 2024-05-10T05:40:42.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:42 smithi131 ceph-mon[37264]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 75 op/s 2024-05-10T05:40:42.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:42 smithi142 ceph-mon[77667]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 75 op/s 2024-05-10T05:40:42.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:42 smithi157 ceph-mon[78427]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 75 op/s 2024-05-10T05:40:43.146 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:40:43.146 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:40:43.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.905+0000 7fb860a00700 1 -- 172.21.15.131:0/1080281157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb85c109bb0 msgr2=0x7fb85c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:43.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.905+0000 7fb860a00700 1 --2- 172.21.15.131:0/1080281157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb85c109bb0 0x7fb85c10bfd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb848009990 tx=0x7fb848009ca0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.907+0000 7fb860a00700 1 -- 172.21.15.131:0/1080281157 shutdown_connections 2024-05-10T05:40:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.908+0000 7fb860a00700 1 --2- 172.21.15.131:0/1080281157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb85c109bb0 0x7fb85c10bfd0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.909+0000 7fb860a00700 1 --2- 172.21.15.131:0/1080281157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85c10efa0 0x7fb85c1114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.909+0000 7fb860a00700 1 --2- 172.21.15.131:0/1080281157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85c10c510 0x7fb85c10e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.909+0000 7fb860a00700 1 -- 172.21.15.131:0/1080281157 >> 172.21.15.131:0/1080281157 conn(0x7fb85c06ea00 msgr2=0x7fb85c070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:43.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.911+0000 7fb860a00700 1 -- 172.21.15.131:0/1080281157 shutdown_connections 2024-05-10T05:40:43.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.911+0000 7fb860a00700 1 -- 172.21.15.131:0/1080281157 wait complete. 2024-05-10T05:40:43.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.914+0000 7fb860a00700 1 Processor -- start 2024-05-10T05:40:43.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.916+0000 7fb860a00700 1 -- start start 2024-05-10T05:40:43.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.916+0000 7fb860a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85c109bb0 0x7fb85c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.916+0000 7fb860a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb85c10c510 0x7fb85c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.916+0000 7fb860a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85c10efa0 0x7fb85c1a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.916+0000 7fb860a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1a61f0 con 0x7fb85c10c510 2024-05-10T05:40:43.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.916+0000 7fb860a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1a6360 con 0x7fb85c109bb0 2024-05-10T05:40:43.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.916+0000 7fb860a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1a6520 con 0x7fb85c10efa0 2024-05-10T05:40:43.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.917+0000 7fb85a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85c10efa0 0x7fb85c1a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.917+0000 7fb85a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85c10efa0 0x7fb85c1a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:38086/0 (socket says 172.21.15.131:38086) 2024-05-10T05:40:43.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.917+0000 7fb85a800700 1 -- 172.21.15.131:0/959299044 learned_addr learned my addr 172.21.15.131:0/959299044 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:43.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.918+0000 7fb859e00700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85c109bb0 0x7fb85c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.918+0000 7fb859400700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb85c10c510 0x7fb85c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.919+0000 7fb85a800700 1 -- 172.21.15.131:0/959299044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85c109bb0 msgr2=0x7fb85c19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:43.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.919+0000 7fb85a800700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85c109bb0 0x7fb85c19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.919+0000 7fb85a800700 1 -- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb85c10c510 msgr2=0x7fb85c19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:43.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.919+0000 7fb85a800700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb85c10c510 0x7fb85c19ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.919+0000 7fb85a800700 1 -- 172.21.15.131:0/959299044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8480095f0 con 0x7fb85c10efa0 2024-05-10T05:40:43.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.919+0000 7fb859400700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb85c10c510 0x7fb85c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:43.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.920+0000 7fb859e00700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85c109bb0 0x7fb85c19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:43.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.923+0000 7fb85a800700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85c10efa0 0x7fb85c1a5be0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b4a0 tx=0x7fb84c00b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:43.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.923+0000 7fb852a00700 1 -- 172.21.15.131:0/959299044 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c011830 con 0x7fb85c10efa0 2024-05-10T05:40:43.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.923+0000 7fb860a00700 1 -- 172.21.15.131:0/959299044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c1a6800 con 0x7fb85c10efa0 2024-05-10T05:40:43.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.924+0000 7fb860a00700 1 -- 172.21.15.131:0/959299044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c1a6d90 con 0x7fb85c10efa0 2024-05-10T05:40:43.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.924+0000 7fb852a00700 1 -- 172.21.15.131:0/959299044 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb84c011e50 con 0x7fb85c10efa0 2024-05-10T05:40:43.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.924+0000 7fb852a00700 1 -- 172.21.15.131:0/959299044 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c010e40 con 0x7fb85c10efa0 2024-05-10T05:40:43.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.930+0000 7fb852a00700 1 -- 172.21.15.131:0/959299044 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb84c01a470 con 0x7fb85c10efa0 2024-05-10T05:40:43.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.933+0000 7fb87883fb80 1 Processor -- start 2024-05-10T05:40:43.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.933+0000 7fb852a00700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb8400a12c0 0x7fb8400a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.934+0000 7fb852a00700 1 -- 172.21.15.131:0/959299044 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb84c0cc020 con 0x7fb85c10efa0 2024-05-10T05:40:43.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.935+0000 7fb87883fb80 1 -- start start 2024-05-10T05:40:43.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.935+0000 7fb859400700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb8400a12c0 0x7fb8400a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.939+0000 7fb859400700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb8400a12c0 0x7fb8400a37d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb85c19f7b0 tx=0x7fb85400d040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.939+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef53b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.940+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef54a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.940+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 0x56399ef5b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.940+0000 7fb87883fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399ee7a1d0 con 0x56399ef53720 2024-05-10T05:40:43.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.940+0000 7fb87883fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399ee77b90 con 0x56399ef545f0 2024-05-10T05:40:43.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.940+0000 7fb87883fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399ee73480 con 0x56399ef54f40 2024-05-10T05:40:43.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.940+0000 7fb859400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef54a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.941+0000 7fb859400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef54a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:54838/0 (socket says 172.21.15.131:54838) 2024-05-10T05:40:43.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.941+0000 7fb859400700 1 -- 172.21.15.131:0/4215032779 learned_addr learned my addr 172.21.15.131:0/4215032779 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:43.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.941+0000 7fb859e00700 1 --2- 172.21.15.131:0/4215032779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef53b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.942+0000 7fb85a800700 1 --2- 172.21.15.131:0/4215032779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 0x56399ef5b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.943+0000 7fb859e00700 1 -- 172.21.15.131:0/4215032779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 msgr2=0x56399ef5b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:43.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.943+0000 7fb859e00700 1 --2- 172.21.15.131:0/4215032779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 0x56399ef5b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.943+0000 7fb859e00700 1 -- 172.21.15.131:0/4215032779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 msgr2=0x56399ef54a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:43.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.943+0000 7fb859e00700 1 --2- 172.21.15.131:0/4215032779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef54a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.943+0000 7fb859e00700 1 -- 172.21.15.131:0/4215032779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56399ecf4430 con 0x56399ef53720 2024-05-10T05:40:43.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.944+0000 7fb859e00700 1 --2- 172.21.15.131:0/4215032779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef53b70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb8480098e0 tx=0x7fb848005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63da178af7189dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:43.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.948+0000 7fb847000700 1 -- 172.21.15.131:0/4215032779 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84801d070 con 0x56399ef53720 2024-05-10T05:40:43.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.948+0000 7fb847000700 1 -- 172.21.15.131:0/4215032779 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb84800be00 con 0x56399ef53720 2024-05-10T05:40:43.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.948+0000 7fb847000700 1 -- 172.21.15.131:0/4215032779 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848005550 con 0x56399ef53720 2024-05-10T05:40:43.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.955+0000 7fb87883fb80 1 -- 172.21.15.131:0/4215032779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 msgr2=0x56399ef53b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:43.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.955+0000 7fb87883fb80 1 --2- 172.21.15.131:0/4215032779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef53b70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb8480098e0 tx=0x7fb848005e80 comp rx=0 tx=0).stop 2024-05-10T05:40:43.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.956+0000 7fb87883fb80 1 -- 172.21.15.131:0/4215032779 shutdown_connections 2024-05-10T05:40:43.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.956+0000 7fb87883fb80 1 --2- 172.21.15.131:0/4215032779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef53b70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.956+0000 7fb87883fb80 1 --2- 172.21.15.131:0/4215032779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 0x56399ef5b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.956+0000 7fb87883fb80 1 --2- 172.21.15.131:0/4215032779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef54a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.956+0000 7fb87883fb80 1 -- 172.21.15.131:0/4215032779 >> 172.21.15.131:0/4215032779 conn(0x56399eeb5ae0 msgr2=0x56399ef4e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:43.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.956+0000 7fb87883fb80 1 -- 172.21.15.131:0/4215032779 shutdown_connections 2024-05-10T05:40:43.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.957+0000 7fb87883fb80 1 -- 172.21.15.131:0/4215032779 wait complete. 2024-05-10T05:40:43.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.958+0000 7fb87883fb80 1 -- start start 2024-05-10T05:40:43.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.960+0000 7fb87883fb80 1 Processor -- start 2024-05-10T05:40:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.965+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef5d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.965+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef5dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.966+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 0x56399ef64c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.966+0000 7fb859400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef5dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.966+0000 7fb859400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef5dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:54840/0 (socket says 172.21.15.131:54840) 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.966+0000 7fb859e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef5d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.966+0000 7fb859e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef5d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50316/0 (socket says 172.21.15.131:50316) 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.967+0000 7fb859400700 1 -- 172.21.15.131:0/2620612992 learned_addr learned my addr 172.21.15.131:0/2620612992 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.967+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399ee0f5e0 con 0x56399ef53720 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.967+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399ee0f280 con 0x56399ef545f0 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.967+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399ecf6720 con 0x56399ef54f40 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.967+0000 7fb85a800700 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 0x56399ef64c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.968+0000 7fb85a800700 1 -- 172.21.15.131:0/2620612992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 msgr2=0x56399ef5dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.968+0000 7fb85a800700 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef5dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.968+0000 7fb85a800700 1 -- 172.21.15.131:0/2620612992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 msgr2=0x56399ef5d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.968+0000 7fb85a800700 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef5d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.968+0000 7fb85a800700 1 -- 172.21.15.131:0/2620612992 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c1120d0 con 0x56399ef54f40 2024-05-10T05:40:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.968+0000 7fb859e00700 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef5d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:40:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.969+0000 7fb87883fb80 10 client.0 _ll_register_callbacks cb 0x56399ee18bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.969+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56399edcc2f0 con 0x56399ef54f40 2024-05-10T05:40:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.969+0000 7fb85a800700 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 0x56399ef64c20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b470 tx=0x7fb84c098e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe69552f710dc554 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.969+0000 7fb83b400700 10 client.26755 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:40:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.970+0000 7fb83b400700 1 -- 172.21.15.131:0/2620612992 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c099410 con 0x56399ef54f40 2024-05-10T05:40:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.970+0000 7fb83b400700 1 -- 172.21.15.131:0/2620612992 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb84c09d240 con 0x56399ef54f40 2024-05-10T05:40:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.970+0000 7fb83b400700 1 -- 172.21.15.131:0/2620612992 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb84c0a1890 con 0x56399ef54f40 2024-05-10T05:40:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.972+0000 7fb87883fb80 10 client.26755 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.972+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x56399eca0be0 con 0x56399ef54f40 2024-05-10T05:40:43.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.976+0000 7fb83b400700 1 -- 172.21.15.131:0/2620612992 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb84c098900 con 0x56399ef54f40 2024-05-10T05:40:43.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.976+0000 7fb87883fb80 10 client.26755 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:43.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.976+0000 7fb87883fb80 10 client.26755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:43.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.976+0000 7fb87883fb80 20 client.26755 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:43.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.976+0000 7fb87883fb80 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56399ef69170 0x56399ef6b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:43.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.976+0000 7fb87883fb80 4 client.26755 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:43.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.976+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56399ef22800 con 0x56399ef69170 2024-05-10T05:40:43.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.976+0000 7fb859e00700 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56399ef69170 0x56399ef6b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:43.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.978+0000 7fb83b400700 10 client.26755 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:43.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.979+0000 7fb859e00700 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56399ef69170 0x56399ef6b680 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b72003b65143a28 server_cookie=a490f11cbd49fdbb in_seq=0 out_seq=0 2024-05-10T05:40:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.987+0000 7fb83b400700 1 -- 172.21.15.131:0/2620612992 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56399ef22800 con 0x56399ef69170 2024-05-10T05:40:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.987+0000 7fb83b400700 10 client.26755 handle_command_reply: tid=0 2024-05-10T05:40:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.988+0000 7fb87883fb80 1 client.26755 shutdown 2024-05-10T05:40:44.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.998+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 msgr2=0x56399ef64c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:44.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:43.998+0000 7fb87883fb80 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 0x56399ef64c20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b470 tx=0x7fb84c098e70 comp rx=0 tx=0).stop 2024-05-10T05:40:44.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.004+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 shutdown_connections 2024-05-10T05:40:44.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.005+0000 7fb87883fb80 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56399ef69170 0x56399ef6b680 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.005+0000 7fb87883fb80 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef53720 0x56399ef5d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.005+0000 7fb87883fb80 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef54f40 0x56399ef64c20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.005+0000 7fb87883fb80 1 --2- 172.21.15.131:0/2620612992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef5dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.005+0000 7fb83b400700 0 client.26755 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:44.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.005+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 >> 172.21.15.131:0/2620612992 conn(0x56399eeb5ae0 msgr2=0x56399ef4e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:44.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.006+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 shutdown_connections 2024-05-10T05:40:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.012+0000 7fb87883fb80 1 -- 172.21.15.131:0/2620612992 wait complete. 2024-05-10T05:40:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.015+0000 7fb87883fb80 20 client.26755 trim_cache size 0 max 16384 2024-05-10T05:40:44.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.148+0000 7fb87883fb80 1 Processor -- start 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.149+0000 7fb87883fb80 1 -- start start 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.150+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef53720 0x56399ef5c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.150+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef64ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.150+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 0x56399ef4cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.153+0000 7fb87883fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399ef68ec0 con 0x56399ef54f40 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.153+0000 7fb87883fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399ef5c740 con 0x56399ef545f0 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.153+0000 7fb87883fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399ef6ffd0 con 0x56399ef53720 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.153+0000 7fb85a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 0x56399ef4cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.153+0000 7fb85a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 0x56399ef4cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50330/0 (socket says 172.21.15.131:50330) 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.153+0000 7fb85a800700 1 -- 172.21.15.131:0/1411840338 learned_addr learned my addr 172.21.15.131:0/1411840338 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.154+0000 7fb85a800700 1 -- 172.21.15.131:0/1411840338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef53720 msgr2=0x56399ef5c200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:40:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.154+0000 7fb85a800700 1 --2- 172.21.15.131:0/1411840338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef53720 0x56399ef5c200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.154+0000 7fb85a800700 1 -- 172.21.15.131:0/1411840338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 msgr2=0x56399ef64ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:40:44.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.154+0000 7fb85a800700 1 --2- 172.21.15.131:0/1411840338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef64ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.154+0000 7fb85a800700 1 -- 172.21.15.131:0/1411840338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56399ee395e0 con 0x56399ef54f40 2024-05-10T05:40:44.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.154+0000 7fb85a800700 1 --2- 172.21.15.131:0/1411840338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 0x56399ef4cd00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0b7390 tx=0x7fb84c09c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cc5dcd35171638 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:44.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.156+0000 7fb847a00700 1 -- 172.21.15.131:0/1411840338 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c09d240 con 0x56399ef54f40 2024-05-10T05:40:44.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.157+0000 7fb847a00700 1 -- 172.21.15.131:0/1411840338 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56399edcc2f0 con 0x56399ef54f40 2024-05-10T05:40:44.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.157+0000 7fb847a00700 1 -- 172.21.15.131:0/1411840338 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0a1660 con 0x56399ef54f40 2024-05-10T05:40:44.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.159+0000 7fb87883fb80 1 -- 172.21.15.131:0/1411840338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 msgr2=0x56399ef4cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:44.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.159+0000 7fb87883fb80 1 --2- 172.21.15.131:0/1411840338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 0x56399ef4cd00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0b7390 tx=0x7fb84c09c000 comp rx=0 tx=0).stop 2024-05-10T05:40:44.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.160+0000 7fb87883fb80 1 -- 172.21.15.131:0/1411840338 shutdown_connections 2024-05-10T05:40:44.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.160+0000 7fb87883fb80 1 --2- 172.21.15.131:0/1411840338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 0x56399ef4cd00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.160+0000 7fb87883fb80 1 --2- 172.21.15.131:0/1411840338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef53720 0x56399ef5c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.160+0000 7fb87883fb80 1 --2- 172.21.15.131:0/1411840338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef64ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.160+0000 7fb87883fb80 1 -- 172.21.15.131:0/1411840338 >> 172.21.15.131:0/1411840338 conn(0x56399eeb5ae0 msgr2=0x56399ef5baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:44.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.160+0000 7fb87883fb80 1 -- 172.21.15.131:0/1411840338 shutdown_connections 2024-05-10T05:40:44.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.160+0000 7fb87883fb80 1 -- 172.21.15.131:0/1411840338 wait complete. 2024-05-10T05:40:44.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.161+0000 7fb87883fb80 1 -- start start 2024-05-10T05:40:44.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.162+0000 7fb87883fb80 1 Processor -- start 2024-05-10T05:40:44.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.172+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef53720 0x56399ef63660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.172+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef63ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.172+0000 7fb87883fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 0x56399ef640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.172+0000 7fb87883fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399efbf780 con 0x56399ef54f40 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.172+0000 7fb87883fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399efbf8f0 con 0x56399ef545f0 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.172+0000 7fb87883fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399efbfab0 con 0x56399ef53720 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.172+0000 7fb87883fb80 10 client.0 _ll_register_callbacks cb 0x56399ee18bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.172+0000 7fb859400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef63ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.173+0000 7fb859400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef63ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:54866/0 (socket says 172.21.15.131:54866) 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.173+0000 7fb859400700 1 -- 172.21.15.131:0/4233171241 learned_addr learned my addr 172.21.15.131:0/4233171241 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.173+0000 7fb85a800700 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 0x56399ef640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.173+0000 7fb859400700 1 -- 172.21.15.131:0/4233171241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef53720 msgr2=0x56399ef63660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.173+0000 7fb859400700 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef53720 0x56399ef63660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.173+0000 7fb859400700 1 -- 172.21.15.131:0/4233171241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 msgr2=0x56399ef640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.173+0000 7fb859400700 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 0x56399ef640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.173+0000 7fb859400700 1 -- 172.21.15.131:0/4233171241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56399eca0be0 con 0x56399ef545f0 2024-05-10T05:40:44.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.174+0000 7fb859400700 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef63ba0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb854015620 tx=0x7fb854004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad4a3d9f95012d8c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:44.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.175+0000 7fb83b400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:44.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.175+0000 7fb83b400700 1 -- 172.21.15.131:0/4233171241 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854010040 con 0x56399ef545f0 2024-05-10T05:40:44.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.176+0000 7fb83b400700 1 -- 172.21.15.131:0/4233171241 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb854006e40 con 0x56399ef545f0 2024-05-10T05:40:44.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.176+0000 7fb87883fb80 1 -- 172.21.15.131:0/4233171241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb84800be00 con 0x56399ef545f0 2024-05-10T05:40:44.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.177+0000 7fb83b400700 1 -- 172.21.15.131:0/4233171241 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb854015ea0 con 0x56399ef545f0 2024-05-10T05:40:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.181+0000 7fb87883fb80 10 client.26705 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:44.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.181+0000 7fb87883fb80 1 -- 172.21.15.131:0/4233171241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56399ecf8d50 con 0x56399ef545f0 2024-05-10T05:40:44.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.182+0000 7fb83b400700 1 -- 172.21.15.131:0/4233171241 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb84800be00 con 0x56399ef545f0 2024-05-10T05:40:44.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.182+0000 7fb83b400700 1 -- 172.21.15.131:0/4233171241 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb854021040 con 0x56399ef545f0 2024-05-10T05:40:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.182+0000 7fb87883fb80 10 client.26705 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.182+0000 7fb87883fb80 10 client.26705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.182+0000 7fb87883fb80 20 client.26705 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.182+0000 7fb87883fb80 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56399efc1550 0x56399efc3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.183+0000 7fb87883fb80 4 client.26705 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:40:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.183+0000 7fb87883fb80 1 -- 172.21.15.131:0/4233171241 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56399ee6bf40 con 0x56399efc1550 2024-05-10T05:40:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.183+0000 7fb859e00700 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56399efc1550 0x56399efc3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.184+0000 7fb83b400700 10 client.26705 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:44.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.185+0000 7fb859e00700 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56399efc1550 0x56399efc3970 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94d8acb9e72fc2d server_cookie=eb7294d13149bb0f in_seq=0 out_seq=0 2024-05-10T05:40:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.187+0000 7fb83b400700 1 -- 172.21.15.131:0/4233171241 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56399ee6bf40 con 0x56399efc1550 2024-05-10T05:40:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.187+0000 7fb83b400700 10 client.26705 handle_command_reply: tid=0 2024-05-10T05:40:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.187+0000 7fb87883fb80 1 client.26705 shutdown 2024-05-10T05:40:44.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.191+0000 7fb87883fb80 1 -- 172.21.15.131:0/4233171241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 msgr2=0x56399ef63ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.191+0000 7fb87883fb80 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef63ba0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb854015620 tx=0x7fb854004ad0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.192+0000 7fb87883fb80 1 -- 172.21.15.131:0/4233171241 shutdown_connections 2024-05-10T05:40:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.192+0000 7fb87883fb80 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56399efc1550 0x56399efc3970 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.192+0000 7fb87883fb80 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56399ef54f40 0x56399ef640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.192+0000 7fb87883fb80 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56399ef53720 0x56399ef63660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.192+0000 7fb87883fb80 1 --2- 172.21.15.131:0/4233171241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56399ef545f0 0x56399ef63ba0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.192+0000 7fb87883fb80 1 -- 172.21.15.131:0/4233171241 >> 172.21.15.131:0/4233171241 conn(0x56399eeb5ae0 msgr2=0x56399ef52330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.192+0000 7fb83b400700 0 client.26705 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.192+0000 7fb87883fb80 1 -- 172.21.15.131:0/4233171241 shutdown_connections 2024-05-10T05:40:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.192+0000 7fb87883fb80 1 -- 172.21.15.131:0/4233171241 wait complete. 2024-05-10T05:40:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.196+0000 7fb87883fb80 20 client.26705 trim_cache size 0 max 16384 2024-05-10T05:40:44.201 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:40:44.201 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:40:44.201 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "4663924f-5800-4aa3-85a3-47d7db2a588d", 2024-05-10T05:40:44.201 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:40:44.201 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:40:44.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.205+0000 7fb860a00700 1 -- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb8400a12c0 msgr2=0x7fb8400a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:44.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.205+0000 7fb860a00700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb8400a12c0 0x7fb8400a37d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb85c19f7b0 tx=0x7fb85400d040 comp rx=0 tx=0).stop 2024-05-10T05:40:44.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.205+0000 7fb860a00700 1 -- 172.21.15.131:0/959299044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85c10efa0 msgr2=0x7fb85c1a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:44.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.205+0000 7fb860a00700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85c10efa0 0x7fb85c1a5be0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b4a0 tx=0x7fb84c00b7b0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.207+0000 7fb860a00700 1 -- 172.21.15.131:0/959299044 shutdown_connections 2024-05-10T05:40:44.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.207+0000 7fb860a00700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb8400a12c0 0x7fb8400a37d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.207+0000 7fb860a00700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85c10efa0 0x7fb85c1a5be0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.207+0000 7fb860a00700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85c109bb0 0x7fb85c19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.207+0000 7fb860a00700 1 --2- 172.21.15.131:0/959299044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb85c10c510 0x7fb85c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.207+0000 7fb860a00700 1 -- 172.21.15.131:0/959299044 >> 172.21.15.131:0/959299044 conn(0x7fb85c06ea00 msgr2=0x7fb85c10f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:44.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.208+0000 7fb860a00700 1 -- 172.21.15.131:0/959299044 shutdown_connections 2024-05-10T05:40:44.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.209+0000 7fb860a00700 1 -- 172.21.15.131:0/959299044 wait complete. 2024-05-10T05:40:44.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:40:44.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:44 smithi142 ceph-mon[77667]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 191 MiB/s wr, 59 op/s 2024-05-10T05:40:44.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:44 smithi131 ceph-mon[37264]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 191 MiB/s wr, 59 op/s 2024-05-10T05:40:44.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:44 smithi157 ceph-mon[78427]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 191 MiB/s wr, 59 op/s 2024-05-10T05:40:44.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.951+0000 7f1a83200700 1 -- 172.21.15.131:0/3481430608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a841024b0 msgr2=0x7f1a841048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:44.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.951+0000 7f1a83200700 1 --2- 172.21.15.131:0/3481430608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a841024b0 0x7f1a841048d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00b0a0 tx=0x7f1a7c00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.952+0000 7f1a83200700 1 -- 172.21.15.131:0/3481430608 shutdown_connections 2024-05-10T05:40:44.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.952+0000 7f1a83200700 1 --2- 172.21.15.131:0/3481430608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a841024b0 0x7f1a841048d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.952+0000 7f1a83200700 1 --2- 172.21.15.131:0/3481430608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a84107770 0x7f1a84109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.952+0000 7f1a83200700 1 --2- 172.21.15.131:0/3481430608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a84104e10 0x7f1a84107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.952+0000 7f1a83200700 1 -- 172.21.15.131:0/3481430608 >> 172.21.15.131:0/3481430608 conn(0x7f1a840fbef0 msgr2=0x7f1a840fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:44.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.952+0000 7f1a83200700 1 -- 172.21.15.131:0/3481430608 shutdown_connections 2024-05-10T05:40:44.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.952+0000 7f1a83200700 1 -- 172.21.15.131:0/3481430608 wait complete. 2024-05-10T05:40:44.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.954+0000 7f1a83200700 1 Processor -- start 2024-05-10T05:40:44.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.954+0000 7f1a83200700 1 -- start start 2024-05-10T05:40:44.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.954+0000 7f1a83200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a841024b0 0x7f1a84101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.954+0000 7f1a83200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a84104e10 0x7f1a841003c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.955+0000 7f1a83200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a84107770 0x7f1a84100900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.955+0000 7f1a83200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84100f60 con 0x7f1a84104e10 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.955+0000 7f1a83200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841010d0 con 0x7f1a841024b0 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.955+0000 7f1a83200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84101290 con 0x7f1a84107770 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.955+0000 7f1a81e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a841024b0 0x7f1a84101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.955+0000 7f1a81e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a841024b0 0x7f1a84101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:54888/0 (socket says 172.21.15.131:54888) 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.955+0000 7f1a81e00700 1 -- 172.21.15.131:0/456657772 learned_addr learned my addr 172.21.15.131:0/456657772 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.955+0000 7f1a82800700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a84107770 0x7f1a84100900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.955+0000 7f1a81400700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a84104e10 0x7f1a841003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.956+0000 7f1a81e00700 1 -- 172.21.15.131:0/456657772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a84107770 msgr2=0x7f1a84100900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.956+0000 7f1a81e00700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a84107770 0x7f1a84100900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.956+0000 7f1a81e00700 1 -- 172.21.15.131:0/456657772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a84104e10 msgr2=0x7f1a841003c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.956+0000 7f1a81e00700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a84104e10 0x7f1a841003c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.956+0000 7f1a81e00700 1 -- 172.21.15.131:0/456657772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c009d00 con 0x7f1a841024b0 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.956+0000 7f1a81e00700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a841024b0 0x7f1a84101d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00b070 tx=0x7f1a7c0078e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.957+0000 7f1a7aa00700 1 -- 172.21.15.131:0/456657772 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c00e040 con 0x7f1a841024b0 2024-05-10T05:40:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.957+0000 7f1a83200700 1 -- 172.21.15.131:0/456657772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a841a2170 con 0x7f1a841024b0 2024-05-10T05:40:44.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.957+0000 7f1a7aa00700 1 -- 172.21.15.131:0/456657772 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a7c00be40 con 0x7f1a841024b0 2024-05-10T05:40:44.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.957+0000 7f1a7aa00700 1 -- 172.21.15.131:0/456657772 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c01b6c0 con 0x7f1a841024b0 2024-05-10T05:40:44.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.957+0000 7f1a83200700 1 -- 172.21.15.131:0/456657772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a841a25f0 con 0x7f1a841024b0 2024-05-10T05:40:44.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.964+0000 7f1aa0a95b80 1 Processor -- start 2024-05-10T05:40:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.964+0000 7f1aa0a95b80 1 -- start start 2024-05-10T05:40:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.965+0000 7f1aa0a95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d7b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.965+0000 7f1aa0a95b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d7c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.965+0000 7f1aa0a95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d83200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.965+0000 7f1aa0a95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25c99150 con 0x55dd25d7c900 2024-05-10T05:40:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.965+0000 7f1aa0a95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25c96b10 con 0x55dd25d7ad40 2024-05-10T05:40:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.965+0000 7f1aa0a95b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25c92400 con 0x55dd25d7bfb0 2024-05-10T05:40:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.967+0000 7f1a7aa00700 1 -- 172.21.15.131:0/456657772 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1a7c01d470 con 0x7f1a841024b0 2024-05-10T05:40:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.969+0000 7f1a82800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d7b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.969+0000 7f1a82800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d7b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:54894/0 (socket says 172.21.15.131:54894) 2024-05-10T05:40:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.969+0000 7f1a82800700 1 -- 172.21.15.131:0/1504012410 learned_addr learned my addr 172.21.15.131:0/1504012410 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.970+0000 7f1a82800700 1 -- 172.21.15.131:0/1504012410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 msgr2=0x55dd25d7c3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:40:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.970+0000 7f1a82800700 1 --2- 172.21.15.131:0/1504012410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d7c3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.970+0000 7f1a82800700 1 -- 172.21.15.131:0/1504012410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 msgr2=0x55dd25d83200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.970+0000 7f1a82800700 1 --2- 172.21.15.131:0/1504012410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d83200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.970+0000 7f1a82800700 1 -- 172.21.15.131:0/1504012410 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd25b133b0 con 0x55dd25d7ad40 2024-05-10T05:40:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.970+0000 7f1a82800700 1 --2- 172.21.15.131:0/1504012410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d7b170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009cb0 tx=0x7f1a70011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=828a72d314c97b70 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.970+0000 7f1a7aa00700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1a680a1220 0x7f1a680a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.970+0000 7f1a6f000700 1 -- 172.21.15.131:0/1504012410 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70011e30 con 0x55dd25d7ad40 2024-05-10T05:40:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.971+0000 7f1a6f000700 1 -- 172.21.15.131:0/1504012410 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a70004cf0 con 0x55dd25d7ad40 2024-05-10T05:40:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.971+0000 7f1a6f000700 1 -- 172.21.15.131:0/1504012410 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70019690 con 0x55dd25d7ad40 2024-05-10T05:40:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.972+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/1504012410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 msgr2=0x55dd25d7b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.972+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/1504012410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d7b170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009cb0 tx=0x7f1a70011440 comp rx=0 tx=0).stop 2024-05-10T05:40:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.973+0000 7f1a81e00700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1a680a1220 0x7f1a680a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.974+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/1504012410 shutdown_connections 2024-05-10T05:40:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.974+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/1504012410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d83200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.974+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/1504012410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d7b170 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.974+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/1504012410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d7c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.974+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/1504012410 >> 172.21.15.131:0/1504012410 conn(0x55dd25cd4a60 msgr2=0x55dd25d6d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.974+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/1504012410 shutdown_connections 2024-05-10T05:40:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.976+0000 7f1a7aa00700 1 -- 172.21.15.131:0/456657772 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1a7c0cd7b0 con 0x7f1a841024b0 2024-05-10T05:40:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.976+0000 7f1a81e00700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1a680a1220 0x7f1a680a3730 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c09abc0 tx=0x7f1a7c00b3b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.976+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/1504012410 wait complete. 2024-05-10T05:40:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.981+0000 7f1aa0a95b80 1 -- start start 2024-05-10T05:40:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.982+0000 7f1aa0a95b80 1 Processor -- start 2024-05-10T05:40:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.987+0000 7f1aa0a95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d85150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.987+0000 7f1aa0a95b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d85690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.988+0000 7f1aa0a95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d8c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.988+0000 7f1aa0a95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25c2e560 con 0x55dd25d7c900 2024-05-10T05:40:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.988+0000 7f1aa0a95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25c2e200 con 0x55dd25d7ad40 2024-05-10T05:40:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.988+0000 7f1aa0a95b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25b156a0 con 0x55dd25d7bfb0 2024-05-10T05:40:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.988+0000 7f1a82800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d85150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.988+0000 7f1aa0a95b80 10 client.0 _ll_register_callbacks cb 0x55dd25c37b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.988+0000 7f1a82800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d85150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:54904/0 (socket says 172.21.15.131:54904) 2024-05-10T05:40:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.988+0000 7f1a82800700 1 -- 172.21.15.131:0/187529424 learned_addr learned my addr 172.21.15.131:0/187529424 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.989+0000 7f1a81400700 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d85690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.989+0000 7f1a82800700 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d8c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.989+0000 7f1a81400700 1 -- 172.21.15.131:0/187529424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 msgr2=0x55dd25d85150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.990+0000 7f1a81400700 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d85150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.990+0000 7f1a81400700 1 -- 172.21.15.131:0/187529424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 msgr2=0x55dd25d8c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.990+0000 7f1a81400700 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d8c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.990+0000 7f1a81400700 1 -- 172.21.15.131:0/187529424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a70009960 con 0x55dd25d7bfb0 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.990+0000 7f1a82800700 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d8c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.990+0000 7f1a81400700 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d85690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1a740116f0 tx=0x7f1a74011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f05cf48d0e607e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.990+0000 7f1a63400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.991+0000 7f1a63400700 1 -- 172.21.15.131:0/187529424 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74004300 con 0x55dd25d7bfb0 2024-05-10T05:40:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.991+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/187529424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd25beb270 con 0x55dd25d7bfb0 2024-05-10T05:40:44.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.991+0000 7f1a63400700 1 -- 172.21.15.131:0/187529424 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1a74004460 con 0x55dd25d7bfb0 2024-05-10T05:40:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.991+0000 7f1a63400700 1 -- 172.21.15.131:0/187529424 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a74003680 con 0x55dd25d7bfb0 2024-05-10T05:40:44.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.991+0000 7f1aa0a95b80 10 client.26764 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:44.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.991+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/187529424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dd25abfb60 con 0x55dd25d7bfb0 2024-05-10T05:40:44.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.992+0000 7f1a63400700 1 -- 172.21.15.131:0/187529424 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd25beb270 con 0x55dd25d7bfb0 2024-05-10T05:40:44.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.992+0000 7f1a63400700 1 -- 172.21.15.131:0/187529424 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a74024900 con 0x55dd25d7bfb0 2024-05-10T05:40:44.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.992+0000 7f1aa0a95b80 10 client.26764 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:44.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.992+0000 7f1aa0a95b80 10 client.26764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:44.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.992+0000 7f1aa0a95b80 20 client.26764 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:44.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.992+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dd25da1de0 0x55dd25da4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.992+0000 7f1aa0a95b80 4 client.26764 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:40:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.992+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/187529424 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd25d41780 con 0x55dd25da1de0 2024-05-10T05:40:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.992+0000 7f1a82800700 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dd25da1de0 0x55dd25da4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.993+0000 7f1a63400700 10 client.26764 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.993+0000 7f1a82800700 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dd25da1de0 0x55dd25da4230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd00678e571fb058 server_cookie=575d9333a361893f in_seq=0 out_seq=0 2024-05-10T05:40:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.994+0000 7f1a63400700 1 -- 172.21.15.131:0/187529424 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd25d41780 con 0x55dd25da1de0 2024-05-10T05:40:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.994+0000 7f1a63400700 10 client.26764 handle_command_reply: tid=0 2024-05-10T05:40:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.994+0000 7f1aa0a95b80 1 client.26764 shutdown 2024-05-10T05:40:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.995+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/187529424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 msgr2=0x55dd25d85690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.995+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d85690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1a740116f0 tx=0x7f1a74011ab0 comp rx=0 tx=0).stop 2024-05-10T05:40:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.997+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/187529424 shutdown_connections 2024-05-10T05:40:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.997+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dd25da1de0 0x55dd25da4230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.997+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d8c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.997+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7ad40 0x55dd25d85150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.997+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/187529424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d85690 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.997+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/187529424 >> 172.21.15.131:0/187529424 conn(0x55dd25cd4a60 msgr2=0x55dd25d79870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:45.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.997+0000 7f1a63400700 0 client.26764 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:45.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.998+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/187529424 shutdown_connections 2024-05-10T05:40:45.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.998+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/187529424 wait complete. 2024-05-10T05:40:45.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:44.998+0000 7f1aa0a95b80 20 client.26764 trim_cache size 0 max 16384 2024-05-10T05:40:45.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.035+0000 7f1aa0a95b80 1 Processor -- start 2024-05-10T05:40:45.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.035+0000 7f1aa0a95b80 1 -- start start 2024-05-10T05:40:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.036+0000 7f1aa0a95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7ad40 0x55dd25da3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.036+0000 7f1aa0a95b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d8c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.036+0000 7f1aa0a95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7c900 0x55dd25da4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.036+0000 7f1aa0a95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25d9e700 con 0x55dd25d7ad40 2024-05-10T05:40:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.036+0000 7f1aa0a95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25d678d0 con 0x55dd25d7c900 2024-05-10T05:40:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.036+0000 7f1aa0a95b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25d7bb20 con 0x55dd25d7bfb0 2024-05-10T05:40:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.037+0000 7f1a81400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d8c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.037+0000 7f1a82800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7ad40 0x55dd25da3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.037+0000 7f1a82800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7ad40 0x55dd25da3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50376/0 (socket says 172.21.15.131:50376) 2024-05-10T05:40:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.037+0000 7f1a82800700 1 -- 172.21.15.131:0/711586657 learned_addr learned my addr 172.21.15.131:0/711586657 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.037+0000 7f1a82800700 1 --2- 172.21.15.131:0/711586657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7c900 0x55dd25da4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:45.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.037+0000 7f1a82800700 1 -- 172.21.15.131:0/711586657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 msgr2=0x55dd25d8c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:45.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.037+0000 7f1a82800700 1 --2- 172.21.15.131:0/711586657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d8c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.037+0000 7f1a82800700 1 -- 172.21.15.131:0/711586657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7c900 msgr2=0x55dd25da4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:45.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.037+0000 7f1a82800700 1 --2- 172.21.15.131:0/711586657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7c900 0x55dd25da4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.037+0000 7f1a82800700 1 -- 172.21.15.131:0/711586657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd25c58560 con 0x55dd25d7ad40 2024-05-10T05:40:45.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.038+0000 7f1a82800700 1 --2- 172.21.15.131:0/711586657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7ad40 0x55dd25da3df0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1a70011fd0 tx=0x7f1a70009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7082d92e7af74e98 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:45.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.038+0000 7f1a6fa00700 1 -- 172.21.15.131:0/711586657 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a700055b0 con 0x55dd25d7ad40 2024-05-10T05:40:45.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.038+0000 7f1a6fa00700 1 -- 172.21.15.131:0/711586657 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a70005710 con 0x55dd25d7ad40 2024-05-10T05:40:45.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.038+0000 7f1a6fa00700 1 -- 172.21.15.131:0/711586657 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7002c440 con 0x55dd25d7ad40 2024-05-10T05:40:45.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.039+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/711586657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7ad40 msgr2=0x55dd25da3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:45.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.039+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/711586657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7ad40 0x55dd25da3df0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1a70011fd0 tx=0x7f1a70009260 comp rx=0 tx=0).stop 2024-05-10T05:40:45.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.040+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/711586657 shutdown_connections 2024-05-10T05:40:45.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.040+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/711586657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7ad40 0x55dd25da3df0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.040+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/711586657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7c900 0x55dd25da4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.040+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/711586657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7bfb0 0x55dd25d8c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.040+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/711586657 >> 172.21.15.131:0/711586657 conn(0x55dd25cd4a60 msgr2=0x55dd25d7fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:45.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.040+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/711586657 shutdown_connections 2024-05-10T05:40:45.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.040+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/711586657 wait complete. 2024-05-10T05:40:45.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.041+0000 7f1aa0a95b80 1 -- start start 2024-05-10T05:40:45.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.041+0000 7f1aa0a95b80 1 Processor -- start 2024-05-10T05:40:45.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.043+0000 7f1aa0a95b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7ad40 0x55dd25d89e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.043+0000 7f1a82800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7ad40 0x55dd25d89e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.043+0000 7f1a82800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7ad40 0x55dd25d89e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:38152/0 (socket says 172.21.15.131:38152) 2024-05-10T05:40:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.043+0000 7f1a82800700 1 -- 172.21.15.131:0/4231636936 learned_addr learned my addr 172.21.15.131:0/4231636936 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:40:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.043+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7bfb0 0x55dd25d8a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.043+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d8b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.043+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25d8b760 con 0x55dd25d7c900 2024-05-10T05:40:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.043+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25d8b8d0 con 0x55dd25d7bfb0 2024-05-10T05:40:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.043+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd25d8ba90 con 0x55dd25d7ad40 2024-05-10T05:40:45.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.044+0000 7f1aa0a95b80 10 client.0 _ll_register_callbacks cb 0x55dd25c37b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:40:45.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.044+0000 7f1a81400700 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7bfb0 0x55dd25d8a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:45.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.044+0000 7f1a81400700 1 -- 172.21.15.131:0/4231636936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7ad40 msgr2=0x55dd25d89e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:40:45.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.044+0000 7f1a81400700 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7ad40 0x55dd25d89e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.044+0000 7f1a81400700 1 -- 172.21.15.131:0/4231636936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 msgr2=0x55dd25d8b220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:40:45.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.045+0000 7f1a82800700 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d8b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:45.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.045+0000 7f1a81400700 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d8b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.045+0000 7f1a81400700 1 -- 172.21.15.131:0/4231636936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd25b133b0 con 0x55dd25d7bfb0 2024-05-10T05:40:45.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.045+0000 7f1a82800700 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d8b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:40:45.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.045+0000 7f1a81400700 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7bfb0 0x55dd25d8a340 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1a74024d60 tx=0x7f1a74024690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b4f6293e48ccae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:40:45.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.045+0000 7f1a63400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:40:45.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.045+0000 7f1a63400700 1 -- 172.21.15.131:0/4231636936 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74005270 con 0x55dd25d7bfb0 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.046+0000 7f1a63400700 1 -- 172.21.15.131:0/4231636936 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1a74000e40 con 0x55dd25d7bfb0 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.046+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a70004cf0 con 0x55dd25d7bfb0 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.046+0000 7f1a63400700 1 -- 172.21.15.131:0/4231636936 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a7400f3c0 con 0x55dd25d7bfb0 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.046+0000 7f1aa0a95b80 10 client.26723 fetch_fsmap learned FSMap version 29 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.046+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dd25b17cd0 con 0x55dd25d7bfb0 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.046+0000 7f1a63400700 1 -- 172.21.15.131:0/4231636936 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a70004cf0 con 0x55dd25d7bfb0 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.046+0000 7f1a63400700 1 -- 172.21.15.131:0/4231636936 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a74003680 con 0x55dd25d7bfb0 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.047+0000 7f1aa0a95b80 10 client.26723 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.047+0000 7f1aa0a95b80 10 client.26723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.047+0000 7f1aa0a95b80 20 client.26723 populate_metadata read hostname 'smithi131' 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.047+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dd25df8bb0 0x55dd25dfafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.047+0000 7f1aa0a95b80 4 client.26723 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:40:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.047+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dd25c8aec0 con 0x55dd25df8bb0 2024-05-10T05:40:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.048+0000 7f1a82800700 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dd25df8bb0 0x55dd25dfafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:40:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.048+0000 7f1a63400700 10 client.26723 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.048+0000 7f1a82800700 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dd25df8bb0 0x55dd25dfafd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac20742c5cd7181d server_cookie=ead79cea37a2dcaa in_seq=0 out_seq=0 2024-05-10T05:40:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1a63400700 1 -- 172.21.15.131:0/4231636936 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dd25c8aec0 con 0x55dd25df8bb0 2024-05-10T05:40:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1a63400700 10 client.26723 handle_command_reply: tid=0 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1aa0a95b80 1 client.26723 shutdown 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7bfb0 msgr2=0x55dd25d8a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7bfb0 0x55dd25d8a340 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1a74024d60 tx=0x7f1a74024690 comp rx=0 tx=0).stop 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 shutdown_connections 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55dd25df8bb0 0x55dd25dfafd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dd25d7c900 0x55dd25d8b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd25d7ad40 0x55dd25d89e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1a63400700 0 client.26723 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1aa0a95b80 1 --2- 172.21.15.131:0/4231636936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd25d7bfb0 0x55dd25d8a340 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.049+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 >> 172.21.15.131:0/4231636936 conn(0x55dd25cd4a60 msgr2=0x55dd25da3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.050+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 shutdown_connections 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.050+0000 7f1aa0a95b80 1 -- 172.21.15.131:0/4231636936 wait complete. 2024-05-10T05:40:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.050+0000 7f1aa0a95b80 20 client.26723 trim_cache size 0 max 16384 2024-05-10T05:40:45.053 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:40:45.053 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:40:45.053 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:40:45.053 INFO:teuthology.orchestra.run.smithi131.stdout: "4663924f-5800-4aa3-85a3-47d7db2a588d": { 2024-05-10T05:40:45.053 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:40:45.053 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "4663924f-5800-4aa3-85a3-47d7db2a588d", 2024-05-10T05:40:45.053 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:40:45.053 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:40:45.053 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:40:45.053 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:40:45.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.056+0000 7f1a83200700 1 -- 172.21.15.131:0/456657772 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1a680a1220 msgr2=0x7f1a680a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.056+0000 7f1a83200700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1a680a1220 0x7f1a680a3730 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c09abc0 tx=0x7f1a7c00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.056+0000 7f1a83200700 1 -- 172.21.15.131:0/456657772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a841024b0 msgr2=0x7f1a84101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:40:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.056+0000 7f1a83200700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a841024b0 0x7f1a84101d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00b070 tx=0x7f1a7c0078e0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.056+0000 7f1a83200700 1 -- 172.21.15.131:0/456657772 shutdown_connections 2024-05-10T05:40:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.056+0000 7f1a83200700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1a680a1220 0x7f1a680a3730 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.056+0000 7f1a83200700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a84107770 0x7f1a84100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.056+0000 7f1a83200700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a841024b0 0x7f1a84101d20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.056+0000 7f1a83200700 1 --2- 172.21.15.131:0/456657772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a84104e10 0x7f1a841003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:40:45.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.056+0000 7f1a83200700 1 -- 172.21.15.131:0/456657772 >> 172.21.15.131:0/456657772 conn(0x7f1a840fbef0 msgr2=0x7f1a841044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:40:45.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.064+0000 7f1a83200700 1 -- 172.21.15.131:0/456657772 shutdown_connections 2024-05-10T05:40:45.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:40:45.065+0000 7f1a83200700 1 -- 172.21.15.131:0/456657772 wait complete. 2024-05-10T05:40:45.203 INFO:tasks.cephfs.filesystem:scrub status for tag:4663924f-5800-4aa3-85a3-47d7db2a588d - {'path': '/', 'tag': '4663924f-5800-4aa3-85a3-47d7db2a588d', 'options': 'recursive,force'} 2024-05-10T05:40:45.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:45 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:40:45.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:45 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:45.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:45 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:40:45.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:45 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:40:45.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:45 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:45.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:45 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:45.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:45 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:40:45.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:45 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:45.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:45 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:40:45.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:45 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:40:45.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:45 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:45.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:45 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:45 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:40:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:45 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:45 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:40:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:45 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:40:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:45 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:45 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:40:46.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:46 smithi142 ceph-mon[77667]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:40:46.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:46 smithi131 ceph-mon[37264]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:40:46.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:46 smithi157 ceph-mon[78427]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:40:48.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:48 smithi142 ceph-mon[77667]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 210 MiB/s wr, 66 op/s 2024-05-10T05:40:48.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:48 smithi131 ceph-mon[37264]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 210 MiB/s wr, 66 op/s 2024-05-10T05:40:48.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:48 smithi157 ceph-mon[78427]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 210 MiB/s wr, 66 op/s 2024-05-10T05:40:50.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:50 smithi142 ceph-mon[77667]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 245 MiB/s wr, 74 op/s 2024-05-10T05:40:50.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:50 smithi131 ceph-mon[37264]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 245 MiB/s wr, 74 op/s 2024-05-10T05:40:50.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:50 smithi157 ceph-mon[78427]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 245 MiB/s wr, 74 op/s 2024-05-10T05:40:51.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:51 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:40:51.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:51 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:40:51.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:51 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:40:51.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:51 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:40:51.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:51 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:40:51.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:51 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:40:52.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:52 smithi131 ceph-mon[37264]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 266 MiB/s wr, 81 op/s 2024-05-10T05:40:52.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:52 smithi157 ceph-mon[78427]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 266 MiB/s wr, 81 op/s 2024-05-10T05:40:52.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:52 smithi142 ceph-mon[77667]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 266 MiB/s wr, 81 op/s 2024-05-10T05:40:54.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:54 smithi131 ceph-mon[37264]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 68 op/s 2024-05-10T05:40:54.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:54 smithi157 ceph-mon[78427]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 68 op/s 2024-05-10T05:40:54.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:54 smithi142 ceph-mon[77667]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 68 op/s 2024-05-10T05:40:55.806 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:55 smithi157 ceph-mon[78427]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 293 MiB/s wr, 87 op/s 2024-05-10T05:40:55.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:55 smithi142 ceph-mon[77667]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 293 MiB/s wr, 87 op/s 2024-05-10T05:40:55.964 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:55 smithi131 ceph-mon[37264]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 293 MiB/s wr, 87 op/s 2024-05-10T05:40:58.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:40:58 smithi157 ceph-mon[78427]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 250 MiB/s wr, 76 op/s 2024-05-10T05:40:58.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:40:58 smithi142 ceph-mon[77667]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 250 MiB/s wr, 76 op/s 2024-05-10T05:40:58.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:40:58 smithi131 ceph-mon[37264]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 250 MiB/s wr, 76 op/s 2024-05-10T05:41:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:00 smithi142 ceph-mon[77667]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 276 MiB/s wr, 82 op/s 2024-05-10T05:41:00.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:00 smithi131 ceph-mon[37264]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 276 MiB/s wr, 82 op/s 2024-05-10T05:41:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:00 smithi157 ceph-mon[78427]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 276 MiB/s wr, 82 op/s 2024-05-10T05:41:02.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:02 smithi142 ceph-mon[77667]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 274 MiB/s wr, 84 op/s 2024-05-10T05:41:02.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:02 smithi131 ceph-mon[37264]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 274 MiB/s wr, 84 op/s 2024-05-10T05:41:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:02 smithi157 ceph-mon[78427]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 274 MiB/s wr, 84 op/s 2024-05-10T05:41:04.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:04 smithi142 ceph-mon[77667]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 226 MiB/s wr, 69 op/s 2024-05-10T05:41:04.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:04 smithi131 ceph-mon[37264]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 226 MiB/s wr, 69 op/s 2024-05-10T05:41:04.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:04 smithi157 ceph-mon[78427]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 226 MiB/s wr, 69 op/s 2024-05-10T05:41:06.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:06 smithi142 ceph-mon[77667]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 296 MiB/s wr, 89 op/s 2024-05-10T05:41:06.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:06 smithi131 ceph-mon[37264]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 296 MiB/s wr, 89 op/s 2024-05-10T05:41:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:06 smithi157 ceph-mon[78427]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 296 MiB/s wr, 89 op/s 2024-05-10T05:41:08.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:08 smithi142 ceph-mon[77667]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 239 MiB/s wr, 75 op/s 2024-05-10T05:41:08.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:08 smithi131 ceph-mon[37264]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 239 MiB/s wr, 75 op/s 2024-05-10T05:41:08.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:08 smithi157 ceph-mon[78427]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 239 MiB/s wr, 75 op/s 2024-05-10T05:41:10.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:10 smithi142 ceph-mon[77667]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 268 MiB/s wr, 83 op/s 2024-05-10T05:41:10.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:10 smithi131 ceph-mon[37264]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 268 MiB/s wr, 83 op/s 2024-05-10T05:41:10.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:10 smithi157 ceph-mon[78427]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 268 MiB/s wr, 83 op/s 2024-05-10T05:41:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:12 smithi131 ceph-mon[37264]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 285 MiB/s wr, 90 op/s 2024-05-10T05:41:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:41:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:12 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:12.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:12 smithi157 ceph-mon[78427]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 285 MiB/s wr, 90 op/s 2024-05-10T05:41:12.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:41:12.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:12.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:12 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:12.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:12 smithi142 ceph-mon[77667]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 285 MiB/s wr, 90 op/s 2024-05-10T05:41:12.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:41:12.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:12.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:12 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:13 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:13 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi142 to 182.4M 2024-05-10T05:41:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:13 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi157 to 182.5M 2024-05-10T05:41:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:13 smithi131 ceph-mon[37264]: Unable to set osd_memory_target on smithi142 to 191357644: error parsing value: Value '191357644' is below minimum 939524096 2024-05-10T05:41:13.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:13 smithi131 ceph-mon[37264]: Unable to set osd_memory_target on smithi157 to 191376281: error parsing value: Value '191376281' is below minimum 939524096 2024-05-10T05:41:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:13 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:13 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi142 to 182.4M 2024-05-10T05:41:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:13 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi157 to 182.5M 2024-05-10T05:41:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:13 smithi157 ceph-mon[78427]: Unable to set osd_memory_target on smithi142 to 191357644: error parsing value: Value '191357644' is below minimum 939524096 2024-05-10T05:41:13.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:13 smithi157 ceph-mon[78427]: Unable to set osd_memory_target on smithi157 to 191376281: error parsing value: Value '191376281' is below minimum 939524096 2024-05-10T05:41:13.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:13 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:13 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi142 to 182.4M 2024-05-10T05:41:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:13 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi157 to 182.5M 2024-05-10T05:41:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:13 smithi142 ceph-mon[77667]: Unable to set osd_memory_target on smithi142 to 191357644: error parsing value: Value '191357644' is below minimum 939524096 2024-05-10T05:41:13.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:13 smithi142 ceph-mon[77667]: Unable to set osd_memory_target on smithi157 to 191376281: error parsing value: Value '191376281' is below minimum 939524096 2024-05-10T05:41:14.717 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:14 smithi131 ceph-mon[37264]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 76 op/s 2024-05-10T05:41:14.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:41:14.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:41:14.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:41:14.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:14 smithi157 ceph-mon[78427]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 76 op/s 2024-05-10T05:41:14.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:41:14.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:41:14.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:41:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:14 smithi142 ceph-mon[77667]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 76 op/s 2024-05-10T05:41:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T05:41:14.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:41:14.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:41:14.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:41:15.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:41:15.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:15 smithi131 ceph-mon[37264]: Adjusting osd_memory_target on smithi131 to 221.2M 2024-05-10T05:41:15.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:15 smithi131 ceph-mon[37264]: Unable to set osd_memory_target on smithi131 to 232010956: error parsing value: Value '232010956' is below minimum 939524096 2024-05-10T05:41:15.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:15 smithi157 ceph-mon[78427]: Adjusting osd_memory_target on smithi131 to 221.2M 2024-05-10T05:41:15.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:15 smithi157 ceph-mon[78427]: Unable to set osd_memory_target on smithi131 to 232010956: error parsing value: Value '232010956' is below minimum 939524096 2024-05-10T05:41:15.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.827+0000 7fd1c1400700 1 -- 172.21.15.131:0/4185403099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1bc10efa0 msgr2=0x7fd1bc1114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:15.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.827+0000 7fd1c1400700 1 --2- 172.21.15.131:0/4185403099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1bc10efa0 0x7fd1bc1114b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009940 tx=0x7fd1a4009c50 comp rx=0 tx=0).stop 2024-05-10T05:41:15.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.828+0000 7fd1c1400700 1 -- 172.21.15.131:0/4185403099 shutdown_connections 2024-05-10T05:41:15.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.828+0000 7fd1c1400700 1 --2- 172.21.15.131:0/4185403099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1bc10efa0 0x7fd1bc1114b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.828+0000 7fd1c1400700 1 --2- 172.21.15.131:0/4185403099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1bc109bb0 0x7fd1bc10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.828+0000 7fd1c1400700 1 --2- 172.21.15.131:0/4185403099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc10c510 0x7fd1bc10e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.828+0000 7fd1c1400700 1 -- 172.21.15.131:0/4185403099 >> 172.21.15.131:0/4185403099 conn(0x7fd1bc06ea00 msgr2=0x7fd1bc070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:15.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.828+0000 7fd1c1400700 1 -- 172.21.15.131:0/4185403099 shutdown_connections 2024-05-10T05:41:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.828+0000 7fd1c1400700 1 -- 172.21.15.131:0/4185403099 wait complete. 2024-05-10T05:41:15.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.830+0000 7fd1c1400700 1 Processor -- start 2024-05-10T05:41:15.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.831+0000 7fd1c1400700 1 -- start start 2024-05-10T05:41:15.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.831+0000 7fd1c1400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1bc109bb0 0x7fd1bc1a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.831+0000 7fd1c1400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc10c510 0x7fd1bc1a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.831+0000 7fd1c1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1bc10efa0 0x7fd1bc1ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.831+0000 7fd1c1400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc1ae900 con 0x7fd1bc109bb0 2024-05-10T05:41:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.831+0000 7fd1c1400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc1aea70 con 0x7fd1bc10efa0 2024-05-10T05:41:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.831+0000 7fd1c1400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc1aec30 con 0x7fd1bc10c510 2024-05-10T05:41:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.832+0000 7fd1bb200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1bc10efa0 0x7fd1bc1ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.832+0000 7fd1bb200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1bc10efa0 0x7fd1bc1ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:42902/0 (socket says 172.21.15.131:42902) 2024-05-10T05:41:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.832+0000 7fd1bb200700 1 -- 172.21.15.131:0/1959063627 learned_addr learned my addr 172.21.15.131:0/1959063627 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.832+0000 7fd1b9e00700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc10c510 0x7fd1bc1a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.832+0000 7fd1bb200700 1 -- 172.21.15.131:0/1959063627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc10c510 msgr2=0x7fd1bc1a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.832+0000 7fd1bb200700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc10c510 0x7fd1bc1a7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.832+0000 7fd1bb200700 1 -- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1bc109bb0 msgr2=0x7fd1bc1a7240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:41:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.833+0000 7fd1ba800700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1bc109bb0 0x7fd1bc1a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.833+0000 7fd1bb200700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1bc109bb0 0x7fd1bc1a7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.833+0000 7fd1bb200700 1 -- 172.21.15.131:0/1959063627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a4009520 con 0x7fd1bc10efa0 2024-05-10T05:41:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.833+0000 7fd1b9e00700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc10c510 0x7fd1bc1a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:41:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.833+0000 7fd1ba800700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1bc109bb0 0x7fd1bc1a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.833+0000 7fd1bb200700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1bc10efa0 0x7fd1bc1ae2c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009910 tx=0x7fd1a400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.834+0000 7fd1b3400700 1 -- 172.21.15.131:0/1959063627 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a401c070 con 0x7fd1bc10efa0 2024-05-10T05:41:15.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.834+0000 7fd1c1400700 1 -- 172.21.15.131:0/1959063627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1bc1aee50 con 0x7fd1bc10efa0 2024-05-10T05:41:15.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.834+0000 7fd1c1400700 1 -- 172.21.15.131:0/1959063627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1bc1af1c0 con 0x7fd1bc10efa0 2024-05-10T05:41:15.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.834+0000 7fd1b3400700 1 -- 172.21.15.131:0/1959063627 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1a4005ce0 con 0x7fd1bc10efa0 2024-05-10T05:41:15.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.834+0000 7fd1b3400700 1 -- 172.21.15.131:0/1959063627 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4003ce0 con 0x7fd1bc10efa0 2024-05-10T05:41:15.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.841+0000 7fd1b3400700 1 -- 172.21.15.131:0/1959063627 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd1a4021420 con 0x7fd1bc10efa0 2024-05-10T05:41:15.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.846+0000 7fd1b3400700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd1a00a1240 0x7fd1a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.846+0000 7fd1b3400700 1 -- 172.21.15.131:0/1959063627 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd1a40ce7e0 con 0x7fd1bc10efa0 2024-05-10T05:41:15.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.848+0000 7fd1d92b4b80 1 Processor -- start 2024-05-10T05:41:15.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.847+0000 7fd1ba800700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd1a00a1240 0x7fd1a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.848+0000 7fd1d92b4b80 1 -- start start 2024-05-10T05:41:15.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.849+0000 7fd1d92b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f36a0 0x55e16c0f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.849+0000 7fd1d92b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c0f4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.849+0000 7fd1d92b4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c0f4ec0 0x55e16c0fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.849+0000 7fd1d92b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16c01a150 con 0x55e16c0f36a0 2024-05-10T05:41:15.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.849+0000 7fd1d92b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16c017b10 con 0x55e16c0f4570 2024-05-10T05:41:15.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.849+0000 7fd1d92b4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16c013400 con 0x55e16c0f4ec0 2024-05-10T05:41:15.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.849+0000 7fd1ba800700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd1a00a1240 0x7fd1a00a3750 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc1a8220 tx=0x7fd1b400b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.850+0000 7fd1ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f36a0 0x55e16c0f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.850+0000 7fd1ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f36a0 0x55e16c0f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46374/0 (socket says 172.21.15.131:46374) 2024-05-10T05:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.850+0000 7fd1ba800700 1 -- 172.21.15.131:0/3810261706 learned_addr learned my addr 172.21.15.131:0/3810261706 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.850+0000 7fd1b9e00700 1 --2- 172.21.15.131:0/3810261706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c0f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.851+0000 7fd1bb200700 1 --2- 172.21.15.131:0/3810261706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c0f4ec0 0x55e16c0fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.852+0000 7fd1ba800700 1 -- 172.21.15.131:0/3810261706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c0f4ec0 msgr2=0x55e16c0fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.852+0000 7fd1ba800700 1 --2- 172.21.15.131:0/3810261706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c0f4ec0 0x55e16c0fb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.852+0000 7fd1ba800700 1 -- 172.21.15.131:0/3810261706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 msgr2=0x55e16c0f4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.852+0000 7fd1ba800700 1 --2- 172.21.15.131:0/3810261706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c0f4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.852+0000 7fd1ba800700 1 -- 172.21.15.131:0/3810261706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e16be943b0 con 0x55e16c0f36a0 2024-05-10T05:41:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.853+0000 7fd1ba800700 1 --2- 172.21.15.131:0/3810261706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f36a0 0x55e16c0f3af0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009600 tx=0x7fd1b4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db117f3b4742f3c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.854+0000 7fd1ab400700 1 -- 172.21.15.131:0/3810261706 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4021400 con 0x55e16c0f36a0 2024-05-10T05:41:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.854+0000 7fd1ab400700 1 -- 172.21.15.131:0/3810261706 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1b4021a20 con 0x55e16c0f36a0 2024-05-10T05:41:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.854+0000 7fd1ab400700 1 -- 172.21.15.131:0/3810261706 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4021400 con 0x55e16c0f36a0 2024-05-10T05:41:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.856+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3810261706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f36a0 msgr2=0x55e16c0f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.856+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/3810261706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f36a0 0x55e16c0f3af0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009600 tx=0x7fd1b4012000 comp rx=0 tx=0).stop 2024-05-10T05:41:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.856+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3810261706 shutdown_connections 2024-05-10T05:41:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.856+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/3810261706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f36a0 0x55e16c0f3af0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.856+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/3810261706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c0f4ec0 0x55e16c0fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.856+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/3810261706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c0f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.856+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3810261706 >> 172.21.15.131:0/3810261706 conn(0x55e16c055a60 msgr2=0x55e16c0edfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.857+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3810261706 shutdown_connections 2024-05-10T05:41:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.857+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3810261706 wait complete. 2024-05-10T05:41:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.858+0000 7fd1d92b4b80 1 -- start start 2024-05-10T05:41:15.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.858+0000 7fd1d92b4b80 1 Processor -- start 2024-05-10T05:41:15.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.859+0000 7fd1d92b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c0fd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.859+0000 7fd1d92b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4ec0 0x55e16c0fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.859+0000 7fd1d92b4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c1050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.859+0000 7fd1d92b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16bfaf560 con 0x55e16c0f4ec0 2024-05-10T05:41:15.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.859+0000 7fd1d92b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16bfaf200 con 0x55e16c0f4570 2024-05-10T05:41:15.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.859+0000 7fd1d92b4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16be966a0 con 0x55e16c102ca0 2024-05-10T05:41:15.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.859+0000 7fd1d92b4b80 10 client.0 _ll_register_callbacks cb 0x55e16bfb8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:15.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.859+0000 7fd1ba800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c0fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.860+0000 7fd1b9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4ec0 0x55e16c0fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.860+0000 7fd1b9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4ec0 0x55e16c0fde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46384/0 (socket says 172.21.15.131:46384) 2024-05-10T05:41:15.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.860+0000 7fd1b9e00700 1 -- 172.21.15.131:0/2119793317 learned_addr learned my addr 172.21.15.131:0/2119793317 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:15.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.860+0000 7fd1bb200700 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c1050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.861+0000 7fd1b9e00700 1 -- 172.21.15.131:0/2119793317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 msgr2=0x55e16c1050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:15.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.861+0000 7fd1b9e00700 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c1050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.862+0000 7fd1b9e00700 1 -- 172.21.15.131:0/2119793317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 msgr2=0x55e16c0fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:15.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.862+0000 7fd1b9e00700 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c0fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.862+0000 7fd1b9e00700 1 -- 172.21.15.131:0/2119793317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b4007d90 con 0x55e16c0f4ec0 2024-05-10T05:41:15.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.862+0000 7fd1ba800700 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c0fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:41:15.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.862+0000 7fd19fe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:41:15.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.862+0000 7fd1b9e00700 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4ec0 0x55e16c0fde10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac012880 tx=0x7fd1ac012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98487c1c9c78dd7c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:15.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.863+0000 7fd19fe00700 1 -- 172.21.15.131:0/2119793317 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac017070 con 0x55e16c0f4ec0 2024-05-10T05:41:15.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.863+0000 7fd19fe00700 1 -- 172.21.15.131:0/2119793317 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd1ac0118d0 con 0x55e16c0f4ec0 2024-05-10T05:41:15.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.864+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2119793317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e16bf6c270 con 0x55e16c0f4ec0 2024-05-10T05:41:15.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.864+0000 7fd19fe00700 1 -- 172.21.15.131:0/2119793317 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1ac017070 con 0x55e16c0f4ec0 2024-05-10T05:41:15.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.865+0000 7fd1d92b4b80 10 client.16503 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:15.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.865+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2119793317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e16be40b60 con 0x55e16c0f4ec0 2024-05-10T05:41:15.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.866+0000 7fd19fe00700 1 -- 172.21.15.131:0/2119793317 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1ac026470 con 0x55e16c0f4ec0 2024-05-10T05:41:15.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.866+0000 7fd19fe00700 1 -- 172.21.15.131:0/2119793317 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd1ac00f460 con 0x55e16c0f4ec0 2024-05-10T05:41:15.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.866+0000 7fd1d92b4b80 10 client.16503 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:15.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.866+0000 7fd1d92b4b80 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:41:15.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.866+0000 7fd1d92b4b80 20 client.16503 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:15.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.867+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e16c109940 0x55e16c10bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.867+0000 7fd1d92b4b80 4 client.16503 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:15.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.867+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2119793317 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e16bf6c270 con 0x55e16c109940 2024-05-10T05:41:15.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.867+0000 7fd1ba800700 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e16c109940 0x55e16c10bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.868+0000 7fd1ba800700 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e16c109940 0x55e16c10bdb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d09f5de5db23cd8 server_cookie=7a228b7afb5f506 in_seq=0 out_seq=0 2024-05-10T05:41:15.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.868+0000 7fd19fe00700 10 client.16503 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:15.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.869+0000 7fd19fe00700 1 -- 172.21.15.131:0/2119793317 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e16bf6c270 con 0x55e16c109940 2024-05-10T05:41:15.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.869+0000 7fd19fe00700 10 client.16503 handle_command_reply: tid=0 2024-05-10T05:41:15.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.869+0000 7fd1d92b4b80 1 client.16503 shutdown 2024-05-10T05:41:15.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.870+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2119793317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4ec0 msgr2=0x55e16c0fde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:15.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.870+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4ec0 0x55e16c0fde10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac012880 tx=0x7fd1ac012b90 comp rx=0 tx=0).stop 2024-05-10T05:41:15.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.870+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2119793317 shutdown_connections 2024-05-10T05:41:15.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.870+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e16c109940 0x55e16c10bdb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.870+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c1050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.870+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c0fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.870+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2119793317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4ec0 0x55e16c0fde10 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac012880 tx=0x7fd1ac012b90 comp rx=0 tx=0).stop 2024-05-10T05:41:15.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.870+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2119793317 >> 172.21.15.131:0/2119793317 conn(0x55e16c055a60 msgr2=0x55e16c0edfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:15.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.871+0000 7fd19fe00700 0 client.16503 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:15.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.872+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2119793317 shutdown_connections 2024-05-10T05:41:15.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.872+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2119793317 wait complete. 2024-05-10T05:41:15.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.878+0000 7fd1d92b4b80 20 client.16503 trim_cache size 0 max 16384 2024-05-10T05:41:15.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:15 smithi142 ceph-mon[77667]: Adjusting osd_memory_target on smithi131 to 221.2M 2024-05-10T05:41:15.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:15 smithi142 ceph-mon[77667]: Unable to set osd_memory_target on smithi131 to 232010956: error parsing value: Value '232010956' is below minimum 939524096 2024-05-10T05:41:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.920+0000 7fd1d92b4b80 1 Processor -- start 2024-05-10T05:41:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.920+0000 7fd1d92b4b80 1 -- start start 2024-05-10T05:41:15.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.921+0000 7fd1d92b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4570 0x55e16c1050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.922+0000 7fd1ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4570 0x55e16c1050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.922+0000 7fd1ba800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4570 0x55e16c1050c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46386/0 (socket says 172.21.15.131:46386) 2024-05-10T05:41:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.923+0000 7fd1ba800700 1 -- 172.21.15.131:0/2076626708 learned_addr learned my addr 172.21.15.131:0/2076626708 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.923+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2076626708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c10b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.923+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2076626708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c10be90 0x55e16c119ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.923+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2076626708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16c0eecc0 con 0x55e16c0f4570 2024-05-10T05:41:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.924+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2076626708 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16c0e87c0 con 0x55e16c10be90 2024-05-10T05:41:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.924+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2076626708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16c0f3f50 con 0x55e16c102ca0 2024-05-10T05:41:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.925+0000 7fd1ba800700 1 -- 172.21.15.131:0/2076626708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 msgr2=0x55e16c10b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.925+0000 7fd1ba800700 1 --2- 172.21.15.131:0/2076626708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c10b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.925+0000 7fd1ba800700 1 -- 172.21.15.131:0/2076626708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c10be90 msgr2=0x55e16c119ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:41:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.925+0000 7fd1ba800700 1 --2- 172.21.15.131:0/2076626708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c10be90 0x55e16c119ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.925+0000 7fd1ba800700 1 -- 172.21.15.131:0/2076626708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e16bfd9560 con 0x55e16c0f4570 2024-05-10T05:41:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.925+0000 7fd1ba800700 1 --2- 172.21.15.131:0/2076626708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4570 0x55e16c1050c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4024040 tx=0x7fd1b4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553cdcbec88eea3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.928+0000 7fd1abe00700 1 -- 172.21.15.131:0/2076626708 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4022d20 con 0x55e16c0f4570 2024-05-10T05:41:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.928+0000 7fd1abe00700 1 -- 172.21.15.131:0/2076626708 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e16bf6c270 con 0x55e16c0f4570 2024-05-10T05:41:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.928+0000 7fd1abe00700 1 -- 172.21.15.131:0/2076626708 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4003a00 con 0x55e16c0f4570 2024-05-10T05:41:15.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.931+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2076626708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4570 msgr2=0x55e16c1050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:15.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.931+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2076626708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4570 0x55e16c1050c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4024040 tx=0x7fd1b4015000 comp rx=0 tx=0).stop 2024-05-10T05:41:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.932+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2076626708 shutdown_connections 2024-05-10T05:41:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.932+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2076626708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c0f4570 0x55e16c1050c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.932+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2076626708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c10be90 0x55e16c119ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.932+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/2076626708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c10b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.932+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2076626708 >> 172.21.15.131:0/2076626708 conn(0x55e16c055a60 msgr2=0x55e16c0edfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:15.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.934+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2076626708 shutdown_connections 2024-05-10T05:41:15.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.934+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/2076626708 wait complete. 2024-05-10T05:41:15.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.935+0000 7fd1d92b4b80 1 -- start start 2024-05-10T05:41:15.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.937+0000 7fd1d92b4b80 1 Processor -- start 2024-05-10T05:41:15.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.940+0000 7fd1d92b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c118b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.940+0000 7fd1d92b4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c119070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.940+0000 7fd1d92b4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c10be90 0x55e16c1195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.940+0000 7fd1d92b4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16c15e9a0 con 0x55e16c10be90 2024-05-10T05:41:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.940+0000 7fd1d92b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16c15eae0 con 0x55e16c0f4570 2024-05-10T05:41:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.940+0000 7fd1d92b4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16c15ec50 con 0x55e16c102ca0 2024-05-10T05:41:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.940+0000 7fd1d92b4b80 10 client.0 _ll_register_callbacks cb 0x55e16bfb8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.940+0000 7fd1b9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c119070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.940+0000 7fd1b9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c119070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52888/0 (socket says 172.21.15.131:52888) 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.941+0000 7fd1b9e00700 1 -- 172.21.15.131:0/3147121486 learned_addr learned my addr 172.21.15.131:0/3147121486 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.940+0000 7fd1ba800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c118b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.941+0000 7fd1b9e00700 1 -- 172.21.15.131:0/3147121486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 msgr2=0x55e16c118b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.941+0000 7fd1b9e00700 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c118b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.941+0000 7fd1b9e00700 1 -- 172.21.15.131:0/3147121486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c10be90 msgr2=0x55e16c1195b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.941+0000 7fd1b9e00700 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c10be90 0x55e16c1195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.941+0000 7fd1b9e00700 1 -- 172.21.15.131:0/3147121486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e16be943b0 con 0x55e16c102ca0 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.942+0000 7fd1b9e00700 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c119070 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac014040 tx=0x7fd1ac00ffa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944978b2f602e064 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:15.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.942+0000 7fd19fe00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:41:15.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.946+0000 7fd19fe00700 1 -- 172.21.15.131:0/3147121486 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac017070 con 0x55e16c102ca0 2024-05-10T05:41:15.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.946+0000 7fd19fe00700 1 -- 172.21.15.131:0/3147121486 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd1ac0118d0 con 0x55e16c102ca0 2024-05-10T05:41:15.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.946+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3147121486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1b4021a20 con 0x55e16c102ca0 2024-05-10T05:41:15.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.946+0000 7fd19fe00700 1 -- 172.21.15.131:0/3147121486 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1ac011a90 con 0x55e16c102ca0 2024-05-10T05:41:15.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.946+0000 7fd1d92b4b80 10 client.26782 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:15.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.946+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3147121486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e16be98cd0 con 0x55e16c102ca0 2024-05-10T05:41:15.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.947+0000 7fd19fe00700 1 -- 172.21.15.131:0/3147121486 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1b4021a20 con 0x55e16c102ca0 2024-05-10T05:41:15.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.947+0000 7fd19fe00700 1 -- 172.21.15.131:0/3147121486 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd1ac02e070 con 0x55e16c102ca0 2024-05-10T05:41:15.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.947+0000 7fd1d92b4b80 10 client.26782 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:15.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.947+0000 7fd1d92b4b80 10 client.26782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:41:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.947+0000 7fd1d92b4b80 20 client.26782 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.947+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e16c160570 0x55e16c162990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.948+0000 7fd1d92b4b80 4 client.26782 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:41:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.948+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3147121486 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e16c0c2780 con 0x55e16c160570 2024-05-10T05:41:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.948+0000 7fd1ba800700 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e16c160570 0x55e16c162990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:15.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.949+0000 7fd19fe00700 10 client.26782 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:15.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.949+0000 7fd1ba800700 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e16c160570 0x55e16c162990 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d347c1414aa5caa5 server_cookie=7abd52135bbb7a48 in_seq=0 out_seq=0 2024-05-10T05:41:15.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.949+0000 7fd19fe00700 1 -- 172.21.15.131:0/3147121486 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e16c0c2780 con 0x55e16c160570 2024-05-10T05:41:15.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.949+0000 7fd19fe00700 10 client.26782 handle_command_reply: tid=0 2024-05-10T05:41:15.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.949+0000 7fd1d92b4b80 1 client.26782 shutdown 2024-05-10T05:41:15.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.950+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3147121486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 msgr2=0x55e16c119070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:15.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.950+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c119070 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac014040 tx=0x7fd1ac00ffa0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.953+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3147121486 shutdown_connections 2024-05-10T05:41:15.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.953+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e16c160570 0x55e16c162990 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.953+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e16c10be90 0x55e16c1195b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.953+0000 7fd19fe00700 0 client.26782 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:15.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.953+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e16c0f4570 0x55e16c118b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.953+0000 7fd1d92b4b80 1 --2- 172.21.15.131:0/3147121486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e16c102ca0 0x55e16c119070 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.953+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3147121486 >> 172.21.15.131:0/3147121486 conn(0x55e16c055a60 msgr2=0x55e16c0fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.954+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3147121486 shutdown_connections 2024-05-10T05:41:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.954+0000 7fd1d92b4b80 1 -- 172.21.15.131:0/3147121486 wait complete. 2024-05-10T05:41:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.955+0000 7fd1d92b4b80 20 client.26782 trim_cache size 0 max 16384 2024-05-10T05:41:15.956 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:41:15.956 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:41:15.956 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:41:15.956 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:41:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.956+0000 7fd1c1400700 1 -- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd1a00a1240 msgr2=0x7fd1a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.956+0000 7fd1c1400700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd1a00a1240 0x7fd1a00a3750 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc1a8220 tx=0x7fd1b400b040 comp rx=0 tx=0).stop 2024-05-10T05:41:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.956+0000 7fd1c1400700 1 -- 172.21.15.131:0/1959063627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1bc10efa0 msgr2=0x7fd1bc1ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.956+0000 7fd1c1400700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1bc10efa0 0x7fd1bc1ae2c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009910 tx=0x7fd1a400b8c0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.956+0000 7fd1c1400700 1 -- 172.21.15.131:0/1959063627 shutdown_connections 2024-05-10T05:41:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.956+0000 7fd1c1400700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd1a00a1240 0x7fd1a00a3750 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.956+0000 7fd1c1400700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1bc109bb0 0x7fd1bc1a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.956+0000 7fd1c1400700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1bc10efa0 0x7fd1bc1ae2c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.956+0000 7fd1c1400700 1 --2- 172.21.15.131:0/1959063627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1bc10c510 0x7fd1bc1a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.956+0000 7fd1c1400700 1 -- 172.21.15.131:0/1959063627 >> 172.21.15.131:0/1959063627 conn(0x7fd1bc06ea00 msgr2=0x7fd1bc10e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.957+0000 7fd1c1400700 1 -- 172.21.15.131:0/1959063627 shutdown_connections 2024-05-10T05:41:15.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:15.958+0000 7fd1c1400700 1 -- 172.21.15.131:0/1959063627 wait complete. 2024-05-10T05:41:16.050 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:41:16.050 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:41:16.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:16 smithi157 ceph-mon[78427]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 294 MiB/s wr, 89 op/s 2024-05-10T05:41:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.817+0000 7fcc13200700 1 -- 172.21.15.131:0/3600740647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c103e00 msgr2=0x7fcc0c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.817+0000 7fcc13200700 1 --2- 172.21.15.131:0/3600740647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c103e00 0x7fcc0c104210 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400b0d0 tx=0x7fcc0400b3e0 comp rx=0 tx=0).stop 2024-05-10T05:41:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.818+0000 7fcc13200700 1 -- 172.21.15.131:0/3600740647 shutdown_connections 2024-05-10T05:41:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.818+0000 7fcc13200700 1 --2- 172.21.15.131:0/3600740647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc0c1058c0 0x7fcc0c10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.818+0000 7fcc13200700 1 --2- 172.21.15.131:0/3600740647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c103e00 0x7fcc0c104210 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.818+0000 7fcc13200700 1 --2- 172.21.15.131:0/3600740647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc0c104f20 0x7fcc0c105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.818+0000 7fcc13200700 1 -- 172.21.15.131:0/3600740647 >> 172.21.15.131:0/3600740647 conn(0x7fcc0c0ff410 msgr2=0x7fcc0c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.819+0000 7fcc13200700 1 -- 172.21.15.131:0/3600740647 shutdown_connections 2024-05-10T05:41:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.819+0000 7fcc13200700 1 -- 172.21.15.131:0/3600740647 wait complete. 2024-05-10T05:41:16.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.823+0000 7fcc13200700 1 Processor -- start 2024-05-10T05:41:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.828+0000 7fcc13200700 1 -- start start 2024-05-10T05:41:16.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.830+0000 7fcc13200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc0c103e00 0x7fcc0c195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:16.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.831+0000 7fcc13200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c104f20 0x7fcc0c196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.831+0000 7fcc13200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc0c1058c0 0x7fcc0c19d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.831+0000 7fcc0be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc0c103e00 0x7fcc0c195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.831+0000 7fcc0b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c104f20 0x7fcc0c196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.831+0000 7fcc0b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c104f20 0x7fcc0c196140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:42956/0 (socket says 172.21.15.131:42956) 2024-05-10T05:41:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.831+0000 7fcc0be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc0c103e00 0x7fcc0c195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46418/0 (socket says 172.21.15.131:46418) 2024-05-10T05:41:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.831+0000 7fcc0b400700 1 -- 172.21.15.131:0/2970200899 learned_addr learned my addr 172.21.15.131:0/2970200899 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:16.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.832+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c1ae740 con 0x7fcc0c103e00 2024-05-10T05:41:16.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.832+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c1ae880 con 0x7fcc0c104f20 2024-05-10T05:41:16.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.832+0000 7fcc11200700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc0c1058c0 0x7fcc0c19d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:16.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.832+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c1aea10 con 0x7fcc0c1058c0 2024-05-10T05:41:16.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.833+0000 7fcc0b400700 1 -- 172.21.15.131:0/2970200899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc0c1058c0 msgr2=0x7fcc0c19d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:16.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.833+0000 7fcc0b400700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc0c1058c0 0x7fcc0c19d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:16.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.833+0000 7fcc0b400700 1 -- 172.21.15.131:0/2970200899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc0c103e00 msgr2=0x7fcc0c195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:16.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.833+0000 7fcc0b400700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc0c103e00 0x7fcc0c195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:16.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.833+0000 7fcc0b400700 1 -- 172.21.15.131:0/2970200899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc04009d00 con 0x7fcc0c104f20 2024-05-10T05:41:16.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.833+0000 7fcc0be00700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc0c103e00 0x7fcc0c195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:41:16.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.834+0000 7fcc0b400700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c104f20 0x7fcc0c196140 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800e9a0 tx=0x7fcbf800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:16.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.834+0000 7fcc08c00700 1 -- 172.21.15.131:0/2970200899 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8009790 con 0x7fcc0c104f20 2024-05-10T05:41:16.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.834+0000 7fcc08c00700 1 -- 172.21.15.131:0/2970200899 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbf800cbd0 con 0x7fcc0c104f20 2024-05-10T05:41:16.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.836+0000 7fcc08c00700 1 -- 172.21.15.131:0/2970200899 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8018680 con 0x7fcc0c104f20 2024-05-10T05:41:16.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.836+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c1aecf0 con 0x7fcc0c104f20 2024-05-10T05:41:16.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.836+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c1af1b0 con 0x7fcc0c104f20 2024-05-10T05:41:16.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.838+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c0694d0 con 0x7fcc0c104f20 2024-05-10T05:41:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.841+0000 7fcc08c00700 1 -- 172.21.15.131:0/2970200899 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fcbf800cd40 con 0x7fcc0c104f20 2024-05-10T05:41:16.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.844+0000 7fcc08c00700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcbf00a1080 0x7fcbf00a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:16.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.845+0000 7fcc08c00700 1 -- 172.21.15.131:0/2970200899 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fcbf8014070 con 0x7fcc0c104f20 2024-05-10T05:41:16.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.845+0000 7fcc0be00700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcbf00a1080 0x7fcbf00a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:16.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.846+0000 7fcc0be00700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcbf00a1080 0x7fcbf00a3590 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009fd0 tx=0x7fcc04006020 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:16.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:16.849+0000 7fcc08c00700 1 -- 172.21.15.131:0/2970200899 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcbf8096400 con 0x7fcc0c104f20 2024-05-10T05:41:16.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:16 smithi142 ceph-mon[77667]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 294 MiB/s wr, 89 op/s 2024-05-10T05:41:16.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:16 smithi131 ceph-mon[37264]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 294 MiB/s wr, 89 op/s 2024-05-10T05:41:17.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.471+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc0c10f630 con 0x7fcc0c104f20 2024-05-10T05:41:17.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.474+0000 7fcc08c00700 1 -- 172.21.15.131:0/2970200899 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fcbf8099a20 con 0x7fcc0c104f20 2024-05-10T05:41:17.485 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:41:17.486 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:41:17.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.492+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcbf00a1080 msgr2=0x7fcbf00a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:17.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.492+0000 7fcc13200700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcbf00a1080 0x7fcbf00a3590 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009fd0 tx=0x7fcc04006020 comp rx=0 tx=0).stop 2024-05-10T05:41:17.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.492+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c104f20 msgr2=0x7fcc0c196140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:17.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.493+0000 7fcc13200700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c104f20 0x7fcc0c196140 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800e9a0 tx=0x7fcbf800ecb0 comp rx=0 tx=0).stop 2024-05-10T05:41:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.502+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 shutdown_connections 2024-05-10T05:41:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.502+0000 7fcc13200700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fcbf00a1080 0x7fcbf00a3590 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.502+0000 7fcc13200700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc0c103e00 0x7fcc0c195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.502+0000 7fcc13200700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc0c1058c0 0x7fcc0c19d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.502+0000 7fcc13200700 1 --2- 172.21.15.131:0/2970200899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0c104f20 0x7fcc0c196140 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.502+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 >> 172.21.15.131:0/2970200899 conn(0x7fcc0c0ff410 msgr2=0x7fcc0c077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.502+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 shutdown_connections 2024-05-10T05:41:17.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:17.502+0000 7fcc13200700 1 -- 172.21.15.131:0/2970200899 wait complete. 2024-05-10T05:41:17.518 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:41:17.641 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:41:17.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:17 smithi157 ceph-mon[78427]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 231 MiB/s wr, 72 op/s 2024-05-10T05:41:17.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:17 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2970200899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:41:17.817 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:17 smithi131 ceph-mon[37264]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 231 MiB/s wr, 72 op/s 2024-05-10T05:41:17.817 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:17 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2970200899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:41:17.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:17 smithi142 ceph-mon[77667]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 231 MiB/s wr, 72 op/s 2024-05-10T05:41:17.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:17 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2970200899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:41:18.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.398+0000 7f75f6e00700 1 -- 172.21.15.131:0/3857378965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75f0109500 msgr2=0x7f75f0109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:18.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.398+0000 7f75f6e00700 1 --2- 172.21.15.131:0/3857378965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75f0109500 0x7f75f0109910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009990 tx=0x7f75e8009ca0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.400+0000 7f75f6e00700 1 -- 172.21.15.131:0/3857378965 shutdown_connections 2024-05-10T05:41:18.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.400+0000 7f75f6e00700 1 --2- 172.21.15.131:0/3857378965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0108b60 0x7f75f010e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.401+0000 7f75f6e00700 1 --2- 172.21.15.131:0/3857378965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75f0109500 0x7f75f0109910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.401+0000 7f75f6e00700 1 --2- 172.21.15.131:0/3857378965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108210 0x7f75f0108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.401+0000 7f75f6e00700 1 -- 172.21.15.131:0/3857378965 >> 172.21.15.131:0/3857378965 conn(0x7f75f006e800 msgr2=0x7f75f0070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.401+0000 7f75f6e00700 1 -- 172.21.15.131:0/3857378965 shutdown_connections 2024-05-10T05:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.401+0000 7f75f6e00700 1 -- 172.21.15.131:0/3857378965 wait complete. 2024-05-10T05:41:18.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.403+0000 7f75f6e00700 1 Processor -- start 2024-05-10T05:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.403+0000 7f75f6e00700 1 -- start start 2024-05-10T05:41:18.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.404+0000 7f75f6e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108210 0x7f75f01a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.404+0000 7f75f6e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0108b60 0x7f75f01a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.404+0000 7f75f6e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75f0109500 0x7f75f01ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.404+0000 7f75f6e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f01ae6c0 con 0x7f75f0108b60 2024-05-10T05:41:18.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.404+0000 7f75f6e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f01ae800 con 0x7f75f0109500 2024-05-10T05:41:18.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.404+0000 7f75f6e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f01ae9c0 con 0x7f75f0108210 2024-05-10T05:41:18.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.405+0000 7f75f6400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75f0109500 0x7f75f01ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.405+0000 7f75f5a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108210 0x7f75f01a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.405+0000 7f75f5a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108210 0x7f75f01a6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52918/0 (socket says 172.21.15.131:52918) 2024-05-10T05:41:18.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.405+0000 7f75f5a00700 1 -- 172.21.15.131:0/1652119548 learned_addr learned my addr 172.21.15.131:0/1652119548 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.406+0000 7f75f5000700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0108b60 0x7f75f01a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.406+0000 7f75f5a00700 1 -- 172.21.15.131:0/1652119548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75f0109500 msgr2=0x7f75f01ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.406+0000 7f75f5a00700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75f0109500 0x7f75f01ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.406+0000 7f75f5a00700 1 -- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0108b60 msgr2=0x7f75f01a71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:18.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.406+0000 7f75f5a00700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0108b60 0x7f75f01a71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.406+0000 7f75f5a00700 1 -- 172.21.15.131:0/1652119548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e80095f0 con 0x7f75f0108210 2024-05-10T05:41:18.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.406+0000 7f75f5000700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0108b60 0x7f75f01a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:18.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.407+0000 7f75f5a00700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108210 0x7f75f01a6c60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f75e8000c00 tx=0x7f75e800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:18.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.409+0000 7f75ee000700 1 -- 172.21.15.131:0/1652119548 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e801d070 con 0x7f75f0108210 2024-05-10T05:41:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.409+0000 7f75ee000700 1 -- 172.21.15.131:0/1652119548 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e800fb80 con 0x7f75f0108210 2024-05-10T05:41:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.409+0000 7f75ee000700 1 -- 172.21.15.131:0/1652119548 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8017650 con 0x7f75f0108210 2024-05-10T05:41:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.410+0000 7f75f6e00700 1 -- 172.21.15.131:0/1652119548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f01aec40 con 0x7f75f0108210 2024-05-10T05:41:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.410+0000 7f75f6e00700 1 -- 172.21.15.131:0/1652119548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f01af010 con 0x7f75f0108210 2024-05-10T05:41:18.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.415+0000 7f75ee000700 1 -- 172.21.15.131:0/1652119548 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f75e80177b0 con 0x7f75f0108210 2024-05-10T05:41:18.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.417+0000 7f76102c2b80 1 Processor -- start 2024-05-10T05:41:18.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.419+0000 7f75ee000700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75d80a1270 0x7f75d80a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.419+0000 7f76102c2b80 1 -- start start 2024-05-10T05:41:18.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.419+0000 7f75ee000700 1 -- 172.21.15.131:0/1652119548 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f75e80cec20 con 0x7f75f0108210 2024-05-10T05:41:18.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.420+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f2530 0x560e2d7f2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.420+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f37a0 0x560e2d7f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.420+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d7f40f0 0x560e2d7fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.420+0000 7f76102c2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d719150 con 0x560e2d7f37a0 2024-05-10T05:41:18.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.420+0000 7f76102c2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d716b10 con 0x560e2d7f2530 2024-05-10T05:41:18.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.420+0000 7f76102c2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d712400 con 0x560e2d7f40f0 2024-05-10T05:41:18.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.420+0000 7f75f6400700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75d80a1270 0x7f75d80a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.421+0000 7f75f5000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f37a0 0x560e2d7f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.421+0000 7f75f5000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f37a0 0x560e2d7f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46440/0 (socket says 172.21.15.131:46440) 2024-05-10T05:41:18.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.421+0000 7f75f5000700 1 -- 172.21.15.131:0/2020728832 learned_addr learned my addr 172.21.15.131:0/2020728832 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:18.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.421+0000 7f75f6400700 1 --2- 172.21.15.131:0/2020728832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d7f40f0 0x560e2d7fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.421+0000 7f75f5a00700 1 --2- 172.21.15.131:0/2020728832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f2530 0x560e2d7f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.422+0000 7f75f5000700 1 -- 172.21.15.131:0/2020728832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d7f40f0 msgr2=0x560e2d7fa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:18.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.422+0000 7f75f5000700 1 --2- 172.21.15.131:0/2020728832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d7f40f0 0x560e2d7fa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.422+0000 7f75f5000700 1 -- 172.21.15.131:0/2020728832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f2530 msgr2=0x560e2d7f2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:18.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.422+0000 7f75f5000700 1 --2- 172.21.15.131:0/2020728832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f2530 0x560e2d7f2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.422+0000 7f75f5000700 1 -- 172.21.15.131:0/2020728832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e2d5933b0 con 0x560e2d7f37a0 2024-05-10T05:41:18.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.422+0000 7f75f6400700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75d80a1270 0x7f75d80a3780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009900 tx=0x7f75e4009ea0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:18.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.424+0000 7f75f5000700 1 --2- 172.21.15.131:0/2020728832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f37a0 0x560e2d7f3bb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00bc10 tx=0x7f75dc00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c69b3842d835cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:18.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.425+0000 7f75e2a00700 1 -- 172.21.15.131:0/2020728832 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00dd00 con 0x560e2d7f37a0 2024-05-10T05:41:18.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.425+0000 7f75e2a00700 1 -- 172.21.15.131:0/2020728832 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75dc00de60 con 0x560e2d7f37a0 2024-05-10T05:41:18.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.426+0000 7f75e2a00700 1 -- 172.21.15.131:0/2020728832 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00f7e0 con 0x560e2d7f37a0 2024-05-10T05:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.427+0000 7f76102c2b80 1 -- 172.21.15.131:0/2020728832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f37a0 msgr2=0x560e2d7f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.427+0000 7f76102c2b80 1 --2- 172.21.15.131:0/2020728832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f37a0 0x560e2d7f3bb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00bc10 tx=0x7f75dc00bf20 comp rx=0 tx=0).stop 2024-05-10T05:41:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.431+0000 7f76102c2b80 1 -- 172.21.15.131:0/2020728832 shutdown_connections 2024-05-10T05:41:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.431+0000 7f76102c2b80 1 --2- 172.21.15.131:0/2020728832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d7f40f0 0x560e2d7fa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.431+0000 7f76102c2b80 1 --2- 172.21.15.131:0/2020728832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f2530 0x560e2d7f2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.431+0000 7f76102c2b80 1 --2- 172.21.15.131:0/2020728832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f37a0 0x560e2d7f3bb0 secure :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00bc10 tx=0x7f75dc00bf20 comp rx=0 tx=0).stop 2024-05-10T05:41:18.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.431+0000 7f76102c2b80 1 -- 172.21.15.131:0/2020728832 >> 172.21.15.131:0/2020728832 conn(0x560e2d754a60 msgr2=0x560e2d7ed330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:18.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.434+0000 7f76102c2b80 1 -- 172.21.15.131:0/2020728832 shutdown_connections 2024-05-10T05:41:18.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.434+0000 7f76102c2b80 1 -- 172.21.15.131:0/2020728832 wait complete. 2024-05-10T05:41:18.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.435+0000 7f76102c2b80 1 -- start start 2024-05-10T05:41:18.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.436+0000 7f76102c2b80 1 Processor -- start 2024-05-10T05:41:18.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.438+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f2530 0x560e2d7fc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.438+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 0x560e2d7fcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.438+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d8042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.438+0000 7f76102c2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d6ae560 con 0x560e2d7f2530 2024-05-10T05:41:18.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.438+0000 7f76102c2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d6ae200 con 0x560e2d7f40f0 2024-05-10T05:41:18.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.438+0000 7f76102c2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d5956a0 con 0x560e2d801e80 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.439+0000 7f75f5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 0x560e2d7fcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.439+0000 7f75f5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 0x560e2d7fcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43004/0 (socket says 172.21.15.131:43004) 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.439+0000 7f75f5000700 1 -- 172.21.15.131:0/3947232447 learned_addr learned my addr 172.21.15.131:0/3947232447 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.439+0000 7f75f6400700 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d8042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.439+0000 7f75f6400700 1 -- 172.21.15.131:0/3947232447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 msgr2=0x560e2d7fcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.439+0000 7f75f6400700 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 0x560e2d7fcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.439+0000 7f75f6400700 1 -- 172.21.15.131:0/3947232447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f2530 msgr2=0x560e2d7fc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.439+0000 7f75f6400700 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f2530 0x560e2d7fc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.439+0000 7f75f6400700 1 -- 172.21.15.131:0/3947232447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc00b840 con 0x560e2d801e80 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.440+0000 7f75f6400700 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d8042a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f75e40154d0 tx=0x7f75e4018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eada47b395e25ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.440+0000 7f75d7400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.440+0000 7f75d7400700 1 -- 172.21.15.131:0/3947232447 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4010040 con 0x560e2d801e80 2024-05-10T05:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.440+0000 7f75d7400700 1 -- 172.21.15.131:0/3947232447 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f75e4006e40 con 0x560e2d801e80 2024-05-10T05:41:18.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.440+0000 7f75d7400700 10 client.0 inactive, discarding osd_map(82..82 src has 1..82) v4 2024-05-10T05:41:18.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.441+0000 7f76102c2b80 10 client.0 _ll_register_callbacks cb 0x560e2d6b7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:18.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.441+0000 7f76102c2b80 1 -- 172.21.15.131:0/3947232447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e2d66b270 con 0x560e2d801e80 2024-05-10T05:41:18.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.442+0000 7f75d7400700 1 -- 172.21.15.131:0/3947232447 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e400ae10 con 0x560e2d801e80 2024-05-10T05:41:18.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.442+0000 7f76102c2b80 10 client.26797 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:18.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.442+0000 7f76102c2b80 1 -- 172.21.15.131:0/3947232447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x560e2d53fb60 con 0x560e2d801e80 2024-05-10T05:41:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.443+0000 7f75d7400700 1 -- 172.21.15.131:0/3947232447 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f75e4021070 con 0x560e2d801e80 2024-05-10T05:41:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.443+0000 7f76102c2b80 10 client.26797 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.443+0000 7f76102c2b80 10 client.26797 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:41:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.443+0000 7f76102c2b80 20 client.26797 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.443+0000 7f76102c2b80 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560e2d808aa0 0x560e2d80afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.443+0000 7f76102c2b80 4 client.26797 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.443+0000 7f76102c2b80 1 -- 172.21.15.131:0/3947232447 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e2d66b270 con 0x560e2d808aa0 2024-05-10T05:41:18.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.443+0000 7f75f5a00700 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560e2d808aa0 0x560e2d80afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.444+0000 7f75d7400700 10 client.26797 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:41:18.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.444+0000 7f75f5a00700 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560e2d808aa0 0x560e2d80afb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e5dd783ec698b04 server_cookie=370853a3ab35d528 in_seq=0 out_seq=0 2024-05-10T05:41:18.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.445+0000 7f75d7400700 1 -- 172.21.15.131:0/3947232447 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e2d66b270 con 0x560e2d808aa0 2024-05-10T05:41:18.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.445+0000 7f75d7400700 10 client.26797 handle_command_reply: tid=0 2024-05-10T05:41:18.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.445+0000 7f76102c2b80 1 client.26797 shutdown 2024-05-10T05:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.446+0000 7f76102c2b80 1 -- 172.21.15.131:0/3947232447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 msgr2=0x560e2d8042a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.446+0000 7f76102c2b80 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d8042a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f75e40154d0 tx=0x7f75e4018000 comp rx=0 tx=0).stop 2024-05-10T05:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.446+0000 7f76102c2b80 1 -- 172.21.15.131:0/3947232447 shutdown_connections 2024-05-10T05:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.446+0000 7f76102c2b80 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560e2d808aa0 0x560e2d80afb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.446+0000 7f76102c2b80 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f2530 0x560e2d7fc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.446+0000 7f76102c2b80 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d8042a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.446+0000 7f76102c2b80 1 --2- 172.21.15.131:0/3947232447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 0x560e2d7fcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.446+0000 7f76102c2b80 1 -- 172.21.15.131:0/3947232447 >> 172.21.15.131:0/3947232447 conn(0x560e2d754a60 msgr2=0x560e2d7f1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.446+0000 7f75d7400700 0 client.26797 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:41:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.447+0000 7f76102c2b80 1 -- 172.21.15.131:0/3947232447 shutdown_connections 2024-05-10T05:41:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.447+0000 7f76102c2b80 1 -- 172.21.15.131:0/3947232447 wait complete. 2024-05-10T05:41:18.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.449+0000 7f76102c2b80 20 client.26797 trim_cache size 0 max 16384 2024-05-10T05:41:18.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.495+0000 7f76102c2b80 1 Processor -- start 2024-05-10T05:41:18.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.505+0000 7f76102c2b80 1 -- start start 2024-05-10T05:41:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.511+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f2530 0x560e2d8049b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.511+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f40f0 0x560e2d818ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.511+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d819420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.511+0000 7f76102c2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d7e9340 con 0x560e2d7f40f0 2024-05-10T05:41:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.511+0000 7f76102c2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d7f32a0 con 0x560e2d7f2530 2024-05-10T05:41:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.511+0000 7f76102c2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d7e4d60 con 0x560e2d801e80 2024-05-10T05:41:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.512+0000 7f75f6400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d819420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.512+0000 7f75f6400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d819420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:52946/0 (socket says 172.21.15.131:52946) 2024-05-10T05:41:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.512+0000 7f75f6400700 1 -- 172.21.15.131:0/3350170238 learned_addr learned my addr 172.21.15.131:0/3350170238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.512+0000 7f75f5000700 1 --2- 172.21.15.131:0/3350170238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f40f0 0x560e2d818ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.512+0000 7f75f6400700 1 -- 172.21.15.131:0/3350170238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f2530 msgr2=0x560e2d8049b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.512+0000 7f75f6400700 1 --2- 172.21.15.131:0/3350170238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f2530 0x560e2d8049b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.512+0000 7f75f6400700 1 -- 172.21.15.131:0/3350170238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f40f0 msgr2=0x560e2d818ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.512+0000 7f75f6400700 1 --2- 172.21.15.131:0/3350170238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f40f0 0x560e2d818ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.512+0000 7f75f6400700 1 -- 172.21.15.131:0/3350170238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e2d6d8560 con 0x560e2d801e80 2024-05-10T05:41:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.512+0000 7f75f5000700 1 --2- 172.21.15.131:0/3350170238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f40f0 0x560e2d818ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:41:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.513+0000 7f75f6400700 1 --2- 172.21.15.131:0/3350170238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d819420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75e402c020 tx=0x7f75e401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=989b329187bf7aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.513+0000 7f75e3400700 1 -- 172.21.15.131:0/3350170238 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4010070 con 0x560e2d801e80 2024-05-10T05:41:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.513+0000 7f75e3400700 1 -- 172.21.15.131:0/3350170238 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e4023450 con 0x560e2d801e80 2024-05-10T05:41:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.514+0000 7f75e3400700 1 -- 172.21.15.131:0/3350170238 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e400ae10 con 0x560e2d801e80 2024-05-10T05:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.516+0000 7f76102c2b80 1 -- 172.21.15.131:0/3350170238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 msgr2=0x560e2d819420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.516+0000 7f76102c2b80 1 --2- 172.21.15.131:0/3350170238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d819420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75e402c020 tx=0x7f75e401d000 comp rx=0 tx=0).stop 2024-05-10T05:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.516+0000 7f76102c2b80 1 -- 172.21.15.131:0/3350170238 shutdown_connections 2024-05-10T05:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.516+0000 7f76102c2b80 1 --2- 172.21.15.131:0/3350170238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d819420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.516+0000 7f76102c2b80 1 --2- 172.21.15.131:0/3350170238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f2530 0x560e2d8049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.516+0000 7f76102c2b80 1 --2- 172.21.15.131:0/3350170238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f40f0 0x560e2d818ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.516+0000 7f76102c2b80 1 -- 172.21.15.131:0/3350170238 >> 172.21.15.131:0/3350170238 conn(0x560e2d754a60 msgr2=0x560e2d8042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.519+0000 7f76102c2b80 1 -- 172.21.15.131:0/3350170238 shutdown_connections 2024-05-10T05:41:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.519+0000 7f76102c2b80 1 -- 172.21.15.131:0/3350170238 wait complete. 2024-05-10T05:41:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.520+0000 7f76102c2b80 1 -- start start 2024-05-10T05:41:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.520+0000 7f76102c2b80 1 Processor -- start 2024-05-10T05:41:18.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.525+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f2530 0x560e2d856760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.525+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 0x560e2d856ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.525+0000 7f76102c2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d85dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.525+0000 7f76102c2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d85e140 con 0x560e2d7f2530 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.525+0000 7f76102c2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d85e280 con 0x560e2d7f40f0 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.525+0000 7f76102c2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2d85e410 con 0x560e2d801e80 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.525+0000 7f76102c2b80 10 client.0 _ll_register_callbacks cb 0x560e2d6b7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.526+0000 7f75f5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 0x560e2d856ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.526+0000 7f75f5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 0x560e2d856ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43028/0 (socket says 172.21.15.131:43028) 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.526+0000 7f75f5000700 1 -- 172.21.15.131:0/779807241 learned_addr learned my addr 172.21.15.131:0/779807241 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.526+0000 7f75f6400700 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d85dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.527+0000 7f75f6400700 1 -- 172.21.15.131:0/779807241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 msgr2=0x560e2d856ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.527+0000 7f75f6400700 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 0x560e2d856ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.527+0000 7f75f6400700 1 -- 172.21.15.131:0/779807241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f2530 msgr2=0x560e2d856760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:18.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.527+0000 7f75f5a00700 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f2530 0x560e2d856760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.527+0000 7f75f6400700 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f2530 0x560e2d856760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.527+0000 7f75f6400700 1 -- 172.21.15.131:0/779807241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e2d53fb60 con 0x560e2d801e80 2024-05-10T05:41:18.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.527+0000 7f75f5a00700 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f2530 0x560e2d856760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:18.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.528+0000 7f75f6400700 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d85dc00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f75e40098d0 tx=0x7f75e4004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f3a859439465e95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:18.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.528+0000 7f75d7400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:41:18.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.528+0000 7f75d7400700 1 -- 172.21.15.131:0/779807241 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4010040 con 0x560e2d801e80 2024-05-10T05:41:18.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.528+0000 7f75d7400700 1 -- 172.21.15.131:0/779807241 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f75e400ae10 con 0x560e2d801e80 2024-05-10T05:41:18.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.528+0000 7f76102c2b80 1 -- 172.21.15.131:0/779807241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75dc00de60 con 0x560e2d801e80 2024-05-10T05:41:18.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.529+0000 7f75d7400700 1 -- 172.21.15.131:0/779807241 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e4021ec0 con 0x560e2d801e80 2024-05-10T05:41:18.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.532+0000 7f76102c2b80 10 client.26803 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:18.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.532+0000 7f76102c2b80 1 -- 172.21.15.131:0/779807241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560e2d597cd0 con 0x560e2d801e80 2024-05-10T05:41:18.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.535+0000 7f75d7400700 1 -- 172.21.15.131:0/779807241 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e403f080 con 0x560e2d801e80 2024-05-10T05:41:18.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.537+0000 7f75d7400700 1 -- 172.21.15.131:0/779807241 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f75e4021070 con 0x560e2d801e80 2024-05-10T05:41:18.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.537+0000 7f76102c2b80 10 client.26803 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:18.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.537+0000 7f76102c2b80 10 client.26803 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:41:18.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.537+0000 7f76102c2b80 20 client.26803 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:18.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.537+0000 7f76102c2b80 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560e2d85f9a0 0x560e2d861dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:18.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.537+0000 7f76102c2b80 4 client.26803 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:41:18.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.537+0000 7f76102c2b80 1 -- 172.21.15.131:0/779807241 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f75dc00de60 con 0x560e2d85f9a0 2024-05-10T05:41:18.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.538+0000 7f75f5a00700 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560e2d85f9a0 0x560e2d861dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:18.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.539+0000 7f75f5a00700 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560e2d85f9a0 0x560e2d861dc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed74f6207dc58c9a server_cookie=19ec394edf9c5a14 in_seq=0 out_seq=0 2024-05-10T05:41:18.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.539+0000 7f75d7400700 10 client.26803 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:41:18.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.539+0000 7f75d7400700 1 -- 172.21.15.131:0/779807241 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f75dc00de60 con 0x560e2d85f9a0 2024-05-10T05:41:18.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.539+0000 7f75d7400700 10 client.26803 handle_command_reply: tid=0 2024-05-10T05:41:18.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.539+0000 7f76102c2b80 1 client.26803 shutdown 2024-05-10T05:41:18.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.540+0000 7f76102c2b80 1 -- 172.21.15.131:0/779807241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 msgr2=0x560e2d85dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:18.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.540+0000 7f76102c2b80 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d85dc00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f75e40098d0 tx=0x7f75e4004be0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.543+0000 7f76102c2b80 1 -- 172.21.15.131:0/779807241 shutdown_connections 2024-05-10T05:41:18.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.543+0000 7f76102c2b80 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560e2d85f9a0 0x560e2d861dc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.543+0000 7f76102c2b80 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e2d7f2530 0x560e2d856760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.543+0000 7f76102c2b80 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e2d801e80 0x560e2d85dc00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.543+0000 7f76102c2b80 1 --2- 172.21.15.131:0/779807241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e2d7f40f0 0x560e2d856ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.543+0000 7f76102c2b80 1 -- 172.21.15.131:0/779807241 >> 172.21.15.131:0/779807241 conn(0x560e2d754a60 msgr2=0x560e2d7ecbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:18.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.543+0000 7f75d7400700 0 client.26803 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:41:18.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.544+0000 7f76102c2b80 1 -- 172.21.15.131:0/779807241 shutdown_connections 2024-05-10T05:41:18.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.549+0000 7f76102c2b80 1 -- 172.21.15.131:0/779807241 wait complete. 2024-05-10T05:41:18.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.551+0000 7f76102c2b80 20 client.26803 trim_cache size 0 max 16384 2024-05-10T05:41:18.553 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:41:18.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.566+0000 7f75f6e00700 1 -- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75d80a1270 msgr2=0x7f75d80a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:18.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.566+0000 7f75f6e00700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75d80a1270 0x7f75d80a3780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009900 tx=0x7f75e4009ea0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.566+0000 7f75f6e00700 1 -- 172.21.15.131:0/1652119548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108210 msgr2=0x7f75f01a6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:18.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.566+0000 7f75f6e00700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108210 0x7f75f01a6c60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f75e8000c00 tx=0x7f75e800f690 comp rx=0 tx=0).stop 2024-05-10T05:41:18.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.566+0000 7f75f6e00700 1 -- 172.21.15.131:0/1652119548 shutdown_connections 2024-05-10T05:41:18.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.566+0000 7f75f6e00700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f75d80a1270 0x7f75d80a3780 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.568+0000 7f75f6e00700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75f0109500 0x7f75f01ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.568+0000 7f75f6e00700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75f0108210 0x7f75f01a6c60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.568+0000 7f75f6e00700 1 --2- 172.21.15.131:0/1652119548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f0108b60 0x7f75f01a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:18.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.568+0000 7f75f6e00700 1 -- 172.21.15.131:0/1652119548 >> 172.21.15.131:0/1652119548 conn(0x7f75f006e800 msgr2=0x7f75f006f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:18.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.577+0000 7f75f6e00700 1 -- 172.21.15.131:0/1652119548 shutdown_connections 2024-05-10T05:41:18.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:18.577+0000 7f75f6e00700 1 -- 172.21.15.131:0/1652119548 wait complete. 2024-05-10T05:41:18.646 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:41:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.415+0000 7f9189800700 1 -- 172.21.15.131:0/3030436775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184102580 msgr2=0x7f9184108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.415+0000 7f9189800700 1 --2- 172.21.15.131:0/3030436775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184102580 0x7f9184108e30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f917c009fc0 tx=0x7f917c00b330 comp rx=0 tx=0).stop 2024-05-10T05:41:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.415+0000 7f9189800700 1 -- 172.21.15.131:0/3030436775 shutdown_connections 2024-05-10T05:41:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.416+0000 7f9189800700 1 --2- 172.21.15.131:0/3030436775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184100ac0 0x7f9184100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.416+0000 7f9189800700 1 --2- 172.21.15.131:0/3030436775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184102580 0x7f9184108e30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.416+0000 7f9189800700 1 --2- 172.21.15.131:0/3030436775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184101be0 0x7f9184101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.416+0000 7f9189800700 1 -- 172.21.15.131:0/3030436775 >> 172.21.15.131:0/3030436775 conn(0x7f91840fc0d0 msgr2=0x7f91840fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.416+0000 7f9189800700 1 -- 172.21.15.131:0/3030436775 shutdown_connections 2024-05-10T05:41:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.416+0000 7f9189800700 1 -- 172.21.15.131:0/3030436775 wait complete. 2024-05-10T05:41:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.418+0000 7f9189800700 1 Processor -- start 2024-05-10T05:41:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.419+0000 7f9189800700 1 -- start start 2024-05-10T05:41:19.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.419+0000 7f9189800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184100ac0 0x7f91841a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.419+0000 7f9189800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184101be0 0x7f91841a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.419+0000 7f9189800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102580 0x7f91841a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.420+0000 7f9182000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184101be0 0x7f91841a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.420+0000 7f9189800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841a5f90 con 0x7f9184100ac0 2024-05-10T05:41:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.420+0000 7f9182000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184101be0 0x7f91841a3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43058/0 (socket says 172.21.15.131:43058) 2024-05-10T05:41:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.420+0000 7f9182000700 1 -- 172.21.15.131:0/1088463993 learned_addr learned my addr 172.21.15.131:0/1088463993 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.420+0000 7f9189800700 1 -- 172.21.15.131:0/1088463993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841a6100 con 0x7f9184101be0 2024-05-10T05:41:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.420+0000 7f9189800700 1 -- 172.21.15.131:0/1088463993 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841a62c0 con 0x7f9184102580 2024-05-10T05:41:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.420+0000 7f9183400700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102580 0x7f91841a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.420+0000 7f9182a00700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184100ac0 0x7f91841a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.421+0000 7f9182000700 1 -- 172.21.15.131:0/1088463993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102580 msgr2=0x7f91841a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.421+0000 7f9182000700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102580 0x7f91841a5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.421+0000 7f9182000700 1 -- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184100ac0 msgr2=0x7f91841a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.421+0000 7f9182000700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184100ac0 0x7f91841a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.421+0000 7f9182000700 1 -- 172.21.15.131:0/1088463993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009c40 con 0x7f9184101be0 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.421+0000 7f9183400700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102580 0x7f91841a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.421+0000 7f9182a00700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184100ac0 0x7f91841a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.421+0000 7f9182000700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184101be0 0x7f91841a3200 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f917000e9a0 tx=0x7f917000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.421+0000 7f917b400700 1 -- 172.21.15.131:0/1088463993 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170009790 con 0x7f9184101be0 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.422+0000 7f917b400700 1 -- 172.21.15.131:0/1088463993 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f917000cbd0 con 0x7f9184101be0 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.422+0000 7f917b400700 1 -- 172.21.15.131:0/1088463993 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170018680 con 0x7f9184101be0 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.422+0000 7f9189800700 1 -- 172.21.15.131:0/1088463993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91841a65a0 con 0x7f9184101be0 2024-05-10T05:41:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.422+0000 7f9189800700 1 -- 172.21.15.131:0/1088463993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91841a6a60 con 0x7f9184101be0 2024-05-10T05:41:19.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.425+0000 7f917b400700 1 -- 172.21.15.131:0/1088463993 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f917000cd40 con 0x7f9184101be0 2024-05-10T05:41:19.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.427+0000 7f917b400700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a1290 0x7f91680a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.427+0000 7f917b400700 1 -- 172.21.15.131:0/1088463993 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9170014070 con 0x7f9184101be0 2024-05-10T05:41:19.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.428+0000 7f9182a00700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a1290 0x7f91680a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.429+0000 7f9182a00700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a1290 0x7f91680a37a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9184101ae0 tx=0x7f9174009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:19.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.431+0000 7f91a166bb80 1 Processor -- start 2024-05-10T05:41:19.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.431+0000 7f91a166bb80 1 -- start start 2024-05-10T05:41:19.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.431+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a46a0 0x559b893a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.431+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 0x559b893a5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.431+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.432+0000 7f91a166bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b892cb150 con 0x559b893a5ec0 2024-05-10T05:41:19.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.432+0000 7f91a166bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b892c8b10 con 0x559b893a46a0 2024-05-10T05:41:19.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.432+0000 7f91a166bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b892c4400 con 0x559b893a5570 2024-05-10T05:41:19.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.432+0000 7f9183400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.432+0000 7f9182000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 0x559b893a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.433+0000 7f9183400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893ac7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46514/0 (socket says 172.21.15.131:46514) 2024-05-10T05:41:19.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.433+0000 7f9183400700 1 -- 172.21.15.131:0/707529183 learned_addr learned my addr 172.21.15.131:0/707529183 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:19.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.433+0000 7f9182a00700 1 --2- 172.21.15.131:0/707529183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a46a0 0x559b893a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.433+0000 7f9182000700 1 -- 172.21.15.131:0/707529183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a46a0 msgr2=0x559b893a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:19.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.433+0000 7f9182000700 1 --2- 172.21.15.131:0/707529183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a46a0 0x559b893a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.433+0000 7f9182000700 1 -- 172.21.15.131:0/707529183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 msgr2=0x559b893ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:19.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.433+0000 7f9182000700 1 --2- 172.21.15.131:0/707529183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.433+0000 7f9182000700 1 -- 172.21.15.131:0/707529183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b891453b0 con 0x559b893a5570 2024-05-10T05:41:19.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.433+0000 7f9183400700 1 --2- 172.21.15.131:0/707529183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:41:19.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.434+0000 7f9182000700 1 --2- 172.21.15.131:0/707529183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 0x559b893a5980 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9170099db0 tx=0x7f9170098290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5251d4cf2b8ecbdf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:19.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.434+0000 7f916fe00700 1 -- 172.21.15.131:0/707529183 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917009a0b0 con 0x559b893a5570 2024-05-10T05:41:19.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.434+0000 7f916fe00700 1 -- 172.21.15.131:0/707529183 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f917009dee0 con 0x559b893a5570 2024-05-10T05:41:19.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.434+0000 7f916fe00700 1 -- 172.21.15.131:0/707529183 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170099010 con 0x559b893a5570 2024-05-10T05:41:19.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.435+0000 7f91a166bb80 1 -- 172.21.15.131:0/707529183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 msgr2=0x559b893a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:19.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.435+0000 7f91a166bb80 1 --2- 172.21.15.131:0/707529183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 0x559b893a5980 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9170099db0 tx=0x7f9170098290 comp rx=0 tx=0).stop 2024-05-10T05:41:19.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.436+0000 7f91a166bb80 1 -- 172.21.15.131:0/707529183 shutdown_connections 2024-05-10T05:41:19.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.436+0000 7f91a166bb80 1 --2- 172.21.15.131:0/707529183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.436+0000 7f91a166bb80 1 --2- 172.21.15.131:0/707529183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a46a0 0x559b893a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.436+0000 7f91a166bb80 1 --2- 172.21.15.131:0/707529183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 0x559b893a5980 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.436+0000 7f91a166bb80 1 -- 172.21.15.131:0/707529183 >> 172.21.15.131:0/707529183 conn(0x559b89306a60 msgr2=0x559b8939efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:19.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.436+0000 7f91a166bb80 1 -- 172.21.15.131:0/707529183 shutdown_connections 2024-05-10T05:41:19.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.436+0000 7f91a166bb80 1 -- 172.21.15.131:0/707529183 wait complete. 2024-05-10T05:41:19.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.437+0000 7f91a166bb80 1 -- start start 2024-05-10T05:41:19.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.437+0000 7f91a166bb80 1 Processor -- start 2024-05-10T05:41:19.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.438+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.438+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a5570 0x559b893aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.438+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.438+0000 7f91a166bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b89260560 con 0x559b893a5ec0 2024-05-10T05:41:19.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.438+0000 7f91a166bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b89260200 con 0x559b893a5570 2024-05-10T05:41:19.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.438+0000 7f91a166bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b891476a0 con 0x559b893a46a0 2024-05-10T05:41:19.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.438+0000 7f91a166bb80 10 client.0 _ll_register_callbacks cb 0x559b89269b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:19.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.439+0000 7f9182a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.439+0000 7f9182a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53006/0 (socket says 172.21.15.131:53006) 2024-05-10T05:41:19.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.439+0000 7f9182a00700 1 -- 172.21.15.131:0/3756961866 learned_addr learned my addr 172.21.15.131:0/3756961866 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:19.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.439+0000 7f9183400700 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.440+0000 7f9182a00700 1 -- 172.21.15.131:0/3756961866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a5570 msgr2=0x559b893aeca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:41:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.440+0000 7f9182a00700 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a5570 0x559b893aeca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.440+0000 7f9182a00700 1 -- 172.21.15.131:0/3756961866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 msgr2=0x559b893b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.440+0000 7f9182a00700 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.440+0000 7f9182a00700 1 -- 172.21.15.131:0/3756961866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91841098c0 con 0x559b893a46a0 2024-05-10T05:41:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.440+0000 7f9183400700 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:41:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.441+0000 7f9182a00700 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893ae760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9174010f10 tx=0x7f9174004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88dd0d41adb0b15d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.441+0000 7f916cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.441+0000 7f916cc00700 1 -- 172.21.15.131:0/3756961866 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174004a30 con 0x559b893a46a0 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.441+0000 7f916cc00700 1 -- 172.21.15.131:0/3756961866 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9174004b90 con 0x559b893a46a0 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.441+0000 7f91a166bb80 1 -- 172.21.15.131:0/3756961866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b8921d270 con 0x559b893a46a0 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f916cc00700 1 -- 172.21.15.131:0/3756961866 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9174012e80 con 0x559b893a46a0 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f91a166bb80 10 client.26815 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f91a166bb80 1 -- 172.21.15.131:0/3756961866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f91841a6a60 con 0x559b893a46a0 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f916cc00700 1 -- 172.21.15.131:0/3756961866 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b8921d270 con 0x559b893a46a0 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f916cc00700 1 -- 172.21.15.131:0/3756961866 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9174026030 con 0x559b893a46a0 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f91a166bb80 10 client.26815 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f91a166bb80 10 client.26815 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f91a166bb80 20 client.26815 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f91a166bb80 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x559b893cb520 0x559b893cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f91a166bb80 4 client.26815 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:19.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.442+0000 7f91a166bb80 1 -- 172.21.15.131:0/3756961866 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b89373780 con 0x559b893cb520 2024-05-10T05:41:19.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.446+0000 7f9182000700 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x559b893cb520 0x559b893cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.446+0000 7f9182000700 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x559b893cb520 0x559b893cd970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f450ed960930cf server_cookie=b7a7fc888ccba288 in_seq=0 out_seq=0 2024-05-10T05:41:19.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.446+0000 7f916cc00700 10 client.26815 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:41:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.447+0000 7f916cc00700 1 -- 172.21.15.131:0/3756961866 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b89373780 con 0x559b893cb520 2024-05-10T05:41:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.447+0000 7f916cc00700 10 client.26815 handle_command_reply: tid=0 2024-05-10T05:41:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.447+0000 7f91a166bb80 1 client.26815 shutdown 2024-05-10T05:41:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f91a166bb80 1 -- 172.21.15.131:0/3756961866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 msgr2=0x559b893ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f91a166bb80 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893ae760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9174010f10 tx=0x7f9174004750 comp rx=0 tx=0).stop 2024-05-10T05:41:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f91a166bb80 1 -- 172.21.15.131:0/3756961866 shutdown_connections 2024-05-10T05:41:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f91a166bb80 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x559b893cb520 0x559b893cd970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f916cc00700 0 client.26815 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:41:19.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f91a166bb80 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a5ec0 0x559b893b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f91a166bb80 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893ae760 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f91a166bb80 1 --2- 172.21.15.131:0/3756961866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a5570 0x559b893aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f91a166bb80 1 -- 172.21.15.131:0/3756961866 >> 172.21.15.131:0/3756961866 conn(0x559b89306a60 msgr2=0x559b8939efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f91a166bb80 1 -- 172.21.15.131:0/3756961866 shutdown_connections 2024-05-10T05:41:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.448+0000 7f91a166bb80 1 -- 172.21.15.131:0/3756961866 wait complete. 2024-05-10T05:41:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.449+0000 7f91a166bb80 20 client.26815 trim_cache size 0 max 16384 2024-05-10T05:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.477+0000 7f91a166bb80 1 Processor -- start 2024-05-10T05:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.477+0000 7f91a166bb80 1 -- start start 2024-05-10T05:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b89306a60 0x559b893a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a5570 0x559b893b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f91a166bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b893ac7b0 con 0x559b89306a60 2024-05-10T05:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f91a166bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b893cb1f0 con 0x559b893a5570 2024-05-10T05:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f91a166bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b8939b180 con 0x559b893a46a0 2024-05-10T05:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f9182a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b89306a60 0x559b893a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f9182a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b89306a60 0x559b893a87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46528/0 (socket says 172.21.15.131:46528) 2024-05-10T05:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f9182000700 1 --2- 172.21.15.131:0/4078723030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f9183400700 1 --2- 172.21.15.131:0/4078723030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a5570 0x559b893b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.478+0000 7f9182a00700 1 -- 172.21.15.131:0/4078723030 learned_addr learned my addr 172.21.15.131:0/4078723030 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.479+0000 7f9182000700 1 -- 172.21.15.131:0/4078723030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a5570 msgr2=0x559b893b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.479+0000 7f9182000700 1 --2- 172.21.15.131:0/4078723030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a5570 0x559b893b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.479+0000 7f9182000700 1 -- 172.21.15.131:0/4078723030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b89306a60 msgr2=0x559b893a87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.479+0000 7f9182000700 1 --2- 172.21.15.131:0/4078723030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b89306a60 0x559b893a87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.479+0000 7f9182000700 1 -- 172.21.15.131:0/4078723030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b890f1b60 con 0x559b893a46a0 2024-05-10T05:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.479+0000 7f9182000700 1 --2- 172.21.15.131:0/4078723030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893b3bb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x559b893af7d0 tx=0x7f9170098160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=617a4fc0c42c3b60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.479+0000 7f9178c00700 1 -- 172.21.15.131:0/4078723030 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917009a130 con 0x559b893a46a0 2024-05-10T05:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.479+0000 7f9178c00700 1 -- 172.21.15.131:0/4078723030 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9170082070 con 0x559b893a46a0 2024-05-10T05:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.479+0000 7f9178c00700 1 -- 172.21.15.131:0/4078723030 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170099010 con 0x559b893a46a0 2024-05-10T05:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.481+0000 7f91a166bb80 1 -- 172.21.15.131:0/4078723030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 msgr2=0x559b893b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.481+0000 7f91a166bb80 1 --2- 172.21.15.131:0/4078723030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893b3bb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x559b893af7d0 tx=0x7f9170098160 comp rx=0 tx=0).stop 2024-05-10T05:41:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.482+0000 7f91a166bb80 1 -- 172.21.15.131:0/4078723030 shutdown_connections 2024-05-10T05:41:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.482+0000 7f91a166bb80 1 --2- 172.21.15.131:0/4078723030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b89306a60 0x559b893a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.482+0000 7f91a166bb80 1 --2- 172.21.15.131:0/4078723030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b893a5570 0x559b893b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.482+0000 7f91a166bb80 1 --2- 172.21.15.131:0/4078723030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a46a0 0x559b893b3bb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.482+0000 7f91a166bb80 1 -- 172.21.15.131:0/4078723030 >> 172.21.15.131:0/4078723030 conn(0x559b8939b450 msgr2=0x559b893a93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.483+0000 7f91a166bb80 1 -- 172.21.15.131:0/4078723030 shutdown_connections 2024-05-10T05:41:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.485+0000 7f91a166bb80 1 -- 172.21.15.131:0/4078723030 wait complete. 2024-05-10T05:41:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.485+0000 7f91a166bb80 1 -- start start 2024-05-10T05:41:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.486+0000 7f91a166bb80 1 Processor -- start 2024-05-10T05:41:19.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.486+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b89306a60 0x559b893b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.486+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a46a0 0x559b893b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.486+0000 7f91a166bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 0x559b8941fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.486+0000 7f91a166bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b89420330 con 0x559b893a46a0 2024-05-10T05:41:19.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.486+0000 7f91a166bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b89420470 con 0x559b89306a60 2024-05-10T05:41:19.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.486+0000 7f91a166bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b89420600 con 0x559b893a5570 2024-05-10T05:41:19.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.487+0000 7f91a166bb80 10 client.0 _ll_register_callbacks cb 0x559b89269b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:19.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.487+0000 7f9182a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b89306a60 0x559b893b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.487+0000 7f9183400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 0x559b8941fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.487+0000 7f9182a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b89306a60 0x559b893b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43108/0 (socket says 172.21.15.131:43108) 2024-05-10T05:41:19.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.487+0000 7f9182a00700 1 -- 172.21.15.131:0/4210434207 learned_addr learned my addr 172.21.15.131:0/4210434207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:19.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.488+0000 7f9182000700 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a46a0 0x559b893b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.488+0000 7f9182a00700 1 -- 172.21.15.131:0/4210434207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 msgr2=0x559b8941fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:19.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.488+0000 7f9182a00700 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 0x559b8941fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.488+0000 7f9182a00700 1 -- 172.21.15.131:0/4210434207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a46a0 msgr2=0x559b893b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:19.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.488+0000 7f9182a00700 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a46a0 0x559b893b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.488+0000 7f9182a00700 1 -- 172.21.15.131:0/4210434207 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b891453b0 con 0x559b89306a60 2024-05-10T05:41:19.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.488+0000 7f9182000700 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a46a0 0x559b893b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:19.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.488+0000 7f9182a00700 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b89306a60 0x559b893b3650 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9174020720 tx=0x7f9174020470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d08ba12dbbda460e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:19.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.488+0000 7f916cc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:41:19.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.489+0000 7f916cc00700 1 -- 172.21.15.131:0/4210434207 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174038070 con 0x559b89306a60 2024-05-10T05:41:19.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.489+0000 7f916cc00700 1 -- 172.21.15.131:0/4210434207 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f9174005250 con 0x559b89306a60 2024-05-10T05:41:19.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.489+0000 7f91a166bb80 1 -- 172.21.15.131:0/4210434207 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f917009dee0 con 0x559b89306a60 2024-05-10T05:41:19.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.489+0000 7f916cc00700 1 -- 172.21.15.131:0/4210434207 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9174012ac0 con 0x559b89306a60 2024-05-10T05:41:19.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.489+0000 7f91a166bb80 10 client.26774 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:19.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.489+0000 7f91a166bb80 1 -- 172.21.15.131:0/4210434207 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559b8928a560 con 0x559b89306a60 2024-05-10T05:41:19.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.490+0000 7f916cc00700 1 -- 172.21.15.131:0/4210434207 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f917009dee0 con 0x559b89306a60 2024-05-10T05:41:19.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.491+0000 7f916cc00700 1 -- 172.21.15.131:0/4210434207 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9174026050 con 0x559b89306a60 2024-05-10T05:41:19.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.491+0000 7f91a166bb80 10 client.26774 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:19.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.491+0000 7f91a166bb80 10 client.26774 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:41:19.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.491+0000 7f91a166bb80 20 client.26774 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:19.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.491+0000 7f91a166bb80 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x559b893a5ec0 0x559b89423ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:19.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.491+0000 7f91a166bb80 4 client.26774 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:41:19.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.491+0000 7f91a166bb80 1 -- 172.21.15.131:0/4210434207 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559b892bcec0 con 0x559b893a5ec0 2024-05-10T05:41:19.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.492+0000 7f9182000700 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x559b893a5ec0 0x559b89423ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:19.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.493+0000 7f916cc00700 10 client.26774 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:41:19.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.493+0000 7f9182000700 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x559b893a5ec0 0x559b89423ec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc513c9801801a4b server_cookie=ce4880a8bacece4a in_seq=0 out_seq=0 2024-05-10T05:41:19.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.493+0000 7f916cc00700 1 -- 172.21.15.131:0/4210434207 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559b892bcec0 con 0x559b893a5ec0 2024-05-10T05:41:19.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.493+0000 7f916cc00700 10 client.26774 handle_command_reply: tid=0 2024-05-10T05:41:19.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.493+0000 7f91a166bb80 1 client.26774 shutdown 2024-05-10T05:41:19.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.494+0000 7f91a166bb80 1 -- 172.21.15.131:0/4210434207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b89306a60 msgr2=0x559b893b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:19.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.494+0000 7f91a166bb80 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b89306a60 0x559b893b3650 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9174020720 tx=0x7f9174020470 comp rx=0 tx=0).stop 2024-05-10T05:41:19.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.494+0000 7f91a166bb80 1 -- 172.21.15.131:0/4210434207 shutdown_connections 2024-05-10T05:41:19.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.494+0000 7f91a166bb80 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x559b893a5ec0 0x559b89423ec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.494+0000 7f91a166bb80 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b893a5570 0x559b8941fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.494+0000 7f91a166bb80 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b89306a60 0x559b893b3650 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.494+0000 7f91a166bb80 1 --2- 172.21.15.131:0/4210434207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b893a46a0 0x559b893b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.494+0000 7f91a166bb80 1 -- 172.21.15.131:0/4210434207 >> 172.21.15.131:0/4210434207 conn(0x559b8939b450 msgr2=0x559b893a87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:19.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.494+0000 7f916cc00700 0 client.26774 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:41:19.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.495+0000 7f91a166bb80 1 -- 172.21.15.131:0/4210434207 shutdown_connections 2024-05-10T05:41:19.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.495+0000 7f91a166bb80 1 -- 172.21.15.131:0/4210434207 wait complete. 2024-05-10T05:41:19.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.495+0000 7f91a166bb80 20 client.26774 trim_cache size 0 max 16384 2024-05-10T05:41:19.496 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:41:19.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.496+0000 7f9189800700 1 -- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a1290 msgr2=0x7f91680a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:19.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.496+0000 7f9189800700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a1290 0x7f91680a37a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9184101ae0 tx=0x7f9174009310 comp rx=0 tx=0).stop 2024-05-10T05:41:19.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.496+0000 7f9189800700 1 -- 172.21.15.131:0/1088463993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184101be0 msgr2=0x7f91841a3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:19.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.496+0000 7f9189800700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184101be0 0x7f91841a3200 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f917000e9a0 tx=0x7f917000ed60 comp rx=0 tx=0).stop 2024-05-10T05:41:19.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.497+0000 7f9189800700 1 -- 172.21.15.131:0/1088463993 shutdown_connections 2024-05-10T05:41:19.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.497+0000 7f9189800700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f91680a1290 0x7f91680a37a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.497+0000 7f9189800700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184100ac0 0x7f91841a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.497+0000 7f9189800700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184102580 0x7f91841a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.497+0000 7f9189800700 1 --2- 172.21.15.131:0/1088463993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9184101be0 0x7f91841a3200 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:19.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.497+0000 7f9189800700 1 -- 172.21.15.131:0/1088463993 >> 172.21.15.131:0/1088463993 conn(0x7f91840fc0d0 msgr2=0x7f918410de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:19.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.497+0000 7f9189800700 1 -- 172.21.15.131:0/1088463993 shutdown_connections 2024-05-10T05:41:19.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:19.501+0000 7f9189800700 1 -- 172.21.15.131:0/1088463993 wait complete. 2024-05-10T05:41:19.559 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:41:20.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.138+0000 7fd18f000700 1 -- 172.21.15.131:0/2024989908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1881036a0 msgr2=0x7fd188109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:20.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.138+0000 7fd18f000700 1 --2- 172.21.15.131:0/2024989908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1881036a0 0x7fd188109f50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd180009fc0 tx=0x7fd18000b330 comp rx=0 tx=0).stop 2024-05-10T05:41:20.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.138+0000 7fd18f000700 1 -- 172.21.15.131:0/2024989908 shutdown_connections 2024-05-10T05:41:20.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.138+0000 7fd18f000700 1 --2- 172.21.15.131:0/2024989908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1881036a0 0x7fd188109f50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.138+0000 7fd18f000700 1 --2- 172.21.15.131:0/2024989908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd188101be0 0x7fd188101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.138+0000 7fd18f000700 1 --2- 172.21.15.131:0/2024989908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd188102d00 0x7fd188103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.138+0000 7fd18f000700 1 -- 172.21.15.131:0/2024989908 >> 172.21.15.131:0/2024989908 conn(0x7fd1880fd1d0 msgr2=0x7fd1880ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:20.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.138+0000 7fd18f000700 1 -- 172.21.15.131:0/2024989908 shutdown_connections 2024-05-10T05:41:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.138+0000 7fd18f000700 1 -- 172.21.15.131:0/2024989908 wait complete. 2024-05-10T05:41:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.140+0000 7fd18f000700 1 Processor -- start 2024-05-10T05:41:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.140+0000 7fd18f000700 1 -- start start 2024-05-10T05:41:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.141+0000 7fd18f000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd188101be0 0x7fd188074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.141+0000 7fd18f000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd188102d00 0x7fd188073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.141+0000 7fd18f000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1881036a0 0x7fd188073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.141+0000 7fd18f000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd188074180 con 0x7fd188102d00 2024-05-10T05:41:20.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.141+0000 7fd18f000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880742f0 con 0x7fd188101be0 2024-05-10T05:41:20.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.141+0000 7fd18f000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1881a20b0 con 0x7fd1881036a0 2024-05-10T05:41:20.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.141+0000 7fd187e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd188101be0 0x7fd188074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.142+0000 7fd187e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd188101be0 0x7fd188074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43126/0 (socket says 172.21.15.131:43126) 2024-05-10T05:41:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.142+0000 7fd187e00700 1 -- 172.21.15.131:0/2321803806 learned_addr learned my addr 172.21.15.131:0/2321803806 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.144+0000 7fd187400700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd188102d00 0x7fd188073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.145+0000 7fd18d000700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1881036a0 0x7fd188073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.145+0000 7fd187400700 1 -- 172.21.15.131:0/2321803806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1881036a0 msgr2=0x7fd188073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.145+0000 7fd187400700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1881036a0 0x7fd188073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.145+0000 7fd187400700 1 -- 172.21.15.131:0/2321803806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd188101be0 msgr2=0x7fd188074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.145+0000 7fd187400700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd188101be0 0x7fd188074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.145+0000 7fd187400700 1 -- 172.21.15.131:0/2321803806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd180009c40 con 0x7fd188102d00 2024-05-10T05:41:20.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.146+0000 7fd187400700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd188102d00 0x7fd188073560 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd17400e9c0 tx=0x7fd17400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:20.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.147+0000 7fd184c00700 1 -- 172.21.15.131:0/2321803806 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17400cbd0 con 0x7fd188102d00 2024-05-10T05:41:20.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.147+0000 7fd184c00700 1 -- 172.21.15.131:0/2321803806 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd174009e50 con 0x7fd188102d00 2024-05-10T05:41:20.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.147+0000 7fd18f000700 1 -- 172.21.15.131:0/2321803806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1881a2310 con 0x7fd188102d00 2024-05-10T05:41:20.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.148+0000 7fd18f000700 1 -- 172.21.15.131:0/2321803806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1881a2840 con 0x7fd188102d00 2024-05-10T05:41:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.148+0000 7fd184c00700 1 -- 172.21.15.131:0/2321803806 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174018680 con 0x7fd188102d00 2024-05-10T05:41:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.160+0000 7fd184c00700 1 -- 172.21.15.131:0/2321803806 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd17400cd30 con 0x7fd188102d00 2024-05-10T05:41:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.162+0000 7fd184c00700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd16c0a1240 0x7fd16c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.163+0000 7fd187e00700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd16c0a1240 0x7fd16c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.164+0000 7fd1a6fecb80 1 Processor -- start 2024-05-10T05:41:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.164+0000 7fd184c00700 1 -- 172.21.15.131:0/2321803806 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd174014070 con 0x7fd188102d00 2024-05-10T05:41:20.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.165+0000 7fd1a6fecb80 1 -- start start 2024-05-10T05:41:20.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.165+0000 7fd187e00700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd16c0a1240 0x7fd16c0a3750 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd178009c50 tx=0x7fd178009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:20.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.166+0000 7fd1a6fecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a11300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.167+0000 7fd187e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a11300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.167+0000 7fd187e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a11300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46570/0 (socket says 172.21.15.131:46570) 2024-05-10T05:41:20.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.167+0000 7fd187e00700 1 -- 172.21.15.131:0/3963601540 learned_addr learned my addr 172.21.15.131:0/3963601540 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.168+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3963601540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a12190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.168+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3963601540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a126d0 0x560b72a18fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.168+0000 7fd187400700 1 --2- 172.21.15.131:0/3963601540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a12190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.168+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3963601540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7292f150 con 0x560b72a10eb0 2024-05-10T05:41:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.169+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3963601540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7292cb10 con 0x560b72a11d80 2024-05-10T05:41:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.169+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3963601540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b72928400 con 0x560b72a126d0 2024-05-10T05:41:20.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.170+0000 7fd187e00700 1 -- 172.21.15.131:0/3963601540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a126d0 msgr2=0x560b72a18fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:41:20.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.170+0000 7fd187e00700 1 --2- 172.21.15.131:0/3963601540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a126d0 0x560b72a18fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.170+0000 7fd187e00700 1 -- 172.21.15.131:0/3963601540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 msgr2=0x560b72a12190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:20.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.170+0000 7fd187e00700 1 --2- 172.21.15.131:0/3963601540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a12190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.170+0000 7fd187e00700 1 -- 172.21.15.131:0/3963601540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b727a93b0 con 0x560b72a10eb0 2024-05-10T05:41:20.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.171+0000 7fd187e00700 1 --2- 172.21.15.131:0/3963601540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a11300 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd178011f80 tx=0x7fd178014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2aab0647d9e359a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:20.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.172+0000 7fd17d600700 1 -- 172.21.15.131:0/3963601540 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178012dc0 con 0x560b72a10eb0 2024-05-10T05:41:20.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.173+0000 7fd17d600700 1 -- 172.21.15.131:0/3963601540 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd178016070 con 0x560b72a10eb0 2024-05-10T05:41:20.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.173+0000 7fd17d600700 1 -- 172.21.15.131:0/3963601540 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17801d610 con 0x560b72a10eb0 2024-05-10T05:41:20.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.174+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3963601540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 msgr2=0x560b72a11300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:20.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.174+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3963601540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a11300 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd178011f80 tx=0x7fd178014000 comp rx=0 tx=0).stop 2024-05-10T05:41:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.175+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3963601540 shutdown_connections 2024-05-10T05:41:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.175+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3963601540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a11300 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.175+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3963601540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a126d0 0x560b72a18fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.175+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3963601540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a12190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.175+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3963601540 >> 172.21.15.131:0/3963601540 conn(0x560b7296aa60 msgr2=0x560b72a02fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.178+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3963601540 shutdown_connections 2024-05-10T05:41:20.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.178+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3963601540 wait complete. 2024-05-10T05:41:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.182+0000 7fd1a6fecb80 1 -- start start 2024-05-10T05:41:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.183+0000 7fd1a6fecb80 1 Processor -- start 2024-05-10T05:41:20.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.186+0000 7fd1a6fecb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a1af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.186+0000 7fd1a6fecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a1b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.186+0000 7fd1a6fecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 0x560b72a22420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.186+0000 7fd1a6fecb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b728c4560 con 0x560b72a126d0 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.186+0000 7fd1a6fecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b728c4200 con 0x560b72a11d80 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.186+0000 7fd1a6fecb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b727ab6a0 con 0x560b72a10eb0 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.186+0000 7fd1a6fecb80 10 client.0 _ll_register_callbacks cb 0x560b728cdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.188+0000 7fd18d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 0x560b72a22420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.188+0000 7fd18d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 0x560b72a22420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46580/0 (socket says 172.21.15.131:46580) 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.188+0000 7fd18d000700 1 -- 172.21.15.131:0/3882607333 learned_addr learned my addr 172.21.15.131:0/3882607333 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.188+0000 7fd187e00700 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a1af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.189+0000 7fd187400700 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a1b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.189+0000 7fd18d000700 1 -- 172.21.15.131:0/3882607333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 msgr2=0x560b72a1af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.189+0000 7fd18d000700 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a1af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.189+0000 7fd18d000700 1 -- 172.21.15.131:0/3882607333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 msgr2=0x560b72a1b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.189+0000 7fd18d000700 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a1b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.189+0000 7fd18d000700 1 -- 172.21.15.131:0/3882607333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178011c60 con 0x560b72a126d0 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.189+0000 7fd187e00700 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a1af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.189+0000 7fd187400700 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a1b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.193+0000 7fd18d000700 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 0x560b72a22420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd1800094f0 tx=0x7fd1800097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ab4ba4373a156a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.195+0000 7fd172000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.195+0000 7fd172000700 1 -- 172.21.15.131:0/3882607333 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000e040 con 0x560b72a126d0 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.195+0000 7fd172000700 1 -- 172.21.15.131:0/3882607333 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd180004c50 con 0x560b72a126d0 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.195+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3882607333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b72881270 con 0x560b72a126d0 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.196+0000 7fd172000700 1 -- 172.21.15.131:0/3882607333 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd180004e10 con 0x560b72a126d0 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.196+0000 7fd1a6fecb80 10 client.16542 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.196+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3882607333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560b72755b60 con 0x560b72a126d0 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.197+0000 7fd172000700 1 -- 172.21.15.131:0/3882607333 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560b72881270 con 0x560b72a126d0 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.197+0000 7fd172000700 1 -- 172.21.15.131:0/3882607333 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd180019040 con 0x560b72a126d0 2024-05-10T05:41:20.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.197+0000 7fd1a6fecb80 10 client.16542 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.197+0000 7fd1a6fecb80 10 client.16542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.197+0000 7fd1a6fecb80 20 client.16542 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.197+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x560b72a48b90 0x560b72a4b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.197+0000 7fd1a6fecb80 4 client.16542 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.197+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3882607333 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b729d7780 con 0x560b72a48b90 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.202+0000 7fd187e00700 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x560b72a48b90 0x560b72a4b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.203+0000 7fd172000700 10 client.16542 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.203+0000 7fd187e00700 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x560b72a48b90 0x560b72a4b060 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7458fde0aba503fb server_cookie=49e2029f82adb2fd in_seq=0 out_seq=0 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.207+0000 7fd172000700 1 -- 172.21.15.131:0/3882607333 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b729d7780 con 0x560b72a48b90 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.207+0000 7fd172000700 10 client.16542 handle_command_reply: tid=0 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.207+0000 7fd1a6fecb80 1 client.16542 shutdown 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.207+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3882607333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 msgr2=0x560b72a22420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.207+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 0x560b72a22420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd1800094f0 tx=0x7fd1800097e0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.208+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3882607333 shutdown_connections 2024-05-10T05:41:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.208+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x560b72a48b90 0x560b72a4b060 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.208+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 0x560b72a22420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.208+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a1af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.208+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/3882607333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a1b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.208+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3882607333 >> 172.21.15.131:0/3882607333 conn(0x560b7296aa60 msgr2=0x560b72a02fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:20.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.208+0000 7fd172000700 0 client.16542 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:41:20.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.208+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3882607333 shutdown_connections 2024-05-10T05:41:20.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.208+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/3882607333 wait complete. 2024-05-10T05:41:20.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.209+0000 7fd1a6fecb80 20 client.16542 trim_cache size 0 max 16384 2024-05-10T05:41:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.239+0000 7fd1a6fecb80 1 Processor -- start 2024-05-10T05:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.239+0000 7fd1a6fecb80 1 -- start start 2024-05-10T05:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.239+0000 7fd1a6fecb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a4aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.239+0000 7fd1a6fecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a223e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.240+0000 7fd187e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a4aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.240+0000 7fd187e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a4aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53056/0 (socket says 172.21.15.131:53056) 2024-05-10T05:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.240+0000 7fd187e00700 1 -- 172.21.15.131:0/1639434313 learned_addr learned my addr 172.21.15.131:0/1639434313 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.240+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 0x560b72a4b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.240+0000 7fd187400700 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.241+0000 7fd18d000700 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 0x560b72a4b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.241+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/1639434313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b72a04770 con 0x560b72a126d0 2024-05-10T05:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.241+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/1639434313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b72a1bff0 con 0x560b72a11d80 2024-05-10T05:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.241+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/1639434313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b72a18e80 con 0x560b72a10eb0 2024-05-10T05:41:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.243+0000 7fd187400700 1 -- 172.21.15.131:0/1639434313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 msgr2=0x560b72a4aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.243+0000 7fd187400700 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a4aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.243+0000 7fd187400700 1 -- 172.21.15.131:0/1639434313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 msgr2=0x560b72a4b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.243+0000 7fd187400700 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 0x560b72a4b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.244+0000 7fd187400700 1 -- 172.21.15.131:0/1639434313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b728ee560 con 0x560b72a11d80 2024-05-10T05:41:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.244+0000 7fd187e00700 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a4aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:41:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.244+0000 7fd187400700 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a223e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd1740949c0 tx=0x7fd17409eb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=427f90327faad0e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.244+0000 7fd17e000700 1 -- 172.21.15.131:0/1639434313 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1740999b0 con 0x560b72a11d80 2024-05-10T05:41:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.244+0000 7fd17e000700 1 -- 172.21.15.131:0/1639434313 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd17409a550 con 0x560b72a11d80 2024-05-10T05:41:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.244+0000 7fd17e000700 1 -- 172.21.15.131:0/1639434313 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174099340 con 0x560b72a11d80 2024-05-10T05:41:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.246+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/1639434313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 msgr2=0x560b72a223e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.246+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a223e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd1740949c0 tx=0x7fd17409eb30 comp rx=0 tx=0).stop 2024-05-10T05:41:20.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.246+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/1639434313 shutdown_connections 2024-05-10T05:41:20.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.246+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a126d0 0x560b72a4b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.246+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a10eb0 0x560b72a4aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.246+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/1639434313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a223e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.246+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/1639434313 >> 172.21.15.131:0/1639434313 conn(0x560b7296aa60 msgr2=0x560b72a15b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.246+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/1639434313 shutdown_connections 2024-05-10T05:41:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.247+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/1639434313 wait complete. 2024-05-10T05:41:20.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.248+0000 7fd1a6fecb80 1 -- start start 2024-05-10T05:41:20.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.250+0000 7fd1a6fecb80 1 Processor -- start 2024-05-10T05:41:20.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.252+0000 7fd1a6fecb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a1fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.253+0000 7fd1a6fecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a1ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.253+0000 7fd1a6fecb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a126d0 0x560b72a20ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.253+0000 7fd1a6fecb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b72a21430 con 0x560b72a10eb0 2024-05-10T05:41:20.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.253+0000 7fd1a6fecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b72a215a0 con 0x560b72a11d80 2024-05-10T05:41:20.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.253+0000 7fd1a6fecb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b72a21760 con 0x560b72a126d0 2024-05-10T05:41:20.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.254+0000 7fd187e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a1fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.254+0000 7fd187e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a1fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46604/0 (socket says 172.21.15.131:46604) 2024-05-10T05:41:20.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.254+0000 7fd1a6fecb80 10 client.0 _ll_register_callbacks cb 0x560b728cdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:20.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.254+0000 7fd187e00700 1 -- 172.21.15.131:0/852521426 learned_addr learned my addr 172.21.15.131:0/852521426 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:20.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.254+0000 7fd187400700 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a1ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.255+0000 7fd18d000700 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a126d0 0x560b72a20ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.255+0000 7fd187e00700 1 -- 172.21.15.131:0/852521426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a126d0 msgr2=0x560b72a20ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:20.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.255+0000 7fd187e00700 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a126d0 0x560b72a20ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.255+0000 7fd187e00700 1 -- 172.21.15.131:0/852521426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 msgr2=0x560b72a1ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:20.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.255+0000 7fd187e00700 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a1ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.255+0000 7fd187e00700 1 -- 172.21.15.131:0/852521426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1881a2840 con 0x560b72a10eb0 2024-05-10T05:41:20.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.256+0000 7fd187e00700 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a1fa80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd178009830 tx=0x7fd178013de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8559e86f8f4dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:20.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.256+0000 7fd172000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:41:20.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.256+0000 7fd187400700 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a1ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:20.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.257+0000 7fd172000700 1 -- 172.21.15.131:0/852521426 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17802d040 con 0x560b72a10eb0 2024-05-10T05:41:20.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.257+0000 7fd172000700 1 -- 172.21.15.131:0/852521426 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd178018070 con 0x560b72a10eb0 2024-05-10T05:41:20.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.257+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/852521426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd178016070 con 0x560b72a10eb0 2024-05-10T05:41:20.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.257+0000 7fd172000700 1 -- 172.21.15.131:0/852521426 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1780323a0 con 0x560b72a10eb0 2024-05-10T05:41:20.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.257+0000 7fd1a6fecb80 10 client.16548 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:20.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.257+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/852521426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560b727adcd0 con 0x560b72a10eb0 2024-05-10T05:41:20.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.258+0000 7fd172000700 1 -- 172.21.15.131:0/852521426 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd178016070 con 0x560b72a10eb0 2024-05-10T05:41:20.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.258+0000 7fd172000700 1 -- 172.21.15.131:0/852521426 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd178004830 con 0x560b72a10eb0 2024-05-10T05:41:20.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.258+0000 7fd1a6fecb80 10 client.16548 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:20.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.258+0000 7fd1a6fecb80 10 client.16548 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:41:20.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.258+0000 7fd1a6fecb80 20 client.16548 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:20.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.258+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x560b72a9f7a0 0x560b72aa1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:20.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.258+0000 7fd1a6fecb80 4 client.16548 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:41:20.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.258+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/852521426 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560b72920ec0 con 0x560b72a9f7a0 2024-05-10T05:41:20.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.259+0000 7fd187400700 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x560b72a9f7a0 0x560b72aa1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:20.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.259+0000 7fd187400700 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x560b72a9f7a0 0x560b72aa1bc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8acb1ad00d708a09 server_cookie=639ec3c08a11c593 in_seq=0 out_seq=0 2024-05-10T05:41:20.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.260+0000 7fd172000700 10 client.16548 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:41:20.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.260+0000 7fd172000700 1 -- 172.21.15.131:0/852521426 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560b72920ec0 con 0x560b72a9f7a0 2024-05-10T05:41:20.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.260+0000 7fd172000700 10 client.16548 handle_command_reply: tid=0 2024-05-10T05:41:20.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.260+0000 7fd1a6fecb80 1 client.16548 shutdown 2024-05-10T05:41:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.261+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/852521426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 msgr2=0x560b72a1fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.261+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a1fa80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd178009830 tx=0x7fd178013de0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.261+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/852521426 shutdown_connections 2024-05-10T05:41:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.261+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x560b72a9f7a0 0x560b72aa1bc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.261+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560b72a10eb0 0x560b72a1fa80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.261+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b72a126d0 0x560b72a20ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.261+0000 7fd1a6fecb80 1 --2- 172.21.15.131:0/852521426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b72a11d80 0x560b72a1ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.261+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/852521426 >> 172.21.15.131:0/852521426 conn(0x560b7296aa60 msgr2=0x560b72a15b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.261+0000 7fd172000700 0 client.16548 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:41:20.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.262+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/852521426 shutdown_connections 2024-05-10T05:41:20.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.262+0000 7fd1a6fecb80 1 -- 172.21.15.131:0/852521426 wait complete. 2024-05-10T05:41:20.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.262+0000 7fd1a6fecb80 20 client.16548 trim_cache size 0 max 16384 2024-05-10T05:41:20.263 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:41:20.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.267+0000 7fd18f000700 1 -- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd16c0a1240 msgr2=0x7fd16c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:20.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.267+0000 7fd18f000700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd16c0a1240 0x7fd16c0a3750 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd178009c50 tx=0x7fd178009260 comp rx=0 tx=0).stop 2024-05-10T05:41:20.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.267+0000 7fd18f000700 1 -- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd188102d00 msgr2=0x7fd188073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:20.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.267+0000 7fd18f000700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd188102d00 0x7fd188073560 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd17400e9c0 tx=0x7fd17400ed80 comp rx=0 tx=0).stop 2024-05-10T05:41:20.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.267+0000 7fd18f000700 1 -- 172.21.15.131:0/2321803806 shutdown_connections 2024-05-10T05:41:20.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.267+0000 7fd18f000700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd16c0a1240 0x7fd16c0a3750 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.267+0000 7fd18f000700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1881036a0 0x7fd188073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.267+0000 7fd18f000700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd188101be0 0x7fd188074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.267+0000 7fd18f000700 1 --2- 172.21.15.131:0/2321803806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd188102d00 0x7fd188073560 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:20.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.267+0000 7fd18f000700 1 -- 172.21.15.131:0/2321803806 >> 172.21.15.131:0/2321803806 conn(0x7fd1880fd1d0 msgr2=0x7fd188105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:20.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.268+0000 7fd18f000700 1 -- 172.21.15.131:0/2321803806 shutdown_connections 2024-05-10T05:41:20.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:20.268+0000 7fd18f000700 1 -- 172.21.15.131:0/2321803806 wait complete. 2024-05-10T05:41:20.328 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:41:20.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:20 smithi131 ceph-mon[37264]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 251 MiB/s wr, 76 op/s 2024-05-10T05:41:20.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:20 smithi142 ceph-mon[77667]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 251 MiB/s wr, 76 op/s 2024-05-10T05:41:20.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:20 smithi157 ceph-mon[78427]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 251 MiB/s wr, 76 op/s 2024-05-10T05:41:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.247+0000 7f61c4c00700 1 -- 172.21.15.131:0/2927822155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c010a9a0 msgr2=0x7f61c010adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.247+0000 7f61c4c00700 1 --2- 172.21.15.131:0/2927822155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c010a9a0 0x7f61c010adb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009f90 tx=0x7f61b800b330 comp rx=0 tx=0).stop 2024-05-10T05:41:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.248+0000 7f61c4c00700 1 -- 172.21.15.131:0/2927822155 shutdown_connections 2024-05-10T05:41:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.248+0000 7f61c4c00700 1 --2- 172.21.15.131:0/2927822155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c01121e0 0x7f61c0114600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.248+0000 7f61c4c00700 1 --2- 172.21.15.131:0/2927822155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c010b750 0x7f61c010a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.248+0000 7f61c4c00700 1 --2- 172.21.15.131:0/2927822155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c010a9a0 0x7f61c010adb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.248+0000 7f61c4c00700 1 -- 172.21.15.131:0/2927822155 >> 172.21.15.131:0/2927822155 conn(0x7f61c006e800 msgr2=0x7f61c0070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:21.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.249+0000 7f61c4c00700 1 -- 172.21.15.131:0/2927822155 shutdown_connections 2024-05-10T05:41:21.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.249+0000 7f61c4c00700 1 -- 172.21.15.131:0/2927822155 wait complete. 2024-05-10T05:41:21.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.251+0000 7f61c4c00700 1 Processor -- start 2024-05-10T05:41:21.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.251+0000 7f61c4c00700 1 -- start start 2024-05-10T05:41:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.253+0000 7f61c4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c010a9a0 0x7f61c01a2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.253+0000 7f61c4c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c010b750 0x7f61c01a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.254+0000 7f61c4c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c01121e0 0x7f61c01a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.253+0000 7f61bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c010a9a0 0x7f61c01a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.254+0000 7f61bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c010a9a0 0x7f61c01a2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43182/0 (socket says 172.21.15.131:43182) 2024-05-10T05:41:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.255+0000 7f61c4c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c01aa220 con 0x7f61c01121e0 2024-05-10T05:41:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.255+0000 7f61c4c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c01aa360 con 0x7f61c010a9a0 2024-05-10T05:41:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.255+0000 7f61bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c01121e0 0x7f61c01a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.255+0000 7f61bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c01121e0 0x7f61c01a9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46622/0 (socket says 172.21.15.131:46622) 2024-05-10T05:41:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.255+0000 7f61bfe00700 1 -- 172.21.15.131:0/959652408 learned_addr learned my addr 172.21.15.131:0/959652408 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.254+0000 7f61bea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c010b750 0x7f61c01a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.256+0000 7f61c4c00700 1 -- 172.21.15.131:0/959652408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c01aa4f0 con 0x7f61c010b750 2024-05-10T05:41:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.257+0000 7f61bfe00700 1 -- 172.21.15.131:0/959652408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c010b750 msgr2=0x7f61c01a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.257+0000 7f61bfe00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c010b750 0x7f61c01a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.257+0000 7f61bfe00700 1 -- 172.21.15.131:0/959652408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c010a9a0 msgr2=0x7f61c01a2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.257+0000 7f61bfe00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c010a9a0 0x7f61c01a2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.257+0000 7f61bfe00700 1 -- 172.21.15.131:0/959652408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b8009c40 con 0x7f61c01121e0 2024-05-10T05:41:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.258+0000 7f61bea00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c010b750 0x7f61c01a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.258+0000 7f61bfe00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c01121e0 0x7f61c01a9c50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e9a0 tx=0x7f61b000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.260+0000 7f61b7e00700 1 -- 172.21.15.131:0/959652408 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b000cbd0 con 0x7f61c01121e0 2024-05-10T05:41:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.260+0000 7f61b7e00700 1 -- 172.21.15.131:0/959652408 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61b000ee80 con 0x7f61c01121e0 2024-05-10T05:41:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.260+0000 7f61c4c00700 1 -- 172.21.15.131:0/959652408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c010be20 con 0x7f61c01121e0 2024-05-10T05:41:21.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.260+0000 7f61b7e00700 1 -- 172.21.15.131:0/959652408 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0005650 con 0x7f61c01121e0 2024-05-10T05:41:21.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.261+0000 7f61c4c00700 1 -- 172.21.15.131:0/959652408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c010c2e0 con 0x7f61c01121e0 2024-05-10T05:41:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.268+0000 7f61b7e00700 1 -- 172.21.15.131:0/959652408 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f61b0018430 con 0x7f61c01121e0 2024-05-10T05:41:21.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.269+0000 7f61de03ab80 1 Processor -- start 2024-05-10T05:41:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.270+0000 7f61de03ab80 1 -- start start 2024-05-10T05:41:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.270+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 0x55bcfdbc3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.270+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbc4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.270+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbcb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.270+0000 7f61de03ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfdaea150 con 0x55bcfdbc47a0 2024-05-10T05:41:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.270+0000 7f61de03ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfdae7b10 con 0x55bcfdbc50f0 2024-05-10T05:41:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.270+0000 7f61de03ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfdae3400 con 0x55bcfdbc3530 2024-05-10T05:41:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.271+0000 7f61bea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbcb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.271+0000 7f61bea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbcb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43184/0 (socket says 172.21.15.131:43184) 2024-05-10T05:41:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.271+0000 7f61bea00700 1 -- 172.21.15.131:0/872518660 learned_addr learned my addr 172.21.15.131:0/872518660 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.271+0000 7f61b7e00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f61a40a1320 0x7f61a40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.271+0000 7f61bea00700 1 --2- 172.21.15.131:0/872518660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 0x55bcfdbc3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.271+0000 7f61bf400700 1 --2- 172.21.15.131:0/872518660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbc4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.272+0000 7f61bfe00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f61a40a1320 0x7f61a40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.272+0000 7f61bea00700 1 -- 172.21.15.131:0/872518660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 msgr2=0x55bcfdbc3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:21.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.272+0000 7f61bea00700 1 --2- 172.21.15.131:0/872518660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 0x55bcfdbc3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.272+0000 7f61bea00700 1 -- 172.21.15.131:0/872518660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 msgr2=0x55bcfdbc4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.272+0000 7f61bea00700 1 --2- 172.21.15.131:0/872518660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbc4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.272+0000 7f61bea00700 1 -- 172.21.15.131:0/872518660 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcfd9643b0 con 0x55bcfdbc50f0 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.272+0000 7f61bf400700 1 --2- 172.21.15.131:0/872518660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbc4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.272+0000 7f61b7e00700 1 -- 172.21.15.131:0/959652408 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f61b0014070 con 0x7f61c01121e0 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.273+0000 7f61bea00700 1 --2- 172.21.15.131:0/872518660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbcb9f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f61b8007b40 tx=0x7f61b8009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ff261a04405bc51 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.273+0000 7f61abe00700 1 -- 172.21.15.131:0/872518660 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8023070 con 0x55bcfdbc50f0 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.273+0000 7f61abe00700 1 -- 172.21.15.131:0/872518660 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61b8007d20 con 0x55bcfdbc50f0 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.273+0000 7f61abe00700 1 -- 172.21.15.131:0/872518660 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b800e040 con 0x55bcfdbc50f0 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.274+0000 7f61de03ab80 1 -- 172.21.15.131:0/872518660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 msgr2=0x55bcfdbcb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.274+0000 7f61bfe00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f61a40a1320 0x7f61a40a3830 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e970 tx=0x7f61b0009530 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.274+0000 7f61de03ab80 1 --2- 172.21.15.131:0/872518660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbcb9f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f61b8007b40 tx=0x7f61b8009490 comp rx=0 tx=0).stop 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.275+0000 7f61de03ab80 1 -- 172.21.15.131:0/872518660 shutdown_connections 2024-05-10T05:41:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.275+0000 7f61de03ab80 1 --2- 172.21.15.131:0/872518660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbcb9f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.275+0000 7f61de03ab80 1 --2- 172.21.15.131:0/872518660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 0x55bcfdbc3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.275+0000 7f61de03ab80 1 --2- 172.21.15.131:0/872518660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbc4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.275+0000 7f61de03ab80 1 -- 172.21.15.131:0/872518660 >> 172.21.15.131:0/872518660 conn(0x55bcfdb25a60 msgr2=0x55bcfdbbe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:21.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.275+0000 7f61de03ab80 1 -- 172.21.15.131:0/872518660 shutdown_connections 2024-05-10T05:41:21.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.275+0000 7f61de03ab80 1 -- 172.21.15.131:0/872518660 wait complete. 2024-05-10T05:41:21.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.276+0000 7f61de03ab80 1 -- start start 2024-05-10T05:41:21.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.276+0000 7f61de03ab80 1 Processor -- start 2024-05-10T05:41:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.277+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc3530 0x55bcfdbcd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.278+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbcdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.278+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.278+0000 7f61de03ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfda7f560 con 0x55bcfdbc3530 2024-05-10T05:41:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.278+0000 7f61de03ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfda7f200 con 0x55bcfdbc47a0 2024-05-10T05:41:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.278+0000 7f61de03ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfd9666a0 con 0x55bcfdbc50f0 2024-05-10T05:41:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.278+0000 7f61de03ab80 10 client.0 _ll_register_callbacks cb 0x55bcfda88b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.278+0000 7f61bea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc3530 0x55bcfdbcd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.278+0000 7f61bea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc3530 0x55bcfdbcd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46632/0 (socket says 172.21.15.131:46632) 2024-05-10T05:41:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.278+0000 7f61bea00700 1 -- 172.21.15.131:0/3504702208 learned_addr learned my addr 172.21.15.131:0/3504702208 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.278+0000 7f61bf400700 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbcdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.279+0000 7f61bf400700 1 -- 172.21.15.131:0/3504702208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc50f0 msgr2=0x55bcfdbd4cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:21.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.279+0000 7f61bf400700 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.279+0000 7f61bf400700 1 -- 172.21.15.131:0/3504702208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc3530 msgr2=0x55bcfdbcd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:21.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.279+0000 7f61bf400700 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc3530 0x55bcfdbcd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.279+0000 7f61bf400700 1 -- 172.21.15.131:0/3504702208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac009830 con 0x55bcfdbc47a0 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.279+0000 7f61bea00700 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc3530 0x55bcfdbcd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.280+0000 7f61bf400700 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbcdd90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009fd0 tx=0x7f61ac012af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c6a683c6a19d842 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.280+0000 7f61a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.280+0000 7f61a8c00700 1 -- 172.21.15.131:0/3504702208 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac017070 con 0x55bcfdbc47a0 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.280+0000 7f61a8c00700 1 -- 172.21.15.131:0/3504702208 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f61ac004320 con 0x55bcfdbc47a0 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.280+0000 7f61de03ab80 1 -- 172.21.15.131:0/3504702208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcfda3c270 con 0x55bcfdbc47a0 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.281+0000 7f61a8c00700 1 -- 172.21.15.131:0/3504702208 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61ac0044e0 con 0x55bcfdbc47a0 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.281+0000 7f61de03ab80 10 client.26798 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.281+0000 7f61de03ab80 1 -- 172.21.15.131:0/3504702208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f61c01151d0 con 0x55bcfdbc47a0 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.281+0000 7f61a8c00700 1 -- 172.21.15.131:0/3504702208 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bcfda3c270 con 0x55bcfdbc47a0 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.281+0000 7f61a8c00700 1 -- 172.21.15.131:0/3504702208 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f61ac0119e0 con 0x55bcfdbc47a0 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.281+0000 7f61de03ab80 10 client.26798 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.281+0000 7f61de03ab80 10 client.26798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.281+0000 7f61de03ab80 20 client.26798 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.282+0000 7f61de03ab80 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bcfdbd95b0 0x55bcfdbdba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.282+0000 7f61de03ab80 4 client.26798 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:21.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.282+0000 7f61de03ab80 1 -- 172.21.15.131:0/3504702208 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcfdb92780 con 0x55bcfdbd95b0 2024-05-10T05:41:21.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.282+0000 7f61bea00700 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bcfdbd95b0 0x55bcfdbdba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.283+0000 7f61bea00700 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bcfdbd95b0 0x55bcfdbdba00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9441b1a078e6eac server_cookie=e7744273659f8c3 in_seq=0 out_seq=0 2024-05-10T05:41:21.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.283+0000 7f61a8c00700 10 client.26798 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:41:21.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.284+0000 7f61a8c00700 1 -- 172.21.15.131:0/3504702208 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bcfdb92780 con 0x55bcfdbd95b0 2024-05-10T05:41:21.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.284+0000 7f61a8c00700 10 client.26798 handle_command_reply: tid=0 2024-05-10T05:41:21.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.284+0000 7f61de03ab80 1 client.26798 shutdown 2024-05-10T05:41:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.285+0000 7f61de03ab80 1 -- 172.21.15.131:0/3504702208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc47a0 msgr2=0x55bcfdbcdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.285+0000 7f61de03ab80 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbcdd90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009fd0 tx=0x7f61ac012af0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.285+0000 7f61de03ab80 1 -- 172.21.15.131:0/3504702208 shutdown_connections 2024-05-10T05:41:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.285+0000 7f61de03ab80 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bcfdbd95b0 0x55bcfdbdba00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.285+0000 7f61a8c00700 0 client.26798 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:41:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.285+0000 7f61de03ab80 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc3530 0x55bcfdbcd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.285+0000 7f61de03ab80 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.285+0000 7f61de03ab80 1 --2- 172.21.15.131:0/3504702208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbcdd90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.285+0000 7f61de03ab80 1 -- 172.21.15.131:0/3504702208 >> 172.21.15.131:0/3504702208 conn(0x55bcfdb25a60 msgr2=0x55bcfdbc2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:21.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.286+0000 7f61de03ab80 1 -- 172.21.15.131:0/3504702208 shutdown_connections 2024-05-10T05:41:21.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.286+0000 7f61de03ab80 1 -- 172.21.15.131:0/3504702208 wait complete. 2024-05-10T05:41:21.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.288+0000 7f61de03ab80 20 client.26798 trim_cache size 0 max 16384 2024-05-10T05:41:21.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.331+0000 7f61de03ab80 1 Processor -- start 2024-05-10T05:41:21.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.333+0000 7f61de03ab80 1 -- start start 2024-05-10T05:41:21.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.333+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 0x55bcfdbbdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.333+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbd4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.333+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.333+0000 7f61de03ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfdbbfba0 con 0x55bcfdbc47a0 2024-05-10T05:41:21.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.333+0000 7f61de03ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfdbb82f0 con 0x55bcfdbc50f0 2024-05-10T05:41:21.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.333+0000 7f61de03ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfdbcb8f0 con 0x55bcfdbc3530 2024-05-10T05:41:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.336+0000 7f61bea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 0x55bcfdbbdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.336+0000 7f61bea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 0x55bcfdbbdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53122/0 (socket says 172.21.15.131:53122) 2024-05-10T05:41:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.337+0000 7f61bea00700 1 -- 172.21.15.131:0/1684157843 learned_addr learned my addr 172.21.15.131:0/1684157843 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.337+0000 7f61bea00700 1 --2- 172.21.15.131:0/1684157843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.337+0000 7f61bea00700 1 -- 172.21.15.131:0/1684157843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 msgr2=0x55bcfdbd52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.337+0000 7f61bea00700 1 --2- 172.21.15.131:0/1684157843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.337+0000 7f61bea00700 1 -- 172.21.15.131:0/1684157843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 msgr2=0x55bcfdbd4cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.337+0000 7f61bea00700 1 --2- 172.21.15.131:0/1684157843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbd4cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.337+0000 7f61bea00700 1 -- 172.21.15.131:0/1684157843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcfd910b60 con 0x55bcfdbc3530 2024-05-10T05:41:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.338+0000 7f61bea00700 1 --2- 172.21.15.131:0/1684157843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 0x55bcfdbbdbb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f61b8007ae0 tx=0x7f61b8009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347a52c1c3b5eb35 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:21.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.343+0000 7f61b5200700 1 -- 172.21.15.131:0/1684157843 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8023040 con 0x55bcfdbc3530 2024-05-10T05:41:21.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.343+0000 7f61b5200700 1 -- 172.21.15.131:0/1684157843 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61b800be90 con 0x55bcfdbc3530 2024-05-10T05:41:21.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.343+0000 7f61b5200700 1 -- 172.21.15.131:0/1684157843 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b800e040 con 0x55bcfdbc3530 2024-05-10T05:41:21.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.345+0000 7f61de03ab80 1 -- 172.21.15.131:0/1684157843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 msgr2=0x55bcfdbbdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:21.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.345+0000 7f61de03ab80 1 --2- 172.21.15.131:0/1684157843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 0x55bcfdbbdbb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f61b8007ae0 tx=0x7f61b8009ac0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.348+0000 7f61de03ab80 1 -- 172.21.15.131:0/1684157843 shutdown_connections 2024-05-10T05:41:21.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.348+0000 7f61de03ab80 1 --2- 172.21.15.131:0/1684157843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.348+0000 7f61de03ab80 1 --2- 172.21.15.131:0/1684157843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc3530 0x55bcfdbbdbb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.348+0000 7f61de03ab80 1 --2- 172.21.15.131:0/1684157843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbd4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.348+0000 7f61de03ab80 1 -- 172.21.15.131:0/1684157843 >> 172.21.15.131:0/1684157843 conn(0x55bcfdb25a60 msgr2=0x55bcfdbcca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:21.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.348+0000 7f61de03ab80 1 -- 172.21.15.131:0/1684157843 shutdown_connections 2024-05-10T05:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.348+0000 7f61de03ab80 1 -- 172.21.15.131:0/1684157843 wait complete. 2024-05-10T05:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.349+0000 7f61de03ab80 1 -- start start 2024-05-10T05:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.349+0000 7f61de03ab80 1 Processor -- start 2024-05-10T05:41:21.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.352+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc3530 0x55bcfdbcfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.352+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbd0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.352+0000 7f61de03ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.353+0000 7f61bea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.353+0000 7f61bea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53124/0 (socket says 172.21.15.131:53124) 2024-05-10T05:41:21.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.353+0000 7f61bf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbd0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.353+0000 7f61bf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbd0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46638/0 (socket says 172.21.15.131:46638) 2024-05-10T05:41:21.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.354+0000 7f61bea00700 1 -- 172.21.15.131:0/2691285261 learned_addr learned my addr 172.21.15.131:0/2691285261 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:21.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.354+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfdbd37d0 con 0x55bcfdbc47a0 2024-05-10T05:41:21.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.354+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfdbd3940 con 0x55bcfdbc3530 2024-05-10T05:41:21.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.354+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfdbd3b00 con 0x55bcfdbc50f0 2024-05-10T05:41:21.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.354+0000 7f61bea00700 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc3530 0x55bcfdbcfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.355+0000 7f61bf400700 1 -- 172.21.15.131:0/2691285261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc50f0 msgr2=0x55bcfdbd3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:21.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.355+0000 7f61bf400700 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.355+0000 7f61bf400700 1 -- 172.21.15.131:0/2691285261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc3530 msgr2=0x55bcfdbcfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:21.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.355+0000 7f61bf400700 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc3530 0x55bcfdbcfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.355+0000 7f61bf400700 1 -- 172.21.15.131:0/2691285261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcfd9643b0 con 0x55bcfdbc47a0 2024-05-10T05:41:21.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.355+0000 7f61de03ab80 10 client.0 _ll_register_callbacks cb 0x55bcfda88b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:21.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.355+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61b8007d20 con 0x55bcfdbc47a0 2024-05-10T05:41:21.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.356+0000 7f61a8c00700 10 client.16563 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:41:21.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.356+0000 7f61bf400700 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbd0350 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f61ac025770 tx=0x7f61ac001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac00a7c10e0afe69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:21.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.357+0000 7f61a8c00700 1 -- 172.21.15.131:0/2691285261 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac017070 con 0x55bcfdbc47a0 2024-05-10T05:41:21.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.358+0000 7f61a8c00700 1 -- 172.21.15.131:0/2691285261 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f61ac0117f0 con 0x55bcfdbc47a0 2024-05-10T05:41:21.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.358+0000 7f61a8c00700 1 -- 172.21.15.131:0/2691285261 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61ac017070 con 0x55bcfdbc47a0 2024-05-10T05:41:21.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.359+0000 7f61de03ab80 10 client.16563 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:21.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.359+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55bcfdaa9560 con 0x55bcfdbc47a0 2024-05-10T05:41:21.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.360+0000 7f61a8c00700 1 -- 172.21.15.131:0/2691285261 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f61ac0119e0 con 0x55bcfdbc47a0 2024-05-10T05:41:21.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.360+0000 7f61de03ab80 10 client.16563 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:21.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.360+0000 7f61de03ab80 10 client.16563 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:41:21.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.360+0000 7f61de03ab80 20 client.16563 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:21.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.360+0000 7f61de03ab80 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bcfdc40a70 0x55bcfdc42e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:21.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.360+0000 7f61de03ab80 4 client.16563 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:41:21.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.360+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bcfdadbec0 con 0x55bcfdc40a70 2024-05-10T05:41:21.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.364+0000 7f61bea00700 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bcfdc40a70 0x55bcfdc42e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:21.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.367+0000 7f61bea00700 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bcfdc40a70 0x55bcfdc42e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a1a47dd3e60e405 server_cookie=780e04a1e06e0038 in_seq=0 out_seq=0 2024-05-10T05:41:21.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.368+0000 7f61a8c00700 10 client.16563 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:41:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.375+0000 7f61a8c00700 1 -- 172.21.15.131:0/2691285261 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bcfdadbec0 con 0x55bcfdc40a70 2024-05-10T05:41:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.375+0000 7f61a8c00700 10 client.16563 handle_command_reply: tid=0 2024-05-10T05:41:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.376+0000 7f61de03ab80 1 client.16563 shutdown 2024-05-10T05:41:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.376+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 msgr2=0x55bcfdbd0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:21.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.376+0000 7f61de03ab80 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbd0350 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f61ac025770 tx=0x7f61ac001e50 comp rx=0 tx=0).stop 2024-05-10T05:41:21.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.377+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 shutdown_connections 2024-05-10T05:41:21.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.377+0000 7f61de03ab80 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bcfdc40a70 0x55bcfdc42e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.377+0000 7f61de03ab80 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcfdbc50f0 0x55bcfdbd3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.377+0000 7f61de03ab80 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcfdbc3530 0x55bcfdbcfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.377+0000 7f61de03ab80 1 --2- 172.21.15.131:0/2691285261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcfdbc47a0 0x55bcfdbd0350 secure :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f61ac025770 tx=0x7f61ac001e50 comp rx=0 tx=0).stop 2024-05-10T05:41:21.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.377+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 >> 172.21.15.131:0/2691285261 conn(0x55bcfdb25a60 msgr2=0x55bcfdbc8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:21.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.378+0000 7f61a8c00700 0 client.16563 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:41:21.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.380+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 shutdown_connections 2024-05-10T05:41:21.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.380+0000 7f61de03ab80 1 -- 172.21.15.131:0/2691285261 wait complete. 2024-05-10T05:41:21.382 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:41:21.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.381+0000 7f61de03ab80 20 client.16563 trim_cache size 0 max 16384 2024-05-10T05:41:21.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.385+0000 7f61c4c00700 1 -- 172.21.15.131:0/959652408 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f61a40a1320 msgr2=0x7f61a40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:21.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.385+0000 7f61c4c00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f61a40a1320 0x7f61a40a3830 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e970 tx=0x7f61b0009530 comp rx=0 tx=0).stop 2024-05-10T05:41:21.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.385+0000 7f61c4c00700 1 -- 172.21.15.131:0/959652408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c01121e0 msgr2=0x7f61c01a9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:21.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.385+0000 7f61c4c00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c01121e0 0x7f61c01a9c50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e9a0 tx=0x7f61b000ed60 comp rx=0 tx=0).stop 2024-05-10T05:41:21.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.386+0000 7f61c4c00700 1 -- 172.21.15.131:0/959652408 shutdown_connections 2024-05-10T05:41:21.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.386+0000 7f61c4c00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f61a40a1320 0x7f61a40a3830 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.386+0000 7f61c4c00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c01121e0 0x7f61c01a9c50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.386+0000 7f61c4c00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c010a9a0 0x7f61c01a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.386+0000 7f61c4c00700 1 --2- 172.21.15.131:0/959652408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c010b750 0x7f61c01a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:21.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.386+0000 7f61c4c00700 1 -- 172.21.15.131:0/959652408 >> 172.21.15.131:0/959652408 conn(0x7f61c006e800 msgr2=0x7f61c0108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:21.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.386+0000 7f61c4c00700 1 -- 172.21.15.131:0/959652408 shutdown_connections 2024-05-10T05:41:21.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:21.386+0000 7f61c4c00700 1 -- 172.21.15.131:0/959652408 wait complete. 2024-05-10T05:41:21.510 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:41:22.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.152+0000 7f0ccb000700 1 -- 172.21.15.131:0/3492809217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cc4101be0 msgr2=0x7f0cc4101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:22.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.152+0000 7f0ccb000700 1 --2- 172.21.15.131:0/3492809217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cc4101be0 0x7f0cc4101ff0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40099c0 tx=0x7f0cb4009cd0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.152+0000 7f0ccb000700 1 -- 172.21.15.131:0/3492809217 shutdown_connections 2024-05-10T05:41:22.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.152+0000 7f0ccb000700 1 --2- 172.21.15.131:0/3492809217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc41036a0 0x7f0cc4109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.152+0000 7f0ccb000700 1 --2- 172.21.15.131:0/3492809217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cc4101be0 0x7f0cc4101ff0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.152+0000 7f0ccb000700 1 --2- 172.21.15.131:0/3492809217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc4102d00 0x7f0cc4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.152+0000 7f0ccb000700 1 -- 172.21.15.131:0/3492809217 >> 172.21.15.131:0/3492809217 conn(0x7f0cc40fd210 msgr2=0x7f0cc40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:22.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.152+0000 7f0ccb000700 1 -- 172.21.15.131:0/3492809217 shutdown_connections 2024-05-10T05:41:22.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.152+0000 7f0ccb000700 1 -- 172.21.15.131:0/3492809217 wait complete. 2024-05-10T05:41:22.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.154+0000 7f0ccb000700 1 Processor -- start 2024-05-10T05:41:22.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.154+0000 7f0ccb000700 1 -- start start 2024-05-10T05:41:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.155+0000 7f0ccb000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc4101be0 0x7f0cc4074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.155+0000 7f0ccb000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc4102d00 0x7f0cc4073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.155+0000 7f0ccb000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cc41036a0 0x7f0cc4073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.155+0000 7f0ccb000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4074100 con 0x7f0cc4102d00 2024-05-10T05:41:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.155+0000 7f0ccb000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4074270 con 0x7f0cc4101be0 2024-05-10T05:41:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.155+0000 7f0ccb000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4074430 con 0x7f0cc41036a0 2024-05-10T05:41:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.155+0000 7f0cc3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc4101be0 0x7f0cc4074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.155+0000 7f0cc3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc4101be0 0x7f0cc4074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43238/0 (socket says 172.21.15.131:43238) 2024-05-10T05:41:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.155+0000 7f0cc3e00700 1 -- 172.21.15.131:0/3146940116 learned_addr learned my addr 172.21.15.131:0/3146940116 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.155+0000 7f0cc9000700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cc41036a0 0x7f0cc4073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.156+0000 7f0cc3400700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc4102d00 0x7f0cc4073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.156+0000 7f0cc9000700 1 -- 172.21.15.131:0/3146940116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc4101be0 msgr2=0x7f0cc4074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.156+0000 7f0cc9000700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc4101be0 0x7f0cc4074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.156+0000 7f0cc9000700 1 -- 172.21.15.131:0/3146940116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc4102d00 msgr2=0x7f0cc4073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.156+0000 7f0cc9000700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc4102d00 0x7f0cc4073560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.156+0000 7f0cc9000700 1 -- 172.21.15.131:0/3146940116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb40095f0 con 0x7f0cc41036a0 2024-05-10T05:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.156+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc4101be0 0x7f0cc4074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.156+0000 7f0cc9000700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cc41036a0 0x7f0cc4073aa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00cc50 tx=0x7f0cbc00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.157+0000 7f0cc0c00700 1 -- 172.21.15.131:0/3146940116 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00ab40 con 0x7f0cc41036a0 2024-05-10T05:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.157+0000 7f0cc0c00700 1 -- 172.21.15.131:0/3146940116 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cbc00aca0 con 0x7f0cc41036a0 2024-05-10T05:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.157+0000 7f0cc0c00700 1 -- 172.21.15.131:0/3146940116 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc0115c0 con 0x7f0cc41036a0 2024-05-10T05:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.157+0000 7f0ccb000700 1 -- 172.21.15.131:0/3146940116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc41a21b0 con 0x7f0cc41036a0 2024-05-10T05:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.157+0000 7f0ccb000700 1 -- 172.21.15.131:0/3146940116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc41a2670 con 0x7f0cc41036a0 2024-05-10T05:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.162+0000 7f0cc0c00700 1 -- 172.21.15.131:0/3146940116 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0cbc012420 con 0x7f0cc41036a0 2024-05-10T05:41:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.164+0000 7f0cc0c00700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ca80a1240 0x7f0ca80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.165+0000 7f0cc0c00700 1 -- 172.21.15.131:0/3146940116 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0cbc0cca00 con 0x7f0cc41036a0 2024-05-10T05:41:22.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.165+0000 7f0ce2f2cb80 1 Processor -- start 2024-05-10T05:41:22.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.165+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ca80a1240 0x7f0ca80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.167+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ca80a1240 0x7f0ca80a3750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4003780 tx=0x7f0cb4019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:22.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.167+0000 7f0ce2f2cb80 1 -- start start 2024-05-10T05:41:22.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 0x55e5417a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 0x55e5417a9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e5417b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5416cf150 con 0x55e5417a86a0 2024-05-10T05:41:22.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5416ccb10 con 0x55e5417a9ec0 2024-05-10T05:41:22.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5416c8400 con 0x55e5417a9570 2024-05-10T05:41:22.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0cc3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 0x55e5417a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0cc9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e5417b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0cc9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e5417b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43244/0 (socket says 172.21.15.131:43244) 2024-05-10T05:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0cc9000700 1 -- 172.21.15.131:0/3585053682 learned_addr learned my addr 172.21.15.131:0/3585053682 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/3585053682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 0x55e5417a8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46650/0 (socket says 172.21.15.131:46650) 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.168+0000 7f0cc3400700 1 --2- 172.21.15.131:0/3585053682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 0x55e5417a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.169+0000 7f0cc9000700 1 -- 172.21.15.131:0/3585053682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 msgr2=0x55e5417a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.169+0000 7f0cc9000700 1 --2- 172.21.15.131:0/3585053682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 0x55e5417a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.169+0000 7f0cc9000700 1 -- 172.21.15.131:0/3585053682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 msgr2=0x55e5417a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.169+0000 7f0cc9000700 1 --2- 172.21.15.131:0/3585053682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 0x55e5417a8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.169+0000 7f0cc9000700 1 -- 172.21.15.131:0/3585053682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5415493b0 con 0x55e5417a9ec0 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.169+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/3585053682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 0x55e5417a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.169+0000 7f0cc9000700 1 --2- 172.21.15.131:0/3585053682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e5417b07c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc099de0 tx=0x7f0cbc099830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3769ee23b97bfe8c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.169+0000 7f0cb9600700 1 -- 172.21.15.131:0/3585053682 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc09dc80 con 0x55e5417a9ec0 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.169+0000 7f0cb9600700 1 -- 172.21.15.131:0/3585053682 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cbc099c20 con 0x55e5417a9ec0 2024-05-10T05:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.169+0000 7f0cb9600700 1 -- 172.21.15.131:0/3585053682 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc099340 con 0x55e5417a9ec0 2024-05-10T05:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.170+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/3585053682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 msgr2=0x55e5417b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.170+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/3585053682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e5417b07c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc099de0 tx=0x7f0cbc099830 comp rx=0 tx=0).stop 2024-05-10T05:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.171+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/3585053682 shutdown_connections 2024-05-10T05:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.171+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/3585053682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 0x55e5417a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.171+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/3585053682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e5417b07c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.171+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/3585053682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 0x55e5417a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.171+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/3585053682 >> 172.21.15.131:0/3585053682 conn(0x55e54170aa60 msgr2=0x55e5417a2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:22.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.171+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/3585053682 shutdown_connections 2024-05-10T05:41:22.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.171+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/3585053682 wait complete. 2024-05-10T05:41:22.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.173+0000 7f0ce2f2cb80 1 -- start start 2024-05-10T05:41:22.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.174+0000 7f0ce2f2cb80 1 Processor -- start 2024-05-10T05:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.175+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a86a0 0x55e5417b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.175+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 0x55e5417b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.175+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9ec0 0x55e5417b9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.175+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e541664560 con 0x55e5417a9570 2024-05-10T05:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.175+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e541664200 con 0x55e5417a86a0 2024-05-10T05:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.175+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54154b6a0 con 0x55e5417a9ec0 2024-05-10T05:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.176+0000 7f0ce2f2cb80 10 client.0 _ll_register_callbacks cb 0x55e54166db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:22.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.176+0000 7f0cc3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a86a0 0x55e5417b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.176+0000 7f0cc3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a86a0 0x55e5417b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43250/0 (socket says 172.21.15.131:43250) 2024-05-10T05:41:22.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.176+0000 7f0cc3e00700 1 -- 172.21.15.131:0/1871161604 learned_addr learned my addr 172.21.15.131:0/1871161604 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:22.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.176+0000 7f0cc9000700 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9ec0 0x55e5417b9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.177+0000 7f0cc3e00700 1 -- 172.21.15.131:0/1871161604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9ec0 msgr2=0x55e5417b9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.177+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9ec0 0x55e5417b9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.177+0000 7f0cc3e00700 1 -- 172.21.15.131:0/1871161604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 msgr2=0x55e5417b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.177+0000 7f0cc3400700 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 0x55e5417b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.177+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 0x55e5417b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.177+0000 7f0cc3e00700 1 -- 172.21.15.131:0/1871161604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc41a2670 con 0x55e5417a86a0 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.177+0000 7f0cc3400700 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 0x55e5417b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.177+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a86a0 0x55e5417b2740 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40098e0 tx=0x7f0cb4009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc7c8c12b7bd128 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.177+0000 7f0cae000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.177+0000 7f0cae000700 1 -- 172.21.15.131:0/1871161604 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4028050 con 0x55e5417a86a0 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.178+0000 7f0cae000700 1 -- 172.21.15.131:0/1871161604 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0cb400fe50 con 0x55e5417a86a0 2024-05-10T05:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.178+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/1871161604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e541621270 con 0x55e5417a86a0 2024-05-10T05:41:22.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.178+0000 7f0cae000700 1 -- 172.21.15.131:0/1871161604 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cb40054e0 con 0x55e5417a86a0 2024-05-10T05:41:22.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.178+0000 7f0ce2f2cb80 10 client.26816 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:22.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.178+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/1871161604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e5414f5b60 con 0x55e5417a86a0 2024-05-10T05:41:22.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.178+0000 7f0cae000700 1 -- 172.21.15.131:0/1871161604 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e541621270 con 0x55e5417a86a0 2024-05-10T05:41:22.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.179+0000 7f0cae000700 1 -- 172.21.15.131:0/1871161604 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0cb402e8a0 con 0x55e5417a86a0 2024-05-10T05:41:22.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.179+0000 7f0ce2f2cb80 10 client.26816 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:22.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.179+0000 7f0ce2f2cb80 10 client.26816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:41:22.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.179+0000 7f0ce2f2cb80 20 client.26816 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:22.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.179+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e5417be360 0x55e5417c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.179+0000 7f0ce2f2cb80 4 client.26816 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:22.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.179+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/1871161604 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e541777780 con 0x55e5417be360 2024-05-10T05:41:22.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.180+0000 7f0cc3400700 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e5417be360 0x55e5417c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.180+0000 7f0cae000700 10 client.26816 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:22.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.180+0000 7f0cc3400700 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e5417be360 0x55e5417c07b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbca9ee6f7e4f197 server_cookie=396ba93730b70ebe in_seq=0 out_seq=0 2024-05-10T05:41:22.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.182+0000 7f0cae000700 1 -- 172.21.15.131:0/1871161604 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e541777780 con 0x55e5417be360 2024-05-10T05:41:22.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.182+0000 7f0cae000700 10 client.26816 handle_command_reply: tid=0 2024-05-10T05:41:22.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.182+0000 7f0ce2f2cb80 1 client.26816 shutdown 2024-05-10T05:41:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/1871161604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a86a0 msgr2=0x55e5417b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a86a0 0x55e5417b2740 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40098e0 tx=0x7f0cb4009cd0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/1871161604 shutdown_connections 2024-05-10T05:41:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e5417be360 0x55e5417c07b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9ec0 0x55e5417b9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a86a0 0x55e5417b2740 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/1871161604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 0x55e5417b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/1871161604 >> 172.21.15.131:0/1871161604 conn(0x55e54170aa60 msgr2=0x55e5417a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:22.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0cae000700 0 client.26816 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:22.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/1871161604 shutdown_connections 2024-05-10T05:41:22.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.183+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/1871161604 wait complete. 2024-05-10T05:41:22.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.184+0000 7f0ce2f2cb80 20 client.26816 trim_cache size 0 max 16384 2024-05-10T05:41:22.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.207+0000 7f0ce2f2cb80 1 Processor -- start 2024-05-10T05:41:22.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.207+0000 7f0ce2f2cb80 1 -- start start 2024-05-10T05:41:22.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.208+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 0x55e5417a4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.208+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 0x55e5417b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.208+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e5417b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.208+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54179d850 con 0x55e5417a86a0 2024-05-10T05:41:22.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.208+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5417a99b0 con 0x55e5417a9ec0 2024-05-10T05:41:22.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.208+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5417be060 con 0x55e5417a9570 2024-05-10T05:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.208+0000 7f0cc3e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 0x55e5417a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.209+0000 7f0cc3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 0x55e5417b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.209+0000 7f0cc3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 0x55e5417b9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53158/0 (socket says 172.21.15.131:53158) 2024-05-10T05:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.209+0000 7f0cc3400700 1 -- 172.21.15.131:0/2915717225 learned_addr learned my addr 172.21.15.131:0/2915717225 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.209+0000 7f0cc9000700 1 --2- 172.21.15.131:0/2915717225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e5417b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.209+0000 7f0cc3400700 1 -- 172.21.15.131:0/2915717225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 msgr2=0x55e5417b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.209+0000 7f0cc3400700 1 --2- 172.21.15.131:0/2915717225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e5417b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.209+0000 7f0cc3400700 1 -- 172.21.15.131:0/2915717225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 msgr2=0x55e5417a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.209+0000 7f0cc3400700 1 --2- 172.21.15.131:0/2915717225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 0x55e5417a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.209+0000 7f0cc3400700 1 -- 172.21.15.131:0/2915717225 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e54168e560 con 0x55e5417a9570 2024-05-10T05:41:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.211+0000 7f0cc3400700 1 --2- 172.21.15.131:0/2915717225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 0x55e5417b9ba0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009c00 tx=0x7f0cb0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d840a6cf180f9325 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.211+0000 7f0cba000700 1 -- 172.21.15.131:0/2915717225 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0018030 con 0x55e5417a9570 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.211+0000 7f0cba000700 1 -- 172.21.15.131:0/2915717225 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cb0019040 con 0x55e5417a9570 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.211+0000 7f0cba000700 1 -- 172.21.15.131:0/2915717225 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0004a20 con 0x55e5417a9570 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.213+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/2915717225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 msgr2=0x55e5417b9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.213+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/2915717225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 0x55e5417b9ba0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009c00 tx=0x7f0cb0009260 comp rx=0 tx=0).stop 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.213+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/2915717225 shutdown_connections 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.213+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/2915717225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a86a0 0x55e5417a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.213+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/2915717225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e5417b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.213+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/2915717225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a9570 0x55e5417b9ba0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.213+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/2915717225 >> 172.21.15.131:0/2915717225 conn(0x55e54170aa60 msgr2=0x55e5417ad350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.215+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/2915717225 shutdown_connections 2024-05-10T05:41:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.215+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/2915717225 wait complete. 2024-05-10T05:41:22.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.216+0000 7f0ce2f2cb80 1 -- start start 2024-05-10T05:41:22.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.216+0000 7f0ce2f2cb80 1 Processor -- start 2024-05-10T05:41:22.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.219+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a86a0 0x55e5417b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.219+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 0x55e5417b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.219+0000 7f0ce2f2cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e54181b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.219+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5417b7760 con 0x55e5417a9570 2024-05-10T05:41:22.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.219+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54181b550 con 0x55e5417a9ec0 2024-05-10T05:41:22.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.219+0000 7f0ce2f2cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54181b710 con 0x55e5417a86a0 2024-05-10T05:41:22.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.220+0000 7f0ce2f2cb80 10 client.0 _ll_register_callbacks cb 0x55e54166db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:22.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.220+0000 7f0cc9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e54181b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.220+0000 7f0cc9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e54181b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43278/0 (socket says 172.21.15.131:43278) 2024-05-10T05:41:22.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.220+0000 7f0cc9000700 1 -- 172.21.15.131:0/4062294657 learned_addr learned my addr 172.21.15.131:0/4062294657 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:22.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.220+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a86a0 0x55e5417b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.220+0000 7f0cc3400700 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 0x55e5417b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.221+0000 7f0cc9000700 1 -- 172.21.15.131:0/4062294657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a86a0 msgr2=0x55e5417b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:22.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.221+0000 7f0cc9000700 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a86a0 0x55e5417b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.221+0000 7f0cc9000700 1 -- 172.21.15.131:0/4062294657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 msgr2=0x55e5417b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:22.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.221+0000 7f0cc9000700 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 0x55e5417b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.221+0000 7f0cc9000700 1 -- 172.21.15.131:0/4062294657 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb00095c0 con 0x55e5417a9ec0 2024-05-10T05:41:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.221+0000 7f0cc9000700 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e54181b010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc012ca0 tx=0x7f0cbc0073e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fab125c99d8f1dad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.222+0000 7f0cae000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:41:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.222+0000 7f0cae000700 1 -- 172.21.15.131:0/4062294657 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc099340 con 0x55e5417a9ec0 2024-05-10T05:41:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.222+0000 7f0cae000700 1 -- 172.21.15.131:0/4062294657 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0cbc0994a0 con 0x55e5417a9ec0 2024-05-10T05:41:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.222+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/4062294657 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cbc099c20 con 0x55e5417a9ec0 2024-05-10T05:41:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.222+0000 7f0cae000700 1 -- 172.21.15.131:0/4062294657 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cbc0833d0 con 0x55e5417a9ec0 2024-05-10T05:41:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.222+0000 7f0ce2f2cb80 10 client.26822 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.222+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/4062294657 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e54154dcd0 con 0x55e5417a9ec0 2024-05-10T05:41:22.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.223+0000 7f0cae000700 1 -- 172.21.15.131:0/4062294657 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cbc099c20 con 0x55e5417a9ec0 2024-05-10T05:41:22.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.224+0000 7f0cae000700 1 -- 172.21.15.131:0/4062294657 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0cbc0add20 con 0x55e5417a9ec0 2024-05-10T05:41:22.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.224+0000 7f0ce2f2cb80 10 client.26822 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.224+0000 7f0ce2f2cb80 10 client.26822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.224+0000 7f0ce2f2cb80 20 client.26822 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.224+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e54181cb90 0x55e54181efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.224+0000 7f0ce2f2cb80 4 client.26822 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.224+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/4062294657 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e5416c0ec0 con 0x55e54181cb90 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.225+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e54181cb90 0x55e54181efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.225+0000 7f0cae000700 10 client.26822 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.225+0000 7f0cc3e00700 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e54181cb90 0x55e54181efb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd90973849317610 server_cookie=230b519f6c0a9c34 in_seq=0 out_seq=0 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.226+0000 7f0cae000700 1 -- 172.21.15.131:0/4062294657 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e5416c0ec0 con 0x55e54181cb90 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.226+0000 7f0cae000700 10 client.26822 handle_command_reply: tid=0 2024-05-10T05:41:22.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.226+0000 7f0ce2f2cb80 1 client.26822 shutdown 2024-05-10T05:41:22.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.226+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/4062294657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 msgr2=0x55e54181b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:22.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.226+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e54181b010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc012ca0 tx=0x7f0cbc0073e0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.227+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/4062294657 shutdown_connections 2024-05-10T05:41:22.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.227+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55e54181cb90 0x55e54181efb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.227+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5417a9ec0 0x55e54181b010 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.227+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e5417a86a0 0x55e5417b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.227+0000 7f0ce2f2cb80 1 --2- 172.21.15.131:0/4062294657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e5417a9570 0x55e5417b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.227+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/4062294657 >> 172.21.15.131:0/4062294657 conn(0x55e54170aa60 msgr2=0x55e5417ad350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:22.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.227+0000 7f0cae000700 0 client.26822 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:22.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.227+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/4062294657 shutdown_connections 2024-05-10T05:41:22.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.227+0000 7f0ce2f2cb80 1 -- 172.21.15.131:0/4062294657 wait complete. 2024-05-10T05:41:22.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.228+0000 7f0ce2f2cb80 20 client.26822 trim_cache size 0 max 16384 2024-05-10T05:41:22.230 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:41:22.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.229+0000 7f0ccb000700 1 -- 172.21.15.131:0/3146940116 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ca80a1240 msgr2=0x7f0ca80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:22.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.229+0000 7f0ccb000700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ca80a1240 0x7f0ca80a3750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4003780 tx=0x7f0cb4019040 comp rx=0 tx=0).stop 2024-05-10T05:41:22.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.229+0000 7f0ccb000700 1 -- 172.21.15.131:0/3146940116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cc41036a0 msgr2=0x7f0cc4073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:22.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.229+0000 7f0ccb000700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cc41036a0 0x7f0cc4073aa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00cc50 tx=0x7f0cbc00cf60 comp rx=0 tx=0).stop 2024-05-10T05:41:22.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.229+0000 7f0ccb000700 1 -- 172.21.15.131:0/3146940116 shutdown_connections 2024-05-10T05:41:22.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.229+0000 7f0ccb000700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0ca80a1240 0x7f0ca80a3750 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.229+0000 7f0ccb000700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cc41036a0 0x7f0cc4073aa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.229+0000 7f0ccb000700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc4101be0 0x7f0cc4074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.229+0000 7f0ccb000700 1 --2- 172.21.15.131:0/3146940116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cc4102d00 0x7f0cc4073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:22.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.229+0000 7f0ccb000700 1 -- 172.21.15.131:0/3146940116 >> 172.21.15.131:0/3146940116 conn(0x7f0cc40fd210 msgr2=0x7f0cc4105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:22.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.230+0000 7f0ccb000700 1 -- 172.21.15.131:0/3146940116 shutdown_connections 2024-05-10T05:41:22.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:22.230+0000 7f0ccb000700 1 -- 172.21.15.131:0/3146940116 wait complete. 2024-05-10T05:41:22.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:22 smithi142 ceph-mon[77667]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 257 MiB/s wr, 78 op/s 2024-05-10T05:41:22.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:22 smithi131 ceph-mon[37264]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 257 MiB/s wr, 78 op/s 2024-05-10T05:41:22.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:22 smithi157 ceph-mon[78427]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 257 MiB/s wr, 78 op/s 2024-05-10T05:41:23.294 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:41:23.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:41:24.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.008+0000 7fca65c00700 1 -- 172.21.15.131:0/2168039943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60108630 msgr2=0x7fca60108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.008+0000 7fca65c00700 1 --2- 172.21.15.131:0/2168039943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60108630 0x7fca60108a40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca480098f0 tx=0x7fca48009c00 comp rx=0 tx=0).stop 2024-05-10T05:41:24.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.008+0000 7fca65c00700 1 -- 172.21.15.131:0/2168039943 shutdown_connections 2024-05-10T05:41:24.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.008+0000 7fca65c00700 1 --2- 172.21.15.131:0/2168039943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60109c50 0x7fca6010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.008+0000 7fca65c00700 1 --2- 172.21.15.131:0/2168039943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60108630 0x7fca60108a40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.008+0000 7fca65c00700 1 --2- 172.21.15.131:0/2168039943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60109300 0x7fca60109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.008+0000 7fca65c00700 1 -- 172.21.15.131:0/2168039943 >> 172.21.15.131:0/2168039943 conn(0x7fca6006ea00 msgr2=0x7fca60070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.008+0000 7fca65c00700 1 -- 172.21.15.131:0/2168039943 shutdown_connections 2024-05-10T05:41:24.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.008+0000 7fca65c00700 1 -- 172.21.15.131:0/2168039943 wait complete. 2024-05-10T05:41:24.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.010+0000 7fca65c00700 1 Processor -- start 2024-05-10T05:41:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.013+0000 7fca65c00700 1 -- start start 2024-05-10T05:41:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.014+0000 7fca65c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60108630 0x7fca60115550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.014+0000 7fca65c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60109300 0x7fca60115a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.014+0000 7fca65c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60109c50 0x7fca601169f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.014+0000 7fca65c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca60117000 con 0x7fca60109c50 2024-05-10T05:41:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.014+0000 7fca65c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca60117170 con 0x7fca60109300 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.014+0000 7fca65c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca60117330 con 0x7fca60108630 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.014+0000 7fca5ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60108630 0x7fca60115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.014+0000 7fca5ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60108630 0x7fca60115550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44302/0 (socket says 172.21.15.131:44302) 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.014+0000 7fca5ea00700 1 -- 172.21.15.131:0/704668073 learned_addr learned my addr 172.21.15.131:0/704668073 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.014+0000 7fca56000700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60109300 0x7fca60115a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.015+0000 7fca5ea00700 1 -- 172.21.15.131:0/704668073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60109300 msgr2=0x7fca60115a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.015+0000 7fca5ea00700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60109300 0x7fca60115a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.015+0000 7fca5ea00700 1 -- 172.21.15.131:0/704668073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60109c50 msgr2=0x7fca601169f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.015+0000 7fca5ea00700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60109c50 0x7fca601169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.015+0000 7fca5ea00700 1 -- 172.21.15.131:0/704668073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca48009520 con 0x7fca60108630 2024-05-10T05:41:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.016+0000 7fca5ea00700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60108630 0x7fca60115550 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fca4800fb40 tx=0x7fca4800b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.017+0000 7fca57e00700 1 -- 172.21.15.131:0/704668073 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4800ba00 con 0x7fca60108630 2024-05-10T05:41:24.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.017+0000 7fca57e00700 1 -- 172.21.15.131:0/704668073 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca4803e070 con 0x7fca60108630 2024-05-10T05:41:24.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.017+0000 7fca57e00700 1 -- 172.21.15.131:0/704668073 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48003be0 con 0x7fca60108630 2024-05-10T05:41:24.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.017+0000 7fca65c00700 1 -- 172.21.15.131:0/704668073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca601175b0 con 0x7fca60108630 2024-05-10T05:41:24.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.017+0000 7fca65c00700 1 -- 172.21.15.131:0/704668073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca60117950 con 0x7fca60108630 2024-05-10T05:41:24.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.031+0000 7fca7dab2b80 1 Processor -- start 2024-05-10T05:41:24.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.032+0000 7fca7dab2b80 1 -- start start 2024-05-10T05:41:24.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.033+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc296b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.033+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc297a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.033+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 0x558dbc29e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.033+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc1bd1d0 con 0x558dbc296720 2024-05-10T05:41:24.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.033+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc1bab90 con 0x558dbc297f40 2024-05-10T05:41:24.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.033+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc1b6480 con 0x558dbc2975f0 2024-05-10T05:41:24.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.034+0000 7fca5f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc296b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.034+0000 7fca5f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc296b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38480/0 (socket says 172.21.15.131:38480) 2024-05-10T05:41:24.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.034+0000 7fca5f400700 1 -- 172.21.15.131:0/4222120273 learned_addr learned my addr 172.21.15.131:0/4222120273 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:24.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.035+0000 7fca56000700 1 --2- 172.21.15.131:0/4222120273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 0x558dbc29e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.036+0000 7fca5f400700 1 -- 172.21.15.131:0/4222120273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 msgr2=0x558dbc297a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.036+0000 7fca5f400700 1 --2- 172.21.15.131:0/4222120273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc297a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.036+0000 7fca5f400700 1 -- 172.21.15.131:0/4222120273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 msgr2=0x558dbc29e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.036+0000 7fca5f400700 1 --2- 172.21.15.131:0/4222120273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 0x558dbc29e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.036+0000 7fca5f400700 1 -- 172.21.15.131:0/4222120273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dbc037430 con 0x558dbc296720 2024-05-10T05:41:24.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.037+0000 7fca5f400700 1 --2- 172.21.15.131:0/4222120273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc296b70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fca58009b70 tx=0x7fca58009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f54ac28e294273c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.038+0000 7fca4fe00700 1 -- 172.21.15.131:0/4222120273 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800ea80 con 0x558dbc296720 2024-05-10T05:41:24.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.039+0000 7fca4fe00700 1 -- 172.21.15.131:0/4222120273 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca58004cf0 con 0x558dbc296720 2024-05-10T05:41:24.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.039+0000 7fca4fe00700 1 -- 172.21.15.131:0/4222120273 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800ea80 con 0x558dbc296720 2024-05-10T05:41:24.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.040+0000 7fca7dab2b80 1 -- 172.21.15.131:0/4222120273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 msgr2=0x558dbc296b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.040+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/4222120273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc296b70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fca58009b70 tx=0x7fca58009e80 comp rx=0 tx=0).stop 2024-05-10T05:41:24.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.041+0000 7fca7dab2b80 1 -- 172.21.15.131:0/4222120273 shutdown_connections 2024-05-10T05:41:24.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.041+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/4222120273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc296b70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.041+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/4222120273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 0x558dbc29e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.041+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/4222120273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc297a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.041+0000 7fca7dab2b80 1 -- 172.21.15.131:0/4222120273 >> 172.21.15.131:0/4222120273 conn(0x558dbc1f8ae0 msgr2=0x558dbc291060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.042+0000 7fca7dab2b80 1 -- 172.21.15.131:0/4222120273 shutdown_connections 2024-05-10T05:41:24.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.042+0000 7fca7dab2b80 1 -- 172.21.15.131:0/4222120273 wait complete. 2024-05-10T05:41:24.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.042+0000 7fca57e00700 1 -- 172.21.15.131:0/704668073 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fca4804c440 con 0x7fca60108630 2024-05-10T05:41:24.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.042+0000 7fca7dab2b80 1 -- start start 2024-05-10T05:41:24.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.043+0000 7fca7dab2b80 1 Processor -- start 2024-05-10T05:41:24.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.044+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc296720 0x558dbc2a0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.044+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc2975f0 0x558dbc2a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.044+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc297f40 0x558dbc2a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.044+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc1525e0 con 0x558dbc2975f0 2024-05-10T05:41:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.044+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc152280 con 0x558dbc296720 2024-05-10T05:41:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.044+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc039720 con 0x558dbc297f40 2024-05-10T05:41:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.044+0000 7fca7dab2b80 10 client.0 _ll_register_callbacks cb 0x558dbc15bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.045+0000 7fca56000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc297f40 0x558dbc2a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.045+0000 7fca56000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc297f40 0x558dbc2a7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44304/0 (socket says 172.21.15.131:44304) 2024-05-10T05:41:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.045+0000 7fca56000700 1 -- 172.21.15.131:0/293478323 learned_addr learned my addr 172.21.15.131:0/293478323 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.046+0000 7fca57e00700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca400a11d0 0x7fca400a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.046+0000 7fca57e00700 1 -- 172.21.15.131:0/704668073 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fca480ef5c0 con 0x7fca60108630 2024-05-10T05:41:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.047+0000 7fca56000700 1 -- 172.21.15.131:0/293478323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc296720 msgr2=0x558dbc2a0870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:41:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.047+0000 7fca5f400700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca400a11d0 0x7fca400a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.047+0000 7fca56000700 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc296720 0x558dbc2a0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.047+0000 7fca56000700 1 -- 172.21.15.131:0/293478323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc2975f0 msgr2=0x558dbc2a0db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:24.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.047+0000 7fca56000700 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc2975f0 0x558dbc2a0db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.047+0000 7fca56000700 1 -- 172.21.15.131:0/293478323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca58009800 con 0x558dbc297f40 2024-05-10T05:41:24.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.050+0000 7fca4cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:41:24.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.050+0000 7fca5f400700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca400a11d0 0x7fca400a36e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fca58009350 tx=0x7fca58004750 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.050+0000 7fca56000700 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc297f40 0x558dbc2a7cc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fca5000af10 tx=0x7fca50009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506aab9c54a809d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.053+0000 7fca4cc00700 1 -- 172.21.15.131:0/293478323 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca50009ba0 con 0x558dbc297f40 2024-05-10T05:41:24.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.053+0000 7fca4cc00700 1 -- 172.21.15.131:0/293478323 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fca50006e40 con 0x558dbc297f40 2024-05-10T05:41:24.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.054+0000 7fca7dab2b80 1 -- 172.21.15.131:0/293478323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558dbc10f2f0 con 0x558dbc297f40 2024-05-10T05:41:24.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.059+0000 7fca4cc00700 1 -- 172.21.15.131:0/293478323 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca50007b30 con 0x558dbc297f40 2024-05-10T05:41:24.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.059+0000 7fca7dab2b80 10 client.26872 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:24.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.059+0000 7fca7dab2b80 1 -- 172.21.15.131:0/293478323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558dbbfe3be0 con 0x558dbc297f40 2024-05-10T05:41:24.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.060+0000 7fca4cc00700 1 -- 172.21.15.131:0/293478323 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558dbc10f2f0 con 0x558dbc297f40 2024-05-10T05:41:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.060+0000 7fca4cc00700 1 -- 172.21.15.131:0/293478323 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fca50014070 con 0x558dbc297f40 2024-05-10T05:41:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.060+0000 7fca7dab2b80 10 client.26872 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.060+0000 7fca7dab2b80 10 client.26872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:41:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.060+0000 7fca7dab2b80 20 client.26872 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:24.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.060+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558dbc2ac5c0 0x558dbc2aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.060+0000 7fca7dab2b80 4 client.26872 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:24.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.060+0000 7fca7dab2b80 1 -- 172.21.15.131:0/293478323 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dbc265800 con 0x558dbc2ac5c0 2024-05-10T05:41:24.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.062+0000 7fca5ea00700 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558dbc2ac5c0 0x558dbc2aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.064+0000 7fca5ea00700 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558dbc2ac5c0 0x558dbc2aea10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8244adf4cd1d3c6 server_cookie=434a77570e994fc5 in_seq=0 out_seq=0 2024-05-10T05:41:24.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.066+0000 7fca4cc00700 10 client.26872 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:24.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.066+0000 7fca4cc00700 1 -- 172.21.15.131:0/293478323 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558dbc265800 con 0x558dbc2ac5c0 2024-05-10T05:41:24.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.066+0000 7fca4cc00700 10 client.26872 handle_command_reply: tid=0 2024-05-10T05:41:24.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.066+0000 7fca7dab2b80 1 client.26872 shutdown 2024-05-10T05:41:24.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.074+0000 7fca7dab2b80 1 -- 172.21.15.131:0/293478323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc297f40 msgr2=0x558dbc2a7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.074+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc297f40 0x558dbc2a7cc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fca5000af10 tx=0x7fca50009d00 comp rx=0 tx=0).stop 2024-05-10T05:41:24.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.074+0000 7fca7dab2b80 1 -- 172.21.15.131:0/293478323 shutdown_connections 2024-05-10T05:41:24.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.074+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558dbc2ac5c0 0x558dbc2aea10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.074+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc297f40 0x558dbc2a7cc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.074+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc296720 0x558dbc2a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.074+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/293478323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc2975f0 0x558dbc2a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.074+0000 7fca4cc00700 0 client.26872 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:24.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.074+0000 7fca7dab2b80 1 -- 172.21.15.131:0/293478323 >> 172.21.15.131:0/293478323 conn(0x558dbc1f8ae0 msgr2=0x558dbc291030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.075+0000 7fca7dab2b80 1 -- 172.21.15.131:0/293478323 shutdown_connections 2024-05-10T05:41:24.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.075+0000 7fca7dab2b80 1 -- 172.21.15.131:0/293478323 wait complete. 2024-05-10T05:41:24.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.081+0000 7fca7dab2b80 20 client.26872 trim_cache size 0 max 16384 2024-05-10T05:41:24.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.106+0000 7fca7dab2b80 1 Processor -- start 2024-05-10T05:41:24.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.106+0000 7fca7dab2b80 1 -- start start 2024-05-10T05:41:24.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.107+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc296720 0x558dbc2a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.107+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc2a7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.107+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc297f40 0x558dbc29eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc2a18c0 con 0x558dbc297f40 2024-05-10T05:41:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc2a99d0 con 0x558dbc296720 2024-05-10T05:41:24.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc2927f0 con 0x558dbc2975f0 2024-05-10T05:41:24.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca56000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc2a7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca5ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc296720 0x558dbc2a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca56000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc2a7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44310/0 (socket says 172.21.15.131:44310) 2024-05-10T05:41:24.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca56000700 1 -- 172.21.15.131:0/2259298209 learned_addr learned my addr 172.21.15.131:0/2259298209 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:24.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca5f400700 1 --2- 172.21.15.131:0/2259298209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc297f40 0x558dbc29eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca5ea00700 1 -- 172.21.15.131:0/2259298209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 msgr2=0x558dbc2a7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca5ea00700 1 --2- 172.21.15.131:0/2259298209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc2a7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca5ea00700 1 -- 172.21.15.131:0/2259298209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc297f40 msgr2=0x558dbc29eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca5ea00700 1 --2- 172.21.15.131:0/2259298209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc297f40 0x558dbc29eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca5ea00700 1 -- 172.21.15.131:0/2259298209 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dbc17c5e0 con 0x558dbc296720 2024-05-10T05:41:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.108+0000 7fca5f400700 1 --2- 172.21.15.131:0/2259298209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc297f40 0x558dbc29eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:41:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.109+0000 7fca5ea00700 1 --2- 172.21.15.131:0/2259298209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc296720 0x558dbc2a01a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x558dbc2a1890 tx=0x7fca48009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d4bf1b214c65ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.109+0000 7fca54c00700 1 -- 172.21.15.131:0/2259298209 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4800bb60 con 0x558dbc296720 2024-05-10T05:41:24.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.109+0000 7fca54c00700 1 -- 172.21.15.131:0/2259298209 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca480bca40 con 0x558dbc296720 2024-05-10T05:41:24.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.109+0000 7fca54c00700 1 -- 172.21.15.131:0/2259298209 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48003d60 con 0x558dbc296720 2024-05-10T05:41:24.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.110+0000 7fca7dab2b80 1 -- 172.21.15.131:0/2259298209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc296720 msgr2=0x558dbc2a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.110+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/2259298209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc296720 0x558dbc2a01a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x558dbc2a1890 tx=0x7fca48009f50 comp rx=0 tx=0).stop 2024-05-10T05:41:24.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.110+0000 7fca7dab2b80 1 -- 172.21.15.131:0/2259298209 shutdown_connections 2024-05-10T05:41:24.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.110+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/2259298209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc297f40 0x558dbc29eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.110+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/2259298209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc296720 0x558dbc2a01a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.110+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/2259298209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc2a7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.110+0000 7fca7dab2b80 1 -- 172.21.15.131:0/2259298209 >> 172.21.15.131:0/2259298209 conn(0x558dbc1f8ae0 msgr2=0x558dbc29fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.110+0000 7fca7dab2b80 1 -- 172.21.15.131:0/2259298209 shutdown_connections 2024-05-10T05:41:24.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.111+0000 7fca7dab2b80 1 -- 172.21.15.131:0/2259298209 wait complete. 2024-05-10T05:41:24.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.111+0000 7fca7dab2b80 1 -- start start 2024-05-10T05:41:24.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.111+0000 7fca7dab2b80 1 Processor -- start 2024-05-10T05:41:24.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.112+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc2a7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.112+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc2a7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.112+0000 7fca7dab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 0x558dbc302d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.112+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc3032d0 con 0x558dbc296720 2024-05-10T05:41:24.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.112+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc303410 con 0x558dbc297f40 2024-05-10T05:41:24.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.112+0000 7fca7dab2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dbc3035d0 con 0x558dbc2975f0 2024-05-10T05:41:24.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.112+0000 7fca7dab2b80 10 client.0 _ll_register_callbacks cb 0x558dbc15bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:24.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.113+0000 7fca5ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc2a7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.113+0000 7fca5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 0x558dbc302d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.113+0000 7fca5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 0x558dbc302d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:58988/0 (socket says 172.21.15.131:58988) 2024-05-10T05:41:24.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.113+0000 7fca5f400700 1 -- 172.21.15.131:0/3414613457 learned_addr learned my addr 172.21.15.131:0/3414613457 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:24.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.113+0000 7fca5ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc2a7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38506/0 (socket says 172.21.15.131:38506) 2024-05-10T05:41:24.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.113+0000 7fca56000700 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc2a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.114+0000 7fca5f400700 1 -- 172.21.15.131:0/3414613457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 msgr2=0x558dbc2a7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.114+0000 7fca5f400700 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc2a7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.114+0000 7fca5f400700 1 -- 172.21.15.131:0/3414613457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 msgr2=0x558dbc2a7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.114+0000 7fca5f400700 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc2a7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.114+0000 7fca5f400700 1 -- 172.21.15.131:0/3414613457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca60117950 con 0x558dbc297f40 2024-05-10T05:41:24.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.114+0000 7fca56000700 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc2a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:41:24.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.114+0000 7fca5f400700 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 0x558dbc302d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fca58009b70 tx=0x7fca58009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a4c02979f5736f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.114+0000 7fca4cc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:41:24.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.115+0000 7fca4cc00700 1 -- 172.21.15.131:0/3414613457 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800ea80 con 0x558dbc297f40 2024-05-10T05:41:24.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.115+0000 7fca4cc00700 1 -- 172.21.15.131:0/3414613457 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fca5802a040 con 0x558dbc297f40 2024-05-10T05:41:24.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.115+0000 7fca7dab2b80 1 -- 172.21.15.131:0/3414613457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca58004cf0 con 0x558dbc297f40 2024-05-10T05:41:24.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.115+0000 7fca4cc00700 1 -- 172.21.15.131:0/3414613457 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca5800ebe0 con 0x558dbc297f40 2024-05-10T05:41:24.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.115+0000 7fca7dab2b80 10 client.26834 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:24.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.115+0000 7fca7dab2b80 1 -- 172.21.15.131:0/3414613457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558dbc03bd50 con 0x558dbc297f40 2024-05-10T05:41:24.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.115+0000 7fca4cc00700 1 -- 172.21.15.131:0/3414613457 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca5803e020 con 0x558dbc297f40 2024-05-10T05:41:24.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.116+0000 7fca4cc00700 1 -- 172.21.15.131:0/3414613457 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fca5802f4c0 con 0x558dbc297f40 2024-05-10T05:41:24.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.116+0000 7fca7dab2b80 10 client.26834 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:24.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.116+0000 7fca7dab2b80 10 client.26834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:41:24.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.116+0000 7fca7dab2b80 20 client.26834 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:24.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.116+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558dbc304f00 0x558dbc307320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.116+0000 7fca7dab2b80 4 client.26834 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:41:24.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.116+0000 7fca7dab2b80 1 -- 172.21.15.131:0/3414613457 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558dbc1aef40 con 0x558dbc304f00 2024-05-10T05:41:24.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.116+0000 7fca5ea00700 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558dbc304f00 0x558dbc307320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.117+0000 7fca4cc00700 10 client.26834 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:24.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.117+0000 7fca5ea00700 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558dbc304f00 0x558dbc307320 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376a18c4cf58b416 server_cookie=75c65f458e19dba4 in_seq=0 out_seq=0 2024-05-10T05:41:24.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.118+0000 7fca4cc00700 1 -- 172.21.15.131:0/3414613457 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558dbc1aef40 con 0x558dbc304f00 2024-05-10T05:41:24.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.118+0000 7fca4cc00700 10 client.26834 handle_command_reply: tid=0 2024-05-10T05:41:24.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.118+0000 7fca7dab2b80 1 client.26834 shutdown 2024-05-10T05:41:24.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.119+0000 7fca7dab2b80 1 -- 172.21.15.131:0/3414613457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 msgr2=0x558dbc302d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.119+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 0x558dbc302d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fca58009b70 tx=0x7fca58009e80 comp rx=0 tx=0).stop 2024-05-10T05:41:24.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.119+0000 7fca7dab2b80 1 -- 172.21.15.131:0/3414613457 shutdown_connections 2024-05-10T05:41:24.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.119+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x558dbc304f00 0x558dbc307320 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.119+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558dbc296720 0x558dbc2a7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.119+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dbc297f40 0x558dbc302d90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.119+0000 7fca7dab2b80 1 --2- 172.21.15.131:0/3414613457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558dbc2975f0 0x558dbc2a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.119+0000 7fca7dab2b80 1 -- 172.21.15.131:0/3414613457 >> 172.21.15.131:0/3414613457 conn(0x558dbc1f8ae0 msgr2=0x558dbc29ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.119+0000 7fca4cc00700 0 client.26834 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:24.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.120+0000 7fca7dab2b80 1 -- 172.21.15.131:0/3414613457 shutdown_connections 2024-05-10T05:41:24.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.120+0000 7fca7dab2b80 1 -- 172.21.15.131:0/3414613457 wait complete. 2024-05-10T05:41:24.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.121+0000 7fca7dab2b80 20 client.26834 trim_cache size 0 max 16384 2024-05-10T05:41:24.135 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:41:24.135 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:41:24.135 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "5125e731-f048-4110-a811-1ab783e5533b", 2024-05-10T05:41:24.136 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:41:24.136 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:41:24.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.121+0000 7fca65c00700 1 -- 172.21.15.131:0/704668073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca400a11d0 msgr2=0x7fca400a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.121+0000 7fca65c00700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca400a11d0 0x7fca400a36e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fca58009350 tx=0x7fca58004750 comp rx=0 tx=0).stop 2024-05-10T05:41:24.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.121+0000 7fca65c00700 1 -- 172.21.15.131:0/704668073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60108630 msgr2=0x7fca60115550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.121+0000 7fca65c00700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60108630 0x7fca60115550 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fca4800fb40 tx=0x7fca4800b560 comp rx=0 tx=0).stop 2024-05-10T05:41:24.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.122+0000 7fca65c00700 1 -- 172.21.15.131:0/704668073 shutdown_connections 2024-05-10T05:41:24.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.122+0000 7fca65c00700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca400a11d0 0x7fca400a36e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.122+0000 7fca65c00700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60109c50 0x7fca601169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.122+0000 7fca65c00700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60108630 0x7fca60115550 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.122+0000 7fca65c00700 1 --2- 172.21.15.131:0/704668073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca60109300 0x7fca60115a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.122+0000 7fca65c00700 1 -- 172.21.15.131:0/704668073 >> 172.21.15.131:0/704668073 conn(0x7fca6006ea00 msgr2=0x7fca6006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.122+0000 7fca65c00700 1 -- 172.21.15.131:0/704668073 shutdown_connections 2024-05-10T05:41:24.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.122+0000 7fca65c00700 1 -- 172.21.15.131:0/704668073 wait complete. 2024-05-10T05:41:24.175 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:41:24.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:24 smithi131 ceph-mon[37264]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 208 MiB/s wr, 62 op/s 2024-05-10T05:41:24.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:24 smithi142 ceph-mon[77667]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 208 MiB/s wr, 62 op/s 2024-05-10T05:41:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.795+0000 7ff01ec00700 1 -- 172.21.15.131:0/2688313145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff018108bb0 msgr2=0x7ff018079470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.795+0000 7ff01ec00700 1 --2- 172.21.15.131:0/2688313145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff018108bb0 0x7ff018079470 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b0a0 tx=0x7ff01000b3b0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.795+0000 7ff01ec00700 1 -- 172.21.15.131:0/2688313145 shutdown_connections 2024-05-10T05:41:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.796+0000 7ff01ec00700 1 --2- 172.21.15.131:0/2688313145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff018108bb0 0x7ff018079470 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.796+0000 7ff01ec00700 1 --2- 172.21.15.131:0/2688313145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff018109710 0x7ff018109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.796+0000 7ff01ec00700 1 --2- 172.21.15.131:0/2688313145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff018108210 0x7ff018108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.796+0000 7ff01ec00700 1 -- 172.21.15.131:0/2688313145 >> 172.21.15.131:0/2688313145 conn(0x7ff01806ea00 msgr2=0x7ff018070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.796+0000 7ff01ec00700 1 -- 172.21.15.131:0/2688313145 shutdown_connections 2024-05-10T05:41:24.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.796+0000 7ff01ec00700 1 -- 172.21.15.131:0/2688313145 wait complete. 2024-05-10T05:41:24.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.800+0000 7ff01ec00700 1 Processor -- start 2024-05-10T05:41:24.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.801+0000 7ff01ec00700 1 -- start start 2024-05-10T05:41:24.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.801+0000 7ff01ec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff018108210 0x7ff0180825c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:24 smithi157 ceph-mon[78427]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 208 MiB/s wr, 62 op/s 2024-05-10T05:41:24.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.802+0000 7ff01ec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff018108bb0 0x7ff018082b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.802+0000 7ff01ec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff018109710 0x7ff018083040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.802+0000 7ff017e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff018108210 0x7ff0180825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.802+0000 7ff017e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff018108210 0x7ff0180825c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44338/0 (socket says 172.21.15.131:44338) 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.802+0000 7ff017400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff018108bb0 0x7ff018082b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.802+0000 7ff017e00700 1 -- 172.21.15.131:0/1620895415 learned_addr learned my addr 172.21.15.131:0/1620895415 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.802+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180842b0 con 0x7ff018109710 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.802+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0181bfa90 con 0x7ff018108bb0 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.802+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0181bfc20 con 0x7ff018108210 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.803+0000 7ff017400700 1 -- 172.21.15.131:0/1620895415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff018108210 msgr2=0x7ff0180825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.803+0000 7ff017400700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff018108210 0x7ff0180825c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.803+0000 7ff017400700 1 -- 172.21.15.131:0/1620895415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff018109710 msgr2=0x7ff018083040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.803+0000 7ff017400700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff018109710 0x7ff018083040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.803+0000 7ff017400700 1 -- 172.21.15.131:0/1620895415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff010009d00 con 0x7ff018108bb0 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.803+0000 7ff017400700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff018108bb0 0x7ff018082b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff00800d950 tx=0x7ff00800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.803+0000 7ff014c00700 1 -- 172.21.15.131:0/1620895415 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008009930 con 0x7ff018108bb0 2024-05-10T05:41:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.804+0000 7ff014c00700 1 -- 172.21.15.131:0/1620895415 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff008010460 con 0x7ff018108bb0 2024-05-10T05:41:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.804+0000 7ff014c00700 1 -- 172.21.15.131:0/1620895415 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800b5d0 con 0x7ff018108bb0 2024-05-10T05:41:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.804+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0181bff00 con 0x7ff018108bb0 2024-05-10T05:41:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.806+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0181c0330 con 0x7ff018108bb0 2024-05-10T05:41:24.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.813+0000 7ff036a97b80 1 Processor -- start 2024-05-10T05:41:24.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.813+0000 7ff036a97b80 1 -- start start 2024-05-10T05:41:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.813+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 0x556326f05af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.813+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f06570 0x556326f06980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06ec0 0x556326f0d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff036a97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326e2c150 con 0x556326f06ec0 2024-05-10T05:41:24.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff036a97b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326e29b10 con 0x556326f056a0 2024-05-10T05:41:24.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff036a97b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326e25400 con 0x556326f06570 2024-05-10T05:41:24.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff01cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 0x556326f05af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff01cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 0x556326f05af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:59008/0 (socket says 172.21.15.131:59008) 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff01cc00700 1 -- 172.21.15.131:0/3476663421 learned_addr learned my addr 172.21.15.131:0/3476663421 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff017e00700 1 --2- 172.21.15.131:0/3476663421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f06570 0x556326f06980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff01cc00700 1 -- 172.21.15.131:0/3476663421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f06570 msgr2=0x556326f06980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff01cc00700 1 --2- 172.21.15.131:0/3476663421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f06570 0x556326f06980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff01cc00700 1 -- 172.21.15.131:0/3476663421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06ec0 msgr2=0x556326f0d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff01cc00700 1 --2- 172.21.15.131:0/3476663421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06ec0 0x556326f0d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.814+0000 7ff01cc00700 1 -- 172.21.15.131:0/3476663421 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556326ca63b0 con 0x556326f056a0 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.815+0000 7ff01cc00700 1 --2- 172.21.15.131:0/3476663421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 0x556326f05af0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b780 tx=0x7ff010009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cd6415c1961a9e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.815+0000 7ff014c00700 1 -- 172.21.15.131:0/1620895415 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff008009a90 con 0x7ff018108bb0 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.816+0000 7ff00d200700 1 -- 172.21.15.131:0/3476663421 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01000e040 con 0x556326f056a0 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.816+0000 7ff00d200700 1 -- 172.21.15.131:0/3476663421 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0100075c0 con 0x556326f056a0 2024-05-10T05:41:24.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.816+0000 7ff00d200700 1 -- 172.21.15.131:0/3476663421 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0100052a0 con 0x556326f056a0 2024-05-10T05:41:24.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.821+0000 7ff036a97b80 1 -- 172.21.15.131:0/3476663421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 msgr2=0x556326f05af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.821+0000 7ff036a97b80 1 --2- 172.21.15.131:0/3476663421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 0x556326f05af0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b780 tx=0x7ff010009780 comp rx=0 tx=0).stop 2024-05-10T05:41:24.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.821+0000 7ff014c00700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feffc0a11d0 0x7feffc0a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.821+0000 7ff014c00700 1 -- 172.21.15.131:0/1620895415 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff0080cf170 con 0x7ff018108bb0 2024-05-10T05:41:24.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.821+0000 7ff036a97b80 1 -- 172.21.15.131:0/3476663421 shutdown_connections 2024-05-10T05:41:24.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.821+0000 7ff036a97b80 1 --2- 172.21.15.131:0/3476663421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06ec0 0x556326f0d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.821+0000 7ff036a97b80 1 --2- 172.21.15.131:0/3476663421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 0x556326f05af0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.821+0000 7ff036a97b80 1 --2- 172.21.15.131:0/3476663421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f06570 0x556326f06980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.821+0000 7ff036a97b80 1 -- 172.21.15.131:0/3476663421 >> 172.21.15.131:0/3476663421 conn(0x556326e67a60 msgr2=0x556326efffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.822+0000 7ff01cc00700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feffc0a11d0 0x7feffc0a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.822+0000 7ff036a97b80 1 -- 172.21.15.131:0/3476663421 shutdown_connections 2024-05-10T05:41:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.822+0000 7ff036a97b80 1 -- 172.21.15.131:0/3476663421 wait complete. 2024-05-10T05:41:24.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.824+0000 7ff036a97b80 1 -- start start 2024-05-10T05:41:24.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.826+0000 7ff036a97b80 1 Processor -- start 2024-05-10T05:41:24.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.826+0000 7ff01cc00700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feffc0a11d0 0x7feffc0a36e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff0100062a0 tx=0x7ff010007b30 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.828+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f0f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.828+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 0x556326f0fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.828+0000 7ff017e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f0f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.828+0000 7ff017e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f0f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:44362/0 (socket says 172.21.15.131:44362) 2024-05-10T05:41:24.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.831+0000 7ff017e00700 1 -- 172.21.15.131:0/2933398591 learned_addr learned my addr 172.21.15.131:0/2933398591 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:24.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.831+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f06ec0 0x556326f16b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.831+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326dc1560 con 0x556326f06570 2024-05-10T05:41:24.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.831+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326dc1200 con 0x556326f06ec0 2024-05-10T05:41:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.831+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326ca86a0 con 0x556326f056a0 2024-05-10T05:41:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.831+0000 7ff036a97b80 10 client.0 _ll_register_callbacks cb 0x556326dcab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.832+0000 7ff017400700 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 0x556326f0fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.832+0000 7ff01cc00700 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f06ec0 0x556326f16b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.832+0000 7ff017e00700 1 -- 172.21.15.131:0/2933398591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f06ec0 msgr2=0x556326f16b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.832+0000 7ff017e00700 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f06ec0 0x556326f16b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.832+0000 7ff017e00700 1 -- 172.21.15.131:0/2933398591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 msgr2=0x556326f0fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.832+0000 7ff017e00700 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 0x556326f0fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.832+0000 7ff017e00700 1 -- 172.21.15.131:0/2933398591 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff018079f60 con 0x556326f056a0 2024-05-10T05:41:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.832+0000 7ff017400700 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 0x556326f0fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:41:24.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.832+0000 7ff005600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:41:24.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.832+0000 7ff017e00700 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f0f6c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff000012850 tx=0x7ff000012c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7332152c6d51a2c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.835+0000 7ff005600700 1 -- 172.21.15.131:0/2933398591 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000017070 con 0x556326f056a0 2024-05-10T05:41:24.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.835+0000 7ff005600700 1 -- 172.21.15.131:0/2933398591 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff000004830 con 0x556326f056a0 2024-05-10T05:41:24.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.835+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556326d7e270 con 0x556326f056a0 2024-05-10T05:41:24.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.835+0000 7ff005600700 1 -- 172.21.15.131:0/2933398591 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0000122e0 con 0x556326f056a0 2024-05-10T05:41:24.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.835+0000 7ff036a97b80 10 client.26890 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:24.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.835+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556326c52b60 con 0x556326f056a0 2024-05-10T05:41:24.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.836+0000 7ff005600700 1 -- 172.21.15.131:0/2933398591 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0000122e0 con 0x556326f056a0 2024-05-10T05:41:24.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.836+0000 7ff005600700 1 -- 172.21.15.131:0/2933398591 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff00001cac0 con 0x556326f056a0 2024-05-10T05:41:24.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.836+0000 7ff036a97b80 10 client.26890 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:24.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.836+0000 7ff036a97b80 10 client.26890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:41:24.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.836+0000 7ff036a97b80 20 client.26890 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:24.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.836+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556326f1b4b0 0x556326f1d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.836+0000 7ff036a97b80 4 client.26890 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:24.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.836+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556326ed4780 con 0x556326f1b4b0 2024-05-10T05:41:24.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.838+0000 7ff017400700 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556326f1b4b0 0x556326f1d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.839+0000 7ff005600700 10 client.26890 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:24.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.839+0000 7ff017400700 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556326f1b4b0 0x556326f1d900 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9074346f38a36a1 server_cookie=12a0ec243b5b44d0 in_seq=0 out_seq=0 2024-05-10T05:41:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.843+0000 7ff005600700 1 -- 172.21.15.131:0/2933398591 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556326ed4780 con 0x556326f1b4b0 2024-05-10T05:41:24.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.843+0000 7ff005600700 10 client.26890 handle_command_reply: tid=0 2024-05-10T05:41:24.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.843+0000 7ff036a97b80 1 client.26890 shutdown 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.843+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 msgr2=0x556326f0f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.843+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f0f6c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff000012850 tx=0x7ff000012c10 comp rx=0 tx=0).stop 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.843+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 shutdown_connections 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.843+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556326f1b4b0 0x556326f1d900 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.844+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f06ec0 0x556326f16b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.844+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f0f6c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.844+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2933398591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 0x556326f0fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.844+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 >> 172.21.15.131:0/2933398591 conn(0x556326e67a60 msgr2=0x556326efffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.844+0000 7ff005600700 0 client.26890 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.844+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 shutdown_connections 2024-05-10T05:41:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.844+0000 7ff036a97b80 1 -- 172.21.15.131:0/2933398591 wait complete. 2024-05-10T05:41:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.844+0000 7ff036a97b80 20 client.26890 trim_cache size 0 max 16384 2024-05-10T05:41:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.876+0000 7ff036a97b80 1 Processor -- start 2024-05-10T05:41:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.877+0000 7ff036a97b80 1 -- start start 2024-05-10T05:41:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.877+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326e67a60 0x556326f14750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.877+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f14c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.877+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 0x556326f0a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.877+0000 7ff036a97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326efa7c0 con 0x556326f06570 2024-05-10T05:41:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.877+0000 7ff036a97b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326f0e410 con 0x556326e67a60 2024-05-10T05:41:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.877+0000 7ff036a97b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326f0e5d0 con 0x556326f056a0 2024-05-10T05:41:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.877+0000 7ff017e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326e67a60 0x556326f14750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.877+0000 7ff017e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326e67a60 0x556326f14750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:59024/0 (socket says 172.21.15.131:59024) 2024-05-10T05:41:24.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.877+0000 7ff017e00700 1 -- 172.21.15.131:0/2637426253 learned_addr learned my addr 172.21.15.131:0/2637426253 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:24.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.878+0000 7ff01cc00700 1 --2- 172.21.15.131:0/2637426253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 0x556326f0a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.878+0000 7ff017400700 1 --2- 172.21.15.131:0/2637426253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f14c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.880+0000 7ff017e00700 1 -- 172.21.15.131:0/2637426253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 msgr2=0x556326f14c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.880+0000 7ff017e00700 1 --2- 172.21.15.131:0/2637426253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f14c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.880+0000 7ff017e00700 1 -- 172.21.15.131:0/2637426253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 msgr2=0x556326f0a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.880+0000 7ff017e00700 1 --2- 172.21.15.131:0/2637426253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 0x556326f0a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.880+0000 7ff017e00700 1 -- 172.21.15.131:0/2637426253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556326deb560 con 0x556326e67a60 2024-05-10T05:41:24.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.880+0000 7ff017400700 1 --2- 172.21.15.131:0/2637426253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f14c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:41:24.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.880+0000 7ff017e00700 1 --2- 172.21.15.131:0/2637426253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326e67a60 0x556326f14750 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff00000de30 tx=0x7ff000000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ae727530ff9276 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.881+0000 7ff00dc00700 1 -- 172.21.15.131:0/2637426253 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000017030 con 0x556326e67a60 2024-05-10T05:41:24.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.881+0000 7ff00dc00700 1 -- 172.21.15.131:0/2637426253 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff000018040 con 0x556326e67a60 2024-05-10T05:41:24.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.881+0000 7ff00dc00700 1 -- 172.21.15.131:0/2637426253 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000e820 con 0x556326e67a60 2024-05-10T05:41:24.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.882+0000 7ff036a97b80 1 -- 172.21.15.131:0/2637426253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326e67a60 msgr2=0x556326f14750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.882+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2637426253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326e67a60 0x556326f14750 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff00000de30 tx=0x7ff000000bc0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.886+0000 7ff036a97b80 1 -- 172.21.15.131:0/2637426253 shutdown_connections 2024-05-10T05:41:24.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.886+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2637426253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326f06570 0x556326f0a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.886+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2637426253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326e67a60 0x556326f14750 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.886+0000 7ff036a97b80 1 --2- 172.21.15.131:0/2637426253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f056a0 0x556326f14c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.886+0000 7ff036a97b80 1 -- 172.21.15.131:0/2637426253 >> 172.21.15.131:0/2637426253 conn(0x556326efc450 msgr2=0x556326f0dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.886+0000 7ff036a97b80 1 -- 172.21.15.131:0/2637426253 shutdown_connections 2024-05-10T05:41:24.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.886+0000 7ff036a97b80 1 -- 172.21.15.131:0/2637426253 wait complete. 2024-05-10T05:41:24.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.887+0000 7ff036a97b80 1 -- start start 2024-05-10T05:41:24.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.887+0000 7ff036a97b80 1 Processor -- start 2024-05-10T05:41:24.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.890+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326e67a60 0x556326f14300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.891+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 0x556326f14860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.891+0000 7ff036a97b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f06570 0x556326f6fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.891+0000 7ff017e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326e67a60 0x556326f14300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.891+0000 7ff036a97b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326f70400 con 0x556326e67a60 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.891+0000 7ff017e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326e67a60 0x556326f14300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38548/0 (socket says 172.21.15.131:38548) 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.891+0000 7ff017e00700 1 -- 172.21.15.131:0/327477465 learned_addr learned my addr 172.21.15.131:0/327477465 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.891+0000 7ff036a97b80 1 -- 172.21.15.131:0/327477465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326f70570 con 0x556326f056a0 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.891+0000 7ff036a97b80 1 -- 172.21.15.131:0/327477465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326f70730 con 0x556326f06570 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.891+0000 7ff01cc00700 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f06570 0x556326f6fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.891+0000 7ff036a97b80 10 client.0 _ll_register_callbacks cb 0x556326dcab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.892+0000 7ff017e00700 1 -- 172.21.15.131:0/327477465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f06570 msgr2=0x556326f6fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.892+0000 7ff017e00700 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f06570 0x556326f6fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.892+0000 7ff017e00700 1 -- 172.21.15.131:0/327477465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 msgr2=0x556326f14860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.892+0000 7ff017e00700 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 0x556326f14860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.892+0000 7ff017e00700 1 -- 172.21.15.131:0/327477465 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556326c52b60 con 0x556326e67a60 2024-05-10T05:41:24.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.892+0000 7ff017e00700 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326e67a60 0x556326f14300 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff00000f6d0 tx=0x7ff00000f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a25714af7ccdf9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:24.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.895+0000 7ff005600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:41:24.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.895+0000 7ff005600700 1 -- 172.21.15.131:0/327477465 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000017040 con 0x556326e67a60 2024-05-10T05:41:24.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.895+0000 7ff005600700 1 -- 172.21.15.131:0/327477465 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff00002a020 con 0x556326e67a60 2024-05-10T05:41:24.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.895+0000 7ff036a97b80 1 -- 172.21.15.131:0/327477465 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0100075c0 con 0x556326e67a60 2024-05-10T05:41:24.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.900+0000 7ff005600700 1 -- 172.21.15.131:0/327477465 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff00000d700 con 0x556326e67a60 2024-05-10T05:41:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.900+0000 7ff036a97b80 10 client.16587 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.900+0000 7ff036a97b80 1 -- 172.21.15.131:0/327477465 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556326caacd0 con 0x556326e67a60 2024-05-10T05:41:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.901+0000 7ff005600700 1 -- 172.21.15.131:0/327477465 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0100075c0 con 0x556326e67a60 2024-05-10T05:41:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.903+0000 7ff005600700 1 -- 172.21.15.131:0/327477465 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff00001cac0 con 0x556326e67a60 2024-05-10T05:41:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.903+0000 7ff036a97b80 10 client.16587 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.903+0000 7ff036a97b80 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:41:24.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.903+0000 7ff036a97b80 20 client.16587 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:24.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.903+0000 7ff036a97b80 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556326f06ec0 0x556326f73bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:24.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.903+0000 7ff036a97b80 4 client.16587 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:41:24.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.904+0000 7ff036a97b80 1 -- 172.21.15.131:0/327477465 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556326e1dec0 con 0x556326f06ec0 2024-05-10T05:41:24.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.904+0000 7ff017400700 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556326f06ec0 0x556326f73bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:24.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.904+0000 7ff005600700 10 client.16587 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:24.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.904+0000 7ff017400700 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556326f06ec0 0x556326f73bc0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57affb57667dce56 server_cookie=f65d1f555df79e0f in_seq=0 out_seq=0 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.905+0000 7ff005600700 1 -- 172.21.15.131:0/327477465 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556326e1dec0 con 0x556326f06ec0 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.905+0000 7ff005600700 10 client.16587 handle_command_reply: tid=0 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.905+0000 7ff036a97b80 1 client.16587 shutdown 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.906+0000 7ff036a97b80 1 -- 172.21.15.131:0/327477465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326e67a60 msgr2=0x556326f14300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.906+0000 7ff036a97b80 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326e67a60 0x556326f14300 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff00000f6d0 tx=0x7ff00000f700 comp rx=0 tx=0).stop 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.906+0000 7ff036a97b80 1 -- 172.21.15.131:0/327477465 shutdown_connections 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.906+0000 7ff036a97b80 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x556326f06ec0 0x556326f73bc0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.906+0000 7ff036a97b80 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556326e67a60 0x556326f14300 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.906+0000 7ff036a97b80 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556326f06570 0x556326f6fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.906+0000 7ff036a97b80 1 --2- 172.21.15.131:0/327477465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556326f056a0 0x556326f14860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.906+0000 7ff036a97b80 1 -- 172.21.15.131:0/327477465 >> 172.21.15.131:0/327477465 conn(0x556326efc450 msgr2=0x556326f1d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.906+0000 7ff005600700 0 client.16587 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:24.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.907+0000 7ff036a97b80 1 -- 172.21.15.131:0/327477465 shutdown_connections 2024-05-10T05:41:24.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.907+0000 7ff036a97b80 1 -- 172.21.15.131:0/327477465 wait complete. 2024-05-10T05:41:24.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.911+0000 7ff036a97b80 20 client.16587 trim_cache size 0 max 16384 2024-05-10T05:41:24.912 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:41:24.912 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:41:24.912 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:41:24.912 INFO:teuthology.orchestra.run.smithi131.stdout: "5125e731-f048-4110-a811-1ab783e5533b": { 2024-05-10T05:41:24.912 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:41:24.912 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "5125e731-f048-4110-a811-1ab783e5533b", 2024-05-10T05:41:24.913 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:41:24.913 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:41:24.913 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:41:24.913 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:41:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.912+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feffc0a11d0 msgr2=0x7feffc0a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.912+0000 7ff01ec00700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feffc0a11d0 0x7feffc0a36e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff0100062a0 tx=0x7ff010007b30 comp rx=0 tx=0).stop 2024-05-10T05:41:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.912+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff018108bb0 msgr2=0x7ff018082b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.912+0000 7ff01ec00700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff018108bb0 0x7ff018082b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff00800d950 tx=0x7ff00800dc60 comp rx=0 tx=0).stop 2024-05-10T05:41:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.913+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 shutdown_connections 2024-05-10T05:41:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.913+0000 7ff01ec00700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7feffc0a11d0 0x7feffc0a36e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.913+0000 7ff01ec00700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff018109710 0x7ff018083040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.913+0000 7ff01ec00700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff018108210 0x7ff0180825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.913+0000 7ff01ec00700 1 --2- 172.21.15.131:0/1620895415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff018108bb0 0x7ff018082b00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:24.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.913+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 >> 172.21.15.131:0/1620895415 conn(0x7ff01806ea00 msgr2=0x7ff018075300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:24.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.913+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 shutdown_connections 2024-05-10T05:41:24.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:24.913+0000 7ff01ec00700 1 -- 172.21.15.131:0/1620895415 wait complete. 2024-05-10T05:41:24.999 INFO:tasks.cephfs.filesystem:scrub status for tag:5125e731-f048-4110-a811-1ab783e5533b - {'path': '/', 'tag': '5125e731-f048-4110-a811-1ab783e5533b', 'options': 'recursive,force'} 2024-05-10T05:41:25.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:25 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:41:25.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:25 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:41:25.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:25 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:41:25.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:25 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:41:25.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:25 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:41:25.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:25 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:41:25.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:25 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:41:25.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:25 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:41:25.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:25 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:41:25.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:25 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:41:25.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:25 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:41:25.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:25 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:41:25.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:25 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:41:25.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:25 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:41:25.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:25 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:41:25.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:25 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:41:25.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:25 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:41:25.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:25 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:41:26.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:26 smithi131 ceph-mon[37264]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 267 MiB/s wr, 79 op/s 2024-05-10T05:41:26.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:26 smithi157 ceph-mon[78427]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 267 MiB/s wr, 79 op/s 2024-05-10T05:41:26.900 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:26 smithi142 ceph-mon[77667]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 267 MiB/s wr, 79 op/s 2024-05-10T05:41:28.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:28 smithi131 ceph-mon[37264]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 224 MiB/s wr, 70 op/s 2024-05-10T05:41:28.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:28 smithi157 ceph-mon[78427]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 224 MiB/s wr, 70 op/s 2024-05-10T05:41:28.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:28 smithi142 ceph-mon[77667]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 224 MiB/s wr, 70 op/s 2024-05-10T05:41:30.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:30 smithi131 ceph-mon[37264]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 75 op/s 2024-05-10T05:41:30.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:30 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:41:30.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:30 smithi157 ceph-mon[78427]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 75 op/s 2024-05-10T05:41:30.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:30 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:41:30.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:30 smithi142 ceph-mon[77667]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 75 op/s 2024-05-10T05:41:30.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:30 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:41:31.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:31 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:41:31.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:31 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:41:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:31 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:41:32.806 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:32 smithi157 ceph-mon[78427]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 262 MiB/s wr, 81 op/s 2024-05-10T05:41:32.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:32 smithi142 ceph-mon[77667]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 262 MiB/s wr, 81 op/s 2024-05-10T05:41:32.968 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:32 smithi131 ceph-mon[37264]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 262 MiB/s wr, 81 op/s 2024-05-10T05:41:33.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:33 smithi142 ceph-mon[77667]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 221 MiB/s wr, 68 op/s 2024-05-10T05:41:33.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:33 smithi131 ceph-mon[37264]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 221 MiB/s wr, 68 op/s 2024-05-10T05:41:34.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:33 smithi157 ceph-mon[78427]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 221 MiB/s wr, 68 op/s 2024-05-10T05:41:36.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:36 smithi142 ceph-mon[77667]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 286 MiB/s wr, 86 op/s 2024-05-10T05:41:36.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:36 smithi131 ceph-mon[37264]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 286 MiB/s wr, 86 op/s 2024-05-10T05:41:36.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:36 smithi157 ceph-mon[78427]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 286 MiB/s wr, 86 op/s 2024-05-10T05:41:38.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:38 smithi142 ceph-mon[77667]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:41:38.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:38 smithi131 ceph-mon[37264]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:41:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:38 smithi157 ceph-mon[78427]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:41:40.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:40 smithi142 ceph-mon[77667]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 269 MiB/s wr, 81 op/s 2024-05-10T05:41:40.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:40 smithi131 ceph-mon[37264]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 269 MiB/s wr, 81 op/s 2024-05-10T05:41:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:40 smithi157 ceph-mon[78427]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 269 MiB/s wr, 81 op/s 2024-05-10T05:41:42.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:42 smithi131 ceph-mon[37264]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 281 MiB/s wr, 87 op/s 2024-05-10T05:41:42.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:42 smithi157 ceph-mon[78427]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 281 MiB/s wr, 87 op/s 2024-05-10T05:41:42.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:42 smithi142 ceph-mon[77667]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 281 MiB/s wr, 87 op/s 2024-05-10T05:41:44.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:44 smithi131 ceph-mon[37264]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:41:44.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:44 smithi157 ceph-mon[78427]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:41:44.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:44 smithi142 ceph-mon[77667]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:41:46.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:46 smithi131 ceph-mon[37264]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 311 MiB/s wr, 93 op/s 2024-05-10T05:41:46.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:46 smithi157 ceph-mon[78427]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 311 MiB/s wr, 93 op/s 2024-05-10T05:41:46.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:46 smithi142 ceph-mon[77667]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 311 MiB/s wr, 93 op/s 2024-05-10T05:41:48.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:48 smithi131 ceph-mon[37264]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 252 MiB/s wr, 78 op/s 2024-05-10T05:41:48.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:48 smithi157 ceph-mon[78427]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 252 MiB/s wr, 78 op/s 2024-05-10T05:41:48.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:48 smithi142 ceph-mon[77667]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 252 MiB/s wr, 78 op/s 2024-05-10T05:41:50.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:50 smithi131 ceph-mon[37264]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 275 MiB/s wr, 83 op/s 2024-05-10T05:41:50.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:50 smithi157 ceph-mon[78427]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 275 MiB/s wr, 83 op/s 2024-05-10T05:41:50.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:50 smithi142 ceph-mon[77667]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 275 MiB/s wr, 83 op/s 2024-05-10T05:41:52.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:52 smithi157 ceph-mon[78427]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 289 MiB/s wr, 89 op/s 2024-05-10T05:41:52.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:52 smithi142 ceph-mon[77667]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 289 MiB/s wr, 89 op/s 2024-05-10T05:41:52.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:52 smithi131 ceph-mon[37264]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 289 MiB/s wr, 89 op/s 2024-05-10T05:41:54.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:54 smithi157 ceph-mon[78427]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:41:54.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:54 smithi142 ceph-mon[77667]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:41:54.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:54 smithi131 ceph-mon[37264]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 243 MiB/s wr, 73 op/s 2024-05-10T05:41:55.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:41:55.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:55 smithi142 ceph-mon[77667]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 302 MiB/s wr, 92 op/s 2024-05-10T05:41:55.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.897+0000 7fd605400700 1 -- 172.21.15.131:0/97864715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd600108630 msgr2=0x7fd600108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:55.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.897+0000 7fd605400700 1 --2- 172.21.15.131:0/97864715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd600108630 0x7fd600108a40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800b0a0 tx=0x7fd5f800b3b0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.898+0000 7fd605400700 1 -- 172.21.15.131:0/97864715 shutdown_connections 2024-05-10T05:41:55.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.898+0000 7fd605400700 1 --2- 172.21.15.131:0/97864715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd600109c50 0x7fd60010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.898+0000 7fd605400700 1 --2- 172.21.15.131:0/97864715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd600108630 0x7fd600108a40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.898+0000 7fd605400700 1 --2- 172.21.15.131:0/97864715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd600109300 0x7fd600109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.898+0000 7fd605400700 1 -- 172.21.15.131:0/97864715 >> 172.21.15.131:0/97864715 conn(0x7fd60006ea00 msgr2=0x7fd600070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:55.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.905+0000 7fd605400700 1 -- 172.21.15.131:0/97864715 shutdown_connections 2024-05-10T05:41:55.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.905+0000 7fd605400700 1 -- 172.21.15.131:0/97864715 wait complete. 2024-05-10T05:41:55.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.907+0000 7fd605400700 1 Processor -- start 2024-05-10T05:41:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.907+0000 7fd605400700 1 -- start start 2024-05-10T05:41:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.907+0000 7fd605400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd600108630 0x7fd6001a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd605400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd600109300 0x7fd6001a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd605400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd600109c50 0x7fd6001ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd605400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001ae830 con 0x7fd600108630 2024-05-10T05:41:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd605400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001ae9a0 con 0x7fd600109c50 2024-05-10T05:41:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd605400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001aeb60 con 0x7fd600109300 2024-05-10T05:41:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd5fde00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd600109300 0x7fd6001a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd5fde00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd600109300 0x7fd6001a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45072/0 (socket says 172.21.15.131:45072) 2024-05-10T05:41:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd5fde00700 1 -- 172.21.15.131:0/3717452018 learned_addr learned my addr 172.21.15.131:0/3717452018 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:55.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd5ff200700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd600109c50 0x7fd6001ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd5fe800700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd600108630 0x7fd6001a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd5fde00700 1 -- 172.21.15.131:0/3717452018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd600109c50 msgr2=0x7fd6001ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd5fde00700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd600109c50 0x7fd6001ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd5fde00700 1 -- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd600108630 msgr2=0x7fd6001a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd5fde00700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd600108630 0x7fd6001a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.908+0000 7fd5fde00700 1 -- 172.21.15.131:0/3717452018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f0009520 con 0x7fd600109300 2024-05-10T05:41:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.909+0000 7fd5fe800700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd600108630 0x7fd6001a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.910+0000 7fd5fde00700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd600109300 0x7fd6001a72f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d950 tx=0x7fd5ec00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.910+0000 7fd5f7400700 1 -- 172.21.15.131:0/3717452018 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec009b40 con 0x7fd600109300 2024-05-10T05:41:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.910+0000 7fd5f7400700 1 -- 172.21.15.131:0/3717452018 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5ec009ca0 con 0x7fd600109300 2024-05-10T05:41:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.910+0000 7fd5f7400700 1 -- 172.21.15.131:0/3717452018 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00f620 con 0x7fd600109300 2024-05-10T05:41:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.911+0000 7fd605400700 1 -- 172.21.15.131:0/3717452018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f8009d00 con 0x7fd600109300 2024-05-10T05:41:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.911+0000 7fd605400700 1 -- 172.21.15.131:0/3717452018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6001af170 con 0x7fd600109300 2024-05-10T05:41:55.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.924+0000 7fd5f7400700 1 -- 172.21.15.131:0/3717452018 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd5ec010460 con 0x7fd600109300 2024-05-10T05:41:55.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.927+0000 7fd5f7400700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd5e40a1240 0x7fd5e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.927+0000 7fd5f7400700 1 -- 172.21.15.131:0/3717452018 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd5ec0ccc40 con 0x7fd600109300 2024-05-10T05:41:55.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.930+0000 7fd61d269b80 1 Processor -- start 2024-05-10T05:41:55.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.930+0000 7fd5fe800700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd5e40a1240 0x7fd5e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:55.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.931+0000 7fd61d269b80 1 -- start start 2024-05-10T05:41:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.932+0000 7fd5fe800700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd5e40a1240 0x7fd5e40a3750 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8000f80 tx=0x7fd5f8007ab0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.932+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd16a0 0x559e4dbd1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.932+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2570 0x559e4dbd2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.932+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2ec0 0x559e4dbd97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.932+0000 7fd61d269b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4daf8150 con 0x559e4dbd2570 2024-05-10T05:41:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.932+0000 7fd61d269b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4daf5b10 con 0x559e4dbd2ec0 2024-05-10T05:41:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.932+0000 7fd61d269b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4daf1400 con 0x559e4dbd16a0 2024-05-10T05:41:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.933+0000 7fd5fe800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd16a0 0x559e4dbd1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.933+0000 7fd5fe800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd16a0 0x559e4dbd1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45086/0 (socket says 172.21.15.131:45086) 2024-05-10T05:41:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.933+0000 7fd5fe800700 1 -- 172.21.15.131:0/3236154202 learned_addr learned my addr 172.21.15.131:0/3236154202 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.933+0000 7fd5fe800700 1 -- 172.21.15.131:0/3236154202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2ec0 msgr2=0x559e4dbd97c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.933+0000 7fd5fe800700 1 --2- 172.21.15.131:0/3236154202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2ec0 0x559e4dbd97c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.933+0000 7fd5fe800700 1 -- 172.21.15.131:0/3236154202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2570 msgr2=0x559e4dbd2980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:55.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.933+0000 7fd5fe800700 1 --2- 172.21.15.131:0/3236154202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2570 0x559e4dbd2980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.933+0000 7fd5fe800700 1 -- 172.21.15.131:0/3236154202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e4d9723b0 con 0x559e4dbd16a0 2024-05-10T05:41:55.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.934+0000 7fd5fe800700 1 --2- 172.21.15.131:0/3236154202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd16a0 0x559e4dbd1af0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800b070 tx=0x7fd5f801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc9252c162a7bf0d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:55.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.934+0000 7fd5eb400700 1 -- 172.21.15.131:0/3236154202 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800e040 con 0x559e4dbd16a0 2024-05-10T05:41:55.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.934+0000 7fd5eb400700 1 -- 172.21.15.131:0/3236154202 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5f801d450 con 0x559e4dbd16a0 2024-05-10T05:41:55.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.934+0000 7fd5eb400700 1 -- 172.21.15.131:0/3236154202 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80052a0 con 0x559e4dbd16a0 2024-05-10T05:41:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.936+0000 7fd61d269b80 1 -- 172.21.15.131:0/3236154202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd16a0 msgr2=0x559e4dbd1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.936+0000 7fd61d269b80 1 --2- 172.21.15.131:0/3236154202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd16a0 0x559e4dbd1af0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800b070 tx=0x7fd5f801a000 comp rx=0 tx=0).stop 2024-05-10T05:41:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.938+0000 7fd61d269b80 1 -- 172.21.15.131:0/3236154202 shutdown_connections 2024-05-10T05:41:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.938+0000 7fd61d269b80 1 --2- 172.21.15.131:0/3236154202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2ec0 0x559e4dbd97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.938+0000 7fd61d269b80 1 --2- 172.21.15.131:0/3236154202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd16a0 0x559e4dbd1af0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.938+0000 7fd61d269b80 1 --2- 172.21.15.131:0/3236154202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2570 0x559e4dbd2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.938+0000 7fd61d269b80 1 -- 172.21.15.131:0/3236154202 >> 172.21.15.131:0/3236154202 conn(0x559e4db33a60 msgr2=0x559e4dbcbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.938+0000 7fd61d269b80 1 -- 172.21.15.131:0/3236154202 shutdown_connections 2024-05-10T05:41:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.939+0000 7fd61d269b80 1 -- 172.21.15.131:0/3236154202 wait complete. 2024-05-10T05:41:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.939+0000 7fd61d269b80 1 -- start start 2024-05-10T05:41:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.940+0000 7fd61d269b80 1 Processor -- start 2024-05-10T05:41:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.941+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd16a0 0x559e4dbdb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.941+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd2570 0x559e4dbdbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.941+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 0x559e4dbe2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd61d269b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4da8d560 con 0x559e4dbd2ec0 2024-05-10T05:41:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd61d269b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4da8d200 con 0x559e4dbd16a0 2024-05-10T05:41:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd61d269b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4d9746a0 con 0x559e4dbd2570 2024-05-10T05:41:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd5fe800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd16a0 0x559e4dbdb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd5fe800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd16a0 0x559e4dbdb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47782/0 (socket says 172.21.15.131:47782) 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd5fe800700 1 -- 172.21.15.131:0/4072801817 learned_addr learned my addr 172.21.15.131:0/4072801817 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd61d269b80 10 client.0 _ll_register_callbacks cb 0x559e4da96b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd5fe800700 1 -- 172.21.15.131:0/4072801817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd2570 msgr2=0x559e4dbdbaa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd5fe800700 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd2570 0x559e4dbdbaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd5fe800700 1 -- 172.21.15.131:0/4072801817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 msgr2=0x559e4dbe2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.942+0000 7fd5ff200700 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 0x559e4dbe2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.943+0000 7fd5fe800700 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 0x559e4dbe2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.943+0000 7fd5fe800700 1 -- 172.21.15.131:0/4072801817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60010efc0 con 0x559e4dbd16a0 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.943+0000 7fd5ff200700 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 0x559e4dbe2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.943+0000 7fd5fe800700 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd16a0 0x559e4dbdb560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800afe0 tx=0x7fd5f801de60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b7f49017540d14 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.943+0000 7fd5dfe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:41:55.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.943+0000 7fd5dfe00700 1 -- 172.21.15.131:0/4072801817 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800e040 con 0x559e4dbd16a0 2024-05-10T05:41:55.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.944+0000 7fd5dfe00700 1 -- 172.21.15.131:0/4072801817 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd5f8036030 con 0x559e4dbd16a0 2024-05-10T05:41:55.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.944+0000 7fd61d269b80 1 -- 172.21.15.131:0/4072801817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e4da4a270 con 0x559e4dbd16a0 2024-05-10T05:41:55.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.944+0000 7fd5dfe00700 1 -- 172.21.15.131:0/4072801817 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5f8030450 con 0x559e4dbd16a0 2024-05-10T05:41:55.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.946+0000 7fd61d269b80 10 client.26858 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:55.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.946+0000 7fd61d269b80 1 -- 172.21.15.131:0/4072801817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559e4d91eb60 con 0x559e4dbd16a0 2024-05-10T05:41:55.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.946+0000 7fd5dfe00700 1 -- 172.21.15.131:0/4072801817 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e4da4a270 con 0x559e4dbd16a0 2024-05-10T05:41:55.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.946+0000 7fd5dfe00700 1 -- 172.21.15.131:0/4072801817 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5f80052f0 con 0x559e4dbd16a0 2024-05-10T05:41:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.947+0000 7fd61d269b80 10 client.26858 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.947+0000 7fd61d269b80 10 client.26858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:41:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.947+0000 7fd61d269b80 20 client.26858 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.947+0000 7fd61d269b80 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559e4dbf7d70 0x559e4dbfa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.947+0000 7fd61d269b80 4 client.26858 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.947+0000 7fd61d269b80 1 -- 172.21.15.131:0/4072801817 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e4dba0780 con 0x559e4dbf7d70 2024-05-10T05:41:55.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.949+0000 7fd5fde00700 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559e4dbf7d70 0x559e4dbfa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:55.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.952+0000 7fd5dfe00700 10 client.26858 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:55.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.952+0000 7fd5fde00700 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559e4dbf7d70 0x559e4dbfa1c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4776ff8f8d2d9de2 server_cookie=6a351576dd55ce96 in_seq=0 out_seq=0 2024-05-10T05:41:55.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.952+0000 7fd5dfe00700 1 -- 172.21.15.131:0/4072801817 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e4dba0780 con 0x559e4dbf7d70 2024-05-10T05:41:55.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.952+0000 7fd5dfe00700 10 client.26858 handle_command_reply: tid=0 2024-05-10T05:41:55.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.952+0000 7fd61d269b80 1 client.26858 shutdown 2024-05-10T05:41:55.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd61d269b80 1 -- 172.21.15.131:0/4072801817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd16a0 msgr2=0x559e4dbdb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd61d269b80 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd16a0 0x559e4dbdb560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800afe0 tx=0x7fd5f801de60 comp rx=0 tx=0).stop 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd61d269b80 1 -- 172.21.15.131:0/4072801817 shutdown_connections 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd61d269b80 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559e4dbf7d70 0x559e4dbfa1c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd61d269b80 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 0x559e4dbe2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd61d269b80 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd16a0 0x559e4dbdb560 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd61d269b80 1 --2- 172.21.15.131:0/4072801817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4dbd2570 0x559e4dbdbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd61d269b80 1 -- 172.21.15.131:0/4072801817 >> 172.21.15.131:0/4072801817 conn(0x559e4db33a60 msgr2=0x559e4dbcbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd5dfe00700 0 client.26858 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd61d269b80 1 -- 172.21.15.131:0/4072801817 shutdown_connections 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.953+0000 7fd61d269b80 1 -- 172.21.15.131:0/4072801817 wait complete. 2024-05-10T05:41:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:55.954+0000 7fd61d269b80 20 client.26858 trim_cache size 0 max 16384 2024-05-10T05:41:55.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:55 smithi131 ceph-mon[37264]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 302 MiB/s wr, 92 op/s 2024-05-10T05:41:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.021+0000 7fd61d269b80 1 Processor -- start 2024-05-10T05:41:56.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.023+0000 7fd61d269b80 1 -- start start 2024-05-10T05:41:56.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.023+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 0x559e4dbfa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.023+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd16a0 0x559e4dbe2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.023+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2570 0x559e4dbd9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.023+0000 7fd61d269b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4dba07b0 con 0x559e4dbd16a0 2024-05-10T05:41:56.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.023+0000 7fd61d269b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4da4a2a0 con 0x559e4dbd2570 2024-05-10T05:41:56.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.023+0000 7fd61d269b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4dbccd20 con 0x559e4db33a60 2024-05-10T05:41:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.024+0000 7fd5fde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd16a0 0x559e4dbe2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.024+0000 7fd5fde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd16a0 0x559e4dbe2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57566/0 (socket says 172.21.15.131:57566) 2024-05-10T05:41:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.024+0000 7fd5fde00700 1 -- 172.21.15.131:0/3390543011 learned_addr learned my addr 172.21.15.131:0/3390543011 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.024+0000 7fd5fde00700 1 -- 172.21.15.131:0/3390543011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 msgr2=0x559e4dbfa490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:41:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.024+0000 7fd5fde00700 1 --2- 172.21.15.131:0/3390543011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 0x559e4dbfa490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.024+0000 7fd5fde00700 1 -- 172.21.15.131:0/3390543011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2570 msgr2=0x559e4dbd9970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.024+0000 7fd5fde00700 1 --2- 172.21.15.131:0/3390543011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2570 0x559e4dbd9970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.024+0000 7fd5fde00700 1 -- 172.21.15.131:0/3390543011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e4dbf4220 con 0x559e4dbd16a0 2024-05-10T05:41:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.025+0000 7fd5fde00700 1 --2- 172.21.15.131:0/3390543011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd16a0 0x559e4dbe2990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec096590 tx=0x7fd5ec0999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a86c4b6a1b5b18 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.025+0000 7fd5ebe00700 1 -- 172.21.15.131:0/3390543011 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec082070 con 0x559e4dbd16a0 2024-05-10T05:41:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.025+0000 7fd5ebe00700 1 -- 172.21.15.131:0/3390543011 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5ec09d050 con 0x559e4dbd16a0 2024-05-10T05:41:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.025+0000 7fd5ebe00700 1 -- 172.21.15.131:0/3390543011 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec010710 con 0x559e4dbd16a0 2024-05-10T05:41:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.026+0000 7fd61d269b80 1 -- 172.21.15.131:0/3390543011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd16a0 msgr2=0x559e4dbe2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.026+0000 7fd61d269b80 1 --2- 172.21.15.131:0/3390543011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd16a0 0x559e4dbe2990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec096590 tx=0x7fd5ec0999f0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.027+0000 7fd61d269b80 1 -- 172.21.15.131:0/3390543011 shutdown_connections 2024-05-10T05:41:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.027+0000 7fd61d269b80 1 --2- 172.21.15.131:0/3390543011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2570 0x559e4dbd9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.027+0000 7fd61d269b80 1 --2- 172.21.15.131:0/3390543011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 0x559e4dbfa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.027+0000 7fd61d269b80 1 --2- 172.21.15.131:0/3390543011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd16a0 0x559e4dbe2990 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.027+0000 7fd61d269b80 1 -- 172.21.15.131:0/3390543011 >> 172.21.15.131:0/3390543011 conn(0x559e4dbc8450 msgr2=0x559e4dbf9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.027+0000 7fd61d269b80 1 -- 172.21.15.131:0/3390543011 shutdown_connections 2024-05-10T05:41:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.027+0000 7fd61d269b80 1 -- 172.21.15.131:0/3390543011 wait complete. 2024-05-10T05:41:56.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.029+0000 7fd61d269b80 1 -- start start 2024-05-10T05:41:56.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.030+0000 7fd61d269b80 1 Processor -- start 2024-05-10T05:41:56.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.033+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 0x559e4dbe0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.033+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2570 0x559e4dbe0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.033+0000 7fd61d269b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 0x559e4dbe1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.033+0000 7fd5fe800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 0x559e4dbe0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.033+0000 7fd61d269b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4dbe1ff0 con 0x559e4dbd2ec0 2024-05-10T05:41:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.033+0000 7fd61d269b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4dbe2160 con 0x559e4dbd2570 2024-05-10T05:41:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.033+0000 7fd61d269b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4dbe2320 con 0x559e4db33a60 2024-05-10T05:41:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.033+0000 7fd5fe800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 0x559e4dbe0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45106/0 (socket says 172.21.15.131:45106) 2024-05-10T05:41:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.033+0000 7fd5fe800700 1 -- 172.21.15.131:0/2524386835 learned_addr learned my addr 172.21.15.131:0/2524386835 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.034+0000 7fd61d269b80 10 client.0 _ll_register_callbacks cb 0x559e4da96b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.034+0000 7fd5ff200700 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 0x559e4dbe1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.034+0000 7fd5fde00700 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2570 0x559e4dbe0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.034+0000 7fd5fe800700 1 -- 172.21.15.131:0/2524386835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2570 msgr2=0x559e4dbe0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.034+0000 7fd5fe800700 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2570 0x559e4dbe0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.034+0000 7fd5fe800700 1 -- 172.21.15.131:0/2524386835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 msgr2=0x559e4dbe1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.034+0000 7fd5fe800700 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 0x559e4dbe1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.034+0000 7fd5fe800700 1 -- 172.21.15.131:0/2524386835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6001af170 con 0x559e4db33a60 2024-05-10T05:41:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.034+0000 7fd5fe800700 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 0x559e4dbe0640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005700 tx=0x7fd5f801d650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c983ee38c30e6d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.034+0000 7fd5dfe00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:41:56.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.036+0000 7fd5dfe00700 1 -- 172.21.15.131:0/2524386835 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800e040 con 0x559e4db33a60 2024-05-10T05:41:56.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.036+0000 7fd61d269b80 1 -- 172.21.15.131:0/2524386835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5f801d450 con 0x559e4db33a60 2024-05-10T05:41:56.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.036+0000 7fd5dfe00700 1 -- 172.21.15.131:0/2524386835 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd5f8036070 con 0x559e4db33a60 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.036+0000 7fd5dfe00700 1 -- 172.21.15.131:0/2524386835 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5f8005410 con 0x559e4db33a60 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.036+0000 7fd61d269b80 10 client.26908 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.036+0000 7fd61d269b80 1 -- 172.21.15.131:0/2524386835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559e4dab7560 con 0x559e4db33a60 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.036+0000 7fd5dfe00700 1 -- 172.21.15.131:0/2524386835 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5f8005920 con 0x559e4db33a60 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.037+0000 7fd5dfe00700 1 -- 172.21.15.131:0/2524386835 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5f8046440 con 0x559e4db33a60 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.037+0000 7fd61d269b80 10 client.26908 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.037+0000 7fd61d269b80 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.037+0000 7fd61d269b80 20 client.26908 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.037+0000 7fd61d269b80 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559e4dc4eb70 0x559e4dc50f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.037+0000 7fd61d269b80 4 client.26908 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:41:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.037+0000 7fd61d269b80 1 -- 172.21.15.131:0/2524386835 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fd5f801d450 con 0x559e4dc4eb70 2024-05-10T05:41:56.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.040+0000 7fd5fde00700 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559e4dc4eb70 0x559e4dc50f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:56.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.040+0000 7fd5fde00700 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559e4dc4eb70 0x559e4dc50f90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b8d3959bb3641e server_cookie=ab878d63f28bc80b in_seq=0 out_seq=0 2024-05-10T05:41:56.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.040+0000 7fd5dfe00700 10 client.26908 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:56.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.040+0000 7fd5dfe00700 1 -- 172.21.15.131:0/2524386835 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fd5f801d450 con 0x559e4dc4eb70 2024-05-10T05:41:56.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.040+0000 7fd5dfe00700 10 client.26908 handle_command_reply: tid=0 2024-05-10T05:41:56.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.040+0000 7fd61d269b80 1 client.26908 shutdown 2024-05-10T05:41:56.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.041+0000 7fd61d269b80 1 -- 172.21.15.131:0/2524386835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 msgr2=0x559e4dbe0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:56.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.041+0000 7fd61d269b80 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 0x559e4dbe0640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005700 tx=0x7fd5f801d650 comp rx=0 tx=0).stop 2024-05-10T05:41:56.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.041+0000 7fd61d269b80 1 -- 172.21.15.131:0/2524386835 shutdown_connections 2024-05-10T05:41:56.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.041+0000 7fd61d269b80 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559e4dc4eb70 0x559e4dc50f90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.042+0000 7fd61d269b80 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e4dbd2ec0 0x559e4dbe1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.042+0000 7fd5dfe00700 0 client.26908 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:41:56.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.042+0000 7fd61d269b80 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e4db33a60 0x559e4dbe0640 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.042+0000 7fd61d269b80 1 --2- 172.21.15.131:0/2524386835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e4dbd2570 0x559e4dbe0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.042+0000 7fd61d269b80 1 -- 172.21.15.131:0/2524386835 >> 172.21.15.131:0/2524386835 conn(0x559e4dbc8450 msgr2=0x559e4dbd9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:56.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.043+0000 7fd61d269b80 1 -- 172.21.15.131:0/2524386835 shutdown_connections 2024-05-10T05:41:56.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.043+0000 7fd61d269b80 1 -- 172.21.15.131:0/2524386835 wait complete. 2024-05-10T05:41:56.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.044+0000 7fd61d269b80 20 client.26908 trim_cache size 0 max 16384 2024-05-10T05:41:56.046 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:41:56.046 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:41:56.046 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:41:56.046 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:41:56.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.045+0000 7fd605400700 1 -- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd5e40a1240 msgr2=0x7fd5e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:56.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.045+0000 7fd605400700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd5e40a1240 0x7fd5e40a3750 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8000f80 tx=0x7fd5f8007ab0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.045+0000 7fd605400700 1 -- 172.21.15.131:0/3717452018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd600109300 msgr2=0x7fd6001a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:56.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.045+0000 7fd605400700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd600109300 0x7fd6001a72f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d950 tx=0x7fd5ec00dd10 comp rx=0 tx=0).stop 2024-05-10T05:41:56.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.046+0000 7fd605400700 1 -- 172.21.15.131:0/3717452018 shutdown_connections 2024-05-10T05:41:56.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.046+0000 7fd605400700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd5e40a1240 0x7fd5e40a3750 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.046+0000 7fd605400700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd600108630 0x7fd6001a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.046+0000 7fd605400700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd600109c50 0x7fd6001ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.046+0000 7fd605400700 1 --2- 172.21.15.131:0/3717452018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd600109300 0x7fd6001a72f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.046+0000 7fd605400700 1 -- 172.21.15.131:0/3717452018 >> 172.21.15.131:0/3717452018 conn(0x7fd60006ea00 msgr2=0x7fd60006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:56.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.046+0000 7fd605400700 1 -- 172.21.15.131:0/3717452018 shutdown_connections 2024-05-10T05:41:56.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.046+0000 7fd605400700 1 -- 172.21.15.131:0/3717452018 wait complete. 2024-05-10T05:41:56.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:55 smithi157 ceph-mon[78427]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 302 MiB/s wr, 92 op/s 2024-05-10T05:41:56.102 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:41:56.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:41:56.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.913+0000 7fca66600700 1 -- 172.21.15.131:0/1215125700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca6010ee70 msgr2=0x7fca60111310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:56.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.913+0000 7fca66600700 1 --2- 172.21.15.131:0/1215125700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca6010ee70 0x7fca60111310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fca500099e0 tx=0x7fca50009cf0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.915+0000 7fca66600700 1 -- 172.21.15.131:0/1215125700 shutdown_connections 2024-05-10T05:41:56.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.915+0000 7fca66600700 1 --2- 172.21.15.131:0/1215125700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca6010ee70 0x7fca60111310 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.915+0000 7fca66600700 1 --2- 172.21.15.131:0/1215125700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60109bb0 0x7fca6010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.915+0000 7fca66600700 1 --2- 172.21.15.131:0/1215125700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6010c510 0x7fca6010e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.915+0000 7fca66600700 1 -- 172.21.15.131:0/1215125700 >> 172.21.15.131:0/1215125700 conn(0x7fca6006e820 msgr2=0x7fca60070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:56.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.918+0000 7fca66600700 1 -- 172.21.15.131:0/1215125700 shutdown_connections 2024-05-10T05:41:56.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.918+0000 7fca66600700 1 -- 172.21.15.131:0/1215125700 wait complete. 2024-05-10T05:41:56.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.920+0000 7fca66600700 1 Processor -- start 2024-05-10T05:41:56.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.920+0000 7fca66600700 1 -- start start 2024-05-10T05:41:56.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.921+0000 7fca66600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60109bb0 0x7fca60119800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.921+0000 7fca66600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6010c510 0x7fca60119d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.922+0000 7fca5fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6010c510 0x7fca60119d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:56.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.922+0000 7fca5fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6010c510 0x7fca60119d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47800/0 (socket says 172.21.15.131:47800) 2024-05-10T05:41:56.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.922+0000 7fca65200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60109bb0 0x7fca60119800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:56.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.922+0000 7fca65200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60109bb0 0x7fca60119800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45114/0 (socket says 172.21.15.131:45114) 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.922+0000 7fca5fe00700 1 -- 172.21.15.131:0/1646535916 learned_addr learned my addr 172.21.15.131:0/1646535916 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.922+0000 7fca66600700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca6010ee70 0x7fca60120c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.922+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca60121250 con 0x7fca6010ee70 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.922+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca601213c0 con 0x7fca6010c510 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.922+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca60121580 con 0x7fca60109bb0 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.922+0000 7fca65c00700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca6010ee70 0x7fca60120c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.923+0000 7fca65200700 1 -- 172.21.15.131:0/1646535916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6010c510 msgr2=0x7fca60119d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.923+0000 7fca65200700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6010c510 0x7fca60119d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.923+0000 7fca65200700 1 -- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca6010ee70 msgr2=0x7fca60120c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.923+0000 7fca65200700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca6010ee70 0x7fca60120c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.923+0000 7fca65200700 1 -- 172.21.15.131:0/1646535916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca500095f0 con 0x7fca60109bb0 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.923+0000 7fca65200700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60109bb0 0x7fca60119800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fca5800bd40 tx=0x7fca5800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:56.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.924+0000 7fca5d600700 1 -- 172.21.15.131:0/1646535916 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800cc90 con 0x7fca60109bb0 2024-05-10T05:41:56.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.924+0000 7fca5d600700 1 -- 172.21.15.131:0/1646535916 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca5800cdf0 con 0x7fca60109bb0 2024-05-10T05:41:56.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.926+0000 7fca5d600700 1 -- 172.21.15.131:0/1646535916 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca58012760 con 0x7fca60109bb0 2024-05-10T05:41:56.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.928+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca60121860 con 0x7fca60109bb0 2024-05-10T05:41:56.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.928+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca601b7aa0 con 0x7fca60109bb0 2024-05-10T05:41:56.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.932+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca600703d0 con 0x7fca60109bb0 2024-05-10T05:41:56.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.938+0000 7fca5d600700 1 -- 172.21.15.131:0/1646535916 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fca580128c0 con 0x7fca60109bb0 2024-05-10T05:41:56.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.940+0000 7fca5d600700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca440a1180 0x7fca440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:56.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.941+0000 7fca5fe00700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca440a1180 0x7fca440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:56.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.942+0000 7fca5fe00700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca440a1180 0x7fca440a3690 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fca6011a850 tx=0x7fca4c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:56.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.941+0000 7fca5d600700 1 -- 172.21.15.131:0/1646535916 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fca58006e40 con 0x7fca60109bb0 2024-05-10T05:41:56.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:56.946+0000 7fca5d600700 1 -- 172.21.15.131:0/1646535916 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fca58095660 con 0x7fca60109bb0 2024-05-10T05:41:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.685+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca601141e0 con 0x7fca60109bb0 2024-05-10T05:41:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.686+0000 7fca5d600700 1 -- 172.21.15.131:0/1646535916 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fca58098c80 con 0x7fca60109bb0 2024-05-10T05:41:57.687 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:41:57.687 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:41:57.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.691+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca440a1180 msgr2=0x7fca440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.691+0000 7fca66600700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca440a1180 0x7fca440a3690 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fca6011a850 tx=0x7fca4c009260 comp rx=0 tx=0).stop 2024-05-10T05:41:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.691+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60109bb0 msgr2=0x7fca60119800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.691+0000 7fca66600700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60109bb0 0x7fca60119800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fca5800bd40 tx=0x7fca5800bd70 comp rx=0 tx=0).stop 2024-05-10T05:41:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.691+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 shutdown_connections 2024-05-10T05:41:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.691+0000 7fca66600700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca440a1180 0x7fca440a3690 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.691+0000 7fca66600700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca6010ee70 0x7fca60120c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.691+0000 7fca66600700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca60109bb0 0x7fca60119800 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fca5800bd40 tx=0x7fca5800bd70 comp rx=0 tx=0).stop 2024-05-10T05:41:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.691+0000 7fca66600700 1 --2- 172.21.15.131:0/1646535916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6010c510 0x7fca60119d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.691+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 >> 172.21.15.131:0/1646535916 conn(0x7fca6006e820 msgr2=0x7fca6010a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.693+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 shutdown_connections 2024-05-10T05:41:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:57.693+0000 7fca66600700 1 -- 172.21.15.131:0/1646535916 wait complete. 2024-05-10T05:41:57.696 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:41:57.753 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:41:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.294+0000 7f2021a00700 1 -- 172.21.15.131:0/4221271206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f201c109130 msgr2=0x7f201c109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.294+0000 7f2021a00700 1 --2- 172.21.15.131:0/4221271206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f201c109130 0x7f201c109540 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2014009f90 tx=0x7f201400b330 comp rx=0 tx=0).stop 2024-05-10T05:41:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.294+0000 7f2021a00700 1 -- 172.21.15.131:0/4221271206 shutdown_connections 2024-05-10T05:41:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.294+0000 7f2021a00700 1 --2- 172.21.15.131:0/4221271206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201c109ad0 0x7f201c110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.295+0000 7f2021a00700 1 --2- 172.21.15.131:0/4221271206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f201c108010 0x7f201c108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.295+0000 7f2021a00700 1 --2- 172.21.15.131:0/4221271206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f201c109130 0x7f201c109540 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.295+0000 7f2021a00700 1 -- 172.21.15.131:0/4221271206 >> 172.21.15.131:0/4221271206 conn(0x7f201c077040 msgr2=0x7f201c106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:58.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.300+0000 7f2021a00700 1 -- 172.21.15.131:0/4221271206 shutdown_connections 2024-05-10T05:41:58.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.300+0000 7f2021a00700 1 -- 172.21.15.131:0/4221271206 wait complete. 2024-05-10T05:41:58.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.302+0000 7f2021a00700 1 Processor -- start 2024-05-10T05:41:58.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.304+0000 7f2021a00700 1 -- start start 2024-05-10T05:41:58.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.304+0000 7f2021a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f201c108010 0x7f201c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.307+0000 7f2021a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201c109130 0x7f201c19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.307+0000 7f2021a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f201c109ad0 0x7f201c1a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.307+0000 7f2021a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c1a5ff0 con 0x7f201c109ad0 2024-05-10T05:41:58.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.307+0000 7f2021a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c1a6130 con 0x7f201c108010 2024-05-10T05:41:58.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.307+0000 7f2021a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c1a62c0 con 0x7f201c109130 2024-05-10T05:41:58.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.307+0000 7f201be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f201c108010 0x7f201c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.308+0000 7f201b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201c109130 0x7f201c19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.308+0000 7f201b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201c109130 0x7f201c19ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45140/0 (socket says 172.21.15.131:45140) 2024-05-10T05:41:58.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.308+0000 7f201b400700 1 -- 172.21.15.131:0/1272515757 learned_addr learned my addr 172.21.15.131:0/1272515757 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:58.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.308+0000 7f201b400700 1 -- 172.21.15.131:0/1272515757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f201c108010 msgr2=0x7f201c19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:58.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.308+0000 7f201b400700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f201c108010 0x7f201c19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.308+0000 7f201b400700 1 -- 172.21.15.131:0/1272515757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f201c109ad0 msgr2=0x7f201c1a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:58.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.308+0000 7f201b400700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f201c109ad0 0x7f201c1a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.308+0000 7f201b400700 1 -- 172.21.15.131:0/1272515757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c009520 con 0x7f201c109130 2024-05-10T05:41:58.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.309+0000 7f201b400700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201c109130 0x7f201c19ea90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2014004d00 tx=0x7f2014004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:58.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.309+0000 7f2018c00700 1 -- 172.21.15.131:0/1272515757 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201400e040 con 0x7f201c109130 2024-05-10T05:41:58.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.309+0000 7f2021a00700 1 -- 172.21.15.131:0/1272515757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20080095f0 con 0x7f201c109130 2024-05-10T05:41:58.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.309+0000 7f2021a00700 1 -- 172.21.15.131:0/1272515757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2014009c40 con 0x7f201c109130 2024-05-10T05:41:58.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.313+0000 7f2018c00700 1 -- 172.21.15.131:0/1272515757 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2014009ac0 con 0x7f201c109130 2024-05-10T05:41:58.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.313+0000 7f2018c00700 1 -- 172.21.15.131:0/1272515757 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014005040 con 0x7f201c109130 2024-05-10T05:41:58.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.325+0000 7f2018c00700 1 -- 172.21.15.131:0/1272515757 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f201401d470 con 0x7f201c109130 2024-05-10T05:41:58.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.325+0000 7f203afe9b80 1 Processor -- start 2024-05-10T05:41:58.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.327+0000 7f2018c00700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20000a11a0 0x7f20000a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.332+0000 7f2018c00700 1 -- 172.21.15.131:0/1272515757 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f201400be00 con 0x7f201c109130 2024-05-10T05:41:58.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.336+0000 7f2021000700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20000a11a0 0x7f20000a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.343+0000 7f203afe9b80 1 -- start start 2024-05-10T05:41:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.352+0000 7f2021000700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20000a11a0 0x7f20000a36b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f20080095c0 tx=0x7f2008009510 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:58.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.357+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a09530 0x560927a09960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.364+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 0x560927a0abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.364+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0b0f0 0x560927a119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.364+0000 7f203afe9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560927930150 con 0x560927a0a7a0 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.364+0000 7f203afe9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092792db10 con 0x560927a0b0f0 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.364+0000 7f203afe9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560927929400 con 0x560927a09530 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.364+0000 7f201b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 0x560927a0abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.364+0000 7f201b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 0x560927a0abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57618/0 (socket says 172.21.15.131:57618) 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.364+0000 7f201b400700 1 -- 172.21.15.131:0/2813007681 learned_addr learned my addr 172.21.15.131:0/2813007681 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.365+0000 7f201b400700 1 -- 172.21.15.131:0/2813007681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a09530 msgr2=0x560927a09960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.358+0000 7f201be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a09530 0x560927a09960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.366+0000 7f201b400700 1 --2- 172.21.15.131:0/2813007681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a09530 0x560927a09960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.366+0000 7f201b400700 1 -- 172.21.15.131:0/2813007681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0b0f0 msgr2=0x560927a119f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:41:58.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.366+0000 7f201b400700 1 --2- 172.21.15.131:0/2813007681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0b0f0 0x560927a119f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.366+0000 7f201b400700 1 -- 172.21.15.131:0/2813007681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609277aa3b0 con 0x560927a0a7a0 2024-05-10T05:41:58.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.367+0000 7f201be00700 1 --2- 172.21.15.131:0/2813007681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a09530 0x560927a09960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:41:58.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.369+0000 7f201b400700 1 --2- 172.21.15.131:0/2813007681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 0x560927a0abb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2014004e50 tx=0x7f20140a1f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ef78ff962671ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:58.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.380+0000 7f2011600700 1 -- 172.21.15.131:0/2813007681 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201409aea0 con 0x560927a0a7a0 2024-05-10T05:41:58.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.380+0000 7f2011600700 1 -- 172.21.15.131:0/2813007681 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f201409aa30 con 0x560927a0a7a0 2024-05-10T05:41:58.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.381+0000 7f2011600700 1 -- 172.21.15.131:0/2813007681 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201409aea0 con 0x560927a0a7a0 2024-05-10T05:41:58.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.383+0000 7f203afe9b80 1 -- 172.21.15.131:0/2813007681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 msgr2=0x560927a0abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:58.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.384+0000 7f203afe9b80 1 --2- 172.21.15.131:0/2813007681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 0x560927a0abb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2014004e50 tx=0x7f20140a1f10 comp rx=0 tx=0).stop 2024-05-10T05:41:58.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.387+0000 7f203afe9b80 1 -- 172.21.15.131:0/2813007681 shutdown_connections 2024-05-10T05:41:58.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.387+0000 7f203afe9b80 1 --2- 172.21.15.131:0/2813007681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0b0f0 0x560927a119f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.387+0000 7f203afe9b80 1 --2- 172.21.15.131:0/2813007681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a09530 0x560927a09960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.387+0000 7f203afe9b80 1 --2- 172.21.15.131:0/2813007681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 0x560927a0abb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.387+0000 7f203afe9b80 1 -- 172.21.15.131:0/2813007681 >> 172.21.15.131:0/2813007681 conn(0x56092796ba60 msgr2=0x560927a04330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:58.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.388+0000 7f203afe9b80 1 -- 172.21.15.131:0/2813007681 shutdown_connections 2024-05-10T05:41:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.389+0000 7f203afe9b80 1 -- 172.21.15.131:0/2813007681 wait complete. 2024-05-10T05:41:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.402+0000 7f203afe9b80 1 -- start start 2024-05-10T05:41:58.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.403+0000 7f203afe9b80 1 Processor -- start 2024-05-10T05:41:58.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.404+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a135d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.404+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0a7a0 0x560927a13b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.405+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a1aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.405+0000 7f201be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.405+0000 7f203afe9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609278c5560 con 0x560927a09530 2024-05-10T05:41:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.405+0000 7f203afe9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609278c5200 con 0x560927a0a7a0 2024-05-10T05:41:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.405+0000 7f203afe9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609277ac6a0 con 0x560927a0b0f0 2024-05-10T05:41:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.405+0000 7f203afe9b80 10 client.0 _ll_register_callbacks cb 0x5609278ceb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.405+0000 7f201be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a135d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57630/0 (socket says 172.21.15.131:57630) 2024-05-10T05:41:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.405+0000 7f201be00700 1 -- 172.21.15.131:0/1825084066 learned_addr learned my addr 172.21.15.131:0/1825084066 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.406+0000 7f201b400700 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0a7a0 0x560927a13b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.406+0000 7f2021000700 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a1aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.408+0000 7f201b400700 1 -- 172.21.15.131:0/1825084066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 msgr2=0x560927a1aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.409+0000 7f201b400700 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a1aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.409+0000 7f201b400700 1 -- 172.21.15.131:0/1825084066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 msgr2=0x560927a135d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:58.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.411+0000 7f201b400700 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a135d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.412+0000 7f201b400700 1 -- 172.21.15.131:0/1825084066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2014009c40 con 0x560927a0a7a0 2024-05-10T05:41:58.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.412+0000 7f201be00700 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:41:58.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.413+0000 7f201b400700 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0a7a0 0x560927a13b10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f20140975d0 tx=0x7f20140097c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcfde6710e5486f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:58.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.413+0000 7f2006000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:41:58.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.413+0000 7f2006000700 1 -- 172.21.15.131:0/1825084066 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201409aea0 con 0x560927a0a7a0 2024-05-10T05:41:58.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.414+0000 7f2006000700 1 -- 172.21.15.131:0/1825084066 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f20140a34c0 con 0x560927a0a7a0 2024-05-10T05:41:58.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.415+0000 7f203afe9b80 1 -- 172.21.15.131:0/1825084066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560927882270 con 0x560927a0a7a0 2024-05-10T05:41:58.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.416+0000 7f2006000700 1 -- 172.21.15.131:0/1825084066 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2014099420 con 0x560927a0a7a0 2024-05-10T05:41:58.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.416+0000 7f203afe9b80 10 client.26876 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:58.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.416+0000 7f203afe9b80 1 -- 172.21.15.131:0/1825084066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560927756b60 con 0x560927a0a7a0 2024-05-10T05:41:58.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.416+0000 7f2006000700 1 -- 172.21.15.131:0/1825084066 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2014099580 con 0x560927a0a7a0 2024-05-10T05:41:58.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.419+0000 7f2006000700 1 -- 172.21.15.131:0/1825084066 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f20140b9170 con 0x560927a0a7a0 2024-05-10T05:41:58.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.420+0000 7f203afe9b80 10 client.26876 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:58.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.420+0000 7f203afe9b80 10 client.26876 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:41:58.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.420+0000 7f203afe9b80 20 client.26876 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:58.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.420+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560927a1eca0 0x560927a210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.420+0000 7f203afe9b80 4 client.26876 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:58.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.420+0000 7f203afe9b80 1 -- 172.21.15.131:0/1825084066 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560927882270 con 0x560927a1eca0 2024-05-10T05:41:58.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.420+0000 7f201be00700 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560927a1eca0 0x560927a210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.423+0000 7f201be00700 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560927a1eca0 0x560927a210c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fcfde7d60f0b9d26 server_cookie=4b28dc1822713aa1 in_seq=0 out_seq=0 2024-05-10T05:41:58.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.425+0000 7f2006000700 10 client.26876 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:41:58.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.425+0000 7f2006000700 1 -- 172.21.15.131:0/1825084066 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560927882270 con 0x560927a1eca0 2024-05-10T05:41:58.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.425+0000 7f2006000700 10 client.26876 handle_command_reply: tid=0 2024-05-10T05:41:58.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.426+0000 7f203afe9b80 1 client.26876 shutdown 2024-05-10T05:41:58.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f203afe9b80 1 -- 172.21.15.131:0/1825084066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0a7a0 msgr2=0x560927a13b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0a7a0 0x560927a13b10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f20140975d0 tx=0x7f20140097c0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f203afe9b80 1 -- 172.21.15.131:0/1825084066 shutdown_connections 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560927a1eca0 0x560927a210c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a1aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1825084066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0a7a0 0x560927a13b10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f203afe9b80 1 -- 172.21.15.131:0/1825084066 >> 172.21.15.131:0/1825084066 conn(0x56092796ba60 msgr2=0x560927a08060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f2006000700 0 client.26876 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f203afe9b80 1 -- 172.21.15.131:0/1825084066 shutdown_connections 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.432+0000 7f203afe9b80 1 -- 172.21.15.131:0/1825084066 wait complete. 2024-05-10T05:41:58.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.433+0000 7f203afe9b80 20 client.26876 trim_cache size 0 max 16384 2024-05-10T05:41:58.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.465+0000 7f203afe9b80 1 Processor -- start 2024-05-10T05:41:58.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.465+0000 7f203afe9b80 1 -- start start 2024-05-10T05:41:58.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.466+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a09530 0x560927a11b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.466+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 0x560927a120c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.466+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a1a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.467+0000 7f203afe9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560927a14670 con 0x560927a0a7a0 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.467+0000 7f2021000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a1a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.467+0000 7f2021000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a1a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45158/0 (socket says 172.21.15.131:45158) 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.467+0000 7f2021000700 1 -- 172.21.15.131:0/3078877441 learned_addr learned my addr 172.21.15.131:0/3078877441 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.467+0000 7f203afe9b80 1 -- 172.21.15.131:0/3078877441 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560927a1bb70 con 0x560927a09530 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.467+0000 7f203afe9b80 1 -- 172.21.15.131:0/3078877441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560927a053e0 con 0x560927a0b0f0 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.467+0000 7f2021000700 1 -- 172.21.15.131:0/3078877441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a09530 msgr2=0x560927a11b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.467+0000 7f2021000700 1 --2- 172.21.15.131:0/3078877441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a09530 0x560927a11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.467+0000 7f2021000700 1 -- 172.21.15.131:0/3078877441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 msgr2=0x560927a120c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.467+0000 7f2021000700 1 --2- 172.21.15.131:0/3078877441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 0x560927a120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.468+0000 7f2021000700 1 -- 172.21.15.131:0/3078877441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560927a0dca0 con 0x560927a0b0f0 2024-05-10T05:41:58.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.468+0000 7f2021000700 1 --2- 172.21.15.131:0/3078877441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a1a8c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f200800ee10 tx=0x7f2008011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df3b0a5ebd6311ee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:58.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.468+0000 7f2012000700 1 -- 172.21.15.131:0/3078877441 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008004220 con 0x560927a0b0f0 2024-05-10T05:41:58.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.469+0000 7f2012000700 1 -- 172.21.15.131:0/3078877441 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2008021070 con 0x560927a0b0f0 2024-05-10T05:41:58.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.469+0000 7f2012000700 1 -- 172.21.15.131:0/3078877441 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008004c90 con 0x560927a0b0f0 2024-05-10T05:41:58.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.473+0000 7f203afe9b80 1 -- 172.21.15.131:0/3078877441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 msgr2=0x560927a1a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:58.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.473+0000 7f203afe9b80 1 --2- 172.21.15.131:0/3078877441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a1a8c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f200800ee10 tx=0x7f2008011000 comp rx=0 tx=0).stop 2024-05-10T05:41:58.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.473+0000 7f203afe9b80 1 -- 172.21.15.131:0/3078877441 shutdown_connections 2024-05-10T05:41:58.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.473+0000 7f203afe9b80 1 --2- 172.21.15.131:0/3078877441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a1a8c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.473+0000 7f203afe9b80 1 --2- 172.21.15.131:0/3078877441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a09530 0x560927a11b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.473+0000 7f203afe9b80 1 --2- 172.21.15.131:0/3078877441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a0a7a0 0x560927a120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.473+0000 7f203afe9b80 1 -- 172.21.15.131:0/3078877441 >> 172.21.15.131:0/3078877441 conn(0x56092796ba60 msgr2=0x560927a0e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:58.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.478+0000 7f203afe9b80 1 -- 172.21.15.131:0/3078877441 shutdown_connections 2024-05-10T05:41:58.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.480+0000 7f203afe9b80 1 -- 172.21.15.131:0/3078877441 wait complete. 2024-05-10T05:41:58.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.486+0000 7f203afe9b80 1 -- start start 2024-05-10T05:41:58.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.486+0000 7f203afe9b80 1 Processor -- start 2024-05-10T05:41:58.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.490+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a18530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.491+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0a7a0 0x560927a18a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.495+0000 7f203afe9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a199a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.495+0000 7f203afe9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560927a19ee0 con 0x560927a09530 2024-05-10T05:41:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.495+0000 7f203afe9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560927a1a050 con 0x560927a0a7a0 2024-05-10T05:41:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.495+0000 7f203afe9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560927a1a210 con 0x560927a0b0f0 2024-05-10T05:41:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.495+0000 7f203afe9b80 10 client.0 _ll_register_callbacks cb 0x5609278ceb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.491+0000 7f201be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a18530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.496+0000 7f201be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a18530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57654/0 (socket says 172.21.15.131:57654) 2024-05-10T05:41:58.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.496+0000 7f201be00700 1 -- 172.21.15.131:0/1157511810 learned_addr learned my addr 172.21.15.131:0/1157511810 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:58.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.499+0000 7f2021000700 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.500+0000 7f201be00700 1 -- 172.21.15.131:0/1157511810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 msgr2=0x560927a199a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:58.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.500+0000 7f201be00700 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a199a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.500+0000 7f201be00700 1 -- 172.21.15.131:0/1157511810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0a7a0 msgr2=0x560927a18a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:58.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.500+0000 7f201be00700 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0a7a0 0x560927a18a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.500+0000 7f201be00700 1 -- 172.21.15.131:0/1157511810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200800eac0 con 0x560927a09530 2024-05-10T05:41:58.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.500+0000 7f2021000700 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:41:58.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.501+0000 7f201be00700 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a18530 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x560927a219c0 tx=0x7f200c014750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545b92e83ea6cdd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:58.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.501+0000 7f2006000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:41:58.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.502+0000 7f2006000700 1 -- 172.21.15.131:0/1157511810 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c018040 con 0x560927a09530 2024-05-10T05:41:58.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.502+0000 7f2006000700 1 -- 172.21.15.131:0/1157511810 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f200c014e30 con 0x560927a09530 2024-05-10T05:41:58.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.502+0000 7f203afe9b80 1 -- 172.21.15.131:0/1157511810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f201409aa30 con 0x560927a09530 2024-05-10T05:41:58.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.506+0000 7f2006000700 1 -- 172.21.15.131:0/1157511810 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f200c01d620 con 0x560927a09530 2024-05-10T05:41:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.507+0000 7f203afe9b80 10 client.16611 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.507+0000 7f203afe9b80 1 -- 172.21.15.131:0/1157511810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5609278ef560 con 0x560927a09530 2024-05-10T05:41:58.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.510+0000 7f2006000700 1 -- 172.21.15.131:0/1157511810 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f201409aa30 con 0x560927a09530 2024-05-10T05:41:58.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.511+0000 7f2006000700 1 -- 172.21.15.131:0/1157511810 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f200c026070 con 0x560927a09530 2024-05-10T05:41:58.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.512+0000 7f203afe9b80 10 client.16611 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:58.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.512+0000 7f203afe9b80 10 client.16611 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:41:58.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.512+0000 7f203afe9b80 20 client.16611 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:58.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.512+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560927a75c40 0x560927a78060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:58.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.512+0000 7f203afe9b80 4 client.16611 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:41:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.512+0000 7f203afe9b80 1 -- 172.21.15.131:0/1157511810 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609279d8780 con 0x560927a75c40 2024-05-10T05:41:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.512+0000 7f201b400700 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560927a75c40 0x560927a78060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.513+0000 7f2006000700 10 client.16611 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:41:58.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.513+0000 7f201b400700 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560927a75c40 0x560927a78060 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e84ac453e8177cb1 server_cookie=f8e488ff642e29cb in_seq=0 out_seq=0 2024-05-10T05:41:58.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.513+0000 7f2006000700 1 -- 172.21.15.131:0/1157511810 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609279d8780 con 0x560927a75c40 2024-05-10T05:41:58.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.513+0000 7f2006000700 10 client.16611 handle_command_reply: tid=0 2024-05-10T05:41:58.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.513+0000 7f203afe9b80 1 client.16611 shutdown 2024-05-10T05:41:58.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.514+0000 7f203afe9b80 1 -- 172.21.15.131:0/1157511810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 msgr2=0x560927a18530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:58.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.514+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a18530 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x560927a219c0 tx=0x7f200c014750 comp rx=0 tx=0).stop 2024-05-10T05:41:58.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.514+0000 7f203afe9b80 1 -- 172.21.15.131:0/1157511810 shutdown_connections 2024-05-10T05:41:58.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.514+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x560927a75c40 0x560927a78060 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.514+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560927a09530 0x560927a18530 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.514+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560927a0b0f0 0x560927a199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.514+0000 7f203afe9b80 1 --2- 172.21.15.131:0/1157511810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560927a0a7a0 0x560927a18a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.514+0000 7f2006000700 0 client.16611 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:41:58.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.515+0000 7f203afe9b80 1 -- 172.21.15.131:0/1157511810 >> 172.21.15.131:0/1157511810 conn(0x56092796ba60 msgr2=0x560927a0e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:58.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.516+0000 7f203afe9b80 1 -- 172.21.15.131:0/1157511810 shutdown_connections 2024-05-10T05:41:58.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.517+0000 7f203afe9b80 1 -- 172.21.15.131:0/1157511810 wait complete. 2024-05-10T05:41:58.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.517+0000 7f203afe9b80 20 client.16611 trim_cache size 0 max 16384 2024-05-10T05:41:58.519 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:41:58.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.521+0000 7f2021a00700 1 -- 172.21.15.131:0/1272515757 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20000a11a0 msgr2=0x7f20000a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:58.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.521+0000 7f2021a00700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20000a11a0 0x7f20000a36b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f20080095c0 tx=0x7f2008009510 comp rx=0 tx=0).stop 2024-05-10T05:41:58.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.521+0000 7f2021a00700 1 -- 172.21.15.131:0/1272515757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201c109130 msgr2=0x7f201c19ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:58.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.521+0000 7f2021a00700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201c109130 0x7f201c19ea90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2014004d00 tx=0x7f2014004d30 comp rx=0 tx=0).stop 2024-05-10T05:41:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.522+0000 7f2021a00700 1 -- 172.21.15.131:0/1272515757 shutdown_connections 2024-05-10T05:41:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.522+0000 7f2021a00700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f20000a11a0 0x7f20000a36b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.522+0000 7f2021a00700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f201c109ad0 0x7f201c1a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.522+0000 7f2021a00700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f201c108010 0x7f201c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.522+0000 7f2021a00700 1 --2- 172.21.15.131:0/1272515757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f201c109130 0x7f201c19ea90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:58.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.522+0000 7f2021a00700 1 -- 172.21.15.131:0/1272515757 >> 172.21.15.131:0/1272515757 conn(0x7f201c077040 msgr2=0x7f201c10c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.522+0000 7f2021a00700 1 -- 172.21.15.131:0/1272515757 shutdown_connections 2024-05-10T05:41:58.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:58.522+0000 7f2021a00700 1 -- 172.21.15.131:0/1272515757 wait complete. 2024-05-10T05:41:58.602 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:41:58.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:58 smithi142 ceph-mon[77667]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 241 MiB/s wr, 77 op/s 2024-05-10T05:41:58.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:41:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1646535916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:41:58.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:58 smithi131 ceph-mon[37264]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 241 MiB/s wr, 77 op/s 2024-05-10T05:41:58.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:41:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1646535916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:41:58.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:58 smithi157 ceph-mon[78427]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 241 MiB/s wr, 77 op/s 2024-05-10T05:41:58.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:41:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1646535916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:41:59.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.467+0000 7f8aa8600700 1 -- 172.21.15.131:0/1784598722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aa0108b60 msgr2=0x7f8aa010e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:59.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.468+0000 7f8aa8600700 1 --2- 172.21.15.131:0/1784598722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aa0108b60 0x7f8aa010e480 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009f90 tx=0x7f8a9800b330 comp rx=0 tx=0).stop 2024-05-10T05:41:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.469+0000 7f8aa8600700 1 -- 172.21.15.131:0/1784598722 shutdown_connections 2024-05-10T05:41:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.470+0000 7f8aa8600700 1 --2- 172.21.15.131:0/1784598722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aa0108b60 0x7f8aa010e480 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.470+0000 7f8aa8600700 1 --2- 172.21.15.131:0/1784598722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa0109500 0x7f8aa0109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.470+0000 7f8aa8600700 1 --2- 172.21.15.131:0/1784598722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aa0108210 0x7f8aa0108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.470+0000 7f8aa8600700 1 -- 172.21.15.131:0/1784598722 >> 172.21.15.131:0/1784598722 conn(0x7f8aa006e830 msgr2=0x7f8aa0070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:59.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.475+0000 7f8aa8600700 1 -- 172.21.15.131:0/1784598722 shutdown_connections 2024-05-10T05:41:59.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.478+0000 7f8aa8600700 1 -- 172.21.15.131:0/1784598722 wait complete. 2024-05-10T05:41:59.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.480+0000 7f8aa8600700 1 Processor -- start 2024-05-10T05:41:59.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.481+0000 7f8aa8600700 1 -- start start 2024-05-10T05:41:59.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.482+0000 7f8aa8600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aa0108210 0x7f8aa01a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.482+0000 7f8aa8600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa0108b60 0x7f8aa01a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.482+0000 7f8aa8600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aa0109500 0x7f8aa01ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.482+0000 7f8aa8600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01ae700 con 0x7f8aa0108210 2024-05-10T05:41:59.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.482+0000 7f8aa8600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01ae840 con 0x7f8aa0108b60 2024-05-10T05:41:59.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.482+0000 7f8aa8600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01aea00 con 0x7f8aa0109500 2024-05-10T05:41:59.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.483+0000 7f8aa5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aa0108210 0x7f8aa01a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.483+0000 7f8aa6600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aa0109500 0x7f8aa01ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.483+0000 7f8aa5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aa0108210 0x7f8aa01a6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57672/0 (socket says 172.21.15.131:57672) 2024-05-10T05:41:59.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.483+0000 7f8aa5c00700 1 -- 172.21.15.131:0/2273838511 learned_addr learned my addr 172.21.15.131:0/2273838511 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:59.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.483+0000 7f8aa5200700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa0108b60 0x7f8aa01a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.484+0000 7f8aa5200700 1 -- 172.21.15.131:0/2273838511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aa0109500 msgr2=0x7f8aa01ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:59.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.484+0000 7f8aa5200700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aa0109500 0x7f8aa01ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.484+0000 7f8aa5200700 1 -- 172.21.15.131:0/2273838511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aa0108210 msgr2=0x7f8aa01a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:59.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.484+0000 7f8aa5200700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aa0108210 0x7f8aa01a6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.484+0000 7f8aa5200700 1 -- 172.21.15.131:0/2273838511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a98009c40 con 0x7f8aa0108b60 2024-05-10T05:41:59.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.484+0000 7f8aa5c00700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aa0108210 0x7f8aa01a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:41:59.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.484+0000 7f8aa5200700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa0108b60 0x7f8aa01a71e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400b450 tx=0x7f8a9400b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:59.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.485+0000 7f8a9e000700 1 -- 172.21.15.131:0/2273838511 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94011830 con 0x7f8aa0108b60 2024-05-10T05:41:59.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.485+0000 7f8a9e000700 1 -- 172.21.15.131:0/2273838511 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a94011e50 con 0x7f8aa0108b60 2024-05-10T05:41:59.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.485+0000 7f8a9e000700 1 -- 172.21.15.131:0/2273838511 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94010d00 con 0x7f8aa0108b60 2024-05-10T05:41:59.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.486+0000 7f8aa8600700 1 -- 172.21.15.131:0/2273838511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa01aece0 con 0x7f8aa0108b60 2024-05-10T05:41:59.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.488+0000 7f8aa8600700 1 -- 172.21.15.131:0/2273838511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa01af1a0 con 0x7f8aa0108b60 2024-05-10T05:41:59.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.494+0000 7f8a9e000700 1 -- 172.21.15.131:0/2273838511 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8a94010380 con 0x7f8aa0108b60 2024-05-10T05:41:59.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.497+0000 7f8ac058ab80 1 Processor -- start 2024-05-10T05:41:59.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.497+0000 7f8a9e000700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a880a1220 0x7f8a880a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.497+0000 7f8a9e000700 1 -- 172.21.15.131:0/2273838511 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8a940cc270 con 0x7f8aa0108b60 2024-05-10T05:41:59.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.499+0000 7f8aa6600700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a880a1220 0x7f8a880a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.499+0000 7f8ac058ab80 1 -- start start 2024-05-10T05:41:59.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.500+0000 7f8ac058ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 0x55e8ba9a1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.500+0000 7f8ac058ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a27a0 0x55e8ba9a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.500+0000 7f8ac058ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.500+0000 7f8ac058ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba8c8150 con 0x55e8ba9a30f0 2024-05-10T05:41:59.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.500+0000 7f8ac058ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba8c5b10 con 0x55e8ba9a27a0 2024-05-10T05:41:59.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.500+0000 7f8ac058ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba8c1400 con 0x55e8ba9a1530 2024-05-10T05:41:59.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.502+0000 7f8aa5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a27a0 0x55e8ba9a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.502+0000 7f8aa5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a27a0 0x55e8ba9a2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47870/0 (socket says 172.21.15.131:47870) 2024-05-10T05:41:59.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.502+0000 7f8aa5200700 1 -- 172.21.15.131:0/1138805444 learned_addr learned my addr 172.21.15.131:0/1138805444 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:59.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.502+0000 7f8aa5c00700 1 --2- 172.21.15.131:0/1138805444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 0x55e8ba9a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.503+0000 7f8aa6600700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a880a1220 0x7f8a880a3730 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009550 tx=0x7f8a98006000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.503+0000 7f8aa6600700 1 --2- 172.21.15.131:0/1138805444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.505+0000 7f8aa6600700 1 -- 172.21.15.131:0/1138805444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 msgr2=0x55e8ba9a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.505+0000 7f8aa6600700 1 --2- 172.21.15.131:0/1138805444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 0x55e8ba9a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.505+0000 7f8aa6600700 1 -- 172.21.15.131:0/1138805444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a27a0 msgr2=0x55e8ba9a2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.505+0000 7f8aa6600700 1 --2- 172.21.15.131:0/1138805444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a27a0 0x55e8ba9a2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.505+0000 7f8aa6600700 1 -- 172.21.15.131:0/1138805444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8ba7423b0 con 0x55e8ba9a30f0 2024-05-10T05:41:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.506+0000 7f8aa6600700 1 --2- 172.21.15.131:0/1138805444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9a99f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8a98007b70 tx=0x7f8a9801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc600f88dbf375d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.507+0000 7f8a92a00700 1 -- 172.21.15.131:0/1138805444 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800e040 con 0x55e8ba9a30f0 2024-05-10T05:41:59.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.507+0000 7f8a92a00700 1 -- 172.21.15.131:0/1138805444 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a9800be50 con 0x55e8ba9a30f0 2024-05-10T05:41:59.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.507+0000 7f8a92a00700 1 -- 172.21.15.131:0/1138805444 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98005330 con 0x55e8ba9a30f0 2024-05-10T05:41:59.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.511+0000 7f8ac058ab80 1 -- 172.21.15.131:0/1138805444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a30f0 msgr2=0x55e8ba9a99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:59.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.511+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/1138805444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9a99f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8a98007b70 tx=0x7f8a9801c000 comp rx=0 tx=0).stop 2024-05-10T05:41:59.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.512+0000 7f8ac058ab80 1 -- 172.21.15.131:0/1138805444 shutdown_connections 2024-05-10T05:41:59.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.512+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/1138805444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9a99f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.512+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/1138805444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 0x55e8ba9a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.512+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/1138805444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a27a0 0x55e8ba9a2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.512+0000 7f8ac058ab80 1 -- 172.21.15.131:0/1138805444 >> 172.21.15.131:0/1138805444 conn(0x55e8ba903a60 msgr2=0x55e8ba99c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.513+0000 7f8ac058ab80 1 -- 172.21.15.131:0/1138805444 shutdown_connections 2024-05-10T05:41:59.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.514+0000 7f8ac058ab80 1 -- 172.21.15.131:0/1138805444 wait complete. 2024-05-10T05:41:59.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.525+0000 7f8ac058ab80 1 -- start start 2024-05-10T05:41:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.533+0000 7f8ac058ab80 1 Processor -- start 2024-05-10T05:41:59.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.534+0000 7f8ac058ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 0x55e8ba9ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.534+0000 7f8ac058ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9abe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.534+0000 7f8ac058ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.534+0000 7f8ac058ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba85d560 con 0x55e8ba9b0d80 2024-05-10T05:41:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.534+0000 7f8ac058ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba85d200 con 0x55e8ba9a30f0 2024-05-10T05:41:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.534+0000 7f8ac058ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba7446a0 con 0x55e8ba9a1530 2024-05-10T05:41:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.535+0000 7f8ac058ab80 10 client.0 _ll_register_callbacks cb 0x55e8ba866b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.535+0000 7f8aa5c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 0x55e8ba9ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.535+0000 7f8aa5c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 0x55e8ba9ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45210/0 (socket says 172.21.15.131:45210) 2024-05-10T05:41:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.535+0000 7f8aa5c00700 1 -- 172.21.15.131:0/3479885850 learned_addr learned my addr 172.21.15.131:0/3479885850 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.535+0000 7f8aa6600700 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.536+0000 7f8aa5200700 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.537+0000 7f8aa6600700 1 -- 172.21.15.131:0/3479885850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 msgr2=0x55e8ba9ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.537+0000 7f8aa6600700 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 0x55e8ba9ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.537+0000 7f8aa6600700 1 -- 172.21.15.131:0/3479885850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a30f0 msgr2=0x55e8ba9abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.537+0000 7f8aa6600700 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.537+0000 7f8aa6600700 1 -- 172.21.15.131:0/3479885850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa010eeb0 con 0x55e8ba9b0d80 2024-05-10T05:41:59.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.538+0000 7f8aa6600700 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9b31a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009810 tx=0x7f8a98009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18d1e51e952318d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:59.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.539+0000 7f8a87400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:41:59.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.541+0000 7f8a87400700 1 -- 172.21.15.131:0/3479885850 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98034070 con 0x55e8ba9b0d80 2024-05-10T05:41:59.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.541+0000 7f8a87400700 1 -- 172.21.15.131:0/3479885850 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8a980050b0 con 0x55e8ba9b0d80 2024-05-10T05:41:59.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.541+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3479885850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8ba81a270 con 0x55e8ba9b0d80 2024-05-10T05:41:59.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.545+0000 7f8a87400700 1 -- 172.21.15.131:0/3479885850 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a98005270 con 0x55e8ba9b0d80 2024-05-10T05:41:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.545+0000 7f8ac058ab80 10 client.16623 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.545+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3479885850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e8ba6eeb60 con 0x55e8ba9b0d80 2024-05-10T05:41:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.546+0000 7f8a87400700 1 -- 172.21.15.131:0/3479885850 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a98005890 con 0x55e8ba9b0d80 2024-05-10T05:41:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.547+0000 7f8a87400700 1 -- 172.21.15.131:0/3479885850 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8a98042b40 con 0x55e8ba9b0d80 2024-05-10T05:41:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.548+0000 7f8ac058ab80 10 client.16623 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.548+0000 7f8ac058ab80 10 client.16623 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:41:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.548+0000 7f8ac058ab80 20 client.16623 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.548+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55e8ba9b7410 0x55e8ba9b9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.549+0000 7f8ac058ab80 4 client.16623 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:41:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.549+0000 7f8aa5c00700 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55e8ba9b7410 0x55e8ba9b9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.549+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3479885850 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8ba81a270 con 0x55e8ba9b7410 2024-05-10T05:41:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.552+0000 7f8aa5c00700 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55e8ba9b7410 0x55e8ba9b9830 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4322137b57d2de03 server_cookie=b9ce87c665309ea9 in_seq=0 out_seq=0 2024-05-10T05:41:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.553+0000 7f8a87400700 10 client.16623 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:41:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.553+0000 7f8a87400700 1 -- 172.21.15.131:0/3479885850 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8ba81a270 con 0x55e8ba9b7410 2024-05-10T05:41:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.553+0000 7f8a87400700 10 client.16623 handle_command_reply: tid=0 2024-05-10T05:41:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.554+0000 7f8ac058ab80 1 client.16623 shutdown 2024-05-10T05:41:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.570+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3479885850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 msgr2=0x55e8ba9b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.570+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9b31a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009810 tx=0x7f8a98009920 comp rx=0 tx=0).stop 2024-05-10T05:41:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.571+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3479885850 shutdown_connections 2024-05-10T05:41:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.571+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55e8ba9b7410 0x55e8ba9b9830 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.571+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9b31a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.571+0000 7f8a87400700 0 client.16623 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:41:59.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.572+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a1530 0x55e8ba9ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.572+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3479885850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.572+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3479885850 >> 172.21.15.131:0/3479885850 conn(0x55e8ba903a60 msgr2=0x55e8ba9a0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:59.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.573+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3479885850 shutdown_connections 2024-05-10T05:41:59.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.573+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3479885850 wait complete. 2024-05-10T05:41:59.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.574+0000 7f8ac058ab80 20 client.16623 trim_cache size 0 max 16384 2024-05-10T05:41:59.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.618+0000 7f8ac058ab80 1 Processor -- start 2024-05-10T05:41:59.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.619+0000 7f8ac058ab80 1 -- start start 2024-05-10T05:41:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.620+0000 7f8ac058ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.621+0000 7f8aa5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.621+0000 7f8aa5c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57716/0 (socket says 172.21.15.131:57716) 2024-05-10T05:41:59.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.621+0000 7f8aa5c00700 1 -- 172.21.15.131:0/3343862337 learned_addr learned my addr 172.21.15.131:0/3343862337 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:59.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.622+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.622+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9a1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.622+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3343862337 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba998340 con 0x55e8ba9a1530 2024-05-10T05:41:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.622+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3343862337 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba9a2560 con 0x55e8ba9b0d80 2024-05-10T05:41:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.622+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3343862337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba9ab570 con 0x55e8ba9a30f0 2024-05-10T05:41:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.623+0000 7f8aa6600700 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.623+0000 7f8aa5200700 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.624+0000 7f8aa6600700 1 -- 172.21.15.131:0/3343862337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a30f0 msgr2=0x55e8ba9c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.624+0000 7f8aa6600700 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.624+0000 7f8aa6600700 1 -- 172.21.15.131:0/3343862337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a1530 msgr2=0x55e8ba9c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.624+0000 7f8aa6600700 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.624+0000 7f8aa6600700 1 -- 172.21.15.131:0/3343862337 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8ba887560 con 0x55e8ba9b0d80 2024-05-10T05:41:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.625+0000 7f8aa6600700 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9a1940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8a980094c0 tx=0x7f8a9802f400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=370691c7c480378e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.625+0000 7f8a93400700 1 -- 172.21.15.131:0/3343862337 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98034070 con 0x55e8ba9b0d80 2024-05-10T05:41:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.626+0000 7f8a93400700 1 -- 172.21.15.131:0/3343862337 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a9802f520 con 0x55e8ba9b0d80 2024-05-10T05:41:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.626+0000 7f8a93400700 1 -- 172.21.15.131:0/3343862337 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98042ad0 con 0x55e8ba9b0d80 2024-05-10T05:41:59.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.628+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3343862337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9b0d80 msgr2=0x55e8ba9a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:59.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.628+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9a1940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8a980094c0 tx=0x7f8a9802f400 comp rx=0 tx=0).stop 2024-05-10T05:41:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.629+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3343862337 shutdown_connections 2024-05-10T05:41:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.629+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.629+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9a1940 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.629+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/3343862337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.629+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3343862337 >> 172.21.15.131:0/3343862337 conn(0x55e8ba903a60 msgr2=0x55e8ba9aae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:59.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.630+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3343862337 shutdown_connections 2024-05-10T05:41:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.630+0000 7f8ac058ab80 1 -- 172.21.15.131:0/3343862337 wait complete. 2024-05-10T05:41:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.632+0000 7f8ac058ab80 1 -- start start 2024-05-10T05:41:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.633+0000 7f8ac058ab80 1 Processor -- start 2024-05-10T05:41:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.636+0000 7f8ac058ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.636+0000 7f8ac058ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.636+0000 7f8ac058ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9c6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.636+0000 7f8ac058ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba9c6ee0 con 0x55e8ba9b0d80 2024-05-10T05:41:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.636+0000 7f8ac058ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ba9c7050 con 0x55e8ba9a1530 2024-05-10T05:41:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.636+0000 7f8ac058ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8baa0c3f0 con 0x55e8ba9a30f0 2024-05-10T05:41:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.636+0000 7f8ac058ab80 10 client.0 _ll_register_callbacks cb 0x55e8ba866b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:41:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.637+0000 7f8aa5c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.638+0000 7f8aa5c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47898/0 (socket says 172.21.15.131:47898) 2024-05-10T05:41:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.638+0000 7f8aa5c00700 1 -- 172.21.15.131:0/4179534180 learned_addr learned my addr 172.21.15.131:0/4179534180 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:41:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.637+0000 7f8aa5200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.638+0000 7f8aa6600700 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.639+0000 7f8aa5c00700 1 -- 172.21.15.131:0/4179534180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a30f0 msgr2=0x55e8ba9c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.639+0000 7f8aa5c00700 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.639+0000 7f8aa5c00700 1 -- 172.21.15.131:0/4179534180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 msgr2=0x55e8ba9c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:41:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.639+0000 7f8aa5c00700 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.639+0000 7f8aa5c00700 1 -- 172.21.15.131:0/4179534180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8ba6eeb60 con 0x55e8ba9a1530 2024-05-10T05:41:59.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.643+0000 7f8aa5c00700 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c012770 tx=0x7f8a8c012650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26d8b84220efb439 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:41:59.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.644+0000 7f8a87400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:41:59.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.644+0000 7f8a87400700 1 -- 172.21.15.131:0/4179534180 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c017040 con 0x55e8ba9a1530 2024-05-10T05:41:59.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.644+0000 7f8a87400700 1 -- 172.21.15.131:0/4179534180 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8a8c003880 con 0x55e8ba9a1530 2024-05-10T05:41:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.644+0000 7f8ac058ab80 1 -- 172.21.15.131:0/4179534180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a9800be50 con 0x55e8ba9a1530 2024-05-10T05:41:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.645+0000 7f8a87400700 1 -- 172.21.15.131:0/4179534180 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a8c013ed0 con 0x55e8ba9a1530 2024-05-10T05:41:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.648+0000 7f8ac058ab80 10 client.26894 fetch_fsmap learned FSMap version 29 2024-05-10T05:41:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.648+0000 7f8ac058ab80 1 -- 172.21.15.131:0/4179534180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e8ba746cd0 con 0x55e8ba9a1530 2024-05-10T05:41:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.653+0000 7f8a87400700 1 -- 172.21.15.131:0/4179534180 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8c018040 con 0x55e8ba9a1530 2024-05-10T05:41:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.653+0000 7f8a87400700 1 -- 172.21.15.131:0/4179534180 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8a8c021750 con 0x55e8ba9a1530 2024-05-10T05:41:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.653+0000 7f8ac058ab80 10 client.26894 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:41:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.653+0000 7f8ac058ab80 10 client.26894 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:41:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.653+0000 7f8ac058ab80 20 client.26894 populate_metadata read hostname 'smithi131' 2024-05-10T05:41:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.653+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55e8baa0dd00 0x55e8baa10120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:41:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.654+0000 7f8ac058ab80 4 client.26894 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:41:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.654+0000 7f8ac058ab80 1 -- 172.21.15.131:0/4179534180 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f8a9800be50 con 0x55e8baa0dd00 2024-05-10T05:41:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.654+0000 7f8aa5200700 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55e8baa0dd00 0x55e8baa10120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:41:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.659+0000 7f8a87400700 10 client.26894 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:41:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.662+0000 7f8aa5200700 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55e8baa0dd00 0x55e8baa10120 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d1671ada4aaaeda server_cookie=a8b319e5ea08e8b8 in_seq=0 out_seq=0 2024-05-10T05:41:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.664+0000 7f8a87400700 1 -- 172.21.15.131:0/4179534180 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f8a9800be50 con 0x55e8baa0dd00 2024-05-10T05:41:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.664+0000 7f8a87400700 10 client.26894 handle_command_reply: tid=0 2024-05-10T05:41:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.664+0000 7f8ac058ab80 1 client.26894 shutdown 2024-05-10T05:41:59.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.665+0000 7f8ac058ab80 1 -- 172.21.15.131:0/4179534180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a1530 msgr2=0x55e8ba9c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.665+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c012770 tx=0x7f8a8c012650 comp rx=0 tx=0).stop 2024-05-10T05:41:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.667+0000 7f8ac058ab80 1 -- 172.21.15.131:0/4179534180 shutdown_connections 2024-05-10T05:41:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.667+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55e8baa0dd00 0x55e8baa10120 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.667+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e8ba9b0d80 0x55e8ba9c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.667+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ba9a1530 0x55e8ba9c7b40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.667+0000 7f8ac058ab80 1 --2- 172.21.15.131:0/4179534180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ba9a30f0 0x55e8ba9c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.667+0000 7f8ac058ab80 1 -- 172.21.15.131:0/4179534180 >> 172.21.15.131:0/4179534180 conn(0x55e8ba903a60 msgr2=0x55e8ba9aae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.668+0000 7f8a87400700 0 client.26894 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:41:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.669+0000 7f8ac058ab80 1 -- 172.21.15.131:0/4179534180 shutdown_connections 2024-05-10T05:41:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.670+0000 7f8ac058ab80 1 -- 172.21.15.131:0/4179534180 wait complete. 2024-05-10T05:41:59.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.671+0000 7f8ac058ab80 20 client.26894 trim_cache size 0 max 16384 2024-05-10T05:41:59.672 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:41:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.672+0000 7f8aa8600700 1 -- 172.21.15.131:0/2273838511 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a880a1220 msgr2=0x7f8a880a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.672+0000 7f8aa8600700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a880a1220 0x7f8a880a3730 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009550 tx=0x7f8a98006000 comp rx=0 tx=0).stop 2024-05-10T05:41:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.672+0000 7f8aa8600700 1 -- 172.21.15.131:0/2273838511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa0108b60 msgr2=0x7f8aa01a71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:41:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.672+0000 7f8aa8600700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa0108b60 0x7f8aa01a71e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400b450 tx=0x7f8a9400b760 comp rx=0 tx=0).stop 2024-05-10T05:41:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.673+0000 7f8aa8600700 1 -- 172.21.15.131:0/2273838511 shutdown_connections 2024-05-10T05:41:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.673+0000 7f8aa8600700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a880a1220 0x7f8a880a3730 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.673+0000 7f8aa8600700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aa0108210 0x7f8aa01a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.673+0000 7f8aa8600700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aa0109500 0x7f8aa01ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.673+0000 7f8aa8600700 1 --2- 172.21.15.131:0/2273838511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa0108b60 0x7f8aa01a71e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:41:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.673+0000 7f8aa8600700 1 -- 172.21.15.131:0/2273838511 >> 172.21.15.131:0/2273838511 conn(0x7f8aa006e830 msgr2=0x7f8aa006f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:41:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.673+0000 7f8aa8600700 1 -- 172.21.15.131:0/2273838511 shutdown_connections 2024-05-10T05:41:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:41:59.673+0000 7f8aa8600700 1 -- 172.21.15.131:0/2273838511 wait complete. 2024-05-10T05:41:59.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:42:00.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.305+0000 7f1042200700 1 -- 172.21.15.131:0/810295701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103c101be0 msgr2=0x7f103c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:00.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.305+0000 7f1042200700 1 --2- 172.21.15.131:0/810295701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103c101be0 0x7f103c101ff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f102c0099c0 tx=0x7f102c009cd0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.305+0000 7f1042200700 1 -- 172.21.15.131:0/810295701 shutdown_connections 2024-05-10T05:42:00.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.305+0000 7f1042200700 1 --2- 172.21.15.131:0/810295701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f103c1036a0 0x7f103c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.305+0000 7f1042200700 1 --2- 172.21.15.131:0/810295701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103c101be0 0x7f103c101ff0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.305+0000 7f1042200700 1 --2- 172.21.15.131:0/810295701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f103c102d00 0x7f103c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.305+0000 7f1042200700 1 -- 172.21.15.131:0/810295701 >> 172.21.15.131:0/810295701 conn(0x7f103c0fd1d0 msgr2=0x7f103c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:00.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.306+0000 7f1042200700 1 -- 172.21.15.131:0/810295701 shutdown_connections 2024-05-10T05:42:00.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.306+0000 7f1042200700 1 -- 172.21.15.131:0/810295701 wait complete. 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.307+0000 7f1042200700 1 Processor -- start 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.307+0000 7f1042200700 1 -- start start 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.307+0000 7f1042200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103c101be0 0x7f103c06ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.307+0000 7f1042200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f103c102d00 0x7f103c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.307+0000 7f1042200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f103c1036a0 0x7f103c1a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.308+0000 7f1042200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1a81e0 con 0x7f103c102d00 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.308+0000 7f1042200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1a8320 con 0x7f103c1036a0 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.308+0000 7f1042200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1a84e0 con 0x7f103c101be0 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.308+0000 7f103be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f103c1036a0 0x7f103c1a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.308+0000 7f103b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103c101be0 0x7f103c06ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.308+0000 7f103be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f103c1036a0 0x7f103c1a7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47918/0 (socket says 172.21.15.131:47918) 2024-05-10T05:42:00.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.308+0000 7f103be00700 1 -- 172.21.15.131:0/1393460914 learned_addr learned my addr 172.21.15.131:0/1393460914 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.308+0000 7f103aa00700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f103c102d00 0x7f103c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.309+0000 7f103b400700 1 -- 172.21.15.131:0/1393460914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f103c1036a0 msgr2=0x7f103c1a7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.309+0000 7f103b400700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f103c1036a0 0x7f103c1a7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.309+0000 7f103b400700 1 -- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f103c102d00 msgr2=0x7f103c070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:00.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.309+0000 7f103b400700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f103c102d00 0x7f103c070450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.309+0000 7f103b400700 1 -- 172.21.15.131:0/1393460914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c0095f0 con 0x7f103c101be0 2024-05-10T05:42:00.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.309+0000 7f103be00700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f103c1036a0 0x7f103c1a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:42:00.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.309+0000 7f103aa00700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f103c102d00 0x7f103c070450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:42:00.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.309+0000 7f103b400700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103c101be0 0x7f103c06ff10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f102c009fd0 tx=0x7f102c00fab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:00.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.311+0000 7f1033e00700 1 -- 172.21.15.131:0/1393460914 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c01d070 con 0x7f103c101be0 2024-05-10T05:42:00.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.311+0000 7f1033e00700 1 -- 172.21.15.131:0/1393460914 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f102c00bdc0 con 0x7f103c101be0 2024-05-10T05:42:00.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.311+0000 7f1033e00700 1 -- 172.21.15.131:0/1393460914 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c0173f0 con 0x7f103c101be0 2024-05-10T05:42:00.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.311+0000 7f1042200700 1 -- 172.21.15.131:0/1393460914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103c1a8760 con 0x7f103c101be0 2024-05-10T05:42:00.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.311+0000 7f1042200700 1 -- 172.21.15.131:0/1393460914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103c1a8b30 con 0x7f103c101be0 2024-05-10T05:42:00.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.317+0000 7f105a0b9b80 1 Processor -- start 2024-05-10T05:42:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.317+0000 7f105a0b9b80 1 -- start start 2024-05-10T05:42:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.318+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.318+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47300 0x563a88b47710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.318+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 0x563a88b58110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.318+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88a6e150 con 0x563a88b47300 2024-05-10T05:42:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.318+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88a6bb10 con 0x563a88b51170 2024-05-10T05:42:00.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.318+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88a67400 con 0x563a88b47c50 2024-05-10T05:42:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.319+0000 7f103be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.319+0000 7f103be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47926/0 (socket says 172.21.15.131:47926) 2024-05-10T05:42:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.319+0000 7f103be00700 1 -- 172.21.15.131:0/3492060830 learned_addr learned my addr 172.21.15.131:0/3492060830 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.319+0000 7f103aa00700 1 --2- 172.21.15.131:0/3492060830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 0x563a88b58110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.320+0000 7f103aa00700 1 -- 172.21.15.131:0/3492060830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 msgr2=0x563a88b515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.320+0000 7f103aa00700 1 --2- 172.21.15.131:0/3492060830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.320+0000 7f103aa00700 1 -- 172.21.15.131:0/3492060830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47300 msgr2=0x563a88b47710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.320+0000 7f103aa00700 1 --2- 172.21.15.131:0/3492060830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47300 0x563a88b47710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.320+0000 7f103aa00700 1 -- 172.21.15.131:0/3492060830 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a888e83b0 con 0x563a88b47c50 2024-05-10T05:42:00.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.320+0000 7f1033e00700 1 -- 172.21.15.131:0/1393460914 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f102c022470 con 0x7f103c101be0 2024-05-10T05:42:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.320+0000 7f103aa00700 1 --2- 172.21.15.131:0/3492060830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 0x563a88b58110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1028009cc0 tx=0x7f10280113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94e0e3661d8ba1c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.321+0000 7f1027e00700 1 -- 172.21.15.131:0/3492060830 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028011c30 con 0x563a88b47c50 2024-05-10T05:42:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.321+0000 7f1027e00700 1 -- 172.21.15.131:0/3492060830 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1028011d90 con 0x563a88b47c50 2024-05-10T05:42:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.321+0000 7f1027e00700 1 -- 172.21.15.131:0/3492060830 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102800f7b0 con 0x563a88b47c50 2024-05-10T05:42:00.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.322+0000 7f105a0b9b80 1 -- 172.21.15.131:0/3492060830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 msgr2=0x563a88b58110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:00.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.322+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/3492060830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 0x563a88b58110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1028009cc0 tx=0x7f10280113e0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.323+0000 7f1033e00700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10200a13b0 0x7f10200a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.323+0000 7f1033e00700 1 -- 172.21.15.131:0/1393460914 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f102c0cd890 con 0x7f103c101be0 2024-05-10T05:42:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.323+0000 7f105a0b9b80 1 -- 172.21.15.131:0/3492060830 shutdown_connections 2024-05-10T05:42:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.323+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/3492060830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 0x563a88b58110 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.323+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/3492060830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.323+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/3492060830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47300 0x563a88b47710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.323+0000 7f105a0b9b80 1 -- 172.21.15.131:0/3492060830 >> 172.21.15.131:0/3492060830 conn(0x563a88aa9a60 msgr2=0x563a88b41fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.323+0000 7f103be00700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10200a13b0 0x7f10200a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.323+0000 7f105a0b9b80 1 -- 172.21.15.131:0/3492060830 shutdown_connections 2024-05-10T05:42:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.324+0000 7f105a0b9b80 1 -- 172.21.15.131:0/3492060830 wait complete. 2024-05-10T05:42:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.324+0000 7f103be00700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10200a13b0 0x7f10200a38c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1034007790 tx=0x7f1034009040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.324+0000 7f105a0b9b80 1 -- start start 2024-05-10T05:42:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.325+0000 7f105a0b9b80 1 Processor -- start 2024-05-10T05:42:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.326+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b59fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.326+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 0x563a88b5a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.326+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b61460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.326+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88a03560 con 0x563a88b47c50 2024-05-10T05:42:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.326+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88a03200 con 0x563a88b51170 2024-05-10T05:42:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.326+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a888ea6a0 con 0x563a88b47300 2024-05-10T05:42:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.326+0000 7f105a0b9b80 10 client.0 _ll_register_callbacks cb 0x563a88a0cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.326+0000 7f103b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b59fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.326+0000 7f103b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b59fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45264/0 (socket says 172.21.15.131:45264) 2024-05-10T05:42:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.326+0000 7f103b400700 1 -- 172.21.15.131:0/2288346785 learned_addr learned my addr 172.21.15.131:0/2288346785 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.327+0000 7f103be00700 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b61460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.327+0000 7f103aa00700 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 0x563a88b5a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.327+0000 7f103b400700 1 -- 172.21.15.131:0/2288346785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 msgr2=0x563a88b61460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.327+0000 7f103b400700 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b61460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.327+0000 7f103b400700 1 -- 172.21.15.131:0/2288346785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 msgr2=0x563a88b5a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:00.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.327+0000 7f103b400700 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 0x563a88b5a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.327+0000 7f103b400700 1 -- 172.21.15.131:0/2288346785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10280098f0 con 0x563a88b47300 2024-05-10T05:42:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.327+0000 7f103aa00700 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 0x563a88b5a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.328+0000 7f103b400700 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b59fc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f102c09adc0 tx=0x7f102c09b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69724e8ff97a504 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.328+0000 7f1024c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:42:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.328+0000 7f1024c00700 1 -- 172.21.15.131:0/2288346785 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c0a2f90 con 0x563a88b47300 2024-05-10T05:42:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.328+0000 7f1024c00700 1 -- 172.21.15.131:0/2288346785 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f102c09be40 con 0x563a88b47300 2024-05-10T05:42:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.328+0000 7f105a0b9b80 1 -- 172.21.15.131:0/2288346785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a889c0270 con 0x563a88b47300 2024-05-10T05:42:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.328+0000 7f1024c00700 1 -- 172.21.15.131:0/2288346785 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f102c09ae30 con 0x563a88b47300 2024-05-10T05:42:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.329+0000 7f105a0b9b80 10 client.26953 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.329+0000 7f105a0b9b80 1 -- 172.21.15.131:0/2288346785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563a88894b60 con 0x563a88b47300 2024-05-10T05:42:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.329+0000 7f1024c00700 1 -- 172.21.15.131:0/2288346785 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a889c0270 con 0x563a88b47300 2024-05-10T05:42:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.329+0000 7f1024c00700 1 -- 172.21.15.131:0/2288346785 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f102c09fb80 con 0x563a88b47300 2024-05-10T05:42:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.329+0000 7f105a0b9b80 10 client.26953 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.330+0000 7f105a0b9b80 10 client.26953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:42:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.330+0000 7f105a0b9b80 20 client.26953 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.330+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563a88b65740 0x563a88b67b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.330+0000 7f105a0b9b80 4 client.26953 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.330+0000 7f105a0b9b80 1 -- 172.21.15.131:0/2288346785 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a88b16780 con 0x563a88b65740 2024-05-10T05:42:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.330+0000 7f103aa00700 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563a88b65740 0x563a88b67b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.332+0000 7f103aa00700 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563a88b65740 0x563a88b67b60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6334c55cae41103e server_cookie=b8c5fa9a387a4ae2 in_seq=0 out_seq=0 2024-05-10T05:42:00.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.332+0000 7f1024c00700 10 client.26953 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:42:00.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.333+0000 7f1024c00700 1 -- 172.21.15.131:0/2288346785 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a88b16780 con 0x563a88b65740 2024-05-10T05:42:00.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.333+0000 7f1024c00700 10 client.26953 handle_command_reply: tid=0 2024-05-10T05:42:00.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.333+0000 7f105a0b9b80 1 client.26953 shutdown 2024-05-10T05:42:00.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.334+0000 7f105a0b9b80 1 -- 172.21.15.131:0/2288346785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 msgr2=0x563a88b59fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:00.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.334+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b59fc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f102c09adc0 tx=0x7f102c09b370 comp rx=0 tx=0).stop 2024-05-10T05:42:00.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.334+0000 7f105a0b9b80 1 -- 172.21.15.131:0/2288346785 shutdown_connections 2024-05-10T05:42:00.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.334+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563a88b65740 0x563a88b67b60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.334+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b61460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.334+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b59fc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.334+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/2288346785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 0x563a88b5a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.334+0000 7f105a0b9b80 1 -- 172.21.15.131:0/2288346785 >> 172.21.15.131:0/2288346785 conn(0x563a88aa9a60 msgr2=0x563a88b41fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:00.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.334+0000 7f1024c00700 0 client.26953 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:42:00.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.334+0000 7f105a0b9b80 1 -- 172.21.15.131:0/2288346785 shutdown_connections 2024-05-10T05:42:00.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.335+0000 7f105a0b9b80 1 -- 172.21.15.131:0/2288346785 wait complete. 2024-05-10T05:42:00.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.335+0000 7f105a0b9b80 20 client.26953 trim_cache size 0 max 16384 2024-05-10T05:42:00.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.363+0000 7f105a0b9b80 1 Processor -- start 2024-05-10T05:42:00.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.363+0000 7f105a0b9b80 1 -- start start 2024-05-10T05:42:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.364+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47300 0x563a88b58480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.364+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 0x563a88b61420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.364+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.364+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88b65530 con 0x563a88b47300 2024-05-10T05:42:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.364+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88b58100 con 0x563a88b51170 2024-05-10T05:42:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.364+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88b58f00 con 0x563a88b47c50 2024-05-10T05:42:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.365+0000 7f103be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.365+0000 7f103aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 0x563a88b61420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.365+0000 7f103aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 0x563a88b61420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45274/0 (socket says 172.21.15.131:45274) 2024-05-10T05:42:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.365+0000 7f103aa00700 1 -- 172.21.15.131:0/3236424215 learned_addr learned my addr 172.21.15.131:0/3236424215 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.365+0000 7f103b400700 1 --2- 172.21.15.131:0/3236424215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47300 0x563a88b58480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.367+0000 7f103be00700 1 -- 172.21.15.131:0/3236424215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 msgr2=0x563a88b61420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.367+0000 7f103be00700 1 --2- 172.21.15.131:0/3236424215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 0x563a88b61420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.367+0000 7f103be00700 1 -- 172.21.15.131:0/3236424215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47300 msgr2=0x563a88b58480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:00.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.367+0000 7f103be00700 1 --2- 172.21.15.131:0/3236424215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47300 0x563a88b58480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.367+0000 7f103be00700 1 -- 172.21.15.131:0/3236424215 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a88a2d560 con 0x563a88b51170 2024-05-10T05:42:00.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.367+0000 7f103b400700 1 --2- 172.21.15.131:0/3236424215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47300 0x563a88b58480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:00.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.368+0000 7f103be00700 1 --2- 172.21.15.131:0/3236424215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b589c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x563a88b48630 tx=0x7f1034014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb9a7afc27df41 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:00.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.368+0000 7f1031200700 1 -- 172.21.15.131:0/3236424215 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034027070 con 0x563a88b51170 2024-05-10T05:42:00.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.369+0000 7f1031200700 1 -- 172.21.15.131:0/3236424215 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1034017e20 con 0x563a88b51170 2024-05-10T05:42:00.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.369+0000 7f1031200700 1 -- 172.21.15.131:0/3236424215 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034015c40 con 0x563a88b51170 2024-05-10T05:42:00.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.370+0000 7f105a0b9b80 1 -- 172.21.15.131:0/3236424215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 msgr2=0x563a88b589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:00.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.370+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/3236424215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b589c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x563a88b48630 tx=0x7f1034014000 comp rx=0 tx=0).stop 2024-05-10T05:42:00.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.371+0000 7f105a0b9b80 1 -- 172.21.15.131:0/3236424215 shutdown_connections 2024-05-10T05:42:00.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.371+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/3236424215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47300 0x563a88b58480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.371+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/3236424215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b589c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.371+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/3236424215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47c50 0x563a88b61420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.371+0000 7f105a0b9b80 1 -- 172.21.15.131:0/3236424215 >> 172.21.15.131:0/3236424215 conn(0x563a88aa9a60 msgr2=0x563a88b54de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:00.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.372+0000 7f105a0b9b80 1 -- 172.21.15.131:0/3236424215 shutdown_connections 2024-05-10T05:42:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.372+0000 7f105a0b9b80 1 -- 172.21.15.131:0/3236424215 wait complete. 2024-05-10T05:42:00.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.374+0000 7f105a0b9b80 1 -- start start 2024-05-10T05:42:00.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.380+0000 7f105a0b9b80 1 Processor -- start 2024-05-10T05:42:00.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.388+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b5f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.389+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 0x563a88b5f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.390+0000 7f105a0b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b60580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.390+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88b60ac0 con 0x563a88b47c50 2024-05-10T05:42:00.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.390+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88b60c30 con 0x563a88b51170 2024-05-10T05:42:00.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.390+0000 7f105a0b9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a88b60df0 con 0x563a88b47300 2024-05-10T05:42:00.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.391+0000 7f105a0b9b80 10 client.0 _ll_register_callbacks cb 0x563a88a0cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:00.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.389+0000 7f103b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b5f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.391+0000 7f103b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b5f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45282/0 (socket says 172.21.15.131:45282) 2024-05-10T05:42:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.391+0000 7f103b400700 1 -- 172.21.15.131:0/908866515 learned_addr learned my addr 172.21.15.131:0/908866515 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.393+0000 7f103aa00700 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 0x563a88b5f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.395+0000 7f103b400700 1 -- 172.21.15.131:0/908866515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 msgr2=0x563a88b60580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:42:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.395+0000 7f103b400700 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b60580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.395+0000 7f103b400700 1 -- 172.21.15.131:0/908866515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 msgr2=0x563a88b5f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.395+0000 7f103b400700 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 0x563a88b5f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.395+0000 7f103b400700 1 -- 172.21.15.131:0/908866515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a88894b60 con 0x563a88b47300 2024-05-10T05:42:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.396+0000 7f103aa00700 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 0x563a88b5f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:42:00.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.397+0000 7f103b400700 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b5f200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f102c0058b0 tx=0x7f102c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66824d3c4f31b8bc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:00.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.397+0000 7f1024c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:42:00.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.398+0000 7f1024c00700 1 -- 172.21.15.131:0/908866515 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c09ec60 con 0x563a88b47300 2024-05-10T05:42:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.398+0000 7f1024c00700 1 -- 172.21.15.131:0/908866515 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f102c09be40 con 0x563a88b47300 2024-05-10T05:42:00.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.398+0000 7f105a0b9b80 1 -- 172.21.15.131:0/908866515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1028011d90 con 0x563a88b47300 2024-05-10T05:42:00.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.399+0000 7f1024c00700 1 -- 172.21.15.131:0/908866515 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f102c0a3220 con 0x563a88b47300 2024-05-10T05:42:00.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.399+0000 7f105a0b9b80 10 client.26959 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:00.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.399+0000 7f105a0b9b80 1 -- 172.21.15.131:0/908866515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f103400f040 con 0x563a88b47300 2024-05-10T05:42:00.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.400+0000 7f1024c00700 1 -- 172.21.15.131:0/908866515 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1028011d90 con 0x563a88b47300 2024-05-10T05:42:00.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.402+0000 7f1024c00700 1 -- 172.21.15.131:0/908866515 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f102c095a40 con 0x563a88b47300 2024-05-10T05:42:00.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.402+0000 7f105a0b9b80 10 client.26959 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:00.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.402+0000 7f105a0b9b80 10 client.26959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:42:00.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.402+0000 7f105a0b9b80 20 client.26959 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:00.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.402+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563a88bbc5c0 0x563a88bbe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:00.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.402+0000 7f105a0b9b80 4 client.26959 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:42:00.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.402+0000 7f105a0b9b80 1 -- 172.21.15.131:0/908866515 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563a88a5fec0 con 0x563a88bbc5c0 2024-05-10T05:42:00.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.403+0000 7f103aa00700 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563a88bbc5c0 0x563a88bbe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:00.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.404+0000 7f103aa00700 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563a88bbc5c0 0x563a88bbe9e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b3ff36c7a0e36a6 server_cookie=471f0edf7397faa8 in_seq=0 out_seq=0 2024-05-10T05:42:00.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.404+0000 7f1024c00700 10 client.26959 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:42:00.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f1024c00700 1 -- 172.21.15.131:0/908866515 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563a88a5fec0 con 0x563a88bbc5c0 2024-05-10T05:42:00.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f1024c00700 10 client.26959 handle_command_reply: tid=0 2024-05-10T05:42:00.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f105a0b9b80 1 client.26959 shutdown 2024-05-10T05:42:00.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f105a0b9b80 1 -- 172.21.15.131:0/908866515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 msgr2=0x563a88b5f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:00.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b5f200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f102c0058b0 tx=0x7f102c009310 comp rx=0 tx=0).stop 2024-05-10T05:42:00.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f105a0b9b80 1 -- 172.21.15.131:0/908866515 shutdown_connections 2024-05-10T05:42:00.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563a88bbc5c0 0x563a88bbe9e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a88b51170 0x563a88b60580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a88b47300 0x563a88b5f200 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f105a0b9b80 1 --2- 172.21.15.131:0/908866515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a88b47c50 0x563a88b5f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.405+0000 7f105a0b9b80 1 -- 172.21.15.131:0/908866515 >> 172.21.15.131:0/908866515 conn(0x563a88aa9a60 msgr2=0x563a88b54de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:00.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.406+0000 7f1024c00700 0 client.26959 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:42:00.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.406+0000 7f105a0b9b80 1 -- 172.21.15.131:0/908866515 shutdown_connections 2024-05-10T05:42:00.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.406+0000 7f105a0b9b80 1 -- 172.21.15.131:0/908866515 wait complete. 2024-05-10T05:42:00.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.407+0000 7f105a0b9b80 20 client.26959 trim_cache size 0 max 16384 2024-05-10T05:42:00.408 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:42:00.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.407+0000 7f1042200700 1 -- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10200a13b0 msgr2=0x7f10200a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:00.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10200a13b0 0x7f10200a38c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1034007790 tx=0x7f1034009040 comp rx=0 tx=0).stop 2024-05-10T05:42:00.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 -- 172.21.15.131:0/1393460914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103c101be0 msgr2=0x7f103c06ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:00.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103c101be0 0x7f103c06ff10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f102c009fd0 tx=0x7f102c00fab0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 -- 172.21.15.131:0/1393460914 shutdown_connections 2024-05-10T05:42:00.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f10200a13b0 0x7f10200a38c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f103c1036a0 0x7f103c1a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f103c101be0 0x7f103c06ff10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 --2- 172.21.15.131:0/1393460914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f103c102d00 0x7f103c070450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:00.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 -- 172.21.15.131:0/1393460914 >> 172.21.15.131:0/1393460914 conn(0x7f103c0fd1d0 msgr2=0x7f103c105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:00.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 -- 172.21.15.131:0/1393460914 shutdown_connections 2024-05-10T05:42:00.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:00.408+0000 7f1042200700 1 -- 172.21.15.131:0/1393460914 wait complete. 2024-05-10T05:42:00.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:42:00.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:00 smithi142 ceph-mon[77667]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 266 MiB/s wr, 83 op/s 2024-05-10T05:42:00.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:00 smithi131 ceph-mon[37264]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 266 MiB/s wr, 83 op/s 2024-05-10T05:42:00.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:00 smithi157 ceph-mon[78427]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 266 MiB/s wr, 83 op/s 2024-05-10T05:42:01.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.206+0000 7fc9be200700 1 -- 172.21.15.131:0/1872604684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8109c50 msgr2=0x7fc9b810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:01.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.206+0000 7fc9be200700 1 --2- 172.21.15.131:0/1872604684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8109c50 0x7fc9b810e560 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009f90 tx=0x7fc9b000b330 comp rx=0 tx=0).stop 2024-05-10T05:42:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.215+0000 7fc9be200700 1 -- 172.21.15.131:0/1872604684 shutdown_connections 2024-05-10T05:42:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.215+0000 7fc9be200700 1 --2- 172.21.15.131:0/1872604684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b8108630 0x7fc9b8108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.215+0000 7fc9be200700 1 --2- 172.21.15.131:0/1872604684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8109c50 0x7fc9b810e560 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.215+0000 7fc9be200700 1 --2- 172.21.15.131:0/1872604684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8109300 0x7fc9b8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.215+0000 7fc9be200700 1 -- 172.21.15.131:0/1872604684 >> 172.21.15.131:0/1872604684 conn(0x7fc9b806ea00 msgr2=0x7fc9b8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:01.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.215+0000 7fc9be200700 1 -- 172.21.15.131:0/1872604684 shutdown_connections 2024-05-10T05:42:01.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.216+0000 7fc9be200700 1 -- 172.21.15.131:0/1872604684 wait complete. 2024-05-10T05:42:01.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.221+0000 7fc9be200700 1 Processor -- start 2024-05-10T05:42:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.222+0000 7fc9be200700 1 -- start start 2024-05-10T05:42:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.222+0000 7fc9be200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b8108630 0x7fc9b81a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.223+0000 7fc9be200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8109300 0x7fc9b81a7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.223+0000 7fc9be200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8109c50 0x7fc9b81ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.223+0000 7fc9be200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b81ae7c0 con 0x7fc9b8108630 2024-05-10T05:42:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.223+0000 7fc9be200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b81ae930 con 0x7fc9b8109300 2024-05-10T05:42:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.223+0000 7fc9be200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b81aeaf0 con 0x7fc9b8109c50 2024-05-10T05:42:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.224+0000 7fc9b7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b8108630 0x7fc9b81a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.224+0000 7fc9b6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8109300 0x7fc9b81a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.224+0000 7fc9b6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8109300 0x7fc9b81a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47958/0 (socket says 172.21.15.131:47958) 2024-05-10T05:42:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.224+0000 7fc9b7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8109c50 0x7fc9b81ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.224+0000 7fc9b6a00700 1 -- 172.21.15.131:0/135645162 learned_addr learned my addr 172.21.15.131:0/135645162 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.224+0000 7fc9b6a00700 1 -- 172.21.15.131:0/135645162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8109c50 msgr2=0x7fc9b81ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.224+0000 7fc9b6a00700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8109c50 0x7fc9b81ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.224+0000 7fc9b6a00700 1 -- 172.21.15.131:0/135645162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b8108630 msgr2=0x7fc9b81a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:01.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.224+0000 7fc9b6a00700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b8108630 0x7fc9b81a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.226+0000 7fc9b6a00700 1 -- 172.21.15.131:0/135645162 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b0009c40 con 0x7fc9b8109300 2024-05-10T05:42:01.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.226+0000 7fc9b6a00700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8109300 0x7fc9b81a7200 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400e9c0 tx=0x7fc9a400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:01.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.226+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8109c50 0x7fc9b81ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:42:01.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.227+0000 7fc9afe00700 1 -- 172.21.15.131:0/135645162 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a400cbd0 con 0x7fc9b8109300 2024-05-10T05:42:01.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.227+0000 7fc9be200700 1 -- 172.21.15.131:0/135645162 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b81aedd0 con 0x7fc9b8109300 2024-05-10T05:42:01.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.227+0000 7fc9afe00700 1 -- 172.21.15.131:0/135645162 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9a400cd30 con 0x7fc9b8109300 2024-05-10T05:42:01.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.227+0000 7fc9afe00700 1 -- 172.21.15.131:0/135645162 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4005550 con 0x7fc9b8109300 2024-05-10T05:42:01.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.229+0000 7fc9be200700 1 -- 172.21.15.131:0/135645162 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b81af250 con 0x7fc9b8109300 2024-05-10T05:42:01.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.244+0000 7fc9afe00700 1 -- 172.21.15.131:0/135645162 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc9a4004cf0 con 0x7fc9b8109300 2024-05-10T05:42:01.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.247+0000 7fc9afe00700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc99c0a1310 0x7fc99c0a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.247+0000 7fc9afe00700 1 -- 172.21.15.131:0/135645162 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc9a4014070 con 0x7fc9b8109300 2024-05-10T05:42:01.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.248+0000 7fc9b7400700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc99c0a1310 0x7fc99c0a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.250+0000 7fc9b7400700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc99c0a1310 0x7fc99c0a3820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc9b81a7b50 tx=0x7fc9a8008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.250+0000 7fc9d6106b80 1 Processor -- start 2024-05-10T05:42:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.250+0000 7fc9d6106b80 1 -- start start 2024-05-10T05:42:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.251+0000 7fc9d6106b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0e9eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.251+0000 7fc9d6106b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0e9f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.251+0000 7fc9d6106b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ea67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.251+0000 7fc9d6106b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0dc5150 con 0x5615e0e9e6a0 2024-05-10T05:42:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.251+0000 7fc9d6106b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0dc2b10 con 0x5615e0e9fec0 2024-05-10T05:42:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.251+0000 7fc9d6106b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0dbe400 con 0x5615e0e9f570 2024-05-10T05:42:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.252+0000 7fc9b7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ea67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.252+0000 7fc9b7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ea67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47964/0 (socket says 172.21.15.131:47964) 2024-05-10T05:42:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.252+0000 7fc9b7e00700 1 -- 172.21.15.131:0/1495946777 learned_addr learned my addr 172.21.15.131:0/1495946777 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.252+0000 7fc9b7400700 1 --2- 172.21.15.131:0/1495946777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0e9eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.253+0000 7fc9b7e00700 1 -- 172.21.15.131:0/1495946777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 msgr2=0x5615e0e9f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.253+0000 7fc9b6a00700 1 --2- 172.21.15.131:0/1495946777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0e9f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.253+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/1495946777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0e9f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.253+0000 7fc9b7e00700 1 -- 172.21.15.131:0/1495946777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 msgr2=0x5615e0e9eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.253+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/1495946777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0e9eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.253+0000 7fc9b7e00700 1 -- 172.21.15.131:0/1495946777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615e0c3f3b0 con 0x5615e0e9fec0 2024-05-10T05:42:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.253+0000 7fc9b6a00700 1 --2- 172.21.15.131:0/1495946777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0e9f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.254+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/1495946777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ea67c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009690 tx=0x7fc9b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8a38a7b8f270b60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.254+0000 7fc9a3e00700 1 -- 172.21.15.131:0/1495946777 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b000e040 con 0x5615e0e9fec0 2024-05-10T05:42:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.254+0000 7fc9a3e00700 1 -- 172.21.15.131:0/1495946777 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9b0007780 con 0x5615e0e9fec0 2024-05-10T05:42:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.254+0000 7fc9a3e00700 1 -- 172.21.15.131:0/1495946777 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b000e040 con 0x5615e0e9fec0 2024-05-10T05:42:01.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.256+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1495946777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 msgr2=0x5615e0ea67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:01.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.256+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/1495946777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ea67c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009690 tx=0x7fc9b0009780 comp rx=0 tx=0).stop 2024-05-10T05:42:01.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.256+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1495946777 shutdown_connections 2024-05-10T05:42:01.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.256+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/1495946777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0e9eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.256+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/1495946777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ea67c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.257+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/1495946777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0e9f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.257+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1495946777 >> 172.21.15.131:0/1495946777 conn(0x5615e0e00a60 msgr2=0x5615e0e98fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.257+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1495946777 shutdown_connections 2024-05-10T05:42:01.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.257+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1495946777 wait complete. 2024-05-10T05:42:01.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.258+0000 7fc9d6106b80 1 -- start start 2024-05-10T05:42:01.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.258+0000 7fc9d6106b80 1 Processor -- start 2024-05-10T05:42:01.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.260+0000 7fc9d6106b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0eb94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.260+0000 7fc9d6106b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0eb99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.260+0000 7fc9d6106b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ec0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.261+0000 7fc9d6106b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0d5a560 con 0x5615e0e9e6a0 2024-05-10T05:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.261+0000 7fc9d6106b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0d5a200 con 0x5615e0e9fec0 2024-05-10T05:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.261+0000 7fc9d6106b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0c416a0 con 0x5615e0e9f570 2024-05-10T05:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.261+0000 7fc9d6106b80 10 client.0 _ll_register_callbacks cb 0x5615e0d63b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.261+0000 7fc9b7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ec0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.261+0000 7fc9b7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ec0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47974/0 (socket says 172.21.15.131:47974) 2024-05-10T05:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.261+0000 7fc9b7e00700 1 -- 172.21.15.131:0/2383809893 learned_addr learned my addr 172.21.15.131:0/2383809893 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.262+0000 7fc9b7400700 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0eb94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.263+0000 7fc9b7e00700 1 -- 172.21.15.131:0/2383809893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 msgr2=0x5615e0eb99e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.263+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0eb99e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.263+0000 7fc9b7e00700 1 -- 172.21.15.131:0/2383809893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 msgr2=0x5615e0eb94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.263+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0eb94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.263+0000 7fc9b7e00700 1 -- 172.21.15.131:0/2383809893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b810efc0 con 0x5615e0e9fec0 2024-05-10T05:42:01.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.264+0000 7fc9a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:01.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.264+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ec0940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0004f40 tx=0x7fc9b000bdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1651be0bc7e16c04 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:01.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.264+0000 7fc9a0c00700 1 -- 172.21.15.131:0/2383809893 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b002d070 con 0x5615e0e9fec0 2024-05-10T05:42:01.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.265+0000 7fc9a0c00700 1 -- 172.21.15.131:0/2383809893 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc9b00138b0 con 0x5615e0e9fec0 2024-05-10T05:42:01.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.265+0000 7fc9d6106b80 1 -- 172.21.15.131:0/2383809893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615e0d17270 con 0x5615e0e9fec0 2024-05-10T05:42:01.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.267+0000 7fc9a0c00700 1 -- 172.21.15.131:0/2383809893 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9b000e040 con 0x5615e0e9fec0 2024-05-10T05:42:01.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.267+0000 7fc9d6106b80 10 client.26918 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:01.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.267+0000 7fc9d6106b80 1 -- 172.21.15.131:0/2383809893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5615e0bebb60 con 0x5615e0e9fec0 2024-05-10T05:42:01.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.269+0000 7fc9a0c00700 1 -- 172.21.15.131:0/2383809893 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615e0d17270 con 0x5615e0e9fec0 2024-05-10T05:42:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.272+0000 7fc9a0c00700 1 -- 172.21.15.131:0/2383809893 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc9b0030020 con 0x5615e0e9fec0 2024-05-10T05:42:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.272+0000 7fc9d6106b80 10 client.26918 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.272+0000 7fc9d6106b80 10 client.26918 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:42:01.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.273+0000 7fc9d6106b80 20 client.26918 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.273+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5615e0ec4c80 0x5615e0ec70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.273+0000 7fc9d6106b80 4 client.26918 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.273+0000 7fc9d6106b80 1 -- 172.21.15.131:0/2383809893 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615e0e6d780 con 0x5615e0ec4c80 2024-05-10T05:42:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.274+0000 7fc9b7400700 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5615e0ec4c80 0x5615e0ec70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.275+0000 7fc9a0c00700 10 client.26918 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:42:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.276+0000 7fc9b7400700 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5615e0ec4c80 0x5615e0ec70d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=781849b3a4be2d77 server_cookie=cb4ebf0f108b485e in_seq=0 out_seq=0 2024-05-10T05:42:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.277+0000 7fc9a0c00700 1 -- 172.21.15.131:0/2383809893 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615e0e6d780 con 0x5615e0ec4c80 2024-05-10T05:42:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.277+0000 7fc9a0c00700 10 client.26918 handle_command_reply: tid=0 2024-05-10T05:42:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.277+0000 7fc9d6106b80 1 client.26918 shutdown 2024-05-10T05:42:01.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.278+0000 7fc9d6106b80 1 -- 172.21.15.131:0/2383809893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 msgr2=0x5615e0ec0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:01.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.278+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ec0940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0004f40 tx=0x7fc9b000bdb0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.279+0000 7fc9d6106b80 1 -- 172.21.15.131:0/2383809893 shutdown_connections 2024-05-10T05:42:01.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.279+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5615e0ec4c80 0x5615e0ec70d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.279+0000 7fc9a0c00700 0 client.26918 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:42:01.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.279+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0eb94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.279+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ec0940 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.279+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/2383809893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0eb99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.279+0000 7fc9d6106b80 1 -- 172.21.15.131:0/2383809893 >> 172.21.15.131:0/2383809893 conn(0x5615e0e00a60 msgr2=0x5615e0e98fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.280+0000 7fc9d6106b80 1 -- 172.21.15.131:0/2383809893 shutdown_connections 2024-05-10T05:42:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.280+0000 7fc9d6106b80 1 -- 172.21.15.131:0/2383809893 wait complete. 2024-05-10T05:42:01.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.286+0000 7fc9d6106b80 20 client.26918 trim_cache size 0 max 16384 2024-05-10T05:42:01.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.365+0000 7fc9d6106b80 1 Processor -- start 2024-05-10T05:42:01.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.366+0000 7fc9d6106b80 1 -- start start 2024-05-10T05:42:01.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.368+0000 7fc9d6106b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0eb7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.368+0000 7fc9d6106b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0ebe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.368+0000 7fc9b6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0ebe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.369+0000 7fc9b6a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0ebe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45316/0 (socket says 172.21.15.131:45316) 2024-05-10T05:42:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.369+0000 7fc9b6a00700 1 -- 172.21.15.131:0/1446326316 learned_addr learned my addr 172.21.15.131:0/1446326316 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.371+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/1446326316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ebee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.371+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1446326316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0e9f3c0 con 0x5615e0e9e6a0 2024-05-10T05:42:01.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.371+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/1446326316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ebee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.371+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1446326316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0ec1a40 con 0x5615e0e9fec0 2024-05-10T05:42:01.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.371+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1446326316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0e9a770 con 0x5615e0e9f570 2024-05-10T05:42:01.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.372+0000 7fc9b7e00700 1 -- 172.21.15.131:0/1446326316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 msgr2=0x5615e0ebe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:01.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.372+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/1446326316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0ebe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.372+0000 7fc9b7e00700 1 -- 172.21.15.131:0/1446326316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 msgr2=0x5615e0eb7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.374+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/1446326316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0eb7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.374+0000 7fc9b7e00700 1 -- 172.21.15.131:0/1446326316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615e0d84560 con 0x5615e0e9fec0 2024-05-10T05:42:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.374+0000 7fc9b6a00700 1 --2- 172.21.15.131:0/1446326316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0ebe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.374+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/1446326316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ebee30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0012520 tx=0x7fc9b001bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e13250d1d863d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:01.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.376+0000 7fc9ad200700 1 -- 172.21.15.131:0/1446326316 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b002d030 con 0x5615e0e9fec0 2024-05-10T05:42:01.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.376+0000 7fc9ad200700 1 -- 172.21.15.131:0/1446326316 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9b0009620 con 0x5615e0e9fec0 2024-05-10T05:42:01.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.376+0000 7fc9ad200700 1 -- 172.21.15.131:0/1446326316 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b000e040 con 0x5615e0e9fec0 2024-05-10T05:42:01.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.378+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1446326316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 msgr2=0x5615e0ebee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:01.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.378+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/1446326316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ebee30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0012520 tx=0x7fc9b001bee0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.378+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1446326316 shutdown_connections 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.378+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/1446326316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9e6a0 0x5615e0eb7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.378+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/1446326316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9fec0 0x5615e0ebee30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.378+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/1446326316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9f570 0x5615e0ebe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.378+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1446326316 >> 172.21.15.131:0/1446326316 conn(0x5615e0e00a60 msgr2=0x5615e0ec70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.379+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1446326316 shutdown_connections 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.379+0000 7fc9d6106b80 1 -- 172.21.15.131:0/1446326316 wait complete. 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.380+0000 7fc9d6106b80 1 -- start start 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.380+0000 7fc9d6106b80 1 Processor -- start 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.381+0000 7fc9d6106b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9e6a0 0x5615e0ebe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.381+0000 7fc9d6106b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9f570 0x5615e0ebe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.381+0000 7fc9b7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9e6a0 0x5615e0ebe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.382+0000 7fc9b7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9e6a0 0x5615e0ebe1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:48000/0 (socket says 172.21.15.131:48000) 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.382+0000 7fc9b7400700 1 -- 172.21.15.131:0/3149721810 learned_addr learned my addr 172.21.15.131:0/3149721810 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.382+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9fec0 0x5615e0f193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.382+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0f19910 con 0x5615e0e9f570 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.382+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0f19a50 con 0x5615e0e9e6a0 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.382+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e0f19be0 con 0x5615e0e9fec0 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.382+0000 7fc9d6106b80 10 client.0 _ll_register_callbacks cb 0x5615e0d63b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.382+0000 7fc9b6a00700 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9f570 0x5615e0ebe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.382+0000 7fc9b7e00700 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9fec0 0x5615e0f193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.383+0000 7fc9b7400700 1 -- 172.21.15.131:0/3149721810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9fec0 msgr2=0x5615e0f193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.383+0000 7fc9b7400700 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9fec0 0x5615e0f193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.383+0000 7fc9b7400700 1 -- 172.21.15.131:0/3149721810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9f570 msgr2=0x5615e0ebe6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.383+0000 7fc9b7400700 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9f570 0x5615e0ebe6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.383+0000 7fc9b7400700 1 -- 172.21.15.131:0/3149721810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b81af250 con 0x5615e0e9e6a0 2024-05-10T05:42:01.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.384+0000 7fc9a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:01.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.384+0000 7fc9b7400700 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9e6a0 0x5615e0ebe1b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800a1a0 tx=0x7fc9a8004800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc51be15e3e9966e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:01.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.388+0000 7fc9a0c00700 1 -- 172.21.15.131:0/3149721810 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80119b0 con 0x5615e0e9e6a0 2024-05-10T05:42:01.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.388+0000 7fc9a0c00700 1 -- 172.21.15.131:0/3149721810 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc9a8011b10 con 0x5615e0e9e6a0 2024-05-10T05:42:01.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.388+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9b0007780 con 0x5615e0e9e6a0 2024-05-10T05:42:01.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.403+0000 7fc9a0c00700 1 -- 172.21.15.131:0/3149721810 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9a8011cd0 con 0x5615e0e9e6a0 2024-05-10T05:42:01.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.403+0000 7fc9d6106b80 10 client.26924 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:01.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.403+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5615e0bebb60 con 0x5615e0e9e6a0 2024-05-10T05:42:01.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.404+0000 7fc9a0c00700 1 -- 172.21.15.131:0/3149721810 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9b0007780 con 0x5615e0e9e6a0 2024-05-10T05:42:01.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.405+0000 7fc9a0c00700 1 -- 172.21.15.131:0/3149721810 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc9a8022420 con 0x5615e0e9e6a0 2024-05-10T05:42:01.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.405+0000 7fc9d6106b80 10 client.26924 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:01.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.405+0000 7fc9d6106b80 10 client.26924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:42:01.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.405+0000 7fc9d6106b80 20 client.26924 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:01.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.405+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5615e0ec4c80 0x5615e0f1d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:01.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.405+0000 7fc9d6106b80 4 client.26924 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:42:01.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.405+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615e0db6ec0 con 0x5615e0ec4c80 2024-05-10T05:42:01.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.406+0000 7fc9b6a00700 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5615e0ec4c80 0x5615e0f1d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:01.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.408+0000 7fc9a0c00700 10 client.26924 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:42:01.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.408+0000 7fc9b6a00700 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5615e0ec4c80 0x5615e0f1d600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d0db51809a3590b server_cookie=6f0c346058563f8c in_seq=0 out_seq=0 2024-05-10T05:42:01.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.409+0000 7fc9a0c00700 1 -- 172.21.15.131:0/3149721810 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615e0db6ec0 con 0x5615e0ec4c80 2024-05-10T05:42:01.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.409+0000 7fc9a0c00700 10 client.26924 handle_command_reply: tid=0 2024-05-10T05:42:01.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.409+0000 7fc9d6106b80 1 client.26924 shutdown 2024-05-10T05:42:01.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.410+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9e6a0 msgr2=0x5615e0ebe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:01.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.410+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9e6a0 0x5615e0ebe1b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800a1a0 tx=0x7fc9a8004800 comp rx=0 tx=0).stop 2024-05-10T05:42:01.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.412+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 shutdown_connections 2024-05-10T05:42:01.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.412+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5615e0ec4c80 0x5615e0f1d600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.412+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615e0e9fec0 0x5615e0f193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.412+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615e0e9e6a0 0x5615e0ebe1b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.412+0000 7fc9a0c00700 0 client.26924 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:42:01.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.412+0000 7fc9d6106b80 1 --2- 172.21.15.131:0/3149721810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615e0e9f570 0x5615e0ebe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.412+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 >> 172.21.15.131:0/3149721810 conn(0x5615e0e00a60 msgr2=0x5615e0eb7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:01.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.415+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 shutdown_connections 2024-05-10T05:42:01.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.415+0000 7fc9d6106b80 1 -- 172.21.15.131:0/3149721810 wait complete. 2024-05-10T05:42:01.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.417+0000 7fc9d6106b80 20 client.26924 trim_cache size 0 max 16384 2024-05-10T05:42:01.419 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:42:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.418+0000 7fc9be200700 1 -- 172.21.15.131:0/135645162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc99c0a1310 msgr2=0x7fc99c0a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.418+0000 7fc9be200700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc99c0a1310 0x7fc99c0a3820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc9b81a7b50 tx=0x7fc9a8008040 comp rx=0 tx=0).stop 2024-05-10T05:42:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.418+0000 7fc9be200700 1 -- 172.21.15.131:0/135645162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8109300 msgr2=0x7fc9b81a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.418+0000 7fc9be200700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8109300 0x7fc9b81a7200 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400e9c0 tx=0x7fc9a400ed80 comp rx=0 tx=0).stop 2024-05-10T05:42:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.419+0000 7fc9be200700 1 -- 172.21.15.131:0/135645162 shutdown_connections 2024-05-10T05:42:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.419+0000 7fc9be200700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc99c0a1310 0x7fc99c0a3820 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.419+0000 7fc9be200700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b8108630 0x7fc9b81a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.420+0000 7fc9be200700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b8109c50 0x7fc9b81ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.420+0000 7fc9be200700 1 --2- 172.21.15.131:0/135645162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8109300 0x7fc9b81a7200 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.420+0000 7fc9be200700 1 -- 172.21.15.131:0/135645162 >> 172.21.15.131:0/135645162 conn(0x7fc9b806ea00 msgr2=0x7fc9b810a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.421+0000 7fc9be200700 1 -- 172.21.15.131:0/135645162 shutdown_connections 2024-05-10T05:42:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:01.421+0000 7fc9be200700 1 -- 172.21.15.131:0/135645162 wait complete. 2024-05-10T05:42:01.484 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:42:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.183+0000 7f7504a00700 1 -- 172.21.15.131:0/893040786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7500104f20 msgr2=0x7f7500105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:02.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.183+0000 7f7504a00700 1 --2- 172.21.15.131:0/893040786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7500104f20 0x7f7500105330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f74f00099c0 tx=0x7f74f0009cd0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.183+0000 7f7504a00700 1 -- 172.21.15.131:0/893040786 shutdown_connections 2024-05-10T05:42:02.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.183+0000 7f7504a00700 1 --2- 172.21.15.131:0/893040786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75001058c0 0x7f750010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.183+0000 7f7504a00700 1 --2- 172.21.15.131:0/893040786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7500103e00 0x7f7500104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.183+0000 7f7504a00700 1 --2- 172.21.15.131:0/893040786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7500104f20 0x7f7500105330 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.183+0000 7f7504a00700 1 -- 172.21.15.131:0/893040786 >> 172.21.15.131:0/893040786 conn(0x7f75000ff410 msgr2=0x7f75001018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:02.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.184+0000 7f7504a00700 1 -- 172.21.15.131:0/893040786 shutdown_connections 2024-05-10T05:42:02.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.184+0000 7f7504a00700 1 -- 172.21.15.131:0/893040786 wait complete. 2024-05-10T05:42:02.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.185+0000 7f7504a00700 1 Processor -- start 2024-05-10T05:42:02.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.186+0000 7f7504a00700 1 -- start start 2024-05-10T05:42:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.186+0000 7f7504a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7500103e00 0x7f750019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.186+0000 7f7504a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7500104f20 0x7f750019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.186+0000 7f7504a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75001058c0 0x7f75001a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.186+0000 7f7504a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75001a6080 con 0x7f75001058c0 2024-05-10T05:42:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.186+0000 7f7504a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75001a61f0 con 0x7f7500103e00 2024-05-10T05:42:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.186+0000 7f7504a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75001a63b0 con 0x7f7500104f20 2024-05-10T05:42:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.186+0000 7f74fde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7500103e00 0x7f750019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.186+0000 7f74fde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7500103e00 0x7f750019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:48024/0 (socket says 172.21.15.131:48024) 2024-05-10T05:42:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.186+0000 7f74fde00700 1 -- 172.21.15.131:0/559121442 learned_addr learned my addr 172.21.15.131:0/559121442 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.187+0000 7f74fde00700 1 -- 172.21.15.131:0/559121442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7500104f20 msgr2=0x7f750019eae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:42:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.187+0000 7f74fde00700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7500104f20 0x7f750019eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.187+0000 7f74fde00700 1 -- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75001058c0 msgr2=0x7f75001a5a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:42:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.187+0000 7f74fe800700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75001058c0 0x7f75001a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.188+0000 7f74fde00700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75001058c0 0x7f75001a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.188+0000 7f74fde00700 1 -- 172.21.15.131:0/559121442 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f00095f0 con 0x7f7500103e00 2024-05-10T05:42:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.188+0000 7f74fe800700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75001058c0 0x7f75001a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.189+0000 7f74fde00700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7500103e00 0x7f750019e5a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b9b0 tx=0x7f74f800bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.189+0000 7f74f6a00700 1 -- 172.21.15.131:0/559121442 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f800ca30 con 0x7f7500103e00 2024-05-10T05:42:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.189+0000 7f74f6a00700 1 -- 172.21.15.131:0/559121442 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74f800cb90 con 0x7f7500103e00 2024-05-10T05:42:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.189+0000 7f7504a00700 1 -- 172.21.15.131:0/559121442 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75001a6690 con 0x7f7500103e00 2024-05-10T05:42:02.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.189+0000 7f7504a00700 1 -- 172.21.15.131:0/559121442 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75001a6bf0 con 0x7f7500103e00 2024-05-10T05:42:02.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.190+0000 7f74f6a00700 1 -- 172.21.15.131:0/559121442 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f80125a0 con 0x7f7500103e00 2024-05-10T05:42:02.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.195+0000 7f751c97fb80 1 Processor -- start 2024-05-10T05:42:02.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.196+0000 7f74f6a00700 1 -- 172.21.15.131:0/559121442 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f74f8012700 con 0x7f7500103e00 2024-05-10T05:42:02.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.196+0000 7f751c97fb80 1 -- start start 2024-05-10T05:42:02.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.197+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b46a0 0x56177a8b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.197+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b5570 0x56177a8b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.197+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8bc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.197+0000 7f751c97fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a7db150 con 0x56177a8b5570 2024-05-10T05:42:02.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.197+0000 7f74fe800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b46a0 0x56177a8b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.197+0000 7f751c97fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a7d8b10 con 0x56177a8b46a0 2024-05-10T05:42:02.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.197+0000 7f751c97fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a7d4400 con 0x56177a8b5ec0 2024-05-10T05:42:02.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.197+0000 7f74fe800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b46a0 0x56177a8b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:48030/0 (socket says 172.21.15.131:48030) 2024-05-10T05:42:02.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.197+0000 7f74fe800700 1 -- 172.21.15.131:0/793068041 learned_addr learned my addr 172.21.15.131:0/793068041 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:02.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.198+0000 7f74fd400700 1 --2- 172.21.15.131:0/793068041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8bc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.198+0000 7f74fde00700 1 --2- 172.21.15.131:0/793068041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b5570 0x56177a8b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.198+0000 7f74f6a00700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f74e40a1240 0x7f74e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.198+0000 7f74fe800700 1 -- 172.21.15.131:0/793068041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 msgr2=0x56177a8bc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:02.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.198+0000 7f74fe800700 1 --2- 172.21.15.131:0/793068041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8bc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.198+0000 7f74fe800700 1 -- 172.21.15.131:0/793068041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b5570 msgr2=0x56177a8b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:02.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.199+0000 7f74f6a00700 1 -- 172.21.15.131:0/559121442 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f74f80cbed0 con 0x7f7500103e00 2024-05-10T05:42:02.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.199+0000 7f74fe800700 1 --2- 172.21.15.131:0/793068041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b5570 0x56177a8b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.199+0000 7f74fe800700 1 -- 172.21.15.131:0/793068041 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56177a6553b0 con 0x56177a8b46a0 2024-05-10T05:42:02.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.199+0000 7f74fde00700 1 --2- 172.21.15.131:0/793068041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b5570 0x56177a8b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:42:02.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.200+0000 7f74fe800700 1 --2- 172.21.15.131:0/793068041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b46a0 0x56177a8b4af0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009c00 tx=0x7f74e8009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e4093b0ca73ba2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:02.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.200+0000 7f74fe800700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f74e40a1240 0x7f74e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.200+0000 7f74ef000700 1 -- 172.21.15.131:0/793068041 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800ebb0 con 0x56177a8b46a0 2024-05-10T05:42:02.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.200+0000 7f74ef000700 1 -- 172.21.15.131:0/793068041 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74e800eeb0 con 0x56177a8b46a0 2024-05-10T05:42:02.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.200+0000 7f74ef000700 1 -- 172.21.15.131:0/793068041 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e80109f0 con 0x56177a8b46a0 2024-05-10T05:42:02.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.202+0000 7f74fe800700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f74e40a1240 0x7f74e40a3750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f74e8018000 tx=0x7f74e8010c30 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:02.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.202+0000 7f751c97fb80 1 -- 172.21.15.131:0/793068041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b46a0 msgr2=0x56177a8b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:02.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.202+0000 7f751c97fb80 1 --2- 172.21.15.131:0/793068041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b46a0 0x56177a8b4af0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009c00 tx=0x7f74e8009f10 comp rx=0 tx=0).stop 2024-05-10T05:42:02.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.202+0000 7f751c97fb80 1 -- 172.21.15.131:0/793068041 shutdown_connections 2024-05-10T05:42:02.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.202+0000 7f751c97fb80 1 --2- 172.21.15.131:0/793068041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8bc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.202+0000 7f751c97fb80 1 --2- 172.21.15.131:0/793068041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b46a0 0x56177a8b4af0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.202+0000 7f751c97fb80 1 --2- 172.21.15.131:0/793068041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b5570 0x56177a8b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.202+0000 7f751c97fb80 1 -- 172.21.15.131:0/793068041 >> 172.21.15.131:0/793068041 conn(0x56177a816a60 msgr2=0x56177a8aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:02.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.202+0000 7f751c97fb80 1 -- 172.21.15.131:0/793068041 shutdown_connections 2024-05-10T05:42:02.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.202+0000 7f751c97fb80 1 -- 172.21.15.131:0/793068041 wait complete. 2024-05-10T05:42:02.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.207+0000 7f751c97fb80 1 -- start start 2024-05-10T05:42:02.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.208+0000 7f751c97fb80 1 Processor -- start 2024-05-10T05:42:02.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.208+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b46a0 0x56177a8be6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.208+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8bec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.208+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.212+0000 7f751c97fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a770560 con 0x56177a8b46a0 2024-05-10T05:42:02.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.212+0000 7f751c97fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a770200 con 0x56177a8b5570 2024-05-10T05:42:02.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.212+0000 7f751c97fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a6576a0 con 0x56177a8b5ec0 2024-05-10T05:42:02.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.213+0000 7f751c97fb80 10 client.0 _ll_register_callbacks cb 0x56177a779b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:02.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.214+0000 7f74fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8bec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.214+0000 7f74fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8bec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:48046/0 (socket says 172.21.15.131:48046) 2024-05-10T05:42:02.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.214+0000 7f74fd400700 1 -- 172.21.15.131:0/4051836725 learned_addr learned my addr 172.21.15.131:0/4051836725 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:02.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.216+0000 7f74fe800700 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.216+0000 7f74fde00700 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b46a0 0x56177a8be6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.216+0000 7f74fd400700 1 -- 172.21.15.131:0/4051836725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 msgr2=0x56177a8c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:02.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.216+0000 7f74fd400700 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.216+0000 7f74fd400700 1 -- 172.21.15.131:0/4051836725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b46a0 msgr2=0x56177a8be6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:02.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.217+0000 7f74fd400700 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b46a0 0x56177a8be6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.217+0000 7f74fd400700 1 -- 172.21.15.131:0/4051836725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e8009860 con 0x56177a8b5570 2024-05-10T05:42:02.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.219+0000 7f74fde00700 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b46a0 0x56177a8be6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:02.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.219+0000 7f74fd400700 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8bec10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f74f0003830 tx=0x7f74f0005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=948452a2a48c750 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:02.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.219+0000 7f74e3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:02.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.219+0000 7f74e3400700 1 -- 172.21.15.131:0/4051836725 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f000bc50 con 0x56177a8b5570 2024-05-10T05:42:02.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.220+0000 7f74e3400700 1 -- 172.21.15.131:0/4051836725 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f74f000bdb0 con 0x56177a8b5570 2024-05-10T05:42:02.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.220+0000 7f751c97fb80 1 -- 172.21.15.131:0/4051836725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56177a72d270 con 0x56177a8b5570 2024-05-10T05:42:02.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.220+0000 7f74e3400700 1 -- 172.21.15.131:0/4051836725 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74f00038d0 con 0x56177a8b5570 2024-05-10T05:42:02.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.220+0000 7f751c97fb80 10 client.26936 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:02.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.220+0000 7f751c97fb80 1 -- 172.21.15.131:0/4051836725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56177a601b60 con 0x56177a8b5570 2024-05-10T05:42:02.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.221+0000 7f74e3400700 1 -- 172.21.15.131:0/4051836725 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56177a72d270 con 0x56177a8b5570 2024-05-10T05:42:02.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.222+0000 7f74e3400700 1 -- 172.21.15.131:0/4051836725 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f74f002a070 con 0x56177a8b5570 2024-05-10T05:42:02.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.224+0000 7f751c97fb80 10 client.26936 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:02.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.224+0000 7f751c97fb80 10 client.26936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:02.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.224+0000 7f751c97fb80 20 client.26936 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:02.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.224+0000 7f751c97fb80 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56177a8db3b0 0x56177a8dd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.224+0000 7f751c97fb80 4 client.26936 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:02.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.224+0000 7f751c97fb80 1 -- 172.21.15.131:0/4051836725 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56177a883780 con 0x56177a8db3b0 2024-05-10T05:42:02.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.225+0000 7f74fde00700 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56177a8db3b0 0x56177a8dd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.225+0000 7f74fde00700 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56177a8db3b0 0x56177a8dd800 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e744d21b116c4c2a server_cookie=4d35c9bb57c8987f in_seq=0 out_seq=0 2024-05-10T05:42:02.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.225+0000 7f74e3400700 10 client.26936 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:02.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.226+0000 7f74e3400700 1 -- 172.21.15.131:0/4051836725 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56177a883780 con 0x56177a8db3b0 2024-05-10T05:42:02.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.226+0000 7f74e3400700 10 client.26936 handle_command_reply: tid=0 2024-05-10T05:42:02.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.227+0000 7f751c97fb80 1 client.26936 shutdown 2024-05-10T05:42:02.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.239+0000 7f751c97fb80 1 -- 172.21.15.131:0/4051836725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 msgr2=0x56177a8bec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:02.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.239+0000 7f751c97fb80 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8bec10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f74f0003830 tx=0x7f74f0005770 comp rx=0 tx=0).stop 2024-05-10T05:42:02.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.246+0000 7f751c97fb80 1 -- 172.21.15.131:0/4051836725 shutdown_connections 2024-05-10T05:42:02.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.246+0000 7f751c97fb80 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56177a8db3b0 0x56177a8dd800 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.246+0000 7f751c97fb80 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b46a0 0x56177a8be6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.246+0000 7f751c97fb80 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.246+0000 7f751c97fb80 1 --2- 172.21.15.131:0/4051836725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8bec10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.246+0000 7f751c97fb80 1 -- 172.21.15.131:0/4051836725 >> 172.21.15.131:0/4051836725 conn(0x56177a816a60 msgr2=0x56177a8aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:02.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.246+0000 7f74e3400700 0 client.26936 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:02.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.247+0000 7f751c97fb80 1 -- 172.21.15.131:0/4051836725 shutdown_connections 2024-05-10T05:42:02.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.250+0000 7f751c97fb80 1 -- 172.21.15.131:0/4051836725 wait complete. 2024-05-10T05:42:02.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.252+0000 7f751c97fb80 20 client.26936 trim_cache size 0 max 16384 2024-05-10T05:42:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.306+0000 7f751c97fb80 1 Processor -- start 2024-05-10T05:42:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.306+0000 7f751c97fb80 1 -- start start 2024-05-10T05:42:02.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.307+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b46a0 0x56177a8afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.307+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.307+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.307+0000 7f751c97fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a8afad0 con 0x56177a8b46a0 2024-05-10T05:42:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.307+0000 7f751c97fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a8bf530 con 0x56177a8b5570 2024-05-10T05:42:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.307+0000 7f751c97fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a8bc640 con 0x56177a8b5ec0 2024-05-10T05:42:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.309+0000 7f74fe800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.310+0000 7f74fe800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45366/0 (socket says 172.21.15.131:45366) 2024-05-10T05:42:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.310+0000 7f74fe800700 1 -- 172.21.15.131:0/3739227334 learned_addr learned my addr 172.21.15.131:0/3739227334 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.310+0000 7f74fe800700 1 -- 172.21.15.131:0/3739227334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 msgr2=0x56177a8b92b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.310+0000 7f74fe800700 1 --2- 172.21.15.131:0/3739227334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8b92b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.310+0000 7f74fe800700 1 -- 172.21.15.131:0/3739227334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b46a0 msgr2=0x56177a8afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.310+0000 7f74fe800700 1 --2- 172.21.15.131:0/3739227334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b46a0 0x56177a8afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.310+0000 7f74fe800700 1 -- 172.21.15.131:0/3739227334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56177a79a560 con 0x56177a8b5ec0 2024-05-10T05:42:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.311+0000 7f74fe800700 1 --2- 172.21.15.131:0/3739227334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c57a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009f30 tx=0x7f74e8024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55f3394a4ecada8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.311+0000 7f74efa00700 1 -- 172.21.15.131:0/3739227334 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e802d050 con 0x56177a8b5ec0 2024-05-10T05:42:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.311+0000 7f74efa00700 1 -- 172.21.15.131:0/3739227334 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74e802e040 con 0x56177a8b5ec0 2024-05-10T05:42:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.311+0000 7f74efa00700 1 -- 172.21.15.131:0/3739227334 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e802d050 con 0x56177a8b5ec0 2024-05-10T05:42:02.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.313+0000 7f751c97fb80 1 -- 172.21.15.131:0/3739227334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 msgr2=0x56177a8c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:02.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.313+0000 7f751c97fb80 1 --2- 172.21.15.131:0/3739227334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c57a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009f30 tx=0x7f74e8024000 comp rx=0 tx=0).stop 2024-05-10T05:42:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.314+0000 7f751c97fb80 1 -- 172.21.15.131:0/3739227334 shutdown_connections 2024-05-10T05:42:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.314+0000 7f751c97fb80 1 --2- 172.21.15.131:0/3739227334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8b46a0 0x56177a8afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.314+0000 7f751c97fb80 1 --2- 172.21.15.131:0/3739227334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c57a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.314+0000 7f751c97fb80 1 --2- 172.21.15.131:0/3739227334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8b92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.314+0000 7f751c97fb80 1 -- 172.21.15.131:0/3739227334 >> 172.21.15.131:0/3739227334 conn(0x56177a816a60 msgr2=0x56177a8b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:02.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.316+0000 7f751c97fb80 1 -- 172.21.15.131:0/3739227334 shutdown_connections 2024-05-10T05:42:02.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.316+0000 7f751c97fb80 1 -- 172.21.15.131:0/3739227334 wait complete. 2024-05-10T05:42:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.319+0000 7f751c97fb80 1 -- start start 2024-05-10T05:42:02.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.326+0000 7f751c97fb80 1 Processor -- start 2024-05-10T05:42:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.328+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.328+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.328+0000 7f751c97fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8c4df0 0x56177a8c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.328+0000 7f751c97fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a8c5740 con 0x56177a8c4df0 2024-05-10T05:42:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.328+0000 7f751c97fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a8c58b0 con 0x56177a8b5570 2024-05-10T05:42:02.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.328+0000 7f751c97fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177a8c5a20 con 0x56177a8b5ec0 2024-05-10T05:42:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.328+0000 7f751c97fb80 10 client.0 _ll_register_callbacks cb 0x56177a779b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.329+0000 7f74fde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.329+0000 7f74fd400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.329+0000 7f74fd400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:45376/0 (socket says 172.21.15.131:45376) 2024-05-10T05:42:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.329+0000 7f74fd400700 1 -- 172.21.15.131:0/334763882 learned_addr learned my addr 172.21.15.131:0/334763882 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.329+0000 7f74fe800700 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8c4df0 0x56177a8c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.331+0000 7f74fde00700 1 -- 172.21.15.131:0/334763882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 msgr2=0x56177a8c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:02.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.331+0000 7f74fde00700 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.331+0000 7f74fde00700 1 -- 172.21.15.131:0/334763882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8c4df0 msgr2=0x56177a8c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.331+0000 7f74fde00700 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8c4df0 0x56177a8c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.331+0000 7f74fde00700 1 -- 172.21.15.131:0/334763882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56177a6553b0 con 0x56177a8b5570 2024-05-10T05:42:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.331+0000 7f74fe800700 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8c4df0 0x56177a8c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.332+0000 7f74fde00700 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8c39d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f74f80c9a20 tx=0x7f74f80960b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f378090d4d585247 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.333+0000 7f74e3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.333+0000 7f74e3400700 1 -- 172.21.15.131:0/334763882 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f80850b0 con 0x56177a8b5570 2024-05-10T05:42:02.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.333+0000 7f751c97fb80 1 -- 172.21.15.131:0/334763882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74e800eeb0 con 0x56177a8b5570 2024-05-10T05:42:02.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.333+0000 7f74e3400700 1 -- 172.21.15.131:0/334763882 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f74f809a370 con 0x56177a8b5570 2024-05-10T05:42:02.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.334+0000 7f74e3400700 1 -- 172.21.15.131:0/334763882 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74f809e1a0 con 0x56177a8b5570 2024-05-10T05:42:02.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.334+0000 7f751c97fb80 10 client.26939 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:02.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.334+0000 7f751c97fb80 1 -- 172.21.15.131:0/334763882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56177a659cd0 con 0x56177a8b5570 2024-05-10T05:42:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.336+0000 7f74e3400700 1 -- 172.21.15.131:0/334763882 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74f80b5020 con 0x56177a8b5570 2024-05-10T05:42:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.336+0000 7f74e3400700 1 -- 172.21.15.131:0/334763882 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f74f80a2360 con 0x56177a8b5570 2024-05-10T05:42:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.336+0000 7f751c97fb80 10 client.26939 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.336+0000 7f751c97fb80 10 client.26939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.336+0000 7f751c97fb80 20 client.26939 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.336+0000 7f751c97fb80 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56177a943b10 0x56177a945f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.337+0000 7f751c97fb80 4 client.26939 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:42:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.337+0000 7f74fd400700 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56177a943b10 0x56177a945f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.337+0000 7f751c97fb80 1 -- 172.21.15.131:0/334763882 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56177a7ccec0 con 0x56177a943b10 2024-05-10T05:42:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.338+0000 7f74fd400700 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56177a943b10 0x56177a945f30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d263888f3faa5047 server_cookie=96f0f614ef964771 in_seq=0 out_seq=0 2024-05-10T05:42:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.338+0000 7f74e3400700 10 client.26939 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.339+0000 7f74e3400700 1 -- 172.21.15.131:0/334763882 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56177a7ccec0 con 0x56177a943b10 2024-05-10T05:42:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.339+0000 7f74e3400700 10 client.26939 handle_command_reply: tid=0 2024-05-10T05:42:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.339+0000 7f751c97fb80 1 client.26939 shutdown 2024-05-10T05:42:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.339+0000 7f751c97fb80 1 -- 172.21.15.131:0/334763882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 msgr2=0x56177a8c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.339+0000 7f751c97fb80 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8c39d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f74f80c9a20 tx=0x7f74f80960b0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.340+0000 7f751c97fb80 1 -- 172.21.15.131:0/334763882 shutdown_connections 2024-05-10T05:42:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.340+0000 7f751c97fb80 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56177a943b10 0x56177a945f30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.340+0000 7f751c97fb80 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56177a8c4df0 0x56177a8c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.340+0000 7f751c97fb80 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56177a8b5570 0x56177a8c39d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.340+0000 7f751c97fb80 1 --2- 172.21.15.131:0/334763882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56177a8b5ec0 0x56177a8c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.340+0000 7f751c97fb80 1 -- 172.21.15.131:0/334763882 >> 172.21.15.131:0/334763882 conn(0x56177a816a60 msgr2=0x56177a8b92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.340+0000 7f74e3400700 0 client.26939 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.340+0000 7f751c97fb80 1 -- 172.21.15.131:0/334763882 shutdown_connections 2024-05-10T05:42:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.340+0000 7f751c97fb80 1 -- 172.21.15.131:0/334763882 wait complete. 2024-05-10T05:42:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.341+0000 7f751c97fb80 20 client.26939 trim_cache size 0 max 16384 2024-05-10T05:42:02.342 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:42:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 -- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f74e40a1240 msgr2=0x7f74e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f74e40a1240 0x7f74e40a3750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f74e8018000 tx=0x7f74e8010c30 comp rx=0 tx=0).stop 2024-05-10T05:42:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 -- 172.21.15.131:0/559121442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7500103e00 msgr2=0x7f750019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7500103e00 0x7f750019e5a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b9b0 tx=0x7f74f800bd70 comp rx=0 tx=0).stop 2024-05-10T05:42:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 -- 172.21.15.131:0/559121442 shutdown_connections 2024-05-10T05:42:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f74e40a1240 0x7f74e40a3750 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75001058c0 0x7f75001a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7500103e00 0x7f750019e5a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 --2- 172.21.15.131:0/559121442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7500104f20 0x7f750019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 -- 172.21.15.131:0/559121442 >> 172.21.15.131:0/559121442 conn(0x7f75000ff410 msgr2=0x7f7500077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:02.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 -- 172.21.15.131:0/559121442 shutdown_connections 2024-05-10T05:42:02.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:02.342+0000 7f7504a00700 1 -- 172.21.15.131:0/559121442 wait complete. 2024-05-10T05:42:02.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:02 smithi131 ceph-mon[37264]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 90 op/s 2024-05-10T05:42:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:02 smithi157 ceph-mon[78427]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 90 op/s 2024-05-10T05:42:02.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:02 smithi142 ceph-mon[77667]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 90 op/s 2024-05-10T05:42:03.402 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:42:03.402 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:42:04.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.085+0000 7fd486c00700 1 -- 172.21.15.131:0/4189946076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48010c190 msgr2=0x7fd48010e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:04.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.085+0000 7fd486c00700 1 --2- 172.21.15.131:0/4189946076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48010c190 0x7fd48010e670 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0098f0 tx=0x7fd46c009c00 comp rx=0 tx=0).stop 2024-05-10T05:42:04.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.085+0000 7fd486c00700 1 -- 172.21.15.131:0/4189946076 shutdown_connections 2024-05-10T05:42:04.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.085+0000 7fd486c00700 1 --2- 172.21.15.131:0/4189946076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd480108620 0x7fd48010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.085+0000 7fd486c00700 1 --2- 172.21.15.131:0/4189946076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd48010ed00 0x7fd480111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.085+0000 7fd486c00700 1 --2- 172.21.15.131:0/4189946076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48010c190 0x7fd48010e670 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0098f0 tx=0x7fd46c009c00 comp rx=0 tx=0).stop 2024-05-10T05:42:04.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.085+0000 7fd486c00700 1 -- 172.21.15.131:0/4189946076 >> 172.21.15.131:0/4189946076 conn(0x7fd48006ec70 msgr2=0x7fd480071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:04.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.087+0000 7fd486c00700 1 -- 172.21.15.131:0/4189946076 shutdown_connections 2024-05-10T05:42:04.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.087+0000 7fd486c00700 1 -- 172.21.15.131:0/4189946076 wait complete. 2024-05-10T05:42:04.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.089+0000 7fd486c00700 1 Processor -- start 2024-05-10T05:42:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.091+0000 7fd486c00700 1 -- start start 2024-05-10T05:42:04.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.092+0000 7fd486c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd480108620 0x7fd48019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.092+0000 7fd486c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48010ed00 0x7fd48019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.093+0000 7fd486c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4801a4b50 0x7fd4801a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.093+0000 7fd486c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801a75b0 con 0x7fd48010ed00 2024-05-10T05:42:04.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.093+0000 7fd486c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801a7720 con 0x7fd4801a4b50 2024-05-10T05:42:04.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.093+0000 7fd486c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801a78e0 con 0x7fd480108620 2024-05-10T05:42:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.093+0000 7fd485800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd480108620 0x7fd48019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.093+0000 7fd485800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd480108620 0x7fd48019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48110/0 (socket says 172.21.15.131:48110) 2024-05-10T05:42:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.093+0000 7fd485800700 1 -- 172.21.15.131:0/3058310144 learned_addr learned my addr 172.21.15.131:0/3058310144 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.093+0000 7fd486200700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4801a4b50 0x7fd4801a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.095+0000 7fd484e00700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48010ed00 0x7fd48019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.095+0000 7fd484e00700 1 -- 172.21.15.131:0/3058310144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd480108620 msgr2=0x7fd48019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.095+0000 7fd484e00700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd480108620 0x7fd48019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.096+0000 7fd484e00700 1 -- 172.21.15.131:0/3058310144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4801a4b50 msgr2=0x7fd4801a6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.096+0000 7fd484e00700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4801a4b50 0x7fd4801a6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.096+0000 7fd484e00700 1 -- 172.21.15.131:0/3058310144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c009520 con 0x7fd48010ed00 2024-05-10T05:42:04.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.097+0000 7fd484e00700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48010ed00 0x7fd48019fc90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd46c008000 tx=0x7fd46c009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:04.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.099+0000 7fd47e000700 1 -- 172.21.15.131:0/3058310144 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c00b990 con 0x7fd48010ed00 2024-05-10T05:42:04.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.099+0000 7fd47e000700 1 -- 172.21.15.131:0/3058310144 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd46c02d070 con 0x7fd48010ed00 2024-05-10T05:42:04.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.099+0000 7fd47e000700 1 -- 172.21.15.131:0/3058310144 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c003a80 con 0x7fd48010ed00 2024-05-10T05:42:04.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.099+0000 7fd486c00700 1 -- 172.21.15.131:0/3058310144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd480108aa0 con 0x7fd48010ed00 2024-05-10T05:42:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.108+0000 7fd486c00700 1 -- 172.21.15.131:0/3058310144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd480108e10 con 0x7fd48010ed00 2024-05-10T05:42:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.109+0000 7fd47e000700 1 -- 172.21.15.131:0/3058310144 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd46c02f070 con 0x7fd48010ed00 2024-05-10T05:42:04.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.127+0000 7fd47e000700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd4680a1270 0x7fd4680a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.127+0000 7fd47e000700 1 -- 172.21.15.131:0/3058310144 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd46c0e1170 con 0x7fd48010ed00 2024-05-10T05:42:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.130+0000 7fd485800700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd4680a1270 0x7fd4680a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.131+0000 7fd485800700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd4680a1270 0x7fd4680a3780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd478009610 tx=0x7fd478009040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:04.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.134+0000 7fd4a01f9b80 1 Processor -- start 2024-05-10T05:42:04.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.135+0000 7fd4a01f9b80 1 -- start start 2024-05-10T05:42:04.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.136+0000 7fd4a01f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.136+0000 7fd4a01f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 0x55bcdb292c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.137+0000 7fd4a01f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb293170 0x55bcdb299a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.137+0000 7fd4a01f9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb1b81d0 con 0x55bcdb293170 2024-05-10T05:42:04.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.137+0000 7fd4a01f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb1b5b90 con 0x55bcdb2915b0 2024-05-10T05:42:04.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.137+0000 7fd4a01f9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb1b1480 con 0x55bcdb292820 2024-05-10T05:42:04.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.137+0000 7fd485800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.137+0000 7fd485800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2919e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:43960/0 (socket says 172.21.15.131:43960) 2024-05-10T05:42:04.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.137+0000 7fd485800700 1 -- 172.21.15.131:0/1280320561 learned_addr learned my addr 172.21.15.131:0/1280320561 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:04.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.137+0000 7fd484e00700 1 --2- 172.21.15.131:0/1280320561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 0x55bcdb292c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.138+0000 7fd485800700 1 -- 172.21.15.131:0/1280320561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 msgr2=0x55bcdb292c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:04.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.138+0000 7fd485800700 1 --2- 172.21.15.131:0/1280320561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 0x55bcdb292c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.138+0000 7fd485800700 1 -- 172.21.15.131:0/1280320561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb293170 msgr2=0x55bcdb299a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:04.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.139+0000 7fd485800700 1 --2- 172.21.15.131:0/1280320561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb293170 0x55bcdb299a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.139+0000 7fd485800700 1 -- 172.21.15.131:0/1280320561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdb032430 con 0x55bcdb2915b0 2024-05-10T05:42:04.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.140+0000 7fd485800700 1 --2- 172.21.15.131:0/1280320561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2919e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd4780156d0 tx=0x7fd47800b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bffe374e15cbfa52 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:04.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.141+0000 7fd472a00700 1 -- 172.21.15.131:0/1280320561 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478016800 con 0x55bcdb2915b0 2024-05-10T05:42:04.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.141+0000 7fd472a00700 1 -- 172.21.15.131:0/1280320561 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd47800f040 con 0x55bcdb2915b0 2024-05-10T05:42:04.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.141+0000 7fd472a00700 1 -- 172.21.15.131:0/1280320561 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478021b50 con 0x55bcdb2915b0 2024-05-10T05:42:04.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.143+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/1280320561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 msgr2=0x55bcdb2919e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:04.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.143+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/1280320561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2919e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd4780156d0 tx=0x7fd47800b050 comp rx=0 tx=0).stop 2024-05-10T05:42:04.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.143+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/1280320561 shutdown_connections 2024-05-10T05:42:04.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.143+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/1280320561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb293170 0x55bcdb299a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.143+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/1280320561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2919e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.143+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/1280320561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 0x55bcdb292c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.144+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/1280320561 >> 172.21.15.131:0/1280320561 conn(0x55bcdb1f3ae0 msgr2=0x55bcdb28c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:04.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.144+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/1280320561 shutdown_connections 2024-05-10T05:42:04.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.152+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/1280320561 wait complete. 2024-05-10T05:42:04.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.153+0000 7fd4a01f9b80 1 -- start start 2024-05-10T05:42:04.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.153+0000 7fd4a01f9b80 1 Processor -- start 2024-05-10T05:42:04.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.159+0000 7fd4a01f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb29b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.162+0000 7fd485800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb29b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.164+0000 7fd485800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb29b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54386/0 (socket says 172.21.15.131:54386) 2024-05-10T05:42:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.164+0000 7fd485800700 1 -- 172.21.15.131:0/3164744001 learned_addr learned my addr 172.21.15.131:0/3164744001 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.164+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb292820 0x55bcdb29bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.167+0000 7fd484e00700 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb292820 0x55bcdb29bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.167+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 0x55bcdb2a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.167+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb14d5e0 con 0x55bcdb2915b0 2024-05-10T05:42:04.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.168+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb14d280 con 0x55bcdb292820 2024-05-10T05:42:04.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.168+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb034720 con 0x55bcdb293170 2024-05-10T05:42:04.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.168+0000 7fd486200700 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 0x55bcdb2a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.169+0000 7fd485800700 1 -- 172.21.15.131:0/3164744001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 msgr2=0x55bcdb2a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:04.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.169+0000 7fd485800700 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 0x55bcdb2a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.169+0000 7fd485800700 1 -- 172.21.15.131:0/3164744001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb292820 msgr2=0x55bcdb29bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:04.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.169+0000 7fd485800700 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb292820 0x55bcdb29bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.169+0000 7fd485800700 1 -- 172.21.15.131:0/3164744001 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd478015440 con 0x55bcdb2915b0 2024-05-10T05:42:04.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.170+0000 7fd486200700 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 0x55bcdb2a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:04.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.171+0000 7fd4a01f9b80 10 client.0 _ll_register_callbacks cb 0x55bcdb156bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:04.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.171+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcdb10a2f0 con 0x55bcdb2915b0 2024-05-10T05:42:04.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.173+0000 7fd485800700 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb29b9a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd478015f60 tx=0x7fd478015f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b05d9c44e4ec4c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:04.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.181+0000 7fd467400700 10 client.16665 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:42:04.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.181+0000 7fd467400700 1 -- 172.21.15.131:0/3164744001 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478016a10 con 0x55bcdb2915b0 2024-05-10T05:42:04.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.181+0000 7fd467400700 1 -- 172.21.15.131:0/3164744001 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd478016b70 con 0x55bcdb2915b0 2024-05-10T05:42:04.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.181+0000 7fd467400700 1 -- 172.21.15.131:0/3164744001 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd478016d50 con 0x55bcdb2915b0 2024-05-10T05:42:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.182+0000 7fd4a01f9b80 10 client.16665 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.182+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55bcdafdebe0 con 0x55bcdb2915b0 2024-05-10T05:42:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.182+0000 7fd467400700 1 -- 172.21.15.131:0/3164744001 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd47802b410 con 0x55bcdb2915b0 2024-05-10T05:42:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.183+0000 7fd4a01f9b80 10 client.16665 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.183+0000 7fd4a01f9b80 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.183+0000 7fd4a01f9b80 20 client.16665 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.183+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bcdb2a70c0 0x55bcdb2a95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.183+0000 7fd4a01f9b80 4 client.16665 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:04.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.183+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcdb260800 con 0x55bcdb2a70c0 2024-05-10T05:42:04.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.185+0000 7fd484e00700 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bcdb2a70c0 0x55bcdb2a95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.190+0000 7fd484e00700 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bcdb2a70c0 0x55bcdb2a95b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c79a3d84a013f308 server_cookie=2529e1aa2c95e28f in_seq=0 out_seq=0 2024-05-10T05:42:04.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.190+0000 7fd467400700 10 client.16665 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:04.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.190+0000 7fd467400700 1 -- 172.21.15.131:0/3164744001 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bcdb260800 con 0x55bcdb2a70c0 2024-05-10T05:42:04.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.190+0000 7fd467400700 10 client.16665 handle_command_reply: tid=0 2024-05-10T05:42:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.191+0000 7fd4a01f9b80 1 client.16665 shutdown 2024-05-10T05:42:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.194+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 msgr2=0x55bcdb29b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.194+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb29b9a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd478015f60 tx=0x7fd478015f90 comp rx=0 tx=0).stop 2024-05-10T05:42:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.194+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 shutdown_connections 2024-05-10T05:42:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.194+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bcdb2a70c0 0x55bcdb2a95b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.194+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb29b9a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.194+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 0x55bcdb2a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.194+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/3164744001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb292820 0x55bcdb29bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.194+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 >> 172.21.15.131:0/3164744001 conn(0x55bcdb1f3ae0 msgr2=0x55bcdb2900e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:04.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.194+0000 7fd467400700 0 client.16665 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:04.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.195+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 shutdown_connections 2024-05-10T05:42:04.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.196+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/3164744001 wait complete. 2024-05-10T05:42:04.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.196+0000 7fd4a01f9b80 20 client.16665 trim_cache size 0 max 16384 2024-05-10T05:42:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.244+0000 7fd4a01f9b80 1 Processor -- start 2024-05-10T05:42:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.245+0000 7fd4a01f9b80 1 -- start start 2024-05-10T05:42:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.245+0000 7fd4a01f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb296c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.246+0000 7fd4a01f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 0x55bcdb2a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.247+0000 7fd485800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb296c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.247+0000 7fd484e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 0x55bcdb2a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.247+0000 7fd484e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 0x55bcdb2a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48130/0 (socket says 172.21.15.131:48130) 2024-05-10T05:42:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.247+0000 7fd484e00700 1 -- 172.21.15.131:0/4213210548 learned_addr learned my addr 172.21.15.131:0/4213210548 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:04.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.247+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/4213210548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb293170 0x55bcdb29a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.248+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/4213210548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb288450 con 0x55bcdb2915b0 2024-05-10T05:42:04.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.248+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/4213210548 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb2a3b10 con 0x55bcdb293170 2024-05-10T05:42:04.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.248+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/4213210548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb292610 con 0x55bcdb292820 2024-05-10T05:42:04.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.248+0000 7fd486200700 1 --2- 172.21.15.131:0/4213210548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb293170 0x55bcdb29a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.249+0000 7fd486200700 1 -- 172.21.15.131:0/4213210548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 msgr2=0x55bcdb2a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:04.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.249+0000 7fd486200700 1 --2- 172.21.15.131:0/4213210548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 0x55bcdb2a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.249+0000 7fd486200700 1 -- 172.21.15.131:0/4213210548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 msgr2=0x55bcdb296c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:04.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.250+0000 7fd486200700 1 --2- 172.21.15.131:0/4213210548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb296c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.250+0000 7fd486200700 1 -- 172.21.15.131:0/4213210548 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdb1775e0 con 0x55bcdb293170 2024-05-10T05:42:04.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.250+0000 7fd485800700 1 --2- 172.21.15.131:0/4213210548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb296c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:04.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.251+0000 7fd486200700 1 --2- 172.21.15.131:0/4213210548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb293170 0x55bcdb29a400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd474011f70 tx=0x7fd4740098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f878784778632c2c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.252+0000 7fd473400700 1 -- 172.21.15.131:0/4213210548 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474015040 con 0x55bcdb293170 2024-05-10T05:42:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.252+0000 7fd473400700 1 -- 172.21.15.131:0/4213210548 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd47401b8b0 con 0x55bcdb293170 2024-05-10T05:42:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.252+0000 7fd473400700 1 -- 172.21.15.131:0/4213210548 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474015040 con 0x55bcdb293170 2024-05-10T05:42:04.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.255+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/4213210548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb293170 msgr2=0x55bcdb29a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:04.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.255+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/4213210548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb293170 0x55bcdb29a400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd474011f70 tx=0x7fd4740098a0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.255+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/4213210548 shutdown_connections 2024-05-10T05:42:04.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.255+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/4213210548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb2915b0 0x55bcdb296c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.255+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/4213210548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb293170 0x55bcdb29a400 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.255+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/4213210548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb292820 0x55bcdb2a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.255+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/4213210548 >> 172.21.15.131:0/4213210548 conn(0x55bcdb1f3ae0 msgr2=0x55bcdb299cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:04.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.255+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/4213210548 shutdown_connections 2024-05-10T05:42:04.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.255+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/4213210548 wait complete. 2024-05-10T05:42:04.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.256+0000 7fd4a01f9b80 1 -- start start 2024-05-10T05:42:04.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.257+0000 7fd4a01f9b80 1 Processor -- start 2024-05-10T05:42:04.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.258+0000 7fd4a01f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.258+0000 7fd4a01f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb292820 0x55bcdb2a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.258+0000 7fd4a01f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 0x55bcdb2a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.259+0000 7fd4a01f9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb2a1df0 con 0x55bcdb292820 2024-05-10T05:42:04.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.259+0000 7fd4a01f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb2a1f60 con 0x55bcdb2915b0 2024-05-10T05:42:04.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.259+0000 7fd4a01f9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb2a2120 con 0x55bcdb293170 2024-05-10T05:42:04.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.259+0000 7fd4a01f9b80 10 client.0 _ll_register_callbacks cb 0x55bcdb156bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:04.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.259+0000 7fd484e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb292820 0x55bcdb2a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.259+0000 7fd485800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.259+0000 7fd484e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb292820 0x55bcdb2a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54408/0 (socket says 172.21.15.131:54408) 2024-05-10T05:42:04.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.259+0000 7fd484e00700 1 -- 172.21.15.131:0/709177582 learned_addr learned my addr 172.21.15.131:0/709177582 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:04.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.259+0000 7fd486200700 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 0x55bcdb2a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.260+0000 7fd485800700 1 -- 172.21.15.131:0/709177582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 msgr2=0x55bcdb2a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:04.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.260+0000 7fd485800700 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 0x55bcdb2a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.260+0000 7fd485800700 1 -- 172.21.15.131:0/709177582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb292820 msgr2=0x55bcdb2a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:04.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.260+0000 7fd485800700 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb292820 0x55bcdb2a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.260+0000 7fd485800700 1 -- 172.21.15.131:0/709177582 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcdafdebe0 con 0x55bcdb2915b0 2024-05-10T05:42:04.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.260+0000 7fd484e00700 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb292820 0x55bcdb2a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:04.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.261+0000 7fd467400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:04.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.262+0000 7fd485800700 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2a2c10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55bcdb28dd30 tx=0x7fd478013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b41e274c7ed6478f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:04.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.263+0000 7fd467400700 1 -- 172.21.15.131:0/709177582 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478016c20 con 0x55bcdb2915b0 2024-05-10T05:42:04.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.263+0000 7fd467400700 1 -- 172.21.15.131:0/709177582 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd478042070 con 0x55bcdb2915b0 2024-05-10T05:42:04.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.263+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/709177582 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd47800f040 con 0x55bcdb2915b0 2024-05-10T05:42:04.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.267+0000 7fd467400700 1 -- 172.21.15.131:0/709177582 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd478009640 con 0x55bcdb2915b0 2024-05-10T05:42:04.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.267+0000 7fd4a01f9b80 10 client.26957 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:04.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.267+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/709177582 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fd474011c50 con 0x55bcdb2915b0 2024-05-10T05:42:04.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.267+0000 7fd467400700 1 -- 172.21.15.131:0/709177582 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd47804e030 con 0x55bcdb2915b0 2024-05-10T05:42:04.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.268+0000 7fd467400700 1 -- 172.21.15.131:0/709177582 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd478047940 con 0x55bcdb2915b0 2024-05-10T05:42:04.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.268+0000 7fd4a01f9b80 10 client.26957 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:04.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.268+0000 7fd4a01f9b80 10 client.26957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:04.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.268+0000 7fd4a01f9b80 20 client.26957 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:04.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.268+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bcdb2fea30 0x55bcdb300e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:04.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.268+0000 7fd4a01f9b80 4 client.26957 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:42:04.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.268+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/709177582 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bcdb1a9f40 con 0x55bcdb2fea30 2024-05-10T05:42:04.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.268+0000 7fd484e00700 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bcdb2fea30 0x55bcdb300e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:04.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.269+0000 7fd467400700 10 client.26957 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:04.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.270+0000 7fd484e00700 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bcdb2fea30 0x55bcdb300e50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707db02255784d1b server_cookie=7170eb4013077d1b in_seq=0 out_seq=0 2024-05-10T05:42:04.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.271+0000 7fd467400700 1 -- 172.21.15.131:0/709177582 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bcdb1a9f40 con 0x55bcdb2fea30 2024-05-10T05:42:04.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.271+0000 7fd467400700 10 client.26957 handle_command_reply: tid=0 2024-05-10T05:42:04.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.271+0000 7fd4a01f9b80 1 client.26957 shutdown 2024-05-10T05:42:04.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.272+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/709177582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 msgr2=0x55bcdb2a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:04.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.272+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2a2c10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55bcdb28dd30 tx=0x7fd478013000 comp rx=0 tx=0).stop 2024-05-10T05:42:04.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.275+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/709177582 shutdown_connections 2024-05-10T05:42:04.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.275+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55bcdb2fea30 0x55bcdb300e50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.275+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bcdb293170 0x55bcdb2a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.275+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdb2915b0 0x55bcdb2a2c10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.275+0000 7fd4a01f9b80 1 --2- 172.21.15.131:0/709177582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcdb292820 0x55bcdb2a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.275+0000 7fd467400700 0 client.26957 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:04.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.275+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/709177582 >> 172.21.15.131:0/709177582 conn(0x55bcdb1f3ae0 msgr2=0x55bcdb285360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:04.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.277+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/709177582 shutdown_connections 2024-05-10T05:42:04.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.279+0000 7fd4a01f9b80 1 -- 172.21.15.131:0/709177582 wait complete. 2024-05-10T05:42:04.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.279+0000 7fd4a01f9b80 20 client.26957 trim_cache size 0 max 16384 2024-05-10T05:42:04.280 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:42:04.280 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:42:04.280 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "db4c625b-d13d-455d-9ece-422622ec90e9", 2024-05-10T05:42:04.280 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:42:04.280 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:42:04.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.282+0000 7fd486c00700 1 -- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd4680a1270 msgr2=0x7fd4680a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:04.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.282+0000 7fd486c00700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd4680a1270 0x7fd4680a3780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd478009610 tx=0x7fd478009040 comp rx=0 tx=0).stop 2024-05-10T05:42:04.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.282+0000 7fd486c00700 1 -- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48010ed00 msgr2=0x7fd48019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:04.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.282+0000 7fd486c00700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48010ed00 0x7fd48019fc90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd46c008000 tx=0x7fd46c009490 comp rx=0 tx=0).stop 2024-05-10T05:42:04.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.283+0000 7fd486c00700 1 -- 172.21.15.131:0/3058310144 shutdown_connections 2024-05-10T05:42:04.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.283+0000 7fd486c00700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd4680a1270 0x7fd4680a3780 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.283+0000 7fd486c00700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4801a4b50 0x7fd4801a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.283+0000 7fd486c00700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd480108620 0x7fd48019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.283+0000 7fd486c00700 1 --2- 172.21.15.131:0/3058310144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48010ed00 0x7fd48019fc90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:04.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.283+0000 7fd486c00700 1 -- 172.21.15.131:0/3058310144 >> 172.21.15.131:0/3058310144 conn(0x7fd48006ec70 msgr2=0x7fd48010e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:04.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.285+0000 7fd486c00700 1 -- 172.21.15.131:0/3058310144 shutdown_connections 2024-05-10T05:42:04.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:04.286+0000 7fd486c00700 1 -- 172.21.15.131:0/3058310144 wait complete. 2024-05-10T05:42:04.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:42:04.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:04 smithi157 ceph-mon[78427]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:42:04.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:04 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:42:04.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:04 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:04.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:04 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:42:04.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:04 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:04.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:04 smithi131 ceph-mon[37264]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:42:04.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:04 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:42:04.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:04 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:04.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:04 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:42:04.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:04 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:04.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:04 smithi142 ceph-mon[77667]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:42:04.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:04 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:42:04.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:04 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:04.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:04 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:42:04.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:04 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:05.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.015+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2550481063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98109300 msgr2=0x7f4b98109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:05.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.015+0000 7f4b9ce00700 1 --2- 172.21.15.131:0/2550481063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98109300 0x7f4b98109710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009990 tx=0x7f4b84009ca0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.015+0000 7f4b94a00700 1 -- 172.21.15.131:0/2550481063 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b84003be0 con 0x7f4b98109300 2024-05-10T05:42:05.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.027+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2550481063 shutdown_connections 2024-05-10T05:42:05.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.027+0000 7f4b9ce00700 1 --2- 172.21.15.131:0/2550481063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98109c50 0x7f4b9810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.027+0000 7f4b9ce00700 1 --2- 172.21.15.131:0/2550481063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b98108630 0x7f4b98108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.027+0000 7f4b9ce00700 1 --2- 172.21.15.131:0/2550481063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98109300 0x7f4b98109710 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.027+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2550481063 >> 172.21.15.131:0/2550481063 conn(0x7f4b9806ea00 msgr2=0x7f4b98070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:05.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.027+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2550481063 shutdown_connections 2024-05-10T05:42:05.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.028+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2550481063 wait complete. 2024-05-10T05:42:05.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.030+0000 7f4b9ce00700 1 Processor -- start 2024-05-10T05:42:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.036+0000 7f4b9ce00700 1 -- start start 2024-05-10T05:42:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.038+0000 7f4b9ce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98108630 0x7f4b98115280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.038+0000 7f4b9ce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98109300 0x7f4b981157c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.038+0000 7f4b9ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b98109c50 0x7f4b981b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.038+0000 7f4b9ce00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b98115d90 con 0x7f4b98109300 2024-05-10T05:42:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.038+0000 7f4b9ce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b981b70d0 con 0x7f4b98109c50 2024-05-10T05:42:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.038+0000 7f4b9ce00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b981b7210 con 0x7f4b98108630 2024-05-10T05:42:05.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.040+0000 7f4b96800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b98109c50 0x7f4b981b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.040+0000 7f4b96800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b98109c50 0x7f4b981b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:44004/0 (socket says 172.21.15.131:44004) 2024-05-10T05:42:05.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.040+0000 7f4b96800700 1 -- 172.21.15.131:0/2252253579 learned_addr learned my addr 172.21.15.131:0/2252253579 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:05.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.041+0000 7f4b95400700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98109300 0x7f4b981157c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.041+0000 7f4b95e00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98108630 0x7f4b98115280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.041+0000 7f4b96800700 1 -- 172.21.15.131:0/2252253579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98108630 msgr2=0x7f4b98115280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:05.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.042+0000 7f4b96800700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98108630 0x7f4b98115280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.042+0000 7f4b96800700 1 -- 172.21.15.131:0/2252253579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98109300 msgr2=0x7f4b981157c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:05.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.042+0000 7f4b96800700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98109300 0x7f4b981157c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.042+0000 7f4b96800700 1 -- 172.21.15.131:0/2252253579 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840095f0 con 0x7f4b98109c50 2024-05-10T05:42:05.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.042+0000 7f4b95e00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98108630 0x7f4b98115280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:05.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.043+0000 7f4b96800700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b98109c50 0x7f4b981b6b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000b970 tx=0x7f4b9000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.047+0000 7f4b8ea00700 1 -- 172.21.15.131:0/2252253579 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000c9b0 con 0x7f4b98109c50 2024-05-10T05:42:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.047+0000 7f4b8ea00700 1 -- 172.21.15.131:0/2252253579 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b90014440 con 0x7f4b98109c50 2024-05-10T05:42:05.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.047+0000 7f4b8ea00700 1 -- 172.21.15.131:0/2252253579 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b900075d0 con 0x7f4b98109c50 2024-05-10T05:42:05.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.052+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2252253579 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b981b7470 con 0x7f4b98109c50 2024-05-10T05:42:05.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.057+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2252253579 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b981b7920 con 0x7f4b98109c50 2024-05-10T05:42:05.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.081+0000 7f4b8ea00700 1 -- 172.21.15.131:0/2252253579 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4b900145b0 con 0x7f4b98109c50 2024-05-10T05:42:05.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.084+0000 7f4b8ea00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4b7c0a1370 0x7f4b7c0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.085+0000 7f4b95e00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4b7c0a1370 0x7f4b7c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.086+0000 7f4b8ea00700 1 -- 172.21.15.131:0/2252253579 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4b90006e40 con 0x7f4b98109c50 2024-05-10T05:42:05.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.087+0000 7f4b95e00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4b7c0a1370 0x7f4b7c0a3880 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4b88005fd0 tx=0x7f4b88005e20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:05.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.097+0000 7f4bb4df6b80 1 Processor -- start 2024-05-10T05:42:05.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.098+0000 7f4bb4df6b80 1 -- start start 2024-05-10T05:42:05.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.104+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257a6a0 0x56448257aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.104+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257b570 0x56448257b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.104+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x5644825827c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.106+0000 7f4b95e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257a6a0 0x56448257aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.107+0000 7f4b95400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257b570 0x56448257b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.107+0000 7f4b95400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257b570 0x56448257b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:44018/0 (socket says 172.21.15.131:44018) 2024-05-10T05:42:05.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.108+0000 7f4b96800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x5644825827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.109+0000 7f4bb4df6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644824a1150 con 0x56448257bec0 2024-05-10T05:42:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.109+0000 7f4bb4df6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448249eb10 con 0x56448257b570 2024-05-10T05:42:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.109+0000 7f4bb4df6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448249a400 con 0x56448257a6a0 2024-05-10T05:42:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.109+0000 7f4b95400700 1 -- 172.21.15.131:0/4034816901 learned_addr learned my addr 172.21.15.131:0/4034816901 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.110+0000 7f4b95400700 1 -- 172.21.15.131:0/4034816901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257a6a0 msgr2=0x56448257aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.110+0000 7f4b95400700 1 --2- 172.21.15.131:0/4034816901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257a6a0 0x56448257aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.110+0000 7f4b95400700 1 -- 172.21.15.131:0/4034816901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 msgr2=0x5644825827c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.110+0000 7f4b95400700 1 --2- 172.21.15.131:0/4034816901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x5644825827c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.111+0000 7f4b95400700 1 -- 172.21.15.131:0/4034816901 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56448231b3b0 con 0x56448257b570 2024-05-10T05:42:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.112+0000 7f4b95400700 1 --2- 172.21.15.131:0/4034816901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257b570 0x56448257b980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4b840036a0 tx=0x7f4b84005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c3534012108cff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:05.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.113+0000 7f4b95e00700 1 --2- 172.21.15.131:0/4034816901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257a6a0 0x56448257aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:05.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.114+0000 7f4b83400700 1 -- 172.21.15.131:0/4034816901 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8401d040 con 0x56448257b570 2024-05-10T05:42:05.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.125+0000 7f4b83400700 1 -- 172.21.15.131:0/4034816901 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b84003d20 con 0x56448257b570 2024-05-10T05:42:05.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.126+0000 7f4b83400700 1 -- 172.21.15.131:0/4034816901 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8400f6c0 con 0x56448257b570 2024-05-10T05:42:05.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.126+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/4034816901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257b570 msgr2=0x56448257b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:05.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.126+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/4034816901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257b570 0x56448257b980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4b840036a0 tx=0x7f4b84005040 comp rx=0 tx=0).stop 2024-05-10T05:42:05.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.126+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/4034816901 shutdown_connections 2024-05-10T05:42:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.126+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/4034816901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x5644825827c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.126+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/4034816901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257a6a0 0x56448257aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.126+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/4034816901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257b570 0x56448257b980 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.126+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/4034816901 >> 172.21.15.131:0/4034816901 conn(0x5644824dca60 msgr2=0x564482574fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.127+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/4034816901 shutdown_connections 2024-05-10T05:42:05.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.127+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/4034816901 wait complete. 2024-05-10T05:42:05.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.128+0000 7f4bb4df6b80 1 -- start start 2024-05-10T05:42:05.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.129+0000 7f4bb4df6b80 1 Processor -- start 2024-05-10T05:42:05.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.133+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482584660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.133+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x564482584ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.134+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x56448258bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.134+0000 7f4b95400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x564482584ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.134+0000 7f4b95400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x564482584ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48152/0 (socket says 172.21.15.131:48152) 2024-05-10T05:42:05.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.134+0000 7f4bb4df6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564482436560 con 0x56448257bec0 2024-05-10T05:42:05.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.134+0000 7f4bb4df6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564482436200 con 0x56448257a6a0 2024-05-10T05:42:05.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.134+0000 7f4bb4df6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448231d6a0 con 0x56448257b570 2024-05-10T05:42:05.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.135+0000 7f4bb4df6b80 10 client.0 _ll_register_callbacks cb 0x56448243fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:05.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.135+0000 7f4b96800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x56448258bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.135+0000 7f4b96800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x56448258bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54436/0 (socket says 172.21.15.131:54436) 2024-05-10T05:42:05.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.135+0000 7f4b96800700 1 -- 172.21.15.131:0/2552999919 learned_addr learned my addr 172.21.15.131:0/2552999919 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:05.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.135+0000 7f4b95e00700 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482584660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.139+0000 7f4b95400700 1 -- 172.21.15.131:0/2552999919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 msgr2=0x564482584660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:05.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.139+0000 7f4b95400700 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482584660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.139+0000 7f4b95400700 1 -- 172.21.15.131:0/2552999919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 msgr2=0x56448258bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:05.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.139+0000 7f4b95400700 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x56448258bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.139+0000 7f4b95400700 1 -- 172.21.15.131:0/2552999919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9810efc0 con 0x56448257b570 2024-05-10T05:42:05.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.140+0000 7f4b95e00700 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482584660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:05.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.141+0000 7f4b77e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:42:05.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.141+0000 7f4b95400700 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x564482584ba0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4b840098e0 tx=0x7f4b840052e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a1b7596d95360 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:05.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.142+0000 7f4b77e00700 1 -- 172.21.15.131:0/2552999919 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8401d070 con 0x56448257b570 2024-05-10T05:42:05.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.142+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/2552999919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644823f3270 con 0x56448257b570 2024-05-10T05:42:05.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.143+0000 7f4b77e00700 1 -- 172.21.15.131:0/2552999919 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4b84003a60 con 0x56448257b570 2024-05-10T05:42:05.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.145+0000 7f4b77e00700 1 -- 172.21.15.131:0/2552999919 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b8401d070 con 0x56448257b570 2024-05-10T05:42:05.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.148+0000 7f4bb4df6b80 10 client.27007 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:05.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.148+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/2552999919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5644822c7b60 con 0x56448257b570 2024-05-10T05:42:05.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.149+0000 7f4b77e00700 1 -- 172.21.15.131:0/2552999919 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5644823f3270 con 0x56448257b570 2024-05-10T05:42:05.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.150+0000 7f4b77e00700 1 -- 172.21.15.131:0/2552999919 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4b8402b440 con 0x56448257b570 2024-05-10T05:42:05.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.151+0000 7f4bb4df6b80 10 client.27007 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:05.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.151+0000 7f4bb4df6b80 10 client.27007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.151+0000 7f4bb4df6b80 20 client.27007 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.151+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56448258fce0 0x564482592100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.151+0000 7f4bb4df6b80 4 client.27007 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.151+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/2552999919 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564482549780 con 0x56448258fce0 2024-05-10T05:42:05.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.155+0000 7f4b95e00700 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56448258fce0 0x564482592100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.156+0000 7f4b95e00700 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56448258fce0 0x564482592100 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17492634a9b283dc server_cookie=3909126fa10f622e in_seq=0 out_seq=0 2024-05-10T05:42:05.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.158+0000 7f4b77e00700 10 client.27007 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.160+0000 7f4b77e00700 1 -- 172.21.15.131:0/2552999919 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564482549780 con 0x56448258fce0 2024-05-10T05:42:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.160+0000 7f4b77e00700 10 client.27007 handle_command_reply: tid=0 2024-05-10T05:42:05.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.161+0000 7f4bb4df6b80 1 client.27007 shutdown 2024-05-10T05:42:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.165+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/2552999919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 msgr2=0x564482584ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.166+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x564482584ba0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4b840098e0 tx=0x7f4b840052e0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.170+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/2552999919 shutdown_connections 2024-05-10T05:42:05.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.170+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56448258fce0 0x564482592100 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.170+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x56448258bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.170+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482584660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.170+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/2552999919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x564482584ba0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.170+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/2552999919 >> 172.21.15.131:0/2552999919 conn(0x5644824dca60 msgr2=0x564482574fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:05.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.171+0000 7f4b77e00700 0 client.27007 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:05.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.171+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/2552999919 shutdown_connections 2024-05-10T05:42:05.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.171+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/2552999919 wait complete. 2024-05-10T05:42:05.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.172+0000 7f4bb4df6b80 20 client.27007 trim_cache size 0 max 16384 2024-05-10T05:42:05.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.219+0000 7f4bb4df6b80 1 Processor -- start 2024-05-10T05:42:05.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.220+0000 7f4bb4df6b80 1 -- start start 2024-05-10T05:42:05.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.220+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482589ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.220+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257b570 0x564482589ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.227+0000 7f4b95e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482589ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.227+0000 7f4b95e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482589ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:44028/0 (socket says 172.21.15.131:44028) 2024-05-10T05:42:05.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.227+0000 7f4b95400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257b570 0x564482589ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.228+0000 7f4b95400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257b570 0x564482589ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54438/0 (socket says 172.21.15.131:54438) 2024-05-10T05:42:05.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.228+0000 7f4b95400700 1 -- 172.21.15.131:0/199872249 learned_addr learned my addr 172.21.15.131:0/199872249 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.228+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/199872249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257bec0 0x56448257f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.228+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/199872249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x564482576770 con 0x56448257b570 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.228+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/199872249 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448256f7c0 con 0x56448257a6a0 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.228+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/199872249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564482582670 con 0x56448257bec0 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.228+0000 7f4b95e00700 1 -- 172.21.15.131:0/199872249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257bec0 msgr2=0x56448257f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.228+0000 7f4b95e00700 1 --2- 172.21.15.131:0/199872249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257bec0 0x56448257f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.228+0000 7f4b95e00700 1 -- 172.21.15.131:0/199872249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257b570 msgr2=0x564482589ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.228+0000 7f4b95e00700 1 --2- 172.21.15.131:0/199872249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257b570 0x564482589ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.229+0000 7f4b95e00700 1 -- 172.21.15.131:0/199872249 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564482460560 con 0x56448257a6a0 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.229+0000 7f4b95400700 1 --2- 172.21.15.131:0/199872249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257b570 0x564482589ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.229+0000 7f4b95e00700 1 --2- 172.21.15.131:0/199872249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482589ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5644825929a0 tx=0x7f4b88013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c009a85c73f51410 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.230+0000 7f4b83e00700 1 -- 172.21.15.131:0/199872249 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88022040 con 0x56448257a6a0 2024-05-10T05:42:05.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.230+0000 7f4b83e00700 1 -- 172.21.15.131:0/199872249 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b880242b0 con 0x56448257a6a0 2024-05-10T05:42:05.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.230+0000 7f4b83e00700 1 -- 172.21.15.131:0/199872249 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88005470 con 0x56448257a6a0 2024-05-10T05:42:05.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.231+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/199872249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 msgr2=0x564482589ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:05.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.231+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/199872249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482589ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5644825929a0 tx=0x7f4b88013000 comp rx=0 tx=0).stop 2024-05-10T05:42:05.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.231+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/199872249 shutdown_connections 2024-05-10T05:42:05.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.231+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/199872249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257bec0 0x56448257f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.231+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/199872249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56448257a6a0 0x564482589ab0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.231+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/199872249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257b570 0x564482589ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.231+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/199872249 >> 172.21.15.131:0/199872249 conn(0x5644824dca60 msgr2=0x564482584660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.234+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/199872249 shutdown_connections 2024-05-10T05:42:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.234+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/199872249 wait complete. 2024-05-10T05:42:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.235+0000 7f4bb4df6b80 1 -- start start 2024-05-10T05:42:05.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.236+0000 7f4bb4df6b80 1 Processor -- start 2024-05-10T05:42:05.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.238+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x5644825e0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.238+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x5644825e2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.238+0000 7f4bb4df6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644825e2bc0 0x5644825e4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.239+0000 7f4b95e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x5644825e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.239+0000 7f4b95e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x5644825e0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:48174/0 (socket says 172.21.15.131:48174) 2024-05-10T05:42:05.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.240+0000 7f4b95400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x5644825e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.241+0000 7f4b95400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x5644825e2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54454/0 (socket says 172.21.15.131:54454) 2024-05-10T05:42:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.244+0000 7f4b95e00700 1 -- 172.21.15.131:0/650779913 learned_addr learned my addr 172.21.15.131:0/650779913 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:05.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.244+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644825e5520 con 0x56448257bec0 2024-05-10T05:42:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.244+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644825e5690 con 0x5644825e2bc0 2024-05-10T05:42:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.244+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644825e5850 con 0x56448257b570 2024-05-10T05:42:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.244+0000 7f4bb4df6b80 10 client.0 _ll_register_callbacks cb 0x56448243fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:05.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.245+0000 7f4b96800700 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644825e2bc0 0x5644825e4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.247+0000 7f4b95e00700 1 -- 172.21.15.131:0/650779913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644825e2bc0 msgr2=0x5644825e4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:05.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.247+0000 7f4b95e00700 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644825e2bc0 0x5644825e4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.247+0000 7f4b95e00700 1 -- 172.21.15.131:0/650779913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 msgr2=0x5644825e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:05.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.247+0000 7f4b95e00700 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x5644825e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.247+0000 7f4b95e00700 1 -- 172.21.15.131:0/650779913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b88024b70 con 0x56448257b570 2024-05-10T05:42:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.251+0000 7f4b95e00700 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x5644825e0130 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4b88027040 tx=0x7f4b88024820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79408b4a93f127a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.251+0000 7f4b77e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:42:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.251+0000 7f4b77e00700 1 -- 172.21.15.131:0/650779913 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88004ce0 con 0x56448257b570 2024-05-10T05:42:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.251+0000 7f4b77e00700 1 -- 172.21.15.131:0/650779913 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f4b880051f0 con 0x56448257b570 2024-05-10T05:42:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.251+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b84003d20 con 0x56448257b570 2024-05-10T05:42:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.252+0000 7f4b77e00700 1 -- 172.21.15.131:0/650779913 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b880053d0 con 0x56448257b570 2024-05-10T05:42:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.255+0000 7f4bb4df6b80 10 client.27010 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.255+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56448231fcd0 con 0x56448257b570 2024-05-10T05:42:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.255+0000 7f4b77e00700 1 -- 172.21.15.131:0/650779913 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b84003d20 con 0x56448257b570 2024-05-10T05:42:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.255+0000 7f4b77e00700 1 -- 172.21.15.131:0/650779913 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4b88038b90 con 0x56448257b570 2024-05-10T05:42:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.259+0000 7f4bb4df6b80 10 client.27010 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.259+0000 7f4bb4df6b80 10 client.27010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.259+0000 7f4bb4df6b80 20 client.27010 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.259+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5644825e7070 0x5644825e9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.259+0000 7f4bb4df6b80 4 client.27010 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:42:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.259+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564482492ec0 con 0x5644825e7070 2024-05-10T05:42:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.260+0000 7f4b95400700 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5644825e7070 0x5644825e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.260+0000 7f4b95400700 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5644825e7070 0x5644825e9490 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd0b7022bfbf0ee server_cookie=2af43399e0ff67e0 in_seq=0 out_seq=0 2024-05-10T05:42:05.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.261+0000 7f4b77e00700 10 client.27010 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:05.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.261+0000 7f4b77e00700 1 -- 172.21.15.131:0/650779913 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564482492ec0 con 0x5644825e7070 2024-05-10T05:42:05.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.261+0000 7f4b77e00700 10 client.27010 handle_command_reply: tid=0 2024-05-10T05:42:05.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.261+0000 7f4bb4df6b80 1 client.27010 shutdown 2024-05-10T05:42:05.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.266+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 msgr2=0x5644825e0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:05.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.267+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x5644825e0130 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4b88027040 tx=0x7f4b88024820 comp rx=0 tx=0).stop 2024-05-10T05:42:05.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.269+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 shutdown_connections 2024-05-10T05:42:05.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.269+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5644825e7070 0x5644825e9490 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.269+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644825e2bc0 0x5644825e4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.269+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56448257b570 0x5644825e0130 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.269+0000 7f4bb4df6b80 1 --2- 172.21.15.131:0/650779913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56448257bec0 0x5644825e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.269+0000 7f4b77e00700 0 client.27010 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:05.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.269+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 >> 172.21.15.131:0/650779913 conn(0x5644824dca60 msgr2=0x5644825842c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:05.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.273+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 shutdown_connections 2024-05-10T05:42:05.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.273+0000 7f4bb4df6b80 1 -- 172.21.15.131:0/650779913 wait complete. 2024-05-10T05:42:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.275+0000 7f4bb4df6b80 20 client.27010 trim_cache size 0 max 16384 2024-05-10T05:42:05.277 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:42:05.278 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:42:05.278 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:42:05.278 INFO:teuthology.orchestra.run.smithi131.stdout: "db4c625b-d13d-455d-9ece-422622ec90e9": { 2024-05-10T05:42:05.278 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:42:05.278 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "db4c625b-d13d-455d-9ece-422622ec90e9", 2024-05-10T05:42:05.278 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:42:05.278 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:42:05.278 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:42:05.278 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:42:05.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.278+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2252253579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4b7c0a1370 msgr2=0x7f4b7c0a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:05.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.278+0000 7f4b9ce00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4b7c0a1370 0x7f4b7c0a3880 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4b88005fd0 tx=0x7f4b88005e20 comp rx=0 tx=0).stop 2024-05-10T05:42:05.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.278+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2252253579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b98109c50 msgr2=0x7f4b981b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.278+0000 7f4b9ce00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b98109c50 0x7f4b981b6b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000b970 tx=0x7f4b9000bd30 comp rx=0 tx=0).stop 2024-05-10T05:42:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.278+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2252253579 shutdown_connections 2024-05-10T05:42:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.278+0000 7f4b9ce00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f4b7c0a1370 0x7f4b7c0a3880 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.278+0000 7f4b9ce00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b98109c50 0x7f4b981b6b90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.278+0000 7f4b9ce00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b98108630 0x7f4b98115280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.278+0000 7f4b9ce00700 1 --2- 172.21.15.131:0/2252253579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b98109300 0x7f4b981157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.278+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2252253579 >> 172.21.15.131:0/2252253579 conn(0x7f4b9806ea00 msgr2=0x7f4b9810a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.281+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2252253579 shutdown_connections 2024-05-10T05:42:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:05.281+0000 7f4b9ce00700 1 -- 172.21.15.131:0/2252253579 wait complete. 2024-05-10T05:42:05.365 INFO:tasks.cephfs.filesystem:scrub status for tag:db4c625b-d13d-455d-9ece-422622ec90e9 - {'path': '/', 'tag': 'db4c625b-d13d-455d-9ece-422622ec90e9', 'options': 'recursive,force'} 2024-05-10T05:42:05.713 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:05 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:42:05.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:05 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:05.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:05 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:42:05.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:05 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:05.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:05 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:42:05.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:05 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:06.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:06 smithi131 ceph-mon[37264]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 283 MiB/s wr, 89 op/s 2024-05-10T05:42:06.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:06 smithi157 ceph-mon[78427]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 283 MiB/s wr, 89 op/s 2024-05-10T05:42:06.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:06 smithi142 ceph-mon[77667]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 283 MiB/s wr, 89 op/s 2024-05-10T05:42:08.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:08 smithi131 ceph-mon[37264]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 228 MiB/s wr, 73 op/s 2024-05-10T05:42:08.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:08 smithi157 ceph-mon[78427]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 228 MiB/s wr, 73 op/s 2024-05-10T05:42:08.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:08 smithi142 ceph-mon[77667]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 228 MiB/s wr, 73 op/s 2024-05-10T05:42:10.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:10 smithi157 ceph-mon[78427]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 252 MiB/s wr, 79 op/s 2024-05-10T05:42:10.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:10 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:42:10.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:10 smithi142 ceph-mon[77667]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 252 MiB/s wr, 79 op/s 2024-05-10T05:42:10.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:10 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:42:10.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:10 smithi131 ceph-mon[37264]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 252 MiB/s wr, 79 op/s 2024-05-10T05:42:10.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:10 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:42:11.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:11 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:42:11.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:11 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:42:11.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:11 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:42:12.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:12 smithi157 ceph-mon[78427]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 262 MiB/s wr, 84 op/s 2024-05-10T05:42:12.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:12 smithi142 ceph-mon[77667]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 262 MiB/s wr, 84 op/s 2024-05-10T05:42:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:12 smithi131 ceph-mon[37264]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 262 MiB/s wr, 84 op/s 2024-05-10T05:42:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:14 smithi142 ceph-mon[77667]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 211 MiB/s wr, 67 op/s 2024-05-10T05:42:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:14 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:42:14.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:14 smithi131 ceph-mon[37264]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 211 MiB/s wr, 67 op/s 2024-05-10T05:42:14.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:14 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:42:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:14 smithi157 ceph-mon[78427]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 211 MiB/s wr, 67 op/s 2024-05-10T05:42:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:14 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:42:16.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:16 smithi142 ceph-mon[77667]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 269 MiB/s wr, 84 op/s 2024-05-10T05:42:16.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:42:16.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:42:16.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:16 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:42:16.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:16 smithi131 ceph-mon[37264]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 269 MiB/s wr, 84 op/s 2024-05-10T05:42:16.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:42:16.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:42:16.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:16 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:42:17.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:16 smithi157 ceph-mon[78427]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 269 MiB/s wr, 84 op/s 2024-05-10T05:42:17.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:42:17.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:42:17.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:16 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:42:17.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:17 smithi131 ceph-mon[37264]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 227 MiB/s wr, 72 op/s 2024-05-10T05:42:18.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:17 smithi157 ceph-mon[78427]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 227 MiB/s wr, 72 op/s 2024-05-10T05:42:18.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:17 smithi142 ceph-mon[77667]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 227 MiB/s wr, 72 op/s 2024-05-10T05:42:20.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:20 smithi131 ceph-mon[37264]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 259 MiB/s wr, 80 op/s 2024-05-10T05:42:20.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:20 smithi157 ceph-mon[78427]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 259 MiB/s wr, 80 op/s 2024-05-10T05:42:20.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:20 smithi142 ceph-mon[77667]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 259 MiB/s wr, 80 op/s 2024-05-10T05:42:22.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:22 smithi131 ceph-mon[37264]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 269 MiB/s wr, 84 op/s 2024-05-10T05:42:22.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:22 smithi157 ceph-mon[78427]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 269 MiB/s wr, 84 op/s 2024-05-10T05:42:22.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:22 smithi142 ceph-mon[77667]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 269 MiB/s wr, 84 op/s 2024-05-10T05:42:24.718 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:24 smithi131 ceph-mon[37264]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 225 MiB/s wr, 69 op/s 2024-05-10T05:42:24.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:24 smithi157 ceph-mon[78427]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 225 MiB/s wr, 69 op/s 2024-05-10T05:42:24.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:24 smithi142 ceph-mon[77667]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 225 MiB/s wr, 69 op/s 2024-05-10T05:42:26.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:26 smithi131 ceph-mon[37264]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 292 MiB/s wr, 88 op/s 2024-05-10T05:42:26.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:26 smithi157 ceph-mon[78427]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 292 MiB/s wr, 88 op/s 2024-05-10T05:42:26.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:26 smithi142 ceph-mon[77667]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 292 MiB/s wr, 88 op/s 2024-05-10T05:42:28.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:28 smithi131 ceph-mon[37264]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 242 MiB/s wr, 75 op/s 2024-05-10T05:42:29.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:28 smithi157 ceph-mon[78427]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 242 MiB/s wr, 75 op/s 2024-05-10T05:42:29.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:28 smithi142 ceph-mon[77667]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 242 MiB/s wr, 75 op/s 2024-05-10T05:42:29.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:29 smithi131 ceph-mon[37264]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 82 op/s 2024-05-10T05:42:30.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:29 smithi157 ceph-mon[78427]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 82 op/s 2024-05-10T05:42:30.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:29 smithi142 ceph-mon[77667]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 273 MiB/s wr, 82 op/s 2024-05-10T05:42:32.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:32 smithi131 ceph-mon[37264]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 285 MiB/s wr, 87 op/s 2024-05-10T05:42:32.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:32 smithi157 ceph-mon[78427]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 285 MiB/s wr, 87 op/s 2024-05-10T05:42:32.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:32 smithi142 ceph-mon[77667]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 285 MiB/s wr, 87 op/s 2024-05-10T05:42:34.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:34 smithi131 ceph-mon[37264]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 240 MiB/s wr, 72 op/s 2024-05-10T05:42:34.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:34 smithi157 ceph-mon[78427]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 240 MiB/s wr, 72 op/s 2024-05-10T05:42:34.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:34 smithi142 ceph-mon[77667]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 240 MiB/s wr, 72 op/s 2024-05-10T05:42:35.366 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:42:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.214+0000 7f054d000700 1 -- 172.21.15.131:0/841686798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0548101d80 msgr2=0x7f0548108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.214+0000 7f054d000700 1 --2- 172.21.15.131:0/841686798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0548101d80 0x7f0548108630 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f054000d3d0 tx=0x7f054000d6e0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.215+0000 7f054d000700 1 -- 172.21.15.131:0/841686798 shutdown_connections 2024-05-10T05:42:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.215+0000 7f054d000700 1 --2- 172.21.15.131:0/841686798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0548101d80 0x7f0548108630 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.215+0000 7f054d000700 1 --2- 172.21.15.131:0/841686798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05481002c0 0x7f05481006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.215+0000 7f054d000700 1 --2- 172.21.15.131:0/841686798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05481013e0 0x7f05481017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.215+0000 7f054d000700 1 -- 172.21.15.131:0/841686798 >> 172.21.15.131:0/841686798 conn(0x7f05480fb8f0 msgr2=0x7f05480fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:36.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.229+0000 7f054d000700 1 -- 172.21.15.131:0/841686798 shutdown_connections 2024-05-10T05:42:36.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.229+0000 7f054d000700 1 -- 172.21.15.131:0/841686798 wait complete. 2024-05-10T05:42:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.231+0000 7f054d000700 1 Processor -- start 2024-05-10T05:42:36.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.234+0000 7f054d000700 1 -- start start 2024-05-10T05:42:36.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.234+0000 7f054d000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05481002c0 0x7f0548111100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.234+0000 7f054d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05481013e0 0x7f0548111640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.234+0000 7f054d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0548101d80 0x7f05481165c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.234+0000 7f054d000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548116bd0 con 0x7f05481002c0 2024-05-10T05:42:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.235+0000 7f054d000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548116d40 con 0x7f05481013e0 2024-05-10T05:42:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.235+0000 7f0545400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05481013e0 0x7f0548111640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.235+0000 7f0545400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05481013e0 0x7f0548111640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35682/0 (socket says 172.21.15.131:35682) 2024-05-10T05:42:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.235+0000 7f0545400700 1 -- 172.21.15.131:0/3032395611 learned_addr learned my addr 172.21.15.131:0/3032395611 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.235+0000 7f0545e00700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05481002c0 0x7f0548111100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.235+0000 7f0546800700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0548101d80 0x7f05481165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.236+0000 7f054d000700 1 -- 172.21.15.131:0/3032395611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548116f00 con 0x7f0548101d80 2024-05-10T05:42:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.237+0000 7f0546800700 1 -- 172.21.15.131:0/3032395611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05481013e0 msgr2=0x7f0548111640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:36.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.237+0000 7f0546800700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05481013e0 0x7f0548111640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.237+0000 7f0546800700 1 -- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05481002c0 msgr2=0x7f0548111100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:36.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.238+0000 7f0546800700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05481002c0 0x7f0548111100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.238+0000 7f0546800700 1 -- 172.21.15.131:0/3032395611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054000d050 con 0x7f0548101d80 2024-05-10T05:42:36.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.238+0000 7f0546800700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0548101d80 0x7f05481165c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f054000db20 tx=0x7f05400169a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:36.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.239+0000 7f053ea00700 1 -- 172.21.15.131:0/3032395611 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540010070 con 0x7f0548101d80 2024-05-10T05:42:36.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.239+0000 7f0545e00700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05481002c0 0x7f0548111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:36.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.239+0000 7f053ea00700 1 -- 172.21.15.131:0/3032395611 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0540011040 con 0x7f0548101d80 2024-05-10T05:42:36.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.239+0000 7f053ea00700 1 -- 172.21.15.131:0/3032395611 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400154a0 con 0x7f0548101d80 2024-05-10T05:42:36.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.239+0000 7f054d000700 1 -- 172.21.15.131:0/3032395611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0548117180 con 0x7f0548101d80 2024-05-10T05:42:36.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.239+0000 7f054d000700 1 -- 172.21.15.131:0/3032395611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0548117660 con 0x7f0548101d80 2024-05-10T05:42:36.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.254+0000 7f053ea00700 1 -- 172.21.15.131:0/3032395611 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0540015600 con 0x7f0548101d80 2024-05-10T05:42:36.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.257+0000 7f053ea00700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f052c0a12c0 0x7f052c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.257+0000 7f053ea00700 1 -- 172.21.15.131:0/3032395611 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f05400cdca0 con 0x7f0548101d80 2024-05-10T05:42:36.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.258+0000 7f0545e00700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f052c0a12c0 0x7f052c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.260+0000 7f0545e00700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f052c0a12c0 0x7f052c0a37d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f05481012e0 tx=0x7f0534009190 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:36.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.261+0000 7f0564f0eb80 1 Processor -- start 2024-05-10T05:42:36.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.261+0000 7f0564f0eb80 1 -- start start 2024-05-10T05:42:36.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.262+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 0x55d1cc52faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.263+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530570 0x55d1cc530980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.263+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc5377c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.264+0000 7f0546800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc5377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.264+0000 7f0546800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc5377c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37718/0 (socket says 172.21.15.131:37718) 2024-05-10T05:42:36.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.264+0000 7f0546800700 1 -- 172.21.15.131:0/783573041 learned_addr learned my addr 172.21.15.131:0/783573041 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:36.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.264+0000 7f0564f0eb80 1 -- 172.21.15.131:0/783573041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc456150 con 0x55d1cc530ec0 2024-05-10T05:42:36.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.264+0000 7f0564f0eb80 1 -- 172.21.15.131:0/783573041 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc453b10 con 0x55d1cc530570 2024-05-10T05:42:36.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.264+0000 7f0564f0eb80 1 -- 172.21.15.131:0/783573041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc44f400 con 0x55d1cc52f6a0 2024-05-10T05:42:36.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.265+0000 7f0546800700 1 -- 172.21.15.131:0/783573041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 msgr2=0x55d1cc52faf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:42:36.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.265+0000 7f0546800700 1 --2- 172.21.15.131:0/783573041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 0x55d1cc52faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.265+0000 7f0546800700 1 -- 172.21.15.131:0/783573041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530570 msgr2=0x55d1cc530980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:36.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.265+0000 7f0546800700 1 --2- 172.21.15.131:0/783573041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530570 0x55d1cc530980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.265+0000 7f0546800700 1 -- 172.21.15.131:0/783573041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1cc2d03b0 con 0x55d1cc530ec0 2024-05-10T05:42:36.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.266+0000 7f0546800700 1 --2- 172.21.15.131:0/783573041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc5377c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f054009b590 tx=0x7f0540016e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bec832e39802c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:36.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.267+0000 7f0533400700 1 -- 172.21.15.131:0/783573041 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054009b600 con 0x55d1cc530ec0 2024-05-10T05:42:36.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.267+0000 7f0533400700 1 -- 172.21.15.131:0/783573041 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f054009f430 con 0x55d1cc530ec0 2024-05-10T05:42:36.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.267+0000 7f0533400700 1 -- 172.21.15.131:0/783573041 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400a10f0 con 0x55d1cc530ec0 2024-05-10T05:42:36.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.269+0000 7f0564f0eb80 1 -- 172.21.15.131:0/783573041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 msgr2=0x55d1cc5377c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:36.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.269+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/783573041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc5377c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f054009b590 tx=0x7f0540016e20 comp rx=0 tx=0).stop 2024-05-10T05:42:36.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.270+0000 7f0564f0eb80 1 -- 172.21.15.131:0/783573041 shutdown_connections 2024-05-10T05:42:36.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.270+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/783573041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc5377c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.270+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/783573041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 0x55d1cc52faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.270+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/783573041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530570 0x55d1cc530980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.270+0000 7f0564f0eb80 1 -- 172.21.15.131:0/783573041 >> 172.21.15.131:0/783573041 conn(0x55d1cc491a60 msgr2=0x55d1cc529fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.270+0000 7f0564f0eb80 1 -- 172.21.15.131:0/783573041 shutdown_connections 2024-05-10T05:42:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.274+0000 7f0564f0eb80 1 -- 172.21.15.131:0/783573041 wait complete. 2024-05-10T05:42:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.274+0000 7f0564f0eb80 1 -- start start 2024-05-10T05:42:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.275+0000 7f0564f0eb80 1 Processor -- start 2024-05-10T05:42:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.278+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc52f6a0 0x55d1cc539570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.278+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530570 0x55d1cc539ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.278+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc530ec0 0x55d1cc540a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.278+0000 7f0564f0eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc3eb560 con 0x55d1cc530570 2024-05-10T05:42:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.278+0000 7f0564f0eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc3eb200 con 0x55d1cc52f6a0 2024-05-10T05:42:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.278+0000 7f0564f0eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc2d26a0 con 0x55d1cc530ec0 2024-05-10T05:42:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.278+0000 7f0564f0eb80 10 client.0 _ll_register_callbacks cb 0x55d1cc3f4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.279+0000 7f0545e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc52f6a0 0x55d1cc539570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.279+0000 7f0545e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc52f6a0 0x55d1cc539570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35688/0 (socket says 172.21.15.131:35688) 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.279+0000 7f0545e00700 1 -- 172.21.15.131:0/1137462981 learned_addr learned my addr 172.21.15.131:0/1137462981 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.279+0000 7f0546800700 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc530ec0 0x55d1cc540a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.279+0000 7f0545400700 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530570 0x55d1cc539ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.279+0000 7f0545e00700 1 -- 172.21.15.131:0/1137462981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc530ec0 msgr2=0x55d1cc540a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.279+0000 7f0545e00700 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc530ec0 0x55d1cc540a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.279+0000 7f0545e00700 1 -- 172.21.15.131:0/1137462981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530570 msgr2=0x55d1cc539ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.279+0000 7f0545e00700 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530570 0x55d1cc539ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.279+0000 7f0545e00700 1 -- 172.21.15.131:0/1137462981 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0548117660 con 0x55d1cc52f6a0 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.281+0000 7f0545e00700 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc52f6a0 0x55d1cc539570 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0534015a30 tx=0x7f05340047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=317d3c716425159 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.281+0000 7f0527e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:36.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.281+0000 7f0527e00700 1 -- 172.21.15.131:0/1137462981 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534013450 con 0x55d1cc52f6a0 2024-05-10T05:42:36.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.281+0000 7f0527e00700 1 -- 172.21.15.131:0/1137462981 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f05340135b0 con 0x55d1cc52f6a0 2024-05-10T05:42:36.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.282+0000 7f0564f0eb80 1 -- 172.21.15.131:0/1137462981 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1cc3a8270 con 0x55d1cc52f6a0 2024-05-10T05:42:36.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.284+0000 7f0527e00700 1 -- 172.21.15.131:0/1137462981 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0534013770 con 0x55d1cc52f6a0 2024-05-10T05:42:36.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.284+0000 7f0564f0eb80 10 client.26978 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:36.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.284+0000 7f0564f0eb80 1 -- 172.21.15.131:0/1137462981 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d1cc27cb60 con 0x55d1cc52f6a0 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.285+0000 7f0527e00700 1 -- 172.21.15.131:0/1137462981 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1cc3a8270 con 0x55d1cc52f6a0 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.285+0000 7f0527e00700 1 -- 172.21.15.131:0/1137462981 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f053401fa00 con 0x55d1cc52f6a0 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.285+0000 7f0564f0eb80 10 client.26978 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.285+0000 7f0564f0eb80 10 client.26978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.285+0000 7f0564f0eb80 20 client.26978 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.285+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d1cc545280 0x55d1cc5476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.285+0000 7f0564f0eb80 4 client.26978 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.285+0000 7f0564f0eb80 1 -- 172.21.15.131:0/1137462981 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1cc4fe780 con 0x55d1cc545280 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.285+0000 7f0545400700 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d1cc545280 0x55d1cc5476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.286+0000 7f0527e00700 10 client.26978 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:36.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.286+0000 7f0545400700 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d1cc545280 0x55d1cc5476d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45bf4469894d1123 server_cookie=9fedac3559d5978e in_seq=0 out_seq=0 2024-05-10T05:42:36.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.287+0000 7f0527e00700 1 -- 172.21.15.131:0/1137462981 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d1cc4fe780 con 0x55d1cc545280 2024-05-10T05:42:36.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.287+0000 7f0527e00700 10 client.26978 handle_command_reply: tid=0 2024-05-10T05:42:36.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.287+0000 7f0564f0eb80 1 client.26978 shutdown 2024-05-10T05:42:36.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.288+0000 7f0564f0eb80 1 -- 172.21.15.131:0/1137462981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc52f6a0 msgr2=0x55d1cc539570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:36.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.288+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc52f6a0 0x55d1cc539570 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0534015a30 tx=0x7f05340047e0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.289+0000 7f0564f0eb80 1 -- 172.21.15.131:0/1137462981 shutdown_connections 2024-05-10T05:42:36.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.289+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d1cc545280 0x55d1cc5476d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.289+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc530ec0 0x55d1cc540a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.289+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc52f6a0 0x55d1cc539570 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.289+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/1137462981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530570 0x55d1cc539ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.289+0000 7f0564f0eb80 1 -- 172.21.15.131:0/1137462981 >> 172.21.15.131:0/1137462981 conn(0x55d1cc491a60 msgr2=0x55d1cc529fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:36.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.289+0000 7f0527e00700 0 client.26978 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:36.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.289+0000 7f0564f0eb80 1 -- 172.21.15.131:0/1137462981 shutdown_connections 2024-05-10T05:42:36.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.289+0000 7f0564f0eb80 1 -- 172.21.15.131:0/1137462981 wait complete. 2024-05-10T05:42:36.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.290+0000 7f0564f0eb80 20 client.26978 trim_cache size 0 max 16384 2024-05-10T05:42:36.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.314+0000 7f0564f0eb80 1 Processor -- start 2024-05-10T05:42:36.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.315+0000 7f0564f0eb80 1 -- start start 2024-05-10T05:42:36.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.316+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc52f6a0 0x55d1cc5479a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.317+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc530570 0x55d1cc53e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.317+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530ec0 0x55d1cc53eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.318+0000 7f0564f0eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc542720 con 0x55d1cc52f6a0 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.318+0000 7f0564f0eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc541440 con 0x55d1cc530ec0 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.318+0000 7f0564f0eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc52ad20 con 0x55d1cc530570 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.318+0000 7f0546800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530ec0 0x55d1cc53eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.318+0000 7f0545400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc530570 0x55d1cc53e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.319+0000 7f0546800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530ec0 0x55d1cc53eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35704/0 (socket says 172.21.15.131:35704) 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.319+0000 7f0546800700 1 -- 172.21.15.131:0/3050626734 learned_addr learned my addr 172.21.15.131:0/3050626734 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.319+0000 7f0545e00700 1 --2- 172.21.15.131:0/3050626734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc52f6a0 0x55d1cc5479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.319+0000 7f0546800700 1 -- 172.21.15.131:0/3050626734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc530570 msgr2=0x55d1cc53e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.319+0000 7f0546800700 1 --2- 172.21.15.131:0/3050626734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc530570 0x55d1cc53e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.319+0000 7f0546800700 1 -- 172.21.15.131:0/3050626734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc52f6a0 msgr2=0x55d1cc5479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.319+0000 7f0546800700 1 --2- 172.21.15.131:0/3050626734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc52f6a0 0x55d1cc5479a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.319+0000 7f0546800700 1 -- 172.21.15.131:0/3050626734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1cc415560 con 0x55d1cc530ec0 2024-05-10T05:42:36.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.320+0000 7f0546800700 1 --2- 172.21.15.131:0/3050626734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530ec0 0x55d1cc53eed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0540015400 tx=0x7f0540086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4655b02eadee07af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:36.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.321+0000 7f0533e00700 1 -- 172.21.15.131:0/3050626734 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054009b600 con 0x55d1cc530ec0 2024-05-10T05:42:36.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.321+0000 7f0533e00700 1 -- 172.21.15.131:0/3050626734 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05400a3510 con 0x55d1cc530ec0 2024-05-10T05:42:36.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.321+0000 7f0533e00700 1 -- 172.21.15.131:0/3050626734 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540086310 con 0x55d1cc530ec0 2024-05-10T05:42:36.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.325+0000 7f0564f0eb80 1 -- 172.21.15.131:0/3050626734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530ec0 msgr2=0x55d1cc53eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:36.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.325+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/3050626734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530ec0 0x55d1cc53eed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0540015400 tx=0x7f0540086040 comp rx=0 tx=0).stop 2024-05-10T05:42:36.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.325+0000 7f0564f0eb80 1 -- 172.21.15.131:0/3050626734 shutdown_connections 2024-05-10T05:42:36.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.325+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/3050626734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc52f6a0 0x55d1cc5479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.325+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/3050626734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530ec0 0x55d1cc53eed0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.325+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/3050626734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc530570 0x55d1cc53e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.325+0000 7f0564f0eb80 1 -- 172.21.15.131:0/3050626734 >> 172.21.15.131:0/3050626734 conn(0x55d1cc491a60 msgr2=0x55d1cc547290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:36.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.326+0000 7f0564f0eb80 1 -- 172.21.15.131:0/3050626734 shutdown_connections 2024-05-10T05:42:36.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.329+0000 7f0564f0eb80 1 -- 172.21.15.131:0/3050626734 wait complete. 2024-05-10T05:42:36.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.330+0000 7f0564f0eb80 1 -- start start 2024-05-10T05:42:36.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.330+0000 7f0564f0eb80 1 Processor -- start 2024-05-10T05:42:36.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.333+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 0x55d1cc53e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.333+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530570 0x55d1cc53e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.333+0000 7f0564f0eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc599ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.334+0000 7f0545400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530570 0x55d1cc53e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.334+0000 7f0545e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 0x55d1cc53e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.335+0000 7f0545e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 0x55d1cc53e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34214/0 (socket says 172.21.15.131:34214) 2024-05-10T05:42:36.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.335+0000 7f0546800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc599ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.335+0000 7f0545400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530570 0x55d1cc53e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35712/0 (socket says 172.21.15.131:35712) 2024-05-10T05:42:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.335+0000 7f0546800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc599ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37748/0 (socket says 172.21.15.131:37748) 2024-05-10T05:42:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.335+0000 7f0545e00700 1 -- 172.21.15.131:0/4191060604 learned_addr learned my addr 172.21.15.131:0/4191060604 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.335+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc599ff0 con 0x55d1cc530ec0 2024-05-10T05:42:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.335+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc59a130 con 0x55d1cc530570 2024-05-10T05:42:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.336+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cc59a2c0 con 0x55d1cc52f6a0 2024-05-10T05:42:36.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.336+0000 7f0564f0eb80 10 client.0 _ll_register_callbacks cb 0x55d1cc3f4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.337+0000 7f0545e00700 1 -- 172.21.15.131:0/4191060604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530570 msgr2=0x55d1cc53e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.337+0000 7f0545e00700 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530570 0x55d1cc53e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.337+0000 7f0545e00700 1 -- 172.21.15.131:0/4191060604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 msgr2=0x55d1cc599ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.337+0000 7f0545e00700 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc599ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.337+0000 7f0545e00700 1 -- 172.21.15.131:0/4191060604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1cc2d03b0 con 0x55d1cc52f6a0 2024-05-10T05:42:36.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.338+0000 7f0545e00700 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 0x55d1cc53e390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f053401fde0 tx=0x7f05340145c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be4fb7914dfd54f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:36.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.339+0000 7f0527e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:42:36.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.340+0000 7f0527e00700 1 -- 172.21.15.131:0/4191060604 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053401f3f0 con 0x55d1cc52f6a0 2024-05-10T05:42:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.340+0000 7f0527e00700 1 -- 172.21.15.131:0/4191060604 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f053401f550 con 0x55d1cc52f6a0 2024-05-10T05:42:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.340+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f054009f430 con 0x55d1cc52f6a0 2024-05-10T05:42:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.341+0000 7f0527e00700 1 -- 172.21.15.131:0/4191060604 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053401f710 con 0x55d1cc52f6a0 2024-05-10T05:42:36.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.341+0000 7f0564f0eb80 10 client.27022 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:36.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.341+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d1cc2d4cd0 con 0x55d1cc52f6a0 2024-05-10T05:42:36.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.341+0000 7f0527e00700 1 -- 172.21.15.131:0/4191060604 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f054009f430 con 0x55d1cc52f6a0 2024-05-10T05:42:36.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.342+0000 7f0527e00700 1 -- 172.21.15.131:0/4191060604 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0534013c40 con 0x55d1cc52f6a0 2024-05-10T05:42:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.342+0000 7f0564f0eb80 10 client.27022 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.342+0000 7f0564f0eb80 10 client.27022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.342+0000 7f0564f0eb80 20 client.27022 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.342+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d1cc59bbf0 0x55d1cc59e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.342+0000 7f0564f0eb80 4 client.27022 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:42:36.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.342+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d1cc447ec0 con 0x55d1cc59bbf0 2024-05-10T05:42:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.344+0000 7f0545400700 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d1cc59bbf0 0x55d1cc59e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.345+0000 7f0545400700 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d1cc59bbf0 0x55d1cc59e010 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97b921af97f6bf7 server_cookie=35c4073dda5f8c5f in_seq=0 out_seq=0 2024-05-10T05:42:36.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.346+0000 7f0527e00700 10 client.27022 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.346+0000 7f0527e00700 1 -- 172.21.15.131:0/4191060604 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d1cc447ec0 con 0x55d1cc59bbf0 2024-05-10T05:42:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.346+0000 7f0527e00700 10 client.27022 handle_command_reply: tid=0 2024-05-10T05:42:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.346+0000 7f0564f0eb80 1 client.27022 shutdown 2024-05-10T05:42:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.347+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 msgr2=0x55d1cc53e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.347+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 0x55d1cc53e390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f053401fde0 tx=0x7f05340145c0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.347+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 shutdown_connections 2024-05-10T05:42:36.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.347+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d1cc59bbf0 0x55d1cc59e010 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.347+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1cc530ec0 0x55d1cc599ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.347+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1cc52f6a0 0x55d1cc53e390 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.347+0000 7f0564f0eb80 1 --2- 172.21.15.131:0/4191060604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1cc530570 0x55d1cc53e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.347+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 >> 172.21.15.131:0/4191060604 conn(0x55d1cc491a60 msgr2=0x55d1cc537a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:36.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.347+0000 7f0527e00700 0 client.27022 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:36.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.348+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 shutdown_connections 2024-05-10T05:42:36.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.348+0000 7f0564f0eb80 1 -- 172.21.15.131:0/4191060604 wait complete. 2024-05-10T05:42:36.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.348+0000 7f0564f0eb80 20 client.27022 trim_cache size 0 max 16384 2024-05-10T05:42:36.350 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:42:36.350 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:42:36.350 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:42:36.350 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:42:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.351+0000 7f054d000700 1 -- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f052c0a12c0 msgr2=0x7f052c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.351+0000 7f054d000700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f052c0a12c0 0x7f052c0a37d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f05481012e0 tx=0x7f0534009190 comp rx=0 tx=0).stop 2024-05-10T05:42:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.351+0000 7f054d000700 1 -- 172.21.15.131:0/3032395611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0548101d80 msgr2=0x7f05481165c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.351+0000 7f054d000700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0548101d80 0x7f05481165c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f054000db20 tx=0x7f05400169a0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.351+0000 7f054d000700 1 -- 172.21.15.131:0/3032395611 shutdown_connections 2024-05-10T05:42:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.351+0000 7f054d000700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f052c0a12c0 0x7f052c0a37d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.351+0000 7f054d000700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05481002c0 0x7f0548111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.351+0000 7f054d000700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0548101d80 0x7f05481165c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.351+0000 7f054d000700 1 --2- 172.21.15.131:0/3032395611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05481013e0 0x7f0548111640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.351+0000 7f054d000700 1 -- 172.21.15.131:0/3032395611 >> 172.21.15.131:0/3032395611 conn(0x7f05480fb8f0 msgr2=0x7f054810d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:36.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.352+0000 7f054d000700 1 -- 172.21.15.131:0/3032395611 shutdown_connections 2024-05-10T05:42:36.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:36.352+0000 7f054d000700 1 -- 172.21.15.131:0/3032395611 wait complete. 2024-05-10T05:42:36.453 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:42:36.453 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:42:36.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:36 smithi131 ceph-mon[37264]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 90 op/s 2024-05-10T05:42:36.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:36 smithi157 ceph-mon[78427]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 90 op/s 2024-05-10T05:42:36.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:36 smithi142 ceph-mon[77667]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 90 op/s 2024-05-10T05:42:37.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.096+0000 7fb779c00700 1 -- 172.21.15.131:0/34291363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774109c50 msgr2=0x7fb77410e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:37.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.096+0000 7fb779c00700 1 --2- 172.21.15.131:0/34291363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774109c50 0x7fb77410e560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00b3e0 tx=0x7fb76c00b6f0 comp rx=0 tx=0).stop 2024-05-10T05:42:37.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.099+0000 7fb779c00700 1 -- 172.21.15.131:0/34291363 shutdown_connections 2024-05-10T05:42:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.099+0000 7fb779c00700 1 --2- 172.21.15.131:0/34291363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774109c50 0x7fb77410e560 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.099+0000 7fb779c00700 1 --2- 172.21.15.131:0/34291363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108630 0x7fb774108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.099+0000 7fb779c00700 1 --2- 172.21.15.131:0/34291363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb774109300 0x7fb774109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.099+0000 7fb779c00700 1 -- 172.21.15.131:0/34291363 >> 172.21.15.131:0/34291363 conn(0x7fb77406ea00 msgr2=0x7fb774070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.100+0000 7fb779c00700 1 -- 172.21.15.131:0/34291363 shutdown_connections 2024-05-10T05:42:37.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.101+0000 7fb779c00700 1 -- 172.21.15.131:0/34291363 wait complete. 2024-05-10T05:42:37.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.102+0000 7fb779c00700 1 Processor -- start 2024-05-10T05:42:37.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.103+0000 7fb779c00700 1 -- start start 2024-05-10T05:42:37.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.103+0000 7fb779c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108630 0x7fb7741a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:37.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.103+0000 7fb779c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb774109300 0x7fb7741a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:37.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.103+0000 7fb779c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774109c50 0x7fb7741ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:37.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.104+0000 7fb773400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774109c50 0x7fb7741ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:37.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.104+0000 7fb772a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108630 0x7fb7741a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:37.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.104+0000 7fb773400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774109c50 0x7fb7741ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37768/0 (socket says 172.21.15.131:37768) 2024-05-10T05:42:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.104+0000 7fb773400700 1 -- 172.21.15.131:0/3409503840 learned_addr learned my addr 172.21.15.131:0/3409503840 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.104+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7741ae950 con 0x7fb774109c50 2024-05-10T05:42:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.104+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7741aeac0 con 0x7fb774108630 2024-05-10T05:42:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.104+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7741aec80 con 0x7fb774109300 2024-05-10T05:42:37.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.105+0000 7fb772000700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb774109300 0x7fb7741a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:37.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.106+0000 7fb773400700 1 -- 172.21.15.131:0/3409503840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb774109300 msgr2=0x7fb7741a73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:37.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.106+0000 7fb773400700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb774109300 0x7fb7741a73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.106+0000 7fb773400700 1 -- 172.21.15.131:0/3409503840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108630 msgr2=0x7fb7741a6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.106+0000 7fb773400700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108630 0x7fb7741a6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.107+0000 7fb773400700 1 -- 172.21.15.131:0/3409503840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb760009520 con 0x7fb774109c50 2024-05-10T05:42:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.107+0000 7fb773400700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774109c50 0x7fb7741ae340 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009cd0 tx=0x7fb76c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.108+0000 7fb76b400700 1 -- 172.21.15.131:0/3409503840 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c00e040 con 0x7fb774109c50 2024-05-10T05:42:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.109+0000 7fb76b400700 1 -- 172.21.15.131:0/3409503840 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb76c0057b0 con 0x7fb774109c50 2024-05-10T05:42:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.109+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76c009d00 con 0x7fb774109c50 2024-05-10T05:42:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.109+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7741af1a0 con 0x7fb774109c50 2024-05-10T05:42:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.110+0000 7fb76b400700 1 -- 172.21.15.131:0/3409503840 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c01ab80 con 0x7fb774109c50 2024-05-10T05:42:37.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.116+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb754005320 con 0x7fb774109c50 2024-05-10T05:42:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.121+0000 7fb76b400700 1 -- 172.21.15.131:0/3409503840 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb76c005920 con 0x7fb774109c50 2024-05-10T05:42:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.127+0000 7fb76b400700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb7580a12f0 0x7fb7580a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.127+0000 7fb772a00700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb7580a12f0 0x7fb7580a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.128+0000 7fb76b400700 1 -- 172.21.15.131:0/3409503840 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fb76c0cddb0 con 0x7fb774109c50 2024-05-10T05:42:37.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.133+0000 7fb772a00700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb7580a12f0 0x7fb7580a3800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb774108320 tx=0x7fb760009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:37.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.133+0000 7fb76b400700 1 -- 172.21.15.131:0/3409503840 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb76c09b000 con 0x7fb774109c50 2024-05-10T05:42:37.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.765+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb754005b50 con 0x7fb774109c50 2024-05-10T05:42:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.769+0000 7fb76b400700 1 -- 172.21.15.131:0/3409503840 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fb76c09ab90 con 0x7fb774109c50 2024-05-10T05:42:37.771 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:42:37.771 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:42:37.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.783+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb7580a12f0 msgr2=0x7fb7580a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:37.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.783+0000 7fb779c00700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb7580a12f0 0x7fb7580a3800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb774108320 tx=0x7fb760009260 comp rx=0 tx=0).stop 2024-05-10T05:42:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.783+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774109c50 msgr2=0x7fb7741ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.783+0000 7fb779c00700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774109c50 0x7fb7741ae340 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009cd0 tx=0x7fb76c003c30 comp rx=0 tx=0).stop 2024-05-10T05:42:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.784+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 shutdown_connections 2024-05-10T05:42:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.784+0000 7fb779c00700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fb7580a12f0 0x7fb7580a3800 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.784+0000 7fb779c00700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb774109c50 0x7fb7741ae340 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.784+0000 7fb779c00700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108630 0x7fb7741a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.784+0000 7fb779c00700 1 --2- 172.21.15.131:0/3409503840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb774109300 0x7fb7741a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.784+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 >> 172.21.15.131:0/3409503840 conn(0x7fb77406ea00 msgr2=0x7fb77406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.784+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 shutdown_connections 2024-05-10T05:42:37.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:37.785+0000 7fb779c00700 1 -- 172.21.15.131:0/3409503840 wait complete. 2024-05-10T05:42:37.792 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:42:37.851 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:42:38.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.569+0000 7f6359200700 1 -- 172.21.15.131:0/900925220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6354101be0 msgr2=0x7f6354101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:38.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.569+0000 7f6359200700 1 --2- 172.21.15.131:0/900925220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6354101be0 0x7f6354101ff0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6344009920 tx=0x7f6344009c30 comp rx=0 tx=0).stop 2024-05-10T05:42:38.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.570+0000 7f6359200700 1 -- 172.21.15.131:0/900925220 shutdown_connections 2024-05-10T05:42:38.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.570+0000 7f6359200700 1 --2- 172.21.15.131:0/900925220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63541036a0 0x7f6354109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.570+0000 7f6359200700 1 --2- 172.21.15.131:0/900925220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6354101be0 0x7f6354101ff0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.570+0000 7f6359200700 1 --2- 172.21.15.131:0/900925220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6354102d00 0x7f6354103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.570+0000 7f6359200700 1 -- 172.21.15.131:0/900925220 >> 172.21.15.131:0/900925220 conn(0x7f63540fd1d0 msgr2=0x7f63540ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:38.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.570+0000 7f6359200700 1 -- 172.21.15.131:0/900925220 shutdown_connections 2024-05-10T05:42:38.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.570+0000 7f6359200700 1 -- 172.21.15.131:0/900925220 wait complete. 2024-05-10T05:42:38.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.572+0000 7f6359200700 1 Processor -- start 2024-05-10T05:42:38.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.572+0000 7f6359200700 1 -- start start 2024-05-10T05:42:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.573+0000 7f6359200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6354101be0 0x7f635419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.573+0000 7f6359200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6354102d00 0x7f635419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.573+0000 7f6359200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63541036a0 0x7f63541a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.573+0000 7f6359200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63541a60a0 con 0x7f6354101be0 2024-05-10T05:42:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.573+0000 7f6359200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63541a6210 con 0x7f6354102d00 2024-05-10T05:42:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.573+0000 7f6359200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63541a63d0 con 0x7f63541036a0 2024-05-10T05:42:38.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.574+0000 7f6351e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6354101be0 0x7f635419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.574+0000 7f6352800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63541036a0 0x7f63541a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.574+0000 7f6351400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6354102d00 0x7f635419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.574+0000 7f6351400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6354102d00 0x7f635419eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35742/0 (socket says 172.21.15.131:35742) 2024-05-10T05:42:38.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.574+0000 7f6351400700 1 -- 172.21.15.131:0/3431892039 learned_addr learned my addr 172.21.15.131:0/3431892039 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:38.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.574+0000 7f6351400700 1 -- 172.21.15.131:0/3431892039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63541036a0 msgr2=0x7f63541a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:38.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.575+0000 7f6351400700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63541036a0 0x7f63541a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.575+0000 7f6351400700 1 -- 172.21.15.131:0/3431892039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6354101be0 msgr2=0x7f635419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.575+0000 7f6351400700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6354101be0 0x7f635419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.575+0000 7f6351400700 1 -- 172.21.15.131:0/3431892039 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63400095f0 con 0x7f6354102d00 2024-05-10T05:42:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.575+0000 7f6351e00700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6354101be0 0x7f635419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.575+0000 7f6351400700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6354102d00 0x7f635419eb00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f634000ec10 tx=0x7f634000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.576+0000 7f634aa00700 1 -- 172.21.15.131:0/3431892039 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634000cd40 con 0x7f6354102d00 2024-05-10T05:42:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.576+0000 7f634aa00700 1 -- 172.21.15.131:0/3431892039 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63400044e0 con 0x7f6354102d00 2024-05-10T05:42:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.576+0000 7f634aa00700 1 -- 172.21.15.131:0/3431892039 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340010690 con 0x7f6354102d00 2024-05-10T05:42:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.576+0000 7f6359200700 1 -- 172.21.15.131:0/3431892039 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634c009c40 con 0x7f6354102d00 2024-05-10T05:42:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.576+0000 7f6359200700 1 -- 172.21.15.131:0/3431892039 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6344009520 con 0x7f6354102d00 2024-05-10T05:42:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.582+0000 7f634aa00700 1 -- 172.21.15.131:0/3431892039 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6340003780 con 0x7f6354102d00 2024-05-10T05:42:38.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.583+0000 7f637109fb80 1 Processor -- start 2024-05-10T05:42:38.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.585+0000 7f634aa00700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f63380a12c0 0x7f63380a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.585+0000 7f634aa00700 1 -- 172.21.15.131:0/3431892039 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6340014070 con 0x7f6354102d00 2024-05-10T05:42:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.587+0000 7f6352800700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f63380a12c0 0x7f63380a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.588+0000 7f637109fb80 1 -- start start 2024-05-10T05:42:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.590+0000 7f6352800700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f63380a12c0 0x7f63380a37d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f634c0094c0 tx=0x7f634c00b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.590+0000 7f637109fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d47baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.590+0000 7f637109fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d47c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.590+0000 7f637109fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47cec0 0x55ac5d4837c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.590+0000 7f637109fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d3a2150 con 0x55ac5d47cec0 2024-05-10T05:42:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.590+0000 7f637109fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d39fb10 con 0x55ac5d47b6a0 2024-05-10T05:42:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.590+0000 7f637109fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d39b400 con 0x55ac5d47c570 2024-05-10T05:42:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.591+0000 7f6351400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d47c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.591+0000 7f6351400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d47c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34246/0 (socket says 172.21.15.131:34246) 2024-05-10T05:42:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.591+0000 7f6351400700 1 -- 172.21.15.131:0/1160322783 learned_addr learned my addr 172.21.15.131:0/1160322783 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.592+0000 7f6351400700 1 -- 172.21.15.131:0/1160322783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 msgr2=0x55ac5d47baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.592+0000 7f6351400700 1 --2- 172.21.15.131:0/1160322783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d47baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.592+0000 7f6351400700 1 -- 172.21.15.131:0/1160322783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47cec0 msgr2=0x55ac5d4837c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.592+0000 7f6351400700 1 --2- 172.21.15.131:0/1160322783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47cec0 0x55ac5d4837c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.592+0000 7f6351400700 1 -- 172.21.15.131:0/1160322783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac5d21c3b0 con 0x55ac5d47c570 2024-05-10T05:42:38.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.594+0000 7f6351400700 1 --2- 172.21.15.131:0/1160322783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d47c980 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6340096cd0 tx=0x7f6340000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db56272ffcf85559 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:38.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.595+0000 7f633f400700 1 -- 172.21.15.131:0/1160322783 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63400af070 con 0x55ac5d47c570 2024-05-10T05:42:38.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.595+0000 7f633f400700 1 -- 172.21.15.131:0/1160322783 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f634009a260 con 0x55ac5d47c570 2024-05-10T05:42:38.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.595+0000 7f633f400700 1 -- 172.21.15.131:0/1160322783 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63400cab00 con 0x55ac5d47c570 2024-05-10T05:42:38.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.596+0000 7f637109fb80 1 -- 172.21.15.131:0/1160322783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 msgr2=0x55ac5d47c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:38.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.596+0000 7f637109fb80 1 --2- 172.21.15.131:0/1160322783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d47c980 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6340096cd0 tx=0x7f6340000bc0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.598+0000 7f637109fb80 1 -- 172.21.15.131:0/1160322783 shutdown_connections 2024-05-10T05:42:38.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.598+0000 7f637109fb80 1 --2- 172.21.15.131:0/1160322783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47cec0 0x55ac5d4837c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.598+0000 7f637109fb80 1 --2- 172.21.15.131:0/1160322783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d47baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.598+0000 7f637109fb80 1 --2- 172.21.15.131:0/1160322783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d47c980 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.598+0000 7f637109fb80 1 -- 172.21.15.131:0/1160322783 >> 172.21.15.131:0/1160322783 conn(0x55ac5d3dda60 msgr2=0x55ac5d475fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:38.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.598+0000 7f637109fb80 1 -- 172.21.15.131:0/1160322783 shutdown_connections 2024-05-10T05:42:38.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.599+0000 7f637109fb80 1 -- 172.21.15.131:0/1160322783 wait complete. 2024-05-10T05:42:38.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.600+0000 7f637109fb80 1 -- start start 2024-05-10T05:42:38.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.603+0000 7f637109fb80 1 Processor -- start 2024-05-10T05:42:38.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.605+0000 7f637109fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d485840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.605+0000 7f637109fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d485d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.605+0000 7f637109fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47cec0 0x55ac5d48cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.609+0000 7f6351400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d485d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.609+0000 7f6351400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d485d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34250/0 (socket says 172.21.15.131:34250) 2024-05-10T05:42:38.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.609+0000 7f6352800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47cec0 0x55ac5d48cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.610+0000 7f6352800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47cec0 0x55ac5d48cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37798/0 (socket says 172.21.15.131:37798) 2024-05-10T05:42:38.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.610+0000 7f6351e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d485840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.610+0000 7f6351400700 1 -- 172.21.15.131:0/596222065 learned_addr learned my addr 172.21.15.131:0/596222065 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:38.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.611+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d337560 con 0x55ac5d47cec0 2024-05-10T05:42:38.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.611+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d337200 con 0x55ac5d47b6a0 2024-05-10T05:42:38.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.611+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d21e6a0 con 0x55ac5d47c570 2024-05-10T05:42:38.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.615+0000 7f637109fb80 10 client.0 _ll_register_callbacks cb 0x55ac5d340b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:38.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.615+0000 7f6351400700 1 -- 172.21.15.131:0/596222065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 msgr2=0x55ac5d485840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:38.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.616+0000 7f6351400700 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d485840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.616+0000 7f6351400700 1 -- 172.21.15.131:0/596222065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47cec0 msgr2=0x55ac5d48cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:38.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.616+0000 7f6351400700 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47cec0 0x55ac5d48cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.616+0000 7f6351400700 1 -- 172.21.15.131:0/596222065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6344009520 con 0x55ac5d47c570 2024-05-10T05:42:38.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.618+0000 7f6351e00700 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d485840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:38.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.619+0000 7f6351400700 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d485d80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6340020010 tx=0x7f634009a880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc2b1f29356cb0d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.621+0000 7f6333e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:42:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.621+0000 7f6333e00700 1 -- 172.21.15.131:0/596222065 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63400af070 con 0x55ac5d47c570 2024-05-10T05:42:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.621+0000 7f6333e00700 1 -- 172.21.15.131:0/596222065 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f634009e090 con 0x55ac5d47c570 2024-05-10T05:42:38.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.622+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac5d2f4270 con 0x55ac5d47c570 2024-05-10T05:42:38.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.622+0000 7f6333e00700 1 -- 172.21.15.131:0/596222065 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63400cab00 con 0x55ac5d47c570 2024-05-10T05:42:38.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.622+0000 7f637109fb80 10 client.27037 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:38.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.622+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ac5d1c8b60 con 0x55ac5d47c570 2024-05-10T05:42:38.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.622+0000 7f6333e00700 1 -- 172.21.15.131:0/596222065 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac5d2f4270 con 0x55ac5d47c570 2024-05-10T05:42:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.623+0000 7f6333e00700 1 -- 172.21.15.131:0/596222065 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6340083bd0 con 0x55ac5d47c570 2024-05-10T05:42:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.623+0000 7f637109fb80 10 client.27037 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.623+0000 7f637109fb80 10 client.27037 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:42:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.623+0000 7f637109fb80 20 client.27037 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.623+0000 7f637109fb80 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ac5d490fd0 0x55ac5d493420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.623+0000 7f637109fb80 4 client.27037 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.623+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac5d44a780 con 0x55ac5d490fd0 2024-05-10T05:42:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.624+0000 7f6351e00700 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ac5d490fd0 0x55ac5d493420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.624+0000 7f6351e00700 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ac5d490fd0 0x55ac5d493420 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8a42250dd052f98 server_cookie=546b0ed42f8a641e in_seq=0 out_seq=0 2024-05-10T05:42:38.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.624+0000 7f6333e00700 10 client.27037 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:42:38.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.625+0000 7f6333e00700 1 -- 172.21.15.131:0/596222065 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ac5d44a780 con 0x55ac5d490fd0 2024-05-10T05:42:38.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.625+0000 7f6333e00700 10 client.27037 handle_command_reply: tid=0 2024-05-10T05:42:38.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.625+0000 7f637109fb80 1 client.27037 shutdown 2024-05-10T05:42:38.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.625+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 msgr2=0x55ac5d485d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:38.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.625+0000 7f637109fb80 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d485d80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6340020010 tx=0x7f634009a880 comp rx=0 tx=0).stop 2024-05-10T05:42:38.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.626+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 shutdown_connections 2024-05-10T05:42:38.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.626+0000 7f637109fb80 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ac5d490fd0 0x55ac5d493420 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.626+0000 7f637109fb80 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47cec0 0x55ac5d48cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.626+0000 7f637109fb80 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d485840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.626+0000 7f637109fb80 1 --2- 172.21.15.131:0/596222065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d485d80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.626+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 >> 172.21.15.131:0/596222065 conn(0x55ac5d3dda60 msgr2=0x55ac5d475fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:38.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.626+0000 7f6333e00700 0 client.27037 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:42:38.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.627+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 shutdown_connections 2024-05-10T05:42:38.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.627+0000 7f637109fb80 1 -- 172.21.15.131:0/596222065 wait complete. 2024-05-10T05:42:38.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.627+0000 7f637109fb80 20 client.27037 trim_cache size 0 max 16384 2024-05-10T05:42:38.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.641+0000 7f637109fb80 1 Processor -- start 2024-05-10T05:42:38.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.641+0000 7f637109fb80 1 -- start start 2024-05-10T05:42:38.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.642+0000 7f637109fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47b6a0 0x55ac5d484db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.642+0000 7f637109fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d48cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.642+0000 7f637109fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47cec0 0x55ac5d485840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.642+0000 7f6351e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47b6a0 0x55ac5d484db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.642+0000 7f6351e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47b6a0 0x55ac5d484db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37812/0 (socket says 172.21.15.131:37812) 2024-05-10T05:42:38.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f6352800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47cec0 0x55ac5d485840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f6352800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47cec0 0x55ac5d485840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35770/0 (socket says 172.21.15.131:35770) 2024-05-10T05:42:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f6352800700 1 -- 172.21.15.131:0/3797841427 learned_addr learned my addr 172.21.15.131:0/3797841427 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f6351400700 1 --2- 172.21.15.131:0/3797841427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d48cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f637109fb80 1 -- 172.21.15.131:0/3797841427 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d470830 con 0x55ac5d47b6a0 2024-05-10T05:42:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f637109fb80 1 -- 172.21.15.131:0/3797841427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d47bf80 con 0x55ac5d47cec0 2024-05-10T05:42:38.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f637109fb80 1 -- 172.21.15.131:0/3797841427 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d490d80 con 0x55ac5d47c570 2024-05-10T05:42:38.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f6352800700 1 -- 172.21.15.131:0/3797841427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 msgr2=0x55ac5d48cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:38.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f6352800700 1 --2- 172.21.15.131:0/3797841427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d48cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f6352800700 1 -- 172.21.15.131:0/3797841427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47b6a0 msgr2=0x55ac5d484db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:38.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.643+0000 7f6352800700 1 --2- 172.21.15.131:0/3797841427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47b6a0 0x55ac5d484db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.644+0000 7f6352800700 1 -- 172.21.15.131:0/3797841427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac5d361560 con 0x55ac5d47cec0 2024-05-10T05:42:38.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.647+0000 7f6352800700 1 --2- 172.21.15.131:0/3797841427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47cec0 0x55ac5d485840 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f634c015570 tx=0x7f634c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=606b8c84e80016f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:38.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.647+0000 7f633fe00700 1 -- 172.21.15.131:0/3797841427 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c00e050 con 0x55ac5d47cec0 2024-05-10T05:42:38.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.648+0000 7f633fe00700 1 -- 172.21.15.131:0/3797841427 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f634c00f040 con 0x55ac5d47cec0 2024-05-10T05:42:38.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.648+0000 7f633fe00700 1 -- 172.21.15.131:0/3797841427 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c016610 con 0x55ac5d47cec0 2024-05-10T05:42:38.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.650+0000 7f637109fb80 1 -- 172.21.15.131:0/3797841427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47cec0 msgr2=0x55ac5d485840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:38.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.650+0000 7f637109fb80 1 --2- 172.21.15.131:0/3797841427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47cec0 0x55ac5d485840 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f634c015570 tx=0x7f634c013000 comp rx=0 tx=0).stop 2024-05-10T05:42:38.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.650+0000 7f637109fb80 1 -- 172.21.15.131:0/3797841427 shutdown_connections 2024-05-10T05:42:38.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.650+0000 7f637109fb80 1 --2- 172.21.15.131:0/3797841427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47b6a0 0x55ac5d484db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.650+0000 7f637109fb80 1 --2- 172.21.15.131:0/3797841427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47cec0 0x55ac5d485840 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.650+0000 7f637109fb80 1 --2- 172.21.15.131:0/3797841427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47c570 0x55ac5d48cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.650+0000 7f637109fb80 1 -- 172.21.15.131:0/3797841427 >> 172.21.15.131:0/3797841427 conn(0x55ac5d3dda60 msgr2=0x55ac5d47f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:38.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.650+0000 7f637109fb80 1 -- 172.21.15.131:0/3797841427 shutdown_connections 2024-05-10T05:42:38.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.650+0000 7f637109fb80 1 -- 172.21.15.131:0/3797841427 wait complete. 2024-05-10T05:42:38.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.651+0000 7f637109fb80 1 -- start start 2024-05-10T05:42:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.651+0000 7f637109fb80 1 Processor -- start 2024-05-10T05:42:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.652+0000 7f637109fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d48a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.652+0000 7f637109fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47c570 0x55ac5d48ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.652+0000 7f637109fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47cec0 0x55ac5d48bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.652+0000 7f637109fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d48c0e0 con 0x55ac5d47c570 2024-05-10T05:42:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.652+0000 7f637109fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d48c250 con 0x55ac5d47b6a0 2024-05-10T05:42:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.652+0000 7f637109fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5d48c410 con 0x55ac5d47cec0 2024-05-10T05:42:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.652+0000 7f637109fb80 10 client.0 _ll_register_callbacks cb 0x55ac5d340b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d48a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47c570 0x55ac5d48ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47c570 0x55ac5d48ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37828/0 (socket says 172.21.15.131:37828) 2024-05-10T05:42:38.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d48a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35772/0 (socket says 172.21.15.131:35772) 2024-05-10T05:42:38.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351400700 1 -- 172.21.15.131:0/3582012913 learned_addr learned my addr 172.21.15.131:0/3582012913 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:38.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6352800700 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47cec0 0x55ac5d48bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351e00700 1 -- 172.21.15.131:0/3582012913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47cec0 msgr2=0x55ac5d48bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:38.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351e00700 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47cec0 0x55ac5d48bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351e00700 1 -- 172.21.15.131:0/3582012913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47c570 msgr2=0x55ac5d48ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:38.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351e00700 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47c570 0x55ac5d48ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351e00700 1 -- 172.21.15.131:0/3582012913 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c01e040 con 0x55ac5d47b6a0 2024-05-10T05:42:38.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.653+0000 7f6351400700 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47c570 0x55ac5d48ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:38.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.654+0000 7f6351e00700 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d48a730 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6344009230 tx=0x7f6344009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=322416f3762df39e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:38.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.654+0000 7f6333e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:38.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.654+0000 7f6333e00700 1 -- 172.21.15.131:0/3582012913 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634401c040 con 0x55ac5d47b6a0 2024-05-10T05:42:38.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.654+0000 7f6333e00700 1 -- 172.21.15.131:0/3582012913 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6344027e40 con 0x55ac5d47b6a0 2024-05-10T05:42:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.654+0000 7f637109fb80 1 -- 172.21.15.131:0/3582012913 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f634009a260 con 0x55ac5d47b6a0 2024-05-10T05:42:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.655+0000 7f6333e00700 1 -- 172.21.15.131:0/3582012913 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f634400b560 con 0x55ac5d47b6a0 2024-05-10T05:42:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.655+0000 7f637109fb80 10 client.27002 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.655+0000 7f637109fb80 1 -- 172.21.15.131:0/3582012913 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ac5d220cd0 con 0x55ac5d47b6a0 2024-05-10T05:42:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.656+0000 7f6333e00700 1 -- 172.21.15.131:0/3582012913 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f634009a260 con 0x55ac5d47b6a0 2024-05-10T05:42:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.656+0000 7f6333e00700 1 -- 172.21.15.131:0/3582012913 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6344015020 con 0x55ac5d47b6a0 2024-05-10T05:42:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.656+0000 7f637109fb80 10 client.27002 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.656+0000 7f637109fb80 10 client.27002 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:42:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.656+0000 7f637109fb80 20 client.27002 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.656+0000 7f637109fb80 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ac5d4e7ca0 0x55ac5d4ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.656+0000 7f637109fb80 4 client.27002 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:42:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.656+0000 7f637109fb80 1 -- 172.21.15.131:0/3582012913 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac5d393ec0 con 0x55ac5d4e7ca0 2024-05-10T05:42:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.657+0000 7f6351400700 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ac5d4e7ca0 0x55ac5d4ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.657+0000 7f6351400700 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ac5d4e7ca0 0x55ac5d4ea0c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fb8a2e75ebbba443 server_cookie=47d69fff97eb4ea4 in_seq=0 out_seq=0 2024-05-10T05:42:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.657+0000 7f6333e00700 10 client.27002 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:42:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f6333e00700 1 -- 172.21.15.131:0/3582012913 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ac5d393ec0 con 0x55ac5d4e7ca0 2024-05-10T05:42:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f6333e00700 10 client.27002 handle_command_reply: tid=0 2024-05-10T05:42:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f637109fb80 1 client.27002 shutdown 2024-05-10T05:42:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f637109fb80 1 -- 172.21.15.131:0/3582012913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 msgr2=0x55ac5d48a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:38.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f637109fb80 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d48a730 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6344009230 tx=0x7f6344009260 comp rx=0 tx=0).stop 2024-05-10T05:42:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f637109fb80 1 -- 172.21.15.131:0/3582012913 shutdown_connections 2024-05-10T05:42:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f637109fb80 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55ac5d4e7ca0 0x55ac5d4ea0c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f637109fb80 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac5d47cec0 0x55ac5d48bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f637109fb80 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac5d47b6a0 0x55ac5d48a730 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f637109fb80 1 --2- 172.21.15.131:0/3582012913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac5d47c570 0x55ac5d48ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f637109fb80 1 -- 172.21.15.131:0/3582012913 >> 172.21.15.131:0/3582012913 conn(0x55ac5d3dda60 msgr2=0x55ac5d47f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.658+0000 7f6333e00700 0 client.27002 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:42:38.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.659+0000 7f637109fb80 1 -- 172.21.15.131:0/3582012913 shutdown_connections 2024-05-10T05:42:38.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.659+0000 7f637109fb80 1 -- 172.21.15.131:0/3582012913 wait complete. 2024-05-10T05:42:38.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.659+0000 7f637109fb80 20 client.27002 trim_cache size 0 max 16384 2024-05-10T05:42:38.661 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.660+0000 7f6359200700 1 -- 172.21.15.131:0/3431892039 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f63380a12c0 msgr2=0x7f63380a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.660+0000 7f6359200700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f63380a12c0 0x7f63380a37d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f634c0094c0 tx=0x7f634c00b040 comp rx=0 tx=0).stop 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.660+0000 7f6359200700 1 -- 172.21.15.131:0/3431892039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6354102d00 msgr2=0x7f635419eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.660+0000 7f6359200700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6354102d00 0x7f635419eb00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f634000ec10 tx=0x7f634000ef20 comp rx=0 tx=0).stop 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.661+0000 7f6359200700 1 -- 172.21.15.131:0/3431892039 shutdown_connections 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.661+0000 7f6359200700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f63380a12c0 0x7f63380a37d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.661+0000 7f6359200700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6354101be0 0x7f635419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.661+0000 7f6359200700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63541036a0 0x7f63541a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.661+0000 7f6359200700 1 --2- 172.21.15.131:0/3431892039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6354102d00 0x7f635419eb00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.661+0000 7f6359200700 1 -- 172.21.15.131:0/3431892039 >> 172.21.15.131:0/3431892039 conn(0x7f63540fd1d0 msgr2=0x7f635410dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.661+0000 7f6359200700 1 -- 172.21.15.131:0/3431892039 shutdown_connections 2024-05-10T05:42:38.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:38.661+0000 7f6359200700 1 -- 172.21.15.131:0/3431892039 wait complete. 2024-05-10T05:42:38.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:38 smithi131 ceph-mon[37264]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 243 MiB/s wr, 75 op/s 2024-05-10T05:42:38.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:38 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3409503840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:42:38.731 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:42:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:38 smithi157 ceph-mon[78427]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 243 MiB/s wr, 75 op/s 2024-05-10T05:42:38.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:38 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3409503840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:42:38.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:38 smithi142 ceph-mon[77667]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 243 MiB/s wr, 75 op/s 2024-05-10T05:42:38.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:38 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3409503840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:42:39.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.409+0000 7fa151000700 1 -- 172.21.15.131:0/1449768619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14c109300 msgr2=0x7fa14c109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:39.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.409+0000 7fa151000700 1 --2- 172.21.15.131:0/1449768619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14c109300 0x7fa14c109710 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa138009990 tx=0x7fa138009ca0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.410+0000 7fa151000700 1 -- 172.21.15.131:0/1449768619 shutdown_connections 2024-05-10T05:42:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.410+0000 7fa151000700 1 --2- 172.21.15.131:0/1449768619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa14c108630 0x7fa14c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.410+0000 7fa151000700 1 --2- 172.21.15.131:0/1449768619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa14c109c50 0x7fa14c10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.410+0000 7fa151000700 1 --2- 172.21.15.131:0/1449768619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14c109300 0x7fa14c109710 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.410+0000 7fa151000700 1 -- 172.21.15.131:0/1449768619 >> 172.21.15.131:0/1449768619 conn(0x7fa14c06ea00 msgr2=0x7fa14c070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.411+0000 7fa151000700 1 -- 172.21.15.131:0/1449768619 shutdown_connections 2024-05-10T05:42:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.412+0000 7fa151000700 1 -- 172.21.15.131:0/1449768619 wait complete. 2024-05-10T05:42:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.413+0000 7fa151000700 1 Processor -- start 2024-05-10T05:42:39.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.415+0000 7fa151000700 1 -- start start 2024-05-10T05:42:39.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.419+0000 7fa151000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa14c108630 0x7fa14c1a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.419+0000 7fa151000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa14c109300 0x7fa14c1a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.420+0000 7fa151000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14c109c50 0x7fa14c1ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.420+0000 7fa151000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c1ae670 con 0x7fa14c108630 2024-05-10T05:42:39.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.420+0000 7fa151000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c1ae7e0 con 0x7fa14c109c50 2024-05-10T05:42:39.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.420+0000 7fa151000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c1ae9a0 con 0x7fa14c109300 2024-05-10T05:42:39.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.420+0000 7fa149e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa14c108630 0x7fa14c1a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.420+0000 7fa149400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa14c109300 0x7fa14c1a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.420+0000 7fa14a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14c109c50 0x7fa14c1ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.420+0000 7fa14a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14c109c50 0x7fa14c1ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35788/0 (socket says 172.21.15.131:35788) 2024-05-10T05:42:39.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.420+0000 7fa14a800700 1 -- 172.21.15.131:0/4105323348 learned_addr learned my addr 172.21.15.131:0/4105323348 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:39.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.421+0000 7fa14a800700 1 -- 172.21.15.131:0/4105323348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa14c109300 msgr2=0x7fa14c1a70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:39.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.421+0000 7fa14a800700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa14c109300 0x7fa14c1a70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.421+0000 7fa14a800700 1 -- 172.21.15.131:0/4105323348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa14c108630 msgr2=0x7fa14c1a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:39.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.421+0000 7fa14a800700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa14c108630 0x7fa14c1a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.421+0000 7fa14a800700 1 -- 172.21.15.131:0/4105323348 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1380095f0 con 0x7fa14c109c50 2024-05-10T05:42:39.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.421+0000 7fa14a800700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14c109c50 0x7fa14c1ae060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa14400ef30 tx=0x7fa14400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:39.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.421+0000 7fa142a00700 1 -- 172.21.15.131:0/4105323348 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144018070 con 0x7fa14c109c50 2024-05-10T05:42:39.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.421+0000 7fa142a00700 1 -- 172.21.15.131:0/4105323348 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa14400f040 con 0x7fa14c109c50 2024-05-10T05:42:39.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.422+0000 7fa142a00700 1 -- 172.21.15.131:0/4105323348 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400cd00 con 0x7fa14c109c50 2024-05-10T05:42:39.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.422+0000 7fa151000700 1 -- 172.21.15.131:0/4105323348 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14c1aec20 con 0x7fa14c109c50 2024-05-10T05:42:39.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.422+0000 7fa151000700 1 -- 172.21.15.131:0/4105323348 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c1af0e0 con 0x7fa14c109c50 2024-05-10T05:42:39.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.428+0000 7fa142a00700 1 -- 172.21.15.131:0/4105323348 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa144004000 con 0x7fa14c109c50 2024-05-10T05:42:39.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.431+0000 7fa142a00700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa1300a12c0 0x7fa1300a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.431+0000 7fa142a00700 1 -- 172.21.15.131:0/4105323348 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa1440cd1e0 con 0x7fa14c109c50 2024-05-10T05:42:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.433+0000 7fa168e4eb80 1 Processor -- start 2024-05-10T05:42:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.434+0000 7fa149e00700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa1300a12c0 0x7fa1300a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.437+0000 7fa149e00700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa1300a12c0 0x7fa1300a37d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005950 tx=0x7fa13c00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.438+0000 7fa168e4eb80 1 -- start start 2024-05-10T05:42:39.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.443+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 0x56500bcedaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.444+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bcee570 0x56500bcee980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.444+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bceeec0 0x56500bcf57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.445+0000 7fa14a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bceeec0 0x56500bcf57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.445+0000 7fa14a800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bceeec0 0x56500bcf57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34294/0 (socket says 172.21.15.131:34294) 2024-05-10T05:42:39.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.445+0000 7fa149400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bcee570 0x56500bcee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.445+0000 7fa149400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bcee570 0x56500bcee980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35794/0 (socket says 172.21.15.131:35794) 2024-05-10T05:42:39.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.446+0000 7fa14a800700 1 -- 172.21.15.131:0/3533884320 learned_addr learned my addr 172.21.15.131:0/3533884320 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:39.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.446+0000 7fa168e4eb80 1 -- 172.21.15.131:0/3533884320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bc14150 con 0x56500bced6a0 2024-05-10T05:42:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.446+0000 7fa149e00700 1 --2- 172.21.15.131:0/3533884320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 0x56500bcedaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.446+0000 7fa168e4eb80 1 -- 172.21.15.131:0/3533884320 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bc11b10 con 0x56500bcee570 2024-05-10T05:42:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.446+0000 7fa168e4eb80 1 -- 172.21.15.131:0/3533884320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bc0d400 con 0x56500bceeec0 2024-05-10T05:42:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.447+0000 7fa14a800700 1 -- 172.21.15.131:0/3533884320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bcee570 msgr2=0x56500bcee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.447+0000 7fa14a800700 1 --2- 172.21.15.131:0/3533884320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bcee570 0x56500bcee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.447+0000 7fa14a800700 1 -- 172.21.15.131:0/3533884320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 msgr2=0x56500bcedaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.448+0000 7fa14a800700 1 --2- 172.21.15.131:0/3533884320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 0x56500bcedaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.448+0000 7fa14a800700 1 -- 172.21.15.131:0/3533884320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56500ba8e3b0 con 0x56500bceeec0 2024-05-10T05:42:39.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.448+0000 7fa149e00700 1 --2- 172.21.15.131:0/3533884320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 0x56500bcedaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:39.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.449+0000 7fa14a800700 1 --2- 172.21.15.131:0/3533884320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bceeec0 0x56500bcf57c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa1440090a0 tx=0x7fa14409a500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a334755d05503fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:39.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.450+0000 7fa137400700 1 -- 172.21.15.131:0/3533884320 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1440ba070 con 0x56500bceeec0 2024-05-10T05:42:39.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.450+0000 7fa137400700 1 -- 172.21.15.131:0/3533884320 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1440cc070 con 0x56500bceeec0 2024-05-10T05:42:39.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.450+0000 7fa137400700 1 -- 172.21.15.131:0/3533884320 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14409e490 con 0x56500bceeec0 2024-05-10T05:42:39.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.451+0000 7fa168e4eb80 1 -- 172.21.15.131:0/3533884320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bceeec0 msgr2=0x56500bcf57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:39.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.451+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/3533884320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bceeec0 0x56500bcf57c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa1440090a0 tx=0x7fa14409a500 comp rx=0 tx=0).stop 2024-05-10T05:42:39.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.454+0000 7fa168e4eb80 1 -- 172.21.15.131:0/3533884320 shutdown_connections 2024-05-10T05:42:39.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.454+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/3533884320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 0x56500bcedaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.454+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/3533884320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bceeec0 0x56500bcf57c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.454+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/3533884320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bcee570 0x56500bcee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.454+0000 7fa168e4eb80 1 -- 172.21.15.131:0/3533884320 >> 172.21.15.131:0/3533884320 conn(0x56500bc4fa60 msgr2=0x56500bce7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.455+0000 7fa168e4eb80 1 -- 172.21.15.131:0/3533884320 shutdown_connections 2024-05-10T05:42:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.455+0000 7fa168e4eb80 1 -- 172.21.15.131:0/3533884320 wait complete. 2024-05-10T05:42:39.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.456+0000 7fa168e4eb80 1 -- start start 2024-05-10T05:42:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.461+0000 7fa168e4eb80 1 Processor -- start 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.468+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 0x56500bcf77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.469+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 0x56500bcf7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.469+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 0x56500bcfec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.469+0000 7fa149e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 0x56500bcf77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.469+0000 7fa149400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 0x56500bcf7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.469+0000 7fa168e4eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bba9560 con 0x56500bced6a0 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.469+0000 7fa168e4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bba9200 con 0x56500bceeec0 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.469+0000 7fa168e4eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500ba906a0 con 0x56500bcee570 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.469+0000 7fa149400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 0x56500bcf7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34300/0 (socket says 172.21.15.131:34300) 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.469+0000 7fa149400700 1 -- 172.21.15.131:0/2461686370 learned_addr learned my addr 172.21.15.131:0/2461686370 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.470+0000 7fa168e4eb80 10 client.0 _ll_register_callbacks cb 0x56500bbb2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:39.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.470+0000 7fa149400700 1 -- 172.21.15.131:0/2461686370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 msgr2=0x56500bcfec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:39.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.470+0000 7fa149400700 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 0x56500bcfec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.470+0000 7fa149400700 1 -- 172.21.15.131:0/2461686370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 msgr2=0x56500bcf77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:39.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.470+0000 7fa149400700 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 0x56500bcf77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.470+0000 7fa149400700 1 -- 172.21.15.131:0/2461686370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c010c20 con 0x56500bcee570 2024-05-10T05:42:39.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.470+0000 7fa149e00700 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 0x56500bcf77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:42:39.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.472+0000 7fa149400700 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 0x56500bcf7d10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b9b0 tx=0x7fa13800b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5fa45e24c23d93 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:39.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.475+0000 7fa12be00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:42:39.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.475+0000 7fa12be00700 1 -- 172.21.15.131:0/2461686370 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138004910 con 0x56500bcee570 2024-05-10T05:42:39.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.475+0000 7fa12be00700 1 -- 172.21.15.131:0/2461686370 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa138004a70 con 0x56500bcee570 2024-05-10T05:42:39.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.475+0000 7fa168e4eb80 1 -- 172.21.15.131:0/2461686370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56500bb66270 con 0x56500bcee570 2024-05-10T05:42:39.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.479+0000 7fa12be00700 1 -- 172.21.15.131:0/2461686370 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa13800be90 con 0x56500bcee570 2024-05-10T05:42:39.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.479+0000 7fa168e4eb80 10 client.27055 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:39.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.479+0000 7fa168e4eb80 1 -- 172.21.15.131:0/2461686370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fa14c1af0e0 con 0x56500bcee570 2024-05-10T05:42:39.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.479+0000 7fa12be00700 1 -- 172.21.15.131:0/2461686370 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56500bb66270 con 0x56500bcee570 2024-05-10T05:42:39.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.481+0000 7fa12be00700 1 -- 172.21.15.131:0/2461686370 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa13802c070 con 0x56500bcee570 2024-05-10T05:42:39.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.481+0000 7fa168e4eb80 10 client.27055 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:39.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.481+0000 7fa168e4eb80 10 client.27055 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:42:39.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.481+0000 7fa168e4eb80 20 client.27055 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:39.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.482+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56500bd033c0 0x56500bd05810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.484+0000 7fa168e4eb80 4 client.27055 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:39.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.484+0000 7fa168e4eb80 1 -- 172.21.15.131:0/2461686370 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56500bcbc780 con 0x56500bd033c0 2024-05-10T05:42:39.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.484+0000 7fa149e00700 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56500bd033c0 0x56500bd05810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.484+0000 7fa149e00700 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56500bd033c0 0x56500bd05810 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb9cd15dfa7b7366 server_cookie=1af896a780f3a03c in_seq=0 out_seq=0 2024-05-10T05:42:39.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.485+0000 7fa12be00700 10 client.27055 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:42:39.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.485+0000 7fa12be00700 1 -- 172.21.15.131:0/2461686370 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56500bcbc780 con 0x56500bd033c0 2024-05-10T05:42:39.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.485+0000 7fa12be00700 10 client.27055 handle_command_reply: tid=0 2024-05-10T05:42:39.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.485+0000 7fa168e4eb80 1 client.27055 shutdown 2024-05-10T05:42:39.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.488+0000 7fa168e4eb80 1 -- 172.21.15.131:0/2461686370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 msgr2=0x56500bcf7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:39.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.488+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 0x56500bcf7d10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b9b0 tx=0x7fa13800b9e0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.489+0000 7fa168e4eb80 1 -- 172.21.15.131:0/2461686370 shutdown_connections 2024-05-10T05:42:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.489+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56500bd033c0 0x56500bd05810 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.489+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bced6a0 0x56500bcf77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.489+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 0x56500bcfec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.489+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/2461686370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 0x56500bcf7d10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.489+0000 7fa168e4eb80 1 -- 172.21.15.131:0/2461686370 >> 172.21.15.131:0/2461686370 conn(0x56500bc4fa60 msgr2=0x56500bce7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:39.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.489+0000 7fa12be00700 0 client.27055 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:42:39.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.489+0000 7fa168e4eb80 1 -- 172.21.15.131:0/2461686370 shutdown_connections 2024-05-10T05:42:39.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.489+0000 7fa168e4eb80 1 -- 172.21.15.131:0/2461686370 wait complete. 2024-05-10T05:42:39.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.492+0000 7fa168e4eb80 20 client.27055 trim_cache size 0 max 16384 2024-05-10T05:42:39.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.560+0000 7fa168e4eb80 1 Processor -- start 2024-05-10T05:42:39.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.560+0000 7fa168e4eb80 1 -- start start 2024-05-10T05:42:39.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.561+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bced6a0 0x56500bcf77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.561+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bcee570 0x56500bcfebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.561+0000 7fa149e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bced6a0 0x56500bcf77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.562+0000 7fa149e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bced6a0 0x56500bcf77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34310/0 (socket says 172.21.15.131:34310) 2024-05-10T05:42:39.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.562+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 0x56500bcf7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.562+0000 7fa168e4eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bce27c0 con 0x56500bcee570 2024-05-10T05:42:39.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.562+0000 7fa14a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 0x56500bcf7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.562+0000 7fa14a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 0x56500bcf7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35814/0 (socket says 172.21.15.131:35814) 2024-05-10T05:42:39.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.562+0000 7fa14a800700 1 -- 172.21.15.131:0/1411512274 learned_addr learned my addr 172.21.15.131:0/1411512274 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:39.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.562+0000 7fa168e4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bd030c0 con 0x56500bceeec0 2024-05-10T05:42:39.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.562+0000 7fa168e4eb80 1 -- 172.21.15.131:0/1411512274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bcf83d0 con 0x56500bced6a0 2024-05-10T05:42:39.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.563+0000 7fa14a800700 1 -- 172.21.15.131:0/1411512274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bced6a0 msgr2=0x56500bcf77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:39.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.563+0000 7fa14a800700 1 --2- 172.21.15.131:0/1411512274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bced6a0 0x56500bcf77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.563+0000 7fa14a800700 1 -- 172.21.15.131:0/1411512274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bcee570 msgr2=0x56500bcfebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:39.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.563+0000 7fa14a800700 1 --2- 172.21.15.131:0/1411512274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bcee570 0x56500bcfebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.563+0000 7fa14a800700 1 -- 172.21.15.131:0/1411512274 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56500ba3ab60 con 0x56500bceeec0 2024-05-10T05:42:39.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.567+0000 7fa14a800700 1 --2- 172.21.15.131:0/1411512274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 0x56500bcf7d10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa1440970d0 tx=0x7fa14409a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f391686bd07f9b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:39.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.567+0000 7fa137e00700 1 -- 172.21.15.131:0/1411512274 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1440ba0c0 con 0x56500bceeec0 2024-05-10T05:42:39.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.567+0000 7fa137e00700 1 -- 172.21.15.131:0/1411512274 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa14409a6d0 con 0x56500bceeec0 2024-05-10T05:42:39.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.567+0000 7fa137e00700 1 -- 172.21.15.131:0/1411512274 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14409e520 con 0x56500bceeec0 2024-05-10T05:42:39.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.569+0000 7fa168e4eb80 1 -- 172.21.15.131:0/1411512274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 msgr2=0x56500bcf7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:39.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.569+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/1411512274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 0x56500bcf7d10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa1440970d0 tx=0x7fa14409a430 comp rx=0 tx=0).stop 2024-05-10T05:42:39.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.571+0000 7fa168e4eb80 1 -- 172.21.15.131:0/1411512274 shutdown_connections 2024-05-10T05:42:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.571+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/1411512274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bceeec0 0x56500bcf7d10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.571+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/1411512274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bced6a0 0x56500bcf77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.571+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/1411512274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bcee570 0x56500bcfebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.571+0000 7fa168e4eb80 1 -- 172.21.15.131:0/1411512274 >> 172.21.15.131:0/1411512274 conn(0x56500bc4fa60 msgr2=0x56500bd0a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.571+0000 7fa168e4eb80 1 -- 172.21.15.131:0/1411512274 shutdown_connections 2024-05-10T05:42:39.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.571+0000 7fa168e4eb80 1 -- 172.21.15.131:0/1411512274 wait complete. 2024-05-10T05:42:39.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.572+0000 7fa168e4eb80 1 -- start start 2024-05-10T05:42:39.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.572+0000 7fa168e4eb80 1 Processor -- start 2024-05-10T05:42:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.576+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bced6a0 0x56500bcfbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.576+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 0x56500bcfc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.576+0000 7fa168e4eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bceeec0 0x56500bcfd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.576+0000 7fa168e4eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bcfd7c0 con 0x56500bceeec0 2024-05-10T05:42:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.576+0000 7fa168e4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bcfd930 con 0x56500bced6a0 2024-05-10T05:42:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.576+0000 7fa168e4eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500bcfdaa0 con 0x56500bcee570 2024-05-10T05:42:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.577+0000 7fa168e4eb80 10 client.0 _ll_register_callbacks cb 0x56500bbb2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.577+0000 7fa14a800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bceeec0 0x56500bcfd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.578+0000 7fa149e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bced6a0 0x56500bcfbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.578+0000 7fa149e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bced6a0 0x56500bcfbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35824/0 (socket says 172.21.15.131:35824) 2024-05-10T05:42:39.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.578+0000 7fa149e00700 1 -- 172.21.15.131:0/143459167 learned_addr learned my addr 172.21.15.131:0/143459167 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:39.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.579+0000 7fa149e00700 1 -- 172.21.15.131:0/143459167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 msgr2=0x56500bcfc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:39.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.579+0000 7fa149e00700 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 0x56500bcfc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.579+0000 7fa149e00700 1 -- 172.21.15.131:0/143459167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bceeec0 msgr2=0x56500bcfd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:39.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.579+0000 7fa149e00700 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bceeec0 0x56500bcfd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.579+0000 7fa149e00700 1 -- 172.21.15.131:0/143459167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56500ba8e3b0 con 0x56500bced6a0 2024-05-10T05:42:39.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.579+0000 7fa149e00700 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bced6a0 0x56500bcfbe60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005d90 tx=0x7fa13c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd9f6d3ea519e9e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:39.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.579+0000 7fa12be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.582+0000 7fa12be00700 1 -- 172.21.15.131:0/143459167 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c026050 con 0x56500bced6a0 2024-05-10T05:42:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.582+0000 7fa12be00700 1 -- 172.21.15.131:0/143459167 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa13c017070 con 0x56500bced6a0 2024-05-10T05:42:39.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.584+0000 7fa168e4eb80 1 -- 172.21.15.131:0/143459167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56500bc05ec0 con 0x56500bced6a0 2024-05-10T05:42:39.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.592+0000 7fa12be00700 1 -- 172.21.15.131:0/143459167 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa13c027d00 con 0x56500bced6a0 2024-05-10T05:42:39.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.592+0000 7fa168e4eb80 10 client.27017 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:39.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.592+0000 7fa168e4eb80 1 -- 172.21.15.131:0/143459167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56500bbd3560 con 0x56500bced6a0 2024-05-10T05:42:39.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.596+0000 7fa12be00700 1 -- 172.21.15.131:0/143459167 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56500bc05ec0 con 0x56500bced6a0 2024-05-10T05:42:39.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.596+0000 7fa12be00700 1 -- 172.21.15.131:0/143459167 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa13c011830 con 0x56500bced6a0 2024-05-10T05:42:39.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.597+0000 7fa168e4eb80 10 client.27017 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:39.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.597+0000 7fa168e4eb80 10 client.27017 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:42:39.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.597+0000 7fa168e4eb80 20 client.27017 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:39.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.597+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56500bd6ac20 0x56500bd6d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:39.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.597+0000 7fa168e4eb80 4 client.27017 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:42:39.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.597+0000 7fa168e4eb80 1 -- 172.21.15.131:0/143459167 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56500ba5bcf0 con 0x56500bd6ac20 2024-05-10T05:42:39.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.598+0000 7fa149400700 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56500bd6ac20 0x56500bd6d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.599+0000 7fa149400700 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56500bd6ac20 0x56500bd6d040 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c85e35d4a0caaaf1 server_cookie=e869b9324857defc in_seq=0 out_seq=0 2024-05-10T05:42:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.599+0000 7fa12be00700 10 client.27017 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:42:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.599+0000 7fa12be00700 1 -- 172.21.15.131:0/143459167 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56500ba5bcf0 con 0x56500bd6ac20 2024-05-10T05:42:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.599+0000 7fa12be00700 10 client.27017 handle_command_reply: tid=0 2024-05-10T05:42:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.599+0000 7fa168e4eb80 1 client.27017 shutdown 2024-05-10T05:42:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa168e4eb80 1 -- 172.21.15.131:0/143459167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bced6a0 msgr2=0x56500bcfbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:39.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bced6a0 0x56500bcfbe60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005d90 tx=0x7fa13c0047c0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa168e4eb80 1 -- 172.21.15.131:0/143459167 shutdown_connections 2024-05-10T05:42:39.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56500bd6ac20 0x56500bd6d040 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56500bceeec0 0x56500bcfd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56500bced6a0 0x56500bcfbe60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa168e4eb80 1 --2- 172.21.15.131:0/143459167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56500bcee570 0x56500bcfc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa168e4eb80 1 -- 172.21.15.131:0/143459167 >> 172.21.15.131:0/143459167 conn(0x56500bc4fa60 msgr2=0x56500bd053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:39.605 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:42:39.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa12be00700 0 client.27017 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:42:39.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa168e4eb80 1 -- 172.21.15.131:0/143459167 shutdown_connections 2024-05-10T05:42:39.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.600+0000 7fa168e4eb80 1 -- 172.21.15.131:0/143459167 wait complete. 2024-05-10T05:42:39.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.601+0000 7fa168e4eb80 20 client.27017 trim_cache size 0 max 16384 2024-05-10T05:42:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.611+0000 7fa151000700 1 -- 172.21.15.131:0/4105323348 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa1300a12c0 msgr2=0x7fa1300a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.611+0000 7fa151000700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa1300a12c0 0x7fa1300a37d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005950 tx=0x7fa13c00b410 comp rx=0 tx=0).stop 2024-05-10T05:42:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.611+0000 7fa151000700 1 -- 172.21.15.131:0/4105323348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14c109c50 msgr2=0x7fa14c1ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.611+0000 7fa151000700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14c109c50 0x7fa14c1ae060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa14400ef30 tx=0x7fa14400c3a0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.611+0000 7fa151000700 1 -- 172.21.15.131:0/4105323348 shutdown_connections 2024-05-10T05:42:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.611+0000 7fa151000700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa1300a12c0 0x7fa1300a37d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.611+0000 7fa151000700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa14c108630 0x7fa14c1a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.611+0000 7fa151000700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14c109c50 0x7fa14c1ae060 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.611+0000 7fa151000700 1 --2- 172.21.15.131:0/4105323348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa14c109300 0x7fa14c1a70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.611+0000 7fa151000700 1 -- 172.21.15.131:0/4105323348 >> 172.21.15.131:0/4105323348 conn(0x7fa14c06ea00 msgr2=0x7fa14c10a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.612+0000 7fa151000700 1 -- 172.21.15.131:0/4105323348 shutdown_connections 2024-05-10T05:42:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:39.614+0000 7fa151000700 1 -- 172.21.15.131:0/4105323348 wait complete. 2024-05-10T05:42:39.776 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.328+0000 7f8d28c00700 1 -- 172.21.15.131:0/1940163982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d24101c00 msgr2=0x7f8d24102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.328+0000 7f8d28c00700 1 --2- 172.21.15.131:0/1940163982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d24101c00 0x7f8d24102010 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0098f0 tx=0x7f8d0c009c00 comp rx=0 tx=0).stop 2024-05-10T05:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.329+0000 7f8d28c00700 1 -- 172.21.15.131:0/1940163982 shutdown_connections 2024-05-10T05:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.329+0000 7f8d28c00700 1 --2- 172.21.15.131:0/1940163982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d24101c00 0x7f8d24102010 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.329+0000 7f8d28c00700 1 --2- 172.21.15.131:0/1940163982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d241036c0 0x7f8d24109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.329+0000 7f8d28c00700 1 --2- 172.21.15.131:0/1940163982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d24102d20 0x7f8d24103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.329+0000 7f8d28c00700 1 -- 172.21.15.131:0/1940163982 >> 172.21.15.131:0/1940163982 conn(0x7f8d240fd210 msgr2=0x7f8d240ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.330+0000 7f8d28c00700 1 -- 172.21.15.131:0/1940163982 shutdown_connections 2024-05-10T05:42:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.330+0000 7f8d28c00700 1 -- 172.21.15.131:0/1940163982 wait complete. 2024-05-10T05:42:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.332+0000 7f8d28c00700 1 Processor -- start 2024-05-10T05:42:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.332+0000 7f8d28c00700 1 -- start start 2024-05-10T05:42:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.333+0000 7f8d28c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d24101c00 0x7f8d2419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.333+0000 7f8d28c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d24102d20 0x7f8d2419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.333+0000 7f8d28c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d241036c0 0x7f8d241a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.334+0000 7f8d22800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d241036c0 0x7f8d241a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.334+0000 7f8d19400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d24102d20 0x7f8d2419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.335+0000 7f8d22800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d241036c0 0x7f8d241a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37904/0 (socket says 172.21.15.131:37904) 2024-05-10T05:42:40.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.335+0000 7f8d22800700 1 -- 172.21.15.131:0/422360035 learned_addr learned my addr 172.21.15.131:0/422360035 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:40.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.335+0000 7f8d21e00700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d24101c00 0x7f8d2419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.335+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d241a1c30 con 0x7f8d241036c0 2024-05-10T05:42:40.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.335+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d241a1da0 con 0x7f8d24101c00 2024-05-10T05:42:40.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.335+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d241a1f60 con 0x7f8d24102d20 2024-05-10T05:42:40.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.336+0000 7f8d22800700 1 -- 172.21.15.131:0/422360035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d24102d20 msgr2=0x7f8d2419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:40.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.336+0000 7f8d22800700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d24102d20 0x7f8d2419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.336+0000 7f8d22800700 1 -- 172.21.15.131:0/422360035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d24101c00 msgr2=0x7f8d2419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:40.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.336+0000 7f8d22800700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d24101c00 0x7f8d2419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.336+0000 7f8d22800700 1 -- 172.21.15.131:0/422360035 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c009520 con 0x7f8d241036c0 2024-05-10T05:42:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.337+0000 7f8d22800700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d241036c0 0x7f8d241a1620 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00d980 tx=0x7f8d1c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.337+0000 7f8d1b400700 1 -- 172.21.15.131:0/422360035 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c009940 con 0x7f8d241036c0 2024-05-10T05:42:40.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.337+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d241a2240 con 0x7f8d241036c0 2024-05-10T05:42:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.338+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d241a2750 con 0x7f8d241036c0 2024-05-10T05:42:40.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.338+0000 7f8d1b400700 1 -- 172.21.15.131:0/422360035 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1c004830 con 0x7f8d241036c0 2024-05-10T05:42:40.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.338+0000 7f8d1b400700 1 -- 172.21.15.131:0/422360035 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c005440 con 0x7f8d241036c0 2024-05-10T05:42:40.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.344+0000 7f8d1b400700 1 -- 172.21.15.131:0/422360035 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8d1c0055a0 con 0x7f8d241036c0 2024-05-10T05:42:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.345+0000 7f8d40c15b80 1 Processor -- start 2024-05-10T05:42:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.346+0000 7f8d1b400700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8d080a1310 0x7f8d080a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.347+0000 7f8d40c15b80 1 -- start start 2024-05-10T05:42:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.347+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 0x563d30096300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.348+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d30096d80 0x563d30097190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.348+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d300976d0 0x563d3009dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.350+0000 7f8d21e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 0x563d30096300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.350+0000 7f8d22800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d300976d0 0x563d3009dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.350+0000 7f8d22800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d300976d0 0x563d3009dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34350/0 (socket says 172.21.15.131:34350) 2024-05-10T05:42:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.350+0000 7f8d22800700 1 -- 172.21.15.131:0/3507801745 learned_addr learned my addr 172.21.15.131:0/3507801745 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.350+0000 7f8d40c15b80 1 -- 172.21.15.131:0/3507801745 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d2ffb4150 con 0x563d30096d80 2024-05-10T05:42:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.350+0000 7f8d40c15b80 1 -- 172.21.15.131:0/3507801745 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d2ffb1b10 con 0x563d30095eb0 2024-05-10T05:42:40.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.350+0000 7f8d40c15b80 1 -- 172.21.15.131:0/3507801745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d2ffad400 con 0x563d300976d0 2024-05-10T05:42:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.351+0000 7f8d22800700 1 -- 172.21.15.131:0/3507801745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 msgr2=0x563d30096300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.351+0000 7f8d22800700 1 --2- 172.21.15.131:0/3507801745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 0x563d30096300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.351+0000 7f8d22800700 1 -- 172.21.15.131:0/3507801745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d30096d80 msgr2=0x563d30097190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.351+0000 7f8d22800700 1 --2- 172.21.15.131:0/3507801745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d30096d80 0x563d30097190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.351+0000 7f8d22800700 1 -- 172.21.15.131:0/3507801745 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d2fe2e3b0 con 0x563d300976d0 2024-05-10T05:42:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.352+0000 7f8d22800700 1 --2- 172.21.15.131:0/3507801745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d300976d0 0x563d3009dfd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0d25b0 tx=0x7f8d1c0d28c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c942b9ccd260b98b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.353+0000 7f8d13000700 1 -- 172.21.15.131:0/3507801745 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0cce70 con 0x563d300976d0 2024-05-10T05:42:40.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.353+0000 7f8d13000700 1 -- 172.21.15.131:0/3507801745 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1c0ce590 con 0x563d300976d0 2024-05-10T05:42:40.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.353+0000 7f8d13000700 1 -- 172.21.15.131:0/3507801745 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0d8ac0 con 0x563d300976d0 2024-05-10T05:42:40.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.353+0000 7f8d1b400700 1 -- 172.21.15.131:0/422360035 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8d1c0cc6f0 con 0x7f8d241036c0 2024-05-10T05:42:40.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.353+0000 7f8d19400700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8d080a1310 0x7f8d080a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.355+0000 7f8d40c15b80 1 -- 172.21.15.131:0/3507801745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d300976d0 msgr2=0x563d3009dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:40.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.355+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/3507801745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d300976d0 0x563d3009dfd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0d25b0 tx=0x7f8d1c0d28c0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.356+0000 7f8d40c15b80 1 -- 172.21.15.131:0/3507801745 shutdown_connections 2024-05-10T05:42:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.360+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/3507801745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d300976d0 0x563d3009dfd0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.359+0000 7f8d19400700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8d080a1310 0x7f8d080a3820 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8d2419b1f0 tx=0x7f8d1400b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.360+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/3507801745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 0x563d30096300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.360+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/3507801745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d30096d80 0x563d30097190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.360+0000 7f8d40c15b80 1 -- 172.21.15.131:0/3507801745 >> 172.21.15.131:0/3507801745 conn(0x563d2ffefa60 msgr2=0x563d30087fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:40.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.368+0000 7f8d40c15b80 1 -- 172.21.15.131:0/3507801745 shutdown_connections 2024-05-10T05:42:40.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.368+0000 7f8d40c15b80 1 -- 172.21.15.131:0/3507801745 wait complete. 2024-05-10T05:42:40.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.369+0000 7f8d40c15b80 1 -- start start 2024-05-10T05:42:40.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.369+0000 7f8d40c15b80 1 Processor -- start 2024-05-10T05:42:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.371+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 0x563d3009ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.371+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 0x563d300a0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.371+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.374+0000 7f8d19400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 0x563d300a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.374+0000 7f8d21e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 0x563d3009ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.374+0000 7f8d19400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 0x563d300a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34356/0 (socket says 172.21.15.131:34356) 2024-05-10T05:42:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.374+0000 7f8d21e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 0x563d3009ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35856/0 (socket says 172.21.15.131:35856) 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.376+0000 7f8d40c15b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d2ff49560 con 0x563d300976d0 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.376+0000 7f8d40c15b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d2ff49200 con 0x563d30095eb0 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.376+0000 7f8d40c15b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d2fe306a0 con 0x563d30096d80 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.376+0000 7f8d40c15b80 10 client.0 _ll_register_callbacks cb 0x563d2ff52b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.378+0000 7f8d22800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.378+0000 7f8d22800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37910/0 (socket says 172.21.15.131:37910) 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.378+0000 7f8d22800700 1 -- 172.21.15.131:0/843842869 learned_addr learned my addr 172.21.15.131:0/843842869 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.379+0000 7f8d22800700 1 -- 172.21.15.131:0/843842869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 msgr2=0x563d300a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.379+0000 7f8d22800700 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 0x563d300a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.379+0000 7f8d22800700 1 -- 172.21.15.131:0/843842869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 msgr2=0x563d3009ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.379+0000 7f8d22800700 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 0x563d3009ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.379+0000 7f8d22800700 1 -- 172.21.15.131:0/843842869 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d241a2750 con 0x563d300976d0 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.379+0000 7f8d22800700 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a73d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c095fe0 tx=0x7f8d1c0cfd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f57f271507ced95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.380+0000 7f8d07400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.380+0000 7f8d07400700 1 -- 172.21.15.131:0/843842869 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0998b0 con 0x563d300976d0 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.380+0000 7f8d40c15b80 1 -- 172.21.15.131:0/843842869 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d2ff06270 con 0x563d300976d0 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.381+0000 7f8d07400700 1 -- 172.21.15.131:0/843842869 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8d1c022300 con 0x563d300976d0 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.381+0000 7f8d19400700 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 0x563d300a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.381+0000 7f8d07400700 1 -- 172.21.15.131:0/843842869 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d1c099620 con 0x563d300976d0 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.381+0000 7f8d40c15b80 10 client.16734 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.381+0000 7f8d21e00700 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 0x563d3009ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-10T05:42:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.381+0000 7f8d40c15b80 1 -- 172.21.15.131:0/843842869 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563d2fddab60 con 0x563d300976d0 2024-05-10T05:42:40.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.382+0000 7f8d07400700 1 -- 172.21.15.131:0/843842869 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d2ff06270 con 0x563d300976d0 2024-05-10T05:42:40.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.382+0000 7f8d07400700 1 -- 172.21.15.131:0/843842869 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d1c0961f0 con 0x563d300976d0 2024-05-10T05:42:40.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.382+0000 7f8d40c15b80 10 client.16734 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:40.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.382+0000 7f8d40c15b80 10 client.16734 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:42:40.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.383+0000 7f8d40c15b80 20 client.16734 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:40.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.383+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d300ab750 0x563d300adba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.383+0000 7f8d40c15b80 4 client.16734 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:40.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.383+0000 7f8d40c15b80 1 -- 172.21.15.131:0/843842869 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d3005c780 con 0x563d300ab750 2024-05-10T05:42:40.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.384+0000 7f8d21e00700 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d300ab750 0x563d300adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.385+0000 7f8d07400700 10 client.16734 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:42:40.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.386+0000 7f8d21e00700 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d300ab750 0x563d300adba0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4d3530dc899a741 server_cookie=5042f8bdf8a679d9 in_seq=0 out_seq=0 2024-05-10T05:42:40.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.390+0000 7f8d07400700 1 -- 172.21.15.131:0/843842869 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d3005c780 con 0x563d300ab750 2024-05-10T05:42:40.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.390+0000 7f8d07400700 10 client.16734 handle_command_reply: tid=0 2024-05-10T05:42:40.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.390+0000 7f8d40c15b80 1 client.16734 shutdown 2024-05-10T05:42:40.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.392+0000 7f8d40c15b80 1 -- 172.21.15.131:0/843842869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 msgr2=0x563d300a73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:40.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.392+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a73d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c095fe0 tx=0x7f8d1c0cfd80 comp rx=0 tx=0).stop 2024-05-10T05:42:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.393+0000 7f8d40c15b80 1 -- 172.21.15.131:0/843842869 shutdown_connections 2024-05-10T05:42:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.393+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d300ab750 0x563d300adba0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.393+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a73d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.393+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30095eb0 0x563d3009ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.393+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/843842869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 0x563d300a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.393+0000 7f8d40c15b80 1 -- 172.21.15.131:0/843842869 >> 172.21.15.131:0/843842869 conn(0x563d2ffefa60 msgr2=0x563d30087fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.393+0000 7f8d07400700 0 client.16734 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:42:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.397+0000 7f8d40c15b80 1 -- 172.21.15.131:0/843842869 shutdown_connections 2024-05-10T05:42:40.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.397+0000 7f8d40c15b80 1 -- 172.21.15.131:0/843842869 wait complete. 2024-05-10T05:42:40.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.404+0000 7f8d40c15b80 20 client.16734 trim_cache size 0 max 16384 2024-05-10T05:42:40.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.506+0000 7f8d40c15b80 1 Processor -- start 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.508+0000 7f8d40c15b80 1 -- start start 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.509+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d30095eb0 0x563d300a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.511+0000 7f8d21e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d30095eb0 0x563d300a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.512+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 0x563d300a7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.513+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d300976d0 0x563d30099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.514+0000 7f8d22800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d300976d0 0x563d30099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.514+0000 7f8d21e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d30095eb0 0x563d300a79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37922/0 (socket says 172.21.15.131:37922) 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.514+0000 7f8d21e00700 1 -- 172.21.15.131:0/2720108886 learned_addr learned my addr 172.21.15.131:0/2720108886 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.514+0000 7f8d40c15b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d300827c0 con 0x563d30095eb0 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.514+0000 7f8d40c15b80 1 -- 172.21.15.131:0/2720108886 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d300ab4a0 con 0x563d300976d0 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.514+0000 7f8d40c15b80 1 -- 172.21.15.131:0/2720108886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3009a4f0 con 0x563d30096d80 2024-05-10T05:42:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.514+0000 7f8d19400700 1 --2- 172.21.15.131:0/2720108886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 0x563d300a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.517+0000 7f8d22800700 1 -- 172.21.15.131:0/2720108886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 msgr2=0x563d300a7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.517+0000 7f8d22800700 1 --2- 172.21.15.131:0/2720108886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 0x563d300a7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.517+0000 7f8d22800700 1 -- 172.21.15.131:0/2720108886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d30095eb0 msgr2=0x563d300a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.517+0000 7f8d22800700 1 --2- 172.21.15.131:0/2720108886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d30095eb0 0x563d300a79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.517+0000 7f8d22800700 1 -- 172.21.15.131:0/2720108886 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d2ff73560 con 0x563d300976d0 2024-05-10T05:42:40.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.518+0000 7f8d22800700 1 --2- 172.21.15.131:0/2720108886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d300976d0 0x563d30099fb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0d2fd0 tx=0x7f8d1c0a20c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3358342afa6c454 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:40.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.518+0000 7f8d13a00700 1 -- 172.21.15.131:0/2720108886 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0cfba0 con 0x563d300976d0 2024-05-10T05:42:40.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.518+0000 7f8d13a00700 1 -- 172.21.15.131:0/2720108886 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1c0d20e0 con 0x563d300976d0 2024-05-10T05:42:40.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.518+0000 7f8d13a00700 1 -- 172.21.15.131:0/2720108886 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c099ae0 con 0x563d300976d0 2024-05-10T05:42:40.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.520+0000 7f8d40c15b80 1 -- 172.21.15.131:0/2720108886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d300976d0 msgr2=0x563d30099fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:40.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.520+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/2720108886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d300976d0 0x563d30099fb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0d2fd0 tx=0x7f8d1c0a20c0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.528+0000 7f8d40c15b80 1 -- 172.21.15.131:0/2720108886 shutdown_connections 2024-05-10T05:42:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.528+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/2720108886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d30095eb0 0x563d300a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.528+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/2720108886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d300976d0 0x563d30099fb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.528+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/2720108886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30096d80 0x563d300a7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.528+0000 7f8d40c15b80 1 -- 172.21.15.131:0/2720108886 >> 172.21.15.131:0/2720108886 conn(0x563d2ffefa60 msgr2=0x563d30087fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.529+0000 7f8d40c15b80 1 -- 172.21.15.131:0/2720108886 shutdown_connections 2024-05-10T05:42:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.530+0000 7f8d40c15b80 1 -- 172.21.15.131:0/2720108886 wait complete. 2024-05-10T05:42:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.530+0000 7f8d40c15b80 1 -- start start 2024-05-10T05:42:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.536+0000 7f8d40c15b80 1 Processor -- start 2024-05-10T05:42:40.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.543+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30095eb0 0x563d300a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.544+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30096d80 0x563d300a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.546+0000 7f8d21e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30095eb0 0x563d300a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.546+0000 7f8d40c15b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.547+0000 7f8d21e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30095eb0 0x563d300a5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34386/0 (socket says 172.21.15.131:34386) 2024-05-10T05:42:40.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.548+0000 7f8d19400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30096d80 0x563d300a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.548+0000 7f8d22800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.549+0000 7f8d19400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30096d80 0x563d300a61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35880/0 (socket says 172.21.15.131:35880) 2024-05-10T05:42:40.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.549+0000 7f8d21e00700 1 -- 172.21.15.131:0/4031950100 learned_addr learned my addr 172.21.15.131:0/4031950100 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:40.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.550+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d300922e0 con 0x563d300976d0 2024-05-10T05:42:40.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.550+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d30092450 con 0x563d30096d80 2024-05-10T05:42:40.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.550+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d30092610 con 0x563d30095eb0 2024-05-10T05:42:40.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.550+0000 7f8d40c15b80 10 client.0 _ll_register_callbacks cb 0x563d2ff52b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:40.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.552+0000 7f8d22800700 1 -- 172.21.15.131:0/4031950100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30095eb0 msgr2=0x563d300a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:40.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.552+0000 7f8d22800700 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30095eb0 0x563d300a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.552+0000 7f8d22800700 1 -- 172.21.15.131:0/4031950100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30096d80 msgr2=0x563d300a61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:40.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.552+0000 7f8d22800700 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30096d80 0x563d300a61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.553+0000 7f8d22800700 1 -- 172.21.15.131:0/4031950100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d2fddab60 con 0x563d300976d0 2024-05-10T05:42:40.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.556+0000 7f8d22800700 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a6700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c096550 tx=0x7f8d1c0a1ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475500999de208a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:40.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.556+0000 7f8d07400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:42:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.559+0000 7f8d07400700 1 -- 172.21.15.131:0/4031950100 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0cfba0 con 0x563d300976d0 2024-05-10T05:42:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.559+0000 7f8d07400700 1 -- 172.21.15.131:0/4031950100 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8d1c0b1030 con 0x563d300976d0 2024-05-10T05:42:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.559+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d1c0ce590 con 0x563d300976d0 2024-05-10T05:42:40.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.563+0000 7f8d07400700 1 -- 172.21.15.131:0/4031950100 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d1c0998b0 con 0x563d300976d0 2024-05-10T05:42:40.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.563+0000 7f8d40c15b80 10 client.16740 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:40.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.563+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563d2fe32cd0 con 0x563d300976d0 2024-05-10T05:42:40.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.564+0000 7f8d07400700 1 -- 172.21.15.131:0/4031950100 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1c07c020 con 0x563d300976d0 2024-05-10T05:42:40.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.564+0000 7f8d07400700 1 -- 172.21.15.131:0/4031950100 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d1c0b2290 con 0x563d300976d0 2024-05-10T05:42:40.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.565+0000 7f8d40c15b80 10 client.16740 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:40.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.565+0000 7f8d40c15b80 10 client.16740 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:42:40.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.565+0000 7f8d40c15b80 20 client.16740 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:40.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.565+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d30093a50 0x563d30093e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:40.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.565+0000 7f8d40c15b80 4 client.16740 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:42:40.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.565+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d2ffa5ec0 con 0x563d30093a50 2024-05-10T05:42:40.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.567+0000 7f8d21e00700 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d30093a50 0x563d30093e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:40.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.570+0000 7f8d07400700 10 client.16740 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:42:40.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.570+0000 7f8d21e00700 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d30093a50 0x563d30093e60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=226e088a422dbc81 server_cookie=695652579f184816 in_seq=0 out_seq=0 2024-05-10T05:42:40.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.572+0000 7f8d07400700 1 -- 172.21.15.131:0/4031950100 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563d2ffa5ec0 con 0x563d30093a50 2024-05-10T05:42:40.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.572+0000 7f8d07400700 10 client.16740 handle_command_reply: tid=0 2024-05-10T05:42:40.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.572+0000 7f8d40c15b80 1 client.16740 shutdown 2024-05-10T05:42:40.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.574+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 msgr2=0x563d300a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:40.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.574+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a6700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c096550 tx=0x7f8d1c0a1ce0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.574+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 shutdown_connections 2024-05-10T05:42:40.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.574+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d30093a50 0x563d30093e60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.575+0000 7f8d07400700 0 client.16740 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:42:40.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.575+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d300976d0 0x563d300a6700 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.575+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d30095eb0 0x563d300a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.575+0000 7f8d40c15b80 1 --2- 172.21.15.131:0/4031950100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d30096d80 0x563d300a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.575+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 >> 172.21.15.131:0/4031950100 conn(0x563d2ffefa60 msgr2=0x563d3009f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:40.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.575+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 shutdown_connections 2024-05-10T05:42:40.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.576+0000 7f8d40c15b80 1 -- 172.21.15.131:0/4031950100 wait complete. 2024-05-10T05:42:40.577 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:42:40.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.576+0000 7f8d40c15b80 20 client.16740 trim_cache size 0 max 16384 2024-05-10T05:42:40.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.584+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8d080a1310 msgr2=0x7f8d080a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:40.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.584+0000 7f8d28c00700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8d080a1310 0x7f8d080a3820 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8d2419b1f0 tx=0x7f8d1400b040 comp rx=0 tx=0).stop 2024-05-10T05:42:40.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.585+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d241036c0 msgr2=0x7f8d241a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:40.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.585+0000 7f8d28c00700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d241036c0 0x7f8d241a1620 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00d980 tx=0x7f8d1c00dd40 comp rx=0 tx=0).stop 2024-05-10T05:42:40.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.586+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 shutdown_connections 2024-05-10T05:42:40.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.586+0000 7f8d28c00700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8d080a1310 0x7f8d080a3820 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.586+0000 7f8d28c00700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d241036c0 0x7f8d241a1620 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.586+0000 7f8d28c00700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d24101c00 0x7f8d2419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.586+0000 7f8d28c00700 1 --2- 172.21.15.131:0/422360035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d24102d20 0x7f8d2419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:40.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.586+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 >> 172.21.15.131:0/422360035 conn(0x7f8d240fd210 msgr2=0x7f8d240fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:40.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.587+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 shutdown_connections 2024-05-10T05:42:40.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:40.587+0000 7f8d28c00700 1 -- 172.21.15.131:0/422360035 wait complete. 2024-05-10T05:42:40.655 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:42:40.798 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:40 smithi131 ceph-mon[37264]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 83 op/s 2024-05-10T05:42:40.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:40 smithi157 ceph-mon[78427]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 83 op/s 2024-05-10T05:42:40.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:40 smithi142 ceph-mon[77667]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 83 op/s 2024-05-10T05:42:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.323+0000 7f1317e00700 1 -- 172.21.15.131:0/1474392817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1318108210 msgr2=0x7f1318108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.323+0000 7f1317e00700 1 --2- 172.21.15.131:0/1474392817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1318108210 0x7f1318108620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f13080098f0 tx=0x7f1308009c00 comp rx=0 tx=0).stop 2024-05-10T05:42:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.324+0000 7f1317e00700 1 -- 172.21.15.131:0/1474392817 shutdown_connections 2024-05-10T05:42:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.324+0000 7f1317e00700 1 --2- 172.21.15.131:0/1474392817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1318109500 0x7f1318109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.324+0000 7f1317e00700 1 --2- 172.21.15.131:0/1474392817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1318108b60 0x7f131810e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.324+0000 7f1317e00700 1 --2- 172.21.15.131:0/1474392817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1318108210 0x7f1318108620 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.324+0000 7f1317e00700 1 -- 172.21.15.131:0/1474392817 >> 172.21.15.131:0/1474392817 conn(0x7f131806e800 msgr2=0x7f1318070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:41.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.327+0000 7f1317e00700 1 -- 172.21.15.131:0/1474392817 shutdown_connections 2024-05-10T05:42:41.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.328+0000 7f1317e00700 1 -- 172.21.15.131:0/1474392817 wait complete. 2024-05-10T05:42:41.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.336+0000 7f1317e00700 1 Processor -- start 2024-05-10T05:42:41.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.340+0000 7f1317e00700 1 -- start start 2024-05-10T05:42:41.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.344+0000 7f1317e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1318108210 0x7f1318118fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.346+0000 7f1316a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1318108210 0x7f1318118fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.346+0000 7f1316a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1318108210 0x7f1318118fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35890/0 (socket says 172.21.15.131:35890) 2024-05-10T05:42:41.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.346+0000 7f1316a00700 1 -- 172.21.15.131:0/1105298343 learned_addr learned my addr 172.21.15.131:0/1105298343 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:41.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.347+0000 7f1317e00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1318108b60 0x7f1318119510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.347+0000 7f1317e00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1318109500 0x7f13181b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.347+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318119cd0 con 0x7f1318109500 2024-05-10T05:42:41.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.347+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318119e40 con 0x7f1318108210 2024-05-10T05:42:41.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.347+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13181b72b0 con 0x7f1318108b60 2024-05-10T05:42:41.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.348+0000 7f1316000700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1318108b60 0x7f1318119510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.349+0000 7f1316a00700 1 -- 172.21.15.131:0/1105298343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1318108b60 msgr2=0x7f1318119510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:41.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.350+0000 7f1316a00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1318108b60 0x7f1318119510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.350+0000 7f1316a00700 1 -- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1318109500 msgr2=0x7f13181b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:41.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.350+0000 7f1317400700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1318109500 0x7f13181b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.351+0000 7f1316a00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1318109500 0x7f13181b6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.352+0000 7f1316a00700 1 -- 172.21.15.131:0/1105298343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308009520 con 0x7f1318108210 2024-05-10T05:42:41.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.353+0000 7f1316000700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1318108b60 0x7f1318119510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:41.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.353+0000 7f1316a00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1318108210 0x7f1318118fd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f130400b500 tx=0x7f130400b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.354+0000 7f131ca00700 1 -- 172.21.15.131:0/1105298343 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304011830 con 0x7f1318108210 2024-05-10T05:42:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.354+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13181b7530 con 0x7f1318108210 2024-05-10T05:42:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.354+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13181b7a90 con 0x7f1318108210 2024-05-10T05:42:41.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.355+0000 7f131ca00700 1 -- 172.21.15.131:0/1105298343 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1304011e50 con 0x7f1318108210 2024-05-10T05:42:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.356+0000 7f131ca00700 1 -- 172.21.15.131:0/1105298343 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304011830 con 0x7f1318108210 2024-05-10T05:42:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.362+0000 7f131ca00700 1 -- 172.21.15.131:0/1105298343 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1304010380 con 0x7f1318108210 2024-05-10T05:42:41.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.363+0000 7f1335c56b80 1 Processor -- start 2024-05-10T05:42:41.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.364+0000 7f1335c56b80 1 -- start start 2024-05-10T05:42:41.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.364+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 0x5568e7803960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.364+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78047a0 0x5568e7804bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.364+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 0x5568e780b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.364+0000 7f1335c56b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e772a150 con 0x5568e7803530 2024-05-10T05:42:41.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.364+0000 7f1335c56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e7727b10 con 0x5568e78050f0 2024-05-10T05:42:41.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.364+0000 7f1335c56b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e7723400 con 0x5568e78047a0 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.365+0000 7f1316000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 0x5568e780b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.365+0000 7f1316000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 0x5568e780b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35902/0 (socket says 172.21.15.131:35902) 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.365+0000 7f1316000700 1 -- 172.21.15.131:0/358087147 learned_addr learned my addr 172.21.15.131:0/358087147 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.366+0000 7f1316000700 1 -- 172.21.15.131:0/358087147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78047a0 msgr2=0x5568e7804bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.366+0000 7f1316000700 1 --2- 172.21.15.131:0/358087147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78047a0 0x5568e7804bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.366+0000 7f1316000700 1 -- 172.21.15.131:0/358087147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 msgr2=0x5568e7803960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.366+0000 7f131ca00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12fc0a11d0 0x7f12fc0a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.367+0000 7f131ca00700 1 -- 172.21.15.131:0/1105298343 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f13040cb5c0 con 0x7f1318108210 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.368+0000 7f1317400700 1 --2- 172.21.15.131:0/358087147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 0x5568e7803960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.368+0000 7f1316000700 1 --2- 172.21.15.131:0/358087147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 0x5568e7803960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.368+0000 7f1316000700 1 -- 172.21.15.131:0/358087147 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568e75a43b0 con 0x5568e78050f0 2024-05-10T05:42:41.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.369+0000 7f1316a00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12fc0a11d0 0x7f12fc0a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.369+0000 7f1316000700 1 --2- 172.21.15.131:0/358087147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 0x5568e780b9f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f130800fc00 tx=0x7f130800fad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dead8be8acc62141 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:41.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.370+0000 7f1303e00700 1 -- 172.21.15.131:0/358087147 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130801c050 con 0x5568e78050f0 2024-05-10T05:42:41.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.370+0000 7f1303e00700 1 -- 172.21.15.131:0/358087147 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f130801d040 con 0x5568e78050f0 2024-05-10T05:42:41.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.370+0000 7f1303e00700 1 -- 172.21.15.131:0/358087147 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308005670 con 0x5568e78050f0 2024-05-10T05:42:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.372+0000 7f1335c56b80 1 -- 172.21.15.131:0/358087147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 msgr2=0x5568e780b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.372+0000 7f1335c56b80 1 --2- 172.21.15.131:0/358087147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 0x5568e780b9f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f130800fc00 tx=0x7f130800fad0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.372+0000 7f1316a00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12fc0a11d0 0x7f12fc0a36e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1304095510 tx=0x7f1304094e70 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.373+0000 7f1335c56b80 1 -- 172.21.15.131:0/358087147 shutdown_connections 2024-05-10T05:42:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.373+0000 7f1335c56b80 1 --2- 172.21.15.131:0/358087147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 0x5568e7803960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.373+0000 7f1335c56b80 1 --2- 172.21.15.131:0/358087147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 0x5568e780b9f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.373+0000 7f1335c56b80 1 --2- 172.21.15.131:0/358087147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78047a0 0x5568e7804bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.373+0000 7f1335c56b80 1 -- 172.21.15.131:0/358087147 >> 172.21.15.131:0/358087147 conn(0x5568e7765a60 msgr2=0x5568e77fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.373+0000 7f1335c56b80 1 -- 172.21.15.131:0/358087147 shutdown_connections 2024-05-10T05:42:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.374+0000 7f1335c56b80 1 -- 172.21.15.131:0/358087147 wait complete. 2024-05-10T05:42:41.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.375+0000 7f1335c56b80 1 -- start start 2024-05-10T05:42:41.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.375+0000 7f1335c56b80 1 Processor -- start 2024-05-10T05:42:41.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.376+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 0x5568e780d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.377+0000 7f1317400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 0x5568e780d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.377+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78047a0 0x5568e780de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78050f0 0x5568e7814da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1335c56b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e76bf560 con 0x5568e7803530 2024-05-10T05:42:41.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1335c56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e76bf200 con 0x5568e78047a0 2024-05-10T05:42:41.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1335c56b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e75a66a0 con 0x5568e78050f0 2024-05-10T05:42:41.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1335c56b80 10 client.0 _ll_register_callbacks cb 0x5568e76c8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:41.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1316000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78047a0 0x5568e780de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1316000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78047a0 0x5568e780de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35906/0 (socket says 172.21.15.131:35906) 2024-05-10T05:42:41.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1316000700 1 -- 172.21.15.131:0/654571521 learned_addr learned my addr 172.21.15.131:0/654571521 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:41.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1316000700 1 -- 172.21.15.131:0/654571521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78050f0 msgr2=0x5568e7814da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:41.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1316000700 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78050f0 0x5568e7814da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.378+0000 7f1316000700 1 -- 172.21.15.131:0/654571521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 msgr2=0x5568e780d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:41.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.380+0000 7f1316000700 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 0x5568e780d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.380+0000 7f1316000700 1 -- 172.21.15.131:0/654571521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131810eeb0 con 0x5568e78047a0 2024-05-10T05:42:41.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.380+0000 7f1317400700 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 0x5568e780d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.401+0000 7f1316000700 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78047a0 0x5568e780de40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1308009810 tx=0x7f130800bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8113e145fc08402e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.401+0000 7f1300c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.402+0000 7f1300c00700 1 -- 172.21.15.131:0/654571521 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130801c070 con 0x5568e78047a0 2024-05-10T05:42:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.402+0000 7f1300c00700 1 -- 172.21.15.131:0/654571521 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f130800bb60 con 0x5568e78047a0 2024-05-10T05:42:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.405+0000 7f1335c56b80 1 -- 172.21.15.131:0/654571521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568e767c270 con 0x5568e78047a0 2024-05-10T05:42:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.405+0000 7f1300c00700 1 -- 172.21.15.131:0/654571521 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1308003680 con 0x5568e78047a0 2024-05-10T05:42:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.406+0000 7f1335c56b80 10 client.27044 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.406+0000 7f1335c56b80 1 -- 172.21.15.131:0/654571521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5568e7550b60 con 0x5568e78047a0 2024-05-10T05:42:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.406+0000 7f1300c00700 1 -- 172.21.15.131:0/654571521 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568e767c270 con 0x5568e78047a0 2024-05-10T05:42:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.406+0000 7f1300c00700 1 -- 172.21.15.131:0/654571521 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1308009c70 con 0x5568e78047a0 2024-05-10T05:42:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.406+0000 7f1335c56b80 10 client.27044 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:41.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.406+0000 7f1335c56b80 10 client.27044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:42:41.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.407+0000 7f1335c56b80 20 client.27044 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:41.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.407+0000 7f1335c56b80 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5568e7819170 0x5568e781b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.407+0000 7f1335c56b80 4 client.27044 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:41.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.407+0000 7f1335c56b80 1 -- 172.21.15.131:0/654571521 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568e77d2780 con 0x5568e7819170 2024-05-10T05:42:41.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.408+0000 7f1317400700 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5568e7819170 0x5568e781b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.410+0000 7f1300c00700 10 client.27044 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:42:41.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.410+0000 7f1317400700 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5568e7819170 0x5568e781b5c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1faeb0668cc414ed server_cookie=7b386c99fde3f927 in_seq=0 out_seq=0 2024-05-10T05:42:41.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.414+0000 7f1300c00700 1 -- 172.21.15.131:0/654571521 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5568e77d2780 con 0x5568e7819170 2024-05-10T05:42:41.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.414+0000 7f1300c00700 10 client.27044 handle_command_reply: tid=0 2024-05-10T05:42:41.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.414+0000 7f1335c56b80 1 client.27044 shutdown 2024-05-10T05:42:41.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.415+0000 7f1335c56b80 1 -- 172.21.15.131:0/654571521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78047a0 msgr2=0x5568e780de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:41.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.415+0000 7f1335c56b80 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78047a0 0x5568e780de40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1308009810 tx=0x7f130800bf70 comp rx=0 tx=0).stop 2024-05-10T05:42:41.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.415+0000 7f1335c56b80 1 -- 172.21.15.131:0/654571521 shutdown_connections 2024-05-10T05:42:41.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.415+0000 7f1335c56b80 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5568e7819170 0x5568e781b5c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.415+0000 7f1335c56b80 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e7803530 0x5568e780d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.415+0000 7f1335c56b80 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78050f0 0x5568e7814da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.415+0000 7f1335c56b80 1 --2- 172.21.15.131:0/654571521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78047a0 0x5568e780de40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.415+0000 7f1335c56b80 1 -- 172.21.15.131:0/654571521 >> 172.21.15.131:0/654571521 conn(0x5568e7765a60 msgr2=0x5568e7802060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:41.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.415+0000 7f1300c00700 0 client.27044 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:42:41.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.416+0000 7f1335c56b80 1 -- 172.21.15.131:0/654571521 shutdown_connections 2024-05-10T05:42:41.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.416+0000 7f1335c56b80 1 -- 172.21.15.131:0/654571521 wait complete. 2024-05-10T05:42:41.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.417+0000 7f1335c56b80 20 client.27044 trim_cache size 0 max 16384 2024-05-10T05:42:41.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.449+0000 7f1335c56b80 1 Processor -- start 2024-05-10T05:42:41.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.450+0000 7f1335c56b80 1 -- start start 2024-05-10T05:42:41.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.450+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e7803530 0x5568e7804bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.451+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e78129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.452+0000 7f1316000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e78129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.453+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 0x5568e7812f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.453+0000 7f1335c56b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e780b930 con 0x5568e78047a0 2024-05-10T05:42:41.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.453+0000 7f1316000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e78129c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37964/0 (socket says 172.21.15.131:37964) 2024-05-10T05:42:41.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.453+0000 7f1316000700 1 -- 172.21.15.131:0/2677635831 learned_addr learned my addr 172.21.15.131:0/2677635831 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:41.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.453+0000 7f1335c56b80 1 -- 172.21.15.131:0/2677635831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e780bcb0 con 0x5568e78050f0 2024-05-10T05:42:41.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.453+0000 7f1335c56b80 1 -- 172.21.15.131:0/2677635831 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e77ff350 con 0x5568e7803530 2024-05-10T05:42:41.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.454+0000 7f1316000700 1 -- 172.21.15.131:0/2677635831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e7803530 msgr2=0x5568e7804bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:42:41.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.454+0000 7f1316000700 1 --2- 172.21.15.131:0/2677635831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e7803530 0x5568e7804bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.454+0000 7f1316000700 1 -- 172.21.15.131:0/2677635831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 msgr2=0x5568e7812f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:41.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.454+0000 7f1316000700 1 --2- 172.21.15.131:0/2677635831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 0x5568e7812f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.454+0000 7f1316000700 1 -- 172.21.15.131:0/2677635831 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568e76e9560 con 0x5568e78047a0 2024-05-10T05:42:41.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.455+0000 7f1316000700 1 --2- 172.21.15.131:0/2677635831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e78129c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x5568e780e970 tx=0x7f1308036a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35cdb3add0a5bd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:41.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.456+0000 7f130d200700 1 -- 172.21.15.131:0/2677635831 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130801c070 con 0x5568e78047a0 2024-05-10T05:42:41.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.456+0000 7f130d200700 1 -- 172.21.15.131:0/2677635831 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1308036bc0 con 0x5568e78047a0 2024-05-10T05:42:41.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.458+0000 7f1335c56b80 1 -- 172.21.15.131:0/2677635831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 msgr2=0x5568e78129c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:41.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.458+0000 7f130d200700 1 -- 172.21.15.131:0/2677635831 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130801c070 con 0x5568e78047a0 2024-05-10T05:42:41.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.458+0000 7f1335c56b80 1 --2- 172.21.15.131:0/2677635831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e78129c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x5568e780e970 tx=0x7f1308036a70 comp rx=0 tx=0).stop 2024-05-10T05:42:41.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.459+0000 7f1335c56b80 1 -- 172.21.15.131:0/2677635831 shutdown_connections 2024-05-10T05:42:41.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.460+0000 7f1335c56b80 1 --2- 172.21.15.131:0/2677635831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e78050f0 0x5568e7812f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.460+0000 7f1335c56b80 1 --2- 172.21.15.131:0/2677635831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e7803530 0x5568e7804bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.460+0000 7f1335c56b80 1 --2- 172.21.15.131:0/2677635831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e78129c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.460+0000 7f1335c56b80 1 -- 172.21.15.131:0/2677635831 >> 172.21.15.131:0/2677635831 conn(0x5568e7765a60 msgr2=0x5568e780d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:41.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.460+0000 7f1335c56b80 1 -- 172.21.15.131:0/2677635831 shutdown_connections 2024-05-10T05:42:41.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.460+0000 7f1335c56b80 1 -- 172.21.15.131:0/2677635831 wait complete. 2024-05-10T05:42:41.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.461+0000 7f1335c56b80 1 -- start start 2024-05-10T05:42:41.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.461+0000 7f1335c56b80 1 Processor -- start 2024-05-10T05:42:41.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.463+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e7803530 0x5568e7812220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.463+0000 7f1335c56b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e7812760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.463+0000 7f1317400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e7803530 0x5568e7812220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.463+0000 7f1317400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e7803530 0x5568e7812220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35908/0 (socket says 172.21.15.131:35908) 2024-05-10T05:42:41.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.464+0000 7f1316000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e7812760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.464+0000 7f1316000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e7812760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37968/0 (socket says 172.21.15.131:37968) 2024-05-10T05:42:41.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.464+0000 7f1317400700 1 -- 172.21.15.131:0/1152993795 learned_addr learned my addr 172.21.15.131:0/1152993795 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:41.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.464+0000 7f1335c56b80 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78050f0 0x5568e786d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.465+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e786dd90 con 0x5568e78047a0 2024-05-10T05:42:41.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.465+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e786ded0 con 0x5568e7803530 2024-05-10T05:42:41.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.465+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e786e090 con 0x5568e78050f0 2024-05-10T05:42:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.466+0000 7f1316000700 1 -- 172.21.15.131:0/1152993795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78050f0 msgr2=0x5568e786d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.466+0000 7f1316000700 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78050f0 0x5568e786d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.466+0000 7f1316000700 1 -- 172.21.15.131:0/1152993795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e7803530 msgr2=0x5568e7812220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.466+0000 7f1316000700 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e7803530 0x5568e7812220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.466+0000 7f1316000700 1 -- 172.21.15.131:0/1152993795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568e7550b60 con 0x5568e78047a0 2024-05-10T05:42:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.467+0000 7f1316000700 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e7812760 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1308009810 tx=0x7f1308036700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0c8fa87526232c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.467+0000 7f1300c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:42:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.468+0000 7f1335c56b80 10 client.0 _ll_register_callbacks cb 0x5568e76c8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:41.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.468+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f130801d040 con 0x5568e78047a0 2024-05-10T05:42:41.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.469+0000 7f1300c00700 1 -- 172.21.15.131:0/1152993795 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130801c040 con 0x5568e78047a0 2024-05-10T05:42:41.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.469+0000 7f1300c00700 1 -- 172.21.15.131:0/1152993795 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1308000e40 con 0x5568e78047a0 2024-05-10T05:42:41.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.469+0000 7f1300c00700 1 -- 172.21.15.131:0/1152993795 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f130801c040 con 0x5568e78047a0 2024-05-10T05:42:41.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.470+0000 7f1335c56b80 10 client.16746 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:41.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.470+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x5568e75a8cd0 con 0x5568e78047a0 2024-05-10T05:42:41.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.471+0000 7f1300c00700 1 -- 172.21.15.131:0/1152993795 <== mon.0 v2:172.21.15.131:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1308046070 con 0x5568e78047a0 2024-05-10T05:42:41.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.472+0000 7f1335c56b80 10 client.16746 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:41.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.472+0000 7f1335c56b80 10 client.16746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:42:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.472+0000 7f1335c56b80 20 client.16746 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.472+0000 7f1335c56b80 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5568e786f310 0x5568e7871820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:41.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.472+0000 7f1335c56b80 4 client.16746 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:42:41.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.472+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5568e771bec0 con 0x5568e786f310 2024-05-10T05:42:41.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.474+0000 7f1317400700 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5568e786f310 0x5568e7871820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.475+0000 7f1317400700 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5568e786f310 0x5568e7871820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e0edcc80b472e6a server_cookie=ccc7d927f5e7a01b in_seq=0 out_seq=0 2024-05-10T05:42:41.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.478+0000 7f1300c00700 10 client.16746 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:42:41.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.484+0000 7f1300c00700 1 -- 172.21.15.131:0/1152993795 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5568e771bec0 con 0x5568e786f310 2024-05-10T05:42:41.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.484+0000 7f1300c00700 10 client.16746 handle_command_reply: tid=0 2024-05-10T05:42:41.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.485+0000 7f1335c56b80 1 client.16746 shutdown 2024-05-10T05:42:41.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.487+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 msgr2=0x5568e7812760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:41.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.487+0000 7f1335c56b80 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e7812760 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1308009810 tx=0x7f1308036700 comp rx=0 tx=0).stop 2024-05-10T05:42:41.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.487+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 shutdown_connections 2024-05-10T05:42:41.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.487+0000 7f1335c56b80 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x5568e786f310 0x5568e7871820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.488+0000 7f1335c56b80 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e78050f0 0x5568e786d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.488+0000 7f1335c56b80 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5568e7803530 0x5568e7812220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.488+0000 7f1335c56b80 1 --2- 172.21.15.131:0/1152993795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e78047a0 0x5568e7812760 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.488+0000 7f1300c00700 0 client.16746 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:42:41.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.488+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 >> 172.21.15.131:0/1152993795 conn(0x5568e7765a60 msgr2=0x5568e77fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:41.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.488+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 shutdown_connections 2024-05-10T05:42:41.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.488+0000 7f1335c56b80 1 -- 172.21.15.131:0/1152993795 wait complete. 2024-05-10T05:42:41.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.490+0000 7f1335c56b80 20 client.16746 trim_cache size 0 max 16384 2024-05-10T05:42:41.492 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:42:41.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.491+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12fc0a11d0 msgr2=0x7f12fc0a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:41.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.491+0000 7f1317e00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12fc0a11d0 0x7f12fc0a36e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1304095510 tx=0x7f1304094e70 comp rx=0 tx=0).stop 2024-05-10T05:42:41.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.492+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1318108210 msgr2=0x7f1318118fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:41.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.492+0000 7f1317e00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1318108210 0x7f1318118fd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f130400b500 tx=0x7f130400b810 comp rx=0 tx=0).stop 2024-05-10T05:42:41.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.493+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 shutdown_connections 2024-05-10T05:42:41.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.493+0000 7f1317e00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f12fc0a11d0 0x7f12fc0a36e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.493+0000 7f1317e00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1318109500 0x7f13181b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.493+0000 7f1317e00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1318108210 0x7f1318118fd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.493+0000 7f1317e00700 1 --2- 172.21.15.131:0/1105298343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1318108b60 0x7f1318119510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:41.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.493+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 >> 172.21.15.131:0/1105298343 conn(0x7f131806e800 msgr2=0x7f131810c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:41.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.493+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 shutdown_connections 2024-05-10T05:42:41.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:41.493+0000 7f1317e00700 1 -- 172.21.15.131:0/1105298343 wait complete. 2024-05-10T05:42:41.558 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:42:42.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.269+0000 7fee1ee00700 1 -- 172.21.15.131:0/3175073240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee18109730 msgr2=0x7fee18108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:42.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.269+0000 7fee1ee00700 1 --2- 172.21.15.131:0/3175073240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee18109730 0x7fee18108210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009990 tx=0x7fee0c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.269+0000 7fee1ee00700 1 -- 172.21.15.131:0/3175073240 shutdown_connections 2024-05-10T05:42:42.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.269+0000 7fee1ee00700 1 --2- 172.21.15.131:0/3175073240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1810fec0 0x7fee18112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.269+0000 7fee1ee00700 1 --2- 172.21.15.131:0/3175073240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee18109730 0x7fee18108210 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.269+0000 7fee1ee00700 1 --2- 172.21.15.131:0/3175073240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee181087e0 0x7fee18108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.269+0000 7fee1ee00700 1 -- 172.21.15.131:0/3175073240 >> 172.21.15.131:0/3175073240 conn(0x7fee1806e800 msgr2=0x7fee18070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:42.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.272+0000 7fee1ee00700 1 -- 172.21.15.131:0/3175073240 shutdown_connections 2024-05-10T05:42:42.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.272+0000 7fee1ee00700 1 -- 172.21.15.131:0/3175073240 wait complete. 2024-05-10T05:42:42.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.281+0000 7fee1ee00700 1 Processor -- start 2024-05-10T05:42:42.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.281+0000 7fee1ee00700 1 -- start start 2024-05-10T05:42:42.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.282+0000 7fee1ee00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee181087e0 0x7fee1810eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.282+0000 7fee1ee00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee18109730 0x7fee1810f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.282+0000 7fee1ee00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee1810fec0 0x7fee181ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.282+0000 7fee1ee00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee181ae5a0 con 0x7fee1810fec0 2024-05-10T05:42:42.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.282+0000 7fee1ee00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee181ae6e0 con 0x7fee181087e0 2024-05-10T05:42:42.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.282+0000 7fee1ee00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee181ae870 con 0x7fee18109730 2024-05-10T05:42:42.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.283+0000 7fee1e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee1810fec0 0x7fee181ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.284+0000 7fee1da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee181087e0 0x7fee1810eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.284+0000 7fee1da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee181087e0 0x7fee1810eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35924/0 (socket says 172.21.15.131:35924) 2024-05-10T05:42:42.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.284+0000 7fee1da00700 1 -- 172.21.15.131:0/507261619 learned_addr learned my addr 172.21.15.131:0/507261619 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:42.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.284+0000 7fee1d000700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee18109730 0x7fee1810f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.285+0000 7fee1da00700 1 -- 172.21.15.131:0/507261619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee18109730 msgr2=0x7fee1810f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:42.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.285+0000 7fee1da00700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee18109730 0x7fee1810f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.285+0000 7fee1da00700 1 -- 172.21.15.131:0/507261619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee1810fec0 msgr2=0x7fee181ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.285+0000 7fee1da00700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee1810fec0 0x7fee181ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.285+0000 7fee1da00700 1 -- 172.21.15.131:0/507261619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c0095f0 con 0x7fee181087e0 2024-05-10T05:42:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.286+0000 7fee1e400700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee1810fec0 0x7fee181ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.286+0000 7fee1da00700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee181087e0 0x7fee1810eaf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fee0c005950 tx=0x7fee0c00f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.287+0000 7fee16000700 1 -- 172.21.15.131:0/507261619 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c01d070 con 0x7fee181087e0 2024-05-10T05:42:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.287+0000 7fee16000700 1 -- 172.21.15.131:0/507261619 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee0c004ca0 con 0x7fee181087e0 2024-05-10T05:42:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.287+0000 7fee16000700 1 -- 172.21.15.131:0/507261619 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c005450 con 0x7fee181087e0 2024-05-10T05:42:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.288+0000 7fee1ee00700 1 -- 172.21.15.131:0/507261619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee181aeaf0 con 0x7fee181087e0 2024-05-10T05:42:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.289+0000 7fee1ee00700 1 -- 172.21.15.131:0/507261619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee181aeec0 con 0x7fee181087e0 2024-05-10T05:42:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.296+0000 7fee3824eb80 1 Processor -- start 2024-05-10T05:42:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.297+0000 7fee3824eb80 1 -- start start 2024-05-10T05:42:42.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.300+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a3b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.300+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a3cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.300+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.300+0000 7fee3824eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2962150 con 0x5648a2a3b530 2024-05-10T05:42:42.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.300+0000 7fee3824eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a295fb10 con 0x5648a2a3c7a0 2024-05-10T05:42:42.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.300+0000 7fee3824eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a295b400 con 0x5648a2a3d0f0 2024-05-10T05:42:42.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.301+0000 7fee1d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.301+0000 7fee1d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34468/0 (socket says 172.21.15.131:34468) 2024-05-10T05:42:42.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.301+0000 7fee1d000700 1 -- 172.21.15.131:0/2793182929 learned_addr learned my addr 172.21.15.131:0/2793182929 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:42.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.301+0000 7fee1d000700 1 -- 172.21.15.131:0/2793182929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 msgr2=0x5648a2a3cbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:42.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.301+0000 7fee1d000700 1 --2- 172.21.15.131:0/2793182929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a3cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.301+0000 7fee1d000700 1 -- 172.21.15.131:0/2793182929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 msgr2=0x5648a2a3b960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:42.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.302+0000 7fee1e400700 1 --2- 172.21.15.131:0/2793182929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a3b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.302+0000 7fee16000700 1 -- 172.21.15.131:0/507261619 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fee0c0055b0 con 0x7fee181087e0 2024-05-10T05:42:42.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.302+0000 7fee1d000700 1 --2- 172.21.15.131:0/2793182929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a3b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.302+0000 7fee1d000700 1 -- 172.21.15.131:0/2793182929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a27dc3b0 con 0x5648a2a3d0f0 2024-05-10T05:42:42.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.303+0000 7fee1e400700 1 --2- 172.21.15.131:0/2793182929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a3b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:42.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.303+0000 7fee1d000700 1 --2- 172.21.15.131:0/2793182929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a439f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fee0400acf0 tx=0x7fee0400f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac6bbef279de07a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:42.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.305+0000 7fee0aa00700 1 -- 172.21.15.131:0/2793182929 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04004020 con 0x5648a2a3d0f0 2024-05-10T05:42:42.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.305+0000 7fee0aa00700 1 -- 172.21.15.131:0/2793182929 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee04003680 con 0x5648a2a3d0f0 2024-05-10T05:42:42.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.305+0000 7fee0aa00700 1 -- 172.21.15.131:0/2793182929 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee040186a0 con 0x5648a2a3d0f0 2024-05-10T05:42:42.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.306+0000 7fee3824eb80 1 -- 172.21.15.131:0/2793182929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 msgr2=0x5648a2a439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:42.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.306+0000 7fee3824eb80 1 --2- 172.21.15.131:0/2793182929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a439f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fee0400acf0 tx=0x7fee0400f3c0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.307+0000 7fee3824eb80 1 -- 172.21.15.131:0/2793182929 shutdown_connections 2024-05-10T05:42:42.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.307+0000 7fee3824eb80 1 --2- 172.21.15.131:0/2793182929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a3b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.307+0000 7fee3824eb80 1 --2- 172.21.15.131:0/2793182929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a439f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.307+0000 7fee3824eb80 1 --2- 172.21.15.131:0/2793182929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a3cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.307+0000 7fee3824eb80 1 -- 172.21.15.131:0/2793182929 >> 172.21.15.131:0/2793182929 conn(0x5648a299da60 msgr2=0x5648a2a36330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:42.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.308+0000 7fee3824eb80 1 -- 172.21.15.131:0/2793182929 shutdown_connections 2024-05-10T05:42:42.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.308+0000 7fee3824eb80 1 -- 172.21.15.131:0/2793182929 wait complete. 2024-05-10T05:42:42.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.309+0000 7fee3824eb80 1 -- start start 2024-05-10T05:42:42.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.309+0000 7fee3824eb80 1 Processor -- start 2024-05-10T05:42:42.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.312+0000 7fee16000700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fee000a1180 0x7fee000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.312+0000 7fee16000700 1 -- 172.21.15.131:0/507261619 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fee0c0cd4f0 con 0x7fee181087e0 2024-05-10T05:42:42.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.313+0000 7fee1e400700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fee000a1180 0x7fee000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.313+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a457c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.313+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a45d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.313+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a4cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.313+0000 7fee3824eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a28f7560 con 0x5648a2a3b530 2024-05-10T05:42:42.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.313+0000 7fee3824eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a28f7200 con 0x5648a2a3c7a0 2024-05-10T05:42:42.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.313+0000 7fee3824eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a27de6a0 con 0x5648a2a3d0f0 2024-05-10T05:42:42.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.314+0000 7fee3824eb80 10 client.0 _ll_register_callbacks cb 0x5648a2900b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.314+0000 7fee1d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a45d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.314+0000 7fee1d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a45d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35932/0 (socket says 172.21.15.131:35932) 2024-05-10T05:42:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.314+0000 7fee1d000700 1 -- 172.21.15.131:0/3757872305 learned_addr learned my addr 172.21.15.131:0/3757872305 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.315+0000 7fee1d000700 1 -- 172.21.15.131:0/3757872305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 msgr2=0x5648a2a4cc60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:42.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.315+0000 7fee1d000700 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a4cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.315+0000 7fee1d000700 1 -- 172.21.15.131:0/3757872305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 msgr2=0x5648a2a457c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:42.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.315+0000 7fee1d000700 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a457c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.315+0000 7fee1d000700 1 -- 172.21.15.131:0/3757872305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0400a950 con 0x5648a2a3c7a0 2024-05-10T05:42:42.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.315+0000 7fee1d000700 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a45d00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fee040045c0 tx=0x7fee04003df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332b8d99940d32a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:42.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.316+0000 7fedff400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:42.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.316+0000 7fee1e400700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fee000a1180 0x7fee000a3690 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5648a2a3b2d0 tx=0x7fee10009040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:42.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.317+0000 7fedff400700 1 -- 172.21.15.131:0/3757872305 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04005450 con 0x5648a2a3c7a0 2024-05-10T05:42:42.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.317+0000 7fee3824eb80 1 -- 172.21.15.131:0/3757872305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648a28b4270 con 0x5648a2a3c7a0 2024-05-10T05:42:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.317+0000 7fedff400700 1 -- 172.21.15.131:0/3757872305 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fee040055b0 con 0x5648a2a3c7a0 2024-05-10T05:42:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.317+0000 7fedff400700 1 -- 172.21.15.131:0/3757872305 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee04005450 con 0x5648a2a3c7a0 2024-05-10T05:42:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.317+0000 7fee3824eb80 10 client.27053 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:42.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.318+0000 7fee3824eb80 1 -- 172.21.15.131:0/3757872305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5648a2788b60 con 0x5648a2a3c7a0 2024-05-10T05:42:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.318+0000 7fedff400700 1 -- 172.21.15.131:0/3757872305 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648a28b4270 con 0x5648a2a3c7a0 2024-05-10T05:42:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.318+0000 7fedff400700 1 -- 172.21.15.131:0/3757872305 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fee04020040 con 0x5648a2a3c7a0 2024-05-10T05:42:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.318+0000 7fee3824eb80 10 client.27053 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:42.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.318+0000 7fee3824eb80 10 client.27053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.319+0000 7fee3824eb80 20 client.27053 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.319+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648a2a62480 0x5648a2a648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.319+0000 7fee3824eb80 4 client.27053 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.319+0000 7fee3824eb80 1 -- 172.21.15.131:0/3757872305 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648a2a0a780 con 0x5648a2a62480 2024-05-10T05:42:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.319+0000 7fee1da00700 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648a2a62480 0x5648a2a648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.320+0000 7fee1da00700 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648a2a62480 0x5648a2a648d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74b01c7cd97df312 server_cookie=7eb1a4af86a17514 in_seq=0 out_seq=0 2024-05-10T05:42:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.320+0000 7fedff400700 10 client.27053 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.322+0000 7fedff400700 1 -- 172.21.15.131:0/3757872305 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648a2a0a780 con 0x5648a2a62480 2024-05-10T05:42:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.322+0000 7fedff400700 10 client.27053 handle_command_reply: tid=0 2024-05-10T05:42:42.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.323+0000 7fee3824eb80 1 client.27053 shutdown 2024-05-10T05:42:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.326+0000 7fee3824eb80 1 -- 172.21.15.131:0/3757872305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 msgr2=0x5648a2a45d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.326+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a45d00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fee040045c0 tx=0x7fee04003df0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.326+0000 7fee3824eb80 1 -- 172.21.15.131:0/3757872305 shutdown_connections 2024-05-10T05:42:42.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.326+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648a2a62480 0x5648a2a648d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.326+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.326+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a4cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.326+0000 7fedff400700 0 client.27053 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:42.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.326+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3757872305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a45d00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.326+0000 7fee3824eb80 1 -- 172.21.15.131:0/3757872305 >> 172.21.15.131:0/3757872305 conn(0x5648a299da60 msgr2=0x5648a2a3a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.329+0000 7fee3824eb80 1 -- 172.21.15.131:0/3757872305 shutdown_connections 2024-05-10T05:42:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.331+0000 7fee3824eb80 1 -- 172.21.15.131:0/3757872305 wait complete. 2024-05-10T05:42:42.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.331+0000 7fee3824eb80 20 client.27053 trim_cache size 0 max 16384 2024-05-10T05:42:42.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.375+0000 7fee3824eb80 1 Processor -- start 2024-05-10T05:42:42.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.375+0000 7fee3824eb80 1 -- start start 2024-05-10T05:42:42.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.375+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a40410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.376+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.376+0000 7fee1d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.376+0000 7fee1d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35942/0 (socket says 172.21.15.131:35942) 2024-05-10T05:42:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.376+0000 7fee1d000700 1 -- 172.21.15.131:0/2620816297 learned_addr learned my addr 172.21.15.131:0/2620816297 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.376+0000 7fee3824eb80 1 --2- 172.21.15.131:0/2620816297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a40950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.377+0000 7fee3824eb80 1 -- 172.21.15.131:0/2620816297 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2a40e90 con 0x5648a2a3b530 2024-05-10T05:42:42.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.377+0000 7fee1e400700 1 --2- 172.21.15.131:0/2620816297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a40950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.377+0000 7fee3824eb80 1 -- 172.21.15.131:0/2620816297 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2a69a30 con 0x5648a2a3c7a0 2024-05-10T05:42:42.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.377+0000 7fee3824eb80 1 -- 172.21.15.131:0/2620816297 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2a3c590 con 0x5648a2a3d0f0 2024-05-10T05:42:42.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.378+0000 7fee1d000700 1 -- 172.21.15.131:0/2620816297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 msgr2=0x5648a2a40950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:42.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.378+0000 7fee1d000700 1 --2- 172.21.15.131:0/2620816297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a40950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.378+0000 7fee1d000700 1 -- 172.21.15.131:0/2620816297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 msgr2=0x5648a2a40410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:42.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.378+0000 7fee1d000700 1 --2- 172.21.15.131:0/2620816297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a40410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.378+0000 7fee1d000700 1 -- 172.21.15.131:0/2620816297 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a2921560 con 0x5648a2a3c7a0 2024-05-10T05:42:42.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.378+0000 7fee1e400700 1 --2- 172.21.15.131:0/2620816297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a40950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:42.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.378+0000 7fee1d000700 1 --2- 172.21.15.131:0/2620816297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4cc20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fee0402aa50 tx=0x7fee0400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936f762a27b208ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:42.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.378+0000 7fee0b400700 1 -- 172.21.15.131:0/2620816297 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04005450 con 0x5648a2a3c7a0 2024-05-10T05:42:42.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.379+0000 7fee0b400700 1 -- 172.21.15.131:0/2620816297 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee040229a0 con 0x5648a2a3c7a0 2024-05-10T05:42:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.379+0000 7fee0b400700 1 -- 172.21.15.131:0/2620816297 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0402a3a0 con 0x5648a2a3c7a0 2024-05-10T05:42:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.380+0000 7fee3824eb80 1 -- 172.21.15.131:0/2620816297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 msgr2=0x5648a2a4cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:42.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.380+0000 7fee3824eb80 1 --2- 172.21.15.131:0/2620816297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4cc20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fee0402aa50 tx=0x7fee0400f3c0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.380+0000 7fee3824eb80 1 -- 172.21.15.131:0/2620816297 shutdown_connections 2024-05-10T05:42:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.380+0000 7fee3824eb80 1 --2- 172.21.15.131:0/2620816297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3b530 0x5648a2a40410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.381+0000 7fee3824eb80 1 --2- 172.21.15.131:0/2620816297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3d0f0 0x5648a2a40950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.381+0000 7fee3824eb80 1 --2- 172.21.15.131:0/2620816297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4cc20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.381+0000 7fee3824eb80 1 -- 172.21.15.131:0/2620816297 >> 172.21.15.131:0/2620816297 conn(0x5648a299da60 msgr2=0x5648a2a3b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:42.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.381+0000 7fee3824eb80 1 -- 172.21.15.131:0/2620816297 shutdown_connections 2024-05-10T05:42:42.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.381+0000 7fee3824eb80 1 -- 172.21.15.131:0/2620816297 wait complete. 2024-05-10T05:42:42.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.382+0000 7fee3824eb80 1 -- start start 2024-05-10T05:42:42.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.383+0000 7fee3824eb80 1 Processor -- start 2024-05-10T05:42:42.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.384+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3b530 0x5648a2a49ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.384+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.385+0000 7fee3824eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3d0f0 0x5648a2a4b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.385+0000 7fee1da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3b530 0x5648a2a49ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.385+0000 7fee1da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3b530 0x5648a2a49ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35946/0 (socket says 172.21.15.131:35946) 2024-05-10T05:42:42.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.385+0000 7fee1d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.385+0000 7fee1d000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34484/0 (socket says 172.21.15.131:34484) 2024-05-10T05:42:42.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.386+0000 7fee1e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3d0f0 0x5648a2a4b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.386+0000 7fee1e400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3d0f0 0x5648a2a4b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38016/0 (socket says 172.21.15.131:38016) 2024-05-10T05:42:42.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.386+0000 7fee1da00700 1 -- 172.21.15.131:0/3991613502 learned_addr learned my addr 172.21.15.131:0/3991613502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:42.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.386+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2a4b850 con 0x5648a2a3d0f0 2024-05-10T05:42:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.386+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2a4b9c0 con 0x5648a2a3b530 2024-05-10T05:42:42.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.386+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2a4bb80 con 0x5648a2a3c7a0 2024-05-10T05:42:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.387+0000 7fee1da00700 1 -- 172.21.15.131:0/3991613502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3c7a0 msgr2=0x5648a2a4a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.387+0000 7fee1da00700 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.387+0000 7fee1da00700 1 -- 172.21.15.131:0/3991613502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3d0f0 msgr2=0x5648a2a4b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.387+0000 7fee1da00700 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3d0f0 0x5648a2a4b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.387+0000 7fee1da00700 1 -- 172.21.15.131:0/3991613502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1000ece0 con 0x5648a2a3b530 2024-05-10T05:42:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.388+0000 7fee1e400700 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3d0f0 0x5648a2a4b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:42.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.388+0000 7fee3824eb80 10 client.0 _ll_register_callbacks cb 0x5648a2900b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:42.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.388+0000 7fee1da00700 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3b530 0x5648a2a49ea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fee0c005000 tx=0x7fee0c09a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca9d0668ebc065d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:42.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.389+0000 7fedff400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:42.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.389+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee04003680 con 0x5648a2a3b530 2024-05-10T05:42:42.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.389+0000 7fedff400700 1 -- 172.21.15.131:0/3991613502 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c09b0d0 con 0x5648a2a3b530 2024-05-10T05:42:42.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.389+0000 7fedff400700 1 -- 172.21.15.131:0/3991613502 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fee0c09a9d0 con 0x5648a2a3b530 2024-05-10T05:42:42.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.389+0000 7fedff400700 1 -- 172.21.15.131:0/3991613502 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee0c09e800 con 0x5648a2a3b530 2024-05-10T05:42:42.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.390+0000 7fee3824eb80 10 client.27059 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:42.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.390+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x5648a2788b60 con 0x5648a2a3b530 2024-05-10T05:42:42.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.390+0000 7fedff400700 1 -- 172.21.15.131:0/3991613502 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fee0c099ec0 con 0x5648a2a3b530 2024-05-10T05:42:42.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.391+0000 7fee3824eb80 10 client.27059 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:42.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.391+0000 7fee3824eb80 10 client.27059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:42.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.391+0000 7fee3824eb80 20 client.27059 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:42.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.391+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648a2ab88f0 0x5648a2abad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:42.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.391+0000 7fee3824eb80 4 client.27059 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:42:42.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.391+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fee04003680 con 0x5648a2ab88f0 2024-05-10T05:42:42.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.391+0000 7fee1d000700 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648a2ab88f0 0x5648a2abad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.395+0000 7fee1d000700 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648a2ab88f0 0x5648a2abad10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7910599c5c96c12a server_cookie=ca9111ddca16e2cc in_seq=0 out_seq=0 2024-05-10T05:42:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.395+0000 7fedff400700 10 client.27059 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.395+0000 7fedff400700 1 -- 172.21.15.131:0/3991613502 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fee04003680 con 0x5648a2ab88f0 2024-05-10T05:42:42.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.395+0000 7fedff400700 10 client.27059 handle_command_reply: tid=0 2024-05-10T05:42:42.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.395+0000 7fee3824eb80 1 client.27059 shutdown 2024-05-10T05:42:42.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.396+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3b530 msgr2=0x5648a2a49ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:42.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.396+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3b530 0x5648a2a49ea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fee0c005000 tx=0x7fee0c09a530 comp rx=0 tx=0).stop 2024-05-10T05:42:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.397+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 shutdown_connections 2024-05-10T05:42:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.397+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5648a2ab88f0 0x5648a2abad10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.397+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5648a2a3d0f0 0x5648a2a4b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.397+0000 7fedff400700 0 client.27059 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.397+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648a2a3b530 0x5648a2a49ea0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.397+0000 7fee3824eb80 1 --2- 172.21.15.131:0/3991613502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5648a2a3c7a0 0x5648a2a4a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.397+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 >> 172.21.15.131:0/3991613502 conn(0x5648a299da60 msgr2=0x5648a2a3b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.397+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 shutdown_connections 2024-05-10T05:42:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.397+0000 7fee3824eb80 1 -- 172.21.15.131:0/3991613502 wait complete. 2024-05-10T05:42:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.404+0000 7fee3824eb80 20 client.27059 trim_cache size 0 max 16384 2024-05-10T05:42:42.407 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:42:42.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.408+0000 7fee1ee00700 1 -- 172.21.15.131:0/507261619 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fee000a1180 msgr2=0x7fee000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:42.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.408+0000 7fee1ee00700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fee000a1180 0x7fee000a3690 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5648a2a3b2d0 tx=0x7fee10009040 comp rx=0 tx=0).stop 2024-05-10T05:42:42.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.409+0000 7fee1ee00700 1 -- 172.21.15.131:0/507261619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee181087e0 msgr2=0x7fee1810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:42.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.409+0000 7fee1ee00700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee181087e0 0x7fee1810eaf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fee0c005950 tx=0x7fee0c00f670 comp rx=0 tx=0).stop 2024-05-10T05:42:42.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.409+0000 7fee1ee00700 1 -- 172.21.15.131:0/507261619 shutdown_connections 2024-05-10T05:42:42.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.409+0000 7fee1ee00700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fee000a1180 0x7fee000a3690 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.409+0000 7fee1ee00700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee1810fec0 0x7fee181ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.409+0000 7fee1ee00700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee181087e0 0x7fee1810eaf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.409+0000 7fee1ee00700 1 --2- 172.21.15.131:0/507261619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee18109730 0x7fee1810f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:42.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.409+0000 7fee1ee00700 1 -- 172.21.15.131:0/507261619 >> 172.21.15.131:0/507261619 conn(0x7fee1806e800 msgr2=0x7fee18116550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:42.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.410+0000 7fee1ee00700 1 -- 172.21.15.131:0/507261619 shutdown_connections 2024-05-10T05:42:42.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:42.410+0000 7fee1ee00700 1 -- 172.21.15.131:0/507261619 wait complete. 2024-05-10T05:42:42.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:42 smithi157 ceph-mon[78427]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 262 MiB/s wr, 82 op/s 2024-05-10T05:42:42.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:42 smithi142 ceph-mon[77667]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 262 MiB/s wr, 82 op/s 2024-05-10T05:42:42.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:42 smithi131 ceph-mon[37264]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 262 MiB/s wr, 82 op/s 2024-05-10T05:42:43.474 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:42:43.475 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:42:44.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.080+0000 7ff066200700 1 -- 172.21.15.131:0/3414368173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff060109730 msgr2=0x7ff060108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.081+0000 7ff066200700 1 --2- 172.21.15.131:0/3414368173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff060109730 0x7ff060108210 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009990 tx=0x7ff04c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.081+0000 7ff066200700 1 -- 172.21.15.131:0/3414368173 shutdown_connections 2024-05-10T05:42:44.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.081+0000 7ff066200700 1 --2- 172.21.15.131:0/3414368173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff06010fec0 0x7ff060112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.081+0000 7ff066200700 1 --2- 172.21.15.131:0/3414368173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff060109730 0x7ff060108210 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.081+0000 7ff066200700 1 --2- 172.21.15.131:0/3414368173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0601087e0 0x7ff060108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.081+0000 7ff066200700 1 -- 172.21.15.131:0/3414368173 >> 172.21.15.131:0/3414368173 conn(0x7ff06006e800 msgr2=0x7ff060070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:44.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.082+0000 7ff066200700 1 -- 172.21.15.131:0/3414368173 shutdown_connections 2024-05-10T05:42:44.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.082+0000 7ff066200700 1 -- 172.21.15.131:0/3414368173 wait complete. 2024-05-10T05:42:44.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.083+0000 7ff066200700 1 Processor -- start 2024-05-10T05:42:44.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.084+0000 7ff066200700 1 -- start start 2024-05-10T05:42:44.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.084+0000 7ff066200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0601087e0 0x7ff0601a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.085+0000 7ff066200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff060109730 0x7ff0601a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.085+0000 7ff066200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff06010fec0 0x7ff0601a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.085+0000 7ff064e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0601087e0 0x7ff0601a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.085+0000 7ff066200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0601a8060 con 0x7ff060109730 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.085+0000 7ff066200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0601a81a0 con 0x7ff06010fec0 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.085+0000 7ff066200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0601a8330 con 0x7ff0601087e0 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.085+0000 7ff05fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff060109730 0x7ff0601a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.085+0000 7ff05fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff060109730 0x7ff0601a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39626/0 (socket says 172.21.15.131:39626) 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.085+0000 7ff05fe00700 1 -- 172.21.15.131:0/4071798077 learned_addr learned my addr 172.21.15.131:0/4071798077 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.085+0000 7ff065800700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff06010fec0 0x7ff0601a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.086+0000 7ff05fe00700 1 -- 172.21.15.131:0/4071798077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0601087e0 msgr2=0x7ff0601a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.086+0000 7ff05fe00700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0601087e0 0x7ff0601a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.086+0000 7ff05fe00700 1 -- 172.21.15.131:0/4071798077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff06010fec0 msgr2=0x7ff0601a7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.086+0000 7ff05fe00700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff06010fec0 0x7ff0601a7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.086+0000 7ff05fe00700 1 -- 172.21.15.131:0/4071798077 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0095f0 con 0x7ff060109730 2024-05-10T05:42:44.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.086+0000 7ff05fe00700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff060109730 0x7ff0601a0b90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff050007b40 tx=0x7ff050007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:44.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.086+0000 7ff05d600700 1 -- 172.21.15.131:0/4071798077 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050014070 con 0x7ff060109730 2024-05-10T05:42:44.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.086+0000 7ff05d600700 1 -- 172.21.15.131:0/4071798077 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff05000fa10 con 0x7ff060109730 2024-05-10T05:42:44.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.086+0000 7ff066200700 1 -- 172.21.15.131:0/4071798077 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff060109cd0 con 0x7ff060109730 2024-05-10T05:42:44.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.086+0000 7ff05d600700 1 -- 172.21.15.131:0/4071798077 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050005390 con 0x7ff060109730 2024-05-10T05:42:44.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.087+0000 7ff066200700 1 -- 172.21.15.131:0/4071798077 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06010a1e0 con 0x7ff060109730 2024-05-10T05:42:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.090+0000 7ff05d600700 1 -- 172.21.15.131:0/4071798077 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff050018070 con 0x7ff060109730 2024-05-10T05:42:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.091+0000 7ff05d600700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0440a1170 0x7ff0440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.092+0000 7ff05d600700 1 -- 172.21.15.131:0/4071798077 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff0500cfa50 con 0x7ff060109730 2024-05-10T05:42:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.092+0000 7ff064e00700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0440a1170 0x7ff0440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.093+0000 7ff064e00700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0440a1170 0x7ff0440a3680 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0098e0 tx=0x7ff04c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:44.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.093+0000 7ff07f785b80 1 Processor -- start 2024-05-10T05:42:44.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.095+0000 7ff07f785b80 1 -- start start 2024-05-10T05:42:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.096+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 0x562ee20549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.096+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2055820 0x562ee2055c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.096+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2056170 0x562ee205ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.096+0000 7ff07f785b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee1f7b1d0 con 0x562ee2056170 2024-05-10T05:42:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.096+0000 7ff07f785b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee1f78b90 con 0x562ee20545b0 2024-05-10T05:42:44.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.096+0000 7ff07f785b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee1f74480 con 0x562ee2055820 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.097+0000 7ff064e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 0x562ee20549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.097+0000 7ff05fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2055820 0x562ee2055c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.097+0000 7ff05fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2055820 0x562ee2055c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58360/0 (socket says 172.21.15.131:58360) 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.097+0000 7ff065800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2056170 0x562ee205ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.097+0000 7ff05fe00700 1 -- 172.21.15.131:0/2187136801 learned_addr learned my addr 172.21.15.131:0/2187136801 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.097+0000 7ff064e00700 1 -- 172.21.15.131:0/2187136801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2055820 msgr2=0x562ee2055c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.097+0000 7ff064e00700 1 --2- 172.21.15.131:0/2187136801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2055820 0x562ee2055c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.097+0000 7ff064e00700 1 -- 172.21.15.131:0/2187136801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2056170 msgr2=0x562ee205ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.097+0000 7ff064e00700 1 --2- 172.21.15.131:0/2187136801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2056170 0x562ee205ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.097+0000 7ff064e00700 1 -- 172.21.15.131:0/2187136801 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ee1df5430 con 0x562ee20545b0 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.098+0000 7ff064e00700 1 --2- 172.21.15.131:0/2187136801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 0x562ee20549e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff04c000c00 tx=0x7ff04c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=556e35541726c482 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:44.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.098+0000 7ff056000700 1 -- 172.21.15.131:0/2187136801 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c003d00 con 0x562ee20545b0 2024-05-10T05:42:44.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.098+0000 7ff056000700 1 -- 172.21.15.131:0/2187136801 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff04c01e040 con 0x562ee20545b0 2024-05-10T05:42:44.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.098+0000 7ff056000700 1 -- 172.21.15.131:0/2187136801 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00fa10 con 0x562ee20545b0 2024-05-10T05:42:44.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.099+0000 7ff07f785b80 1 -- 172.21.15.131:0/2187136801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 msgr2=0x562ee20549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.099+0000 7ff07f785b80 1 --2- 172.21.15.131:0/2187136801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 0x562ee20549e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff04c000c00 tx=0x7ff04c01f000 comp rx=0 tx=0).stop 2024-05-10T05:42:44.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.099+0000 7ff07f785b80 1 -- 172.21.15.131:0/2187136801 shutdown_connections 2024-05-10T05:42:44.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.099+0000 7ff07f785b80 1 --2- 172.21.15.131:0/2187136801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2056170 0x562ee205ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.099+0000 7ff07f785b80 1 --2- 172.21.15.131:0/2187136801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 0x562ee20549e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.099+0000 7ff07f785b80 1 --2- 172.21.15.131:0/2187136801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2055820 0x562ee2055c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.099+0000 7ff07f785b80 1 -- 172.21.15.131:0/2187136801 >> 172.21.15.131:0/2187136801 conn(0x562ee1fb6ae0 msgr2=0x562ee204f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:44.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.099+0000 7ff07f785b80 1 -- 172.21.15.131:0/2187136801 shutdown_connections 2024-05-10T05:42:44.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.099+0000 7ff07f785b80 1 -- 172.21.15.131:0/2187136801 wait complete. 2024-05-10T05:42:44.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.100+0000 7ff07f785b80 1 -- start start 2024-05-10T05:42:44.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.100+0000 7ff07f785b80 1 Processor -- start 2024-05-10T05:42:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee20545b0 0x562ee205e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2055820 0x562ee205ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2056170 0x562ee2065dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff07f785b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee1f105e0 con 0x562ee2055820 2024-05-10T05:42:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff07f785b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee1f10280 con 0x562ee2056170 2024-05-10T05:42:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff07f785b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee1df7720 con 0x562ee20545b0 2024-05-10T05:42:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff07f785b80 10 client.0 _ll_register_callbacks cb 0x562ee1f19bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff064e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee20545b0 0x562ee205e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff064e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee20545b0 0x562ee205e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58374/0 (socket says 172.21.15.131:58374) 2024-05-10T05:42:44.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff064e00700 1 -- 172.21.15.131:0/3539409644 learned_addr learned my addr 172.21.15.131:0/3539409644 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.101+0000 7ff065800700 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2056170 0x562ee2065dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff05fe00700 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2055820 0x562ee205ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff064e00700 1 -- 172.21.15.131:0/3539409644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2056170 msgr2=0x562ee2065dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff064e00700 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2056170 0x562ee2065dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff064e00700 1 -- 172.21.15.131:0/3539409644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2055820 msgr2=0x562ee205ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff064e00700 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2055820 0x562ee205ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff064e00700 1 -- 172.21.15.131:0/3539409644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff060112fb0 con 0x562ee20545b0 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff05fe00700 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2055820 0x562ee205ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff064e00700 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee20545b0 0x562ee205e930 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009ce0 tx=0x7ff04c003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=923d9605c994d149 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff04aa00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff04aa00700 1 -- 172.21.15.131:0/3539409644 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c0056c0 con 0x562ee20545b0 2024-05-10T05:42:44.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff07f785b80 1 -- 172.21.15.131:0/3539409644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ee1ecd2f0 con 0x562ee20545b0 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.102+0000 7ff04aa00700 1 -- 172.21.15.131:0/3539409644 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff04c005070 con 0x562ee20545b0 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff04aa00700 1 -- 172.21.15.131:0/3539409644 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff04c005230 con 0x562ee20545b0 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff07f785b80 10 client.27106 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff07f785b80 1 -- 172.21.15.131:0/3539409644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562ee1da1be0 con 0x562ee20545b0 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff04aa00700 1 -- 172.21.15.131:0/3539409644 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff04c046020 con 0x562ee20545b0 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff04aa00700 1 -- 172.21.15.131:0/3539409644 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff04c037440 con 0x562ee20545b0 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff07f785b80 10 client.27106 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff07f785b80 10 client.27106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff07f785b80 20 client.27106 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff07f785b80 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562ee206a0d0 0x562ee206c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff07f785b80 4 client.27106 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.103+0000 7ff07f785b80 1 -- 172.21.15.131:0/3539409644 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ee2023800 con 0x562ee206a0d0 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.104+0000 7ff05fe00700 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562ee206a0d0 0x562ee206c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.104+0000 7ff05fe00700 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562ee206a0d0 0x562ee206c4f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82fe3616f32b77e9 server_cookie=3ef55159bb8b55e5 in_seq=0 out_seq=0 2024-05-10T05:42:44.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.104+0000 7ff04aa00700 10 client.27106 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:44.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff04aa00700 1 -- 172.21.15.131:0/3539409644 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ee2023800 con 0x562ee206a0d0 2024-05-10T05:42:44.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff04aa00700 10 client.27106 handle_command_reply: tid=0 2024-05-10T05:42:44.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 client.27106 shutdown 2024-05-10T05:42:44.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 -- 172.21.15.131:0/3539409644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee20545b0 msgr2=0x562ee205e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee20545b0 0x562ee205e930 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009ce0 tx=0x7ff04c003db0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 -- 172.21.15.131:0/3539409644 shutdown_connections 2024-05-10T05:42:44.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562ee206a0d0 0x562ee206c4f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2056170 0x562ee2065dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee20545b0 0x562ee205e930 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 --2- 172.21.15.131:0/3539409644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2055820 0x562ee205ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 -- 172.21.15.131:0/3539409644 >> 172.21.15.131:0/3539409644 conn(0x562ee1fb6ae0 msgr2=0x562ee20530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:44.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff04aa00700 0 client.27106 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:44.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 -- 172.21.15.131:0/3539409644 shutdown_connections 2024-05-10T05:42:44.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.105+0000 7ff07f785b80 1 -- 172.21.15.131:0/3539409644 wait complete. 2024-05-10T05:42:44.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.106+0000 7ff07f785b80 20 client.27106 trim_cache size 0 max 16384 2024-05-10T05:42:44.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.145+0000 7ff07f785b80 1 Processor -- start 2024-05-10T05:42:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.152+0000 7ff07f785b80 1 -- start start 2024-05-10T05:42:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.153+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 0x562ee205d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.153+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2055820 0x562ee2063d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.153+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 0x562ee20642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.153+0000 7ff07f785b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee2066f10 con 0x562ee2055820 2024-05-10T05:42:44.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.153+0000 7ff07f785b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee2049980 con 0x562ee20545b0 2024-05-10T05:42:44.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.153+0000 7ff07f785b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee205a020 con 0x562ee2056170 2024-05-10T05:42:44.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.154+0000 7ff065800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 0x562ee20642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.154+0000 7ff065800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 0x562ee20642c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58390/0 (socket says 172.21.15.131:58390) 2024-05-10T05:42:44.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.154+0000 7ff065800700 1 -- 172.21.15.131:0/312653441 learned_addr learned my addr 172.21.15.131:0/312653441 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:44.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.155+0000 7ff064e00700 1 --2- 172.21.15.131:0/312653441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 0x562ee205d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.155+0000 7ff065800700 1 -- 172.21.15.131:0/312653441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 msgr2=0x562ee205d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:44.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.155+0000 7ff065800700 1 --2- 172.21.15.131:0/312653441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 0x562ee205d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.155+0000 7ff065800700 1 -- 172.21.15.131:0/312653441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2055820 msgr2=0x562ee2063d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:42:44.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.155+0000 7ff065800700 1 --2- 172.21.15.131:0/312653441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2055820 0x562ee2063d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.155+0000 7ff065800700 1 -- 172.21.15.131:0/312653441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ee1f3a5e0 con 0x562ee2056170 2024-05-10T05:42:44.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.164+0000 7ff065800700 1 --2- 172.21.15.131:0/312653441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 0x562ee20642c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff058006090 tx=0x7ff0580157f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9227046236b52f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:44.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.167+0000 7ff056a00700 1 -- 172.21.15.131:0/312653441 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058015ea0 con 0x562ee2056170 2024-05-10T05:42:44.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.167+0000 7ff056a00700 1 -- 172.21.15.131:0/312653441 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff058016630 con 0x562ee2056170 2024-05-10T05:42:44.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.167+0000 7ff056a00700 1 -- 172.21.15.131:0/312653441 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058004280 con 0x562ee2056170 2024-05-10T05:42:44.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.168+0000 7ff07f785b80 1 -- 172.21.15.131:0/312653441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 msgr2=0x562ee20642c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.168+0000 7ff07f785b80 1 --2- 172.21.15.131:0/312653441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 0x562ee20642c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff058006090 tx=0x7ff0580157f0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.169+0000 7ff07f785b80 1 -- 172.21.15.131:0/312653441 shutdown_connections 2024-05-10T05:42:44.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.169+0000 7ff07f785b80 1 --2- 172.21.15.131:0/312653441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 0x562ee20642c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.169+0000 7ff07f785b80 1 --2- 172.21.15.131:0/312653441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee20545b0 0x562ee205d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.169+0000 7ff07f785b80 1 --2- 172.21.15.131:0/312653441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee2055820 0x562ee2063d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.169+0000 7ff07f785b80 1 -- 172.21.15.131:0/312653441 >> 172.21.15.131:0/312653441 conn(0x562ee1fb6ae0 msgr2=0x562ee205cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:44.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.169+0000 7ff07f785b80 1 -- 172.21.15.131:0/312653441 shutdown_connections 2024-05-10T05:42:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.170+0000 7ff07f785b80 1 -- 172.21.15.131:0/312653441 wait complete. 2024-05-10T05:42:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.173+0000 7ff07f785b80 1 -- start start 2024-05-10T05:42:44.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.174+0000 7ff07f785b80 1 Processor -- start 2024-05-10T05:42:44.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.175+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee20545b0 0x562ee20b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.179+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2055820 0x562ee20b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.179+0000 7ff07f785b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 0x562ee20bfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.179+0000 7ff05fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2055820 0x562ee20b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.179+0000 7ff05fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2055820 0x562ee20b8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39324/0 (socket says 172.21.15.131:39324) 2024-05-10T05:42:44.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.179+0000 7ff05fe00700 1 -- 172.21.15.131:0/428039928 learned_addr learned my addr 172.21.15.131:0/428039928 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:44.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.179+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee20c0060 con 0x562ee20545b0 2024-05-10T05:42:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.180+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee20c01d0 con 0x562ee2055820 2024-05-10T05:42:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.180+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee20c0390 con 0x562ee2056170 2024-05-10T05:42:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.180+0000 7ff065800700 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 0x562ee20bfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.180+0000 7ff05fe00700 1 -- 172.21.15.131:0/428039928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 msgr2=0x562ee20bfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.180+0000 7ff05fe00700 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 0x562ee20bfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.180+0000 7ff07f785b80 10 client.0 _ll_register_callbacks cb 0x562ee1f19bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.180+0000 7ff05fe00700 1 -- 172.21.15.131:0/428039928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee20545b0 msgr2=0x562ee20b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:44.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.180+0000 7ff05fe00700 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee20545b0 0x562ee20b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.181+0000 7ff05fe00700 1 -- 172.21.15.131:0/428039928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0580154d0 con 0x562ee2055820 2024-05-10T05:42:44.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.181+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff04c01e040 con 0x562ee2055820 2024-05-10T05:42:44.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.181+0000 7ff05fe00700 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2055820 0x562ee20b8ca0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff050009520 tx=0x7ff0500a0df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f707f5ca7b3956 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:44.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.181+0000 7ff04aa00700 10 client.27077 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:44.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.182+0000 7ff04aa00700 1 -- 172.21.15.131:0/428039928 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05009b040 con 0x562ee2055820 2024-05-10T05:42:44.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.182+0000 7ff04aa00700 1 -- 172.21.15.131:0/428039928 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff05009c330 con 0x562ee2055820 2024-05-10T05:42:44.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.182+0000 7ff04aa00700 1 -- 172.21.15.131:0/428039928 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff05009c4f0 con 0x562ee2055820 2024-05-10T05:42:44.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.183+0000 7ff07f785b80 10 client.27077 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:44.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.184+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x562ee1df9d50 con 0x562ee2055820 2024-05-10T05:42:44.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.184+0000 7ff04aa00700 1 -- 172.21.15.131:0/428039928 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff0500054f0 con 0x562ee2055820 2024-05-10T05:42:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.184+0000 7ff07f785b80 10 client.27077 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.184+0000 7ff07f785b80 10 client.27077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.185+0000 7ff07f785b80 20 client.27077 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.185+0000 7ff07f785b80 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562ee20c1b10 0x562ee20c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.185+0000 7ff07f785b80 4 client.27077 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:42:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.185+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562ee1f6cf40 con 0x562ee20c1b10 2024-05-10T05:42:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.185+0000 7ff064e00700 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562ee20c1b10 0x562ee20c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.186+0000 7ff04aa00700 10 client.27077 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:44.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.186+0000 7ff064e00700 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562ee20c1b10 0x562ee20c3f60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9542003feb373725 server_cookie=683a6ee307a3e9c6 in_seq=0 out_seq=0 2024-05-10T05:42:44.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.188+0000 7ff04aa00700 1 -- 172.21.15.131:0/428039928 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562ee1f6cf40 con 0x562ee20c1b10 2024-05-10T05:42:44.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.188+0000 7ff04aa00700 10 client.27077 handle_command_reply: tid=0 2024-05-10T05:42:44.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.188+0000 7ff07f785b80 1 client.27077 shutdown 2024-05-10T05:42:44.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.188+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2055820 msgr2=0x562ee20b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.189+0000 7ff07f785b80 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2055820 0x562ee20b8ca0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff050009520 tx=0x7ff0500a0df0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.189+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 shutdown_connections 2024-05-10T05:42:44.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.189+0000 7ff07f785b80 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x562ee20c1b10 0x562ee20c3f60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.189+0000 7ff07f785b80 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ee20545b0 0x562ee20b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.189+0000 7ff07f785b80 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee2056170 0x562ee20bfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.189+0000 7ff07f785b80 1 --2- 172.21.15.131:0/428039928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ee2055820 0x562ee20b8ca0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.189+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 >> 172.21.15.131:0/428039928 conn(0x562ee1fb6ae0 msgr2=0x562ee2048360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:44.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.189+0000 7ff04aa00700 0 client.27077 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:44.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.190+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 shutdown_connections 2024-05-10T05:42:44.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.190+0000 7ff07f785b80 1 -- 172.21.15.131:0/428039928 wait complete. 2024-05-10T05:42:44.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.191+0000 7ff07f785b80 20 client.27077 trim_cache size 0 max 16384 2024-05-10T05:42:44.192 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:42:44.193 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:42:44.193 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "d27f10bf-85fb-46be-a9d8-e3604380e9fc", 2024-05-10T05:42:44.193 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:42:44.193 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:42:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.193+0000 7ff066200700 1 -- 172.21.15.131:0/4071798077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0440a1170 msgr2=0x7ff0440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.193+0000 7ff066200700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0440a1170 0x7ff0440a3680 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0098e0 tx=0x7ff04c01a040 comp rx=0 tx=0).stop 2024-05-10T05:42:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.193+0000 7ff066200700 1 -- 172.21.15.131:0/4071798077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff060109730 msgr2=0x7ff0601a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.193+0000 7ff066200700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff060109730 0x7ff0601a0b90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff050007b40 tx=0x7ff050007e50 comp rx=0 tx=0).stop 2024-05-10T05:42:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.195+0000 7ff066200700 1 -- 172.21.15.131:0/4071798077 shutdown_connections 2024-05-10T05:42:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.195+0000 7ff066200700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0440a1170 0x7ff0440a3680 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.195+0000 7ff066200700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff06010fec0 0x7ff0601a7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.195+0000 7ff066200700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0601087e0 0x7ff0601a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.195+0000 7ff066200700 1 --2- 172.21.15.131:0/4071798077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff060109730 0x7ff0601a0b90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.195+0000 7ff066200700 1 -- 172.21.15.131:0/4071798077 >> 172.21.15.131:0/4071798077 conn(0x7ff06006e800 msgr2=0x7ff06010bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.196+0000 7ff066200700 1 -- 172.21.15.131:0/4071798077 shutdown_connections 2024-05-10T05:42:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.196+0000 7ff066200700 1 -- 172.21.15.131:0/4071798077 wait complete. 2024-05-10T05:42:44.233 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:42:44.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:44 smithi157 ceph-mon[78427]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 211 MiB/s wr, 65 op/s 2024-05-10T05:42:44.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:44 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:42:44.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:44 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:44.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:44 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:42:44.804 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:44 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:44.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:44 smithi142 ceph-mon[77667]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 211 MiB/s wr, 65 op/s 2024-05-10T05:42:44.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:44 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:42:44.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:44 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:44.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:44 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:42:44.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:44 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:44.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:44 smithi131 ceph-mon[37264]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 211 MiB/s wr, 65 op/s 2024-05-10T05:42:44.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:44 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:42:44.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:44 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:44.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:44 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:42:44.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:44 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.987+0000 7f0005600700 1 -- 172.21.15.131:0/3770382813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0000108210 msgr2=0x7f0000108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.987+0000 7f0005600700 1 --2- 172.21.15.131:0/3770382813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0000108210 0x7f0000108620 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7effec009990 tx=0x7effec009ca0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.987+0000 7f0005600700 1 -- 172.21.15.131:0/3770382813 shutdown_connections 2024-05-10T05:42:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.987+0000 7f0005600700 1 --2- 172.21.15.131:0/3770382813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0000108210 0x7f0000108620 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.987+0000 7f0005600700 1 --2- 172.21.15.131:0/3770382813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0000110320 0x7f0000112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.987+0000 7f0005600700 1 --2- 172.21.15.131:0/3770382813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0000108b60 0x7f000010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.987+0000 7f0005600700 1 -- 172.21.15.131:0/3770382813 >> 172.21.15.131:0/3770382813 conn(0x7f000006ea00 msgr2=0x7f0000070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.987+0000 7f0005600700 1 -- 172.21.15.131:0/3770382813 shutdown_connections 2024-05-10T05:42:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.987+0000 7f0005600700 1 -- 172.21.15.131:0/3770382813 wait complete. 2024-05-10T05:42:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.991+0000 7f0005600700 1 Processor -- start 2024-05-10T05:42:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.992+0000 7f0005600700 1 -- start start 2024-05-10T05:42:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.993+0000 7f0005600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0000108210 0x7f00001a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.993+0000 7f0005600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0000108b60 0x7f00001a72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.993+0000 7f0005600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0000110320 0x7f00001ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.993+0000 7f0005600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001ae860 con 0x7f0000108b60 2024-05-10T05:42:44.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.993+0000 7f0005600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001ae9d0 con 0x7f0000110320 2024-05-10T05:42:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.993+0000 7f0005600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001aeb90 con 0x7f0000108210 2024-05-10T05:42:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.994+0000 7efffea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0000108210 0x7f00001a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.994+0000 7efffea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0000108210 0x7f00001a6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58434/0 (socket says 172.21.15.131:58434) 2024-05-10T05:42:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.994+0000 7efffea00700 1 -- 172.21.15.131:0/270143910 learned_addr learned my addr 172.21.15.131:0/270143910 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.994+0000 7effff400700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0000110320 0x7f00001ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.994+0000 7efffe000700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0000108b60 0x7f00001a72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.994+0000 7efffea00700 1 -- 172.21.15.131:0/270143910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0000110320 msgr2=0x7f00001ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.994+0000 7efffea00700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0000110320 0x7f00001ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.994+0000 7efffea00700 1 -- 172.21.15.131:0/270143910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0000108b60 msgr2=0x7f00001a72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.995+0000 7efffea00700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0000108b60 0x7f00001a72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.995+0000 7efffea00700 1 -- 172.21.15.131:0/270143910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec0095f0 con 0x7f0000108210 2024-05-10T05:42:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.995+0000 7efffea00700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0000108210 0x7f00001a6d90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7effec005950 tx=0x7effec00f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.995+0000 7efff7400700 1 -- 172.21.15.131:0/270143910 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec01d070 con 0x7f0000108210 2024-05-10T05:42:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.995+0000 7efff7400700 1 -- 172.21.15.131:0/270143910 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effec00fe40 con 0x7f0000108210 2024-05-10T05:42:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.995+0000 7efff7400700 1 -- 172.21.15.131:0/270143910 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec0054f0 con 0x7f0000108210 2024-05-10T05:42:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.996+0000 7f0005600700 1 -- 172.21.15.131:0/270143910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00001aee10 con 0x7f0000108210 2024-05-10T05:42:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:44.996+0000 7f0005600700 1 -- 172.21.15.131:0/270143910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00001af200 con 0x7f0000108210 2024-05-10T05:42:45.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.004+0000 7f001d508b80 1 Processor -- start 2024-05-10T05:42:45.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.004+0000 7efff7400700 1 -- 172.21.15.131:0/270143910 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7effec004800 con 0x7f0000108210 2024-05-10T05:42:45.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.006+0000 7f001d508b80 1 -- start start 2024-05-10T05:42:45.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.007+0000 7f001d508b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b40af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.007+0000 7f001d508b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 0x560273b41980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.007+0000 7f001d508b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.007+0000 7f001d508b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560273a67150 con 0x560273b406a0 2024-05-10T05:42:45.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.008+0000 7efffe000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.008+0000 7effff400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b40af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.008+0000 7efffe000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58436/0 (socket says 172.21.15.131:58436) 2024-05-10T05:42:45.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.008+0000 7efffe000700 1 -- 172.21.15.131:0/112024405 learned_addr learned my addr 172.21.15.131:0/112024405 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:45.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.008+0000 7f001d508b80 1 -- 172.21.15.131:0/112024405 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560273a64b10 con 0x560273b41570 2024-05-10T05:42:45.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.008+0000 7f001d508b80 1 -- 172.21.15.131:0/112024405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560273a60400 con 0x560273b41ec0 2024-05-10T05:42:45.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.009+0000 7efffea00700 1 --2- 172.21.15.131:0/112024405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 0x560273b41980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.009+0000 7efff7400700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7effe40a13b0 0x7effe40a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.009+0000 7effff400700 1 -- 172.21.15.131:0/112024405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 msgr2=0x560273b487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:45.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.009+0000 7effff400700 1 --2- 172.21.15.131:0/112024405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.009+0000 7effff400700 1 -- 172.21.15.131:0/112024405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 msgr2=0x560273b41980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:45.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.009+0000 7effff400700 1 --2- 172.21.15.131:0/112024405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 0x560273b41980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.009+0000 7effff400700 1 -- 172.21.15.131:0/112024405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602738e13b0 con 0x560273b406a0 2024-05-10T05:42:45.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.009+0000 7efffe000700 1 --2- 172.21.15.131:0/112024405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:42:45.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.009+0000 7efff7400700 1 -- 172.21.15.131:0/270143910 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7effec0cd930 con 0x7f0000108210 2024-05-10T05:42:45.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.010+0000 7effff400700 1 --2- 172.21.15.131:0/112024405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b40af0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efff8009ed0 tx=0x7efff8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f609928c84c6109f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:45.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.011+0000 7effff400700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7effe40a13b0 0x7effe40a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.011+0000 7effeb400700 1 -- 172.21.15.131:0/112024405 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8019030 con 0x560273b406a0 2024-05-10T05:42:45.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.011+0000 7effeb400700 1 -- 172.21.15.131:0/112024405 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efff8014d60 con 0x560273b406a0 2024-05-10T05:42:45.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.011+0000 7effeb400700 1 -- 172.21.15.131:0/112024405 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8010040 con 0x560273b406a0 2024-05-10T05:42:45.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.013+0000 7f001d508b80 1 -- 172.21.15.131:0/112024405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 msgr2=0x560273b40af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:45.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.013+0000 7effff400700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7effe40a13b0 0x7effe40a38c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efff8019000 tx=0x7efff80137d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:45.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.013+0000 7f001d508b80 1 --2- 172.21.15.131:0/112024405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b40af0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efff8009ed0 tx=0x7efff8009650 comp rx=0 tx=0).stop 2024-05-10T05:42:45.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.013+0000 7f001d508b80 1 -- 172.21.15.131:0/112024405 shutdown_connections 2024-05-10T05:42:45.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.013+0000 7f001d508b80 1 --2- 172.21.15.131:0/112024405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b40af0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.013+0000 7f001d508b80 1 --2- 172.21.15.131:0/112024405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.013+0000 7f001d508b80 1 --2- 172.21.15.131:0/112024405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 0x560273b41980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.013+0000 7f001d508b80 1 -- 172.21.15.131:0/112024405 >> 172.21.15.131:0/112024405 conn(0x560273aa2a60 msgr2=0x560273b3afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:45.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.013+0000 7f001d508b80 1 -- 172.21.15.131:0/112024405 shutdown_connections 2024-05-10T05:42:45.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.014+0000 7f001d508b80 1 -- 172.21.15.131:0/112024405 wait complete. 2024-05-10T05:42:45.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.015+0000 7f001d508b80 1 -- start start 2024-05-10T05:42:45.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.015+0000 7f001d508b80 1 Processor -- start 2024-05-10T05:42:45.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.016+0000 7f001d508b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b4a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.017+0000 7f001d508b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41570 0x560273b4ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.017+0000 7f001d508b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41ec0 0x560273b51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7f001d508b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602739fc560 con 0x560273b406a0 2024-05-10T05:42:45.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7f001d508b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602739fc200 con 0x560273b41ec0 2024-05-10T05:42:45.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7f001d508b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602738e36a0 con 0x560273b41570 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7f001d508b80 10 client.0 _ll_register_callbacks cb 0x560273a05b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7effff400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41ec0 0x560273b51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7effff400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41ec0 0x560273b51b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:39354/0 (socket says 172.21.15.131:39354) 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7effff400700 1 -- 172.21.15.131:0/3985096755 learned_addr learned my addr 172.21.15.131:0/3985096755 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7efffea00700 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b4a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7effff400700 1 -- 172.21.15.131:0/3985096755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41570 msgr2=0x560273b4ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7effff400700 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41570 0x560273b4ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7effff400700 1 -- 172.21.15.131:0/3985096755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 msgr2=0x560273b4a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7effff400700 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b4a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7effff400700 1 -- 172.21.15.131:0/3985096755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff0009940 con 0x560273b41ec0 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.018+0000 7efffea00700 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b4a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.019+0000 7effdfe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.019+0000 7effff400700 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41ec0 0x560273b51b40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efff8009690 tx=0x7efff8023070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb961b168943259c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:45.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.019+0000 7effdfe00700 1 -- 172.21.15.131:0/3985096755 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8010050 con 0x560273b41ec0 2024-05-10T05:42:45.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.019+0000 7f001d508b80 1 -- 172.21.15.131:0/3985096755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602739b9270 con 0x560273b41ec0 2024-05-10T05:42:45.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.019+0000 7effdfe00700 1 -- 172.21.15.131:0/3985096755 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7efff8027070 con 0x560273b41ec0 2024-05-10T05:42:45.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.019+0000 7effdfe00700 1 -- 172.21.15.131:0/3985096755 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff80203c0 con 0x560273b41ec0 2024-05-10T05:42:45.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.020+0000 7f001d508b80 10 client.27089 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:45.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.020+0000 7f001d508b80 1 -- 172.21.15.131:0/3985096755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7efff8009b00 con 0x560273b41ec0 2024-05-10T05:42:45.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.020+0000 7effdfe00700 1 -- 172.21.15.131:0/3985096755 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efff803b020 con 0x560273b41ec0 2024-05-10T05:42:45.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.020+0000 7effdfe00700 1 -- 172.21.15.131:0/3985096755 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efff802b560 con 0x560273b41ec0 2024-05-10T05:42:45.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.020+0000 7f001d508b80 10 client.27089 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:45.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.020+0000 7f001d508b80 10 client.27089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:45.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.020+0000 7f001d508b80 20 client.27089 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:45.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.020+0000 7f001d508b80 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560273b562a0 0x560273b586f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.020+0000 7f001d508b80 4 client.27089 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:42:45.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.020+0000 7f001d508b80 1 -- 172.21.15.131:0/3985096755 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560273b0f780 con 0x560273b562a0 2024-05-10T05:42:45.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.021+0000 7efffea00700 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560273b562a0 0x560273b586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.022+0000 7efffea00700 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560273b562a0 0x560273b586f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a45833f324171302 server_cookie=da6765ba8af547ed in_seq=0 out_seq=0 2024-05-10T05:42:45.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.022+0000 7effdfe00700 10 client.27089 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:45.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.022+0000 7effdfe00700 1 -- 172.21.15.131:0/3985096755 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560273b0f780 con 0x560273b562a0 2024-05-10T05:42:45.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.022+0000 7effdfe00700 10 client.27089 handle_command_reply: tid=0 2024-05-10T05:42:45.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.023+0000 7f001d508b80 1 client.27089 shutdown 2024-05-10T05:42:45.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.023+0000 7f001d508b80 1 -- 172.21.15.131:0/3985096755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41ec0 msgr2=0x560273b51b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:45.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.024+0000 7f001d508b80 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41ec0 0x560273b51b40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efff8009690 tx=0x7efff8023070 comp rx=0 tx=0).stop 2024-05-10T05:42:45.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.024+0000 7f001d508b80 1 -- 172.21.15.131:0/3985096755 shutdown_connections 2024-05-10T05:42:45.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.024+0000 7f001d508b80 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560273b562a0 0x560273b586f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.024+0000 7effdfe00700 0 client.27089 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:45.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.024+0000 7f001d508b80 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b4a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.024+0000 7f001d508b80 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41ec0 0x560273b51b40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.024+0000 7f001d508b80 1 --2- 172.21.15.131:0/3985096755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41570 0x560273b4ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.024+0000 7f001d508b80 1 -- 172.21.15.131:0/3985096755 >> 172.21.15.131:0/3985096755 conn(0x560273aa2a60 msgr2=0x560273b3afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:45.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.025+0000 7f001d508b80 1 -- 172.21.15.131:0/3985096755 shutdown_connections 2024-05-10T05:42:45.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.025+0000 7f001d508b80 1 -- 172.21.15.131:0/3985096755 wait complete. 2024-05-10T05:42:45.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.026+0000 7f001d508b80 20 client.27089 trim_cache size 0 max 16384 2024-05-10T05:42:45.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.085+0000 7f001d508b80 1 Processor -- start 2024-05-10T05:42:45.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.087+0000 7f001d508b80 1 -- start start 2024-05-10T05:42:45.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.087+0000 7f001d508b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b406a0 0x560273b49cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.090+0000 7f001d508b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b41570 0x560273b4faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.090+0000 7f001d508b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b50030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.090+0000 7f001d508b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560273b357c0 con 0x560273b41570 2024-05-10T05:42:45.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.090+0000 7f001d508b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560273b3c770 con 0x560273b406a0 2024-05-10T05:42:45.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.090+0000 7f001d508b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560273b40f50 con 0x560273b41ec0 2024-05-10T05:42:45.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.091+0000 7effff400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b50030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.091+0000 7effff400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b50030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58444/0 (socket says 172.21.15.131:58444) 2024-05-10T05:42:45.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.093+0000 7effff400700 1 -- 172.21.15.131:0/1263457647 learned_addr learned my addr 172.21.15.131:0/1263457647 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:45.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.093+0000 7efffe000700 1 --2- 172.21.15.131:0/1263457647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b41570 0x560273b4faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.094+0000 7effff400700 1 -- 172.21.15.131:0/1263457647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b406a0 msgr2=0x560273b49cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:42:45.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.094+0000 7effff400700 1 --2- 172.21.15.131:0/1263457647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b406a0 0x560273b49cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.094+0000 7effff400700 1 -- 172.21.15.131:0/1263457647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b41570 msgr2=0x560273b4faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:45.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.094+0000 7effff400700 1 --2- 172.21.15.131:0/1263457647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b41570 0x560273b4faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.094+0000 7effff400700 1 -- 172.21.15.131:0/1263457647 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560273b3b260 con 0x560273b41ec0 2024-05-10T05:42:45.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.094+0000 7efffe000700 1 --2- 172.21.15.131:0/1263457647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b41570 0x560273b4faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:42:45.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.096+0000 7effff400700 1 --2- 172.21.15.131:0/1263457647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b50030 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efff8002230 tx=0x7efff8004a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d20b9f45a9e00288 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:45.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.097+0000 7effebe00700 1 -- 172.21.15.131:0/1263457647 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8010050 con 0x560273b41ec0 2024-05-10T05:42:45.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.098+0000 7effebe00700 1 -- 172.21.15.131:0/1263457647 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efff803b030 con 0x560273b41ec0 2024-05-10T05:42:45.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.101+0000 7effebe00700 1 -- 172.21.15.131:0/1263457647 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8010050 con 0x560273b41ec0 2024-05-10T05:42:45.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.105+0000 7f001d508b80 1 -- 172.21.15.131:0/1263457647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 msgr2=0x560273b50030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:45.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.105+0000 7f001d508b80 1 --2- 172.21.15.131:0/1263457647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b50030 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efff8002230 tx=0x7efff8004a50 comp rx=0 tx=0).stop 2024-05-10T05:42:45.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.105+0000 7f001d508b80 1 -- 172.21.15.131:0/1263457647 shutdown_connections 2024-05-10T05:42:45.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.105+0000 7f001d508b80 1 --2- 172.21.15.131:0/1263457647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273b50030 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.105+0000 7f001d508b80 1 --2- 172.21.15.131:0/1263457647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b406a0 0x560273b49cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.105+0000 7f001d508b80 1 --2- 172.21.15.131:0/1263457647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b41570 0x560273b4faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.105+0000 7f001d508b80 1 -- 172.21.15.131:0/1263457647 >> 172.21.15.131:0/1263457647 conn(0x560273aa2a60 msgr2=0x560273b3bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:45.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.106+0000 7f001d508b80 1 -- 172.21.15.131:0/1263457647 shutdown_connections 2024-05-10T05:42:45.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.107+0000 7f001d508b80 1 -- 172.21.15.131:0/1263457647 wait complete. 2024-05-10T05:42:45.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.109+0000 7f001d508b80 1 -- start start 2024-05-10T05:42:45.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.110+0000 7f001d508b80 1 Processor -- start 2024-05-10T05:42:45.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.113+0000 7f001d508b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b4f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.114+0000 7f001d508b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 0x560273b4fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.114+0000 7f001d508b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273baaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.115+0000 7effff400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273baaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.117+0000 7f001d508b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560273bab010 con 0x560273b406a0 2024-05-10T05:42:45.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.117+0000 7f001d508b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560273bab150 con 0x560273b41570 2024-05-10T05:42:45.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.117+0000 7f001d508b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560273bab310 con 0x560273b41ec0 2024-05-10T05:42:45.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.117+0000 7effff400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273baaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:58458/0 (socket says 172.21.15.131:58458) 2024-05-10T05:42:45.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.117+0000 7effff400700 1 -- 172.21.15.131:0/2425700931 learned_addr learned my addr 172.21.15.131:0/2425700931 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:42:45.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.119+0000 7efffe000700 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 0x560273b4fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.120+0000 7f001d508b80 10 client.0 _ll_register_callbacks cb 0x560273a05b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:42:45.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.120+0000 7efffea00700 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b4f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.120+0000 7efffe000700 1 -- 172.21.15.131:0/2425700931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 msgr2=0x560273baaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:45.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.120+0000 7efffe000700 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273baaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.121+0000 7efffe000700 1 -- 172.21.15.131:0/2425700931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 msgr2=0x560273b4f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:42:45.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.121+0000 7efffe000700 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b4f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.121+0000 7efffe000700 1 -- 172.21.15.131:0/2425700931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff8009b00 con 0x560273b41570 2024-05-10T05:42:45.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.122+0000 7effdfe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:42:45.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.123+0000 7efffe000700 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 0x560273b4fa50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x560273b3af60 tx=0x7efff0012ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d114e3bb2095ec02 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:42:45.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.123+0000 7effdfe00700 1 -- 172.21.15.131:0/2425700931 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0011b30 con 0x560273b41570 2024-05-10T05:42:45.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.123+0000 7effdfe00700 1 -- 172.21.15.131:0/2425700931 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7efff0011c90 con 0x560273b41570 2024-05-10T05:42:45.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.124+0000 7f001d508b80 1 -- 172.21.15.131:0/2425700931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efff8014d60 con 0x560273b41570 2024-05-10T05:42:45.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.124+0000 7effdfe00700 1 -- 172.21.15.131:0/2425700931 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff0011b30 con 0x560273b41570 2024-05-10T05:42:45.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.124+0000 7f001d508b80 10 client.27092 fetch_fsmap learned FSMap version 29 2024-05-10T05:42:45.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.124+0000 7f001d508b80 1 -- 172.21.15.131:0/2425700931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560273b37450 con 0x560273b41570 2024-05-10T05:42:45.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.126+0000 7effdfe00700 1 -- 172.21.15.131:0/2425700931 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efff8014d60 con 0x560273b41570 2024-05-10T05:42:45.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.126+0000 7effdfe00700 1 -- 172.21.15.131:0/2425700931 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efff0023020 con 0x560273b41570 2024-05-10T05:42:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.126+0000 7f001d508b80 10 client.27092 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:42:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.126+0000 7f001d508b80 10 client.27092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:42:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.126+0000 7f001d508b80 20 client.27092 populate_metadata read hostname 'smithi131' 2024-05-10T05:42:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.126+0000 7f001d508b80 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560273b562a0 0x560273baedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:42:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.126+0000 7f001d508b80 4 client.27092 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:42:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.126+0000 7f001d508b80 1 -- 172.21.15.131:0/2425700931 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560273a58ec0 con 0x560273b562a0 2024-05-10T05:42:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.128+0000 7efffea00700 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560273b562a0 0x560273baedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:42:45.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.130+0000 7efffea00700 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560273b562a0 0x560273baedb0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36cfb59e2231f60 server_cookie=caed5c5e9ee46f75 in_seq=0 out_seq=0 2024-05-10T05:42:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.130+0000 7effdfe00700 10 client.27092 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.130+0000 7effdfe00700 1 -- 172.21.15.131:0/2425700931 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560273a58ec0 con 0x560273b562a0 2024-05-10T05:42:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.130+0000 7effdfe00700 10 client.27092 handle_command_reply: tid=0 2024-05-10T05:42:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.130+0000 7f001d508b80 1 client.27092 shutdown 2024-05-10T05:42:45.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.131+0000 7f001d508b80 1 -- 172.21.15.131:0/2425700931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 msgr2=0x560273b4fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:45.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.131+0000 7f001d508b80 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 0x560273b4fa50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x560273b3af60 tx=0x7efff0012ca0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.132+0000 7f001d508b80 1 -- 172.21.15.131:0/2425700931 shutdown_connections 2024-05-10T05:42:45.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.132+0000 7f001d508b80 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560273b562a0 0x560273baedb0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.132+0000 7f001d508b80 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560273b406a0 0x560273b4f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.132+0000 7f001d508b80 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560273b41ec0 0x560273baaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.132+0000 7f001d508b80 1 --2- 172.21.15.131:0/2425700931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560273b41570 0x560273b4fa50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.132+0000 7f001d508b80 1 -- 172.21.15.131:0/2425700931 >> 172.21.15.131:0/2425700931 conn(0x560273aa2a60 msgr2=0x560273b3bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.132+0000 7effdfe00700 0 client.27092 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:42:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.133+0000 7f001d508b80 1 -- 172.21.15.131:0/2425700931 shutdown_connections 2024-05-10T05:42:45.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.133+0000 7f001d508b80 1 -- 172.21.15.131:0/2425700931 wait complete. 2024-05-10T05:42:45.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.134+0000 7f001d508b80 20 client.27092 trim_cache size 0 max 16384 2024-05-10T05:42:45.135 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:42:45.136 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:42:45.136 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:42:45.136 INFO:teuthology.orchestra.run.smithi131.stdout: "d27f10bf-85fb-46be-a9d8-e3604380e9fc": { 2024-05-10T05:42:45.136 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:42:45.136 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "d27f10bf-85fb-46be-a9d8-e3604380e9fc", 2024-05-10T05:42:45.136 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:42:45.136 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:42:45.136 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:42:45.136 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:42:45.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.135+0000 7f0005600700 1 -- 172.21.15.131:0/270143910 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7effe40a13b0 msgr2=0x7effe40a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:45.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.135+0000 7f0005600700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7effe40a13b0 0x7effe40a38c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efff8019000 tx=0x7efff80137d0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.135+0000 7f0005600700 1 -- 172.21.15.131:0/270143910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0000108210 msgr2=0x7f00001a6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:42:45.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.135+0000 7f0005600700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0000108210 0x7f00001a6d90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7effec005950 tx=0x7effec00f670 comp rx=0 tx=0).stop 2024-05-10T05:42:45.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.136+0000 7f0005600700 1 -- 172.21.15.131:0/270143910 shutdown_connections 2024-05-10T05:42:45.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.136+0000 7f0005600700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7effe40a13b0 0x7effe40a38c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.136+0000 7f0005600700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0000110320 0x7f00001ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.136+0000 7f0005600700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0000108210 0x7f00001a6d90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.136+0000 7f0005600700 1 --2- 172.21.15.131:0/270143910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0000108b60 0x7f00001a72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:42:45.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.136+0000 7f0005600700 1 -- 172.21.15.131:0/270143910 >> 172.21.15.131:0/270143910 conn(0x7f000006ea00 msgr2=0x7f0000114760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:42:45.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.136+0000 7f0005600700 1 -- 172.21.15.131:0/270143910 shutdown_connections 2024-05-10T05:42:45.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:42:45.137+0000 7f0005600700 1 -- 172.21.15.131:0/270143910 wait complete. 2024-05-10T05:42:45.194 INFO:tasks.cephfs.filesystem:scrub status for tag:d27f10bf-85fb-46be-a9d8-e3604380e9fc - {'path': '/', 'tag': 'd27f10bf-85fb-46be-a9d8-e3604380e9fc', 'options': 'recursive,force'} 2024-05-10T05:42:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:45 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:42:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:45 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:45.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:45 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:42:45.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:45 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:45.964 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:45 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:42:45.964 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:45 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:42:46.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:46 smithi157 ceph-mon[78427]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 265 MiB/s wr, 81 op/s 2024-05-10T05:42:46.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:46 smithi142 ceph-mon[77667]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 265 MiB/s wr, 81 op/s 2024-05-10T05:42:46.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:46 smithi131 ceph-mon[37264]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 265 MiB/s wr, 81 op/s 2024-05-10T05:42:48.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:48 smithi157 ceph-mon[78427]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 212 MiB/s wr, 67 op/s 2024-05-10T05:42:48.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:48 smithi142 ceph-mon[77667]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 212 MiB/s wr, 67 op/s 2024-05-10T05:42:48.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:48 smithi131 ceph-mon[37264]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 212 MiB/s wr, 67 op/s 2024-05-10T05:42:50.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:50 smithi157 ceph-mon[78427]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 251 MiB/s wr, 76 op/s 2024-05-10T05:42:50.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:50 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:42:50.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:50 smithi142 ceph-mon[77667]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 251 MiB/s wr, 76 op/s 2024-05-10T05:42:50.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:50 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:42:50.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:50 smithi131 ceph-mon[37264]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 251 MiB/s wr, 76 op/s 2024-05-10T05:42:50.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:50 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:42:51.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:51 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:42:51.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:51 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:42:52.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:51 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:42:52.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:52 smithi142 ceph-mon[77667]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:42:52.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:52 smithi131 ceph-mon[37264]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:42:53.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:52 smithi157 ceph-mon[78427]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:42:54.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:54 smithi142 ceph-mon[77667]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 220 MiB/s wr, 66 op/s 2024-05-10T05:42:54.968 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:54 smithi131 ceph-mon[37264]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 220 MiB/s wr, 66 op/s 2024-05-10T05:42:55.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:54 smithi157 ceph-mon[78427]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 220 MiB/s wr, 66 op/s 2024-05-10T05:42:56.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:56 smithi142 ceph-mon[77667]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 286 MiB/s wr, 85 op/s 2024-05-10T05:42:56.964 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:56 smithi131 ceph-mon[37264]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 286 MiB/s wr, 85 op/s 2024-05-10T05:42:57.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:56 smithi157 ceph-mon[78427]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 286 MiB/s wr, 85 op/s 2024-05-10T05:42:58.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:58 smithi142 ceph-mon[77667]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 242 MiB/s wr, 73 op/s 2024-05-10T05:42:58.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:58 smithi131 ceph-mon[37264]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 242 MiB/s wr, 73 op/s 2024-05-10T05:42:59.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:58 smithi157 ceph-mon[78427]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 242 MiB/s wr, 73 op/s 2024-05-10T05:42:59.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:42:59 smithi131 ceph-mon[37264]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 290 MiB/s wr, 85 op/s 2024-05-10T05:43:00.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:42:59 smithi157 ceph-mon[78427]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 290 MiB/s wr, 85 op/s 2024-05-10T05:43:00.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:42:59 smithi142 ceph-mon[77667]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 290 MiB/s wr, 85 op/s 2024-05-10T05:43:02.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:02 smithi131 ceph-mon[37264]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 275 MiB/s wr, 83 op/s 2024-05-10T05:43:02.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:02 smithi157 ceph-mon[78427]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 275 MiB/s wr, 83 op/s 2024-05-10T05:43:02.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:02 smithi142 ceph-mon[77667]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 275 MiB/s wr, 83 op/s 2024-05-10T05:43:04.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:04 smithi157 ceph-mon[78427]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 238 MiB/s wr, 71 op/s 2024-05-10T05:43:04.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:04 smithi142 ceph-mon[77667]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 238 MiB/s wr, 71 op/s 2024-05-10T05:43:04.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:04 smithi131 ceph-mon[37264]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 238 MiB/s wr, 71 op/s 2024-05-10T05:43:06.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:06 smithi157 ceph-mon[78427]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 305 MiB/s wr, 91 op/s 2024-05-10T05:43:06.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:06 smithi142 ceph-mon[77667]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 305 MiB/s wr, 91 op/s 2024-05-10T05:43:06.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:06 smithi131 ceph-mon[37264]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 305 MiB/s wr, 91 op/s 2024-05-10T05:43:08.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:08 smithi157 ceph-mon[78427]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:43:08.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:08 smithi142 ceph-mon[77667]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:43:08.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:08 smithi131 ceph-mon[37264]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:43:10.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:10 smithi142 ceph-mon[77667]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 282 MiB/s wr, 84 op/s 2024-05-10T05:43:10.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:10 smithi131 ceph-mon[37264]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 282 MiB/s wr, 84 op/s 2024-05-10T05:43:11.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:10 smithi157 ceph-mon[78427]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 282 MiB/s wr, 84 op/s 2024-05-10T05:43:12.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:12 smithi142 ceph-mon[77667]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 261 MiB/s wr, 81 op/s 2024-05-10T05:43:12.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:12 smithi131 ceph-mon[37264]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 261 MiB/s wr, 81 op/s 2024-05-10T05:43:13.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:12 smithi157 ceph-mon[78427]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 261 MiB/s wr, 81 op/s 2024-05-10T05:43:14.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:14 smithi142 ceph-mon[77667]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 69 op/s 2024-05-10T05:43:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:14 smithi131 ceph-mon[37264]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 69 op/s 2024-05-10T05:43:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:14 smithi157 ceph-mon[78427]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 228 MiB/s wr, 69 op/s 2024-05-10T05:43:15.195 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:43:15.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.972+0000 7f7de2e00700 1 -- 172.21.15.131:0/1062624520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ddc102580 msgr2=0x7f7ddc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:15.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.972+0000 7f7de2e00700 1 --2- 172.21.15.131:0/1062624520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ddc102580 0x7f7ddc108e30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400b0d0 tx=0x7f7dd400b3e0 comp rx=0 tx=0).stop 2024-05-10T05:43:15.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.972+0000 7f7de2e00700 1 -- 172.21.15.131:0/1062624520 shutdown_connections 2024-05-10T05:43:15.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.972+0000 7f7de2e00700 1 --2- 172.21.15.131:0/1062624520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ddc102580 0x7f7ddc108e30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:15.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.972+0000 7f7de2e00700 1 --2- 172.21.15.131:0/1062624520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ddc100ac0 0x7f7ddc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:15.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.972+0000 7f7de2e00700 1 --2- 172.21.15.131:0/1062624520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ddc101be0 0x7f7ddc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:15.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.973+0000 7f7de2e00700 1 -- 172.21.15.131:0/1062624520 >> 172.21.15.131:0/1062624520 conn(0x7f7ddc0fc0d0 msgr2=0x7f7ddc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.978+0000 7f7de2e00700 1 -- 172.21.15.131:0/1062624520 shutdown_connections 2024-05-10T05:43:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.978+0000 7f7de2e00700 1 -- 172.21.15.131:0/1062624520 wait complete. 2024-05-10T05:43:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.980+0000 7f7de2e00700 1 Processor -- start 2024-05-10T05:43:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.980+0000 7f7de2e00700 1 -- start start 2024-05-10T05:43:15.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.980+0000 7f7de2e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ddc100ac0 0x7f7ddc106630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:15.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.981+0000 7f7de2e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ddc101be0 0x7f7ddc106b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:15.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.981+0000 7f7de2e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ddc102580 0x7f7ddc103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:15.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.981+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ddc100ac0 0x7f7ddc106630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:15.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.981+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ddc100ac0 0x7f7ddc106630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34430/0 (socket says 172.21.15.131:34430) 2024-05-10T05:43:15.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.981+0000 7f7de2e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc103940 con 0x7f7ddc102580 2024-05-10T05:43:15.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.981+0000 7f7de2e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc103ab0 con 0x7f7ddc101be0 2024-05-10T05:43:15.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.981+0000 7f7de2e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc103c70 con 0x7f7ddc100ac0 2024-05-10T05:43:15.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.981+0000 7f7ddbe00700 1 -- 172.21.15.131:0/2209580112 learned_addr learned my addr 172.21.15.131:0/2209580112 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.982+0000 7f7de0e00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ddc102580 0x7f7ddc103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.982+0000 7f7ddbe00700 1 -- 172.21.15.131:0/2209580112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ddc101be0 msgr2=0x7f7ddc106b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:43:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.982+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ddc101be0 0x7f7ddc106b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.982+0000 7f7ddbe00700 1 -- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ddc102580 msgr2=0x7f7ddc103240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.982+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ddc102580 0x7f7ddc103240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.982+0000 7f7ddbe00700 1 -- 172.21.15.131:0/2209580112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8009520 con 0x7f7ddc100ac0 2024-05-10T05:43:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.982+0000 7f7de0e00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ddc102580 0x7f7ddc103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.983+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ddc100ac0 0x7f7ddc106630 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0097b0 tx=0x7f7dcc009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.984+0000 7f7dd8c00700 1 -- 172.21.15.131:0/2209580112 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc010040 con 0x7f7ddc100ac0 2024-05-10T05:43:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.984+0000 7f7dd8c00700 1 -- 172.21.15.131:0/2209580112 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dcc0044e0 con 0x7f7ddc100ac0 2024-05-10T05:43:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.984+0000 7f7dd8c00700 1 -- 172.21.15.131:0/2209580112 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc014870 con 0x7f7ddc100ac0 2024-05-10T05:43:15.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.985+0000 7f7de2e00700 1 -- 172.21.15.131:0/2209580112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd4009d00 con 0x7f7ddc100ac0 2024-05-10T05:43:15.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.986+0000 7f7de2e00700 1 -- 172.21.15.131:0/2209580112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc06af90 con 0x7f7ddc100ac0 2024-05-10T05:43:15.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.993+0000 7f7dd8c00700 1 -- 172.21.15.131:0/2209580112 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7dcc003f00 con 0x7f7ddc100ac0 2024-05-10T05:43:15.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.994+0000 7f7dfad5fb80 1 Processor -- start 2024-05-10T05:43:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.994+0000 7f7dfad5fb80 1 -- start start 2024-05-10T05:43:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.995+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b8eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.995+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b8f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.996+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.996+0000 7f7de0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b8eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:15.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.996+0000 7f7de0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b8eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38508/0 (socket says 172.21.15.131:38508) 2024-05-10T05:43:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.996+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b8f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.996+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34ab5150 con 0x560e34b8fec0 2024-05-10T05:43:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.996+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34ab2b10 con 0x560e34b8e6a0 2024-05-10T05:43:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.996+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34aae400 con 0x560e34b8f570 2024-05-10T05:43:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.996+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b8f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34438/0 (socket says 172.21.15.131:34438) 2024-05-10T05:43:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.996+0000 7f7de0e00700 1 -- 172.21.15.131:0/91397726 learned_addr learned my addr 172.21.15.131:0/91397726 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.996+0000 7f7ddb400700 1 --2- 172.21.15.131:0/91397726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.997+0000 7f7dd8c00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7dc00a1330 0x7f7dc00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.998+0000 7f7dd8c00700 1 -- 172.21.15.131:0/2209580112 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f7dcc0cbc00 con 0x7f7ddc100ac0 2024-05-10T05:43:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.998+0000 7f7ddb400700 1 -- 172.21.15.131:0/91397726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 msgr2=0x560e34b8f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.998+0000 7f7ddb400700 1 --2- 172.21.15.131:0/91397726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b8f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.998+0000 7f7ddb400700 1 -- 172.21.15.131:0/91397726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 msgr2=0x560e34b8eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:16.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.998+0000 7f7ddb400700 1 --2- 172.21.15.131:0/91397726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b8eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.998+0000 7f7ddb400700 1 -- 172.21.15.131:0/91397726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e3492f3b0 con 0x560e34b8fec0 2024-05-10T05:43:16.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.999+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/91397726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b8f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:16.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:15.999+0000 7f7de0e00700 1 --2- 172.21.15.131:0/91397726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b8eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:16.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.000+0000 7f7de0e00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7dc00a1330 0x7f7dc00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.000+0000 7f7ddb400700 1 --2- 172.21.15.131:0/91397726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b967c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009c00 tx=0x7f7dc8009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b90f7a806a3e2ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:16.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.001+0000 7f7de0e00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7dc00a1330 0x7f7dc00a3840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x560e34b8e380 tx=0x7f7dd40099e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:16.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.001+0000 7f7dd1600700 1 -- 172.21.15.131:0/91397726 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8004230 con 0x560e34b8fec0 2024-05-10T05:43:16.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.002+0000 7f7dd1600700 1 -- 172.21.15.131:0/91397726 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc800eda0 con 0x560e34b8fec0 2024-05-10T05:43:16.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.002+0000 7f7dd1600700 1 -- 172.21.15.131:0/91397726 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8003de0 con 0x560e34b8fec0 2024-05-10T05:43:16.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.009+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/91397726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 msgr2=0x560e34b967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:16.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.009+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/91397726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b967c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009c00 tx=0x7f7dc8009f10 comp rx=0 tx=0).stop 2024-05-10T05:43:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.011+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/91397726 shutdown_connections 2024-05-10T05:43:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.011+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/91397726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b967c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.011+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/91397726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b8eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.011+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/91397726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b8f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.011+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/91397726 >> 172.21.15.131:0/91397726 conn(0x560e34af0a60 msgr2=0x560e34b88fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:16.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.011+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/91397726 shutdown_connections 2024-05-10T05:43:16.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.012+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/91397726 wait complete. 2024-05-10T05:43:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.015+0000 7f7dfad5fb80 1 -- start start 2024-05-10T05:43:16.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.015+0000 7f7dfad5fb80 1 Processor -- start 2024-05-10T05:43:16.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.017+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b98710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.018+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8f570 0x560e34b98c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.018+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8fec0 0x560e34b9fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.018+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34a4a560 con 0x560e34b8f570 2024-05-10T05:43:16.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.018+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34a4a200 con 0x560e34b8e6a0 2024-05-10T05:43:16.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.018+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e349316a0 con 0x560e34b8fec0 2024-05-10T05:43:16.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.018+0000 7f7dfad5fb80 10 client.0 _ll_register_callbacks cb 0x560e34a53b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:16.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.018+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b98710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.018+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b98710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38514/0 (socket says 172.21.15.131:38514) 2024-05-10T05:43:16.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.018+0000 7f7ddbe00700 1 -- 172.21.15.131:0/1483723775 learned_addr learned my addr 172.21.15.131:0/1483723775 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:16.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.018+0000 7f7ddb400700 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8f570 0x560e34b98c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.019+0000 7f7de0e00700 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8fec0 0x560e34b9fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.019+0000 7f7ddb400700 1 -- 172.21.15.131:0/1483723775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8fec0 msgr2=0x560e34b9fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:16.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.019+0000 7f7ddb400700 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8fec0 0x560e34b9fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.019+0000 7f7ddb400700 1 -- 172.21.15.131:0/1483723775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 msgr2=0x560e34b98710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:16.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.019+0000 7f7ddb400700 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b98710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.019+0000 7f7ddb400700 1 -- 172.21.15.131:0/1483723775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc06af90 con 0x560e34b8f570 2024-05-10T05:43:16.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.019+0000 7f7de0e00700 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8fec0 0x560e34b9fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:16.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.019+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b98710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:16.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.020+0000 7f7ddb400700 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8f570 0x560e34b98c50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80036c0 tx=0x7f7dc8003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce952526092b4b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:16.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.020+0000 7f7dc6000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:43:16.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.021+0000 7f7dc6000700 1 -- 172.21.15.131:0/1483723775 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8026070 con 0x560e34b8f570 2024-05-10T05:43:16.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.021+0000 7f7dc6000700 1 -- 172.21.15.131:0/1483723775 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f7dc8014070 con 0x560e34b8f570 2024-05-10T05:43:16.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.021+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/1483723775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e34a07270 con 0x560e34b8f570 2024-05-10T05:43:16.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.021+0000 7f7dc6000700 1 -- 172.21.15.131:0/1483723775 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc80040e0 con 0x560e34b8f570 2024-05-10T05:43:16.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.021+0000 7f7dfad5fb80 10 client.16782 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:16.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.021+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/1483723775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f7dc80098e0 con 0x560e34b8f570 2024-05-10T05:43:16.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.022+0000 7f7dc6000700 1 -- 172.21.15.131:0/1483723775 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e34a07270 con 0x560e34b8f570 2024-05-10T05:43:16.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.024+0000 7f7dc6000700 1 -- 172.21.15.131:0/1483723775 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7dc8029030 con 0x560e34b8f570 2024-05-10T05:43:16.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.025+0000 7f7dfad5fb80 10 client.16782 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:16.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.025+0000 7f7dfad5fb80 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:43:16.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.025+0000 7f7dfad5fb80 20 client.16782 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:16.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.025+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560e34ba41d0 0x560e34ba6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.025+0000 7f7dfad5fb80 4 client.16782 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:43:16.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.025+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/1483723775 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e34b5d780 con 0x560e34ba41d0 2024-05-10T05:43:16.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.026+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560e34ba41d0 0x560e34ba6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.027+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560e34ba41d0 0x560e34ba6620 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c147ba79132b28b9 server_cookie=b1bbe0ff04744a36 in_seq=0 out_seq=0 2024-05-10T05:43:16.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.027+0000 7f7dc6000700 10 client.16782 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:16.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.028+0000 7f7dc6000700 1 -- 172.21.15.131:0/1483723775 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e34b5d780 con 0x560e34ba41d0 2024-05-10T05:43:16.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.028+0000 7f7dc6000700 10 client.16782 handle_command_reply: tid=0 2024-05-10T05:43:16.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.028+0000 7f7dfad5fb80 1 client.16782 shutdown 2024-05-10T05:43:16.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.030+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/1483723775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8f570 msgr2=0x560e34b98c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:16.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.030+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8f570 0x560e34b98c50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80036c0 tx=0x7f7dc8003e40 comp rx=0 tx=0).stop 2024-05-10T05:43:16.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.031+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/1483723775 shutdown_connections 2024-05-10T05:43:16.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.031+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560e34ba41d0 0x560e34ba6620 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.031+0000 7f7dc6000700 0 client.16782 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:16.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.031+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8fec0 0x560e34b9fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.031+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b98710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.031+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/1483723775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8f570 0x560e34b98c50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.031+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/1483723775 >> 172.21.15.131:0/1483723775 conn(0x560e34af0a60 msgr2=0x560e34b88fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:16.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.031+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/1483723775 shutdown_connections 2024-05-10T05:43:16.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.032+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/1483723775 wait complete. 2024-05-10T05:43:16.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.032+0000 7f7dfad5fb80 20 client.16782 trim_cache size 0 max 16384 2024-05-10T05:43:16.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.072+0000 7f7dfad5fb80 1 Processor -- start 2024-05-10T05:43:16.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.072+0000 7f7dfad5fb80 1 -- start start 2024-05-10T05:43:16.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.073+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8e6a0 0x560e34b97c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.073+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8f570 0x560e34b9fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.073+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.073+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34b99580 con 0x560e34b8fec0 2024-05-10T05:43:16.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.073+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34b99760 con 0x560e34b8f570 2024-05-10T05:43:16.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.073+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34b837c0 con 0x560e34b8e6a0 2024-05-10T05:43:16.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.073+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8e6a0 0x560e34b97c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.074+0000 7f7ddbe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8e6a0 0x560e34b97c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34452/0 (socket says 172.21.15.131:34452) 2024-05-10T05:43:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.074+0000 7f7ddb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8f570 0x560e34b9fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.074+0000 7f7ddbe00700 1 -- 172.21.15.131:0/599747892 learned_addr learned my addr 172.21.15.131:0/599747892 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.074+0000 7f7de0e00700 1 --2- 172.21.15.131:0/599747892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.074+0000 7f7ddb400700 1 -- 172.21.15.131:0/599747892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8e6a0 msgr2=0x560e34b97c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.074+0000 7f7ddb400700 1 --2- 172.21.15.131:0/599747892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8e6a0 0x560e34b97c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.074+0000 7f7ddb400700 1 -- 172.21.15.131:0/599747892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 msgr2=0x560e34b933f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.074+0000 7f7ddb400700 1 --2- 172.21.15.131:0/599747892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b933f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.074+0000 7f7ddb400700 1 -- 172.21.15.131:0/599747892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0732b0 con 0x560e34b8f570 2024-05-10T05:43:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.074+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/599747892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8e6a0 0x560e34b97c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:16.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.075+0000 7f7ddb400700 1 --2- 172.21.15.131:0/599747892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8f570 0x560e34b9fb20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8010c60 tx=0x7f7dc8001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93933e938197f40e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:16.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.075+0000 7f7dd2000700 1 -- 172.21.15.131:0/599747892 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8026070 con 0x560e34b8f570 2024-05-10T05:43:16.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.075+0000 7f7dd2000700 1 -- 172.21.15.131:0/599747892 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc80036f0 con 0x560e34b8f570 2024-05-10T05:43:16.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.075+0000 7f7dd2000700 1 -- 172.21.15.131:0/599747892 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8010650 con 0x560e34b8f570 2024-05-10T05:43:16.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.076+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/599747892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8f570 msgr2=0x560e34b9fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:16.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.076+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/599747892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8f570 0x560e34b9fb20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8010c60 tx=0x7f7dc8001e70 comp rx=0 tx=0).stop 2024-05-10T05:43:16.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.077+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/599747892 shutdown_connections 2024-05-10T05:43:16.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.077+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/599747892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.077+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/599747892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8e6a0 0x560e34b97c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.077+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/599747892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8f570 0x560e34b9fb20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.077+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/599747892 >> 172.21.15.131:0/599747892 conn(0x560e34af0a60 msgr2=0x560e34b927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.077+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/599747892 shutdown_connections 2024-05-10T05:43:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.077+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/599747892 wait complete. 2024-05-10T05:43:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.078+0000 7f7dfad5fb80 1 -- start start 2024-05-10T05:43:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.078+0000 7f7dfad5fb80 1 Processor -- start 2024-05-10T05:43:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.079+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b9d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.079+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b9d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.079+0000 7f7dfad5fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b9e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.079+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34b9ed40 con 0x560e34b8fec0 2024-05-10T05:43:16.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.079+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34b9eeb0 con 0x560e34b8e6a0 2024-05-10T05:43:16.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.079+0000 7f7dfad5fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e34b9f070 con 0x560e34b8f570 2024-05-10T05:43:16.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.079+0000 7f7dfad5fb80 10 client.0 _ll_register_callbacks cb 0x560e34a53b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:16.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.079+0000 7f7ddb400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b9d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.079+0000 7f7ddb400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b9d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34454/0 (socket says 172.21.15.131:34454) 2024-05-10T05:43:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.079+0000 7f7ddb400700 1 -- 172.21.15.131:0/890876443 learned_addr learned my addr 172.21.15.131:0/890876443 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.080+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b9d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.080+0000 7f7de0e00700 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b9e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.080+0000 7f7ddb400700 1 -- 172.21.15.131:0/890876443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 msgr2=0x560e34b9d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.080+0000 7f7ddb400700 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b9d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.080+0000 7f7ddb400700 1 -- 172.21.15.131:0/890876443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 msgr2=0x560e34b9e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.080+0000 7f7ddb400700 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b9e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.080+0000 7f7ddb400700 1 -- 172.21.15.131:0/890876443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc021070 con 0x560e34b8f570 2024-05-10T05:43:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.080+0000 7f7de0e00700 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b9e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.081+0000 7f7ddb400700 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b9d8d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8010be0 tx=0x7f7dc8005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a2f98cb4cf35e5f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:16.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.081+0000 7f7dc6000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:43:16.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.081+0000 7f7dc6000700 1 -- 172.21.15.131:0/890876443 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8026040 con 0x560e34b8f570 2024-05-10T05:43:16.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.081+0000 7f7dc6000700 1 -- 172.21.15.131:0/890876443 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f7dc8014070 con 0x560e34b8f570 2024-05-10T05:43:16.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.081+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/890876443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dc800eda0 con 0x560e34b8f570 2024-05-10T05:43:16.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dc6000700 1 -- 172.21.15.131:0/890876443 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc80188b0 con 0x560e34b8f570 2024-05-10T05:43:16.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dfad5fb80 10 client.27139 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:16.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/890876443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f7dc80098e0 con 0x560e34b8f570 2024-05-10T05:43:16.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dc6000700 1 -- 172.21.15.131:0/890876443 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc800eda0 con 0x560e34b8f570 2024-05-10T05:43:16.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dc6000700 1 -- 172.21.15.131:0/890876443 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7dc800e350 con 0x560e34b8f570 2024-05-10T05:43:16.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dfad5fb80 10 client.27139 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:16.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dfad5fb80 10 client.27139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:43:16.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dfad5fb80 20 client.27139 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:16.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560e34ba41d0 0x560e34bfcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:16.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dfad5fb80 4 client.27139 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:43:16.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.082+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/890876443 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e34aa6ec0 con 0x560e34ba41d0 2024-05-10T05:43:16.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.083+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560e34ba41d0 0x560e34bfcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.083+0000 7f7dc6000700 10 client.27139 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.083+0000 7f7ddbe00700 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560e34ba41d0 0x560e34bfcc80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80dd69a98d149739 server_cookie=22e0b2ea90825e5c in_seq=0 out_seq=0 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dc6000700 1 -- 172.21.15.131:0/890876443 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560e34aa6ec0 con 0x560e34ba41d0 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dc6000700 10 client.27139 handle_command_reply: tid=0 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 client.27139 shutdown 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/890876443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 msgr2=0x560e34b9d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b9d8d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8010be0 tx=0x7f7dc8005dc0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/890876443 shutdown_connections 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x560e34ba41d0 0x560e34bfcc80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e34b8fec0 0x560e34b9e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e34b8e6a0 0x560e34b9d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 --2- 172.21.15.131:0/890876443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e34b8f570 0x560e34b9d8d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/890876443 >> 172.21.15.131:0/890876443 conn(0x560e34af0a60 msgr2=0x560e34b927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:16.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dc6000700 0 client.27139 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:16.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/890876443 shutdown_connections 2024-05-10T05:43:16.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.084+0000 7f7dfad5fb80 1 -- 172.21.15.131:0/890876443 wait complete. 2024-05-10T05:43:16.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.085+0000 7f7dfad5fb80 20 client.27139 trim_cache size 0 max 16384 2024-05-10T05:43:16.087 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:43:16.087 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:43:16.087 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:43:16.087 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:43:16.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 -- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7dc00a1330 msgr2=0x7f7dc00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:16.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7dc00a1330 0x7f7dc00a3840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x560e34b8e380 tx=0x7f7dd40099e0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 -- 172.21.15.131:0/2209580112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ddc100ac0 msgr2=0x7f7ddc106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:16.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ddc100ac0 0x7f7ddc106630 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0097b0 tx=0x7f7dcc009b70 comp rx=0 tx=0).stop 2024-05-10T05:43:16.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 -- 172.21.15.131:0/2209580112 shutdown_connections 2024-05-10T05:43:16.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f7dc00a1330 0x7f7dc00a3840 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ddc102580 0x7f7ddc103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ddc100ac0 0x7f7ddc106630 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 --2- 172.21.15.131:0/2209580112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ddc101be0 0x7f7ddc106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 -- 172.21.15.131:0/2209580112 >> 172.21.15.131:0/2209580112 conn(0x7f7ddc0fc0d0 msgr2=0x7f7ddc190aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.086+0000 7f7de2e00700 1 -- 172.21.15.131:0/2209580112 shutdown_connections 2024-05-10T05:43:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:16.087+0000 7f7de2e00700 1 -- 172.21.15.131:0/2209580112 wait complete. 2024-05-10T05:43:16.209 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:43:16.209 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:43:16.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:16 smithi142 ceph-mon[77667]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 90 op/s 2024-05-10T05:43:16.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:16 smithi131 ceph-mon[37264]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 90 op/s 2024-05-10T05:43:17.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.026+0000 7f267a200700 1 -- 172.21.15.131:0/1595845773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674105760 msgr2=0x7f2674105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:17.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.026+0000 7f267a200700 1 --2- 172.21.15.131:0/1595845773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674105760 0x7f2674105b70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f266c0098f0 tx=0x7f266c009c00 comp rx=0 tx=0).stop 2024-05-10T05:43:17.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.031+0000 7f267a200700 1 -- 172.21.15.131:0/1595845773 shutdown_connections 2024-05-10T05:43:17.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.031+0000 7f267a200700 1 --2- 172.21.15.131:0/1595845773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26741004a0 0x7f26741028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:17.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.031+0000 7f267a200700 1 --2- 172.21.15.131:0/1595845773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674105760 0x7f2674105b70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:17.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.031+0000 7f267a200700 1 --2- 172.21.15.131:0/1595845773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2674102e00 0x7f2674105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:17.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.032+0000 7f267a200700 1 -- 172.21.15.131:0/1595845773 >> 172.21.15.131:0/1595845773 conn(0x7f26740fc0d0 msgr2=0x7f26740fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:17.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.035+0000 7f267a200700 1 -- 172.21.15.131:0/1595845773 shutdown_connections 2024-05-10T05:43:17.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.035+0000 7f267a200700 1 -- 172.21.15.131:0/1595845773 wait complete. 2024-05-10T05:43:17.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:16 smithi157 ceph-mon[78427]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 90 op/s 2024-05-10T05:43:17.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.050+0000 7f267a200700 1 Processor -- start 2024-05-10T05:43:17.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.056+0000 7f267a200700 1 -- start start 2024-05-10T05:43:17.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.057+0000 7f267a200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26741004a0 0x7f267410be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.064+0000 7f267a200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674102e00 0x7f267410c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.064+0000 7f267a200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2674105760 0x7f267410d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.064+0000 7f267a200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267410d910 con 0x7f26741004a0 2024-05-10T05:43:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.064+0000 7f267a200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267410da80 con 0x7f2674105760 2024-05-10T05:43:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.064+0000 7f267a200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267410dc40 con 0x7f2674102e00 2024-05-10T05:43:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.064+0000 7f2672a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674102e00 0x7f267410c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.064+0000 7f2672a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674102e00 0x7f267410c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34458/0 (socket says 172.21.15.131:34458) 2024-05-10T05:43:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.064+0000 7f2672a00700 1 -- 172.21.15.131:0/316038792 learned_addr learned my addr 172.21.15.131:0/316038792 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.065+0000 7f2672a00700 1 -- 172.21.15.131:0/316038792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2674105760 msgr2=0x7f267410d300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:43:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.065+0000 7f2672a00700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2674105760 0x7f267410d300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.065+0000 7f2672a00700 1 -- 172.21.15.131:0/316038792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26741004a0 msgr2=0x7f267410be60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:43:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.065+0000 7f2672a00700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26741004a0 0x7f267410be60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.065+0000 7f2672a00700 1 -- 172.21.15.131:0/316038792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2664009d00 con 0x7f2674102e00 2024-05-10T05:43:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.065+0000 7f2672a00700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674102e00 0x7f267410c3a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f266400e3c0 tx=0x7f266400e780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.068+0000 7f266be00700 1 -- 172.21.15.131:0/316038792 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664018070 con 0x7f2674102e00 2024-05-10T05:43:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.068+0000 7f266be00700 1 -- 172.21.15.131:0/316038792 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f266400f040 con 0x7f2674102e00 2024-05-10T05:43:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.068+0000 7f266be00700 1 -- 172.21.15.131:0/316038792 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664004480 con 0x7f2674102e00 2024-05-10T05:43:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.068+0000 7f267a200700 1 -- 172.21.15.131:0/316038792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c009520 con 0x7f2674102e00 2024-05-10T05:43:17.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.070+0000 7f267a200700 1 -- 172.21.15.131:0/316038792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267410e1f0 con 0x7f2674102e00 2024-05-10T05:43:17.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.074+0000 7f267a200700 1 -- 172.21.15.131:0/316038792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26740694d0 con 0x7f2674102e00 2024-05-10T05:43:17.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.087+0000 7f266be00700 1 -- 172.21.15.131:0/316038792 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2664003a00 con 0x7f2674102e00 2024-05-10T05:43:17.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.090+0000 7f266be00700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f26580a12a0 0x7f26580a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:17.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.099+0000 7f2673400700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f26580a12a0 0x7f26580a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:17.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.099+0000 7f266be00700 1 -- 172.21.15.131:0/316038792 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2664099f50 con 0x7f2674102e00 2024-05-10T05:43:17.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.100+0000 7f266be00700 1 -- 172.21.15.131:0/316038792 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2664096320 con 0x7f2674102e00 2024-05-10T05:43:17.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.134+0000 7f2673400700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f26580a12a0 0x7f26580a37b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2674068de0 tx=0x7f266000b3f0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:17.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.952+0000 7f267a200700 1 -- 172.21.15.131:0/316038792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2674113a10 con 0x7f2674102e00 2024-05-10T05:43:17.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.953+0000 7f266be00700 1 -- 172.21.15.131:0/316038792 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f26640964c0 con 0x7f2674102e00 2024-05-10T05:43:17.954 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:43:17.954 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:43:17.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.960+0000 7f267a200700 1 -- 172.21.15.131:0/316038792 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f26580a12a0 msgr2=0x7f26580a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:17.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.960+0000 7f267a200700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f26580a12a0 0x7f26580a37b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2674068de0 tx=0x7f266000b3f0 comp rx=0 tx=0).stop 2024-05-10T05:43:17.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.960+0000 7f267a200700 1 -- 172.21.15.131:0/316038792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674102e00 msgr2=0x7f267410c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:17.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.960+0000 7f267a200700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674102e00 0x7f267410c3a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f266400e3c0 tx=0x7f266400e780 comp rx=0 tx=0).stop 2024-05-10T05:43:17.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.961+0000 7f267a200700 1 -- 172.21.15.131:0/316038792 shutdown_connections 2024-05-10T05:43:17.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.961+0000 7f267a200700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f26580a12a0 0x7f26580a37b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:17.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.961+0000 7f267a200700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26741004a0 0x7f267410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:17.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.961+0000 7f267a200700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2674105760 0x7f267410d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:17.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.961+0000 7f267a200700 1 --2- 172.21.15.131:0/316038792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2674102e00 0x7f267410c3a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:17.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.961+0000 7f267a200700 1 -- 172.21.15.131:0/316038792 >> 172.21.15.131:0/316038792 conn(0x7f26740fc0d0 msgr2=0x7f2674101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:17.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.961+0000 7f267a200700 1 -- 172.21.15.131:0/316038792 shutdown_connections 2024-05-10T05:43:17.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:17 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:43:17.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:17 smithi131 ceph-mon[37264]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 235 MiB/s wr, 74 op/s 2024-05-10T05:43:17.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:17.965+0000 7f267a200700 1 -- 172.21.15.131:0/316038792 wait complete. 2024-05-10T05:43:17.970 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:43:18.035 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:43:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:17 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:43:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:17 smithi157 ceph-mon[78427]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 235 MiB/s wr, 74 op/s 2024-05-10T05:43:18.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:17 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:43:18.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:17 smithi142 ceph-mon[77667]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 235 MiB/s wr, 74 op/s 2024-05-10T05:43:18.679 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:18 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/316038792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:43:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.817+0000 7eff9fe00700 1 -- 172.21.15.131:0/1654493080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa01087e0 msgr2=0x7effa0108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.817+0000 7eff9fe00700 1 --2- 172.21.15.131:0/1654493080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa01087e0 0x7effa0108c30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7eff88009990 tx=0x7eff88009ca0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.818+0000 7eff9fe00700 1 -- 172.21.15.131:0/1654493080 shutdown_connections 2024-05-10T05:43:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.818+0000 7eff9fe00700 1 --2- 172.21.15.131:0/1654493080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effa010fec0 0x7effa0112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.818+0000 7eff9fe00700 1 --2- 172.21.15.131:0/1654493080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa0109730 0x7effa0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.818+0000 7eff9fe00700 1 --2- 172.21.15.131:0/1654493080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa01087e0 0x7effa0108c30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.818+0000 7eff9fe00700 1 -- 172.21.15.131:0/1654493080 >> 172.21.15.131:0/1654493080 conn(0x7effa006e800 msgr2=0x7effa0070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.819+0000 7eff9fe00700 1 -- 172.21.15.131:0/1654493080 shutdown_connections 2024-05-10T05:43:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.819+0000 7eff9fe00700 1 -- 172.21.15.131:0/1654493080 wait complete. 2024-05-10T05:43:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.825+0000 7eff9fe00700 1 Processor -- start 2024-05-10T05:43:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.828+0000 7eff9fe00700 1 -- start start 2024-05-10T05:43:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.828+0000 7eff9fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa01087e0 0x7effa010b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.828+0000 7eff9fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa0109730 0x7effa0109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.828+0000 7eff9fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effa010fec0 0x7effa010a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.828+0000 7eff9fe00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa01a43d0 con 0x7effa010fec0 2024-05-10T05:43:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.829+0000 7eff9fe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa01a4540 con 0x7effa0109730 2024-05-10T05:43:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.829+0000 7eff9fe00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa01a4700 con 0x7effa01087e0 2024-05-10T05:43:18.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.830+0000 7eff9f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effa010fec0 0x7effa010a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.831+0000 7eff9f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effa010fec0 0x7effa010a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51702/0 (socket says 172.21.15.131:51702) 2024-05-10T05:43:18.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.831+0000 7eff9f400700 1 -- 172.21.15.131:0/3111216440 learned_addr learned my addr 172.21.15.131:0/3111216440 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:18.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.831+0000 7eff9f400700 1 -- 172.21.15.131:0/3111216440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa01087e0 msgr2=0x7effa010b630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:43:18.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.831+0000 7eff9f400700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa01087e0 0x7effa010b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.831+0000 7eff9f400700 1 -- 172.21.15.131:0/3111216440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa0109730 msgr2=0x7effa0109cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:43:18.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.831+0000 7eff9f400700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa0109730 0x7effa0109cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.832+0000 7eff9f400700 1 -- 172.21.15.131:0/3111216440 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff880095f0 con 0x7effa010fec0 2024-05-10T05:43:18.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.837+0000 7eff9f400700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effa010fec0 0x7effa010a210 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7eff98007fa0 tx=0x7eff9800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:18.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.838+0000 7eff97e00700 1 -- 172.21.15.131:0/3111216440 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98017070 con 0x7effa010fec0 2024-05-10T05:43:18.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.838+0000 7eff97e00700 1 -- 172.21.15.131:0/3111216440 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff9800f040 con 0x7effa010fec0 2024-05-10T05:43:18.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.838+0000 7eff9fe00700 1 -- 172.21.15.131:0/3111216440 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa01a49e0 con 0x7effa010fec0 2024-05-10T05:43:18.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.838+0000 7eff9fe00700 1 -- 172.21.15.131:0/3111216440 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa01a4ef0 con 0x7effa010fec0 2024-05-10T05:43:18.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.839+0000 7eff97e00700 1 -- 172.21.15.131:0/3111216440 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98017070 con 0x7effa010fec0 2024-05-10T05:43:18.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.851+0000 7eff97e00700 1 -- 172.21.15.131:0/3111216440 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7eff98004ad0 con 0x7effa010fec0 2024-05-10T05:43:18.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.854+0000 7eff97e00700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff840a11d0 0x7eff840a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.857+0000 7effbd634b80 1 Processor -- start 2024-05-10T05:43:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.857+0000 7eff9ea00700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff840a11d0 0x7eff840a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.857+0000 7eff97e00700 1 -- 172.21.15.131:0/3111216440 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7eff98022250 con 0x7effa010fec0 2024-05-10T05:43:18.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.863+0000 7effbd634b80 1 -- start start 2024-05-10T05:43:18.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.866+0000 7effbd634b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f357960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.866+0000 7effbd634b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3587a0 0x56283f358bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.866+0000 7effbd634b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f35f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.866+0000 7effbd634b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f27e150 con 0x56283f357530 2024-05-10T05:43:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.866+0000 7effbd634b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f27bb10 con 0x56283f3590f0 2024-05-10T05:43:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.866+0000 7effbd634b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f277400 con 0x56283f3587a0 2024-05-10T05:43:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.866+0000 7eff9ea00700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff840a11d0 0x7eff840a36e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7effa0072e70 tx=0x7eff90009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.867+0000 7eff9f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f35f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.867+0000 7eff9f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f35f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38588/0 (socket says 172.21.15.131:38588) 2024-05-10T05:43:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.867+0000 7eff9f400700 1 -- 172.21.15.131:0/3015919509 learned_addr learned my addr 172.21.15.131:0/3015919509 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.867+0000 7eff9f400700 1 -- 172.21.15.131:0/3015919509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3587a0 msgr2=0x56283f358bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:43:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.867+0000 7eff9f400700 1 --2- 172.21.15.131:0/3015919509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3587a0 0x56283f358bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.867+0000 7eff9f400700 1 -- 172.21.15.131:0/3015919509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 msgr2=0x56283f357960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.867+0000 7eff9f400700 1 --2- 172.21.15.131:0/3015919509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f357960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.867+0000 7eff9f400700 1 -- 172.21.15.131:0/3015919509 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56283f0f83b0 con 0x56283f3590f0 2024-05-10T05:43:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.868+0000 7eff9f400700 1 --2- 172.21.15.131:0/3015919509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f35f9f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7eff980a10d0 tx=0x7eff98099900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a03db3dc64e24326 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:18.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.881+0000 7eff8fe00700 1 -- 172.21.15.131:0/3015919509 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98017070 con 0x56283f3590f0 2024-05-10T05:43:18.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.881+0000 7eff8fe00700 1 -- 172.21.15.131:0/3015919509 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff980a1600 con 0x56283f3590f0 2024-05-10T05:43:18.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.881+0000 7eff8fe00700 1 -- 172.21.15.131:0/3015919509 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980988d0 con 0x56283f3590f0 2024-05-10T05:43:18.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.883+0000 7effbd634b80 1 -- 172.21.15.131:0/3015919509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 msgr2=0x56283f35f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:18.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.883+0000 7effbd634b80 1 --2- 172.21.15.131:0/3015919509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f35f9f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7eff980a10d0 tx=0x7eff98099900 comp rx=0 tx=0).stop 2024-05-10T05:43:18.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.885+0000 7effbd634b80 1 -- 172.21.15.131:0/3015919509 shutdown_connections 2024-05-10T05:43:18.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.885+0000 7effbd634b80 1 --2- 172.21.15.131:0/3015919509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f357960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.885+0000 7effbd634b80 1 --2- 172.21.15.131:0/3015919509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f35f9f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.885+0000 7effbd634b80 1 --2- 172.21.15.131:0/3015919509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3587a0 0x56283f358bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.885+0000 7effbd634b80 1 -- 172.21.15.131:0/3015919509 >> 172.21.15.131:0/3015919509 conn(0x56283f2b9a60 msgr2=0x56283f352330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:18.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.886+0000 7effbd634b80 1 -- 172.21.15.131:0/3015919509 shutdown_connections 2024-05-10T05:43:18.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.886+0000 7effbd634b80 1 -- 172.21.15.131:0/3015919509 wait complete. 2024-05-10T05:43:18.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.887+0000 7effbd634b80 1 -- start start 2024-05-10T05:43:18.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.889+0000 7effbd634b80 1 Processor -- start 2024-05-10T05:43:18.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.889+0000 7effbd634b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f3619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.890+0000 7effbd634b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3587a0 0x56283f361ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.890+0000 7eff9ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f3619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.890+0000 7eff9ea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f3619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51712/0 (socket says 172.21.15.131:51712) 2024-05-10T05:43:18.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.890+0000 7eff96000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3587a0 0x56283f361ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.890+0000 7eff9ea00700 1 -- 172.21.15.131:0/2100564528 learned_addr learned my addr 172.21.15.131:0/2100564528 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.890+0000 7effbd634b80 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 0x56283f368e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.895+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f213560 con 0x56283f357530 2024-05-10T05:43:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.895+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f213200 con 0x56283f3587a0 2024-05-10T05:43:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.895+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f0fa6a0 con 0x56283f3590f0 2024-05-10T05:43:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.895+0000 7effbd634b80 10 client.0 _ll_register_callbacks cb 0x56283f21cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:18.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.895+0000 7eff9f400700 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 0x56283f368e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.896+0000 7eff9f400700 1 -- 172.21.15.131:0/2100564528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3587a0 msgr2=0x56283f361ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:18.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.896+0000 7eff9f400700 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3587a0 0x56283f361ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.896+0000 7eff9f400700 1 -- 172.21.15.131:0/2100564528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 msgr2=0x56283f3619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:18.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.896+0000 7eff9f400700 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f3619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.896+0000 7eff9f400700 1 -- 172.21.15.131:0/2100564528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa01a4ef0 con 0x56283f3590f0 2024-05-10T05:43:18.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.897+0000 7eff9f400700 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 0x56283f368e50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7eff980a10d0 tx=0x7eff98097730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa54c0549b23adb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:18.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.897+0000 7eff8cc00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:43:18.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.897+0000 7eff8cc00700 1 -- 172.21.15.131:0/2100564528 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98017040 con 0x56283f3590f0 2024-05-10T05:43:18.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.897+0000 7eff8cc00700 1 -- 172.21.15.131:0/2100564528 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7eff9809b030 con 0x56283f3590f0 2024-05-10T05:43:18.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.897+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56283f1d0270 con 0x56283f3590f0 2024-05-10T05:43:18.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.898+0000 7eff8cc00700 1 -- 172.21.15.131:0/2100564528 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff980a1370 con 0x56283f3590f0 2024-05-10T05:43:18.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.898+0000 7effbd634b80 10 client.27151 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:18.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.898+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56283f0a4b60 con 0x56283f3590f0 2024-05-10T05:43:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.899+0000 7eff8cc00700 1 -- 172.21.15.131:0/2100564528 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56283f1d0270 con 0x56283f3590f0 2024-05-10T05:43:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.899+0000 7eff8cc00700 1 -- 172.21.15.131:0/2100564528 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7eff98079070 con 0x56283f3590f0 2024-05-10T05:43:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.899+0000 7effbd634b80 10 client.27151 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.899+0000 7effbd634b80 10 client.27151 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:43:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.899+0000 7effbd634b80 20 client.27151 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.899+0000 7effbd634b80 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x56283f36d290 0x56283f36f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.899+0000 7effbd634b80 4 client.27151 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:43:18.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.899+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56283f326780 con 0x56283f36d290 2024-05-10T05:43:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.901+0000 7eff9ea00700 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x56283f36d290 0x56283f36f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.902+0000 7eff9ea00700 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x56283f36d290 0x56283f36f6e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25a7188ee3deae9c server_cookie=bfc9311c644c99a7 in_seq=0 out_seq=0 2024-05-10T05:43:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.902+0000 7eff8cc00700 10 client.27151 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:43:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.902+0000 7eff8cc00700 1 -- 172.21.15.131:0/2100564528 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56283f326780 con 0x56283f36d290 2024-05-10T05:43:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.902+0000 7eff8cc00700 10 client.27151 handle_command_reply: tid=0 2024-05-10T05:43:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.903+0000 7effbd634b80 1 client.27151 shutdown 2024-05-10T05:43:18.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.903+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 msgr2=0x56283f368e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:18.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.903+0000 7effbd634b80 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 0x56283f368e50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7eff980a10d0 tx=0x7eff98097730 comp rx=0 tx=0).stop 2024-05-10T05:43:18.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.904+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 shutdown_connections 2024-05-10T05:43:18.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.904+0000 7effbd634b80 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x56283f36d290 0x56283f36f6e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.904+0000 7effbd634b80 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f3619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.904+0000 7effbd634b80 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 0x56283f368e50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.904+0000 7effbd634b80 1 --2- 172.21.15.131:0/2100564528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3587a0 0x56283f361ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.904+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 >> 172.21.15.131:0/2100564528 conn(0x56283f2b9a60 msgr2=0x56283f356060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:18.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.904+0000 7eff8cc00700 0 client.27151 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:43:18.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.904+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 shutdown_connections 2024-05-10T05:43:18.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.904+0000 7effbd634b80 1 -- 172.21.15.131:0/2100564528 wait complete. 2024-05-10T05:43:18.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.906+0000 7effbd634b80 20 client.27151 trim_cache size 0 max 16384 2024-05-10T05:43:18.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.966+0000 7effbd634b80 1 Processor -- start 2024-05-10T05:43:18.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.968+0000 7effbd634b80 1 -- start start 2024-05-10T05:43:18.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.969+0000 7effbd634b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f357530 0x56283f36f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.969+0000 7effbd634b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f3587a0 0x56283f368e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.969+0000 7eff9ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f357530 0x56283f36f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.969+0000 7eff9ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f357530 0x56283f36f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38606/0 (socket says 172.21.15.131:38606) 2024-05-10T05:43:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.970+0000 7eff9ea00700 1 -- 172.21.15.131:0/1656258576 learned_addr learned my addr 172.21.15.131:0/1656258576 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.970+0000 7eff96000700 1 --2- 172.21.15.131:0/1656258576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f3587a0 0x56283f368e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.970+0000 7effbd634b80 1 --2- 172.21.15.131:0/1656258576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 0x56283f35fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.970+0000 7effbd634b80 1 -- 172.21.15.131:0/1656258576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f357dc0 con 0x56283f3587a0 2024-05-10T05:43:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.970+0000 7effbd634b80 1 -- 172.21.15.131:0/1656258576 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f35d020 con 0x56283f357530 2024-05-10T05:43:18.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.970+0000 7effbd634b80 1 -- 172.21.15.131:0/1656258576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f35f980 con 0x56283f3590f0 2024-05-10T05:43:18.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.970+0000 7eff9f400700 1 --2- 172.21.15.131:0/1656258576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 0x56283f35fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.970+0000 7eff9ea00700 1 -- 172.21.15.131:0/1656258576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 msgr2=0x56283f35fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:18.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.971+0000 7eff9ea00700 1 --2- 172.21.15.131:0/1656258576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 0x56283f35fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.971+0000 7eff9ea00700 1 -- 172.21.15.131:0/1656258576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f3587a0 msgr2=0x56283f368e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:18.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.971+0000 7eff9ea00700 1 --2- 172.21.15.131:0/1656258576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f3587a0 0x56283f368e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.971+0000 7eff9ea00700 1 -- 172.21.15.131:0/1656258576 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56283f23d560 con 0x56283f357530 2024-05-10T05:43:18.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.974+0000 7eff9ea00700 1 --2- 172.21.15.131:0/1656258576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f357530 0x56283f36f9b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7eff90025be0 tx=0x7eff90011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42948e41c3bdee53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:18.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.975+0000 7eff94c00700 1 -- 172.21.15.131:0/1656258576 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90022040 con 0x56283f357530 2024-05-10T05:43:18.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.975+0000 7eff94c00700 1 -- 172.21.15.131:0/1656258576 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff9000eeb0 con 0x56283f357530 2024-05-10T05:43:18.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.975+0000 7eff94c00700 1 -- 172.21.15.131:0/1656258576 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9001dd00 con 0x56283f357530 2024-05-10T05:43:18.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.978+0000 7effbd634b80 1 -- 172.21.15.131:0/1656258576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f357530 msgr2=0x56283f36f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:18.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.978+0000 7effbd634b80 1 --2- 172.21.15.131:0/1656258576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f357530 0x56283f36f9b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7eff90025be0 tx=0x7eff90011000 comp rx=0 tx=0).stop 2024-05-10T05:43:18.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.979+0000 7effbd634b80 1 -- 172.21.15.131:0/1656258576 shutdown_connections 2024-05-10T05:43:18.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.979+0000 7effbd634b80 1 --2- 172.21.15.131:0/1656258576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3590f0 0x56283f35fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.979+0000 7effbd634b80 1 --2- 172.21.15.131:0/1656258576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f357530 0x56283f36f9b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.979+0000 7effbd634b80 1 --2- 172.21.15.131:0/1656258576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f3587a0 0x56283f368e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.979+0000 7effbd634b80 1 -- 172.21.15.131:0/1656258576 >> 172.21.15.131:0/1656258576 conn(0x56283f2b9a60 msgr2=0x56283f36f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:18.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.979+0000 7effbd634b80 1 -- 172.21.15.131:0/1656258576 shutdown_connections 2024-05-10T05:43:18.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.983+0000 7effbd634b80 1 -- 172.21.15.131:0/1656258576 wait complete. 2024-05-10T05:43:18.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.983+0000 7effbd634b80 1 -- start start 2024-05-10T05:43:18.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.984+0000 7effbd634b80 1 Processor -- start 2024-05-10T05:43:18.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.984+0000 7effbd634b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f366290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.984+0000 7effbd634b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3587a0 0x56283f3667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.984+0000 7effbd634b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f367700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.984+0000 7effbd634b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f367c40 con 0x56283f357530 2024-05-10T05:43:18.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.984+0000 7effbd634b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f367db0 con 0x56283f3590f0 2024-05-10T05:43:18.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.984+0000 7effbd634b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56283f367f70 con 0x56283f3587a0 2024-05-10T05:43:18.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.984+0000 7effbd634b80 10 client.0 _ll_register_callbacks cb 0x56283f21cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:18.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.985+0000 7eff9f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f367700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.985+0000 7eff96000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3587a0 0x56283f3667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.985+0000 7eff9f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f367700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38608/0 (socket says 172.21.15.131:38608) 2024-05-10T05:43:18.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.985+0000 7eff9f400700 1 -- 172.21.15.131:0/383841912 learned_addr learned my addr 172.21.15.131:0/383841912 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:18.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.986+0000 7eff9ea00700 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f366290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.986+0000 7eff9f400700 1 -- 172.21.15.131:0/383841912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3587a0 msgr2=0x56283f3667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:18.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.986+0000 7eff9f400700 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3587a0 0x56283f3667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.986+0000 7eff9f400700 1 -- 172.21.15.131:0/383841912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 msgr2=0x56283f366290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:18.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.986+0000 7eff9f400700 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f366290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.986+0000 7eff9f400700 1 -- 172.21.15.131:0/383841912 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9000e8a0 con 0x56283f3590f0 2024-05-10T05:43:18.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.986+0000 7eff9ea00700 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f366290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:43:18.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.987+0000 7eff9f400700 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f367700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7eff98098e80 tx=0x7eff980a13a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0809cc7b56b2096 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:18.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.987+0000 7eff8cc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:43:18.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.988+0000 7eff8cc00700 1 -- 172.21.15.131:0/383841912 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98017030 con 0x56283f3590f0 2024-05-10T05:43:18.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.988+0000 7eff8cc00700 1 -- 172.21.15.131:0/383841912 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7eff9809b070 con 0x56283f3590f0 2024-05-10T05:43:18.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.988+0000 7effbd634b80 1 -- 172.21.15.131:0/383841912 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff980a1600 con 0x56283f3590f0 2024-05-10T05:43:18.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.988+0000 7eff8cc00700 1 -- 172.21.15.131:0/383841912 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff980980e0 con 0x56283f3590f0 2024-05-10T05:43:18.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.988+0000 7effbd634b80 10 client.27125 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:18.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.988+0000 7effbd634b80 1 -- 172.21.15.131:0/383841912 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56283f0fccd0 con 0x56283f3590f0 2024-05-10T05:43:18.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.989+0000 7eff8cc00700 1 -- 172.21.15.131:0/383841912 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff980a1600 con 0x56283f3590f0 2024-05-10T05:43:18.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.990+0000 7eff8cc00700 1 -- 172.21.15.131:0/383841912 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7eff98079070 con 0x56283f3590f0 2024-05-10T05:43:18.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.991+0000 7effbd634b80 10 client.27125 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:18.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.992+0000 7effbd634b80 10 client.27125 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:43:18.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.992+0000 7effbd634b80 20 client.27125 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:18.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.992+0000 7effbd634b80 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x56283f3e5750 0x56283f3e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:18.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.994+0000 7effbd634b80 4 client.27125 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:43:18.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.994+0000 7effbd634b80 1 -- 172.21.15.131:0/383841912 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56283f26fec0 con 0x56283f3e5750 2024-05-10T05:43:18.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.996+0000 7eff9ea00700 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x56283f3e5750 0x56283f3e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:18.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.997+0000 7eff9ea00700 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x56283f3e5750 0x56283f3e7b70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34116634f4556307 server_cookie=5db21402b5b8b893 in_seq=0 out_seq=0 2024-05-10T05:43:18.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.997+0000 7eff8cc00700 10 client.27125 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:43:18.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.998+0000 7eff8cc00700 1 -- 172.21.15.131:0/383841912 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56283f26fec0 con 0x56283f3e5750 2024-05-10T05:43:18.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.998+0000 7eff8cc00700 10 client.27125 handle_command_reply: tid=0 2024-05-10T05:43:18.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.998+0000 7effbd634b80 1 client.27125 shutdown 2024-05-10T05:43:18.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.998+0000 7effbd634b80 1 -- 172.21.15.131:0/383841912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 msgr2=0x56283f367700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:18.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.998+0000 7effbd634b80 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f367700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7eff98098e80 tx=0x7eff980a13a0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.999+0000 7effbd634b80 1 -- 172.21.15.131:0/383841912 shutdown_connections 2024-05-10T05:43:18.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.999+0000 7effbd634b80 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x56283f3e5750 0x56283f3e7b70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.999+0000 7effbd634b80 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56283f357530 0x56283f366290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.999+0000 7effbd634b80 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56283f3590f0 0x56283f367700 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.999+0000 7effbd634b80 1 --2- 172.21.15.131:0/383841912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56283f3587a0 0x56283f3667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:18.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:18.999+0000 7effbd634b80 1 -- 172.21.15.131:0/383841912 >> 172.21.15.131:0/383841912 conn(0x56283f2b9a60 msgr2=0x56283f351bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:19.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.001+0000 7eff8cc00700 0 client.27125 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:43:19.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.001+0000 7effbd634b80 1 -- 172.21.15.131:0/383841912 shutdown_connections 2024-05-10T05:43:19.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.002+0000 7effbd634b80 1 -- 172.21.15.131:0/383841912 wait complete. 2024-05-10T05:43:19.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.002+0000 7effbd634b80 20 client.27125 trim_cache size 0 max 16384 2024-05-10T05:43:19.003 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:43:19.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.007+0000 7eff9fe00700 1 -- 172.21.15.131:0/3111216440 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff840a11d0 msgr2=0x7eff840a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:19.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.007+0000 7eff9fe00700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff840a11d0 0x7eff840a36e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7effa0072e70 tx=0x7eff90009260 comp rx=0 tx=0).stop 2024-05-10T05:43:19.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.007+0000 7eff9fe00700 1 -- 172.21.15.131:0/3111216440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effa010fec0 msgr2=0x7effa010a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:19.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.007+0000 7eff9fe00700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effa010fec0 0x7effa010a210 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7eff98007fa0 tx=0x7eff9800d460 comp rx=0 tx=0).stop 2024-05-10T05:43:19.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.008+0000 7eff9fe00700 1 -- 172.21.15.131:0/3111216440 shutdown_connections 2024-05-10T05:43:19.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.008+0000 7eff9fe00700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7eff840a11d0 0x7eff840a36e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.008+0000 7eff9fe00700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effa010fec0 0x7effa010a210 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.009+0000 7eff9fe00700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effa01087e0 0x7effa010b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.009+0000 7eff9fe00700 1 --2- 172.21.15.131:0/3111216440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa0109730 0x7effa0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.009+0000 7eff9fe00700 1 -- 172.21.15.131:0/3111216440 >> 172.21.15.131:0/3111216440 conn(0x7effa006e800 msgr2=0x7effa010bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:19.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.009+0000 7eff9fe00700 1 -- 172.21.15.131:0/3111216440 shutdown_connections 2024-05-10T05:43:19.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.009+0000 7eff9fe00700 1 -- 172.21.15.131:0/3111216440 wait complete. 2024-05-10T05:43:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:18 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/316038792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:43:19.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:18 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/316038792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:43:19.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:43:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.923+0000 7f0583000700 1 -- 172.21.15.131:0/1468904663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f057c073b40 msgr2=0x7f057c074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.923+0000 7f0583000700 1 --2- 172.21.15.131:0/1468904663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f057c073b40 0x7f057c074010 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f056c009990 tx=0x7f056c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.925+0000 7f0583000700 1 -- 172.21.15.131:0/1468904663 shutdown_connections 2024-05-10T05:43:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.925+0000 7f0583000700 1 --2- 172.21.15.131:0/1468904663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f057c10e220 0x7f057c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.925+0000 7f0583000700 1 --2- 172.21.15.131:0/1468904663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f057c075140 0x7f057c073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.925+0000 7f0583000700 1 --2- 172.21.15.131:0/1468904663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f057c073b40 0x7f057c074010 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.925+0000 7f0583000700 1 -- 172.21.15.131:0/1468904663 >> 172.21.15.131:0/1468904663 conn(0x7f057c0fd210 msgr2=0x7f057c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.927+0000 7f0583000700 1 -- 172.21.15.131:0/1468904663 shutdown_connections 2024-05-10T05:43:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.927+0000 7f0583000700 1 -- 172.21.15.131:0/1468904663 wait complete. 2024-05-10T05:43:19.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.933+0000 7f0583000700 1 Processor -- start 2024-05-10T05:43:19.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.934+0000 7f0583000700 1 -- start start 2024-05-10T05:43:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.934+0000 7f0583000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f057c073b40 0x7f057c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.934+0000 7f0583000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f057c075140 0x7f057c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.934+0000 7f0583000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f057c10e220 0x7f057c1a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.934+0000 7f0583000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c1a60a0 con 0x7f057c073b40 2024-05-10T05:43:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.934+0000 7f0583000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c06ac30 con 0x7f057c10e220 2024-05-10T05:43:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.934+0000 7f0583000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c06adf0 con 0x7f057c075140 2024-05-10T05:43:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.935+0000 7f057be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f057c073b40 0x7f057c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.935+0000 7f057be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f057c073b40 0x7f057c19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51758/0 (socket says 172.21.15.131:51758) 2024-05-10T05:43:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.935+0000 7f057be00700 1 -- 172.21.15.131:0/3856013444 learned_addr learned my addr 172.21.15.131:0/3856013444 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.935+0000 7f0581000700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f057c10e220 0x7f057c1a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.936+0000 7f057be00700 1 -- 172.21.15.131:0/3856013444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f057c075140 msgr2=0x7f057c19eb40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.936+0000 7f057be00700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f057c075140 0x7f057c19eb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.936+0000 7f057be00700 1 -- 172.21.15.131:0/3856013444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f057c10e220 msgr2=0x7f057c1a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.936+0000 7f057be00700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f057c10e220 0x7f057c1a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.936+0000 7f057be00700 1 -- 172.21.15.131:0/3856013444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c0095f0 con 0x7f057c073b40 2024-05-10T05:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.936+0000 7f0581000700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f057c10e220 0x7f057c1a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:19.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.945+0000 7f057be00700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f057c073b40 0x7f057c19e600 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f056800b750 tx=0x7f056800bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:19.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.949+0000 7f0578c00700 1 -- 172.21.15.131:0/3856013444 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056800d660 con 0x7f057c073b40 2024-05-10T05:43:19.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.949+0000 7f0578c00700 1 -- 172.21.15.131:0/3856013444 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f056800dc80 con 0x7f057c073b40 2024-05-10T05:43:19.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.949+0000 7f0578c00700 1 -- 172.21.15.131:0/3856013444 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056800eda0 con 0x7f057c073b40 2024-05-10T05:43:19.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.949+0000 7f0583000700 1 -- 172.21.15.131:0/3856013444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c06b0d0 con 0x7f057c073b40 2024-05-10T05:43:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.952+0000 7f0583000700 1 -- 172.21.15.131:0/3856013444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c06b5b0 con 0x7f057c073b40 2024-05-10T05:43:19.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:19 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:43:19.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:19 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:43:19.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:19 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:43:19.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:19 smithi131 ceph-mon[37264]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 268 MiB/s wr, 82 op/s 2024-05-10T05:43:19.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.960+0000 7f0578c00700 1 -- 172.21.15.131:0/3856013444 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f056803e580 con 0x7f057c073b40 2024-05-10T05:43:19.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.962+0000 7f0578c00700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05600a1330 0x7f05600a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:19.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.963+0000 7f0578c00700 1 -- 172.21.15.131:0/3856013444 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f056800d820 con 0x7f057c073b40 2024-05-10T05:43:19.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.967+0000 7f059af7eb80 1 Processor -- start 2024-05-10T05:43:19.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.968+0000 7f059af7eb80 1 -- start start 2024-05-10T05:43:19.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.969+0000 7f059af7eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e5406a0 0x56336e540af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:19.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.969+0000 7f059af7eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 0x56336e541980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:19.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.969+0000 7f057be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e5406a0 0x56336e540af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:19.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.969+0000 7f057be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e5406a0 0x56336e540af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38638/0 (socket says 172.21.15.131:38638) 2024-05-10T05:43:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.970+0000 7f057be00700 1 -- 172.21.15.131:0/442766104 learned_addr learned my addr 172.21.15.131:0/442766104 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.970+0000 7f057b400700 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 0x56336e541980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.970+0000 7f059af7eb80 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e5487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:19.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.971+0000 7f0581000700 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e5487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:19.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.971+0000 7f059af7eb80 1 -- 172.21.15.131:0/442766104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e467150 con 0x56336e541ec0 2024-05-10T05:43:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.972+0000 7f059af7eb80 1 -- 172.21.15.131:0/442766104 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e464b10 con 0x56336e5406a0 2024-05-10T05:43:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.972+0000 7f059af7eb80 1 -- 172.21.15.131:0/442766104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e460400 con 0x56336e541570 2024-05-10T05:43:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.972+0000 7f057b400700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05600a1330 0x7f05600a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.972+0000 7f057b400700 1 -- 172.21.15.131:0/442766104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e5406a0 msgr2=0x56336e540af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.972+0000 7f057b400700 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e5406a0 0x56336e540af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.972+0000 7f057b400700 1 -- 172.21.15.131:0/442766104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 msgr2=0x56336e5487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.972+0000 7f057b400700 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e5487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.972+0000 7f057b400700 1 -- 172.21.15.131:0/442766104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56336e2e13b0 con 0x56336e541570 2024-05-10T05:43:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.973+0000 7f057b400700 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 0x56336e541980 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f056c00b5c0 tx=0x7f056c010470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ea3a9027ae5dcb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.973+0000 7f0581000700 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e5487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.974+0000 7f057b400700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05600a1330 0x7f05600a3840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f056c0038c0 tx=0x7f056c004000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.975+0000 7f0571600700 1 -- 172.21.15.131:0/442766104 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c003e60 con 0x56336e541570 2024-05-10T05:43:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.975+0000 7f0571600700 1 -- 172.21.15.131:0/442766104 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f056c0050e0 con 0x56336e541570 2024-05-10T05:43:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.975+0000 7f0571600700 1 -- 172.21.15.131:0/442766104 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c026650 con 0x56336e541570 2024-05-10T05:43:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.976+0000 7f059af7eb80 1 -- 172.21.15.131:0/442766104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 msgr2=0x56336e541980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.976+0000 7f059af7eb80 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 0x56336e541980 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f056c00b5c0 tx=0x7f056c010470 comp rx=0 tx=0).stop 2024-05-10T05:43:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.976+0000 7f059af7eb80 1 -- 172.21.15.131:0/442766104 shutdown_connections 2024-05-10T05:43:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.976+0000 7f059af7eb80 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e5487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.976+0000 7f059af7eb80 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e5406a0 0x56336e540af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.976+0000 7f059af7eb80 1 --2- 172.21.15.131:0/442766104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 0x56336e541980 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.976+0000 7f059af7eb80 1 -- 172.21.15.131:0/442766104 >> 172.21.15.131:0/442766104 conn(0x56336e4a2a60 msgr2=0x56336e53afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.977+0000 7f059af7eb80 1 -- 172.21.15.131:0/442766104 shutdown_connections 2024-05-10T05:43:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.977+0000 7f059af7eb80 1 -- 172.21.15.131:0/442766104 wait complete. 2024-05-10T05:43:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.978+0000 7f059af7eb80 1 -- start start 2024-05-10T05:43:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.979+0000 7f059af7eb80 1 Processor -- start 2024-05-10T05:43:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.983+0000 7f059af7eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e5406a0 0x56336e54a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.983+0000 7f059af7eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541570 0x56336e54ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.983+0000 7f059af7eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541ec0 0x56336e551ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.983+0000 7f057b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541570 0x56336e54ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.984+0000 7f059af7eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e3fc560 con 0x56336e541570 2024-05-10T05:43:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.984+0000 7f059af7eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e3fc200 con 0x56336e541ec0 2024-05-10T05:43:19.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.984+0000 7f059af7eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e2e36a0 con 0x56336e5406a0 2024-05-10T05:43:19.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.984+0000 7f057be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e5406a0 0x56336e54a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.984+0000 7f057b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541570 0x56336e54ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51784/0 (socket says 172.21.15.131:51784) 2024-05-10T05:43:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.984+0000 7f057b400700 1 -- 172.21.15.131:0/244612064 learned_addr learned my addr 172.21.15.131:0/244612064 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.984+0000 7f059af7eb80 10 client.0 _ll_register_callbacks cb 0x56336e405b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.984+0000 7f0581000700 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541ec0 0x56336e551ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:19.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.986+0000 7f057be00700 1 -- 172.21.15.131:0/244612064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541ec0 msgr2=0x56336e551ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:19.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.986+0000 7f057be00700 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541ec0 0x56336e551ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.986+0000 7f057be00700 1 -- 172.21.15.131:0/244612064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541570 msgr2=0x56336e54ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:19.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.986+0000 7f057be00700 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541570 0x56336e54ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:19.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.986+0000 7f057be00700 1 -- 172.21.15.131:0/244612064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c110fc0 con 0x56336e5406a0 2024-05-10T05:43:19.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.987+0000 7f0581000700 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541ec0 0x56336e551ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:43:19.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.987+0000 7f057be00700 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e5406a0 0x56336e54a620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x56336e540380 tx=0x7f0568098e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a657efff15508e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:19.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.987+0000 7f0566000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:43:19.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.987+0000 7f057b400700 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541570 0x56336e54ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:19.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.992+0000 7f0566000700 1 -- 172.21.15.131:0/244612064 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568099c90 con 0x56336e5406a0 2024-05-10T05:43:19.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.992+0000 7f0566000700 1 -- 172.21.15.131:0/244612064 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0568099df0 con 0x56336e5406a0 2024-05-10T05:43:19.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:19.992+0000 7f059af7eb80 1 -- 172.21.15.131:0/244612064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56336e3b9270 con 0x56336e5406a0 2024-05-10T05:43:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.003+0000 7f0566000700 1 -- 172.21.15.131:0/244612064 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05680b72b0 con 0x56336e5406a0 2024-05-10T05:43:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.003+0000 7f059af7eb80 10 client.27163 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.003+0000 7f059af7eb80 1 -- 172.21.15.131:0/244612064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56336e28db60 con 0x56336e5406a0 2024-05-10T05:43:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.006+0000 7f0566000700 1 -- 172.21.15.131:0/244612064 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f056803e830 con 0x56336e5406a0 2024-05-10T05:43:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.008+0000 7f0566000700 1 -- 172.21.15.131:0/244612064 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f05680991d0 con 0x56336e5406a0 2024-05-10T05:43:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.008+0000 7f059af7eb80 10 client.27163 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.008+0000 7f059af7eb80 10 client.27163 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:43:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.009+0000 7f059af7eb80 20 client.27163 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.009+0000 7f059af7eb80 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56336e5560c0 0x56336e558510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.009+0000 7f059af7eb80 4 client.27163 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:43:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.009+0000 7f059af7eb80 1 -- 172.21.15.131:0/244612064 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56336e3b9270 con 0x56336e5560c0 2024-05-10T05:43:20.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.021+0000 7f057b400700 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56336e5560c0 0x56336e558510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.024+0000 7f057b400700 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56336e5560c0 0x56336e558510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cbe7030e275a6db0 server_cookie=f05895edce57e978 in_seq=0 out_seq=0 2024-05-10T05:43:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.025+0000 7f0566000700 10 client.27163 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:43:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.025+0000 7f0566000700 1 -- 172.21.15.131:0/244612064 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56336e3b9270 con 0x56336e5560c0 2024-05-10T05:43:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.025+0000 7f0566000700 10 client.27163 handle_command_reply: tid=0 2024-05-10T05:43:20.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.028+0000 7f059af7eb80 1 client.27163 shutdown 2024-05-10T05:43:20.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.028+0000 7f059af7eb80 1 -- 172.21.15.131:0/244612064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e5406a0 msgr2=0x56336e54a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:20.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.028+0000 7f059af7eb80 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e5406a0 0x56336e54a620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x56336e540380 tx=0x7f0568098e70 comp rx=0 tx=0).stop 2024-05-10T05:43:20.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.029+0000 7f059af7eb80 1 -- 172.21.15.131:0/244612064 shutdown_connections 2024-05-10T05:43:20.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.029+0000 7f059af7eb80 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56336e5560c0 0x56336e558510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.029+0000 7f059af7eb80 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541ec0 0x56336e551ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.029+0000 7f059af7eb80 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e5406a0 0x56336e54a620 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x56336e540380 tx=0x7f0568098e70 comp rx=0 tx=0).stop 2024-05-10T05:43:20.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.029+0000 7f059af7eb80 1 --2- 172.21.15.131:0/244612064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541570 0x56336e54ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.029+0000 7f059af7eb80 1 -- 172.21.15.131:0/244612064 >> 172.21.15.131:0/244612064 conn(0x56336e4a2a60 msgr2=0x56336e53afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:20.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.029+0000 7f0566000700 0 client.27163 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:43:20.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.029+0000 7f059af7eb80 1 -- 172.21.15.131:0/244612064 shutdown_connections 2024-05-10T05:43:20.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.029+0000 7f059af7eb80 1 -- 172.21.15.131:0/244612064 wait complete. 2024-05-10T05:43:20.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.032+0000 7f059af7eb80 20 client.27163 trim_cache size 0 max 16384 2024-05-10T05:43:20.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:19 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:43:20.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:19 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:43:20.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:19 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:43:20.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:19 smithi157 ceph-mon[78427]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 268 MiB/s wr, 82 op/s 2024-05-10T05:43:20.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.062+0000 7f059af7eb80 1 Processor -- start 2024-05-10T05:43:20.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.062+0000 7f059af7eb80 1 -- start start 2024-05-10T05:43:20.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.063+0000 7f059af7eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e4a2a60 0x56336e548a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:20.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.063+0000 7f057be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e4a2a60 0x56336e548a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:20.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.063+0000 7f057be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e4a2a60 0x56336e548a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38652/0 (socket says 172.21.15.131:38652) 2024-05-10T05:43:20.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.063+0000 7f057be00700 1 -- 172.21.15.131:0/1451498741 learned_addr learned my addr 172.21.15.131:0/1451498741 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:20.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.063+0000 7f059af7eb80 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 0x56336e551a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:20.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.064+0000 7f059af7eb80 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e548f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:20.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.064+0000 7f057b400700 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 0x56336e551a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:20.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.064+0000 7f0581000700 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e548f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:20.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.064+0000 7f059af7eb80 1 -- 172.21.15.131:0/1451498741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e555c50 con 0x56336e541ec0 2024-05-10T05:43:20.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.065+0000 7f059af7eb80 1 -- 172.21.15.131:0/1451498741 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e53c770 con 0x56336e4a2a60 2024-05-10T05:43:20.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.065+0000 7f059af7eb80 1 -- 172.21.15.131:0/1451498741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e555e10 con 0x56336e541570 2024-05-10T05:43:20.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.066+0000 7f057b400700 1 -- 172.21.15.131:0/1451498741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e4a2a60 msgr2=0x56336e548a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:20.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.066+0000 7f057b400700 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e4a2a60 0x56336e548a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.066+0000 7f057b400700 1 -- 172.21.15.131:0/1451498741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 msgr2=0x56336e548f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:20.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.066+0000 7f057b400700 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e548f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.066+0000 7f057b400700 1 -- 172.21.15.131:0/1451498741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56336e426560 con 0x56336e541570 2024-05-10T05:43:20.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.066+0000 7f0581000700 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e548f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:20.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.066+0000 7f057be00700 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e4a2a60 0x56336e548a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:43:20.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.067+0000 7f057b400700 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 0x56336e551a50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x56336e558d70 tx=0x7f056c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4a44646e7a00e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.067+0000 7f0572000700 1 -- 172.21.15.131:0/1451498741 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c0052d0 con 0x56336e541570 2024-05-10T05:43:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.067+0000 7f0572000700 1 -- 172.21.15.131:0/1451498741 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f056c005430 con 0x56336e541570 2024-05-10T05:43:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.067+0000 7f0572000700 1 -- 172.21.15.131:0/1451498741 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c0387b0 con 0x56336e541570 2024-05-10T05:43:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.069+0000 7f059af7eb80 1 -- 172.21.15.131:0/1451498741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 msgr2=0x56336e551a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.069+0000 7f059af7eb80 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 0x56336e551a50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x56336e558d70 tx=0x7f056c009310 comp rx=0 tx=0).stop 2024-05-10T05:43:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.069+0000 7f059af7eb80 1 -- 172.21.15.131:0/1451498741 shutdown_connections 2024-05-10T05:43:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.069+0000 7f059af7eb80 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e548f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.069+0000 7f059af7eb80 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e4a2a60 0x56336e548a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.069+0000 7f059af7eb80 1 --2- 172.21.15.131:0/1451498741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e541570 0x56336e551a50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.069+0000 7f059af7eb80 1 -- 172.21.15.131:0/1451498741 >> 172.21.15.131:0/1451498741 conn(0x56336e537450 msgr2=0x56336e545350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.069+0000 7f059af7eb80 1 -- 172.21.15.131:0/1451498741 shutdown_connections 2024-05-10T05:43:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.069+0000 7f059af7eb80 1 -- 172.21.15.131:0/1451498741 wait complete. 2024-05-10T05:43:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.070+0000 7f059af7eb80 1 -- start start 2024-05-10T05:43:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.070+0000 7f059af7eb80 1 Processor -- start 2024-05-10T05:43:20.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.071+0000 7f059af7eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e4a2a60 0x56336e54f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:20.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.071+0000 7f059af7eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541570 0x56336e54f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:20.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.071+0000 7f059af7eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e5507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:20.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.071+0000 7f059af7eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e550ce0 con 0x56336e541ec0 2024-05-10T05:43:20.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.071+0000 7f059af7eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e550e50 con 0x56336e541570 2024-05-10T05:43:20.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.071+0000 7f059af7eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336e551010 con 0x56336e4a2a60 2024-05-10T05:43:20.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.071+0000 7f059af7eb80 10 client.0 _ll_register_callbacks cb 0x56336e405b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:20.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.071+0000 7f057be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e4a2a60 0x56336e54f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:20.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.071+0000 7f057be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e4a2a60 0x56336e54f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34578/0 (socket says 172.21.15.131:34578) 2024-05-10T05:43:20.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.071+0000 7f057be00700 1 -- 172.21.15.131:0/4061042762 learned_addr learned my addr 172.21.15.131:0/4061042762 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:20.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.072+0000 7f057b400700 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541570 0x56336e54f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:20.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.072+0000 7f057b400700 1 -- 172.21.15.131:0/4061042762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e4a2a60 msgr2=0x56336e54f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:20.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.072+0000 7f057b400700 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e4a2a60 0x56336e54f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.072+0000 7f057b400700 1 -- 172.21.15.131:0/4061042762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 msgr2=0x56336e5507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.072+0000 7f0581000700 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e5507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.072+0000 7f057b400700 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e5507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.073+0000 7f057b400700 1 -- 172.21.15.131:0/4061042762 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0574007c50 con 0x56336e541570 2024-05-10T05:43:20.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.073+0000 7f0581000700 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e5507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.073+0000 7f057b400700 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541570 0x56336e54f870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f056c00b5c0 tx=0x7f056c005350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9f3c2e49d34a331 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.073+0000 7f0566000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:43:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.074+0000 7f0566000700 1 -- 172.21.15.131:0/4061042762 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c0056a0 con 0x56336e541570 2024-05-10T05:43:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.074+0000 7f0566000700 1 -- 172.21.15.131:0/4061042762 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f056c047030 con 0x56336e541570 2024-05-10T05:43:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.074+0000 7f059af7eb80 1 -- 172.21.15.131:0/4061042762 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f056c0050e0 con 0x56336e541570 2024-05-10T05:43:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.074+0000 7f0566000700 1 -- 172.21.15.131:0/4061042762 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f056c010040 con 0x56336e541570 2024-05-10T05:43:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.074+0000 7f059af7eb80 10 client.27140 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:20.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.074+0000 7f059af7eb80 1 -- 172.21.15.131:0/4061042762 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f056c032040 con 0x56336e541570 2024-05-10T05:43:20.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.074+0000 7f0566000700 1 -- 172.21.15.131:0/4061042762 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f056c0050e0 con 0x56336e541570 2024-05-10T05:43:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.075+0000 7f0566000700 1 -- 172.21.15.131:0/4061042762 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f056c02c070 con 0x56336e541570 2024-05-10T05:43:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.075+0000 7f059af7eb80 10 client.27140 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.075+0000 7f059af7eb80 10 client.27140 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:43:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.075+0000 7f059af7eb80 20 client.27140 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.075+0000 7f059af7eb80 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56336e5aca90 0x56336e5aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.075+0000 7f059af7eb80 4 client.27140 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:43:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.075+0000 7f059af7eb80 1 -- 172.21.15.131:0/4061042762 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56336e50f780 con 0x56336e5aca90 2024-05-10T05:43:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.075+0000 7f057be00700 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56336e5aca90 0x56336e5aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.076+0000 7f0566000700 10 client.27140 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:43:20.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.076+0000 7f057be00700 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56336e5aca90 0x56336e5aeeb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4fc91d40fa1c103a server_cookie=368289130de113b4 in_seq=0 out_seq=0 2024-05-10T05:43:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.076+0000 7f0566000700 1 -- 172.21.15.131:0/4061042762 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56336e50f780 con 0x56336e5aca90 2024-05-10T05:43:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.076+0000 7f0566000700 10 client.27140 handle_command_reply: tid=0 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.076+0000 7f059af7eb80 1 client.27140 shutdown 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f059af7eb80 1 -- 172.21.15.131:0/4061042762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541570 msgr2=0x56336e54f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f059af7eb80 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541570 0x56336e54f870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f056c00b5c0 tx=0x7f056c005350 comp rx=0 tx=0).stop 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f059af7eb80 1 -- 172.21.15.131:0/4061042762 shutdown_connections 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f059af7eb80 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x56336e5aca90 0x56336e5aeeb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f0566000700 0 client.27140 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f059af7eb80 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56336e541ec0 0x56336e5507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f059af7eb80 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56336e4a2a60 0x56336e54f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f059af7eb80 1 --2- 172.21.15.131:0/4061042762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56336e541570 0x56336e54f870 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f059af7eb80 1 -- 172.21.15.131:0/4061042762 >> 172.21.15.131:0/4061042762 conn(0x56336e537450 msgr2=0x56336e5488c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f059af7eb80 1 -- 172.21.15.131:0/4061042762 shutdown_connections 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.077+0000 7f059af7eb80 1 -- 172.21.15.131:0/4061042762 wait complete. 2024-05-10T05:43:20.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.078+0000 7f059af7eb80 20 client.27140 trim_cache size 0 max 16384 2024-05-10T05:43:20.079 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:43:20.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.078+0000 7f0583000700 1 -- 172.21.15.131:0/3856013444 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05600a1330 msgr2=0x7f05600a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.078+0000 7f0583000700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05600a1330 0x7f05600a3840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f056c0038c0 tx=0x7f056c004000 comp rx=0 tx=0).stop 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.078+0000 7f0583000700 1 -- 172.21.15.131:0/3856013444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f057c073b40 msgr2=0x7f057c19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.078+0000 7f0583000700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f057c073b40 0x7f057c19e600 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f056800b750 tx=0x7f056800bb10 comp rx=0 tx=0).stop 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.079+0000 7f0583000700 1 -- 172.21.15.131:0/3856013444 shutdown_connections 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.079+0000 7f0583000700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f05600a1330 0x7f05600a3840 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.079+0000 7f0583000700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f057c073b40 0x7f057c19e600 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.079+0000 7f0583000700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f057c10e220 0x7f057c1a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.079+0000 7f0583000700 1 --2- 172.21.15.131:0/3856013444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f057c075140 0x7f057c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.079+0000 7f0583000700 1 -- 172.21.15.131:0/3856013444 >> 172.21.15.131:0/3856013444 conn(0x7f057c0fd210 msgr2=0x7f057c1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.079+0000 7f0583000700 1 -- 172.21.15.131:0/3856013444 shutdown_connections 2024-05-10T05:43:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.079+0000 7f0583000700 1 -- 172.21.15.131:0/3856013444 wait complete. 2024-05-10T05:43:20.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:19 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:43:20.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:19 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:43:20.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:19 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:43:20.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:19 smithi142 ceph-mon[77667]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 268 MiB/s wr, 82 op/s 2024-05-10T05:43:20.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:43:20.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.993+0000 7f09c6c00700 1 -- 172.21.15.131:0/208828136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c0108630 msgr2=0x7f09c0108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:20.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.993+0000 7f09c6c00700 1 --2- 172.21.15.131:0/208828136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c0108630 0x7f09c0108a40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0099c0 tx=0x7f09ac009cd0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.998+0000 7f09c6c00700 1 -- 172.21.15.131:0/208828136 shutdown_connections 2024-05-10T05:43:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.998+0000 7f09c6c00700 1 --2- 172.21.15.131:0/208828136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c010a0c0 0x7f09c0110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.998+0000 7f09c6c00700 1 --2- 172.21.15.131:0/208828136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c0108630 0x7f09c0108a40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.998+0000 7f09c6c00700 1 --2- 172.21.15.131:0/208828136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0109720 0x7f09c0109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.998+0000 7f09c6c00700 1 -- 172.21.15.131:0/208828136 >> 172.21.15.131:0/208828136 conn(0x7f09c006ee40 msgr2=0x7f09c00712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.998+0000 7f09c6c00700 1 -- 172.21.15.131:0/208828136 shutdown_connections 2024-05-10T05:43:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:20.998+0000 7f09c6c00700 1 -- 172.21.15.131:0/208828136 wait complete. 2024-05-10T05:43:21.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.000+0000 7f09c6c00700 1 Processor -- start 2024-05-10T05:43:21.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.003+0000 7f09c6c00700 1 -- start start 2024-05-10T05:43:21.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.004+0000 7f09c6c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c0108630 0x7f09c019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.007+0000 7f09c6c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0109720 0x7f09c019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.007+0000 7f09c6c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c010a0c0 0x7f09c01996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.007+0000 7f09c6c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c0199df0 con 0x7f09c0109720 2024-05-10T05:43:21.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.007+0000 7f09c6c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c0199f60 con 0x7f09c010a0c0 2024-05-10T05:43:21.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.007+0000 7f09c6c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c019a120 con 0x7f09c0108630 2024-05-10T05:43:21.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.007+0000 7f09c4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c010a0c0 0x7f09c01996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.007+0000 7f09c4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c010a0c0 0x7f09c01996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38680/0 (socket says 172.21.15.131:38680) 2024-05-10T05:43:21.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.007+0000 7f09c4c00700 1 -- 172.21.15.131:0/970740530 learned_addr learned my addr 172.21.15.131:0/970740530 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:21.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.007+0000 7f09bfe00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c0108630 0x7f09c019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.007+0000 7f09bf400700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0109720 0x7f09c019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.008+0000 7f09c4c00700 1 -- 172.21.15.131:0/970740530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c0108630 msgr2=0x7f09c019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:21.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.008+0000 7f09c4c00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c0108630 0x7f09c019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.008+0000 7f09c4c00700 1 -- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0109720 msgr2=0x7f09c019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:21.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.008+0000 7f09c4c00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0109720 0x7f09c019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.008+0000 7f09c4c00700 1 -- 172.21.15.131:0/970740530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0095f0 con 0x7f09c010a0c0 2024-05-10T05:43:21.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.008+0000 7f09bf400700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0109720 0x7f09c019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:21.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.009+0000 7f09c4c00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c010a0c0 0x7f09c01996f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f09b000db10 tx=0x7f09b000de20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:21.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.011+0000 7f09bcc00700 1 -- 172.21.15.131:0/970740530 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000ac10 con 0x7f09c010a0c0 2024-05-10T05:43:21.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.011+0000 7f09bcc00700 1 -- 172.21.15.131:0/970740530 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b000ad70 con 0x7f09c010a0c0 2024-05-10T05:43:21.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.011+0000 7f09c6c00700 1 -- 172.21.15.131:0/970740530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c01aee80 con 0x7f09c010a0c0 2024-05-10T05:43:21.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.012+0000 7f09bcc00700 1 -- 172.21.15.131:0/970740530 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000e450 con 0x7f09c010a0c0 2024-05-10T05:43:21.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.012+0000 7f09c6c00700 1 -- 172.21.15.131:0/970740530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c01af2a0 con 0x7f09c010a0c0 2024-05-10T05:43:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.027+0000 7f09bcc00700 1 -- 172.21.15.131:0/970740530 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f09b0004830 con 0x7f09c010a0c0 2024-05-10T05:43:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.029+0000 7f09bcc00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09a40a1400 0x7f09a40a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.033+0000 7f09bcc00700 1 -- 172.21.15.131:0/970740530 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f09b00cc580 con 0x7f09c010a0c0 2024-05-10T05:43:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.033+0000 7f09bfe00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09a40a1400 0x7f09a40a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.035+0000 7f09debeeb80 1 Processor -- start 2024-05-10T05:43:21.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.036+0000 7f09debeeb80 1 -- start start 2024-05-10T05:43:21.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.041+0000 7f09debeeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e205af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.041+0000 7f09debeeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 0x56221e206980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.041+0000 7f09debeeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206ec0 0x56221e20d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.041+0000 7f09debeeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e12c150 con 0x56221e206570 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.041+0000 7f09debeeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e129b10 con 0x56221e206ec0 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.041+0000 7f09debeeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e125400 con 0x56221e2056a0 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.042+0000 7f09bf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 0x56221e206980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.043+0000 7f09bf400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 0x56221e206980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51828/0 (socket says 172.21.15.131:51828) 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.043+0000 7f09bf400700 1 -- 172.21.15.131:0/966940544 learned_addr learned my addr 172.21.15.131:0/966940544 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.043+0000 7f09c4c00700 1 --2- 172.21.15.131:0/966940544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206ec0 0x56221e20d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.044+0000 7f09bfe00700 1 --2- 172.21.15.131:0/966940544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e205af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.044+0000 7f09bfe00700 1 -- 172.21.15.131:0/966940544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206ec0 msgr2=0x56221e20d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.044+0000 7f09bfe00700 1 --2- 172.21.15.131:0/966940544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206ec0 0x56221e20d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.044+0000 7f09bfe00700 1 -- 172.21.15.131:0/966940544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 msgr2=0x56221e206980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:21.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.044+0000 7f09bfe00700 1 --2- 172.21.15.131:0/966940544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 0x56221e206980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.044+0000 7f09bfe00700 1 -- 172.21.15.131:0/966940544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56221dfa63b0 con 0x56221e2056a0 2024-05-10T05:43:21.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.044+0000 7f09bf400700 1 --2- 172.21.15.131:0/966940544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 0x56221e206980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:21.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.066+0000 7f09bfe00700 1 --2- 172.21.15.131:0/966940544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e205af0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0098e0 tx=0x7f09ac009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a82825a6a6e1cf4e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:21.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.066+0000 7f09bfe00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09a40a1400 0x7f09a40a3910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009f50 tx=0x7f09ac01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:21.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.077+0000 7f09b5200700 1 -- 172.21.15.131:0/966940544 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac003ac0 con 0x56221e2056a0 2024-05-10T05:43:21.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.077+0000 7f09b5200700 1 -- 172.21.15.131:0/966940544 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09ac01e040 con 0x56221e2056a0 2024-05-10T05:43:21.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.077+0000 7f09b5200700 1 -- 172.21.15.131:0/966940544 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac02ec40 con 0x56221e2056a0 2024-05-10T05:43:21.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.081+0000 7f09debeeb80 1 -- 172.21.15.131:0/966940544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 msgr2=0x56221e205af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:21.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.081+0000 7f09debeeb80 1 --2- 172.21.15.131:0/966940544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e205af0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0098e0 tx=0x7f09ac009310 comp rx=0 tx=0).stop 2024-05-10T05:43:21.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.087+0000 7f09debeeb80 1 -- 172.21.15.131:0/966940544 shutdown_connections 2024-05-10T05:43:21.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.087+0000 7f09debeeb80 1 --2- 172.21.15.131:0/966940544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206ec0 0x56221e20d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.087+0000 7f09debeeb80 1 --2- 172.21.15.131:0/966940544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e205af0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.087+0000 7f09debeeb80 1 --2- 172.21.15.131:0/966940544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 0x56221e206980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.087+0000 7f09debeeb80 1 -- 172.21.15.131:0/966940544 >> 172.21.15.131:0/966940544 conn(0x56221e167a60 msgr2=0x56221e1fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:21.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.091+0000 7f09debeeb80 1 -- 172.21.15.131:0/966940544 shutdown_connections 2024-05-10T05:43:21.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.091+0000 7f09debeeb80 1 -- 172.21.15.131:0/966940544 wait complete. 2024-05-10T05:43:21.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.094+0000 7f09debeeb80 1 -- start start 2024-05-10T05:43:21.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.103+0000 7f09debeeb80 1 Processor -- start 2024-05-10T05:43:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.107+0000 7f09debeeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e20f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.109+0000 7f09bfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e20f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.109+0000 7f09bfe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e20f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34620/0 (socket says 172.21.15.131:34620) 2024-05-10T05:43:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.110+0000 7f09bfe00700 1 -- 172.21.15.131:0/1954121505 learned_addr learned my addr 172.21.15.131:0/1954121505 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.110+0000 7f09debeeb80 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 0x56221e20fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.110+0000 7f09debeeb80 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206ec0 0x56221e216bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.110+0000 7f09bf400700 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 0x56221e20fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.112+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e0c1560 con 0x56221e206ec0 2024-05-10T05:43:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.112+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e0c1200 con 0x56221e206570 2024-05-10T05:43:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.112+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221dfa86a0 con 0x56221e2056a0 2024-05-10T05:43:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.113+0000 7f09debeeb80 10 client.0 _ll_register_callbacks cb 0x56221e0cab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.113+0000 7f09c4c00700 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206ec0 0x56221e216bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.113+0000 7f09bfe00700 1 -- 172.21.15.131:0/1954121505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 msgr2=0x56221e20fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.113+0000 7f09bfe00700 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 0x56221e20fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.113+0000 7f09bfe00700 1 -- 172.21.15.131:0/1954121505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206ec0 msgr2=0x56221e216bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.113+0000 7f09bfe00700 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206ec0 0x56221e216bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.113+0000 7f09bfe00700 1 -- 172.21.15.131:0/1954121505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b80099e0 con 0x56221e2056a0 2024-05-10T05:43:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.113+0000 7f09bf400700 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 0x56221e20fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.114+0000 7f09bfe00700 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e20f710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f09ac003b20 tx=0x7f09ac003b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc4167b6e2fe74c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:21.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.114+0000 7f09a9600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:43:21.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.115+0000 7f09a9600700 1 -- 172.21.15.131:0/1954121505 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac02e580 con 0x56221e2056a0 2024-05-10T05:43:21.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.117+0000 7f09a9600700 1 -- 172.21.15.131:0/1954121505 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f09ac00f460 con 0x56221e2056a0 2024-05-10T05:43:21.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.117+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56221e07e270 con 0x56221e2056a0 2024-05-10T05:43:21.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.119+0000 7f09a9600700 1 -- 172.21.15.131:0/1954121505 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09ac00f620 con 0x56221e2056a0 2024-05-10T05:43:21.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.119+0000 7f09debeeb80 10 client.27181 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:21.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.119+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f09c006a540 con 0x56221e2056a0 2024-05-10T05:43:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.121+0000 7f09a9600700 1 -- 172.21.15.131:0/1954121505 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56221e07e270 con 0x56221e2056a0 2024-05-10T05:43:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.121+0000 7f09a9600700 1 -- 172.21.15.131:0/1954121505 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f09ac041070 con 0x56221e2056a0 2024-05-10T05:43:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.121+0000 7f09debeeb80 10 client.27181 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.121+0000 7f09debeeb80 10 client.27181 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:43:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.121+0000 7f09debeeb80 20 client.27181 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.121+0000 7f09debeeb80 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56221e21aec0 0x56221e21d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.121+0000 7f09debeeb80 4 client.27181 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:43:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.121+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56221e1d4780 con 0x56221e21aec0 2024-05-10T05:43:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.122+0000 7f09bf400700 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56221e21aec0 0x56221e21d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.124+0000 7f09bf400700 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56221e21aec0 0x56221e21d310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d09baffa78117461 server_cookie=960ced7f7d474ad0 in_seq=0 out_seq=0 2024-05-10T05:43:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.125+0000 7f09a9600700 10 client.27181 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:43:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.129+0000 7f09a9600700 1 -- 172.21.15.131:0/1954121505 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56221e1d4780 con 0x56221e21aec0 2024-05-10T05:43:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.129+0000 7f09a9600700 10 client.27181 handle_command_reply: tid=0 2024-05-10T05:43:21.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.131+0000 7f09debeeb80 1 client.27181 shutdown 2024-05-10T05:43:21.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.132+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 msgr2=0x56221e20f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:21.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.136+0000 7f09debeeb80 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e20f710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f09ac003b20 tx=0x7f09ac003b50 comp rx=0 tx=0).stop 2024-05-10T05:43:21.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.136+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 shutdown_connections 2024-05-10T05:43:21.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.136+0000 7f09debeeb80 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56221e21aec0 0x56221e21d310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.136+0000 7f09debeeb80 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206ec0 0x56221e216bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.136+0000 7f09debeeb80 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e2056a0 0x56221e20f710 secure :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f09ac003b20 tx=0x7f09ac003b50 comp rx=0 tx=0).stop 2024-05-10T05:43:21.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.136+0000 7f09debeeb80 1 --2- 172.21.15.131:0/1954121505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 0x56221e20fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.136+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 >> 172.21.15.131:0/1954121505 conn(0x56221e167a60 msgr2=0x56221e1fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:21.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.136+0000 7f09a9600700 0 client.27181 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:43:21.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.138+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 shutdown_connections 2024-05-10T05:43:21.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.138+0000 7f09debeeb80 1 -- 172.21.15.131:0/1954121505 wait complete. 2024-05-10T05:43:21.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.142+0000 7f09debeeb80 20 client.27181 trim_cache size 0 max 16384 2024-05-10T05:43:21.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.176+0000 7f09debeeb80 1 Processor -- start 2024-05-10T05:43:21.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.176+0000 7f09debeeb80 1 -- start start 2024-05-10T05:43:21.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.177+0000 7f09debeeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e167a60 0x56221e20a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.178+0000 7f09bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e167a60 0x56221e20a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.179+0000 7f09bfe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e167a60 0x56221e20a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51848/0 (socket says 172.21.15.131:51848) 2024-05-10T05:43:21.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.179+0000 7f09bfe00700 1 -- 172.21.15.131:0/272627919 learned_addr learned my addr 172.21.15.131:0/272627919 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:21.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.179+0000 7f09debeeb80 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 0x56221e216b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.179+0000 7f09debeeb80 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 0x56221e20a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.180+0000 7f09debeeb80 1 -- 172.21.15.131:0/272627919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e07e270 con 0x56221e167a60 2024-05-10T05:43:21.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.180+0000 7f09bf400700 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 0x56221e216b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.180+0000 7f09debeeb80 1 -- 172.21.15.131:0/272627919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e20add0 con 0x56221e206570 2024-05-10T05:43:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.180+0000 7f09debeeb80 1 -- 172.21.15.131:0/272627919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e1fcd30 con 0x56221e206ec0 2024-05-10T05:43:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.180+0000 7f09c4c00700 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 0x56221e20a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.181+0000 7f09bf400700 1 -- 172.21.15.131:0/272627919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 msgr2=0x56221e20a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.181+0000 7f09bf400700 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 0x56221e20a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.181+0000 7f09bf400700 1 -- 172.21.15.131:0/272627919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e167a60 msgr2=0x56221e20a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.181+0000 7f09bf400700 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e167a60 0x56221e20a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.181+0000 7f09bf400700 1 -- 172.21.15.131:0/272627919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56221df52b60 con 0x56221e206570 2024-05-10T05:43:21.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.181+0000 7f09bfe00700 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e167a60 0x56221e20a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:43:21.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.182+0000 7f09bf400700 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 0x56221e216b70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x56221e21b2d0 tx=0x7f09b8009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d3fc112944de23 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:21.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.184+0000 7f09b5c00700 1 -- 172.21.15.131:0/272627919 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8007e90 con 0x56221e206570 2024-05-10T05:43:21.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.185+0000 7f09b5c00700 1 -- 172.21.15.131:0/272627919 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b801b150 con 0x56221e206570 2024-05-10T05:43:21.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.186+0000 7f09debeeb80 1 -- 172.21.15.131:0/272627919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 msgr2=0x56221e216b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:21.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.186+0000 7f09debeeb80 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 0x56221e216b70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x56221e21b2d0 tx=0x7f09b8009040 comp rx=0 tx=0).stop 2024-05-10T05:43:21.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.186+0000 7f09debeeb80 1 -- 172.21.15.131:0/272627919 shutdown_connections 2024-05-10T05:43:21.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.187+0000 7f09debeeb80 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e167a60 0x56221e20a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.187+0000 7f09debeeb80 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 0x56221e20a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.187+0000 7f09debeeb80 1 --2- 172.21.15.131:0/272627919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e206570 0x56221e216b70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.187+0000 7f09debeeb80 1 -- 172.21.15.131:0/272627919 >> 172.21.15.131:0/272627919 conn(0x56221e1fc450 msgr2=0x56221e200c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:21.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.187+0000 7f09debeeb80 1 -- 172.21.15.131:0/272627919 shutdown_connections 2024-05-10T05:43:21.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.187+0000 7f09debeeb80 1 -- 172.21.15.131:0/272627919 wait complete. 2024-05-10T05:43:21.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.188+0000 7f09debeeb80 1 -- start start 2024-05-10T05:43:21.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.188+0000 7f09debeeb80 1 Processor -- start 2024-05-10T05:43:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.194+0000 7f09debeeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e167a60 0x56221e2145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.194+0000 7f09debeeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 0x56221e214ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.194+0000 7f09debeeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 0x56221e215a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.194+0000 7f09debeeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e215f50 con 0x56221e206570 2024-05-10T05:43:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.194+0000 7f09debeeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e2160c0 con 0x56221e167a60 2024-05-10T05:43:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.194+0000 7f09debeeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221e216280 con 0x56221e206ec0 2024-05-10T05:43:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.194+0000 7f09debeeb80 10 client.0 _ll_register_callbacks cb 0x56221e0cab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.194+0000 7f09c4c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 0x56221e215a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.194+0000 7f09c4c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 0x56221e215a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34650/0 (socket says 172.21.15.131:34650) 2024-05-10T05:43:21.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.194+0000 7f09c4c00700 1 -- 172.21.15.131:0/389820182 learned_addr learned my addr 172.21.15.131:0/389820182 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:21.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.195+0000 7f09bfe00700 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e167a60 0x56221e2145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.195+0000 7f09c4c00700 1 -- 172.21.15.131:0/389820182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e167a60 msgr2=0x56221e2145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:21.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.195+0000 7f09c4c00700 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e167a60 0x56221e2145a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.195+0000 7f09c4c00700 1 -- 172.21.15.131:0/389820182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 msgr2=0x56221e214ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:21.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.195+0000 7f09c4c00700 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 0x56221e214ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.195+0000 7f09c4c00700 1 -- 172.21.15.131:0/389820182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b8011040 con 0x56221e206ec0 2024-05-10T05:43:21.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.196+0000 7f09c4c00700 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 0x56221e215a10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f09b0099900 tx=0x7f09b0010040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf5b0c0047ac896 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:21.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.196+0000 7f09a9600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:43:21.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.196+0000 7f09a9600700 1 -- 172.21.15.131:0/389820182 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0099970 con 0x56221e206ec0 2024-05-10T05:43:21.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.198+0000 7f09a9600700 1 -- 172.21.15.131:0/389820182 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f09b009d7a0 con 0x56221e206ec0 2024-05-10T05:43:21.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.198+0000 7f09debeeb80 1 -- 172.21.15.131:0/389820182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09ac01e040 con 0x56221e206ec0 2024-05-10T05:43:21.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.198+0000 7f09a9600700 1 -- 172.21.15.131:0/389820182 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09b00a1df0 con 0x56221e206ec0 2024-05-10T05:43:21.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.198+0000 7f09debeeb80 10 client.27187 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:21.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.199+0000 7f09debeeb80 1 -- 172.21.15.131:0/389820182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56221e0eb560 con 0x56221e206ec0 2024-05-10T05:43:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.199+0000 7f09a9600700 1 -- 172.21.15.131:0/389820182 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09ac01e040 con 0x56221e206ec0 2024-05-10T05:43:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.200+0000 7f09a9600700 1 -- 172.21.15.131:0/389820182 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f09b00cc1e0 con 0x56221e206ec0 2024-05-10T05:43:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.201+0000 7f09debeeb80 10 client.27187 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.201+0000 7f09debeeb80 10 client.27187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:43:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.201+0000 7f09debeeb80 20 client.27187 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:21.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.202+0000 7f09debeeb80 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56221e271d90 0x56221e2741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:21.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.202+0000 7f09debeeb80 4 client.27187 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:43:21.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.202+0000 7f09debeeb80 1 -- 172.21.15.131:0/389820182 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56221e11dec0 con 0x56221e271d90 2024-05-10T05:43:21.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.202+0000 7f09bfe00700 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56221e271d90 0x56221e2741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.204+0000 7f09bfe00700 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56221e271d90 0x56221e2741b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4916612d9433b138 server_cookie=825d67c17b1a53b3 in_seq=0 out_seq=0 2024-05-10T05:43:21.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.205+0000 7f09a9600700 10 client.27187 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:43:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.206+0000 7f09a9600700 1 -- 172.21.15.131:0/389820182 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56221e11dec0 con 0x56221e271d90 2024-05-10T05:43:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.207+0000 7f09a9600700 10 client.27187 handle_command_reply: tid=0 2024-05-10T05:43:21.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.207+0000 7f09debeeb80 1 client.27187 shutdown 2024-05-10T05:43:21.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.213+0000 7f09debeeb80 1 -- 172.21.15.131:0/389820182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 msgr2=0x56221e215a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:21.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.213+0000 7f09debeeb80 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 0x56221e215a10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f09b0099900 tx=0x7f09b0010040 comp rx=0 tx=0).stop 2024-05-10T05:43:21.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.214+0000 7f09debeeb80 1 -- 172.21.15.131:0/389820182 shutdown_connections 2024-05-10T05:43:21.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.214+0000 7f09debeeb80 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x56221e271d90 0x56221e2741b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.214+0000 7f09debeeb80 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56221e206ec0 0x56221e215a10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.214+0000 7f09debeeb80 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56221e167a60 0x56221e2145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.214+0000 7f09debeeb80 1 --2- 172.21.15.131:0/389820182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56221e206570 0x56221e214ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.214+0000 7f09debeeb80 1 -- 172.21.15.131:0/389820182 >> 172.21.15.131:0/389820182 conn(0x56221e1fc450 msgr2=0x56221e20a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.214+0000 7f09a9600700 0 client.27187 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:43:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.214+0000 7f09debeeb80 1 -- 172.21.15.131:0/389820182 shutdown_connections 2024-05-10T05:43:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.214+0000 7f09debeeb80 1 -- 172.21.15.131:0/389820182 wait complete. 2024-05-10T05:43:21.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.215+0000 7f09debeeb80 20 client.27187 trim_cache size 0 max 16384 2024-05-10T05:43:21.216 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:43:21.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.216+0000 7f09c6c00700 1 -- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09a40a1400 msgr2=0x7f09a40a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:21.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.216+0000 7f09c6c00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09a40a1400 0x7f09a40a3910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009f50 tx=0x7f09ac01a040 comp rx=0 tx=0).stop 2024-05-10T05:43:21.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.217+0000 7f09c6c00700 1 -- 172.21.15.131:0/970740530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c010a0c0 msgr2=0x7f09c01996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:21.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.217+0000 7f09c6c00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c010a0c0 0x7f09c01996f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f09b000db10 tx=0x7f09b000de20 comp rx=0 tx=0).stop 2024-05-10T05:43:21.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.217+0000 7f09c6c00700 1 -- 172.21.15.131:0/970740530 shutdown_connections 2024-05-10T05:43:21.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.217+0000 7f09c6c00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09a40a1400 0x7f09a40a3910 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.217+0000 7f09c6c00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c010a0c0 0x7f09c01996f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.217+0000 7f09c6c00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c0108630 0x7f09c019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.217+0000 7f09c6c00700 1 --2- 172.21.15.131:0/970740530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0109720 0x7f09c019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:21.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.217+0000 7f09c6c00700 1 -- 172.21.15.131:0/970740530 >> 172.21.15.131:0/970740530 conn(0x7f09c006ee40 msgr2=0x7f09c010c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:21.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.218+0000 7f09c6c00700 1 -- 172.21.15.131:0/970740530 shutdown_connections 2024-05-10T05:43:21.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:21.218+0000 7f09c6c00700 1 -- 172.21.15.131:0/970740530 wait complete. 2024-05-10T05:43:21.275 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:43:22.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.241+0000 7fa883600700 1 -- 172.21.15.131:0/1143920218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8740a6ee0 msgr2=0x7fa8740ab7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:22.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.241+0000 7fa883600700 1 --2- 172.21.15.131:0/1143920218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8740a6ee0 0x7fa8740ab7f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0672f0 tx=0x7fa87c069d30 comp rx=0 tx=0).stop 2024-05-10T05:43:22.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.241+0000 7fa883600700 1 -- 172.21.15.131:0/1143920218 shutdown_connections 2024-05-10T05:43:22.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.241+0000 7fa883600700 1 --2- 172.21.15.131:0/1143920218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8740a6ee0 0x7fa8740ab7f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.241+0000 7fa883600700 1 --2- 172.21.15.131:0/1143920218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8740a55b0 0x7fa8740a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.241+0000 7fa883600700 1 --2- 172.21.15.131:0/1143920218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8740a6590 0x7fa8740a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.241+0000 7fa883600700 1 -- 172.21.15.131:0/1143920218 >> 172.21.15.131:0/1143920218 conn(0x7fa8740a0a40 msgr2=0x7fa8740a2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:22.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.244+0000 7fa883600700 1 -- 172.21.15.131:0/1143920218 shutdown_connections 2024-05-10T05:43:22.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.245+0000 7fa883600700 1 -- 172.21.15.131:0/1143920218 wait complete. 2024-05-10T05:43:22.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.248+0000 7fa883600700 1 Processor -- start 2024-05-10T05:43:22.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.250+0000 7fa883600700 1 -- start start 2024-05-10T05:43:22.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.253+0000 7fa883600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8740a55b0 0x7fa874143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.253+0000 7fa883600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8740a6590 0x7fa8741444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.253+0000 7fa883600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8740a6ee0 0x7fa87414b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.254+0000 7fa883600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87414ba50 con 0x7fa8740a55b0 2024-05-10T05:43:22.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.254+0000 7fa881600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8740a6ee0 0x7fa87414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.254+0000 7fa881600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8740a6ee0 0x7fa87414b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38742/0 (socket says 172.21.15.131:38742) 2024-05-10T05:43:22.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.254+0000 7fa881600700 1 -- 172.21.15.131:0/1503131588 learned_addr learned my addr 172.21.15.131:0/1503131588 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:22.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.254+0000 7fa883600700 1 -- 172.21.15.131:0/1503131588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87414bb90 con 0x7fa8740a6ee0 2024-05-10T05:43:22.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.254+0000 7fa883600700 1 -- 172.21.15.131:0/1503131588 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87414bd20 con 0x7fa8740a6590 2024-05-10T05:43:22.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.255+0000 7fa87be00700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8740a6590 0x7fa8741444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.255+0000 7fa880c00700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8740a55b0 0x7fa874143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.256+0000 7fa881600700 1 -- 172.21.15.131:0/1503131588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8740a6590 msgr2=0x7fa8741444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:22.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.256+0000 7fa881600700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8740a6590 0x7fa8741444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.256+0000 7fa881600700 1 -- 172.21.15.131:0/1503131588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8740a55b0 msgr2=0x7fa874143fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:22.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.256+0000 7fa881600700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8740a55b0 0x7fa874143fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.256+0000 7fa881600700 1 -- 172.21.15.131:0/1503131588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c069700 con 0x7fa8740a6ee0 2024-05-10T05:43:22.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.257+0000 7fa881600700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8740a6ee0 0x7fa87414b480 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa87c050560 tx=0x7fa87c0692e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:22.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.259+0000 7fa879600700 1 -- 172.21.15.131:0/1503131588 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c067c60 con 0x7fa8740a6ee0 2024-05-10T05:43:22.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.259+0000 7fa879600700 1 -- 172.21.15.131:0/1503131588 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa87c067dc0 con 0x7fa8740a6ee0 2024-05-10T05:43:22.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.259+0000 7fa879600700 1 -- 172.21.15.131:0/1503131588 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c07fea0 con 0x7fa8740a6ee0 2024-05-10T05:43:22.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.259+0000 7fa883600700 1 -- 172.21.15.131:0/1503131588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87414bfa0 con 0x7fa8740a6ee0 2024-05-10T05:43:22.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.260+0000 7fa883600700 1 -- 172.21.15.131:0/1503131588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87414c310 con 0x7fa8740a6ee0 2024-05-10T05:43:22.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.268+0000 7fa879600700 1 -- 172.21.15.131:0/1503131588 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa87c07b040 con 0x7fa8740a6ee0 2024-05-10T05:43:22.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.276+0000 7fa879600700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa8640a11f0 0x7fa8640a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.278+0000 7fa880c00700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa8640a11f0 0x7fa8640a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.278+0000 7fa879600700 1 -- 172.21.15.131:0/1503131588 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa87c130f30 con 0x7fa8740a6ee0 2024-05-10T05:43:22.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.279+0000 7fa880c00700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa8640a11f0 0x7fa8640a3700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa8740a52a0 tx=0x7fa868009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:22.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.283+0000 7fa89b4eeb80 1 Processor -- start 2024-05-10T05:43:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.284+0000 7fa89b4eeb80 1 -- start start 2024-05-10T05:43:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.285+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.285+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 0x563339dfabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.285+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfb0f0 0x563339e019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.285+0000 7fa89b4eeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339d20150 con 0x563339dfa7a0 2024-05-10T05:43:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.285+0000 7fa89b4eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339d1db10 con 0x563339df9530 2024-05-10T05:43:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.285+0000 7fa89b4eeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339d19400 con 0x563339dfb0f0 2024-05-10T05:43:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.286+0000 7fa880c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.286+0000 7fa880c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:38750/0 (socket says 172.21.15.131:38750) 2024-05-10T05:43:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.286+0000 7fa880c00700 1 -- 172.21.15.131:0/3744415181 learned_addr learned my addr 172.21.15.131:0/3744415181 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.287+0000 7fa880c00700 1 -- 172.21.15.131:0/3744415181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfb0f0 msgr2=0x563339e019f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:43:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.287+0000 7fa880c00700 1 --2- 172.21.15.131:0/3744415181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfb0f0 0x563339e019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.287+0000 7fa880c00700 1 -- 172.21.15.131:0/3744415181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 msgr2=0x563339dfabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.287+0000 7fa87be00700 1 --2- 172.21.15.131:0/3744415181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 0x563339dfabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.288+0000 7fa880c00700 1 --2- 172.21.15.131:0/3744415181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 0x563339dfabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.288+0000 7fa880c00700 1 -- 172.21.15.131:0/3744415181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563339b9a3b0 con 0x563339df9530 2024-05-10T05:43:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.288+0000 7fa87be00700 1 --2- 172.21.15.131:0/3744415181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 0x563339dfabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.288+0000 7fa880c00700 1 --2- 172.21.15.131:0/3744415181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df9960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa868011de0 tx=0x7fa868014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19fddc9c6b47c10 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.290+0000 7fa86dc00700 1 -- 172.21.15.131:0/3744415181 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86801f420 con 0x563339df9530 2024-05-10T05:43:22.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.290+0000 7fa86dc00700 1 -- 172.21.15.131:0/3744415181 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa868016070 con 0x563339df9530 2024-05-10T05:43:22.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.290+0000 7fa86dc00700 1 -- 172.21.15.131:0/3744415181 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86801d950 con 0x563339df9530 2024-05-10T05:43:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.291+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3744415181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 msgr2=0x563339df9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.291+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/3744415181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df9960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa868011de0 tx=0x7fa868014000 comp rx=0 tx=0).stop 2024-05-10T05:43:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.291+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3744415181 shutdown_connections 2024-05-10T05:43:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.291+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/3744415181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfb0f0 0x563339e019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.291+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/3744415181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df9960 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.291+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/3744415181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 0x563339dfabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.291+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3744415181 >> 172.21.15.131:0/3744415181 conn(0x563339d5ba60 msgr2=0x563339df4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.292+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3744415181 shutdown_connections 2024-05-10T05:43:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.292+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3744415181 wait complete. 2024-05-10T05:43:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.293+0000 7fa89b4eeb80 1 -- start start 2024-05-10T05:43:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.293+0000 7fa89b4eeb80 1 Processor -- start 2024-05-10T05:43:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.294+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339df9530 0x563339e03970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.294+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e03eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.294+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 0x563339e0ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.295+0000 7fa880c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339df9530 0x563339e03970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.295+0000 7fa87be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e03eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.295+0000 7fa87be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e03eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34688/0 (socket says 172.21.15.131:34688) 2024-05-10T05:43:22.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.295+0000 7fa87be00700 1 -- 172.21.15.131:0/3780146791 learned_addr learned my addr 172.21.15.131:0/3780146791 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:22.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.295+0000 7fa881600700 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 0x563339e0ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.295+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339cb5560 con 0x563339df9530 2024-05-10T05:43:22.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.295+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339cb5200 con 0x563339dfb0f0 2024-05-10T05:43:22.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.295+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339b9c6a0 con 0x563339dfa7a0 2024-05-10T05:43:22.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.295+0000 7fa89b4eeb80 10 client.0 _ll_register_callbacks cb 0x563339cbeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:22.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.296+0000 7fa87be00700 1 -- 172.21.15.131:0/3780146791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 msgr2=0x563339e0ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:22.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.296+0000 7fa87be00700 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 0x563339e0ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.296+0000 7fa87be00700 1 -- 172.21.15.131:0/3780146791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339df9530 msgr2=0x563339e03970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:22.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.296+0000 7fa87be00700 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339df9530 0x563339e03970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.296+0000 7fa87be00700 1 -- 172.21.15.131:0/3780146791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa868011ac0 con 0x563339dfa7a0 2024-05-10T05:43:22.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.297+0000 7fa880c00700 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339df9530 0x563339e03970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:22.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.297+0000 7fa862000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:43:22.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.297+0000 7fa87be00700 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e03eb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa87000b9f0 tx=0x7fa87000bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25b8e01a021ebf82 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:22.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.297+0000 7fa862000700 1 -- 172.21.15.131:0/3780146791 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000be30 con 0x563339dfa7a0 2024-05-10T05:43:22.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.298+0000 7fa862000700 1 -- 172.21.15.131:0/3780146791 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa870012450 con 0x563339dfa7a0 2024-05-10T05:43:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.298+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563339c72270 con 0x563339dfa7a0 2024-05-10T05:43:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.298+0000 7fa862000700 1 -- 172.21.15.131:0/3780146791 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa870012610 con 0x563339dfa7a0 2024-05-10T05:43:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.298+0000 7fa89b4eeb80 10 client.27196 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.298+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563339b46b60 con 0x563339dfa7a0 2024-05-10T05:43:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.299+0000 7fa862000700 1 -- 172.21.15.131:0/3780146791 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563339c72270 con 0x563339dfa7a0 2024-05-10T05:43:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.299+0000 7fa862000700 1 -- 172.21.15.131:0/3780146791 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa87001b840 con 0x563339dfa7a0 2024-05-10T05:43:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.299+0000 7fa89b4eeb80 10 client.27196 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.299+0000 7fa89b4eeb80 10 client.27196 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:43:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.299+0000 7fa89b4eeb80 20 client.27196 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:22.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.299+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563339e0f5a0 0x563339e119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.299+0000 7fa89b4eeb80 4 client.27196 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:43:22.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.299+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563339dc8780 con 0x563339e0f5a0 2024-05-10T05:43:22.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.300+0000 7fa880c00700 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563339e0f5a0 0x563339e119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.301+0000 7fa862000700 10 client.27196 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:43:22.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.301+0000 7fa880c00700 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563339e0f5a0 0x563339e119f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be36798d5bf55bb4 server_cookie=153810475289a296 in_seq=0 out_seq=0 2024-05-10T05:43:22.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.303+0000 7fa862000700 1 -- 172.21.15.131:0/3780146791 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563339dc8780 con 0x563339e0f5a0 2024-05-10T05:43:22.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.303+0000 7fa862000700 10 client.27196 handle_command_reply: tid=0 2024-05-10T05:43:22.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.303+0000 7fa89b4eeb80 1 client.27196 shutdown 2024-05-10T05:43:22.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.304+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 msgr2=0x563339e03eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:22.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.304+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e03eb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa87000b9f0 tx=0x7fa87000bd00 comp rx=0 tx=0).stop 2024-05-10T05:43:22.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.304+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 shutdown_connections 2024-05-10T05:43:22.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.304+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563339e0f5a0 0x563339e119f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.304+0000 7fa862000700 0 client.27196 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:43:22.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.304+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339df9530 0x563339e03970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.304+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 0x563339e0ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.304+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/3780146791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e03eb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.304+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 >> 172.21.15.131:0/3780146791 conn(0x563339d5ba60 msgr2=0x563339df8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:22.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.306+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 shutdown_connections 2024-05-10T05:43:22.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.307+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/3780146791 wait complete. 2024-05-10T05:43:22.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.310+0000 7fa89b4eeb80 20 client.27196 trim_cache size 0 max 16384 2024-05-10T05:43:22.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.341+0000 7fa89b4eeb80 1 Processor -- start 2024-05-10T05:43:22.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.341+0000 7fa89b4eeb80 1 -- start start 2024-05-10T05:43:22.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.342+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.342+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e0add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.342+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfb0f0 0x563339e0b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.343+0000 7fa881600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfb0f0 0x563339e0b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.343+0000 7fa881600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfb0f0 0x563339e0b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51926/0 (socket says 172.21.15.131:51926) 2024-05-10T05:43:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.343+0000 7fa880c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.344+0000 7fa881600700 1 -- 172.21.15.131:0/4012338642 learned_addr learned my addr 172.21.15.131:0/4012338642 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.344+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/4012338642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339dfa590 con 0x563339dfb0f0 2024-05-10T05:43:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.344+0000 7fa87be00700 1 --2- 172.21.15.131:0/4012338642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e0add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.344+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/4012338642 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339e01940 con 0x563339df9530 2024-05-10T05:43:22.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.344+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/4012338642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339df4810 con 0x563339dfa7a0 2024-05-10T05:43:22.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.345+0000 7fa881600700 1 -- 172.21.15.131:0/4012338642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 msgr2=0x563339e0add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:22.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.345+0000 7fa881600700 1 --2- 172.21.15.131:0/4012338642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e0add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.345+0000 7fa881600700 1 -- 172.21.15.131:0/4012338642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 msgr2=0x563339df3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:22.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.345+0000 7fa881600700 1 --2- 172.21.15.131:0/4012338642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.345+0000 7fa881600700 1 -- 172.21.15.131:0/4012338642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563339cdf560 con 0x563339dfb0f0 2024-05-10T05:43:22.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.345+0000 7fa87be00700 1 --2- 172.21.15.131:0/4012338642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e0add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:22.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.345+0000 7fa880c00700 1 --2- 172.21.15.131:0/4012338642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:22.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.346+0000 7fa881600700 1 --2- 172.21.15.131:0/4012338642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfb0f0 0x563339e0b3f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0fae80 tx=0x7fa87c0fa7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef19cb7650963b26 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:22.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.347+0000 7fa86e600700 1 -- 172.21.15.131:0/4012338642 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0fdd70 con 0x563339dfb0f0 2024-05-10T05:43:22.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.347+0000 7fa86e600700 1 -- 172.21.15.131:0/4012338642 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa87c0feb10 con 0x563339dfb0f0 2024-05-10T05:43:22.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.347+0000 7fa86e600700 1 -- 172.21.15.131:0/4012338642 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0e90d0 con 0x563339dfb0f0 2024-05-10T05:43:22.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.348+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/4012338642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfb0f0 msgr2=0x563339e0b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:22.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.348+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/4012338642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfb0f0 0x563339e0b3f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0fae80 tx=0x7fa87c0fa7d0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.349+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/4012338642 shutdown_connections 2024-05-10T05:43:22.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.349+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/4012338642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfb0f0 0x563339e0b3f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.349+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/4012338642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339df9530 0x563339df3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.349+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/4012338642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339dfa7a0 0x563339e0add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.349+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/4012338642 >> 172.21.15.131:0/4012338642 conn(0x563339d5ba60 msgr2=0x563339e03970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:22.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.351+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/4012338642 shutdown_connections 2024-05-10T05:43:22.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.352+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/4012338642 wait complete. 2024-05-10T05:43:22.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.354+0000 7fa89b4eeb80 1 -- start start 2024-05-10T05:43:22.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.358+0000 7fa89b4eeb80 1 Processor -- start 2024-05-10T05:43:22.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.360+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339df9530 0x563339e08730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.360+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 0x563339e08c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.362+0000 7fa89b4eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 0x563339e09ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.362+0000 7fa89b4eeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339e0a0e0 con 0x563339dfa7a0 2024-05-10T05:43:22.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.363+0000 7fa89b4eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339e0a250 con 0x563339dfb0f0 2024-05-10T05:43:22.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.363+0000 7fa89b4eeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563339e0a410 con 0x563339df9530 2024-05-10T05:43:22.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.363+0000 7fa89b4eeb80 10 client.0 _ll_register_callbacks cb 0x563339cbeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:22.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.364+0000 7fa87be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 0x563339e08c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.365+0000 7fa87be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 0x563339e08c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51938/0 (socket says 172.21.15.131:51938) 2024-05-10T05:43:22.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.365+0000 7fa87be00700 1 -- 172.21.15.131:0/281457762 learned_addr learned my addr 172.21.15.131:0/281457762 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:22.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.365+0000 7fa880c00700 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339df9530 0x563339e08730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.365+0000 7fa881600700 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 0x563339e09ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.366+0000 7fa881600700 1 -- 172.21.15.131:0/281457762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339df9530 msgr2=0x563339e08730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:22.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.366+0000 7fa881600700 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339df9530 0x563339e08730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.366+0000 7fa881600700 1 -- 172.21.15.131:0/281457762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 msgr2=0x563339e08c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:22.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.366+0000 7fa881600700 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 0x563339e08c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.366+0000 7fa881600700 1 -- 172.21.15.131:0/281457762 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87414c310 con 0x563339dfb0f0 2024-05-10T05:43:22.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.366+0000 7fa881600700 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 0x563339e09ba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0fae80 tx=0x7fa87c12e9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30dc808ef5870bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:22.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.366+0000 7fa862000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:43:22.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.367+0000 7fa862000700 1 -- 172.21.15.131:0/281457762 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c106570 con 0x563339dfb0f0 2024-05-10T05:43:22.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.367+0000 7fa862000700 1 -- 172.21.15.131:0/281457762 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa87c0fe410 con 0x563339dfb0f0 2024-05-10T05:43:22.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.367+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/281457762 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa868016070 con 0x563339dfb0f0 2024-05-10T05:43:22.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.367+0000 7fa862000700 1 -- 172.21.15.131:0/281457762 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa87c0e90d0 con 0x563339dfb0f0 2024-05-10T05:43:22.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.367+0000 7fa89b4eeb80 10 client.27170 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:22.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.367+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/281457762 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563339b9ecd0 con 0x563339dfb0f0 2024-05-10T05:43:22.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.368+0000 7fa862000700 1 -- 172.21.15.131:0/281457762 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa87c0e1080 con 0x563339dfb0f0 2024-05-10T05:43:22.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.368+0000 7fa862000700 1 -- 172.21.15.131:0/281457762 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa87c0fd500 con 0x563339dfb0f0 2024-05-10T05:43:22.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.369+0000 7fa89b4eeb80 10 client.27170 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:22.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.369+0000 7fa89b4eeb80 10 client.27170 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:43:22.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.369+0000 7fa89b4eeb80 20 client.27170 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:22.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.369+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563339e65df0 0x563339e68210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:22.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.369+0000 7fa89b4eeb80 4 client.27170 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:43:22.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.369+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/281457762 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fa868016070 con 0x563339e65df0 2024-05-10T05:43:22.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.370+0000 7fa880c00700 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563339e65df0 0x563339e68210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:22.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.370+0000 7fa880c00700 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563339e65df0 0x563339e68210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18c11085f343b3c7 server_cookie=1af6ac0c05583cc7 in_seq=0 out_seq=0 2024-05-10T05:43:22.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.371+0000 7fa862000700 10 client.27170 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:43:22.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.372+0000 7fa862000700 1 -- 172.21.15.131:0/281457762 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fa868016070 con 0x563339e65df0 2024-05-10T05:43:22.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.373+0000 7fa862000700 10 client.27170 handle_command_reply: tid=0 2024-05-10T05:43:22.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.373+0000 7fa89b4eeb80 1 client.27170 shutdown 2024-05-10T05:43:22.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.374+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/281457762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 msgr2=0x563339e09ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:22.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.374+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 0x563339e09ba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0fae80 tx=0x7fa87c12e9d0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.375+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/281457762 shutdown_connections 2024-05-10T05:43:22.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.375+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x563339e65df0 0x563339e68210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.375+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563339dfb0f0 0x563339e09ba0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.375+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563339df9530 0x563339e08730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.375+0000 7fa89b4eeb80 1 --2- 172.21.15.131:0/281457762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563339dfa7a0 0x563339e08c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.375+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/281457762 >> 172.21.15.131:0/281457762 conn(0x563339d5ba60 msgr2=0x563339e01c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:22.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.375+0000 7fa862000700 0 client.27170 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:43:22.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.376+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/281457762 shutdown_connections 2024-05-10T05:43:22.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.376+0000 7fa89b4eeb80 1 -- 172.21.15.131:0/281457762 wait complete. 2024-05-10T05:43:22.385 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:43:22.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.384+0000 7fa89b4eeb80 20 client.27170 trim_cache size 0 max 16384 2024-05-10T05:43:22.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.388+0000 7fa883600700 1 -- 172.21.15.131:0/1503131588 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa8640a11f0 msgr2=0x7fa8640a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:22.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.388+0000 7fa883600700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa8640a11f0 0x7fa8640a3700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa8740a52a0 tx=0x7fa868009260 comp rx=0 tx=0).stop 2024-05-10T05:43:22.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.388+0000 7fa883600700 1 -- 172.21.15.131:0/1503131588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8740a6ee0 msgr2=0x7fa87414b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:22.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.388+0000 7fa883600700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8740a6ee0 0x7fa87414b480 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa87c050560 tx=0x7fa87c0692e0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.389+0000 7fa883600700 1 -- 172.21.15.131:0/1503131588 shutdown_connections 2024-05-10T05:43:22.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.389+0000 7fa883600700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa8640a11f0 0x7fa8640a3700 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.389+0000 7fa883600700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8740a55b0 0x7fa874143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.389+0000 7fa883600700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8740a6ee0 0x7fa87414b480 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.389+0000 7fa883600700 1 --2- 172.21.15.131:0/1503131588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8740a6590 0x7fa8741444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:22.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.389+0000 7fa883600700 1 -- 172.21.15.131:0/1503131588 >> 172.21.15.131:0/1503131588 conn(0x7fa8740a0a40 msgr2=0x7fa8740a1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:22.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.389+0000 7fa883600700 1 -- 172.21.15.131:0/1503131588 shutdown_connections 2024-05-10T05:43:22.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:22.390+0000 7fa883600700 1 -- 172.21.15.131:0/1503131588 wait complete. 2024-05-10T05:43:22.455 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:43:22.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:22 smithi131 ceph-mon[37264]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 254 MiB/s wr, 81 op/s 2024-05-10T05:43:22.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:22 smithi157 ceph-mon[78427]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 254 MiB/s wr, 81 op/s 2024-05-10T05:43:22.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:22 smithi142 ceph-mon[77667]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 254 MiB/s wr, 81 op/s 2024-05-10T05:43:23.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.150+0000 7f9278600700 1 -- 172.21.15.131:0/1366729447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9270108210 msgr2=0x7f9270108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.150+0000 7f9278600700 1 --2- 172.21.15.131:0/1366729447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9270108210 0x7f9270108620 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f926800b0d0 tx=0x7f926800b3e0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.150+0000 7f9278600700 1 -- 172.21.15.131:0/1366729447 shutdown_connections 2024-05-10T05:43:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.150+0000 7f9278600700 1 --2- 172.21.15.131:0/1366729447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9270110320 0x7f9270112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.150+0000 7f9278600700 1 --2- 172.21.15.131:0/1366729447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9270108210 0x7f9270108620 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.150+0000 7f9278600700 1 --2- 172.21.15.131:0/1366729447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9270108b60 0x7f927010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.150+0000 7f9278600700 1 -- 172.21.15.131:0/1366729447 >> 172.21.15.131:0/1366729447 conn(0x7f927006ea00 msgr2=0x7f9270070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:23.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.155+0000 7f9278600700 1 -- 172.21.15.131:0/1366729447 shutdown_connections 2024-05-10T05:43:23.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.155+0000 7f9278600700 1 -- 172.21.15.131:0/1366729447 wait complete. 2024-05-10T05:43:23.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.158+0000 7f9278600700 1 Processor -- start 2024-05-10T05:43:23.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.159+0000 7f9278600700 1 -- start start 2024-05-10T05:43:23.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.159+0000 7f9278600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9270108210 0x7f92701a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.159+0000 7f9278600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9270108b60 0x7f92701a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.159+0000 7f9278600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9270110320 0x7f92701a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.159+0000 7f9278600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92701aa550 con 0x7f9270108210 2024-05-10T05:43:23.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.159+0000 7f9278600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92701aa6c0 con 0x7f9270108b60 2024-05-10T05:43:23.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.159+0000 7f9278600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92701aa880 con 0x7f9270110320 2024-05-10T05:43:23.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.159+0000 7f9275200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9270108b60 0x7f92701a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.160+0000 7f9275200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9270108b60 0x7f92701a2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47614/0 (socket says 172.21.15.131:47614) 2024-05-10T05:43:23.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.160+0000 7f9275200700 1 -- 172.21.15.131:0/807127389 learned_addr learned my addr 172.21.15.131:0/807127389 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.160+0000 7f9275200700 1 -- 172.21.15.131:0/807127389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9270110320 msgr2=0x7f92701a9f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:43:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.161+0000 7f9276600700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9270110320 0x7f92701a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.162+0000 7f9275200700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9270110320 0x7f92701a9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.162+0000 7f9275200700 1 -- 172.21.15.131:0/807127389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9270108210 msgr2=0x7f92701a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.162+0000 7f9275200700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9270108210 0x7f92701a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.162+0000 7f9275200700 1 -- 172.21.15.131:0/807127389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9268009d00 con 0x7f9270108b60 2024-05-10T05:43:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.163+0000 7f9276600700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9270110320 0x7f92701a9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.163+0000 7f9275200700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9270108b60 0x7f92701a2fb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f926400d950 tx=0x7f926400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.163+0000 7f926e000700 1 -- 172.21.15.131:0/807127389 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264009b40 con 0x7f9270108b60 2024-05-10T05:43:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.164+0000 7f9278600700 1 -- 172.21.15.131:0/807127389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92701aab60 con 0x7f9270108b60 2024-05-10T05:43:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.164+0000 7f926e000700 1 -- 172.21.15.131:0/807127389 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9264009ca0 con 0x7f9270108b60 2024-05-10T05:43:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.164+0000 7f9278600700 1 -- 172.21.15.131:0/807127389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92701ab0c0 con 0x7f9270108b60 2024-05-10T05:43:23.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.165+0000 7f926e000700 1 -- 172.21.15.131:0/807127389 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264010660 con 0x7f9270108b60 2024-05-10T05:43:23.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.175+0000 7f926e000700 1 -- 172.21.15.131:0/807127389 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9264019360 con 0x7f9270108b60 2024-05-10T05:43:23.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.185+0000 7f926e000700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92580a1280 0x7f92580a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.185+0000 7f926e000700 1 -- 172.21.15.131:0/807127389 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f92640cbfb0 con 0x7f9270108b60 2024-05-10T05:43:23.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.186+0000 7f9275c00700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92580a1280 0x7f92580a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.192+0000 7f9275c00700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92580a1280 0x7f92580a3790 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9268009900 tx=0x7f926800b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.192+0000 7f92905f9b80 1 Processor -- start 2024-05-10T05:43:23.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.193+0000 7f92905f9b80 1 -- start start 2024-05-10T05:43:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.196+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f60f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.196+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1570 0x5566f60f1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.196+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f60f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.196+0000 7f92905f9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f6017150 con 0x5566f60f1570 2024-05-10T05:43:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.196+0000 7f92905f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f6014b10 con 0x5566f60f06a0 2024-05-10T05:43:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.196+0000 7f92905f9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f6010400 con 0x5566f60f1ec0 2024-05-10T05:43:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.198+0000 7f9275c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f60f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.198+0000 7f9275200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1570 0x5566f60f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.198+0000 7f9275200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1570 0x5566f60f1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44842/0 (socket says 172.21.15.131:44842) 2024-05-10T05:43:23.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.198+0000 7f9275200700 1 -- 172.21.15.131:0/425827748 learned_addr learned my addr 172.21.15.131:0/425827748 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:23.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.198+0000 7f9276600700 1 --2- 172.21.15.131:0/425827748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f60f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.199+0000 7f9275200700 1 -- 172.21.15.131:0/425827748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 msgr2=0x5566f60f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.199+0000 7f9275200700 1 --2- 172.21.15.131:0/425827748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f60f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.199+0000 7f9275200700 1 -- 172.21.15.131:0/425827748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 msgr2=0x5566f60f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.199+0000 7f9275200700 1 --2- 172.21.15.131:0/425827748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f60f0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.199+0000 7f9275200700 1 -- 172.21.15.131:0/425827748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566f5e913b0 con 0x5566f60f1570 2024-05-10T05:43:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.200+0000 7f9275200700 1 --2- 172.21.15.131:0/425827748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1570 0x5566f60f1980 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9264095f00 tx=0x7f9264098fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3da7181e53d6d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.201+0000 7f9262a00700 1 -- 172.21.15.131:0/425827748 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264010660 con 0x5566f60f1570 2024-05-10T05:43:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.201+0000 7f9262a00700 1 -- 172.21.15.131:0/425827748 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9264009ca0 con 0x5566f60f1570 2024-05-10T05:43:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.201+0000 7f9262a00700 1 -- 172.21.15.131:0/425827748 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640975d0 con 0x5566f60f1570 2024-05-10T05:43:23.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.203+0000 7f92905f9b80 1 -- 172.21.15.131:0/425827748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1570 msgr2=0x5566f60f1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:23.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.203+0000 7f92905f9b80 1 --2- 172.21.15.131:0/425827748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1570 0x5566f60f1980 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9264095f00 tx=0x7f9264098fd0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.204+0000 7f92905f9b80 1 -- 172.21.15.131:0/425827748 shutdown_connections 2024-05-10T05:43:23.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.204+0000 7f92905f9b80 1 --2- 172.21.15.131:0/425827748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f60f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.204+0000 7f92905f9b80 1 --2- 172.21.15.131:0/425827748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f60f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.204+0000 7f92905f9b80 1 --2- 172.21.15.131:0/425827748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1570 0x5566f60f1980 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.204+0000 7f92905f9b80 1 -- 172.21.15.131:0/425827748 >> 172.21.15.131:0/425827748 conn(0x5566f6052a60 msgr2=0x5566f60eafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:23.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.204+0000 7f92905f9b80 1 -- 172.21.15.131:0/425827748 shutdown_connections 2024-05-10T05:43:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.204+0000 7f92905f9b80 1 -- 172.21.15.131:0/425827748 wait complete. 2024-05-10T05:43:23.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.207+0000 7f92905f9b80 1 -- start start 2024-05-10T05:43:23.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.208+0000 7f92905f9b80 1 Processor -- start 2024-05-10T05:43:23.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.210+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f06a0 0x5566f61030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.210+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1ec0 0x5566f61036d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.210+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f61084c0 0x5566f610a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.210+0000 7f92905f9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f5fac560 con 0x5566f60f1ec0 2024-05-10T05:43:23.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.210+0000 7f92905f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f5fac200 con 0x5566f61084c0 2024-05-10T05:43:23.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.210+0000 7f92905f9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f5e936a0 con 0x5566f60f06a0 2024-05-10T05:43:23.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.211+0000 7f92905f9b80 10 client.0 _ll_register_callbacks cb 0x5566f5fb5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:23.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.211+0000 7f9276600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f61084c0 0x5566f610a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.211+0000 7f9276600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f61084c0 0x5566f610a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47624/0 (socket says 172.21.15.131:47624) 2024-05-10T05:43:23.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.211+0000 7f9276600700 1 -- 172.21.15.131:0/1941875468 learned_addr learned my addr 172.21.15.131:0/1941875468 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:23.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.212+0000 7f9275c00700 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f06a0 0x5566f61030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.212+0000 7f9276600700 1 -- 172.21.15.131:0/1941875468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f06a0 msgr2=0x5566f61030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:23.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.212+0000 7f9276600700 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f06a0 0x5566f61030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.212+0000 7f9276600700 1 -- 172.21.15.131:0/1941875468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1ec0 msgr2=0x5566f61036d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:43:23.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.214+0000 7f9275200700 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1ec0 0x5566f61036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.214+0000 7f9276600700 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1ec0 0x5566f61036d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.214+0000 7f9276600700 1 -- 172.21.15.131:0/1941875468 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92701ab0c0 con 0x5566f61084c0 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.215+0000 7f9275200700 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1ec0 0x5566f61036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.215+0000 7f9276600700 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f61084c0 0x5566f610a8e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f925c0136e0 tx=0x7f925c013aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ee1063e133e2ae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.215+0000 7f9257400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.216+0000 7f9257400700 1 -- 172.21.15.131:0/1941875468 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c009850 con 0x5566f61084c0 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.216+0000 7f92905f9b80 1 -- 172.21.15.131:0/1941875468 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566f5f69270 con 0x5566f61084c0 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.216+0000 7f9257400700 1 -- 172.21.15.131:0/1941875468 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f925c004020 con 0x5566f61084c0 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.216+0000 7f9257400700 1 -- 172.21.15.131:0/1941875468 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f925c004220 con 0x5566f61084c0 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.217+0000 7f92905f9b80 10 client.27182 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.217+0000 7f92905f9b80 1 -- 172.21.15.131:0/1941875468 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5566f5e3db60 con 0x5566f61084c0 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.217+0000 7f9257400700 1 -- 172.21.15.131:0/1941875468 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5566f5f69270 con 0x5566f61084c0 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.217+0000 7f9257400700 1 -- 172.21.15.131:0/1941875468 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f925c003a10 con 0x5566f61084c0 2024-05-10T05:43:23.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.217+0000 7f92905f9b80 10 client.27182 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:23.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.217+0000 7f92905f9b80 10 client.27182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:43:23.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.217+0000 7f92905f9b80 20 client.27182 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:23.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.218+0000 7f92905f9b80 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5566f610f260 0x5566f61116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.218+0000 7f92905f9b80 4 client.27182 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:43:23.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.218+0000 7f92905f9b80 1 -- 172.21.15.131:0/1941875468 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566f60bf780 con 0x5566f610f260 2024-05-10T05:43:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.218+0000 7f9275c00700 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5566f610f260 0x5566f61116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.219+0000 7f9257400700 10 client.27182 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.219+0000 7f9275c00700 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5566f610f260 0x5566f61116b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ae99acc11bee94a server_cookie=1960f62873fe3ce4 in_seq=0 out_seq=0 2024-05-10T05:43:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.220+0000 7f9257400700 1 -- 172.21.15.131:0/1941875468 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5566f60bf780 con 0x5566f610f260 2024-05-10T05:43:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.220+0000 7f9257400700 10 client.27182 handle_command_reply: tid=0 2024-05-10T05:43:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.220+0000 7f92905f9b80 1 client.27182 shutdown 2024-05-10T05:43:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.220+0000 7f92905f9b80 1 -- 172.21.15.131:0/1941875468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f61084c0 msgr2=0x5566f610a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.221+0000 7f92905f9b80 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f61084c0 0x5566f610a8e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f925c0136e0 tx=0x7f925c013aa0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.221+0000 7f92905f9b80 1 -- 172.21.15.131:0/1941875468 shutdown_connections 2024-05-10T05:43:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.221+0000 7f92905f9b80 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5566f610f260 0x5566f61116b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.221+0000 7f92905f9b80 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f61084c0 0x5566f610a8e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.221+0000 7f92905f9b80 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f06a0 0x5566f61030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.221+0000 7f92905f9b80 1 --2- 172.21.15.131:0/1941875468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f60f1ec0 0x5566f61036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.221+0000 7f92905f9b80 1 -- 172.21.15.131:0/1941875468 >> 172.21.15.131:0/1941875468 conn(0x5566f6052a60 msgr2=0x5566f60eafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.221+0000 7f9257400700 0 client.27182 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.222+0000 7f92905f9b80 1 -- 172.21.15.131:0/1941875468 shutdown_connections 2024-05-10T05:43:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.222+0000 7f92905f9b80 1 -- 172.21.15.131:0/1941875468 wait complete. 2024-05-10T05:43:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.223+0000 7f92905f9b80 20 client.27182 trim_cache size 0 max 16384 2024-05-10T05:43:23.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.254+0000 7f92905f9b80 1 Processor -- start 2024-05-10T05:43:23.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.257+0000 7f92905f9b80 1 -- start start 2024-05-10T05:43:23.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.257+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f61030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.257+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f6103670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.257+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f60ec770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.262+0000 7f9275c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f61030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.262+0000 7f9275c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f61030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47636/0 (socket says 172.21.15.131:47636) 2024-05-10T05:43:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.262+0000 7f9276600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f60ec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.264+0000 7f9276600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f60ec770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44870/0 (socket says 172.21.15.131:44870) 2024-05-10T05:43:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.265+0000 7f9275200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f6103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.265+0000 7f9275c00700 1 -- 172.21.15.131:0/3769805664 learned_addr learned my addr 172.21.15.131:0/3769805664 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.265+0000 7f92905f9b80 1 -- 172.21.15.131:0/3769805664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f60f8850 con 0x5566f61084c0 2024-05-10T05:43:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.265+0000 7f92905f9b80 1 -- 172.21.15.131:0/3769805664 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f61080c0 con 0x5566f60f06a0 2024-05-10T05:43:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.265+0000 7f92905f9b80 1 -- 172.21.15.131:0/3769805664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f6108280 con 0x5566f60f1ec0 2024-05-10T05:43:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.266+0000 7f9276600700 1 -- 172.21.15.131:0/3769805664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 msgr2=0x5566f6103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.266+0000 7f9276600700 1 --2- 172.21.15.131:0/3769805664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f6103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.266+0000 7f9276600700 1 -- 172.21.15.131:0/3769805664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 msgr2=0x5566f61030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.266+0000 7f9276600700 1 --2- 172.21.15.131:0/3769805664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f61030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.266+0000 7f9276600700 1 -- 172.21.15.131:0/3769805664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566f5fd6560 con 0x5566f61084c0 2024-05-10T05:43:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.267+0000 7f9275200700 1 --2- 172.21.15.131:0/3769805664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f6103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.267+0000 7f9275c00700 1 --2- 172.21.15.131:0/3769805664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f61030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.267+0000 7f9276600700 1 --2- 172.21.15.131:0/3769805664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f60ec770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f925c026020 tx=0x7f925c0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7400a217db22e22c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.271+0000 7f9263400700 1 -- 172.21.15.131:0/3769805664 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c004190 con 0x5566f61084c0 2024-05-10T05:43:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.271+0000 7f9263400700 1 -- 172.21.15.131:0/3769805664 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f925c005810 con 0x5566f61084c0 2024-05-10T05:43:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.273+0000 7f92905f9b80 1 -- 172.21.15.131:0/3769805664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 msgr2=0x5566f60ec770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.273+0000 7f9263400700 1 -- 172.21.15.131:0/3769805664 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c004190 con 0x5566f61084c0 2024-05-10T05:43:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.273+0000 7f92905f9b80 1 --2- 172.21.15.131:0/3769805664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f60ec770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f925c026020 tx=0x7f925c0098d0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.274+0000 7f92905f9b80 1 -- 172.21.15.131:0/3769805664 shutdown_connections 2024-05-10T05:43:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.274+0000 7f92905f9b80 1 --2- 172.21.15.131:0/3769805664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f60ec770 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.274+0000 7f92905f9b80 1 --2- 172.21.15.131:0/3769805664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f61030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.274+0000 7f92905f9b80 1 --2- 172.21.15.131:0/3769805664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f6103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.274+0000 7f92905f9b80 1 -- 172.21.15.131:0/3769805664 >> 172.21.15.131:0/3769805664 conn(0x5566f6052a60 msgr2=0x5566f60f47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:23.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.277+0000 7f92905f9b80 1 -- 172.21.15.131:0/3769805664 shutdown_connections 2024-05-10T05:43:23.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.277+0000 7f92905f9b80 1 -- 172.21.15.131:0/3769805664 wait complete. 2024-05-10T05:43:23.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.278+0000 7f92905f9b80 1 -- start start 2024-05-10T05:43:23.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.279+0000 7f92905f9b80 1 Processor -- start 2024-05-10T05:43:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.282+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f611fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.282+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f611e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.282+0000 7f92905f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f611e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.283+0000 7f9275c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f611fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.283+0000 7f9275c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f611fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47638/0 (socket says 172.21.15.131:47638) 2024-05-10T05:43:23.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.283+0000 7f9276600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f611e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.283+0000 7f9276600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f611e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44886/0 (socket says 172.21.15.131:44886) 2024-05-10T05:43:23.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.283+0000 7f9275c00700 1 -- 172.21.15.131:0/54334226 learned_addr learned my addr 172.21.15.131:0/54334226 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:23.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.283+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f611ee50 con 0x5566f61084c0 2024-05-10T05:43:23.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.283+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f611efc0 con 0x5566f60f06a0 2024-05-10T05:43:23.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.284+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f6164290 con 0x5566f60f1ec0 2024-05-10T05:43:23.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.284+0000 7f9275200700 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f611e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.284+0000 7f92905f9b80 10 client.0 _ll_register_callbacks cb 0x5566f5fb5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:23.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.284+0000 7f9276600700 1 -- 172.21.15.131:0/54334226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 msgr2=0x5566f611e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:23.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.284+0000 7f9276600700 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f611e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.284+0000 7f9276600700 1 -- 172.21.15.131:0/54334226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 msgr2=0x5566f611fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:23.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.285+0000 7f9276600700 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f611fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.285+0000 7f9276600700 1 -- 172.21.15.131:0/54334226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566f5e3db60 con 0x5566f61084c0 2024-05-10T05:43:23.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.285+0000 7f9275c00700 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f611fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:43:23.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.286+0000 7f9276600700 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f611e7e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5566f60f8c90 tx=0x7f925c009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68e8f70389fd710 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:23.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.286+0000 7f9257400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:43:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.290+0000 7f9257400700 1 -- 172.21.15.131:0/54334226 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c003a90 con 0x5566f61084c0 2024-05-10T05:43:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.290+0000 7f9257400700 1 -- 172.21.15.131:0/54334226 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f925c000e40 con 0x5566f61084c0 2024-05-10T05:43:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.290+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9264009ca0 con 0x5566f61084c0 2024-05-10T05:43:23.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.291+0000 7f9257400700 1 -- 172.21.15.131:0/54334226 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f925c003bf0 con 0x5566f61084c0 2024-05-10T05:43:23.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.291+0000 7f92905f9b80 10 client.16851 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:23.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.291+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5566f5e95cd0 con 0x5566f61084c0 2024-05-10T05:43:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.295+0000 7f9257400700 1 -- 172.21.15.131:0/54334226 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9264009ca0 con 0x5566f61084c0 2024-05-10T05:43:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.295+0000 7f9257400700 1 -- 172.21.15.131:0/54334226 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f925c042070 con 0x5566f61084c0 2024-05-10T05:43:23.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.296+0000 7f92905f9b80 10 client.16851 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:23.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.296+0000 7f92905f9b80 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:43:23.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.296+0000 7f92905f9b80 20 client.16851 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:23.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.296+0000 7f92905f9b80 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5566f6165760 0x5566f6167b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:23.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.296+0000 7f92905f9b80 4 client.16851 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:43:23.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.296+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5566f6008ec0 con 0x5566f6165760 2024-05-10T05:43:23.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.296+0000 7f9275c00700 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5566f6165760 0x5566f6167b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:23.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.299+0000 7f9257400700 10 client.16851 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.299+0000 7f9275c00700 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5566f6165760 0x5566f6167b80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a840b1abbda63a8 server_cookie=4292ae164d7cdf3e in_seq=0 out_seq=0 2024-05-10T05:43:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.299+0000 7f9257400700 1 -- 172.21.15.131:0/54334226 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5566f6008ec0 con 0x5566f6165760 2024-05-10T05:43:23.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.299+0000 7f9257400700 10 client.16851 handle_command_reply: tid=0 2024-05-10T05:43:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.303+0000 7f92905f9b80 1 client.16851 shutdown 2024-05-10T05:43:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 msgr2=0x5566f611e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:23.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f92905f9b80 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f611e7e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5566f60f8c90 tx=0x7f925c009940 comp rx=0 tx=0).stop 2024-05-10T05:43:23.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 shutdown_connections 2024-05-10T05:43:23.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f92905f9b80 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x5566f6165760 0x5566f6167b80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f92905f9b80 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566f61084c0 0x5566f611e7e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f92905f9b80 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566f60f06a0 0x5566f611fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f92905f9b80 1 --2- 172.21.15.131:0/54334226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566f60f1ec0 0x5566f611e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 >> 172.21.15.131:0/54334226 conn(0x5566f6052a60 msgr2=0x5566f60f47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:23.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f9257400700 0 client.16851 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:23.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 shutdown_connections 2024-05-10T05:43:23.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.304+0000 7f92905f9b80 1 -- 172.21.15.131:0/54334226 wait complete. 2024-05-10T05:43:23.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.305+0000 7f92905f9b80 20 client.16851 trim_cache size 0 max 16384 2024-05-10T05:43:23.306 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:43:23.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 -- 172.21.15.131:0/807127389 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92580a1280 msgr2=0x7f92580a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:23.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92580a1280 0x7f92580a3790 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9268009900 tx=0x7f926800b040 comp rx=0 tx=0).stop 2024-05-10T05:43:23.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 -- 172.21.15.131:0/807127389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9270108b60 msgr2=0x7f92701a2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9270108b60 0x7f92701a2fb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f926400d950 tx=0x7f926400dd10 comp rx=0 tx=0).stop 2024-05-10T05:43:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 -- 172.21.15.131:0/807127389 shutdown_connections 2024-05-10T05:43:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92580a1280 0x7f92580a3790 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9270108210 0x7f92701a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9270110320 0x7f92701a9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 --2- 172.21.15.131:0/807127389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9270108b60 0x7f92701a2fb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 -- 172.21.15.131:0/807127389 >> 172.21.15.131:0/807127389 conn(0x7f927006ea00 msgr2=0x7f92701124e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 -- 172.21.15.131:0/807127389 shutdown_connections 2024-05-10T05:43:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:23.307+0000 7f9278600700 1 -- 172.21.15.131:0/807127389 wait complete. 2024-05-10T05:43:24.383 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:43:24.384 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:43:24.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:24 smithi157 ceph-mon[78427]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 221 MiB/s wr, 69 op/s 2024-05-10T05:43:24.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:24 smithi142 ceph-mon[77667]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 221 MiB/s wr, 69 op/s 2024-05-10T05:43:24.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:24 smithi131 ceph-mon[37264]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 221 MiB/s wr, 69 op/s 2024-05-10T05:43:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.263+0000 7fea9de00700 1 -- 172.21.15.131:0/590409404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea98108630 msgr2=0x7fea98108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.263+0000 7fea9de00700 1 --2- 172.21.15.131:0/590409404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea98108630 0x7fea98108a40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fea84009990 tx=0x7fea84009ca0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.263+0000 7fea9de00700 1 -- 172.21.15.131:0/590409404 shutdown_connections 2024-05-10T05:43:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.263+0000 7fea9de00700 1 --2- 172.21.15.131:0/590409404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98109c50 0x7fea9810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.263+0000 7fea9de00700 1 --2- 172.21.15.131:0/590409404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea98108630 0x7fea98108a40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.263+0000 7fea9de00700 1 --2- 172.21.15.131:0/590409404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea98109300 0x7fea98109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.263+0000 7fea9de00700 1 -- 172.21.15.131:0/590409404 >> 172.21.15.131:0/590409404 conn(0x7fea9806ea00 msgr2=0x7fea98070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:25.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.266+0000 7fea9de00700 1 -- 172.21.15.131:0/590409404 shutdown_connections 2024-05-10T05:43:25.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.266+0000 7fea9de00700 1 -- 172.21.15.131:0/590409404 wait complete. 2024-05-10T05:43:25.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.269+0000 7fea9de00700 1 Processor -- start 2024-05-10T05:43:25.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.269+0000 7fea9de00700 1 -- start start 2024-05-10T05:43:25.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.270+0000 7fea9de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea98108630 0x7fea981a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.270+0000 7fea9de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea98109300 0x7fea981a7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.270+0000 7fea9de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98109c50 0x7fea981ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.271+0000 7fea9de00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea981ae8a0 con 0x7fea98109c50 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.271+0000 7fea9de00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea981ae9e0 con 0x7fea98108630 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.271+0000 7fea9de00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea981aeb70 con 0x7fea98109300 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.271+0000 7fea96000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea98109300 0x7fea981a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.271+0000 7fea96000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea98109300 0x7fea981a7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46598/0 (socket says 172.21.15.131:46598) 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.271+0000 7fea96000700 1 -- 172.21.15.131:0/3012416215 learned_addr learned my addr 172.21.15.131:0/3012416215 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.272+0000 7fea96a00700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea98108630 0x7fea981a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.272+0000 7fea97400700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98109c50 0x7fea981ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.272+0000 7fea96000700 1 -- 172.21.15.131:0/3012416215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea98108630 msgr2=0x7fea981a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.272+0000 7fea96000700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea98108630 0x7fea981a6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.273+0000 7fea96000700 1 -- 172.21.15.131:0/3012416215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98109c50 msgr2=0x7fea981ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.273+0000 7fea96000700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98109c50 0x7fea981ae2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.273+0000 7fea96000700 1 -- 172.21.15.131:0/3012416215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840095f0 con 0x7fea98109300 2024-05-10T05:43:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.274+0000 7fea96000700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea98109300 0x7fea981a7340 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea9000ba80 tx=0x7fea9000bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.275+0000 7fea9ca00700 1 -- 172.21.15.131:0/3012416215 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9000c9b0 con 0x7fea98109300 2024-05-10T05:43:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.275+0000 7fea9ca00700 1 -- 172.21.15.131:0/3012416215 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea90014440 con 0x7fea98109300 2024-05-10T05:43:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.275+0000 7fea9de00700 1 -- 172.21.15.131:0/3012416215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea981aee50 con 0x7fea98109300 2024-05-10T05:43:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.275+0000 7fea9ca00700 1 -- 172.21.15.131:0/3012416215 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90007590 con 0x7fea98109300 2024-05-10T05:43:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.277+0000 7fea9de00700 1 -- 172.21.15.131:0/3012416215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea981af2a0 con 0x7fea98109300 2024-05-10T05:43:25.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.290+0000 7fea9ca00700 1 -- 172.21.15.131:0/3012416215 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fea900145b0 con 0x7fea98109300 2024-05-10T05:43:25.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.292+0000 7fea9ca00700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea7c0a1240 0x7fea7c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.293+0000 7fea9ca00700 1 -- 172.21.15.131:0/3012416215 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fea90099cd0 con 0x7fea98109300 2024-05-10T05:43:25.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.294+0000 7fea96a00700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea7c0a1240 0x7fea7c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.295+0000 7fea96a00700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea7c0a1240 0x7fea7c0a3750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fea84003800 tx=0x7fea8401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:25.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.311+0000 7feab5d95b80 1 Processor -- start 2024-05-10T05:43:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.312+0000 7feab5d95b80 1 -- start start 2024-05-10T05:43:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.313+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5087b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.314+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 0x55f7e5088a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.314+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5088f40 0x55f7e508f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.314+0000 7feab5d95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e4fae1d0 con 0x55f7e5087720 2024-05-10T05:43:25.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.314+0000 7feab5d95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e4fabb90 con 0x55f7e5088f40 2024-05-10T05:43:25.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.314+0000 7feab5d95b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e4fa7480 con 0x55f7e50885f0 2024-05-10T05:43:25.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.315+0000 7fea96000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 0x55f7e5088a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.315+0000 7fea96a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5087b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.315+0000 7fea96000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 0x55f7e5088a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46606/0 (socket says 172.21.15.131:46606) 2024-05-10T05:43:25.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.315+0000 7fea96000700 1 -- 172.21.15.131:0/3477843429 learned_addr learned my addr 172.21.15.131:0/3477843429 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:25.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.316+0000 7fea96000700 1 -- 172.21.15.131:0/3477843429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5088f40 msgr2=0x55f7e508f840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:43:25.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.316+0000 7fea96000700 1 --2- 172.21.15.131:0/3477843429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5088f40 0x55f7e508f840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.316+0000 7fea96000700 1 -- 172.21.15.131:0/3477843429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 msgr2=0x55f7e5087b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:25.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.316+0000 7fea96000700 1 --2- 172.21.15.131:0/3477843429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5087b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.316+0000 7fea96000700 1 -- 172.21.15.131:0/3477843429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7e4e28430 con 0x55f7e50885f0 2024-05-10T05:43:25.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.316+0000 7fea96a00700 1 --2- 172.21.15.131:0/3477843429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5087b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:43:25.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.317+0000 7fea96000700 1 --2- 172.21.15.131:0/3477843429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 0x55f7e5088a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fea90009fd0 tx=0x7fea900a0410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6234091f83b76e95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:25.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.317+0000 7fea83e00700 1 -- 172.21.15.131:0/3477843429 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90009930 con 0x55f7e50885f0 2024-05-10T05:43:25.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.317+0000 7fea83e00700 1 -- 172.21.15.131:0/3477843429 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea90098f30 con 0x55f7e50885f0 2024-05-10T05:43:25.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.317+0000 7fea83e00700 1 -- 172.21.15.131:0/3477843429 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90095740 con 0x55f7e50885f0 2024-05-10T05:43:25.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.319+0000 7feab5d95b80 1 -- 172.21.15.131:0/3477843429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 msgr2=0x55f7e5088a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:25.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.319+0000 7feab5d95b80 1 --2- 172.21.15.131:0/3477843429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 0x55f7e5088a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fea90009fd0 tx=0x7fea900a0410 comp rx=0 tx=0).stop 2024-05-10T05:43:25.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.319+0000 7feab5d95b80 1 -- 172.21.15.131:0/3477843429 shutdown_connections 2024-05-10T05:43:25.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.319+0000 7feab5d95b80 1 --2- 172.21.15.131:0/3477843429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5087b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.319+0000 7feab5d95b80 1 --2- 172.21.15.131:0/3477843429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5088f40 0x55f7e508f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.319+0000 7feab5d95b80 1 --2- 172.21.15.131:0/3477843429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 0x55f7e5088a00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.319+0000 7feab5d95b80 1 -- 172.21.15.131:0/3477843429 >> 172.21.15.131:0/3477843429 conn(0x55f7e4fe9ae0 msgr2=0x55f7e5082060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:25.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.320+0000 7feab5d95b80 1 -- 172.21.15.131:0/3477843429 shutdown_connections 2024-05-10T05:43:25.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.320+0000 7feab5d95b80 1 -- 172.21.15.131:0/3477843429 wait complete. 2024-05-10T05:43:25.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.321+0000 7feab5d95b80 1 -- start start 2024-05-10T05:43:25.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.321+0000 7feab5d95b80 1 Processor -- start 2024-05-10T05:43:25.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.322+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5087720 0x55f7e50916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.322+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e50885f0 0x55f7e5091be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.322+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 0x55f7e5098b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.322+0000 7feab5d95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e4f435e0 con 0x55f7e50885f0 2024-05-10T05:43:25.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.322+0000 7feab5d95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e4f43280 con 0x55f7e5087720 2024-05-10T05:43:25.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.322+0000 7feab5d95b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e4e2a720 con 0x55f7e5088f40 2024-05-10T05:43:25.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.322+0000 7feab5d95b80 10 client.0 _ll_register_callbacks cb 0x55f7e4f4cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:25.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.323+0000 7fea96a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5087720 0x55f7e50916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.323+0000 7fea96a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5087720 0x55f7e50916a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47688/0 (socket says 172.21.15.131:47688) 2024-05-10T05:43:25.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.323+0000 7fea96a00700 1 -- 172.21.15.131:0/959083807 learned_addr learned my addr 172.21.15.131:0/959083807 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:25.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.323+0000 7fea96000700 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e50885f0 0x55f7e5091be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.324+0000 7fea97400700 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 0x55f7e5098b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.325+0000 7fea96a00700 1 -- 172.21.15.131:0/959083807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 msgr2=0x55f7e5098b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:25.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.325+0000 7fea96a00700 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 0x55f7e5098b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.325+0000 7fea96a00700 1 -- 172.21.15.131:0/959083807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e50885f0 msgr2=0x55f7e5091be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:25.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.325+0000 7fea96a00700 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e50885f0 0x55f7e5091be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.325+0000 7fea96a00700 1 -- 172.21.15.131:0/959083807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea981af2a0 con 0x55f7e5087720 2024-05-10T05:43:25.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.325+0000 7fea96000700 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e50885f0 0x55f7e5091be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:25.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.325+0000 7fea97400700 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 0x55f7e5098b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:25.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.325+0000 7fea96a00700 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5087720 0x55f7e50916a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fea84009990 tx=0x7fea84009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d84ebf6852b93c13 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:25.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.326+0000 7fea80c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:43:25.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.326+0000 7fea80c00700 1 -- 172.21.15.131:0/959083807 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84003e10 con 0x55f7e5087720 2024-05-10T05:43:25.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.326+0000 7feab5d95b80 1 -- 172.21.15.131:0/959083807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7e4f002f0 con 0x55f7e5087720 2024-05-10T05:43:25.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.326+0000 7fea80c00700 1 -- 172.21.15.131:0/959083807 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fea8400f670 con 0x55f7e5087720 2024-05-10T05:43:25.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.327+0000 7fea80c00700 1 -- 172.21.15.131:0/959083807 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea8400f830 con 0x55f7e5087720 2024-05-10T05:43:25.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.327+0000 7feab5d95b80 10 client.27194 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:25.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.327+0000 7feab5d95b80 1 -- 172.21.15.131:0/959083807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f7e4dd4be0 con 0x55f7e5087720 2024-05-10T05:43:25.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.327+0000 7fea80c00700 1 -- 172.21.15.131:0/959083807 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7e4f002f0 con 0x55f7e5087720 2024-05-10T05:43:25.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.328+0000 7fea80c00700 1 -- 172.21.15.131:0/959083807 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fea8402fa20 con 0x55f7e5087720 2024-05-10T05:43:25.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.328+0000 7feab5d95b80 10 client.27194 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:25.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.328+0000 7feab5d95b80 10 client.27194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:43:25.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.328+0000 7feab5d95b80 20 client.27194 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:25.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.328+0000 7feab5d95b80 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f7e509d2f0 0x55f7e509f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.328+0000 7feab5d95b80 4 client.27194 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:43:25.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.328+0000 7feab5d95b80 1 -- 172.21.15.131:0/959083807 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7e5056800 con 0x55f7e509d2f0 2024-05-10T05:43:25.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.328+0000 7fea96000700 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f7e509d2f0 0x55f7e509f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.329+0000 7fea96000700 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f7e509d2f0 0x55f7e509f740 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbabad9bb979e14 server_cookie=feff583d5ffad1fa in_seq=0 out_seq=0 2024-05-10T05:43:25.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.329+0000 7fea80c00700 10 client.27194 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:25.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.330+0000 7fea80c00700 1 -- 172.21.15.131:0/959083807 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f7e5056800 con 0x55f7e509d2f0 2024-05-10T05:43:25.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.330+0000 7fea80c00700 10 client.27194 handle_command_reply: tid=0 2024-05-10T05:43:25.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.330+0000 7feab5d95b80 1 client.27194 shutdown 2024-05-10T05:43:25.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.331+0000 7feab5d95b80 1 -- 172.21.15.131:0/959083807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5087720 msgr2=0x55f7e50916a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:25.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.331+0000 7feab5d95b80 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5087720 0x55f7e50916a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fea84009990 tx=0x7fea84009310 comp rx=0 tx=0).stop 2024-05-10T05:43:25.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.331+0000 7feab5d95b80 1 -- 172.21.15.131:0/959083807 shutdown_connections 2024-05-10T05:43:25.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.331+0000 7feab5d95b80 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f7e509d2f0 0x55f7e509f740 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.331+0000 7feab5d95b80 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 0x55f7e5098b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.331+0000 7feab5d95b80 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5087720 0x55f7e50916a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.331+0000 7feab5d95b80 1 --2- 172.21.15.131:0/959083807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e50885f0 0x55f7e5091be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.331+0000 7feab5d95b80 1 -- 172.21.15.131:0/959083807 >> 172.21.15.131:0/959083807 conn(0x55f7e4fe9ae0 msgr2=0x55f7e5082030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:25.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.331+0000 7fea80c00700 0 client.27194 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:25.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.332+0000 7feab5d95b80 1 -- 172.21.15.131:0/959083807 shutdown_connections 2024-05-10T05:43:25.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.332+0000 7feab5d95b80 1 -- 172.21.15.131:0/959083807 wait complete. 2024-05-10T05:43:25.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.332+0000 7feab5d95b80 20 client.27194 trim_cache size 0 max 16384 2024-05-10T05:43:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.364+0000 7feab5d95b80 1 Processor -- start 2024-05-10T05:43:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.364+0000 7feab5d95b80 1 -- start start 2024-05-10T05:43:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.365+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5082bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.365+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 0x55f7e5098ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.365+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5088f40 0x55f7e508c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.365+0000 7feab5d95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e5083130 con 0x55f7e5087720 2024-05-10T05:43:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.365+0000 7feab5d95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e50832a0 con 0x55f7e5088f40 2024-05-10T05:43:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.365+0000 7feab5d95b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e508f740 con 0x55f7e50885f0 2024-05-10T05:43:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.365+0000 7fea96a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5082bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.365+0000 7fea96000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 0x55f7e5098ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.365+0000 7fea96a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5082bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44936/0 (socket says 172.21.15.131:44936) 2024-05-10T05:43:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.365+0000 7fea96a00700 1 -- 172.21.15.131:0/3722572928 learned_addr learned my addr 172.21.15.131:0/3722572928 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.366+0000 7fea97400700 1 --2- 172.21.15.131:0/3722572928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5088f40 0x55f7e508c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.366+0000 7fea96a00700 1 -- 172.21.15.131:0/3722572928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 msgr2=0x55f7e5098ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.366+0000 7fea96a00700 1 --2- 172.21.15.131:0/3722572928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 0x55f7e5098ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.366+0000 7fea96a00700 1 -- 172.21.15.131:0/3722572928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5088f40 msgr2=0x55f7e508c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.366+0000 7fea96a00700 1 --2- 172.21.15.131:0/3722572928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5088f40 0x55f7e508c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.366+0000 7fea96a00700 1 -- 172.21.15.131:0/3722572928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7e507e4d0 con 0x55f7e5087720 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.366+0000 7fea97400700 1 --2- 172.21.15.131:0/3722572928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5088f40 0x55f7e508c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.366+0000 7fea96a00700 1 --2- 172.21.15.131:0/3722572928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5082bf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fea84018450 tx=0x7fea84018480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d81ac0de2895327 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.367+0000 7fea8d200700 1 -- 172.21.15.131:0/3722572928 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400b9e0 con 0x55f7e5087720 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.367+0000 7fea8d200700 1 -- 172.21.15.131:0/3722572928 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea8401e040 con 0x55f7e5087720 2024-05-10T05:43:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.367+0000 7fea8d200700 1 -- 172.21.15.131:0/3722572928 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400fe20 con 0x55f7e5087720 2024-05-10T05:43:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.368+0000 7feab5d95b80 1 -- 172.21.15.131:0/3722572928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 msgr2=0x55f7e5082bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.368+0000 7feab5d95b80 1 --2- 172.21.15.131:0/3722572928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5082bf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fea84018450 tx=0x7fea84018480 comp rx=0 tx=0).stop 2024-05-10T05:43:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.369+0000 7feab5d95b80 1 -- 172.21.15.131:0/3722572928 shutdown_connections 2024-05-10T05:43:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.369+0000 7feab5d95b80 1 --2- 172.21.15.131:0/3722572928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5082bf0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.369+0000 7feab5d95b80 1 --2- 172.21.15.131:0/3722572928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e5088f40 0x55f7e508c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.369+0000 7feab5d95b80 1 --2- 172.21.15.131:0/3722572928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e50885f0 0x55f7e5098ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.369+0000 7feab5d95b80 1 -- 172.21.15.131:0/3722572928 >> 172.21.15.131:0/3722572928 conn(0x55f7e4fe9ae0 msgr2=0x55f7e5087d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.369+0000 7feab5d95b80 1 -- 172.21.15.131:0/3722572928 shutdown_connections 2024-05-10T05:43:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.369+0000 7feab5d95b80 1 -- 172.21.15.131:0/3722572928 wait complete. 2024-05-10T05:43:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.369+0000 7feab5d95b80 1 -- start start 2024-05-10T05:43:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.370+0000 7feab5d95b80 1 Processor -- start 2024-05-10T05:43:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.371+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5096560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.371+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e50885f0 0x55f7e5096aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.371+0000 7feab5d95b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 0x55f7e5097930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.371+0000 7feab5d95b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e5097e70 con 0x55f7e5087720 2024-05-10T05:43:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.371+0000 7feab5d95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e5097fe0 con 0x55f7e50885f0 2024-05-10T05:43:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.371+0000 7feab5d95b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7e50981a0 con 0x55f7e5088f40 2024-05-10T05:43:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.371+0000 7feab5d95b80 10 client.0 _ll_register_callbacks cb 0x55f7e4f4cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:25.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.371+0000 7fea96a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5096560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.372+0000 7fea96000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e50885f0 0x55f7e5096aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.372+0000 7fea96a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5096560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44952/0 (socket says 172.21.15.131:44952) 2024-05-10T05:43:25.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.372+0000 7fea96a00700 1 -- 172.21.15.131:0/739356470 learned_addr learned my addr 172.21.15.131:0/739356470 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:25.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.372+0000 7fea97400700 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 0x55f7e5097930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.372+0000 7fea96a00700 1 -- 172.21.15.131:0/739356470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 msgr2=0x55f7e5097930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.372+0000 7fea96a00700 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 0x55f7e5097930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.372+0000 7fea96a00700 1 -- 172.21.15.131:0/739356470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e50885f0 msgr2=0x55f7e5096aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.372+0000 7fea96a00700 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e50885f0 0x55f7e5096aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.372+0000 7fea96a00700 1 -- 172.21.15.131:0/739356470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7e4dd4be0 con 0x55f7e5087720 2024-05-10T05:43:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.372+0000 7fea96000700 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e50885f0 0x55f7e5096aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.373+0000 7fea96a00700 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5096560 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fea84009ce0 tx=0x7fea8400ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebb2198844a16f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.373+0000 7fea80c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:43:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.373+0000 7fea80c00700 1 -- 172.21.15.131:0/739356470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400f7b0 con 0x55f7e5087720 2024-05-10T05:43:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.373+0000 7fea80c00700 1 -- 172.21.15.131:0/739356470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fea8400fdb0 con 0x55f7e5087720 2024-05-10T05:43:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.373+0000 7feab5d95b80 1 -- 172.21.15.131:0/739356470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea90098f30 con 0x55f7e5087720 2024-05-10T05:43:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7fea80c00700 1 -- 172.21.15.131:0/739356470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea8402f480 con 0x55f7e5087720 2024-05-10T05:43:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7feab5d95b80 10 client.16863 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7feab5d95b80 1 -- 172.21.15.131:0/739356470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f7e4f6d5e0 con 0x55f7e5087720 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7fea80c00700 1 -- 172.21.15.131:0/739356470 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea90098f30 con 0x55f7e5087720 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7fea80c00700 1 -- 172.21.15.131:0/739356470 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fea84018b60 con 0x55f7e5087720 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7feab5d95b80 10 client.16863 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7feab5d95b80 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7feab5d95b80 20 client.16863 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7feab5d95b80 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f7e50f3d60 0x55f7e50f6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7feab5d95b80 4 client.16863 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.374+0000 7feab5d95b80 1 -- 172.21.15.131:0/739356470 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f7e4f9ff40 con 0x55f7e50f3d60 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.375+0000 7fea96000700 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f7e50f3d60 0x55f7e50f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.375+0000 7fea80c00700 10 client.16863 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.375+0000 7fea96000700 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f7e50f3d60 0x55f7e50f6180 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec091c4ae44c2cd server_cookie=2e514064f770ff01 in_seq=0 out_seq=0 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.377+0000 7fea80c00700 1 -- 172.21.15.131:0/739356470 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f7e4f9ff40 con 0x55f7e50f3d60 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.377+0000 7fea80c00700 10 client.16863 handle_command_reply: tid=0 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.377+0000 7feab5d95b80 1 client.16863 shutdown 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.377+0000 7feab5d95b80 1 -- 172.21.15.131:0/739356470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 msgr2=0x55f7e5096560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.377+0000 7feab5d95b80 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5096560 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fea84009ce0 tx=0x7fea8400ba00 comp rx=0 tx=0).stop 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.378+0000 7feab5d95b80 1 -- 172.21.15.131:0/739356470 shutdown_connections 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.378+0000 7feab5d95b80 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f7e50f3d60 0x55f7e50f6180 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.378+0000 7feab5d95b80 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f7e5087720 0x55f7e5096560 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.378+0000 7feab5d95b80 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7e5088f40 0x55f7e5097930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.378+0000 7feab5d95b80 1 --2- 172.21.15.131:0/739356470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7e50885f0 0x55f7e5096aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.378+0000 7feab5d95b80 1 -- 172.21.15.131:0/739356470 >> 172.21.15.131:0/739356470 conn(0x55f7e4fe9ae0 msgr2=0x55f7e5087d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:25.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.378+0000 7fea80c00700 0 client.16863 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:25.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.378+0000 7feab5d95b80 1 -- 172.21.15.131:0/739356470 shutdown_connections 2024-05-10T05:43:25.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.378+0000 7feab5d95b80 1 -- 172.21.15.131:0/739356470 wait complete. 2024-05-10T05:43:25.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.379+0000 7feab5d95b80 20 client.16863 trim_cache size 0 max 16384 2024-05-10T05:43:25.379 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:43:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:43:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "94ce672f-ebcc-436a-9b46-ca5e32c334d0", 2024-05-10T05:43:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:43:25.380 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:43:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.380+0000 7fea9de00700 1 -- 172.21.15.131:0/3012416215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea7c0a1240 msgr2=0x7fea7c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.380+0000 7fea9de00700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea7c0a1240 0x7fea7c0a3750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fea84003800 tx=0x7fea8401a040 comp rx=0 tx=0).stop 2024-05-10T05:43:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.380+0000 7fea9de00700 1 -- 172.21.15.131:0/3012416215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea98109300 msgr2=0x7fea981a7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.380+0000 7fea9de00700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea98109300 0x7fea981a7340 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea9000ba80 tx=0x7fea9000bd90 comp rx=0 tx=0).stop 2024-05-10T05:43:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.381+0000 7fea9de00700 1 -- 172.21.15.131:0/3012416215 shutdown_connections 2024-05-10T05:43:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.381+0000 7fea9de00700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea7c0a1240 0x7fea7c0a3750 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.381+0000 7fea9de00700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98109c50 0x7fea981ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.381+0000 7fea9de00700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea98108630 0x7fea981a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.381+0000 7fea9de00700 1 --2- 172.21.15.131:0/3012416215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea98109300 0x7fea981a7340 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:25.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.381+0000 7fea9de00700 1 -- 172.21.15.131:0/3012416215 >> 172.21.15.131:0/3012416215 conn(0x7fea9806ea00 msgr2=0x7fea9806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:25.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.381+0000 7fea9de00700 1 -- 172.21.15.131:0/3012416215 shutdown_connections 2024-05-10T05:43:25.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:25.381+0000 7fea9de00700 1 -- 172.21.15.131:0/3012416215 wait complete. 2024-05-10T05:43:25.423 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:43:25.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:25 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:43:25.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:25 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:43:25.705 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:25 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:43:25.705 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:25 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:43:25.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:25 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:43:25.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:25 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:43:25.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:25 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:43:25.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:25 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:43:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:25 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:43:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:25 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:43:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:25 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:43:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:25 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:43:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.284+0000 7f38bb000700 1 -- 172.21.15.131:0/958840851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38b4077460 msgr2=0x7f38b4079880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.284+0000 7f38bb000700 1 --2- 172.21.15.131:0/958840851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38b4077460 0x7f38b4079880 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00b0a0 tx=0x7f38ac00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.285+0000 7f38bb000700 1 -- 172.21.15.131:0/958840851 shutdown_connections 2024-05-10T05:43:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.285+0000 7f38bb000700 1 --2- 172.21.15.131:0/958840851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38b4109650 0x7f38b4109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.285+0000 7f38bb000700 1 --2- 172.21.15.131:0/958840851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38b4077460 0x7f38b4079880 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.285+0000 7f38bb000700 1 --2- 172.21.15.131:0/958840851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b4108660 0x7f38b4108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.285+0000 7f38bb000700 1 -- 172.21.15.131:0/958840851 >> 172.21.15.131:0/958840851 conn(0x7f38b406e970 msgr2=0x7f38b4070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.285+0000 7f38bb000700 1 -- 172.21.15.131:0/958840851 shutdown_connections 2024-05-10T05:43:26.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.285+0000 7f38bb000700 1 -- 172.21.15.131:0/958840851 wait complete. 2024-05-10T05:43:26.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.287+0000 7f38bb000700 1 Processor -- start 2024-05-10T05:43:26.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.287+0000 7f38bb000700 1 -- start start 2024-05-10T05:43:26.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.288+0000 7f38bb000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b4077460 0x7f38b41b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.289+0000 7f38bb000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38b4108660 0x7f38b41b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.289+0000 7f38b3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b4077460 0x7f38b41b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.289+0000 7f38b3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b4077460 0x7f38b41b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46658/0 (socket says 172.21.15.131:46658) 2024-05-10T05:43:26.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.289+0000 7f38b3e00700 1 -- 172.21.15.131:0/1808087403 learned_addr learned my addr 172.21.15.131:0/1808087403 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:26.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.289+0000 7f38bb000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38b4109650 0x7f38b41b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.289+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b41b92e0 con 0x7f38b4109650 2024-05-10T05:43:26.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.289+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b41b9420 con 0x7f38b4108660 2024-05-10T05:43:26.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.289+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b41b95e0 con 0x7f38b4077460 2024-05-10T05:43:26.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.289+0000 7f38b3400700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38b4108660 0x7f38b41b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.290+0000 7f38b9000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38b4109650 0x7f38b41b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.291+0000 7f38b3e00700 1 -- 172.21.15.131:0/1808087403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38b4108660 msgr2=0x7f38b41b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:26.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.291+0000 7f38b3e00700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38b4108660 0x7f38b41b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.291+0000 7f38b3e00700 1 -- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38b4109650 msgr2=0x7f38b41b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:26.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.291+0000 7f38b3e00700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38b4109650 0x7f38b41b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.291+0000 7f38b3e00700 1 -- 172.21.15.131:0/1808087403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac009d00 con 0x7f38b4077460 2024-05-10T05:43:26.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.291+0000 7f38b9000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38b4109650 0x7f38b41b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:26.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.291+0000 7f38b3400700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38b4108660 0x7f38b41b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:26.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.292+0000 7f38b3e00700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b4077460 0x7f38b41b1890 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f38a400b500 tx=0x7f38a400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:26.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.292+0000 7f38b0c00700 1 -- 172.21.15.131:0/1808087403 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4011830 con 0x7f38b4077460 2024-05-10T05:43:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.293+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b41b98c0 con 0x7f38b4077460 2024-05-10T05:43:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.293+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b41b9e50 con 0x7f38b4077460 2024-05-10T05:43:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.293+0000 7f38b0c00700 1 -- 172.21.15.131:0/1808087403 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a4011e50 con 0x7f38b4077460 2024-05-10T05:43:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.293+0000 7f38b0c00700 1 -- 172.21.15.131:0/1808087403 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4010de0 con 0x7f38b4077460 2024-05-10T05:43:26.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.300+0000 7f38b0c00700 1 -- 172.21.15.131:0/1808087403 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f38a401a400 con 0x7f38b4077460 2024-05-10T05:43:26.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.301+0000 7f38d300bb80 1 Processor -- start 2024-05-10T05:43:26.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.303+0000 7f38d300bb80 1 -- start start 2024-05-10T05:43:26.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.303+0000 7f38b0c00700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f38980a12c0 0x7f38980a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.303+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f90036a0 0x55f2f9003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.303+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004570 0x55f2f9004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.303+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004ec0 0x55f2f900b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.303+0000 7f38d300bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f8f2a150 con 0x55f2f9004ec0 2024-05-10T05:43:26.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.303+0000 7f38d300bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f8f27b10 con 0x55f2f90036a0 2024-05-10T05:43:26.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.303+0000 7f38d300bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f8f23400 con 0x55f2f9004570 2024-05-10T05:43:26.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b0c00700 1 -- 172.21.15.131:0/1808087403 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f38a40cc770 con 0x7f38b4077460 2024-05-10T05:43:26.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004570 0x55f2f9004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f90036a0 0x55f2f9003af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b3400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004570 0x55f2f9004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:46664/0 (socket says 172.21.15.131:46664) 2024-05-10T05:43:26.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b3400700 1 -- 172.21.15.131:0/2189904216 learned_addr learned my addr 172.21.15.131:0/2189904216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:26.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f90036a0 0x55f2f9003af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47736/0 (socket says 172.21.15.131:47736) 2024-05-10T05:43:26.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b9000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f38980a12c0 0x7f38980a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b3400700 1 -- 172.21.15.131:0/2189904216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f90036a0 msgr2=0x55f2f9003af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:26.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b3400700 1 --2- 172.21.15.131:0/2189904216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f90036a0 0x55f2f9003af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b3400700 1 -- 172.21.15.131:0/2189904216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004ec0 msgr2=0x55f2f900b7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:43:26.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b3400700 1 --2- 172.21.15.131:0/2189904216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004ec0 0x55f2f900b7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.305+0000 7f38b3400700 1 -- 172.21.15.131:0/2189904216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2f8da43b0 con 0x55f2f9004570 2024-05-10T05:43:26.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.306+0000 7f38b3400700 1 --2- 172.21.15.131:0/2189904216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004570 0x55f2f9004980 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f38a00115f0 tx=0x7f38a0011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9fd7dbf5aa31a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:26.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.306+0000 7f38a9600700 1 -- 172.21.15.131:0/2189904216 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0004030 con 0x55f2f9004570 2024-05-10T05:43:26.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.306+0000 7f38a9600700 1 -- 172.21.15.131:0/2189904216 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a0003680 con 0x55f2f9004570 2024-05-10T05:43:26.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.306+0000 7f38a9600700 1 -- 172.21.15.131:0/2189904216 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0019730 con 0x55f2f9004570 2024-05-10T05:43:26.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.308+0000 7f38d300bb80 1 -- 172.21.15.131:0/2189904216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004570 msgr2=0x55f2f9004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:26.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.308+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2189904216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004570 0x55f2f9004980 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f38a00115f0 tx=0x7f38a0011900 comp rx=0 tx=0).stop 2024-05-10T05:43:26.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.308+0000 7f38b9000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f38980a12c0 0x7f38980a37d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009900 tx=0x7f38ac00b700 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:26.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.308+0000 7f38d300bb80 1 -- 172.21.15.131:0/2189904216 shutdown_connections 2024-05-10T05:43:26.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.308+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2189904216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004ec0 0x55f2f900b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.308+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2189904216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f90036a0 0x55f2f9003af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.308+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2189904216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004570 0x55f2f9004980 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.308+0000 7f38d300bb80 1 -- 172.21.15.131:0/2189904216 >> 172.21.15.131:0/2189904216 conn(0x55f2f8f65a60 msgr2=0x55f2f8ffdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:26.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.308+0000 7f38d300bb80 1 -- 172.21.15.131:0/2189904216 shutdown_connections 2024-05-10T05:43:26.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.308+0000 7f38d300bb80 1 -- 172.21.15.131:0/2189904216 wait complete. 2024-05-10T05:43:26.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.309+0000 7f38d300bb80 1 -- start start 2024-05-10T05:43:26.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.310+0000 7f38d300bb80 1 Processor -- start 2024-05-10T05:43:26.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.311+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f90036a0 0x55f2f9015ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.311+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f9016420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.311+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004ec0 0x55f2f901d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.311+0000 7f38d300bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f8ebf560 con 0x55f2f9004ec0 2024-05-10T05:43:26.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.311+0000 7f38d300bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f8ebf200 con 0x55f2f9004570 2024-05-10T05:43:26.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.311+0000 7f38d300bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f8da66a0 con 0x55f2f90036a0 2024-05-10T05:43:26.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.311+0000 7f38d300bb80 10 client.0 _ll_register_callbacks cb 0x55f2f8ec8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:26.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.312+0000 7f38b3e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f90036a0 0x55f2f9015ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.312+0000 7f38b3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f9016420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.312+0000 7f38b3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f9016420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47742/0 (socket says 172.21.15.131:47742) 2024-05-10T05:43:26.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.312+0000 7f38b3400700 1 -- 172.21.15.131:0/2423364717 learned_addr learned my addr 172.21.15.131:0/2423364717 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:26.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.313+0000 7f38b3400700 1 -- 172.21.15.131:0/2423364717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f90036a0 msgr2=0x55f2f9015ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:26.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.313+0000 7f38b3400700 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f90036a0 0x55f2f9015ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.313+0000 7f38b3400700 1 -- 172.21.15.131:0/2423364717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004ec0 msgr2=0x55f2f901d380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:43:26.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.313+0000 7f38b3400700 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004ec0 0x55f2f901d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.313+0000 7f38b3400700 1 -- 172.21.15.131:0/2423364717 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a0009d50 con 0x55f2f9004570 2024-05-10T05:43:26.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.313+0000 7f38b3e00700 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f90036a0 0x55f2f9015ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:26.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.313+0000 7f38b3400700 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f9016420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f38a000a010 tx=0x7f38a0004210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcfdff37db08e15c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:26.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.313+0000 7f389e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:43:26.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.314+0000 7f389e000700 1 -- 172.21.15.131:0/2423364717 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a00044e0 con 0x55f2f9004570 2024-05-10T05:43:26.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.314+0000 7f38d300bb80 1 -- 172.21.15.131:0/2423364717 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2f8e7c270 con 0x55f2f9004570 2024-05-10T05:43:26.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.314+0000 7f389e000700 1 -- 172.21.15.131:0/2423364717 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f38a0005240 con 0x55f2f9004570 2024-05-10T05:43:26.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.314+0000 7f389e000700 1 -- 172.21.15.131:0/2423364717 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38a0005400 con 0x55f2f9004570 2024-05-10T05:43:26.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.315+0000 7f38d300bb80 10 client.27212 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:26.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.315+0000 7f38d300bb80 1 -- 172.21.15.131:0/2423364717 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f2f8d50b60 con 0x55f2f9004570 2024-05-10T05:43:26.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.317+0000 7f389e000700 1 -- 172.21.15.131:0/2423364717 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2f8e7c270 con 0x55f2f9004570 2024-05-10T05:43:26.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.317+0000 7f389e000700 1 -- 172.21.15.131:0/2423364717 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38a0019610 con 0x55f2f9004570 2024-05-10T05:43:26.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.317+0000 7f38d300bb80 10 client.27212 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:26.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.317+0000 7f38d300bb80 10 client.27212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:43:26.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.318+0000 7f38d300bb80 20 client.27212 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:26.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.318+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f2f90100c0 0x55f2f9012510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.318+0000 7f38d300bb80 4 client.27212 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:43:26.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.318+0000 7f38d300bb80 1 -- 172.21.15.131:0/2423364717 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2f8fd2780 con 0x55f2f90100c0 2024-05-10T05:43:26.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.322+0000 7f38b3e00700 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f2f90100c0 0x55f2f9012510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.322+0000 7f38b3e00700 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f2f90100c0 0x55f2f9012510 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b1e14989b71a19 server_cookie=9e7ec57f2fb7c37a in_seq=0 out_seq=0 2024-05-10T05:43:26.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.322+0000 7f389e000700 10 client.27212 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.323+0000 7f389e000700 1 -- 172.21.15.131:0/2423364717 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2f8fd2780 con 0x55f2f90100c0 2024-05-10T05:43:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.323+0000 7f389e000700 10 client.27212 handle_command_reply: tid=0 2024-05-10T05:43:26.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.324+0000 7f38d300bb80 1 client.27212 shutdown 2024-05-10T05:43:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.324+0000 7f38d300bb80 1 -- 172.21.15.131:0/2423364717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 msgr2=0x55f2f9016420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.324+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f9016420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f38a000a010 tx=0x7f38a0004210 comp rx=0 tx=0).stop 2024-05-10T05:43:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.325+0000 7f38d300bb80 1 -- 172.21.15.131:0/2423364717 shutdown_connections 2024-05-10T05:43:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.325+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f2f90100c0 0x55f2f9012510 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.325+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004ec0 0x55f2f901d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.325+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f90036a0 0x55f2f9015ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.325+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2423364717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f9016420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.325+0000 7f389e000700 0 client.27212 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:26.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.325+0000 7f38d300bb80 1 -- 172.21.15.131:0/2423364717 >> 172.21.15.131:0/2423364717 conn(0x55f2f8f65a60 msgr2=0x55f2f8ffdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:26.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.325+0000 7f38d300bb80 1 -- 172.21.15.131:0/2423364717 shutdown_connections 2024-05-10T05:43:26.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.325+0000 7f38d300bb80 1 -- 172.21.15.131:0/2423364717 wait complete. 2024-05-10T05:43:26.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.326+0000 7f38d300bb80 20 client.27212 trim_cache size 0 max 16384 2024-05-10T05:43:26.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.364+0000 7f38d300bb80 1 Processor -- start 2024-05-10T05:43:26.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.364+0000 7f38d300bb80 1 -- start start 2024-05-10T05:43:26.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.365+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f90036a0 0x55f2f900bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.365+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f901d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.365+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004ec0 0x55f2f9015ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.365+0000 7f38d300bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f9005360 con 0x55f2f90036a0 2024-05-10T05:43:26.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.365+0000 7f38d300bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f9016d40 con 0x55f2f9004570 2024-05-10T05:43:26.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.365+0000 7f38d300bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f900b670 con 0x55f2f9004ec0 2024-05-10T05:43:26.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.365+0000 7f38b3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f901d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.365+0000 7f38b3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f901d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:47756/0 (socket says 172.21.15.131:47756) 2024-05-10T05:43:26.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.365+0000 7f38b3400700 1 -- 172.21.15.131:0/2275198702 learned_addr learned my addr 172.21.15.131:0/2275198702 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:26.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.365+0000 7f38b9000700 1 --2- 172.21.15.131:0/2275198702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004ec0 0x55f2f9015ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.366+0000 7f38b3e00700 1 --2- 172.21.15.131:0/2275198702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f90036a0 0x55f2f900bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.366+0000 7f38b9000700 1 -- 172.21.15.131:0/2275198702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 msgr2=0x55f2f901d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:26.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.366+0000 7f38b9000700 1 --2- 172.21.15.131:0/2275198702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f901d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.366+0000 7f38b9000700 1 -- 172.21.15.131:0/2275198702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f90036a0 msgr2=0x55f2f900bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:26.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.366+0000 7f38b9000700 1 --2- 172.21.15.131:0/2275198702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f90036a0 0x55f2f900bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.366+0000 7f38b9000700 1 -- 172.21.15.131:0/2275198702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2f8ee9560 con 0x55f2f9004ec0 2024-05-10T05:43:26.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.366+0000 7f38b9000700 1 --2- 172.21.15.131:0/2275198702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004ec0 0x55f2f9015ea0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00b070 tx=0x7f38ac01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=814c6f127cf0e305 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:26.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.367+0000 7f38aa000700 1 -- 172.21.15.131:0/2275198702 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00e040 con 0x55f2f9004ec0 2024-05-10T05:43:26.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.367+0000 7f38aa000700 1 -- 172.21.15.131:0/2275198702 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38ac00bd30 con 0x55f2f9004ec0 2024-05-10T05:43:26.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.367+0000 7f38aa000700 1 -- 172.21.15.131:0/2275198702 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00e040 con 0x55f2f9004ec0 2024-05-10T05:43:26.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.368+0000 7f38d300bb80 1 -- 172.21.15.131:0/2275198702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004ec0 msgr2=0x55f2f9015ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:26.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.368+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2275198702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004ec0 0x55f2f9015ea0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00b070 tx=0x7f38ac01d000 comp rx=0 tx=0).stop 2024-05-10T05:43:26.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.369+0000 7f38d300bb80 1 -- 172.21.15.131:0/2275198702 shutdown_connections 2024-05-10T05:43:26.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.369+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2275198702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f90036a0 0x55f2f900bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.369+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2275198702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004ec0 0x55f2f9015ea0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.369+0000 7f38d300bb80 1 --2- 172.21.15.131:0/2275198702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f9004570 0x55f2f901d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.369+0000 7f38d300bb80 1 -- 172.21.15.131:0/2275198702 >> 172.21.15.131:0/2275198702 conn(0x55f2f8f65a60 msgr2=0x55f2f9012510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:26.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.369+0000 7f38d300bb80 1 -- 172.21.15.131:0/2275198702 shutdown_connections 2024-05-10T05:43:26.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.369+0000 7f38d300bb80 1 -- 172.21.15.131:0/2275198702 wait complete. 2024-05-10T05:43:26.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.370+0000 7f38d300bb80 1 -- start start 2024-05-10T05:43:26.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.370+0000 7f38d300bb80 1 Processor -- start 2024-05-10T05:43:26.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.371+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f90036a0 0x55f2f901b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.371+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004570 0x55f2f901b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.371+0000 7f38d300bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004ec0 0x55f2f901c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.371+0000 7f38d300bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f901cc50 con 0x55f2f9004570 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.371+0000 7f38d300bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f901cdc0 con 0x55f2f90036a0 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.371+0000 7f38d300bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f901cf80 con 0x55f2f9004ec0 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.371+0000 7f38d300bb80 10 client.0 _ll_register_callbacks cb 0x55f2f8ec8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.372+0000 7f38b3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004570 0x55f2f901b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.372+0000 7f38b3400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004570 0x55f2f901b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44982/0 (socket says 172.21.15.131:44982) 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.372+0000 7f38b3400700 1 -- 172.21.15.131:0/360462946 learned_addr learned my addr 172.21.15.131:0/360462946 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.373+0000 7f38b3400700 1 -- 172.21.15.131:0/360462946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004ec0 msgr2=0x55f2f901c710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.373+0000 7f38b3400700 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004ec0 0x55f2f901c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.373+0000 7f38b3400700 1 -- 172.21.15.131:0/360462946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f90036a0 msgr2=0x55f2f901b2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.373+0000 7f38b3400700 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f90036a0 0x55f2f901b2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.373+0000 7f38b3400700 1 -- 172.21.15.131:0/360462946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b407a8d0 con 0x55f2f9004570 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.374+0000 7f38b3400700 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004570 0x55f2f901b7e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f38a0019a20 tx=0x7f38a00092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535b833dec9e0606 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:26.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.374+0000 7f389e000700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:43:26.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.374+0000 7f389e000700 1 -- 172.21.15.131:0/360462946 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0019c90 con 0x55f2f9004570 2024-05-10T05:43:26.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.375+0000 7f389e000700 1 -- 172.21.15.131:0/360462946 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f38a0000e40 con 0x55f2f9004570 2024-05-10T05:43:26.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.375+0000 7f38d300bb80 1 -- 172.21.15.131:0/360462946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38a0003680 con 0x55f2f9004570 2024-05-10T05:43:26.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.375+0000 7f389e000700 1 -- 172.21.15.131:0/360462946 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38a0005450 con 0x55f2f9004570 2024-05-10T05:43:26.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.375+0000 7f38d300bb80 10 client.16872 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:26.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.375+0000 7f38d300bb80 1 -- 172.21.15.131:0/360462946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f2f8da8cd0 con 0x55f2f9004570 2024-05-10T05:43:26.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.376+0000 7f389e000700 1 -- 172.21.15.131:0/360462946 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a0003680 con 0x55f2f9004570 2024-05-10T05:43:26.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.376+0000 7f389e000700 1 -- 172.21.15.131:0/360462946 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38a003d070 con 0x55f2f9004570 2024-05-10T05:43:26.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.378+0000 7f38d300bb80 10 client.16872 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:26.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.378+0000 7f38d300bb80 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:43:26.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.378+0000 7f38d300bb80 20 client.16872 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:26.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.378+0000 7f38d300bb80 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f2f90100c0 0x55f2f9082c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:26.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.378+0000 7f38d300bb80 4 client.16872 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.378+0000 7f38d300bb80 1 -- 172.21.15.131:0/360462946 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f2f8f1bec0 con 0x55f2f90100c0 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.379+0000 7f38b3e00700 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f2f90100c0 0x55f2f9082c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.379+0000 7f389e000700 10 client.16872 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.379+0000 7f38b3e00700 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f2f90100c0 0x55f2f9082c90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78eb9fd686e8bbc8 server_cookie=4e90f370e27a88da in_seq=0 out_seq=0 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f389e000700 1 -- 172.21.15.131:0/360462946 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f2f8f1bec0 con 0x55f2f90100c0 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f389e000700 10 client.16872 handle_command_reply: tid=0 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f38d300bb80 1 client.16872 shutdown 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f38d300bb80 1 -- 172.21.15.131:0/360462946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004570 msgr2=0x55f2f901b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f38d300bb80 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004570 0x55f2f901b7e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f38a0019a20 tx=0x7f38a00092d0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f38d300bb80 1 -- 172.21.15.131:0/360462946 shutdown_connections 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f38d300bb80 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55f2f90100c0 0x55f2f9082c90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f38d300bb80 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2f9004ec0 0x55f2f901c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f38d300bb80 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2f90036a0 0x55f2f901b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f38d300bb80 1 --2- 172.21.15.131:0/360462946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f2f9004570 0x55f2f901b7e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.380+0000 7f38d300bb80 1 -- 172.21.15.131:0/360462946 >> 172.21.15.131:0/360462946 conn(0x55f2f8f65a60 msgr2=0x55f2f9012510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:26.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.381+0000 7f389e000700 0 client.16872 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:26.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.381+0000 7f38d300bb80 1 -- 172.21.15.131:0/360462946 shutdown_connections 2024-05-10T05:43:26.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.381+0000 7f38d300bb80 1 -- 172.21.15.131:0/360462946 wait complete. 2024-05-10T05:43:26.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.382+0000 7f38d300bb80 20 client.16872 trim_cache size 0 max 16384 2024-05-10T05:43:26.384 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:43:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:43:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:43:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "94ce672f-ebcc-436a-9b46-ca5e32c334d0": { 2024-05-10T05:43:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:43:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "94ce672f-ebcc-436a-9b46-ca5e32c334d0", 2024-05-10T05:43:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:43:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:43:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:43:26.385 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:43:26.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.390+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f38980a12c0 msgr2=0x7f38980a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:26.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.390+0000 7f38bb000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f38980a12c0 0x7f38980a37d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009900 tx=0x7f38ac00b700 comp rx=0 tx=0).stop 2024-05-10T05:43:26.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.390+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b4077460 msgr2=0x7f38b41b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:26.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.390+0000 7f38bb000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b4077460 0x7f38b41b1890 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f38a400b500 tx=0x7f38a400b810 comp rx=0 tx=0).stop 2024-05-10T05:43:26.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.391+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 shutdown_connections 2024-05-10T05:43:26.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.391+0000 7f38bb000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f38980a12c0 0x7f38980a37d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.391+0000 7f38bb000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38b4109650 0x7f38b41b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.391+0000 7f38bb000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b4077460 0x7f38b41b1890 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.391+0000 7f38bb000700 1 --2- 172.21.15.131:0/1808087403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38b4108660 0x7f38b41b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:26.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.391+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 >> 172.21.15.131:0/1808087403 conn(0x7f38b406e970 msgr2=0x7f38b4078700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:26.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.392+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 shutdown_connections 2024-05-10T05:43:26.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:26.392+0000 7f38bb000700 1 -- 172.21.15.131:0/1808087403 wait complete. 2024-05-10T05:43:26.468 INFO:tasks.cephfs.filesystem:scrub status for tag:94ce672f-ebcc-436a-9b46-ca5e32c334d0 - {'path': '/', 'tag': '94ce672f-ebcc-436a-9b46-ca5e32c334d0', 'options': 'recursive,force'} 2024-05-10T05:43:26.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:26 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:43:26.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:26 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:43:26.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:26 smithi131 ceph-mon[37264]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 284 MiB/s wr, 88 op/s 2024-05-10T05:43:26.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:26 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:43:26.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:26 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:43:26.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:26 smithi142 ceph-mon[77667]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 284 MiB/s wr, 88 op/s 2024-05-10T05:43:27.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:26 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:43:27.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:26 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:43:27.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:26 smithi157 ceph-mon[78427]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 284 MiB/s wr, 88 op/s 2024-05-10T05:43:28.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:28 smithi142 ceph-mon[77667]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:43:28.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:28 smithi131 ceph-mon[37264]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:43:29.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:28 smithi157 ceph-mon[78427]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 233 MiB/s wr, 73 op/s 2024-05-10T05:43:30.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:30 smithi142 ceph-mon[77667]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 275 MiB/s wr, 84 op/s 2024-05-10T05:43:30.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:30 smithi131 ceph-mon[37264]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 275 MiB/s wr, 84 op/s 2024-05-10T05:43:31.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:30 smithi157 ceph-mon[78427]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 275 MiB/s wr, 84 op/s 2024-05-10T05:43:32.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:32 smithi142 ceph-mon[77667]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 268 MiB/s wr, 83 op/s 2024-05-10T05:43:32.964 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:32 smithi131 ceph-mon[37264]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 268 MiB/s wr, 83 op/s 2024-05-10T05:43:33.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:32 smithi157 ceph-mon[78427]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 268 MiB/s wr, 83 op/s 2024-05-10T05:43:34.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:34 smithi142 ceph-mon[77667]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 238 MiB/s wr, 73 op/s 2024-05-10T05:43:34.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:34 smithi131 ceph-mon[37264]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 238 MiB/s wr, 73 op/s 2024-05-10T05:43:35.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:34 smithi157 ceph-mon[78427]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 238 MiB/s wr, 73 op/s 2024-05-10T05:43:35.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:35 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:43:36.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:35 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:43:36.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:35 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:43:36.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:36 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:43:36.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:36 smithi131 ceph-mon[37264]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 300 MiB/s wr, 91 op/s 2024-05-10T05:43:37.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:36 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:43:37.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:36 smithi157 ceph-mon[78427]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 300 MiB/s wr, 91 op/s 2024-05-10T05:43:37.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:36 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:43:37.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:36 smithi142 ceph-mon[77667]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 300 MiB/s wr, 91 op/s 2024-05-10T05:43:38.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:38 smithi131 ceph-mon[37264]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:43:39.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:38 smithi157 ceph-mon[78427]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:43:39.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:38 smithi142 ceph-mon[77667]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:43:40.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:39 smithi157 ceph-mon[78427]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 85 op/s 2024-05-10T05:43:40.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:39 smithi142 ceph-mon[77667]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 85 op/s 2024-05-10T05:43:40.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:39 smithi131 ceph-mon[37264]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 85 op/s 2024-05-10T05:43:42.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:42 smithi157 ceph-mon[78427]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-05-10T05:43:42.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:42 smithi142 ceph-mon[77667]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-05-10T05:43:42.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:42 smithi131 ceph-mon[37264]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-05-10T05:43:44.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:44 smithi157 ceph-mon[78427]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 MiB/s wr, 73 op/s 2024-05-10T05:43:44.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:44 smithi142 ceph-mon[77667]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 MiB/s wr, 73 op/s 2024-05-10T05:43:44.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:44 smithi131 ceph-mon[37264]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 MiB/s wr, 73 op/s 2024-05-10T05:43:46.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:46 smithi157 ceph-mon[78427]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 295 MiB/s wr, 89 op/s 2024-05-10T05:43:46.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:46 smithi142 ceph-mon[77667]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 295 MiB/s wr, 89 op/s 2024-05-10T05:43:46.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:46 smithi131 ceph-mon[37264]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 295 MiB/s wr, 89 op/s 2024-05-10T05:43:48.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:48 smithi157 ceph-mon[78427]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:43:48.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:48 smithi142 ceph-mon[77667]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:43:48.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:48 smithi131 ceph-mon[37264]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:43:50.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:50 smithi142 ceph-mon[77667]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 281 MiB/s wr, 83 op/s 2024-05-10T05:43:50.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:50 smithi131 ceph-mon[37264]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 281 MiB/s wr, 83 op/s 2024-05-10T05:43:51.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:50 smithi157 ceph-mon[78427]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 281 MiB/s wr, 83 op/s 2024-05-10T05:43:52.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:52 smithi142 ceph-mon[77667]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 272 MiB/s wr, 82 op/s 2024-05-10T05:43:52.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:52 smithi131 ceph-mon[37264]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 272 MiB/s wr, 82 op/s 2024-05-10T05:43:53.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:52 smithi157 ceph-mon[78427]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 272 MiB/s wr, 82 op/s 2024-05-10T05:43:54.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:54 smithi142 ceph-mon[77667]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 253 MiB/s wr, 74 op/s 2024-05-10T05:43:54.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:54 smithi131 ceph-mon[37264]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 253 MiB/s wr, 74 op/s 2024-05-10T05:43:55.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:54 smithi157 ceph-mon[78427]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 253 MiB/s wr, 74 op/s 2024-05-10T05:43:56.470 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:43:56.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:56 smithi142 ceph-mon[77667]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 291 MiB/s wr, 87 op/s 2024-05-10T05:43:56.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:56 smithi131 ceph-mon[37264]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 291 MiB/s wr, 87 op/s 2024-05-10T05:43:57.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:56 smithi157 ceph-mon[78427]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 291 MiB/s wr, 87 op/s 2024-05-10T05:43:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.279+0000 7f5ff0400700 1 -- 172.21.15.131:0/3600694550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe8108b60 msgr2=0x7f5fe810fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.279+0000 7f5ff0400700 1 --2- 172.21.15.131:0/3600694550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe8108b60 0x7f5fe810fde0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40099c0 tx=0x7f5fd4009cd0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.280+0000 7f5ff0400700 1 -- 172.21.15.131:0/3600694550 shutdown_connections 2024-05-10T05:43:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.280+0000 7f5ff0400700 1 --2- 172.21.15.131:0/3600694550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe8108210 0x7f5fe8108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.280+0000 7f5ff0400700 1 --2- 172.21.15.131:0/3600694550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8110320 0x7f5fe8112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.280+0000 7f5ff0400700 1 --2- 172.21.15.131:0/3600694550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe8108b60 0x7f5fe810fde0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.281+0000 7f5ff0400700 1 -- 172.21.15.131:0/3600694550 >> 172.21.15.131:0/3600694550 conn(0x7f5fe806ea00 msgr2=0x7f5fe8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.281+0000 7f5ff0400700 1 -- 172.21.15.131:0/3600694550 shutdown_connections 2024-05-10T05:43:57.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.282+0000 7f5ff0400700 1 -- 172.21.15.131:0/3600694550 wait complete. 2024-05-10T05:43:57.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.284+0000 7f5ff0400700 1 Processor -- start 2024-05-10T05:43:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.285+0000 7f5ff0400700 1 -- start start 2024-05-10T05:43:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.285+0000 7f5ff0400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe8108210 0x7f5fe810b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.285+0000 7f5ff0400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8108b60 0x7f5fe8109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.286+0000 7f5ff0400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe8110320 0x7f5fe810a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.286+0000 7f5fed000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8108b60 0x7f5fe8109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.286+0000 7f5fed000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8108b60 0x7f5fe8109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54276/0 (socket says 172.21.15.131:54276) 2024-05-10T05:43:57.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.286+0000 7f5fed000700 1 -- 172.21.15.131:0/2817142520 learned_addr learned my addr 172.21.15.131:0/2817142520 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:57.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.286+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a42e0 con 0x7f5fe8108210 2024-05-10T05:43:57.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.287+0000 7f5fee400700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe8110320 0x7f5fe810a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.287+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a4450 con 0x7f5fe8110320 2024-05-10T05:43:57.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.287+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe810a7d0 con 0x7f5fe8108b60 2024-05-10T05:43:57.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.287+0000 7f5fed000700 1 -- 172.21.15.131:0/2817142520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe8110320 msgr2=0x7f5fe810a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:57.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.287+0000 7f5fed000700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe8110320 0x7f5fe810a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.287+0000 7f5fed000700 1 -- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe8108210 msgr2=0x7f5fe810b630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:43:57.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.288+0000 7f5feda00700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe8108210 0x7f5fe810b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.288+0000 7f5fed000700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe8108210 0x7f5fe810b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.288+0000 7f5fed000700 1 -- 172.21.15.131:0/2817142520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd40095f0 con 0x7f5fe8108b60 2024-05-10T05:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.288+0000 7f5fee400700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe8110320 0x7f5fe810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.289+0000 7f5fed000700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8108b60 0x7f5fe8109cd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400bd30 tx=0x7f5fd400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.289+0000 7f5fe6000700 1 -- 172.21.15.131:0/2817142520 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd401d070 con 0x7f5fe8108b60 2024-05-10T05:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.289+0000 7f5fe6000700 1 -- 172.21.15.131:0/2817142520 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fd4003df0 con 0x7f5fe8108b60 2024-05-10T05:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.289+0000 7f5fe6000700 1 -- 172.21.15.131:0/2817142520 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400fe80 con 0x7f5fe8108b60 2024-05-10T05:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.290+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe810aa50 con 0x7f5fe8108b60 2024-05-10T05:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.290+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe81aefb0 con 0x7f5fe8108b60 2024-05-10T05:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.290+0000 7f5feda00700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe8108210 0x7f5fe810b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:43:57.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.294+0000 7f5fe6000700 1 -- 172.21.15.131:0/2817142520 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5fd402b430 con 0x7f5fe8108b60 2024-05-10T05:43:57.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.297+0000 7f5fe6000700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5fcc0a1380 0x7f5fcc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.297+0000 7f5fe6000700 1 -- 172.21.15.131:0/2817142520 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5fd40cec90 con 0x7f5fe8108b60 2024-05-10T05:43:57.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.298+0000 7f5feda00700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5fcc0a1380 0x7f5fcc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.299+0000 7f5feda00700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5fcc0a1380 0x7f5fcc0a3890 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005fd0 tx=0x7f5fd8005e20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:57.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.301+0000 7f600828fb80 1 Processor -- start 2024-05-10T05:43:57.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.302+0000 7f600828fb80 1 -- start start 2024-05-10T05:43:57.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.302+0000 7f600828fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017e3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.302+0000 7f600828fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017e4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.303+0000 7f600828fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4ec0 0x55fb017eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.303+0000 7f600828fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb0170a150 con 0x55fb017e4ec0 2024-05-10T05:43:57.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.303+0000 7f600828fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb01707b10 con 0x55fb017e36a0 2024-05-10T05:43:57.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.303+0000 7f600828fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb01703400 con 0x55fb017e4570 2024-05-10T05:43:57.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.303+0000 7f5feda00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017e3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.303+0000 7f5feda00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017e3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56812/0 (socket says 172.21.15.131:56812) 2024-05-10T05:43:57.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.303+0000 7f5feda00700 1 -- 172.21.15.131:0/539021497 learned_addr learned my addr 172.21.15.131:0/539021497 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:57.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.303+0000 7f5fee400700 1 --2- 172.21.15.131:0/539021497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4ec0 0x55fb017eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.303+0000 7f5fed000700 1 --2- 172.21.15.131:0/539021497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.304+0000 7f5fee400700 1 -- 172.21.15.131:0/539021497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 msgr2=0x55fb017e4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:57.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.304+0000 7f5fee400700 1 --2- 172.21.15.131:0/539021497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017e4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.304+0000 7f5fee400700 1 -- 172.21.15.131:0/539021497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 msgr2=0x55fb017e3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:57.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.304+0000 7f5fee400700 1 --2- 172.21.15.131:0/539021497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017e3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.304+0000 7f5fee400700 1 -- 172.21.15.131:0/539021497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb015843b0 con 0x55fb017e4ec0 2024-05-10T05:43:57.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.304+0000 7f5fed000700 1 --2- 172.21.15.131:0/539021497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:57.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.304+0000 7f5fee400700 1 --2- 172.21.15.131:0/539021497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4ec0 0x55fb017eb7c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009cf0 tx=0x7f5fe0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07f74c21d99b7e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:57.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.305+0000 7f5fdea00700 1 -- 172.21.15.131:0/539021497 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00045d0 con 0x55fb017e4ec0 2024-05-10T05:43:57.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.305+0000 7f5fdea00700 1 -- 172.21.15.131:0/539021497 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fe000f040 con 0x55fb017e4ec0 2024-05-10T05:43:57.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.305+0000 7f5fdea00700 1 -- 172.21.15.131:0/539021497 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000e750 con 0x55fb017e4ec0 2024-05-10T05:43:57.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.306+0000 7f600828fb80 1 -- 172.21.15.131:0/539021497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4ec0 msgr2=0x55fb017eb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:57.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.306+0000 7f600828fb80 1 --2- 172.21.15.131:0/539021497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4ec0 0x55fb017eb7c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009cf0 tx=0x7f5fe0009920 comp rx=0 tx=0).stop 2024-05-10T05:43:57.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.306+0000 7f600828fb80 1 -- 172.21.15.131:0/539021497 shutdown_connections 2024-05-10T05:43:57.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.306+0000 7f600828fb80 1 --2- 172.21.15.131:0/539021497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4ec0 0x55fb017eb7c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.306+0000 7f600828fb80 1 --2- 172.21.15.131:0/539021497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017e3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.306+0000 7f600828fb80 1 --2- 172.21.15.131:0/539021497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.306+0000 7f600828fb80 1 -- 172.21.15.131:0/539021497 >> 172.21.15.131:0/539021497 conn(0x55fb01745a60 msgr2=0x55fb017ddfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:57.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.307+0000 7f600828fb80 1 -- 172.21.15.131:0/539021497 shutdown_connections 2024-05-10T05:43:57.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.307+0000 7f600828fb80 1 -- 172.21.15.131:0/539021497 wait complete. 2024-05-10T05:43:57.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.307+0000 7f600828fb80 1 -- start start 2024-05-10T05:43:57.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.308+0000 7f600828fb80 1 Processor -- start 2024-05-10T05:43:57.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.308+0000 7f600828fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017ed710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.308+0000 7f600828fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4570 0x55fb017edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.308+0000 7f600828fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 0x55fb017f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.308+0000 7f600828fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb0169f560 con 0x55fb017e4570 2024-05-10T05:43:57.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.308+0000 7f600828fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb0169f200 con 0x55fb017e36a0 2024-05-10T05:43:57.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.308+0000 7f600828fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb015866a0 con 0x55fb017e4ec0 2024-05-10T05:43:57.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.308+0000 7f600828fb80 10 client.0 _ll_register_callbacks cb 0x55fb016a8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5feda00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017ed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5feda00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017ed710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56822/0 (socket says 172.21.15.131:56822) 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5feda00700 1 -- 172.21.15.131:0/236837898 learned_addr learned my addr 172.21.15.131:0/236837898 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5fed000700 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4570 0x55fb017edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5feda00700 1 -- 172.21.15.131:0/236837898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 msgr2=0x55fb017f4bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5fee400700 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 0x55fb017f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5feda00700 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 0x55fb017f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5feda00700 1 -- 172.21.15.131:0/236837898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4570 msgr2=0x55fb017edc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5feda00700 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4570 0x55fb017edc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5feda00700 1 -- 172.21.15.131:0/236837898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe81aefb0 con 0x55fb017e36a0 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.309+0000 7f5fed000700 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4570 0x55fb017edc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.310+0000 7f5feda00700 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017ed710 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800f770 tx=0x7f5fd8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ded2e93659e2f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.310+0000 7f5fd3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:43:57.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.310+0000 7f5fd3400700 1 -- 172.21.15.131:0/236837898 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8026670 con 0x55fb017e36a0 2024-05-10T05:43:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.310+0000 7f5fd3400700 1 -- 172.21.15.131:0/236837898 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5fd80267d0 con 0x55fb017e36a0 2024-05-10T05:43:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.310+0000 7f600828fb80 1 -- 172.21.15.131:0/236837898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb0165c270 con 0x55fb017e36a0 2024-05-10T05:43:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.310+0000 7f5fd3400700 1 -- 172.21.15.131:0/236837898 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fd8026990 con 0x55fb017e36a0 2024-05-10T05:43:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.310+0000 7f600828fb80 10 client.27227 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:57.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.311+0000 7f600828fb80 1 -- 172.21.15.131:0/236837898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f5fe0003a00 con 0x55fb017e36a0 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.311+0000 7f5fd3400700 1 -- 172.21.15.131:0/236837898 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fb0165c270 con 0x55fb017e36a0 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.311+0000 7f5fd3400700 1 -- 172.21.15.131:0/236837898 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5fd80315c0 con 0x55fb017e36a0 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.311+0000 7f600828fb80 10 client.27227 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.311+0000 7f600828fb80 10 client.27227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.311+0000 7f600828fb80 20 client.27227 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.311+0000 7f600828fb80 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fb017f9440 0x55fb017fb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.311+0000 7f600828fb80 4 client.27227 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.311+0000 7f600828fb80 1 -- 172.21.15.131:0/236837898 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb017b2780 con 0x55fb017f9440 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.312+0000 7f5fed000700 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fb017f9440 0x55fb017fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.312+0000 7f5fed000700 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fb017f9440 0x55fb017fb890 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a13bab5323edb191 server_cookie=1f0214648709f5a8 in_seq=0 out_seq=0 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.312+0000 7f5fd3400700 10 client.27227 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:57.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f5fd3400700 1 -- 172.21.15.131:0/236837898 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb017b2780 con 0x55fb017f9440 2024-05-10T05:43:57.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f5fd3400700 10 client.27227 handle_command_reply: tid=0 2024-05-10T05:43:57.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f600828fb80 1 client.27227 shutdown 2024-05-10T05:43:57.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f600828fb80 1 -- 172.21.15.131:0/236837898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 msgr2=0x55fb017ed710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:57.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f600828fb80 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017ed710 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800f770 tx=0x7f5fd8004750 comp rx=0 tx=0).stop 2024-05-10T05:43:57.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f600828fb80 1 -- 172.21.15.131:0/236837898 shutdown_connections 2024-05-10T05:43:57.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f600828fb80 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fb017f9440 0x55fb017fb890 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f600828fb80 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 0x55fb017f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f600828fb80 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017ed710 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f600828fb80 1 --2- 172.21.15.131:0/236837898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4570 0x55fb017edc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.313+0000 7f600828fb80 1 -- 172.21.15.131:0/236837898 >> 172.21.15.131:0/236837898 conn(0x55fb01745a60 msgr2=0x55fb017ddfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:57.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.314+0000 7f5fd3400700 0 client.27227 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:57.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.314+0000 7f600828fb80 1 -- 172.21.15.131:0/236837898 shutdown_connections 2024-05-10T05:43:57.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.314+0000 7f600828fb80 1 -- 172.21.15.131:0/236837898 wait complete. 2024-05-10T05:43:57.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.314+0000 7f600828fb80 20 client.27227 trim_cache size 0 max 16384 2024-05-10T05:43:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.342+0000 7f600828fb80 1 Processor -- start 2024-05-10T05:43:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.342+0000 7f600828fb80 1 -- start start 2024-05-10T05:43:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.343+0000 7f600828fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017da180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.343+0000 7f600828fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4570 0x55fb017f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.343+0000 7f600828fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 0x55fb017e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.343+0000 7f600828fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb017df770 con 0x55fb017e4570 2024-05-10T05:43:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.343+0000 7f600828fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb01800a70 con 0x55fb017e36a0 2024-05-10T05:43:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.343+0000 7f600828fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb017ec730 con 0x55fb017e4ec0 2024-05-10T05:43:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.343+0000 7f5fee400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 0x55fb017e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.344+0000 7f5fee400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 0x55fb017e77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54308/0 (socket says 172.21.15.131:54308) 2024-05-10T05:43:57.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.344+0000 7f5fee400700 1 -- 172.21.15.131:0/1797572628 learned_addr learned my addr 172.21.15.131:0/1797572628 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:57.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.345+0000 7f5fee400700 1 -- 172.21.15.131:0/1797572628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 msgr2=0x55fb017da180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:43:57.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.345+0000 7f5fee400700 1 --2- 172.21.15.131:0/1797572628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017da180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.345+0000 7f5fee400700 1 -- 172.21.15.131:0/1797572628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4570 msgr2=0x55fb017f4b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:43:57.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.345+0000 7f5fee400700 1 --2- 172.21.15.131:0/1797572628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4570 0x55fb017f4b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.345+0000 7f5fee400700 1 -- 172.21.15.131:0/1797572628 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb01530b60 con 0x55fb017e4ec0 2024-05-10T05:43:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.346+0000 7f5fee400700 1 --2- 172.21.15.131:0/1797572628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 0x55fb017e77a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0003cf0 tx=0x7f5fe00060e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4cc508350455728 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.348+0000 7f5fdf400700 1 -- 172.21.15.131:0/1797572628 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00047d0 con 0x55fb017e4ec0 2024-05-10T05:43:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.348+0000 7f5fdf400700 1 -- 172.21.15.131:0/1797572628 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fe0004df0 con 0x55fb017e4ec0 2024-05-10T05:43:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.348+0000 7f5fdf400700 1 -- 172.21.15.131:0/1797572628 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000e4f0 con 0x55fb017e4ec0 2024-05-10T05:43:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.350+0000 7f600828fb80 1 -- 172.21.15.131:0/1797572628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 msgr2=0x55fb017e77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.350+0000 7f600828fb80 1 --2- 172.21.15.131:0/1797572628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 0x55fb017e77a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0003cf0 tx=0x7f5fe00060e0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.350+0000 7f600828fb80 1 -- 172.21.15.131:0/1797572628 shutdown_connections 2024-05-10T05:43:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.350+0000 7f600828fb80 1 --2- 172.21.15.131:0/1797572628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4ec0 0x55fb017e77a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.350+0000 7f600828fb80 1 --2- 172.21.15.131:0/1797572628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e36a0 0x55fb017da180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.350+0000 7f600828fb80 1 --2- 172.21.15.131:0/1797572628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e4570 0x55fb017f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.350+0000 7f600828fb80 1 -- 172.21.15.131:0/1797572628 >> 172.21.15.131:0/1797572628 conn(0x55fb01745a60 msgr2=0x55fb017ddfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.350+0000 7f600828fb80 1 -- 172.21.15.131:0/1797572628 shutdown_connections 2024-05-10T05:43:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.350+0000 7f600828fb80 1 -- 172.21.15.131:0/1797572628 wait complete. 2024-05-10T05:43:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.351+0000 7f600828fb80 1 -- start start 2024-05-10T05:43:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.351+0000 7f600828fb80 1 Processor -- start 2024-05-10T05:43:57.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.352+0000 7f600828fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e36a0 0x55fb017f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.352+0000 7f600828fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.352+0000 7f600828fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e4ec0 0x55fb017f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.353+0000 7f5fed000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.353+0000 7f5fed000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54322/0 (socket says 172.21.15.131:54322) 2024-05-10T05:43:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.353+0000 7f5feda00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e36a0 0x55fb017f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.353+0000 7f600828fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb017f3460 con 0x55fb017e36a0 2024-05-10T05:43:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.353+0000 7f5fed000700 1 -- 172.21.15.131:0/3134628963 learned_addr learned my addr 172.21.15.131:0/3134628963 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.353+0000 7f600828fb80 1 -- 172.21.15.131:0/3134628963 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb017f35d0 con 0x55fb017e4ec0 2024-05-10T05:43:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.353+0000 7f600828fb80 1 -- 172.21.15.131:0/3134628963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb017f3790 con 0x55fb017e4570 2024-05-10T05:43:57.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.353+0000 7f600828fb80 10 client.0 _ll_register_callbacks cb 0x55fb016a8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:57.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.353+0000 7f5fee400700 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e4ec0 0x55fb017f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.353+0000 7f5feda00700 1 -- 172.21.15.131:0/3134628963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 msgr2=0x55fb017f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:57.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.355+0000 7f5feda00700 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.355+0000 7f5feda00700 1 -- 172.21.15.131:0/3134628963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e4ec0 msgr2=0x55fb017f2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:57.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.355+0000 7f5feda00700 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e4ec0 0x55fb017f2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.355+0000 7f5feda00700 1 -- 172.21.15.131:0/3134628963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb015843b0 con 0x55fb017e36a0 2024-05-10T05:43:57.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.355+0000 7f5fee400700 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e4ec0 0x55fb017f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:43:57.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.355+0000 7f5fed000700 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:57.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.355+0000 7f5feda00700 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e36a0 0x55fb017f4620 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8008740 tx=0x7f5fd8031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6411f06c9b38b3c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:57.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.355+0000 7f5fd3400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:43:57.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.356+0000 7f5fd3400700 1 -- 172.21.15.131:0/3134628963 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8026880 con 0x55fb017e36a0 2024-05-10T05:43:57.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.356+0000 7f5fd3400700 1 -- 172.21.15.131:0/3134628963 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5fd80269e0 con 0x55fb017e36a0 2024-05-10T05:43:57.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.356+0000 7f600828fb80 1 -- 172.21.15.131:0/3134628963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fe000f040 con 0x55fb017e36a0 2024-05-10T05:43:57.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.357+0000 7f5fd3400700 1 -- 172.21.15.131:0/3134628963 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fd800fbe0 con 0x55fb017e36a0 2024-05-10T05:43:57.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.357+0000 7f600828fb80 10 client.16881 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:57.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.357+0000 7f600828fb80 1 -- 172.21.15.131:0/3134628963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fb016c9560 con 0x55fb017e36a0 2024-05-10T05:43:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.357+0000 7f5fd3400700 1 -- 172.21.15.131:0/3134628963 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fe000f040 con 0x55fb017e36a0 2024-05-10T05:43:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.357+0000 7f5fd3400700 1 -- 172.21.15.131:0/3134628963 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5fd801d430 con 0x55fb017e36a0 2024-05-10T05:43:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.358+0000 7f600828fb80 10 client.16881 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.358+0000 7f600828fb80 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:43:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.358+0000 7f600828fb80 20 client.16881 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.358+0000 7f600828fb80 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fb01850d00 0x55fb01853120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.358+0000 7f600828fb80 4 client.16881 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:43:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.358+0000 7f600828fb80 1 -- 172.21.15.131:0/3134628963 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fb016fbec0 con 0x55fb01850d00 2024-05-10T05:43:57.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.358+0000 7f5fed000700 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fb01850d00 0x55fb01853120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:57.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.359+0000 7f5fd3400700 10 client.16881 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:57.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.359+0000 7f5fed000700 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fb01850d00 0x55fb01853120 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31dda1d353e0b816 server_cookie=6a740a2b26c0a6de in_seq=0 out_seq=0 2024-05-10T05:43:57.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.359+0000 7f5fd3400700 1 -- 172.21.15.131:0/3134628963 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fb016fbec0 con 0x55fb01850d00 2024-05-10T05:43:57.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.359+0000 7f5fd3400700 10 client.16881 handle_command_reply: tid=0 2024-05-10T05:43:57.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.359+0000 7f600828fb80 1 client.16881 shutdown 2024-05-10T05:43:57.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.359+0000 7f600828fb80 1 -- 172.21.15.131:0/3134628963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e36a0 msgr2=0x55fb017f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:57.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.359+0000 7f600828fb80 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e36a0 0x55fb017f4620 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8008740 tx=0x7f5fd8031760 comp rx=0 tx=0).stop 2024-05-10T05:43:57.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.360+0000 7f600828fb80 1 -- 172.21.15.131:0/3134628963 shutdown_connections 2024-05-10T05:43:57.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.360+0000 7f600828fb80 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fb01850d00 0x55fb01853120 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.360+0000 7f600828fb80 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb017e36a0 0x55fb017f4620 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.360+0000 7f600828fb80 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fb017e4ec0 0x55fb017f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.360+0000 7f600828fb80 1 --2- 172.21.15.131:0/3134628963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb017e4570 0x55fb017f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.360+0000 7f600828fb80 1 -- 172.21.15.131:0/3134628963 >> 172.21.15.131:0/3134628963 conn(0x55fb01745a60 msgr2=0x55fb017e82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:57.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.360+0000 7f5fd3400700 0 client.16881 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:43:57.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.360+0000 7f600828fb80 1 -- 172.21.15.131:0/3134628963 shutdown_connections 2024-05-10T05:43:57.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.360+0000 7f600828fb80 1 -- 172.21.15.131:0/3134628963 wait complete. 2024-05-10T05:43:57.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.360+0000 7f600828fb80 20 client.16881 trim_cache size 0 max 16384 2024-05-10T05:43:57.363 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:43:57.363 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:43:57.363 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:43:57.363 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.361+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5fcc0a1380 msgr2=0x7f5fcc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.361+0000 7f5ff0400700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5fcc0a1380 0x7f5fcc0a3890 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005fd0 tx=0x7f5fd8005e20 comp rx=0 tx=0).stop 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.361+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8108b60 msgr2=0x7f5fe8109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.361+0000 7f5ff0400700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8108b60 0x7f5fe8109cd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400bd30 tx=0x7f5fd400bd60 comp rx=0 tx=0).stop 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.362+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 shutdown_connections 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.362+0000 7f5ff0400700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f5fcc0a1380 0x7f5fcc0a3890 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.362+0000 7f5ff0400700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe8108210 0x7f5fe810b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.362+0000 7f5ff0400700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe8110320 0x7f5fe810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.362+0000 7f5ff0400700 1 --2- 172.21.15.131:0/2817142520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe8108b60 0x7f5fe8109cd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.362+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 >> 172.21.15.131:0/2817142520 conn(0x7f5fe806ea00 msgr2=0x7f5fe810bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:57.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.362+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 shutdown_connections 2024-05-10T05:43:57.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:57.362+0000 7f5ff0400700 1 -- 172.21.15.131:0/2817142520 wait complete. 2024-05-10T05:43:57.406 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:43:57.406 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:43:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.015+0000 7fea1b400700 1 -- 172.21.15.131:0/2017239096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14102580 msgr2=0x7fea14108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.015+0000 7fea1b400700 1 --2- 172.21.15.131:0/2017239096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14102580 0x7fea14108e30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0099c0 tx=0x7fea0c009cd0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.027+0000 7fea1b400700 1 -- 172.21.15.131:0/2017239096 shutdown_connections 2024-05-10T05:43:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.027+0000 7fea1b400700 1 --2- 172.21.15.131:0/2017239096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14102580 0x7fea14108e30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.027+0000 7fea1b400700 1 --2- 172.21.15.131:0/2017239096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea14100ac0 0x7fea14100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.027+0000 7fea1b400700 1 --2- 172.21.15.131:0/2017239096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea14101be0 0x7fea14101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.027+0000 7fea1b400700 1 -- 172.21.15.131:0/2017239096 >> 172.21.15.131:0/2017239096 conn(0x7fea140fc0d0 msgr2=0x7fea140fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.027+0000 7fea1b400700 1 -- 172.21.15.131:0/2017239096 shutdown_connections 2024-05-10T05:43:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.027+0000 7fea1b400700 1 -- 172.21.15.131:0/2017239096 wait complete. 2024-05-10T05:43:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.029+0000 7fea1b400700 1 Processor -- start 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.033+0000 7fea1b400700 1 -- start start 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.034+0000 7fea1b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea14100ac0 0x7fea1406c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.034+0000 7fea1b400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14101be0 0x7fea1406ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.034+0000 7fea1b400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea14102580 0x7fea14071e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.035+0000 7fea19400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea14102580 0x7fea14071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.039+0000 7fea19400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea14102580 0x7fea14071e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34366/0 (socket says 172.21.15.131:34366) 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.039+0000 7fea1b400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea14072440 con 0x7fea14102580 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.039+0000 7fea1b400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea140725b0 con 0x7fea14100ac0 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.039+0000 7fea1b400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea14072770 con 0x7fea14101be0 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.040+0000 7fea0be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14101be0 0x7fea1406ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.040+0000 7fea0be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14101be0 0x7fea1406ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54342/0 (socket says 172.21.15.131:54342) 2024-05-10T05:43:58.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.040+0000 7fea0be00700 1 -- 172.21.15.131:0/3315426205 learned_addr learned my addr 172.21.15.131:0/3315426205 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:58.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.040+0000 7fea18a00700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea14100ac0 0x7fea1406c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:58.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.042+0000 7fea0be00700 1 -- 172.21.15.131:0/3315426205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea14100ac0 msgr2=0x7fea1406c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:58.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.042+0000 7fea0be00700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea14100ac0 0x7fea1406c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.042+0000 7fea0be00700 1 -- 172.21.15.131:0/3315426205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea14102580 msgr2=0x7fea14071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:58.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.042+0000 7fea0be00700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea14102580 0x7fea14071e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.042+0000 7fea0be00700 1 -- 172.21.15.131:0/3315426205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c0095f0 con 0x7fea14101be0 2024-05-10T05:43:58.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.042+0000 7fea19400700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea14102580 0x7fea14071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:43:58.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.043+0000 7fea0be00700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14101be0 0x7fea1406ceb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fea0400ef10 tx=0x7fea0400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:58.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.045+0000 7fea12000700 1 -- 172.21.15.131:0/3315426205 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0400ca70 con 0x7fea14101be0 2024-05-10T05:43:58.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.045+0000 7fea12000700 1 -- 172.21.15.131:0/3315426205 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0400f040 con 0x7fea14101be0 2024-05-10T05:43:58.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.046+0000 7fea12000700 1 -- 172.21.15.131:0/3315426205 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04014480 con 0x7fea14101be0 2024-05-10T05:43:58.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.048+0000 7fea1b400700 1 -- 172.21.15.131:0/3315426205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea140729f0 con 0x7fea14101be0 2024-05-10T05:43:58.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.048+0000 7fea1b400700 1 -- 172.21.15.131:0/3315426205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea14072df0 con 0x7fea14101be0 2024-05-10T05:43:58.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.057+0000 7fea1b400700 1 -- 172.21.15.131:0/3315426205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea140694d0 con 0x7fea14101be0 2024-05-10T05:43:58.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.080+0000 7fea12000700 1 -- 172.21.15.131:0/3315426205 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fea04007500 con 0x7fea14101be0 2024-05-10T05:43:58.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.092+0000 7fea12000700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9f80a1200 0x7fe9f80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:58.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.093+0000 7fea12000700 1 -- 172.21.15.131:0/3315426205 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fea040ccc40 con 0x7fea14101be0 2024-05-10T05:43:58.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.095+0000 7fea12000700 1 -- 172.21.15.131:0/3315426205 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fea040d02a0 con 0x7fea14101be0 2024-05-10T05:43:58.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.098+0000 7fea18a00700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9f80a1200 0x7fe9f80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:58.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.106+0000 7fea18a00700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9f80a1200 0x7fe9f80a3710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fea14101ae0 tx=0x7fe9fc00a380 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:58.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.741+0000 7fea1b400700 1 -- 172.21.15.131:0/3315426205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea1410aa10 con 0x7fea14101be0 2024-05-10T05:43:58.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.743+0000 7fea12000700 1 -- 172.21.15.131:0/3315426205 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fea04096370 con 0x7fea14101be0 2024-05-10T05:43:58.744 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:43:58.744 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:43:58.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.750+0000 7fea1b400700 1 -- 172.21.15.131:0/3315426205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9f80a1200 msgr2=0x7fe9f80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.750+0000 7fea1b400700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9f80a1200 0x7fe9f80a3710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fea14101ae0 tx=0x7fe9fc00a380 comp rx=0 tx=0).stop 2024-05-10T05:43:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.750+0000 7fea1b400700 1 -- 172.21.15.131:0/3315426205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14101be0 msgr2=0x7fea1406ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.750+0000 7fea1b400700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14101be0 0x7fea1406ceb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fea0400ef10 tx=0x7fea0400c3a0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.750+0000 7fea1b400700 1 -- 172.21.15.131:0/3315426205 shutdown_connections 2024-05-10T05:43:58.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.750+0000 7fea1b400700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9f80a1200 0x7fe9f80a3710 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.750+0000 7fea1b400700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea14102580 0x7fea14071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.750+0000 7fea1b400700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea14100ac0 0x7fea1406c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.750+0000 7fea1b400700 1 --2- 172.21.15.131:0/3315426205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea14101be0 0x7fea1406ceb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:58.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.750+0000 7fea1b400700 1 -- 172.21.15.131:0/3315426205 >> 172.21.15.131:0/3315426205 conn(0x7fea140fc0d0 msgr2=0x7fea1410d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:58.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.751+0000 7fea1b400700 1 -- 172.21.15.131:0/3315426205 shutdown_connections 2024-05-10T05:43:58.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:58.751+0000 7fea1b400700 1 -- 172.21.15.131:0/3315426205 wait complete. 2024-05-10T05:43:58.759 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:43:58.809 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:43:58.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:58 smithi142 ceph-mon[77667]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:43:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:58 smithi131 ceph-mon[37264]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:43:59.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:58 smithi157 ceph-mon[78427]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 245 MiB/s wr, 75 op/s 2024-05-10T05:43:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.585+0000 7f4405e00700 1 -- 172.21.15.131:0/2967458178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f440006b160 msgr2=0x7f440006b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.585+0000 7f4405e00700 1 --2- 172.21.15.131:0/2967458178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f440006b160 0x7f440006b570 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009990 tx=0x7f43f0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.587+0000 7f4405e00700 1 -- 172.21.15.131:0/2967458178 shutdown_connections 2024-05-10T05:43:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.587+0000 7f4405e00700 1 --2- 172.21.15.131:0/2967458178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f440006cc20 0x7f4400071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.587+0000 7f4405e00700 1 --2- 172.21.15.131:0/2967458178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f440006b160 0x7f440006b570 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.587+0000 7f4405e00700 1 --2- 172.21.15.131:0/2967458178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f440006c2d0 0x7f440006c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.587+0000 7f4405e00700 1 -- 172.21.15.131:0/2967458178 >> 172.21.15.131:0/2967458178 conn(0x7f44000fbf20 msgr2=0x7f44000fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.593+0000 7f4405e00700 1 -- 172.21.15.131:0/2967458178 shutdown_connections 2024-05-10T05:43:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.593+0000 7f4405e00700 1 -- 172.21.15.131:0/2967458178 wait complete. 2024-05-10T05:43:59.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.599+0000 7f4405e00700 1 Processor -- start 2024-05-10T05:43:59.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.599+0000 7f4405e00700 1 -- start start 2024-05-10T05:43:59.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.600+0000 7f4405e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f440006b160 0x7f440018e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.600+0000 7f4405e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f440006c2d0 0x7f440018eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.600+0000 7f4405e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f440006cc20 0x7f4400195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.601+0000 7f43fea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f440006b160 0x7f440018e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.601+0000 7f43fea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f440006b160 0x7f440018e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34392/0 (socket says 172.21.15.131:34392) 2024-05-10T05:43:59.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.601+0000 7f43fe000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f440006c2d0 0x7f440018eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.601+0000 7f43fe000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f440006c2d0 0x7f440018eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56878/0 (socket says 172.21.15.131:56878) 2024-05-10T05:43:59.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.603+0000 7f43fea00700 1 -- 172.21.15.131:0/1284820380 learned_addr learned my addr 172.21.15.131:0/1284820380 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:59.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.603+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4400068450 con 0x7f440006b160 2024-05-10T05:43:59.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.603+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000685c0 con 0x7f440006c2d0 2024-05-10T05:43:59.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.603+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4400068780 con 0x7f440006cc20 2024-05-10T05:43:59.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.605+0000 7f43ff400700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f440006cc20 0x7f4400195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.606+0000 7f43fe000700 1 -- 172.21.15.131:0/1284820380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f440006cc20 msgr2=0x7f4400195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:59.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.606+0000 7f43fe000700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f440006cc20 0x7f4400195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.607+0000 7f43fe000700 1 -- 172.21.15.131:0/1284820380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f440006b160 msgr2=0x7f440018e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:59.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.607+0000 7f43fe000700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f440006b160 0x7f440018e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.607+0000 7f43fe000700 1 -- 172.21.15.131:0/1284820380 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f00095f0 con 0x7f440006c2d0 2024-05-10T05:43:59.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.607+0000 7f43fea00700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f440006b160 0x7f440018e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:43:59.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.608+0000 7f43fe000700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f440006c2d0 0x7f440018eca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00b450 tx=0x7f43ec00b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:59.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.608+0000 7f4404a00700 1 -- 172.21.15.131:0/1284820380 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec011830 con 0x7f440006c2d0 2024-05-10T05:43:59.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.608+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4400068a60 con 0x7f440006c2d0 2024-05-10T05:43:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.609+0000 7f4404a00700 1 -- 172.21.15.131:0/1284820380 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43ec00be60 con 0x7f440006c2d0 2024-05-10T05:43:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.610+0000 7f4404a00700 1 -- 172.21.15.131:0/1284820380 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec010db0 con 0x7f440006c2d0 2024-05-10T05:43:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.610+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4400068f70 con 0x7f440006c2d0 2024-05-10T05:43:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.615+0000 7f4404a00700 1 -- 172.21.15.131:0/1284820380 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f43ec011990 con 0x7f440006c2d0 2024-05-10T05:43:59.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.622+0000 7f4404a00700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43e40c30d0 0x7f43e40c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.622+0000 7f4404a00700 1 -- 172.21.15.131:0/1284820380 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f43ec0cf170 con 0x7f440006c2d0 2024-05-10T05:43:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.624+0000 7f43fea00700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43e40c30d0 0x7f43e40c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.625+0000 7f441dd5bb80 1 Processor -- start 2024-05-10T05:43:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.626+0000 7f441dd5bb80 1 -- start start 2024-05-10T05:43:59.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.626+0000 7f43fea00700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43e40c30d0 0x7f43e40c55e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f43f00098e0 tx=0x7f43f0003860 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.627+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x555860406170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.627+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x5558604073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.630+0000 7f43fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x555860406170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.630+0000 7f43fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x555860406170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54374/0 (socket says 172.21.15.131:54374) 2024-05-10T05:43:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.630+0000 7f43fea00700 1 -- 172.21.15.131:0/387320474 learned_addr learned my addr 172.21.15.131:0/387320474 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.630+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/387320474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x55586040e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.631+0000 7f43ff400700 1 --2- 172.21.15.131:0/387320474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x55586040e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.632+0000 7f43fe000700 1 --2- 172.21.15.131:0/387320474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x5558604073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.633+0000 7f441dd5bb80 1 -- 172.21.15.131:0/387320474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555860324150 con 0x555860406fb0 2024-05-10T05:43:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.633+0000 7f441dd5bb80 1 -- 172.21.15.131:0/387320474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555860321b10 con 0x555860407900 2024-05-10T05:43:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.633+0000 7f441dd5bb80 1 -- 172.21.15.131:0/387320474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55586031d400 con 0x555860405d40 2024-05-10T05:43:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.635+0000 7f43fea00700 1 -- 172.21.15.131:0/387320474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 msgr2=0x55586040e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.635+0000 7f43fea00700 1 --2- 172.21.15.131:0/387320474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x55586040e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.635+0000 7f43fea00700 1 -- 172.21.15.131:0/387320474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 msgr2=0x5558604073c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.635+0000 7f43fea00700 1 --2- 172.21.15.131:0/387320474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x5558604073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.636+0000 7f43fea00700 1 -- 172.21.15.131:0/387320474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55586019e3b0 con 0x555860405d40 2024-05-10T05:43:59.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.639+0000 7f43fea00700 1 --2- 172.21.15.131:0/387320474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x555860406170 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f43f0003a40 tx=0x7f43f001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76918810beb3c9c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.639+0000 7f43ebe00700 1 -- 172.21.15.131:0/387320474 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f001d070 con 0x555860405d40 2024-05-10T05:43:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.640+0000 7f43ebe00700 1 -- 172.21.15.131:0/387320474 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f0003e10 con 0x555860405d40 2024-05-10T05:43:59.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.641+0000 7f441dd5bb80 1 -- 172.21.15.131:0/387320474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 msgr2=0x555860406170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:59.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.641+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/387320474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x555860406170 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f43f0003a40 tx=0x7f43f001e000 comp rx=0 tx=0).stop 2024-05-10T05:43:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.645+0000 7f441dd5bb80 1 -- 172.21.15.131:0/387320474 shutdown_connections 2024-05-10T05:43:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.645+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/387320474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x55586040e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.645+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/387320474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x555860406170 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.645+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/387320474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x5558604073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.646+0000 7f441dd5bb80 1 -- 172.21.15.131:0/387320474 >> 172.21.15.131:0/387320474 conn(0x55586035fa60 msgr2=0x5558603f8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:59.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.648+0000 7f441dd5bb80 1 -- 172.21.15.131:0/387320474 shutdown_connections 2024-05-10T05:43:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.648+0000 7f441dd5bb80 1 -- 172.21.15.131:0/387320474 wait complete. 2024-05-10T05:43:59.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.649+0000 7f441dd5bb80 1 -- start start 2024-05-10T05:43:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.649+0000 7f441dd5bb80 1 Processor -- start 2024-05-10T05:43:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.650+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.650+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860410620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.650+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x555860417580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.650+0000 7f441dd5bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558602b9560 con 0x555860406fb0 2024-05-10T05:43:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.650+0000 7f441dd5bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558602b9200 con 0x555860407900 2024-05-10T05:43:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.650+0000 7f441dd5bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558601a06a0 con 0x555860405d40 2024-05-10T05:43:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.650+0000 7f441dd5bb80 10 client.0 _ll_register_callbacks cb 0x5558602c2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.651+0000 7f43fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.651+0000 7f43fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54384/0 (socket says 172.21.15.131:54384) 2024-05-10T05:43:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.651+0000 7f43fea00700 1 -- 172.21.15.131:0/1656732909 learned_addr learned my addr 172.21.15.131:0/1656732909 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.651+0000 7f43ff400700 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x555860417580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.652+0000 7f43fea00700 1 -- 172.21.15.131:0/1656732909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 msgr2=0x555860417580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.652+0000 7f43fea00700 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x555860417580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.652+0000 7f43fea00700 1 -- 172.21.15.131:0/1656732909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 msgr2=0x555860410620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.652+0000 7f43fea00700 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860410620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.652+0000 7f43fea00700 1 -- 172.21.15.131:0/1656732909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4400072020 con 0x555860405d40 2024-05-10T05:43:59.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.652+0000 7f43fea00700 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604100e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43f0003a10 tx=0x7f43f000f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d91d23deabcaeaa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.652+0000 7f43e8c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:43:59.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.653+0000 7f43e8c00700 1 -- 172.21.15.131:0/1656732909 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0029820 con 0x555860405d40 2024-05-10T05:43:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.654+0000 7f43e8c00700 1 -- 172.21.15.131:0/1656732909 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f43f0029980 con 0x555860405d40 2024-05-10T05:43:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.654+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1656732909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555860276270 con 0x555860405d40 2024-05-10T05:43:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.671+0000 7f43e8c00700 1 -- 172.21.15.131:0/1656732909 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43f0029b40 con 0x555860405d40 2024-05-10T05:43:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.672+0000 7f441dd5bb80 10 client.27265 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:59.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.673+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1656732909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55586014ab60 con 0x555860405d40 2024-05-10T05:43:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.673+0000 7f43e8c00700 1 -- 172.21.15.131:0/1656732909 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555860276270 con 0x555860405d40 2024-05-10T05:43:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.673+0000 7f43e8c00700 1 -- 172.21.15.131:0/1656732909 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f43f00173f0 con 0x555860405d40 2024-05-10T05:43:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.673+0000 7f441dd5bb80 10 client.27265 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.673+0000 7f441dd5bb80 10 client.27265 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:43:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.673+0000 7f441dd5bb80 20 client.27265 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.673+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55586041b920 0x55586041dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.674+0000 7f441dd5bb80 4 client.27265 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:43:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.674+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1656732909 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558603cc780 con 0x55586041b920 2024-05-10T05:43:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.674+0000 7f43fe000700 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55586041b920 0x55586041dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.675+0000 7f43e8c00700 10 client.27265 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:43:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.675+0000 7f43fe000700 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55586041b920 0x55586041dd70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a4a218508951c10 server_cookie=87c5565bba3e1fc2 in_seq=0 out_seq=0 2024-05-10T05:43:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.675+0000 7f43e8c00700 1 -- 172.21.15.131:0/1656732909 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558603cc780 con 0x55586041b920 2024-05-10T05:43:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.675+0000 7f43e8c00700 10 client.27265 handle_command_reply: tid=0 2024-05-10T05:43:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.675+0000 7f441dd5bb80 1 client.27265 shutdown 2024-05-10T05:43:59.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.676+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1656732909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 msgr2=0x5558604100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:59.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.676+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604100e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43f0003a10 tx=0x7f43f000f710 comp rx=0 tx=0).stop 2024-05-10T05:43:59.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.677+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1656732909 shutdown_connections 2024-05-10T05:43:59.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.677+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55586041b920 0x55586041dd70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.677+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x555860417580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.677+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604100e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.677+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1656732909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860410620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.677+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1656732909 >> 172.21.15.131:0/1656732909 conn(0x55586035fa60 msgr2=0x555860404870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.677+0000 7f43e8c00700 0 client.27265 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:43:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.677+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1656732909 shutdown_connections 2024-05-10T05:43:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.677+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1656732909 wait complete. 2024-05-10T05:43:59.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.678+0000 7f441dd5bb80 20 client.27265 trim_cache size 0 max 16384 2024-05-10T05:43:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.707+0000 7f441dd5bb80 1 Processor -- start 2024-05-10T05:43:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.708+0000 7f441dd5bb80 1 -- start start 2024-05-10T05:43:59.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.708+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x55586041e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.708+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860417540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.708+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x5558603f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.709+0000 7f441dd5bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55586040a860 con 0x555860406fb0 2024-05-10T05:43:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.709+0000 7f441dd5bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558603efd60 con 0x555860407900 2024-05-10T05:43:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.709+0000 7f441dd5bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558604065d0 con 0x555860405d40 2024-05-10T05:43:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.709+0000 7f43fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x55586041e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.709+0000 7f43fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x55586041e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54396/0 (socket says 172.21.15.131:54396) 2024-05-10T05:43:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.709+0000 7f43fea00700 1 -- 172.21.15.131:0/1127406396 learned_addr learned my addr 172.21.15.131:0/1127406396 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.709+0000 7f43ff400700 1 --2- 172.21.15.131:0/1127406396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x5558603f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.710+0000 7f43fe000700 1 --2- 172.21.15.131:0/1127406396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860417540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.710+0000 7f43fea00700 1 -- 172.21.15.131:0/1127406396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 msgr2=0x5558603f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.710+0000 7f43fea00700 1 --2- 172.21.15.131:0/1127406396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x5558603f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.710+0000 7f43fea00700 1 -- 172.21.15.131:0/1127406396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 msgr2=0x555860417540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.710+0000 7f43fea00700 1 --2- 172.21.15.131:0/1127406396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860417540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.710+0000 7f43fea00700 1 -- 172.21.15.131:0/1127406396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558602e3560 con 0x555860405d40 2024-05-10T05:43:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.711+0000 7f43fea00700 1 --2- 172.21.15.131:0/1127406396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x55586041e040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f43f0017fd0 tx=0x7f43f0003720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f784b363f42bd6ef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.711+0000 7f43f5200700 1 -- 172.21.15.131:0/1127406396 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0017a20 con 0x555860405d40 2024-05-10T05:43:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.711+0000 7f43f5200700 1 -- 172.21.15.131:0/1127406396 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f003c810 con 0x555860405d40 2024-05-10T05:43:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.711+0000 7f43f5200700 1 -- 172.21.15.131:0/1127406396 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f003cc30 con 0x555860405d40 2024-05-10T05:43:59.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.716+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1127406396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 msgr2=0x55586041e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:59.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.716+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1127406396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x55586041e040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f43f0017fd0 tx=0x7f43f0003720 comp rx=0 tx=0).stop 2024-05-10T05:43:59.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.716+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1127406396 shutdown_connections 2024-05-10T05:43:59.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.716+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1127406396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x5558603f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.716+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1127406396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x55586041e040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.716+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1127406396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860417540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.716+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1127406396 >> 172.21.15.131:0/1127406396 conn(0x55586035fa60 msgr2=0x55586041d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:59.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.716+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1127406396 shutdown_connections 2024-05-10T05:43:59.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.716+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1127406396 wait complete. 2024-05-10T05:43:59.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.717+0000 7f441dd5bb80 1 -- start start 2024-05-10T05:43:59.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.717+0000 7f441dd5bb80 1 Processor -- start 2024-05-10T05:43:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.718+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.718+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860413e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f43fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f43fea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604172d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54410/0 (socket says 172.21.15.131:54410) 2024-05-10T05:43:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f441dd5bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x555860414390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f43fe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860413e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f43fe000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860413e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34434/0 (socket says 172.21.15.131:34434) 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f441dd5bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558604148d0 con 0x555860406fb0 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f441dd5bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555860414a40 con 0x555860407900 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f441dd5bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55586046fd80 con 0x555860405d40 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f441dd5bb80 10 client.0 _ll_register_callbacks cb 0x5558602c2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f43fea00700 1 -- 172.21.15.131:0/1243368872 learned_addr learned my addr 172.21.15.131:0/1243368872 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f43ff400700 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x555860414390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f43fea00700 1 -- 172.21.15.131:0/1243368872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 msgr2=0x555860414390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f43fea00700 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x555860414390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.719+0000 7f43fea00700 1 -- 172.21.15.131:0/1243368872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 msgr2=0x555860413e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.720+0000 7f43fea00700 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860413e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.720+0000 7f43fea00700 1 -- 172.21.15.131:0/1243368872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55586014ab60 con 0x555860405d40 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.720+0000 7f43fea00700 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604172d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f43f00178a0 tx=0x7f43f0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d51f49ee1de393e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.720+0000 7f43e8c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:43:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.720+0000 7f43e8c00700 1 -- 172.21.15.131:0/1243368872 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0017a20 con 0x555860405d40 2024-05-10T05:43:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.720+0000 7f43e8c00700 1 -- 172.21.15.131:0/1243368872 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f43f003c980 con 0x555860405d40 2024-05-10T05:43:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.720+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1243368872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43f0003e10 con 0x555860405d40 2024-05-10T05:43:59.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f43e8c00700 1 -- 172.21.15.131:0/1243368872 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43f003cb40 con 0x555860405d40 2024-05-10T05:43:59.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f441dd5bb80 10 client.27271 fetch_fsmap learned FSMap version 29 2024-05-10T05:43:59.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1243368872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5558601a2cd0 con 0x555860405d40 2024-05-10T05:43:59.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f43e8c00700 1 -- 172.21.15.131:0/1243368872 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f0003e10 con 0x555860405d40 2024-05-10T05:43:59.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f43e8c00700 1 -- 172.21.15.131:0/1243368872 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f43f003e450 con 0x555860405d40 2024-05-10T05:43:59.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f441dd5bb80 10 client.27271 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f441dd5bb80 10 client.27271 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f441dd5bb80 20 client.27271 populate_metadata read hostname 'smithi131' 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x555860471270 0x555860473690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f441dd5bb80 4 client.27271 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.721+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1243368872 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555860315ec0 con 0x555860471270 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.722+0000 7f43fe000700 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x555860471270 0x555860473690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.722+0000 7f43e8c00700 10 client.27271 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.722+0000 7f43fe000700 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x555860471270 0x555860473690 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=13587f096096eabe server_cookie=8f78574d8d78439c in_seq=0 out_seq=0 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f43e8c00700 1 -- 172.21.15.131:0/1243368872 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555860315ec0 con 0x555860471270 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f43e8c00700 10 client.27271 handle_command_reply: tid=0 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f441dd5bb80 1 client.27271 shutdown 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1243368872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 msgr2=0x5558604172d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604172d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f43f00178a0 tx=0x7f43f0009310 comp rx=0 tx=0).stop 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1243368872 shutdown_connections 2024-05-10T05:43:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x555860471270 0x555860473690 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f43e8c00700 0 client.27271 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:43:59.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555860407900 0x555860414390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555860405d40 0x5558604172d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f441dd5bb80 1 --2- 172.21.15.131:0/1243368872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555860406fb0 0x555860413e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.723+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1243368872 >> 172.21.15.131:0/1243368872 conn(0x55586035fa60 msgr2=0x5558603f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:59.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.724+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1243368872 shutdown_connections 2024-05-10T05:43:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.724+0000 7f441dd5bb80 1 -- 172.21.15.131:0/1243368872 wait complete. 2024-05-10T05:43:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.724+0000 7f441dd5bb80 20 client.27271 trim_cache size 0 max 16384 2024-05-10T05:43:59.727 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:43:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.726+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43e40c30d0 msgr2=0x7f43e40c55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.727+0000 7f4405e00700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43e40c30d0 0x7f43e40c55e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f43f00098e0 tx=0x7f43f0003860 comp rx=0 tx=0).stop 2024-05-10T05:43:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.727+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f440006c2d0 msgr2=0x7f440018eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:43:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.727+0000 7f4405e00700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f440006c2d0 0x7f440018eca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00b450 tx=0x7f43ec00b760 comp rx=0 tx=0).stop 2024-05-10T05:43:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.727+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 shutdown_connections 2024-05-10T05:43:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.727+0000 7f4405e00700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43e40c30d0 0x7f43e40c55e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.727+0000 7f4405e00700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f440006b160 0x7f440018e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.727+0000 7f4405e00700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f440006cc20 0x7f4400195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.727+0000 7f4405e00700 1 --2- 172.21.15.131:0/1284820380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f440006c2d0 0x7f440018eca0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:43:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.727+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 >> 172.21.15.131:0/1284820380 conn(0x7f44000fbf20 msgr2=0x7f440006f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:43:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.727+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 shutdown_connections 2024-05-10T05:43:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:43:59.728+0000 7f4405e00700 1 -- 172.21.15.131:0/1284820380 wait complete. 2024-05-10T05:43:59.772 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:43:59.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:43:59 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3315426205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:44:00.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:43:59 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3315426205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:44:00.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:43:59 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3315426205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:44:00.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.431+0000 7f8ac0600700 1 -- 172.21.15.131:0/341873566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab8108bb0 msgr2=0x7f8ab8079470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:00.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.431+0000 7f8ac0600700 1 --2- 172.21.15.131:0/341873566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab8108bb0 0x7f8ab8079470 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009990 tx=0x7f8aac009ca0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.433+0000 7f8ac0600700 1 -- 172.21.15.131:0/341873566 shutdown_connections 2024-05-10T05:44:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.433+0000 7f8ac0600700 1 --2- 172.21.15.131:0/341873566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab8108bb0 0x7f8ab8079470 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.433+0000 7f8ac0600700 1 --2- 172.21.15.131:0/341873566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab8109710 0x7f8ab8109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.433+0000 7f8ac0600700 1 --2- 172.21.15.131:0/341873566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab8108210 0x7f8ab8108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.433+0000 7f8ac0600700 1 -- 172.21.15.131:0/341873566 >> 172.21.15.131:0/341873566 conn(0x7f8ab806ea00 msgr2=0x7f8ab8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.433+0000 7f8ac0600700 1 -- 172.21.15.131:0/341873566 shutdown_connections 2024-05-10T05:44:00.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.434+0000 7f8ac0600700 1 -- 172.21.15.131:0/341873566 wait complete. 2024-05-10T05:44:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.438+0000 7f8ac0600700 1 Processor -- start 2024-05-10T05:44:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.438+0000 7f8ac0600700 1 -- start start 2024-05-10T05:44:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.439+0000 7f8ac0600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab8108210 0x7f8ab8083da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.439+0000 7f8ac0600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab8108bb0 0x7f8ab8082440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.439+0000 7f8ac0600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab8109710 0x7f8ab8082a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.439+0000 7f8ac0600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8083130 con 0x7f8ab8108bb0 2024-05-10T05:44:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.439+0000 7f8ac0600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80832a0 con 0x7f8ab8109710 2024-05-10T05:44:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.439+0000 7f8ac0600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab81bfac0 con 0x7f8ab8108210 2024-05-10T05:44:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.439+0000 7f8abe600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab8109710 0x7f8ab8082a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.439+0000 7f8abe600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab8109710 0x7f8ab8082a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56916/0 (socket says 172.21.15.131:56916) 2024-05-10T05:44:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.439+0000 7f8abe600700 1 -- 172.21.15.131:0/878149976 learned_addr learned my addr 172.21.15.131:0/878149976 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:00.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.440+0000 7f8abe600700 1 -- 172.21.15.131:0/878149976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab8108210 msgr2=0x7f8ab8083da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:44:00.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.440+0000 7f8abe600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab8108210 0x7f8ab8083da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.440+0000 7f8abe600700 1 -- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab8108bb0 msgr2=0x7f8ab8082440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T05:44:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.440+0000 7f8abd200700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab8108bb0 0x7f8ab8082440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.441+0000 7f8abe600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab8108bb0 0x7f8ab8082440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.441+0000 7f8abe600700 1 -- 172.21.15.131:0/878149976 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0095f0 con 0x7f8ab8109710 2024-05-10T05:44:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.441+0000 7f8abd200700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab8108bb0 0x7f8ab8082440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.441+0000 7f8abe600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab8109710 0x7f8ab8082a30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8aac003e00 tx=0x7f8aac003ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.442+0000 7f8ab6000700 1 -- 172.21.15.131:0/878149976 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac01d070 con 0x7f8ab8109710 2024-05-10T05:44:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.442+0000 7f8ac0600700 1 -- 172.21.15.131:0/878149976 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab81bfd40 con 0x7f8ab8109710 2024-05-10T05:44:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.442+0000 7f8ac0600700 1 -- 172.21.15.131:0/878149976 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab81c0220 con 0x7f8ab8109710 2024-05-10T05:44:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.442+0000 7f8ab6000700 1 -- 172.21.15.131:0/878149976 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aac00f460 con 0x7f8ab8109710 2024-05-10T05:44:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.443+0000 7f8ab6000700 1 -- 172.21.15.131:0/878149976 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac01d070 con 0x7f8ab8109710 2024-05-10T05:44:00.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.455+0000 7f8ab6000700 1 -- 172.21.15.131:0/878149976 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8aac00f5d0 con 0x7f8ab8109710 2024-05-10T05:44:00.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.457+0000 7f8ad84f8b80 1 Processor -- start 2024-05-10T05:44:00.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.457+0000 7f8ad84f8b80 1 -- start start 2024-05-10T05:44:00.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.459+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a4542daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.459+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 0x563a4542e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.460+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 0x563a454357c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.460+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a45354150 con 0x563a4542e570 2024-05-10T05:44:00.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.460+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a45351b10 con 0x563a4542eec0 2024-05-10T05:44:00.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.460+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4534d400 con 0x563a4542d6a0 2024-05-10T05:44:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.461+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 0x563a4542e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.462+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 0x563a4542e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34468/0 (socket says 172.21.15.131:34468) 2024-05-10T05:44:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.462+0000 7f8abdc00700 1 -- 172.21.15.131:0/2030654926 learned_addr learned my addr 172.21.15.131:0/2030654926 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.462+0000 7f8abd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a4542daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.462+0000 7f8abd200700 1 --2- 172.21.15.131:0/2030654926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 0x563a454357c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.463+0000 7f8ab6000700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8aa00a1350 0x7f8aa00a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.465+0000 7f8ab6000700 1 -- 172.21.15.131:0/878149976 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8aac0cd070 con 0x7f8ab8109710 2024-05-10T05:44:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.465+0000 7f8abdc00700 1 -- 172.21.15.131:0/2030654926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 msgr2=0x563a4542daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.465+0000 7f8abdc00700 1 --2- 172.21.15.131:0/2030654926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a4542daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.465+0000 7f8abdc00700 1 -- 172.21.15.131:0/2030654926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 msgr2=0x563a454357c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.465+0000 7f8abdc00700 1 --2- 172.21.15.131:0/2030654926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 0x563a454357c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.465+0000 7f8abdc00700 1 -- 172.21.15.131:0/2030654926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a451ce3b0 con 0x563a4542e570 2024-05-10T05:44:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.465+0000 7f8abe600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8aa00a1350 0x7f8aa00a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.466+0000 7f8abd200700 1 --2- 172.21.15.131:0/2030654926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 0x563a454357c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.466+0000 7f8abdc00700 1 --2- 172.21.15.131:0/2030654926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 0x563a4542e980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009740 tx=0x7f8aa4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0da82a8f00a090b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.467+0000 7f8aaaa00700 1 -- 172.21.15.131:0/2030654926 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4011830 con 0x563a4542e570 2024-05-10T05:44:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.467+0000 7f8aaaa00700 1 -- 172.21.15.131:0/2030654926 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa4011e50 con 0x563a4542e570 2024-05-10T05:44:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.467+0000 7f8aaaa00700 1 -- 172.21.15.131:0/2030654926 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400f2e0 con 0x563a4542e570 2024-05-10T05:44:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.468+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2030654926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 msgr2=0x563a4542e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.468+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/2030654926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 0x563a4542e980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009740 tx=0x7f8aa4009a50 comp rx=0 tx=0).stop 2024-05-10T05:44:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.469+0000 7f8abe600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8aa00a1350 0x7f8aa00a3860 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0a2440 tx=0x7f8aac009ca0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.469+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2030654926 shutdown_connections 2024-05-10T05:44:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.469+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/2030654926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 0x563a454357c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.469+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/2030654926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a4542daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.469+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/2030654926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 0x563a4542e980 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.469+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2030654926 >> 172.21.15.131:0/2030654926 conn(0x563a4538fa60 msgr2=0x563a45427fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:00.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.469+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2030654926 shutdown_connections 2024-05-10T05:44:00.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.470+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2030654926 wait complete. 2024-05-10T05:44:00.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.470+0000 7f8ad84f8b80 1 -- start start 2024-05-10T05:44:00.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.471+0000 7f8ad84f8b80 1 Processor -- start 2024-05-10T05:44:00.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.472+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a45437750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.472+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 0x563a45437c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.473+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 0x563a4543ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.473+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a452e9560 con 0x563a4542e570 2024-05-10T05:44:00.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.473+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a452e9200 con 0x563a4542eec0 2024-05-10T05:44:00.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.473+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a451d06a0 con 0x563a4542d6a0 2024-05-10T05:44:00.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.473+0000 7f8ad84f8b80 10 client.0 _ll_register_callbacks cb 0x563a452f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.473+0000 7f8abd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a45437750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.473+0000 7f8abd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a45437750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54422/0 (socket says 172.21.15.131:54422) 2024-05-10T05:44:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.473+0000 7f8abd200700 1 -- 172.21.15.131:0/2862729415 learned_addr learned my addr 172.21.15.131:0/2862729415 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.474+0000 7f8abd200700 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 0x563a4543ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.474+0000 7f8abd200700 1 -- 172.21.15.131:0/2862729415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 msgr2=0x563a4543ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:00.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.474+0000 7f8abd200700 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 0x563a4543ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.474+0000 7f8abd200700 1 -- 172.21.15.131:0/2862729415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 msgr2=0x563a45437c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:00.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.474+0000 7f8abd200700 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 0x563a45437c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.474+0000 7f8abd200700 1 -- 172.21.15.131:0/2862729415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa4009370 con 0x563a4542d6a0 2024-05-10T05:44:00.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.475+0000 7f8a9f400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:00.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.475+0000 7f8abd200700 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a45437750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0017a90 tx=0x7f8ab0004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf26179fbf1e92c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.476+0000 7f8a9f400700 1 -- 172.21.15.131:0/2862729415 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0017310 con 0x563a4542d6a0 2024-05-10T05:44:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.476+0000 7f8a9f400700 1 -- 172.21.15.131:0/2862729415 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8ab001e030 con 0x563a4542d6a0 2024-05-10T05:44:00.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.476+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2862729415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a452a6270 con 0x563a4542d6a0 2024-05-10T05:44:00.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.477+0000 7f8a9f400700 1 -- 172.21.15.131:0/2862729415 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ab00135e0 con 0x563a4542d6a0 2024-05-10T05:44:00.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.477+0000 7f8ad84f8b80 10 client.27277 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:00.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.477+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2862729415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563a4517ab60 con 0x563a4542d6a0 2024-05-10T05:44:00.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.477+0000 7f8a9f400700 1 -- 172.21.15.131:0/2862729415 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ab0027030 con 0x563a4542d6a0 2024-05-10T05:44:00.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.478+0000 7f8a9f400700 1 -- 172.21.15.131:0/2862729415 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8ab0015570 con 0x563a4542d6a0 2024-05-10T05:44:00.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.478+0000 7f8ad84f8b80 10 client.27277 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:00.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.478+0000 7f8ad84f8b80 10 client.27277 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:44:00.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.478+0000 7f8ad84f8b80 20 client.27277 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:00.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.478+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x563a45443580 0x563a454459d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.478+0000 7f8ad84f8b80 4 client.27277 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:00.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.478+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2862729415 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a453fc780 con 0x563a45443580 2024-05-10T05:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.484+0000 7f8abdc00700 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x563a45443580 0x563a454459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.490+0000 7f8abdc00700 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x563a45443580 0x563a454459d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c10393169f8d26 server_cookie=f3bc0fe7b085658 in_seq=0 out_seq=0 2024-05-10T05:44:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.490+0000 7f8a9f400700 10 client.27277 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:44:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.491+0000 7f8a9f400700 1 -- 172.21.15.131:0/2862729415 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a453fc780 con 0x563a45443580 2024-05-10T05:44:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.491+0000 7f8a9f400700 10 client.27277 handle_command_reply: tid=0 2024-05-10T05:44:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.491+0000 7f8ad84f8b80 1 client.27277 shutdown 2024-05-10T05:44:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.491+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2862729415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 msgr2=0x563a45437750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.491+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a45437750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0017a90 tx=0x7f8ab0004b80 comp rx=0 tx=0).stop 2024-05-10T05:44:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.491+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2862729415 shutdown_connections 2024-05-10T05:44:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.491+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x563a45443580 0x563a454459d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.492+0000 7f8a9f400700 0 client.27277 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:44:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.492+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542eec0 0x563a4543ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.492+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a45437750 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.492+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/2862729415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542e570 0x563a45437c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.492+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2862729415 >> 172.21.15.131:0/2862729415 conn(0x563a4538fa60 msgr2=0x563a45427fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.492+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2862729415 shutdown_connections 2024-05-10T05:44:00.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.492+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/2862729415 wait complete. 2024-05-10T05:44:00.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.498+0000 7f8ad84f8b80 20 client.27277 trim_cache size 0 max 16384 2024-05-10T05:44:00.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.536+0000 7f8ad84f8b80 1 Processor -- start 2024-05-10T05:44:00.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.536+0000 7f8ad84f8b80 1 -- start start 2024-05-10T05:44:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.537+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542d6a0 0x563a454317a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.537+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.537+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542eec0 0x563a4543d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.537+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a45422850 con 0x563a4542d6a0 2024-05-10T05:44:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.537+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4543f650 con 0x563a4542e570 2024-05-10T05:44:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.537+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a45440900 con 0x563a4542eec0 2024-05-10T05:44:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.538+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.538+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56950/0 (socket says 172.21.15.131:56950) 2024-05-10T05:44:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.538+0000 7f8abdc00700 1 -- 172.21.15.131:0/3530880063 learned_addr learned my addr 172.21.15.131:0/3530880063 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.538+0000 7f8abd200700 1 --2- 172.21.15.131:0/3530880063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542d6a0 0x563a454317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.538+0000 7f8abdc00700 1 -- 172.21.15.131:0/3530880063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542eec0 msgr2=0x563a4543d0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:44:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.538+0000 7f8abdc00700 1 --2- 172.21.15.131:0/3530880063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542eec0 0x563a4543d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.538+0000 7f8abdc00700 1 -- 172.21.15.131:0/3530880063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542d6a0 msgr2=0x563a454317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.539+0000 7f8abdc00700 1 --2- 172.21.15.131:0/3530880063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542d6a0 0x563a454317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.539+0000 7f8abdc00700 1 -- 172.21.15.131:0/3530880063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a45313560 con 0x563a4542e570 2024-05-10T05:44:00.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.539+0000 7f8abd200700 1 --2- 172.21.15.131:0/3530880063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542d6a0 0x563a454317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.539+0000 7f8abdc00700 1 --2- 172.21.15.131:0/3530880063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543cb70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x563a454387c0 tx=0x7f8aa4009690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=756e24b8cea3092d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:00.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.539+0000 7f8aab400700 1 -- 172.21.15.131:0/3530880063 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4020040 con 0x563a4542e570 2024-05-10T05:44:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.540+0000 7f8aab400700 1 -- 172.21.15.131:0/3530880063 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa4010a80 con 0x563a4542e570 2024-05-10T05:44:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.540+0000 7f8aab400700 1 -- 172.21.15.131:0/3530880063 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4023630 con 0x563a4542e570 2024-05-10T05:44:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.541+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/3530880063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 msgr2=0x563a4543cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.541+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/3530880063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543cb70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x563a454387c0 tx=0x7f8aa4009690 comp rx=0 tx=0).stop 2024-05-10T05:44:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.541+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/3530880063 shutdown_connections 2024-05-10T05:44:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.541+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/3530880063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542d6a0 0x563a454317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.541+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/3530880063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542eec0 0x563a4543d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.541+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/3530880063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543cb70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.541+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/3530880063 >> 172.21.15.131:0/3530880063 conn(0x563a4538fa60 msgr2=0x563a4542dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.541+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/3530880063 shutdown_connections 2024-05-10T05:44:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.542+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/3530880063 wait complete. 2024-05-10T05:44:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.542+0000 7f8ad84f8b80 1 -- start start 2024-05-10T05:44:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.543+0000 7f8ad84f8b80 1 Processor -- start 2024-05-10T05:44:00.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.550+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a4543c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.550+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.551+0000 7f8ad84f8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542eec0 0x563a45497d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.551+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.553+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a454982a0 con 0x563a4542eec0 2024-05-10T05:44:00.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.553+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a454983e0 con 0x563a4542e570 2024-05-10T05:44:00.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.553+0000 7f8ad84f8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a45498570 con 0x563a4542d6a0 2024-05-10T05:44:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.553+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56960/0 (socket says 172.21.15.131:56960) 2024-05-10T05:44:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.553+0000 7f8abdc00700 1 -- 172.21.15.131:0/217974513 learned_addr learned my addr 172.21.15.131:0/217974513 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.553+0000 7f8ad84f8b80 10 client.0 _ll_register_callbacks cb 0x563a452f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.552+0000 7f8abd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a4543c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.556+0000 7f8abd200700 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542eec0 0x563a45497d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.557+0000 7f8abd200700 1 -- 172.21.15.131:0/217974513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 msgr2=0x563a4543c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:00.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.557+0000 7f8abd200700 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a4543c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.557+0000 7f8abd200700 1 -- 172.21.15.131:0/217974513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 msgr2=0x563a4543c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:00.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.557+0000 7f8abd200700 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.557+0000 7f8abd200700 1 -- 172.21.15.131:0/217974513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a451ce3b0 con 0x563a4542eec0 2024-05-10T05:44:00.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.557+0000 7f8abd200700 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542eec0 0x563a45497d60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0015900 tx=0x7f8ab0004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddb21cb63a4a041 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:00.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.557+0000 7f8a9f400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:44:00.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.558+0000 7f8a9f400700 1 -- 172.21.15.131:0/217974513 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0015650 con 0x563a4542eec0 2024-05-10T05:44:00.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.558+0000 7f8a9f400700 1 -- 172.21.15.131:0/217974513 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8ab001e050 con 0x563a4542eec0 2024-05-10T05:44:00.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.558+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/217974513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8aa4011e50 con 0x563a4542eec0 2024-05-10T05:44:00.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.558+0000 7f8a9f400700 1 -- 172.21.15.131:0/217974513 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ab0003ac0 con 0x563a4542eec0 2024-05-10T05:44:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.560+0000 7f8ad84f8b80 10 client.16908 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.560+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/217974513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563a451d2cd0 con 0x563a4542eec0 2024-05-10T05:44:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.560+0000 7f8a9f400700 1 -- 172.21.15.131:0/217974513 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa4011e50 con 0x563a4542eec0 2024-05-10T05:44:00.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.560+0000 7f8a9f400700 1 -- 172.21.15.131:0/217974513 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8ab0013c40 con 0x563a4542eec0 2024-05-10T05:44:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.565+0000 7f8ad84f8b80 10 client.16908 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.565+0000 7f8ad84f8b80 10 client.16908 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:44:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.565+0000 7f8ad84f8b80 20 client.16908 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.565+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x563a45443580 0x563a4549bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.566+0000 7f8ad84f8b80 4 client.16908 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:44:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.566+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/217974513 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563a45345ec0 con 0x563a45443580 2024-05-10T05:44:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.567+0000 7f8abdc00700 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x563a45443580 0x563a4549bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:00.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.570+0000 7f8a9f400700 10 client.16908 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:44:00.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.570+0000 7f8abdc00700 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x563a45443580 0x563a4549bda0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b04aa6d92909fb3 server_cookie=f11d98ded7cc16ec in_seq=0 out_seq=0 2024-05-10T05:44:00.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.573+0000 7f8a9f400700 1 -- 172.21.15.131:0/217974513 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563a45345ec0 con 0x563a45443580 2024-05-10T05:44:00.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.573+0000 7f8a9f400700 10 client.16908 handle_command_reply: tid=0 2024-05-10T05:44:00.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.573+0000 7f8ad84f8b80 1 client.16908 shutdown 2024-05-10T05:44:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.574+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/217974513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542eec0 msgr2=0x563a45497d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.574+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542eec0 0x563a45497d60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0015900 tx=0x7f8ab0004b80 comp rx=0 tx=0).stop 2024-05-10T05:44:00.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.579+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/217974513 shutdown_connections 2024-05-10T05:44:00.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.579+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x563a45443580 0x563a4549bda0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.579+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a4542eec0 0x563a45497d60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.579+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a4542d6a0 0x563a4543c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.579+0000 7f8ad84f8b80 1 --2- 172.21.15.131:0/217974513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a4542e570 0x563a4543c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.579+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/217974513 >> 172.21.15.131:0/217974513 conn(0x563a4538fa60 msgr2=0x563a454322d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.579+0000 7f8a9f400700 0 client.16908 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:44:00.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.582+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/217974513 shutdown_connections 2024-05-10T05:44:00.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.582+0000 7f8ad84f8b80 1 -- 172.21.15.131:0/217974513 wait complete. 2024-05-10T05:44:00.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.584+0000 7f8ad84f8b80 20 client.16908 trim_cache size 0 max 16384 2024-05-10T05:44:00.585 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:44:00.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.586+0000 7f8ac0600700 1 -- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8aa00a1350 msgr2=0x7f8aa00a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:00.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.586+0000 7f8ac0600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8aa00a1350 0x7f8aa00a3860 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0a2440 tx=0x7f8aac009ca0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.586+0000 7f8ac0600700 1 -- 172.21.15.131:0/878149976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab8109710 msgr2=0x7f8ab8082a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:00.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.586+0000 7f8ac0600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab8109710 0x7f8ab8082a30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8aac003e00 tx=0x7f8aac003ee0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.590+0000 7f8ac0600700 1 -- 172.21.15.131:0/878149976 shutdown_connections 2024-05-10T05:44:00.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.590+0000 7f8ac0600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8aa00a1350 0x7f8aa00a3860 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.590+0000 7f8ac0600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab8109710 0x7f8ab8082a30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.590+0000 7f8ac0600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab8108210 0x7f8ab8083da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.590+0000 7f8ac0600700 1 --2- 172.21.15.131:0/878149976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab8108bb0 0x7f8ab8082440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:00.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.590+0000 7f8ac0600700 1 -- 172.21.15.131:0/878149976 >> 172.21.15.131:0/878149976 conn(0x7f8ab806ea00 msgr2=0x7f8ab80753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:00.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.592+0000 7f8ac0600700 1 -- 172.21.15.131:0/878149976 shutdown_connections 2024-05-10T05:44:00.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:00.592+0000 7f8ac0600700 1 -- 172.21.15.131:0/878149976 wait complete. 2024-05-10T05:44:00.674 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:44:00.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:00 smithi131 ceph-mon[37264]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 277 MiB/s wr, 84 op/s 2024-05-10T05:44:01.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:00 smithi157 ceph-mon[78427]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 277 MiB/s wr, 84 op/s 2024-05-10T05:44:01.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:00 smithi142 ceph-mon[77667]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 277 MiB/s wr, 84 op/s 2024-05-10T05:44:01.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.380+0000 7fe993000700 1 -- 172.21.15.131:0/2424909768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98409dd20 msgr2=0x7fe98409e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:01.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.380+0000 7fe993000700 1 --2- 172.21.15.131:0/2424909768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98409dd20 0x7fe98409e130 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe9780099c0 tx=0x7fe978009cd0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.381+0000 7fe993000700 1 -- 172.21.15.131:0/2424909768 shutdown_connections 2024-05-10T05:44:01.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.382+0000 7fe993000700 1 --2- 172.21.15.131:0/2424909768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe98409cc00 0x7fe98409d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.382+0000 7fe993000700 1 --2- 172.21.15.131:0/2424909768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe98409e6c0 0x7fe9840a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.382+0000 7fe993000700 1 --2- 172.21.15.131:0/2424909768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98409dd20 0x7fe98409e130 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.382+0000 7fe993000700 1 -- 172.21.15.131:0/2424909768 >> 172.21.15.131:0/2424909768 conn(0x7fe98400bc40 msgr2=0x7fe98409a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:01.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.389+0000 7fe993000700 1 -- 172.21.15.131:0/2424909768 shutdown_connections 2024-05-10T05:44:01.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.389+0000 7fe993000700 1 -- 172.21.15.131:0/2424909768 wait complete. 2024-05-10T05:44:01.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.391+0000 7fe993000700 1 Processor -- start 2024-05-10T05:44:01.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.394+0000 7fe993000700 1 -- start start 2024-05-10T05:44:01.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.398+0000 7fe993000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98409cc00 0x7fe984013550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.399+0000 7fe993000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe98409dd20 0x7fe984013a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.399+0000 7fe993000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe98409e6c0 0x7fe984142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.399+0000 7fe993000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe984143310 con 0x7fe98409dd20 2024-05-10T05:44:01.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.399+0000 7fe993000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe984143450 con 0x7fe98409e6c0 2024-05-10T05:44:01.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.399+0000 7fe993000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe984143590 con 0x7fe98409cc00 2024-05-10T05:44:01.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.399+0000 7fe991c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98409cc00 0x7fe984013550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.400+0000 7fe992600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe98409e6c0 0x7fe984142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.400+0000 7fe992600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe98409e6c0 0x7fe984142dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56962/0 (socket says 172.21.15.131:56962) 2024-05-10T05:44:01.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.400+0000 7fe992600700 1 -- 172.21.15.131:0/2597260026 learned_addr learned my addr 172.21.15.131:0/2597260026 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:01.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.400+0000 7fe992600700 1 -- 172.21.15.131:0/2597260026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98409cc00 msgr2=0x7fe984013550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:01.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.400+0000 7fe992600700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98409cc00 0x7fe984013550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.400+0000 7fe992600700 1 -- 172.21.15.131:0/2597260026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe98409dd20 msgr2=0x7fe984013a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:01.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.400+0000 7fe992600700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe98409dd20 0x7fe984013a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.400+0000 7fe992600700 1 -- 172.21.15.131:0/2597260026 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780095f0 con 0x7fe98409e6c0 2024-05-10T05:44:01.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.401+0000 7fe992600700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe98409e6c0 0x7fe984142dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe98c064650 tx=0x7fe98c0783c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:01.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.402+0000 7fe98a000700 1 -- 172.21.15.131:0/2597260026 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c078db0 con 0x7fe98409e6c0 2024-05-10T05:44:01.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.402+0000 7fe98a000700 1 -- 172.21.15.131:0/2597260026 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe98c0624e0 con 0x7fe98409e6c0 2024-05-10T05:44:01.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.402+0000 7fe98a000700 1 -- 172.21.15.131:0/2597260026 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c07d600 con 0x7fe98409e6c0 2024-05-10T05:44:01.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.403+0000 7fe993000700 1 -- 172.21.15.131:0/2597260026 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe984143840 con 0x7fe98409e6c0 2024-05-10T05:44:01.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.403+0000 7fe993000700 1 -- 172.21.15.131:0/2597260026 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe984143d00 con 0x7fe98409e6c0 2024-05-10T05:44:01.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.417+0000 7fe98a000700 1 -- 172.21.15.131:0/2597260026 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe98c0617e0 con 0x7fe98409e6c0 2024-05-10T05:44:01.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.420+0000 7fe98a000700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9740a1270 0x7fe9740a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.420+0000 7fe9ac48bb80 1 Processor -- start 2024-05-10T05:44:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.421+0000 7fe9ac48bb80 1 -- start start 2024-05-10T05:44:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.421+0000 7fe991c00700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9740a1270 0x7fe9740a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.421+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 0x563d23f8a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.421+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8b7a0 0x563d23f8bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.421+0000 7fe98a000700 1 -- 172.21.15.131:0/2597260026 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe98c136cf0 con 0x7fe98409e6c0 2024-05-10T05:44:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.421+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 0x563d23f929f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.421+0000 7fe9ac48bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23eb1150 con 0x563d23f8a530 2024-05-10T05:44:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.421+0000 7fe9ac48bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23eaeb10 con 0x563d23f8c0f0 2024-05-10T05:44:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.421+0000 7fe9ac48bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23eaa400 con 0x563d23f8b7a0 2024-05-10T05:44:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.422+0000 7fe991200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8b7a0 0x563d23f8bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.422+0000 7fe991200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8b7a0 0x563d23f8bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54460/0 (socket says 172.21.15.131:54460) 2024-05-10T05:44:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.422+0000 7fe991200700 1 -- 172.21.15.131:0/1414760558 learned_addr learned my addr 172.21.15.131:0/1414760558 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:01.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.422+0000 7fe992600700 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 0x563d23f929f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.422+0000 7fe991c00700 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 0x563d23f8a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.423+0000 7fe991200700 1 -- 172.21.15.131:0/1414760558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 msgr2=0x563d23f929f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:01.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.423+0000 7fe991200700 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 0x563d23f929f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.423+0000 7fe991200700 1 -- 172.21.15.131:0/1414760558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 msgr2=0x563d23f8a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:01.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.423+0000 7fe991200700 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 0x563d23f8a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.423+0000 7fe991200700 1 -- 172.21.15.131:0/1414760558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d23d2b3b0 con 0x563d23f8b7a0 2024-05-10T05:44:01.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.423+0000 7fe992600700 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 0x563d23f929f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:01.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.423+0000 7fe991c00700 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 0x563d23f8a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:01.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.424+0000 7fe991200700 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8b7a0 0x563d23f8bbb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe978009990 tx=0x7fe978000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10796bfee55d0743 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:01.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.424+0000 7fe991c00700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9740a1270 0x7fe9740a3780 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe980005fd0 tx=0x7fe98000e630 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:01.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.424+0000 7fe97ea00700 1 -- 172.21.15.131:0/1414760558 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97801d070 con 0x563d23f8b7a0 2024-05-10T05:44:01.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.424+0000 7fe97ea00700 1 -- 172.21.15.131:0/1414760558 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9780046b0 con 0x563d23f8b7a0 2024-05-10T05:44:01.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.425+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1414760558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8b7a0 msgr2=0x563d23f8bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:01.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.425+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8b7a0 0x563d23f8bbb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe978009990 tx=0x7fe978000bc0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.426+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1414760558 shutdown_connections 2024-05-10T05:44:01.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.426+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 0x563d23f8a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.426+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 0x563d23f929f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.426+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1414760558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8b7a0 0x563d23f8bbb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.426+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1414760558 >> 172.21.15.131:0/1414760558 conn(0x563d23eeca60 msgr2=0x563d23f85330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:01.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.427+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1414760558 shutdown_connections 2024-05-10T05:44:01.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.427+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1414760558 wait complete. 2024-05-10T05:44:01.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.427+0000 7fe9ac48bb80 1 -- start start 2024-05-10T05:44:01.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.428+0000 7fe9ac48bb80 1 Processor -- start 2024-05-10T05:44:01.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.429+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8a530 0x563d23f94860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.429+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 0x563d23f94da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.429+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 0x563d23f9bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.429+0000 7fe9ac48bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23e46560 con 0x563d23f8b7a0 2024-05-10T05:44:01.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.429+0000 7fe9ac48bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23e46200 con 0x563d23f8c0f0 2024-05-10T05:44:01.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.429+0000 7fe9ac48bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23d2d6a0 con 0x563d23f8a530 2024-05-10T05:44:01.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.430+0000 7fe991c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8a530 0x563d23f94860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.430+0000 7fe991c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8a530 0x563d23f94860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54464/0 (socket says 172.21.15.131:54464) 2024-05-10T05:44:01.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.430+0000 7fe991c00700 1 -- 172.21.15.131:0/1552164804 learned_addr learned my addr 172.21.15.131:0/1552164804 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:01.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.430+0000 7fe9ac48bb80 10 client.0 _ll_register_callbacks cb 0x563d23e4fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:01.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.430+0000 7fe992600700 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 0x563d23f9bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.431+0000 7fe991c00700 1 -- 172.21.15.131:0/1552164804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 msgr2=0x563d23f9bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:01.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.431+0000 7fe991c00700 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 0x563d23f9bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.431+0000 7fe991c00700 1 -- 172.21.15.131:0/1552164804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 msgr2=0x563d23f94da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:01.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.431+0000 7fe991c00700 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 0x563d23f94da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.431+0000 7fe991c00700 1 -- 172.21.15.131:0/1552164804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe984143d00 con 0x563d23f8a530 2024-05-10T05:44:01.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.432+0000 7fe991c00700 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8a530 0x563d23f94860 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe980012a50 tx=0x7fe9800047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f8ef45c4660678 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:01.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.432+0000 7fe973400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:01.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.432+0000 7fe973400700 1 -- 172.21.15.131:0/1552164804 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980018030 con 0x563d23f8a530 2024-05-10T05:44:01.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.433+0000 7fe973400700 1 -- 172.21.15.131:0/1552164804 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe980019070 con 0x563d23f8a530 2024-05-10T05:44:01.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.433+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1552164804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d23f59780 con 0x563d23f8a530 2024-05-10T05:44:01.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.434+0000 7fe973400700 1 -- 172.21.15.131:0/1552164804 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe98000ae70 con 0x563d23f8a530 2024-05-10T05:44:01.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.434+0000 7fe9ac48bb80 10 client.27292 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:01.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.434+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1552164804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe9840a5a00 con 0x563d23f8a530 2024-05-10T05:44:01.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.438+0000 7fe973400700 1 -- 172.21.15.131:0/1552164804 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe980011c80 con 0x563d23f8a530 2024-05-10T05:44:01.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.440+0000 7fe973400700 1 -- 172.21.15.131:0/1552164804 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe980024070 con 0x563d23f8a530 2024-05-10T05:44:01.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.440+0000 7fe9ac48bb80 10 client.27292 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:01.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.440+0000 7fe9ac48bb80 10 client.27292 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:44:01.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.440+0000 7fe9ac48bb80 20 client.27292 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:01.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.440+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d23fa0540 0x563d23fa2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.440+0000 7fe9ac48bb80 4 client.27292 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:01.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.440+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1552164804 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d23f59780 con 0x563d23fa0540 2024-05-10T05:44:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.443+0000 7fe991200700 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d23fa0540 0x563d23fa2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.444+0000 7fe991200700 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d23fa0540 0x563d23fa2a30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b936a69e8115e94 server_cookie=33397164b3a22e0a in_seq=0 out_seq=0 2024-05-10T05:44:01.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.446+0000 7fe973400700 10 client.27292 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:44:01.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.446+0000 7fe973400700 1 -- 172.21.15.131:0/1552164804 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d23f59780 con 0x563d23fa0540 2024-05-10T05:44:01.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.446+0000 7fe973400700 10 client.27292 handle_command_reply: tid=0 2024-05-10T05:44:01.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.446+0000 7fe9ac48bb80 1 client.27292 shutdown 2024-05-10T05:44:01.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.448+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1552164804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8a530 msgr2=0x563d23f94860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:01.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.448+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8a530 0x563d23f94860 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe980012a50 tx=0x7fe9800047c0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.450+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1552164804 shutdown_connections 2024-05-10T05:44:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.450+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d23fa0540 0x563d23fa2a30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.450+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8c0f0 0x563d23f9bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.450+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8a530 0x563d23f94860 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.450+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1552164804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 0x563d23f94da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.450+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1552164804 >> 172.21.15.131:0/1552164804 conn(0x563d23eeca60 msgr2=0x563d23f89060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.450+0000 7fe973400700 0 client.27292 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:44:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.450+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1552164804 shutdown_connections 2024-05-10T05:44:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.450+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1552164804 wait complete. 2024-05-10T05:44:01.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.451+0000 7fe9ac48bb80 20 client.27292 trim_cache size 0 max 16384 2024-05-10T05:44:01.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.480+0000 7fe9ac48bb80 1 Processor -- start 2024-05-10T05:44:01.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.480+0000 7fe9ac48bb80 1 -- start start 2024-05-10T05:44:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.481+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 0x563d23f99cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.481+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8b7a0 0x563d23f9a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.481+0000 7fe991200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8b7a0 0x563d23f9a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.481+0000 7fe991200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8b7a0 0x563d23f9a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:56990/0 (socket says 172.21.15.131:56990) 2024-05-10T05:44:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.482+0000 7fe991c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 0x563d23f99cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.481+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8c0f0 0x563d23fa2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.482+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/3371434726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23f8b560 con 0x563d23f8a530 2024-05-10T05:44:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.482+0000 7fe991200700 1 -- 172.21.15.131:0/3371434726 learned_addr learned my addr 172.21.15.131:0/3371434726 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.483+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/3371434726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23f7f8d0 con 0x563d23f8b7a0 2024-05-10T05:44:01.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.483+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/3371434726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23f9c790 con 0x563d23f8c0f0 2024-05-10T05:44:01.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.485+0000 7fe991200700 1 -- 172.21.15.131:0/3371434726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8c0f0 msgr2=0x563d23fa2550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:44:01.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.485+0000 7fe991200700 1 --2- 172.21.15.131:0/3371434726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8c0f0 0x563d23fa2550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.485+0000 7fe991200700 1 -- 172.21.15.131:0/3371434726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 msgr2=0x563d23f99cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:01.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.485+0000 7fe991200700 1 --2- 172.21.15.131:0/3371434726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 0x563d23f99cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.485+0000 7fe991200700 1 -- 172.21.15.131:0/3371434726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d23cd7b60 con 0x563d23f8b7a0 2024-05-10T05:44:01.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.486+0000 7fe991c00700 1 --2- 172.21.15.131:0/3371434726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 0x563d23f99cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:01.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.487+0000 7fe991200700 1 --2- 172.21.15.131:0/3371434726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8b7a0 0x563d23f9a1f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe978009990 tx=0x7fe97801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95fe378971e9cfa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:01.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.488+0000 7fe97f400700 1 -- 172.21.15.131:0/3371434726 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97802e070 con 0x563d23f8b7a0 2024-05-10T05:44:01.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.488+0000 7fe97f400700 1 -- 172.21.15.131:0/3371434726 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe97802a8e0 con 0x563d23f8b7a0 2024-05-10T05:44:01.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.488+0000 7fe97f400700 1 -- 172.21.15.131:0/3371434726 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97801d040 con 0x563d23f8b7a0 2024-05-10T05:44:01.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.491+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/3371434726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8b7a0 msgr2=0x563d23f9a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:01.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.491+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/3371434726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8b7a0 0x563d23f9a1f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe978009990 tx=0x7fe97801a040 comp rx=0 tx=0).stop 2024-05-10T05:44:01.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.492+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/3371434726 shutdown_connections 2024-05-10T05:44:01.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.492+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/3371434726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8a530 0x563d23f99cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.492+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/3371434726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8c0f0 0x563d23fa2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.492+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/3371434726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8b7a0 0x563d23f9a1f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.493+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/3371434726 >> 172.21.15.131:0/3371434726 conn(0x563d23eeca60 msgr2=0x563d23f8f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:01.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.494+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/3371434726 shutdown_connections 2024-05-10T05:44:01.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.494+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/3371434726 wait complete. 2024-05-10T05:44:01.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.496+0000 7fe9ac48bb80 1 -- start start 2024-05-10T05:44:01.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.497+0000 7fe9ac48bb80 1 Processor -- start 2024-05-10T05:44:01.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.499+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8a530 0x563d23f98e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.500+0000 7fe9ac48bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 0x563d23f99340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.502+0000 7fe991200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 0x563d23f99340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.502+0000 7fe991200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 0x563d23f99340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34560/0 (socket says 172.21.15.131:34560) 2024-05-10T05:44:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.502+0000 7fe991200700 1 -- 172.21.15.131:0/1809685217 learned_addr learned my addr 172.21.15.131:0/1809685217 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.502+0000 7fe991c00700 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8a530 0x563d23f98e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.502+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8c0f0 0x563d23ff4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.502+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23f99b60 con 0x563d23f8b7a0 2024-05-10T05:44:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.502+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23f99cd0 con 0x563d23f8a530 2024-05-10T05:44:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.502+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d23ff5060 con 0x563d23f8c0f0 2024-05-10T05:44:01.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.502+0000 7fe9ac48bb80 10 client.0 _ll_register_callbacks cb 0x563d23e4fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:01.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.504+0000 7fe992600700 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8c0f0 0x563d23ff4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.506+0000 7fe991200700 1 -- 172.21.15.131:0/1809685217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8c0f0 msgr2=0x563d23ff4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:01.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.506+0000 7fe991200700 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8c0f0 0x563d23ff4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.506+0000 7fe991200700 1 -- 172.21.15.131:0/1809685217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8a530 msgr2=0x563d23f98e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:01.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.506+0000 7fe991200700 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8a530 0x563d23f98e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.506+0000 7fe991200700 1 -- 172.21.15.131:0/1809685217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9840a5a00 con 0x563d23f8b7a0 2024-05-10T05:44:01.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.513+0000 7fe991200700 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 0x563d23f99340 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe9780099c0 tx=0x7fe97800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f1c98e8009365 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:01.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.513+0000 7fe973400700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:44:01.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.515+0000 7fe973400700 1 -- 172.21.15.131:0/1809685217 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97801d050 con 0x563d23f8b7a0 2024-05-10T05:44:01.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.515+0000 7fe973400700 1 -- 172.21.15.131:0/1809685217 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fe97800bdf0 con 0x563d23f8b7a0 2024-05-10T05:44:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.516+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d23e03270 con 0x563d23f8b7a0 2024-05-10T05:44:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.518+0000 7fe973400700 1 -- 172.21.15.131:0/1809685217 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe97802e040 con 0x563d23f8b7a0 2024-05-10T05:44:01.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.518+0000 7fe9ac48bb80 10 client.16917 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:01.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.518+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe978026040 con 0x563d23f8b7a0 2024-05-10T05:44:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.523+0000 7fe973400700 1 -- 172.21.15.131:0/1809685217 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d23e03270 con 0x563d23f8b7a0 2024-05-10T05:44:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.523+0000 7fe973400700 1 -- 172.21.15.131:0/1809685217 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe978039070 con 0x563d23f8b7a0 2024-05-10T05:44:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.523+0000 7fe9ac48bb80 10 client.16917 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.523+0000 7fe9ac48bb80 10 client.16917 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:44:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.523+0000 7fe9ac48bb80 20 client.16917 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.523+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d23ff64c0 0x563d23ff88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.523+0000 7fe9ac48bb80 4 client.16917 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:44:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.524+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d23ea2ec0 con 0x563d23ff64c0 2024-05-10T05:44:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.524+0000 7fe991c00700 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d23ff64c0 0x563d23ff88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:01.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.534+0000 7fe991c00700 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d23ff64c0 0x563d23ff88e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5ad2cb881f40df7 server_cookie=fa803dc788b2c541 in_seq=0 out_seq=0 2024-05-10T05:44:01.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.535+0000 7fe973400700 10 client.16917 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:44:01.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.536+0000 7fe973400700 1 -- 172.21.15.131:0/1809685217 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563d23ea2ec0 con 0x563d23ff64c0 2024-05-10T05:44:01.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.536+0000 7fe973400700 10 client.16917 handle_command_reply: tid=0 2024-05-10T05:44:01.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.537+0000 7fe9ac48bb80 1 client.16917 shutdown 2024-05-10T05:44:01.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.540+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 msgr2=0x563d23f99340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:01.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.540+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 0x563d23f99340 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe9780099c0 tx=0x7fe97800b890 comp rx=0 tx=0).stop 2024-05-10T05:44:01.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.541+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 shutdown_connections 2024-05-10T05:44:01.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.541+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x563d23ff64c0 0x563d23ff88e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.541+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563d23f8c0f0 0x563d23ff4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.541+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d23f8a530 0x563d23f98e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.541+0000 7fe9ac48bb80 1 --2- 172.21.15.131:0/1809685217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d23f8b7a0 0x563d23f99340 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.541+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 >> 172.21.15.131:0/1809685217 conn(0x563d23eeca60 msgr2=0x563d23fa2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:01.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.541+0000 7fe973400700 0 client.16917 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:44:01.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.542+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 shutdown_connections 2024-05-10T05:44:01.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.542+0000 7fe9ac48bb80 1 -- 172.21.15.131:0/1809685217 wait complete. 2024-05-10T05:44:01.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.543+0000 7fe9ac48bb80 20 client.16917 trim_cache size 0 max 16384 2024-05-10T05:44:01.544 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:44:01.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.545+0000 7fe993000700 1 -- 172.21.15.131:0/2597260026 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9740a1270 msgr2=0x7fe9740a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:01.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.545+0000 7fe993000700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9740a1270 0x7fe9740a3780 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe980005fd0 tx=0x7fe98000e630 comp rx=0 tx=0).stop 2024-05-10T05:44:01.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.545+0000 7fe993000700 1 -- 172.21.15.131:0/2597260026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe98409e6c0 msgr2=0x7fe984142dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:01.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.545+0000 7fe993000700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe98409e6c0 0x7fe984142dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe98c064650 tx=0x7fe98c0783c0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.546+0000 7fe993000700 1 -- 172.21.15.131:0/2597260026 shutdown_connections 2024-05-10T05:44:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.546+0000 7fe993000700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fe9740a1270 0x7fe9740a3780 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.546+0000 7fe993000700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe98409e6c0 0x7fe984142dd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.546+0000 7fe993000700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98409cc00 0x7fe984013550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.546+0000 7fe993000700 1 --2- 172.21.15.131:0/2597260026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe98409dd20 0x7fe984013a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.546+0000 7fe993000700 1 -- 172.21.15.131:0/2597260026 >> 172.21.15.131:0/2597260026 conn(0x7fe98400bc40 msgr2=0x7fe9840a0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:01.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.547+0000 7fe993000700 1 -- 172.21.15.131:0/2597260026 shutdown_connections 2024-05-10T05:44:01.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:01.547+0000 7fe993000700 1 -- 172.21.15.131:0/2597260026 wait complete. 2024-05-10T05:44:01.675 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:44:02.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.510+0000 7f1458e00700 1 -- 172.21.15.131:0/4147464176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1454109300 msgr2=0x7f1454109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:02.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.510+0000 7f1458e00700 1 --2- 172.21.15.131:0/4147464176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1454109300 0x7f1454109710 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1444009990 tx=0x7f1444009ca0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.514+0000 7f1458e00700 1 -- 172.21.15.131:0/4147464176 shutdown_connections 2024-05-10T05:44:02.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.514+0000 7f1458e00700 1 --2- 172.21.15.131:0/4147464176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1454109c50 0x7f145410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.514+0000 7f1458e00700 1 --2- 172.21.15.131:0/4147464176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1454108630 0x7f1454108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.514+0000 7f1458e00700 1 --2- 172.21.15.131:0/4147464176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1454109300 0x7f1454109710 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.514+0000 7f1458e00700 1 -- 172.21.15.131:0/4147464176 >> 172.21.15.131:0/4147464176 conn(0x7f145406ea00 msgr2=0x7f1454070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:02.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.515+0000 7f1458e00700 1 -- 172.21.15.131:0/4147464176 shutdown_connections 2024-05-10T05:44:02.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.515+0000 7f1458e00700 1 -- 172.21.15.131:0/4147464176 wait complete. 2024-05-10T05:44:02.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.516+0000 7f1458e00700 1 Processor -- start 2024-05-10T05:44:02.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.517+0000 7f1458e00700 1 -- start start 2024-05-10T05:44:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.518+0000 7f1458e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1454108630 0x7f1454115540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.518+0000 7f1458e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1454109300 0x7f1454115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.518+0000 7f1458e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1454109c50 0x7f14541169e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.518+0000 7f1458e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454116ff0 con 0x7f1454108630 2024-05-10T05:44:02.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.518+0000 7f1458e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454117160 con 0x7f1454109c50 2024-05-10T05:44:02.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.518+0000 7f1458e00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454117320 con 0x7f1454109300 2024-05-10T05:44:02.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.519+0000 7f1451e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1454108630 0x7f1454115540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.520+0000 7f1451400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1454109300 0x7f1454115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.520+0000 7f1451400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1454109300 0x7f1454115a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54480/0 (socket says 172.21.15.131:54480) 2024-05-10T05:44:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.520+0000 7f1451400700 1 -- 172.21.15.131:0/391679207 learned_addr learned my addr 172.21.15.131:0/391679207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.520+0000 7f1451400700 1 -- 172.21.15.131:0/391679207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1454109c50 msgr2=0x7f14541169e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:44:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.520+0000 7f1451400700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1454109c50 0x7f14541169e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.520+0000 7f1451400700 1 -- 172.21.15.131:0/391679207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1454108630 msgr2=0x7f1454115540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.520+0000 7f1451400700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1454108630 0x7f1454115540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.520+0000 7f1451400700 1 -- 172.21.15.131:0/391679207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440095f0 con 0x7f1454109300 2024-05-10T05:44:02.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.521+0000 7f1451400700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1454109300 0x7f1454115a80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1444003800 tx=0x7f1444003b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:02.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.522+0000 7f144aa00700 1 -- 172.21.15.131:0/391679207 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144401d070 con 0x7f1454109300 2024-05-10T05:44:02.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.522+0000 7f144aa00700 1 -- 172.21.15.131:0/391679207 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f144400f460 con 0x7f1454109300 2024-05-10T05:44:02.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.525+0000 7f1458e00700 1 -- 172.21.15.131:0/391679207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14541175a0 con 0x7f1454109300 2024-05-10T05:44:02.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.527+0000 7f144aa00700 1 -- 172.21.15.131:0/391679207 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444017650 con 0x7f1454109300 2024-05-10T05:44:02.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.527+0000 7f1458e00700 1 -- 172.21.15.131:0/391679207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1454117ad0 con 0x7f1454109300 2024-05-10T05:44:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.531+0000 7f144aa00700 1 -- 172.21.15.131:0/391679207 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f14440052d0 con 0x7f1454109300 2024-05-10T05:44:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.533+0000 7f144aa00700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14380a1130 0x7f14380a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.534+0000 7f144aa00700 1 -- 172.21.15.131:0/391679207 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f14440d0400 con 0x7f1454109300 2024-05-10T05:44:02.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.535+0000 7f1451e00700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14380a1130 0x7f14380a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.536+0000 7f1470cccb80 1 Processor -- start 2024-05-10T05:44:02.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.536+0000 7f1470cccb80 1 -- start start 2024-05-10T05:44:02.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.536+0000 7f1451e00700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14380a1130 0x7f14380a3640 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1454108320 tx=0x7f1440009190 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:02.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.536+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a613af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.536+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614570 0x55f99a614980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.536+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a61b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.537+0000 7f1451e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a613af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.537+0000 7f1451e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a613af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34590/0 (socket says 172.21.15.131:34590) 2024-05-10T05:44:02.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.538+0000 7f1451e00700 1 -- 172.21.15.131:0/526827038 learned_addr learned my addr 172.21.15.131:0/526827038 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:02.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.538+0000 7f1470cccb80 1 -- 172.21.15.131:0/526827038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a53a150 con 0x55f99a6136a0 2024-05-10T05:44:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.538+0000 7f1470cccb80 1 -- 172.21.15.131:0/526827038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a537b10 con 0x55f99a614570 2024-05-10T05:44:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.538+0000 7f1470cccb80 1 -- 172.21.15.131:0/526827038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a533400 con 0x55f99a614ec0 2024-05-10T05:44:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.538+0000 7f1452800700 1 --2- 172.21.15.131:0/526827038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a61b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.538+0000 7f1451400700 1 --2- 172.21.15.131:0/526827038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614570 0x55f99a614980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.539+0000 7f1451e00700 1 -- 172.21.15.131:0/526827038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 msgr2=0x55f99a61b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.539+0000 7f1451e00700 1 --2- 172.21.15.131:0/526827038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a61b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.539+0000 7f1451e00700 1 -- 172.21.15.131:0/526827038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614570 msgr2=0x55f99a614980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.539+0000 7f1451e00700 1 --2- 172.21.15.131:0/526827038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614570 0x55f99a614980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.539+0000 7f1451e00700 1 -- 172.21.15.131:0/526827038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f99a3b43b0 con 0x55f99a6136a0 2024-05-10T05:44:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.539+0000 7f1452800700 1 --2- 172.21.15.131:0/526827038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a61b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.540+0000 7f1451e00700 1 --2- 172.21.15.131:0/526827038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a613af0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f144000e9f0 tx=0x7f1440011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cadaa818874e32f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.540+0000 7f143f400700 1 -- 172.21.15.131:0/526827038 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000eeb0 con 0x55f99a6136a0 2024-05-10T05:44:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.540+0000 7f143f400700 1 -- 172.21.15.131:0/526827038 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1440015940 con 0x55f99a6136a0 2024-05-10T05:44:02.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.540+0000 7f143f400700 1 -- 172.21.15.131:0/526827038 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440013a80 con 0x55f99a6136a0 2024-05-10T05:44:02.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.543+0000 7f1470cccb80 1 -- 172.21.15.131:0/526827038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 msgr2=0x55f99a613af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:02.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.543+0000 7f1470cccb80 1 --2- 172.21.15.131:0/526827038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a613af0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f144000e9f0 tx=0x7f1440011000 comp rx=0 tx=0).stop 2024-05-10T05:44:02.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.545+0000 7f1470cccb80 1 -- 172.21.15.131:0/526827038 shutdown_connections 2024-05-10T05:44:02.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.545+0000 7f1470cccb80 1 --2- 172.21.15.131:0/526827038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a613af0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.545+0000 7f1470cccb80 1 --2- 172.21.15.131:0/526827038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a61b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.545+0000 7f1470cccb80 1 --2- 172.21.15.131:0/526827038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614570 0x55f99a614980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.545+0000 7f1470cccb80 1 -- 172.21.15.131:0/526827038 >> 172.21.15.131:0/526827038 conn(0x55f99a575a60 msgr2=0x55f99a60dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:02.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.547+0000 7f1470cccb80 1 -- 172.21.15.131:0/526827038 shutdown_connections 2024-05-10T05:44:02.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.547+0000 7f1470cccb80 1 -- 172.21.15.131:0/526827038 wait complete. 2024-05-10T05:44:02.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.548+0000 7f1470cccb80 1 -- start start 2024-05-10T05:44:02.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.548+0000 7f1470cccb80 1 Processor -- start 2024-05-10T05:44:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.549+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a6136a0 0x55f99a61d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.549+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a614570 0x55f99a61dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.550+0000 7f1451400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a614570 0x55f99a61dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.550+0000 7f1451e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a6136a0 0x55f99a61d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.550+0000 7f1451400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a614570 0x55f99a61dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34600/0 (socket says 172.21.15.131:34600) 2024-05-10T05:44:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.550+0000 7f1451400700 1 -- 172.21.15.131:0/929200019 learned_addr learned my addr 172.21.15.131:0/929200019 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.550+0000 7f1470cccb80 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614ec0 0x55f99a624c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.551+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a4cf560 con 0x55f99a614570 2024-05-10T05:44:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.551+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a4cf200 con 0x55f99a614ec0 2024-05-10T05:44:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.551+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a3b66a0 con 0x55f99a6136a0 2024-05-10T05:44:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.552+0000 7f1451e00700 1 -- 172.21.15.131:0/929200019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614ec0 msgr2=0x55f99a624c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:44:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.552+0000 7f1451e00700 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614ec0 0x55f99a624c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.552+0000 7f1451e00700 1 -- 172.21.15.131:0/929200019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a614570 msgr2=0x55f99a61dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.552+0000 7f1470cccb80 10 client.0 _ll_register_callbacks cb 0x55f99a4d8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.552+0000 7f1451e00700 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a614570 0x55f99a61dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.553+0000 7f1451e00700 1 -- 172.21.15.131:0/929200019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c009910 con 0x55f99a6136a0 2024-05-10T05:44:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.553+0000 7f1451400700 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a614570 0x55f99a61dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.553+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f99a48c270 con 0x55f99a6136a0 2024-05-10T05:44:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.554+0000 7f1433e00700 10 client.27304 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.554+0000 7f1451e00700 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a6136a0 0x55f99a61d7f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f14400097f0 tx=0x7f1440015ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cc04a38f59267af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.555+0000 7f1433e00700 1 -- 172.21.15.131:0/929200019 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000eeb0 con 0x55f99a6136a0 2024-05-10T05:44:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.556+0000 7f1433e00700 1 -- 172.21.15.131:0/929200019 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1440004460 con 0x55f99a6136a0 2024-05-10T05:44:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.557+0000 7f1433e00700 1 -- 172.21.15.131:0/929200019 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1440004660 con 0x55f99a6136a0 2024-05-10T05:44:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.558+0000 7f1470cccb80 10 client.27304 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.558+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x7f144000e6d0 con 0x55f99a6136a0 2024-05-10T05:44:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.561+0000 7f1433e00700 1 -- 172.21.15.131:0/929200019 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14400137c0 con 0x55f99a6136a0 2024-05-10T05:44:02.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.564+0000 7f1470cccb80 10 client.27304 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.564+0000 7f1470cccb80 10 client.27304 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:44:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.564+0000 7f1470cccb80 20 client.27304 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.564+0000 7f1470cccb80 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55f99a6292c0 0x55f99a62b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.564+0000 7f1470cccb80 4 client.27304 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.564+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f99a5e2780 con 0x55f99a6292c0 2024-05-10T05:44:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.565+0000 7f1451400700 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55f99a6292c0 0x55f99a62b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.566+0000 7f1451400700 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55f99a6292c0 0x55f99a62b710 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff8561441d58038a server_cookie=97ff6eb0257cb7fa in_seq=0 out_seq=0 2024-05-10T05:44:02.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.566+0000 7f1433e00700 10 client.27304 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:44:02.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.568+0000 7f1433e00700 1 -- 172.21.15.131:0/929200019 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f99a5e2780 con 0x55f99a6292c0 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.568+0000 7f1433e00700 10 client.27304 handle_command_reply: tid=0 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.569+0000 7f1470cccb80 1 client.27304 shutdown 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.569+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a6136a0 msgr2=0x55f99a61d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.569+0000 7f1470cccb80 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a6136a0 0x55f99a61d7f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f14400097f0 tx=0x7f1440015ce0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.570+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 shutdown_connections 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.570+0000 7f1470cccb80 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55f99a6292c0 0x55f99a62b710 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.570+0000 7f1470cccb80 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614ec0 0x55f99a624c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.570+0000 7f1470cccb80 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a6136a0 0x55f99a61d7f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.570+0000 7f1433e00700 0 client.27304 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.570+0000 7f1470cccb80 1 --2- 172.21.15.131:0/929200019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a614570 0x55f99a61dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.570+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 >> 172.21.15.131:0/929200019 conn(0x55f99a575a60 msgr2=0x55f99a60dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.570+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 shutdown_connections 2024-05-10T05:44:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.571+0000 7f1470cccb80 1 -- 172.21.15.131:0/929200019 wait complete. 2024-05-10T05:44:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.571+0000 7f1470cccb80 20 client.27304 trim_cache size 0 max 16384 2024-05-10T05:44:02.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.640+0000 7f1470cccb80 1 Processor -- start 2024-05-10T05:44:02.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.641+0000 7f1470cccb80 1 -- start start 2024-05-10T05:44:02.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.642+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a622c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.642+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614570 0x55f99a623180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.642+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614ec0 0x55f99a61bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.642+0000 7f1470cccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a60ead0 con 0x55f99a6136a0 2024-05-10T05:44:02.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.642+0000 7f1470cccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a62b6e0 con 0x55f99a614ec0 2024-05-10T05:44:02.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.642+0000 7f1470cccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a62b8a0 con 0x55f99a614570 2024-05-10T05:44:02.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.644+0000 7f1451400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614570 0x55f99a623180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.644+0000 7f1451400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614570 0x55f99a623180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54500/0 (socket says 172.21.15.131:54500) 2024-05-10T05:44:02.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.644+0000 7f1451400700 1 -- 172.21.15.131:0/1969794001 learned_addr learned my addr 172.21.15.131:0/1969794001 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:02.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.644+0000 7f1452800700 1 --2- 172.21.15.131:0/1969794001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614ec0 0x55f99a61bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.645+0000 7f1451400700 1 -- 172.21.15.131:0/1969794001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614ec0 msgr2=0x55f99a61bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:02.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.645+0000 7f1451400700 1 --2- 172.21.15.131:0/1969794001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614ec0 0x55f99a61bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.645+0000 7f1451400700 1 -- 172.21.15.131:0/1969794001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 msgr2=0x55f99a622c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:02.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.645+0000 7f1451400700 1 --2- 172.21.15.131:0/1969794001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a622c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.645+0000 7f1451400700 1 -- 172.21.15.131:0/1969794001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f99a360b60 con 0x55f99a614570 2024-05-10T05:44:02.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.646+0000 7f1451400700 1 --2- 172.21.15.131:0/1969794001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614570 0x55f99a623180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1444096cb0 tx=0x7f1444085040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccbbe210d3016e69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:02.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.647+0000 7f143fe00700 1 -- 172.21.15.131:0/1969794001 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144409e5e0 con 0x55f99a614570 2024-05-10T05:44:02.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.647+0000 7f143fe00700 1 -- 172.21.15.131:0/1969794001 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14440a2ba0 con 0x55f99a614570 2024-05-10T05:44:02.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.648+0000 7f143fe00700 1 -- 172.21.15.131:0/1969794001 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144409e5e0 con 0x55f99a614570 2024-05-10T05:44:02.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.649+0000 7f1470cccb80 1 -- 172.21.15.131:0/1969794001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614570 msgr2=0x55f99a623180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:02.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.649+0000 7f1470cccb80 1 --2- 172.21.15.131:0/1969794001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614570 0x55f99a623180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1444096cb0 tx=0x7f1444085040 comp rx=0 tx=0).stop 2024-05-10T05:44:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.655+0000 7f1470cccb80 1 -- 172.21.15.131:0/1969794001 shutdown_connections 2024-05-10T05:44:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.655+0000 7f1470cccb80 1 --2- 172.21.15.131:0/1969794001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a622c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.655+0000 7f1470cccb80 1 --2- 172.21.15.131:0/1969794001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614ec0 0x55f99a61bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.655+0000 7f1470cccb80 1 --2- 172.21.15.131:0/1969794001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614570 0x55f99a623180 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.655+0000 7f1470cccb80 1 -- 172.21.15.131:0/1969794001 >> 172.21.15.131:0/1969794001 conn(0x55f99a575a60 msgr2=0x55f99a618450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.656+0000 7f1470cccb80 1 -- 172.21.15.131:0/1969794001 shutdown_connections 2024-05-10T05:44:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.656+0000 7f1470cccb80 1 -- 172.21.15.131:0/1969794001 wait complete. 2024-05-10T05:44:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.656+0000 7f1470cccb80 1 -- start start 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.657+0000 7f1470cccb80 1 Processor -- start 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.657+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a6228d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.657+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614570 0x55f99a622e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.658+0000 7f1470cccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a67de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.658+0000 7f1470cccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a67e380 con 0x55f99a6136a0 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.658+0000 7f1470cccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a67e4c0 con 0x55f99a614570 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.658+0000 7f1470cccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f99a67e650 con 0x55f99a614ec0 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.658+0000 7f1470cccb80 10 client.0 _ll_register_callbacks cb 0x55f99a4d8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.658+0000 7f1452800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a67de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.658+0000 7f1452800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a67de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54510/0 (socket says 172.21.15.131:54510) 2024-05-10T05:44:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.658+0000 7f1452800700 1 -- 172.21.15.131:0/536477219 learned_addr learned my addr 172.21.15.131:0/536477219 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.659+0000 7f1451400700 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614570 0x55f99a622e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.659+0000 7f1452800700 1 -- 172.21.15.131:0/536477219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614570 msgr2=0x55f99a622e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.659+0000 7f1452800700 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614570 0x55f99a622e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.659+0000 7f1452800700 1 -- 172.21.15.131:0/536477219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 msgr2=0x55f99a6228d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.659+0000 7f1451e00700 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a6228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.659+0000 7f1452800700 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a6228d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.659+0000 7f1452800700 1 -- 172.21.15.131:0/536477219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1454117ad0 con 0x55f99a614ec0 2024-05-10T05:44:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.659+0000 7f1451e00700 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a6228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.660+0000 7f1452800700 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a67de40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f144c00abd0 tx=0x7f144c00af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9eda0eed86387 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.660+0000 7f1433e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.660+0000 7f1433e00700 1 -- 172.21.15.131:0/536477219 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c010040 con 0x55f99a614ec0 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.660+0000 7f1433e00700 1 -- 172.21.15.131:0/536477219 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f144c011c00 con 0x55f99a614ec0 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.660+0000 7f1470cccb80 1 -- 172.21.15.131:0/536477219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1440015940 con 0x55f99a614ec0 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.660+0000 7f1433e00700 1 -- 172.21.15.131:0/536477219 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f144c011dc0 con 0x55f99a614ec0 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.661+0000 7f1470cccb80 10 client.27310 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.661+0000 7f1470cccb80 1 -- 172.21.15.131:0/536477219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f99a4f9560 con 0x55f99a614ec0 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.661+0000 7f1433e00700 1 -- 172.21.15.131:0/536477219 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1440015940 con 0x55f99a614ec0 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.662+0000 7f1433e00700 1 -- 172.21.15.131:0/536477219 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f144c008030 con 0x55f99a614ec0 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.664+0000 7f1470cccb80 10 client.27310 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.664+0000 7f1470cccb80 10 client.27310 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.664+0000 7f1470cccb80 20 client.27310 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.664+0000 7f1470cccb80 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55f99a67ff90 0x55f99a6823b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:02.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.664+0000 7f1470cccb80 4 client.27310 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:44:02.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.664+0000 7f1470cccb80 1 -- 172.21.15.131:0/536477219 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f99a52bec0 con 0x55f99a67ff90 2024-05-10T05:44:02.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.667+0000 7f1451e00700 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55f99a67ff90 0x55f99a6823b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:02.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.674+0000 7f1451e00700 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55f99a67ff90 0x55f99a6823b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6faaa17997f59468 server_cookie=4b6b760aca78d27e in_seq=0 out_seq=0 2024-05-10T05:44:02.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.675+0000 7f1433e00700 10 client.27310 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:44:02.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.678+0000 7f1433e00700 1 -- 172.21.15.131:0/536477219 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f1440040020 con 0x55f99a67ff90 2024-05-10T05:44:02.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.678+0000 7f1433e00700 10 client.27310 handle_command_reply: tid=0 2024-05-10T05:44:02.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.678+0000 7f1470cccb80 1 client.27310 shutdown 2024-05-10T05:44:02.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.680+0000 7f1470cccb80 1 -- 172.21.15.131:0/536477219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 msgr2=0x55f99a67de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:02.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.680+0000 7f1470cccb80 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a67de40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f144c00abd0 tx=0x7f144c00af90 comp rx=0 tx=0).stop 2024-05-10T05:44:02.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.680+0000 7f1470cccb80 1 -- 172.21.15.131:0/536477219 shutdown_connections 2024-05-10T05:44:02.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.680+0000 7f1470cccb80 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55f99a67ff90 0x55f99a6823b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.680+0000 7f1433e00700 0 client.27310 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:44:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.680+0000 7f1470cccb80 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f99a6136a0 0x55f99a6228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.680+0000 7f1470cccb80 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f99a614ec0 0x55f99a67de40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.680+0000 7f1470cccb80 1 --2- 172.21.15.131:0/536477219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f99a614570 0x55f99a622e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.680+0000 7f1470cccb80 1 -- 172.21.15.131:0/536477219 >> 172.21.15.131:0/536477219 conn(0x55f99a575a60 msgr2=0x55f99a61b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.681+0000 7f1470cccb80 1 -- 172.21.15.131:0/536477219 shutdown_connections 2024-05-10T05:44:02.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.681+0000 7f1470cccb80 1 -- 172.21.15.131:0/536477219 wait complete. 2024-05-10T05:44:02.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.683+0000 7f1470cccb80 20 client.27310 trim_cache size 0 max 16384 2024-05-10T05:44:02.684 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:44:02.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.689+0000 7f1458e00700 1 -- 172.21.15.131:0/391679207 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14380a1130 msgr2=0x7f14380a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:02.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.689+0000 7f1458e00700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14380a1130 0x7f14380a3640 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1454108320 tx=0x7f1440009190 comp rx=0 tx=0).stop 2024-05-10T05:44:02.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.689+0000 7f1458e00700 1 -- 172.21.15.131:0/391679207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1454109300 msgr2=0x7f1454115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:02.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.689+0000 7f1458e00700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1454109300 0x7f1454115a80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1444003800 tx=0x7f1444003b70 comp rx=0 tx=0).stop 2024-05-10T05:44:02.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.690+0000 7f1458e00700 1 -- 172.21.15.131:0/391679207 shutdown_connections 2024-05-10T05:44:02.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.690+0000 7f1458e00700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f14380a1130 0x7f14380a3640 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.690+0000 7f1458e00700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1454108630 0x7f1454115540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.690+0000 7f1458e00700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1454109c50 0x7f14541169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.690+0000 7f1458e00700 1 --2- 172.21.15.131:0/391679207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1454109300 0x7f1454115a80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:02.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.690+0000 7f1458e00700 1 -- 172.21.15.131:0/391679207 >> 172.21.15.131:0/391679207 conn(0x7f145406ea00 msgr2=0x7f145406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:02.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.691+0000 7f1458e00700 1 -- 172.21.15.131:0/391679207 shutdown_connections 2024-05-10T05:44:02.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:02.691+0000 7f1458e00700 1 -- 172.21.15.131:0/391679207 wait complete. 2024-05-10T05:44:02.753 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:44:02.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:02 smithi131 ceph-mon[37264]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 253 MiB/s wr, 81 op/s 2024-05-10T05:44:03.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:02 smithi157 ceph-mon[78427]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 253 MiB/s wr, 81 op/s 2024-05-10T05:44:03.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:02 smithi142 ceph-mon[77667]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 253 MiB/s wr, 81 op/s 2024-05-10T05:44:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.454+0000 7ff0ec800700 1 -- 172.21.15.131:0/4077687306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e4073b40 msgr2=0x7ff0e4074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.454+0000 7ff0ec800700 1 --2- 172.21.15.131:0/4077687306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e4073b40 0x7ff0e4074010 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009990 tx=0x7ff0d4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.454+0000 7ff0ec800700 1 -- 172.21.15.131:0/4077687306 shutdown_connections 2024-05-10T05:44:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.454+0000 7ff0ec800700 1 --2- 172.21.15.131:0/4077687306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e410e220 0x7ff0e4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.454+0000 7ff0ec800700 1 --2- 172.21.15.131:0/4077687306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4075140 0x7ff0e4073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.454+0000 7ff0ec800700 1 --2- 172.21.15.131:0/4077687306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e4073b40 0x7ff0e4074010 secure :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009990 tx=0x7ff0d4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.454+0000 7ff0ec800700 1 -- 172.21.15.131:0/4077687306 >> 172.21.15.131:0/4077687306 conn(0x7ff0e40fd1d0 msgr2=0x7ff0e40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.455+0000 7ff0ec800700 1 -- 172.21.15.131:0/4077687306 shutdown_connections 2024-05-10T05:44:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.455+0000 7ff0ec800700 1 -- 172.21.15.131:0/4077687306 wait complete. 2024-05-10T05:44:03.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.458+0000 7ff0ec800700 1 Processor -- start 2024-05-10T05:44:03.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.459+0000 7ff0ec800700 1 -- start start 2024-05-10T05:44:03.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.460+0000 7ff0ec800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4075140 0x7ff0e406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.460+0000 7ff0ec800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e410e220 0x7ff0e40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.460+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4075140 0x7ff0e406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.460+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4075140 0x7ff0e406fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35172/0 (socket says 172.21.15.131:35172) 2024-05-10T05:44:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.460+0000 7ff0ec800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e4071490 0x7ff0e40718a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4071f80 con 0x7ff0e410e220 2024-05-10T05:44:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40720f0 con 0x7ff0e4075140 2024-05-10T05:44:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40722b0 con 0x7ff0e4071490 2024-05-10T05:44:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0e9e00700 1 -- 172.21.15.131:0/3518234835 learned_addr learned my addr 172.21.15.131:0/3518234835 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0e9400700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e410e220 0x7ff0e40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0ea800700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e4071490 0x7ff0e40718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0e9e00700 1 -- 172.21.15.131:0/3518234835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e4071490 msgr2=0x7ff0e40718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:03.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e4071490 0x7ff0e40718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0e9e00700 1 -- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e410e220 msgr2=0x7ff0e40705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:03.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e410e220 0x7ff0e40705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0e9e00700 1 -- 172.21.15.131:0/3518234835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d8009520 con 0x7ff0e4075140 2024-05-10T05:44:03.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.461+0000 7ff0e9400700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e410e220 0x7ff0e40705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:03.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.462+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4075140 0x7ff0e406fff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800ea30 tx=0x7ff0d800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:03.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.462+0000 7ff0dea00700 1 -- 172.21.15.131:0/3518234835 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8009820 con 0x7ff0e4075140 2024-05-10T05:44:03.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.462+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d40095f0 con 0x7ff0e4075140 2024-05-10T05:44:03.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.462+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e4072960 con 0x7ff0e4075140 2024-05-10T05:44:03.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.462+0000 7ff0dea00700 1 -- 172.21.15.131:0/3518234835 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0d80044e0 con 0x7ff0e4075140 2024-05-10T05:44:03.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.462+0000 7ff0dea00700 1 -- 172.21.15.131:0/3518234835 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8003e80 con 0x7ff0e4075140 2024-05-10T05:44:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.468+0000 7ff0dea00700 1 -- 172.21.15.131:0/3518234835 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff0d8010510 con 0x7ff0e4075140 2024-05-10T05:44:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.470+0000 7ff0dea00700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0cc0a1240 0x7ff0cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.470+0000 7ff1046f9b80 1 Processor -- start 2024-05-10T05:44:03.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.471+0000 7ff1046f9b80 1 -- start start 2024-05-10T05:44:03.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.471+0000 7ff0dea00700 1 -- 172.21.15.131:0/3518234835 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff0d8014070 con 0x7ff0e4075140 2024-05-10T05:44:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.471+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.471+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cf570 0x55d2693cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.472+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d2693d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.472+0000 7ff1046f9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2692f5150 con 0x55d2693cf570 2024-05-10T05:44:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.472+0000 7ff1046f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2692f2b10 con 0x55d2693cfec0 2024-05-10T05:44:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.472+0000 7ff1046f9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2692ee400 con 0x55d2693ce6a0 2024-05-10T05:44:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.472+0000 7ff0ea800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d2693d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.472+0000 7ff0ea800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d2693d67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35184/0 (socket says 172.21.15.131:35184) 2024-05-10T05:44:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.472+0000 7ff0ea800700 1 -- 172.21.15.131:0/2097310704 learned_addr learned my addr 172.21.15.131:0/2097310704 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:03.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.472+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/2097310704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.472+0000 7ff0ea800700 1 -- 172.21.15.131:0/2097310704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 msgr2=0x55d2693ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.473+0000 7ff0ea800700 1 --2- 172.21.15.131:0/2097310704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.473+0000 7ff0ea800700 1 -- 172.21.15.131:0/2097310704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cf570 msgr2=0x55d2693cf980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:44:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.473+0000 7ff0ea800700 1 --2- 172.21.15.131:0/2097310704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cf570 0x55d2693cf980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.473+0000 7ff0ea800700 1 -- 172.21.15.131:0/2097310704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d26916f3b0 con 0x55d2693cfec0 2024-05-10T05:44:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.473+0000 7ff0e9400700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0cc0a1240 0x7ff0cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.473+0000 7ff0ea800700 1 --2- 172.21.15.131:0/2097310704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d2693d67c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009dc0 tx=0x7ff0e00077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b54f02435701f92e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.473+0000 7ff0d3000700 1 -- 172.21.15.131:0/2097310704 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0014610 con 0x55d2693cfec0 2024-05-10T05:44:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.474+0000 7ff0d3000700 1 -- 172.21.15.131:0/2097310704 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0e000f050 con 0x55d2693cfec0 2024-05-10T05:44:03.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.474+0000 7ff0d3000700 1 -- 172.21.15.131:0/2097310704 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00139a0 con 0x55d2693cfec0 2024-05-10T05:44:03.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.475+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2097310704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 msgr2=0x55d2693d67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:03.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.475+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/2097310704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d2693d67c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009dc0 tx=0x7ff0e00077e0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.475+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2097310704 shutdown_connections 2024-05-10T05:44:03.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.475+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/2097310704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d2693d67c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.475+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/2097310704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.475+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/2097310704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cf570 0x55d2693cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.475+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2097310704 >> 172.21.15.131:0/2097310704 conn(0x55d269330a60 msgr2=0x55d2693c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:03.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.475+0000 7ff0e9400700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0cc0a1240 0x7ff0cc0a3750 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40098e0 tx=0x7ff0d400b750 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:03.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.476+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2097310704 shutdown_connections 2024-05-10T05:44:03.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.476+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2097310704 wait complete. 2024-05-10T05:44:03.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.477+0000 7ff1046f9b80 1 -- start start 2024-05-10T05:44:03.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.477+0000 7ff1046f9b80 1 Processor -- start 2024-05-10T05:44:03.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693d8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cf570 0x55d2693d8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d2693dfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff1046f9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26928a560 con 0x55d2693cf570 2024-05-10T05:44:03.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff1046f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26928a200 con 0x55d2693cfec0 2024-05-10T05:44:03.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff1046f9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2691716a0 con 0x55d2693ce6a0 2024-05-10T05:44:03.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff1046f9b80 10 client.0 _ll_register_callbacks cb 0x55d269293b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:03.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693d8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693d8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56614/0 (socket says 172.21.15.131:56614) 2024-05-10T05:44:03.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff0e9e00700 1 -- 172.21.15.131:0/4235258667 learned_addr learned my addr 172.21.15.131:0/4235258667 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:03.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.478+0000 7ff0ea800700 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d2693dfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.479+0000 7ff0e9400700 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cf570 0x55d2693d8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.479+0000 7ff0e9e00700 1 -- 172.21.15.131:0/4235258667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 msgr2=0x55d2693dfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:03.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.479+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d2693dfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.479+0000 7ff0e9e00700 1 -- 172.21.15.131:0/4235258667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cf570 msgr2=0x55d2693d8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:03.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.479+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cf570 0x55d2693d8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.479+0000 7ff0e9e00700 1 -- 172.21.15.131:0/4235258667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e0009a20 con 0x55d2693ce6a0 2024-05-10T05:44:03.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.480+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693d8740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8097bb0 tx=0x7ff0d8003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3afe547f32ee451 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:03.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.480+0000 7ff0c7400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:03.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.480+0000 7ff0c7400700 1 -- 172.21.15.131:0/4235258667 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d809a830 con 0x55d2693ce6a0 2024-05-10T05:44:03.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.480+0000 7ff0c7400700 1 -- 172.21.15.131:0/4235258667 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff0d809ef70 con 0x55d2693ce6a0 2024-05-10T05:44:03.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.480+0000 7ff1046f9b80 1 -- 172.21.15.131:0/4235258667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d269247270 con 0x55d2693ce6a0 2024-05-10T05:44:03.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.480+0000 7ff0c7400700 1 -- 172.21.15.131:0/4235258667 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0d809b3a0 con 0x55d2693ce6a0 2024-05-10T05:44:03.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.480+0000 7ff1046f9b80 10 client.27319 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:03.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.480+0000 7ff1046f9b80 1 -- 172.21.15.131:0/4235258667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d26911bb60 con 0x55d2693ce6a0 2024-05-10T05:44:03.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.480+0000 7ff0c7400700 1 -- 172.21.15.131:0/4235258667 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0d8083020 con 0x55d2693ce6a0 2024-05-10T05:44:03.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.481+0000 7ff0c7400700 1 -- 172.21.15.131:0/4235258667 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff0d807d020 con 0x55d2693ce6a0 2024-05-10T05:44:03.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.481+0000 7ff1046f9b80 10 client.27319 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:03.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.481+0000 7ff1046f9b80 10 client.27319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:03.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.481+0000 7ff1046f9b80 20 client.27319 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:03.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.481+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d2693e4470 0x55d2693e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.481+0000 7ff1046f9b80 4 client.27319 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:03.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.481+0000 7ff1046f9b80 1 -- 172.21.15.131:0/4235258667 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d26939d780 con 0x55d2693e4470 2024-05-10T05:44:03.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.481+0000 7ff0e9400700 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d2693e4470 0x55d2693e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.482+0000 7ff0e9400700 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d2693e4470 0x55d2693e68c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b83886bc2d617b0 server_cookie=da297bd44a5ef932 in_seq=0 out_seq=0 2024-05-10T05:44:03.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.482+0000 7ff0c7400700 10 client.27319 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:03.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.483+0000 7ff0c7400700 1 -- 172.21.15.131:0/4235258667 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d26939d780 con 0x55d2693e4470 2024-05-10T05:44:03.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.483+0000 7ff0c7400700 10 client.27319 handle_command_reply: tid=0 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.483+0000 7ff1046f9b80 1 client.27319 shutdown 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.483+0000 7ff1046f9b80 1 -- 172.21.15.131:0/4235258667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 msgr2=0x55d2693d8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.483+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693d8740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8097bb0 tx=0x7ff0d8003730 comp rx=0 tx=0).stop 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.484+0000 7ff1046f9b80 1 -- 172.21.15.131:0/4235258667 shutdown_connections 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.484+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d2693e4470 0x55d2693e68c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.484+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d2693dfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.484+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693d8740 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.484+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/4235258667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cf570 0x55d2693d8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.484+0000 7ff0c7400700 0 client.27319 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.484+0000 7ff1046f9b80 1 -- 172.21.15.131:0/4235258667 >> 172.21.15.131:0/4235258667 conn(0x55d269330a60 msgr2=0x55d2693c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:03.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.484+0000 7ff1046f9b80 1 -- 172.21.15.131:0/4235258667 shutdown_connections 2024-05-10T05:44:03.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.484+0000 7ff1046f9b80 1 -- 172.21.15.131:0/4235258667 wait complete. 2024-05-10T05:44:03.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.485+0000 7ff1046f9b80 20 client.27319 trim_cache size 0 max 16384 2024-05-10T05:44:03.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.515+0000 7ff1046f9b80 1 Processor -- start 2024-05-10T05:44:03.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.515+0000 7ff1046f9b80 1 -- start start 2024-05-10T05:44:03.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.516+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.516+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cf570 0x55d2693dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.516+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cfec0 0x55d2693d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.517+0000 7ff1046f9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2693e4170 con 0x55d2693cfec0 2024-05-10T05:44:03.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.517+0000 7ff1046f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2693d2ce0 con 0x55d2693cf570 2024-05-10T05:44:03.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.517+0000 7ff1046f9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2693d2ea0 con 0x55d2693ce6a0 2024-05-10T05:44:03.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.517+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.517+0000 7ff0ea800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cfec0 0x55d2693d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.517+0000 7ff0ea800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cfec0 0x55d2693d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59024/0 (socket says 172.21.15.131:59024) 2024-05-10T05:44:03.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.517+0000 7ff0ea800700 1 -- 172.21.15.131:0/1220107406 learned_addr learned my addr 172.21.15.131:0/1220107406 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:03.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.518+0000 7ff0e9400700 1 --2- 172.21.15.131:0/1220107406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cf570 0x55d2693dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.519+0000 7ff0e9e00700 1 -- 172.21.15.131:0/1220107406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cf570 msgr2=0x55d2693dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.519+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/1220107406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cf570 0x55d2693dfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.519+0000 7ff0e9e00700 1 -- 172.21.15.131:0/1220107406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cfec0 msgr2=0x55d2693d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.519+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/1220107406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cfec0 0x55d2693d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.519+0000 7ff0e9e00700 1 -- 172.21.15.131:0/1220107406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2692b4560 con 0x55d2693ce6a0 2024-05-10T05:44:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.519+0000 7ff0ea800700 1 --2- 172.21.15.131:0/1220107406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cfec0 0x55d2693d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.519+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/1220107406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693cf980 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80058f0 tx=0x7ff0d8005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1cdf4e1e9db84f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.519+0000 7ff0d3a00700 1 -- 172.21.15.131:0/1220107406 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d809b840 con 0x55d2693ce6a0 2024-05-10T05:44:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.520+0000 7ff0d3a00700 1 -- 172.21.15.131:0/1220107406 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0d8083020 con 0x55d2693ce6a0 2024-05-10T05:44:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.520+0000 7ff0d3a00700 1 -- 172.21.15.131:0/1220107406 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d809a630 con 0x55d2693ce6a0 2024-05-10T05:44:03.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.521+0000 7ff1046f9b80 1 -- 172.21.15.131:0/1220107406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 msgr2=0x55d2693cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:03.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.521+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/1220107406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693cf980 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80058f0 tx=0x7ff0d8005760 comp rx=0 tx=0).stop 2024-05-10T05:44:03.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.522+0000 7ff1046f9b80 1 -- 172.21.15.131:0/1220107406 shutdown_connections 2024-05-10T05:44:03.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.522+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/1220107406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693cfec0 0x55d2693d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.522+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/1220107406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693ce6a0 0x55d2693cf980 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.522+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/1220107406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cf570 0x55d2693dfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.522+0000 7ff1046f9b80 1 -- 172.21.15.131:0/1220107406 >> 172.21.15.131:0/1220107406 conn(0x55d269330a60 msgr2=0x55d2693c8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:03.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.522+0000 7ff1046f9b80 1 -- 172.21.15.131:0/1220107406 shutdown_connections 2024-05-10T05:44:03.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.522+0000 7ff1046f9b80 1 -- 172.21.15.131:0/1220107406 wait complete. 2024-05-10T05:44:03.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.523+0000 7ff1046f9b80 1 -- start start 2024-05-10T05:44:03.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.523+0000 7ff1046f9b80 1 Processor -- start 2024-05-10T05:44:03.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.524+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693ce6a0 0x55d2693dcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.524+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693cf570 0x55d2693dd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.524+0000 7ff1046f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d269438930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.525+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693ce6a0 0x55d2693dcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.525+0000 7ff0e9e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693ce6a0 0x55d2693dcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59026/0 (socket says 172.21.15.131:59026) 2024-05-10T05:44:03.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.525+0000 7ff0e9e00700 1 -- 172.21.15.131:0/2387927893 learned_addr learned my addr 172.21.15.131:0/2387927893 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.525+0000 7ff0ea800700 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d269438930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.525+0000 7ff1046f9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d269438e70 con 0x55d2693ce6a0 2024-05-10T05:44:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.526+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2387927893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d269438fb0 con 0x55d2693cfec0 2024-05-10T05:44:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.526+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2387927893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d269439140 con 0x55d2693cf570 2024-05-10T05:44:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.526+0000 7ff1046f9b80 10 client.0 _ll_register_callbacks cb 0x55d269293b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.527+0000 7ff0ea800700 1 -- 172.21.15.131:0/2387927893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693cf570 msgr2=0x55d2693dd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.527+0000 7ff0e9400700 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693cf570 0x55d2693dd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.527+0000 7ff0ea800700 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693cf570 0x55d2693dd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.527+0000 7ff0ea800700 1 -- 172.21.15.131:0/2387927893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693ce6a0 msgr2=0x55d2693dcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.527+0000 7ff0ea800700 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693ce6a0 0x55d2693dcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.527+0000 7ff0ea800700 1 -- 172.21.15.131:0/2387927893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d26916f3b0 con 0x55d2693cfec0 2024-05-10T05:44:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.527+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693ce6a0 0x55d2693dcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.527+0000 7ff0e9400700 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693cf570 0x55d2693dd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.527+0000 7ff0ea800700 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d269438930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009690 tx=0x7ff0e00052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c328859a262ed99 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.527+0000 7ff0c7400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:44:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.528+0000 7ff0c7400700 1 -- 172.21.15.131:0/2387927893 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0023040 con 0x55d2693cfec0 2024-05-10T05:44:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.528+0000 7ff0c7400700 1 -- 172.21.15.131:0/2387927893 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff0e0005910 con 0x55d2693cfec0 2024-05-10T05:44:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.529+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2387927893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0e000f050 con 0x55d2693cfec0 2024-05-10T05:44:03.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.529+0000 7ff0c7400700 1 -- 172.21.15.131:0/2387927893 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0e0028480 con 0x55d2693cfec0 2024-05-10T05:44:03.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.529+0000 7ff1046f9b80 10 client.27308 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:03.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.529+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2387927893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d26911bb60 con 0x55d2693cfec0 2024-05-10T05:44:03.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.530+0000 7ff0c7400700 1 -- 172.21.15.131:0/2387927893 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0e000f050 con 0x55d2693cfec0 2024-05-10T05:44:03.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.530+0000 7ff0c7400700 1 -- 172.21.15.131:0/2387927893 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff0e00146e0 con 0x55d2693cfec0 2024-05-10T05:44:03.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.530+0000 7ff1046f9b80 10 client.27308 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:03.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.530+0000 7ff1046f9b80 10 client.27308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:03.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.530+0000 7ff1046f9b80 20 client.27308 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:03.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.530+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d26943a940 0x55d26943cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:03.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.530+0000 7ff1046f9b80 4 client.27308 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:44:03.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.530+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2387927893 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d2692e6ec0 con 0x55d26943a940 2024-05-10T05:44:03.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.531+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d26943a940 0x55d26943cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:03.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.531+0000 7ff0e9e00700 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d26943a940 0x55d26943cd60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94a0b8ad351f05f server_cookie=9b2c82a4a1ba5bd in_seq=0 out_seq=0 2024-05-10T05:44:03.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.531+0000 7ff0c7400700 10 client.27308 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:03.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.532+0000 7ff0c7400700 1 -- 172.21.15.131:0/2387927893 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d2692e6ec0 con 0x55d26943a940 2024-05-10T05:44:03.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.532+0000 7ff0c7400700 10 client.27308 handle_command_reply: tid=0 2024-05-10T05:44:03.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.532+0000 7ff1046f9b80 1 client.27308 shutdown 2024-05-10T05:44:03.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.532+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2387927893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 msgr2=0x55d269438930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:03.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.532+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d269438930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009690 tx=0x7ff0e00052e0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.532+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2387927893 shutdown_connections 2024-05-10T05:44:03.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.532+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55d26943a940 0x55d26943cd60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.532+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2693ce6a0 0x55d2693dcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.533+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2693cfec0 0x55d269438930 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.533+0000 7ff1046f9b80 1 --2- 172.21.15.131:0/2387927893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2693cf570 0x55d2693dd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.533+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2387927893 >> 172.21.15.131:0/2387927893 conn(0x55d269330a60 msgr2=0x55d2693c8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:03.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.533+0000 7ff0c7400700 0 client.27308 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:03.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.533+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2387927893 shutdown_connections 2024-05-10T05:44:03.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.533+0000 7ff1046f9b80 1 -- 172.21.15.131:0/2387927893 wait complete. 2024-05-10T05:44:03.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.533+0000 7ff1046f9b80 20 client.27308 trim_cache size 0 max 16384 2024-05-10T05:44:03.535 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:44:03.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.534+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0cc0a1240 msgr2=0x7ff0cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:03.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.534+0000 7ff0ec800700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0cc0a1240 0x7ff0cc0a3750 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40098e0 tx=0x7ff0d400b750 comp rx=0 tx=0).stop 2024-05-10T05:44:03.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.534+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4075140 msgr2=0x7ff0e406fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:03.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.534+0000 7ff0ec800700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4075140 0x7ff0e406fff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800ea30 tx=0x7ff0d800edf0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.535+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 shutdown_connections 2024-05-10T05:44:03.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.535+0000 7ff0ec800700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff0cc0a1240 0x7ff0cc0a3750 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.535+0000 7ff0ec800700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0e4071490 0x7ff0e40718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.535+0000 7ff0ec800700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0e4075140 0x7ff0e406fff0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.535+0000 7ff0ec800700 1 --2- 172.21.15.131:0/3518234835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0e410e220 0x7ff0e40705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:03.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.535+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 >> 172.21.15.131:0/3518234835 conn(0x7ff0e40fd1d0 msgr2=0x7ff0e41015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:03.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.536+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 shutdown_connections 2024-05-10T05:44:03.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:03.538+0000 7ff0ec800700 1 -- 172.21.15.131:0/3518234835 wait complete. 2024-05-10T05:44:04.595 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:44:04.595 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:44:04.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:04 smithi131 ceph-mon[37264]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 224 MiB/s wr, 71 op/s 2024-05-10T05:44:05.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:04 smithi157 ceph-mon[78427]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 224 MiB/s wr, 71 op/s 2024-05-10T05:44:05.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:04 smithi142 ceph-mon[77667]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 224 MiB/s wr, 71 op/s 2024-05-10T05:44:05.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.314+0000 7f2f37000700 1 -- 172.21.15.131:0/1314529499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f3010a0a0 msgr2=0x7f2f301109a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:05.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.314+0000 7f2f37000700 1 --2- 172.21.15.131:0/1314529499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f3010a0a0 0x7f2f301109a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2f28007510 tx=0x7f2f28007820 comp rx=0 tx=0).stop 2024-05-10T05:44:05.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.314+0000 7f2f37000700 1 -- 172.21.15.131:0/1314529499 shutdown_connections 2024-05-10T05:44:05.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.314+0000 7f2f37000700 1 --2- 172.21.15.131:0/1314529499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f3010a0a0 0x7f2f301109a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.314+0000 7f2f37000700 1 --2- 172.21.15.131:0/1314529499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f301085e0 0x7f2f301089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.314+0000 7f2f37000700 1 --2- 172.21.15.131:0/1314529499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f30109750 0x7f2f30109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.314+0000 7f2f37000700 1 -- 172.21.15.131:0/1314529499 >> 172.21.15.131:0/1314529499 conn(0x7f2f3006ea00 msgr2=0x7f2f30070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:05.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.317+0000 7f2f37000700 1 -- 172.21.15.131:0/1314529499 shutdown_connections 2024-05-10T05:44:05.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.317+0000 7f2f37000700 1 -- 172.21.15.131:0/1314529499 wait complete. 2024-05-10T05:44:05.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.318+0000 7f2f37000700 1 Processor -- start 2024-05-10T05:44:05.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.320+0000 7f2f37000700 1 -- start start 2024-05-10T05:44:05.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.321+0000 7f2f37000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301085e0 0x7f2f3019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.322+0000 7f2f2fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301085e0 0x7f2f3019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.322+0000 7f2f2fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301085e0 0x7f2f3019e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59036/0 (socket says 172.21.15.131:59036) 2024-05-10T05:44:05.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.322+0000 7f2f37000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f30109750 0x7f2f3019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.322+0000 7f2f2fe00700 1 -- 172.21.15.131:0/1726365160 learned_addr learned my addr 172.21.15.131:0/1726365160 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.323+0000 7f2f37000700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f3010a0a0 0x7f2f301a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.323+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301b7020 con 0x7f2f301085e0 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.323+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301b7160 con 0x7f2f30109750 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.323+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301b72d0 con 0x7f2f3010a0a0 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.323+0000 7f2f27400700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f30109750 0x7f2f3019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.323+0000 7f2f35000700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f3010a0a0 0x7f2f301a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.323+0000 7f2f27400700 1 -- 172.21.15.131:0/1726365160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f3010a0a0 msgr2=0x7f2f301a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.323+0000 7f2f27400700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f3010a0a0 0x7f2f301a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.323+0000 7f2f27400700 1 -- 172.21.15.131:0/1726365160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301085e0 msgr2=0x7f2f3019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.324+0000 7f2f27400700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301085e0 0x7f2f3019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.324+0000 7f2f27400700 1 -- 172.21.15.131:0/1726365160 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f280071c0 con 0x7f2f30109750 2024-05-10T05:44:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.324+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301085e0 0x7f2f3019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.324+0000 7f2f27400700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f30109750 0x7f2f3019ea60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2f18006fd0 tx=0x7f2f1800b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.324+0000 7f2f2d600700 1 -- 172.21.15.131:0/1726365160 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18011830 con 0x7f2f30109750 2024-05-10T05:44:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.325+0000 7f2f2d600700 1 -- 172.21.15.131:0/1726365160 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f18011e50 con 0x7f2f30109750 2024-05-10T05:44:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.325+0000 7f2f2d600700 1 -- 172.21.15.131:0/1726365160 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18010e40 con 0x7f2f30109750 2024-05-10T05:44:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.325+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f301b75b0 con 0x7f2f30109750 2024-05-10T05:44:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.325+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f301b7a70 con 0x7f2f30109750 2024-05-10T05:44:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.330+0000 7f2f2d600700 1 -- 172.21.15.131:0/1726365160 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2f18011990 con 0x7f2f30109750 2024-05-10T05:44:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.335+0000 7f2f2d600700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2f140a1310 0x7f2f140a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.335+0000 7f2f2d600700 1 -- 172.21.15.131:0/1726365160 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2f180cb6c0 con 0x7f2f30109750 2024-05-10T05:44:05.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.336+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2f140a1310 0x7f2f140a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.337+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2f140a1310 0x7f2f140a3820 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009fd0 tx=0x7f2f2000b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:05.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.340+0000 7f2f4efc6b80 1 Processor -- start 2024-05-10T05:44:05.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.341+0000 7f2f4efc6b80 1 -- start start 2024-05-10T05:44:05.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.341+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 0x55fcff26fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.341+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff270a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.342+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 0x55fcff277840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.342+0000 7f2f4efc6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff1961d0 con 0x55fcff26f720 2024-05-10T05:44:05.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.342+0000 7f2f4efc6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff193b90 con 0x55fcff270f40 2024-05-10T05:44:05.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.342+0000 7f2f4efc6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff18f480 con 0x55fcff2705f0 2024-05-10T05:44:05.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.343+0000 7f2f27400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff270a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.343+0000 7f2f2fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 0x55fcff26fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.343+0000 7f2f27400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff270a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56644/0 (socket says 172.21.15.131:56644) 2024-05-10T05:44:05.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.343+0000 7f2f27400700 1 -- 172.21.15.131:0/838680406 learned_addr learned my addr 172.21.15.131:0/838680406 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:05.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.343+0000 7f2f35000700 1 --2- 172.21.15.131:0/838680406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 0x55fcff277840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.344+0000 7f2f27400700 1 -- 172.21.15.131:0/838680406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 msgr2=0x55fcff277840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:05.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.344+0000 7f2f27400700 1 --2- 172.21.15.131:0/838680406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 0x55fcff277840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.344+0000 7f2f27400700 1 -- 172.21.15.131:0/838680406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 msgr2=0x55fcff26fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:05.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.344+0000 7f2f27400700 1 --2- 172.21.15.131:0/838680406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 0x55fcff26fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.344+0000 7f2f27400700 1 -- 172.21.15.131:0/838680406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcff010430 con 0x55fcff2705f0 2024-05-10T05:44:05.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.344+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/838680406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 0x55fcff26fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:05.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.345+0000 7f2f27400700 1 --2- 172.21.15.131:0/838680406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff270a00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800bb30 tx=0x7f2f18098950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8769e78ed61d3122 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:05.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.345+0000 7f2f25600700 1 -- 172.21.15.131:0/838680406 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180991b0 con 0x55fcff2705f0 2024-05-10T05:44:05.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.345+0000 7f2f25600700 1 -- 172.21.15.131:0/838680406 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f18098ab0 con 0x55fcff2705f0 2024-05-10T05:44:05.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.347+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/838680406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 msgr2=0x55fcff270a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:05.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.347+0000 7f2f25600700 1 -- 172.21.15.131:0/838680406 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180991b0 con 0x55fcff2705f0 2024-05-10T05:44:05.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.347+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/838680406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff270a00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800bb30 tx=0x7f2f18098950 comp rx=0 tx=0).stop 2024-05-10T05:44:05.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.348+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/838680406 shutdown_connections 2024-05-10T05:44:05.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.348+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/838680406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 0x55fcff26fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.348+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/838680406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 0x55fcff277840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.348+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/838680406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff270a00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.348+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/838680406 >> 172.21.15.131:0/838680406 conn(0x55fcff1d1ae0 msgr2=0x55fcff26a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:05.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.349+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/838680406 shutdown_connections 2024-05-10T05:44:05.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.349+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/838680406 wait complete. 2024-05-10T05:44:05.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.350+0000 7f2f4efc6b80 1 -- start start 2024-05-10T05:44:05.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.350+0000 7f2f4efc6b80 1 Processor -- start 2024-05-10T05:44:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.351+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff26f720 0x55fcff279630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.351+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff2705f0 0x55fcff279b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.351+0000 7f2f27400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff2705f0 0x55fcff279b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.351+0000 7f2f27400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff2705f0 0x55fcff279b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59064/0 (socket says 172.21.15.131:59064) 2024-05-10T05:44:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.352+0000 7f2f27400700 1 -- 172.21.15.131:0/1275268954 learned_addr learned my addr 172.21.15.131:0/1275268954 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.352+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff270f40 0x55fcff280a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.352+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff12b5e0 con 0x55fcff2705f0 2024-05-10T05:44:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.352+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff12b280 con 0x55fcff26f720 2024-05-10T05:44:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.352+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff012720 con 0x55fcff270f40 2024-05-10T05:44:05.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.352+0000 7f2f4efc6b80 10 client.0 _ll_register_callbacks cb 0x55fcff134bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:05.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.353+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff26f720 0x55fcff279630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.353+0000 7f2f35000700 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff270f40 0x55fcff280a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.353+0000 7f2f2fe00700 1 -- 172.21.15.131:0/1275268954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff270f40 msgr2=0x55fcff280a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:05.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.353+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff270f40 0x55fcff280a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.353+0000 7f2f2fe00700 1 -- 172.21.15.131:0/1275268954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff2705f0 msgr2=0x55fcff279b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:05.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.354+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff2705f0 0x55fcff279b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.354+0000 7f2f2fe00700 1 -- 172.21.15.131:0/1275268954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2000e5f0 con 0x55fcff26f720 2024-05-10T05:44:05.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.354+0000 7f2f27400700 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff2705f0 0x55fcff279b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:05.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.355+0000 7f2f1e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:44:05.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.355+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff26f720 0x55fcff279630 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009120 tx=0x7f2f20009050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb0ad80b1db9cdc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:05.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.355+0000 7f2f1e000700 1 -- 172.21.15.131:0/1275268954 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20020040 con 0x55fcff26f720 2024-05-10T05:44:05.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.355+0000 7f2f1e000700 1 -- 172.21.15.131:0/1275268954 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2f2000ee40 con 0x55fcff26f720 2024-05-10T05:44:05.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.356+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcff0e82f0 con 0x55fcff26f720 2024-05-10T05:44:05.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.356+0000 7f2f1e000700 1 -- 172.21.15.131:0/1275268954 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f20020040 con 0x55fcff26f720 2024-05-10T05:44:05.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.356+0000 7f2f4efc6b80 10 client.27320 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:05.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.356+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f2f301b7a70 con 0x55fcff26f720 2024-05-10T05:44:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.356+0000 7f2f1e000700 1 -- 172.21.15.131:0/1275268954 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fcff0e82f0 con 0x55fcff26f720 2024-05-10T05:44:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.357+0000 7f2f1e000700 1 -- 172.21.15.131:0/1275268954 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2f20017880 con 0x55fcff26f720 2024-05-10T05:44:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.357+0000 7f2f4efc6b80 10 client.27320 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.357+0000 7f2f4efc6b80 10 client.27320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.357+0000 7f2f4efc6b80 20 client.27320 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.357+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fcff285100 0x55fcff287550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.357+0000 7f2f4efc6b80 4 client.27320 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:05.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.357+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcff23e800 con 0x55fcff285100 2024-05-10T05:44:05.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.358+0000 7f2f27400700 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fcff285100 0x55fcff287550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.358+0000 7f2f27400700 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fcff285100 0x55fcff287550 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d7e254a938d8f2e server_cookie=a4bdbad435753374 in_seq=0 out_seq=0 2024-05-10T05:44:05.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.358+0000 7f2f1e000700 10 client.27320 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:05.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.359+0000 7f2f1e000700 1 -- 172.21.15.131:0/1275268954 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fcff23e800 con 0x55fcff285100 2024-05-10T05:44:05.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.359+0000 7f2f1e000700 10 client.27320 handle_command_reply: tid=0 2024-05-10T05:44:05.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.359+0000 7f2f4efc6b80 1 client.27320 shutdown 2024-05-10T05:44:05.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.360+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff26f720 msgr2=0x55fcff279630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:05.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.360+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff26f720 0x55fcff279630 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009120 tx=0x7f2f20009050 comp rx=0 tx=0).stop 2024-05-10T05:44:05.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.360+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 shutdown_connections 2024-05-10T05:44:05.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.360+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fcff285100 0x55fcff287550 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.360+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff270f40 0x55fcff280a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.360+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff26f720 0x55fcff279630 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.360+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/1275268954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff2705f0 0x55fcff279b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.360+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 >> 172.21.15.131:0/1275268954 conn(0x55fcff1d1ae0 msgr2=0x55fcff26a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:05.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.361+0000 7f2f1e000700 0 client.27320 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:05.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.361+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 shutdown_connections 2024-05-10T05:44:05.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.361+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/1275268954 wait complete. 2024-05-10T05:44:05.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.362+0000 7f2f4efc6b80 20 client.27320 trim_cache size 0 max 16384 2024-05-10T05:44:05.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.404+0000 7f2f4efc6b80 1 Processor -- start 2024-05-10T05:44:05.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.406+0000 7f2f4efc6b80 1 -- start start 2024-05-10T05:44:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.407+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 0x55fcff277d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.407+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff2782c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.407+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 0x55fcff2809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.407+0000 7f2f4efc6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff277ae0 con 0x55fcff26f720 2024-05-10T05:44:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.407+0000 7f2f4efc6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff287520 con 0x55fcff270f40 2024-05-10T05:44:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.407+0000 7f2f4efc6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff287690 con 0x55fcff2705f0 2024-05-10T05:44:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.407+0000 7f2f35000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 0x55fcff2809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.407+0000 7f2f35000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 0x55fcff2809f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35276/0 (socket says 172.21.15.131:35276) 2024-05-10T05:44:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.407+0000 7f2f35000700 1 -- 172.21.15.131:0/679549274 learned_addr learned my addr 172.21.15.131:0/679549274 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:05.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.408+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/679549274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 0x55fcff277d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.408+0000 7f2f35000700 1 -- 172.21.15.131:0/679549274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 msgr2=0x55fcff2782c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:44:05.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.408+0000 7f2f35000700 1 --2- 172.21.15.131:0/679549274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff2782c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.408+0000 7f2f35000700 1 -- 172.21.15.131:0/679549274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 msgr2=0x55fcff277d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:05.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.408+0000 7f2f35000700 1 --2- 172.21.15.131:0/679549274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 0x55fcff277d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.408+0000 7f2f35000700 1 -- 172.21.15.131:0/679549274 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcfefbcbe0 con 0x55fcff270f40 2024-05-10T05:44:05.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.410+0000 7f2f35000700 1 --2- 172.21.15.131:0/679549274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 0x55fcff2809f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2f28007a30 tx=0x7f2f28007a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757a533f8df74f9a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:05.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.411+0000 7f2f26000700 1 -- 172.21.15.131:0/679549274 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800f040 con 0x55fcff270f40 2024-05-10T05:44:05.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.411+0000 7f2f26000700 1 -- 172.21.15.131:0/679549274 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f28007e30 con 0x55fcff270f40 2024-05-10T05:44:05.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.411+0000 7f2f26000700 1 -- 172.21.15.131:0/679549274 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800c5c0 con 0x55fcff270f40 2024-05-10T05:44:05.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.420+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/679549274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 msgr2=0x55fcff2809f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:05.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.420+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/679549274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 0x55fcff2809f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2f28007a30 tx=0x7f2f28007a60 comp rx=0 tx=0).stop 2024-05-10T05:44:05.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.422+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/679549274 shutdown_connections 2024-05-10T05:44:05.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.422+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/679549274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff26f720 0x55fcff277d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.422+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/679549274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff270f40 0x55fcff2809f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.422+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/679549274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff2782c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.422+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/679549274 >> 172.21.15.131:0/679549274 conn(0x55fcff1d1ae0 msgr2=0x55fcff266200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:05.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.430+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/679549274 shutdown_connections 2024-05-10T05:44:05.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.430+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/679549274 wait complete. 2024-05-10T05:44:05.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.434+0000 7f2f4efc6b80 1 -- start start 2024-05-10T05:44:05.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.435+0000 7f2f4efc6b80 1 Processor -- start 2024-05-10T05:44:05.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.444+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff26f720 0x55fcff2803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.444+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff280930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.444+0000 7f2f4efc6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff270f40 0x55fcff27eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.444+0000 7f2f27400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff280930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.444+0000 7f2f27400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff280930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56670/0 (socket says 172.21.15.131:56670) 2024-05-10T05:44:05.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.445+0000 7f2f35000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff270f40 0x55fcff27eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.445+0000 7f2f35000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff270f40 0x55fcff27eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59088/0 (socket says 172.21.15.131:59088) 2024-05-10T05:44:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.446+0000 7f2f27400700 1 -- 172.21.15.131:0/2851263473 learned_addr learned my addr 172.21.15.131:0/2851263473 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.446+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff27f170 con 0x55fcff270f40 2024-05-10T05:44:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.446+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff27f2e0 con 0x55fcff26f720 2024-05-10T05:44:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.446+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcff27f4a0 con 0x55fcff2705f0 2024-05-10T05:44:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.446+0000 7f2f4efc6b80 10 client.0 _ll_register_callbacks cb 0x55fcff134bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.446+0000 7f2f27400700 1 -- 172.21.15.131:0/2851263473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff26f720 msgr2=0x55fcff2803f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:44:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.446+0000 7f2f27400700 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff26f720 0x55fcff2803f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.446+0000 7f2f27400700 1 -- 172.21.15.131:0/2851263473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff270f40 msgr2=0x55fcff27eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:05.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.446+0000 7f2f27400700 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff270f40 0x55fcff27eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.446+0000 7f2f27400700 1 -- 172.21.15.131:0/2851263473 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f30111460 con 0x55fcff2705f0 2024-05-10T05:44:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.447+0000 7f2f27400700 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff280930 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2f180c94d0 tx=0x7f2f180108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6975000f5b10fe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.447+0000 7f2f1e000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.447+0000 7f2f1e000700 1 -- 172.21.15.131:0/2851263473 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180b1410 con 0x55fcff2705f0 2024-05-10T05:44:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.447+0000 7f2f1e000700 1 -- 172.21.15.131:0/2851263473 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2f180b6070 con 0x55fcff2705f0 2024-05-10T05:44:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.448+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f18098ab0 con 0x55fcff2705f0 2024-05-10T05:44:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.448+0000 7f2f1e000700 1 -- 172.21.15.131:0/2851263473 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f18096e50 con 0x55fcff2705f0 2024-05-10T05:44:05.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.462+0000 7f2f4efc6b80 10 client.27334 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:05.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.462+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fcff1555e0 con 0x55fcff2705f0 2024-05-10T05:44:05.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.464+0000 7f2f1e000700 1 -- 172.21.15.131:0/2851263473 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f18098ab0 con 0x55fcff2705f0 2024-05-10T05:44:05.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.464+0000 7f2f1e000700 1 -- 172.21.15.131:0/2851263473 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2f18077be0 con 0x55fcff2705f0 2024-05-10T05:44:05.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.467+0000 7f2f4efc6b80 10 client.27334 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:05.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.467+0000 7f2f4efc6b80 10 client.27334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.467+0000 7f2f4efc6b80 20 client.27334 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.467+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fcff2dc680 0x55fcff2deaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.467+0000 7f2f4efc6b80 4 client.27334 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:44:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.467+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fcff187f40 con 0x55fcff2dc680 2024-05-10T05:44:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.467+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fcff2dc680 0x55fcff2deaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.468+0000 7f2f2fe00700 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fcff2dc680 0x55fcff2deaa0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf185e2755500e8 server_cookie=6b3f32eed756bc57 in_seq=0 out_seq=0 2024-05-10T05:44:05.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.468+0000 7f2f1e000700 10 client.27334 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:05.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.469+0000 7f2f1e000700 1 -- 172.21.15.131:0/2851263473 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fcff187f40 con 0x55fcff2dc680 2024-05-10T05:44:05.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.471+0000 7f2f1e000700 10 client.27334 handle_command_reply: tid=0 2024-05-10T05:44:05.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.471+0000 7f2f4efc6b80 1 client.27334 shutdown 2024-05-10T05:44:05.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.473+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 msgr2=0x55fcff280930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:05.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.473+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff280930 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2f180c94d0 tx=0x7f2f180108e0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.475+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 shutdown_connections 2024-05-10T05:44:05.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.475+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55fcff2dc680 0x55fcff2deaa0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.475+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fcff270f40 0x55fcff27eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.475+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcff26f720 0x55fcff2803f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.475+0000 7f2f4efc6b80 1 --2- 172.21.15.131:0/2851263473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fcff2705f0 0x55fcff280930 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.475+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 >> 172.21.15.131:0/2851263473 conn(0x55fcff1d1ae0 msgr2=0x55fcff266200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:05.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.476+0000 7f2f1e000700 0 client.27334 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:05.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.476+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 shutdown_connections 2024-05-10T05:44:05.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.476+0000 7f2f4efc6b80 1 -- 172.21.15.131:0/2851263473 wait complete. 2024-05-10T05:44:05.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.478+0000 7f2f4efc6b80 20 client.27334 trim_cache size 0 max 16384 2024-05-10T05:44:05.478 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:44:05.479 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:44:05.479 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "bc55f0f5-e3e4-4418-914e-d7165cbd75b2", 2024-05-10T05:44:05.479 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:44:05.479 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:44:05.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.479+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2f140a1310 msgr2=0x7f2f140a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:05.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.479+0000 7f2f37000700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2f140a1310 0x7f2f140a3820 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009fd0 tx=0x7f2f2000b040 comp rx=0 tx=0).stop 2024-05-10T05:44:05.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.479+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f30109750 msgr2=0x7f2f3019ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:05.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.479+0000 7f2f37000700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f30109750 0x7f2f3019ea60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2f18006fd0 tx=0x7f2f1800b730 comp rx=0 tx=0).stop 2024-05-10T05:44:05.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.493+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 shutdown_connections 2024-05-10T05:44:05.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.493+0000 7f2f37000700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2f140a1310 0x7f2f140a3820 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.493+0000 7f2f37000700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f301085e0 0x7f2f3019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.493+0000 7f2f37000700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f3010a0a0 0x7f2f301a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.493+0000 7f2f37000700 1 --2- 172.21.15.131:0/1726365160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f30109750 0x7f2f3019ea60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:05.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.493+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 >> 172.21.15.131:0/1726365160 conn(0x7f2f3006ea00 msgr2=0x7f2f3010c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:05.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.501+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 shutdown_connections 2024-05-10T05:44:05.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:05.504+0000 7f2f37000700 1 -- 172.21.15.131:0/1726365160 wait complete. 2024-05-10T05:44:05.650 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:44:06.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.258+0000 7f7369400700 1 -- 172.21.15.131:0/4252270328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73641081b0 msgr2=0x7f73641085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:06.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.258+0000 7f7369400700 1 --2- 172.21.15.131:0/4252270328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73641081b0 0x7f73641085c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f735c00b3e0 tx=0x7f735c00b6f0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.265+0000 7f7369400700 1 -- 172.21.15.131:0/4252270328 shutdown_connections 2024-05-10T05:44:06.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.266+0000 7f7369400700 1 --2- 172.21.15.131:0/4252270328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73641081b0 0x7f73641085c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.266+0000 7f7369400700 1 --2- 172.21.15.131:0/4252270328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7364109c70 0x7f7364110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.266+0000 7f7369400700 1 --2- 172.21.15.131:0/4252270328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73641092d0 0x7f73641096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.267+0000 7f7369400700 1 -- 172.21.15.131:0/4252270328 >> 172.21.15.131:0/4252270328 conn(0x7f73640771f0 msgr2=0x7f7364106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:06.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.273+0000 7f7369400700 1 -- 172.21.15.131:0/4252270328 shutdown_connections 2024-05-10T05:44:06.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.273+0000 7f7369400700 1 -- 172.21.15.131:0/4252270328 wait complete. 2024-05-10T05:44:06.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.279+0000 7f7369400700 1 Processor -- start 2024-05-10T05:44:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.280+0000 7f7369400700 1 -- start start 2024-05-10T05:44:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.284+0000 7f7369400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73641081b0 0x7f736419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.286+0000 7f7369400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73641092d0 0x7f736419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.286+0000 7f7362800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73641081b0 0x7f736419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.286+0000 7f7362800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73641081b0 0x7f736419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35302/0 (socket says 172.21.15.131:35302) 2024-05-10T05:44:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.287+0000 7f7361e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73641092d0 0x7f736419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.287+0000 7f7361e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73641092d0 0x7f736419eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59110/0 (socket says 172.21.15.131:59110) 2024-05-10T05:44:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.287+0000 7f7362800700 1 -- 172.21.15.131:0/2633670377 learned_addr learned my addr 172.21.15.131:0/2633670377 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.287+0000 7f7369400700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7364109c70 0x7f73641a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.292+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73641a6200 con 0x7f73641092d0 2024-05-10T05:44:06.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.292+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73641a6340 con 0x7f73641081b0 2024-05-10T05:44:06.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.292+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73641a64d0 con 0x7f7364109c70 2024-05-10T05:44:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.288+0000 7f7363200700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7364109c70 0x7f73641a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.295+0000 7f7361e00700 1 -- 172.21.15.131:0/2633670377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7364109c70 msgr2=0x7f73641a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.295+0000 7f7361e00700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7364109c70 0x7f73641a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.295+0000 7f7361e00700 1 -- 172.21.15.131:0/2633670377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73641081b0 msgr2=0x7f736419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.295+0000 7f7361e00700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73641081b0 0x7f736419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.295+0000 7f7361e00700 1 -- 172.21.15.131:0/2633670377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c009c40 con 0x7f73641092d0 2024-05-10T05:44:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.296+0000 7f7361e00700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73641092d0 0x7f736419eca0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f735000d9f0 tx=0x7f735000dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.297+0000 7f735b400700 1 -- 172.21.15.131:0/2633670377 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350009930 con 0x7f73641092d0 2024-05-10T05:44:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.300+0000 7f735b400700 1 -- 172.21.15.131:0/2633670377 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f735000de90 con 0x7f73641092d0 2024-05-10T05:44:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.300+0000 7f735b400700 1 -- 172.21.15.131:0/2633670377 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735000f3c0 con 0x7f73641092d0 2024-05-10T05:44:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.303+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73641a67b0 con 0x7f73641092d0 2024-05-10T05:44:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.303+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73641a6c40 con 0x7f73641092d0 2024-05-10T05:44:06.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.341+0000 7f735b400700 1 -- 172.21.15.131:0/2633670377 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7350010640 con 0x7f73641092d0 2024-05-10T05:44:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.356+0000 7f735b400700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73480a1290 0x7f73480a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.357+0000 7f735b400700 1 -- 172.21.15.131:0/2633670377 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f73500cbed0 con 0x7f73641092d0 2024-05-10T05:44:06.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.358+0000 7f7362800700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73480a1290 0x7f73480a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.358+0000 7f73812c3b80 1 Processor -- start 2024-05-10T05:44:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.362+0000 7f73812c3b80 1 -- start start 2024-05-10T05:44:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.362+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d98586a0 0x55a2d9858af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.362+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859570 0x55a2d9859980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.362+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859ec0 0x55a2d98607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.362+0000 7f73812c3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d977f150 con 0x55a2d9859570 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.362+0000 7f73812c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d977cb10 con 0x55a2d98586a0 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.363+0000 7f73812c3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d9778400 con 0x55a2d9859ec0 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.363+0000 7f7363200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859ec0 0x55a2d98607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.364+0000 7f7363200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859ec0 0x55a2d98607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56702/0 (socket says 172.21.15.131:56702) 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.364+0000 7f7363200700 1 -- 172.21.15.131:0/2621992900 learned_addr learned my addr 172.21.15.131:0/2621992900 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.364+0000 7f7363200700 1 -- 172.21.15.131:0/2621992900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d98586a0 msgr2=0x55a2d9858af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.364+0000 7f7363200700 1 --2- 172.21.15.131:0/2621992900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d98586a0 0x55a2d9858af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.364+0000 7f7363200700 1 -- 172.21.15.131:0/2621992900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859570 msgr2=0x55a2d9859980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.364+0000 7f7363200700 1 --2- 172.21.15.131:0/2621992900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859570 0x55a2d9859980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.364+0000 7f7363200700 1 -- 172.21.15.131:0/2621992900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2d95f93b0 con 0x55a2d9859ec0 2024-05-10T05:44:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.364+0000 7f7363200700 1 --2- 172.21.15.131:0/2621992900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859ec0 0x55a2d98607c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7354009a70 tx=0x7f7354009d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=713b8c4433ff140c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.365+0000 7f7362800700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73480a1290 0x7f73480a37a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f735c009840 tx=0x7f735c00bb30 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.366+0000 7f734f400700 1 -- 172.21.15.131:0/2621992900 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735400e8d0 con 0x55a2d9859ec0 2024-05-10T05:44:06.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.370+0000 7f734f400700 1 -- 172.21.15.131:0/2621992900 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7354013470 con 0x55a2d9859ec0 2024-05-10T05:44:06.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.370+0000 7f734f400700 1 -- 172.21.15.131:0/2621992900 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354012630 con 0x55a2d9859ec0 2024-05-10T05:44:06.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.377+0000 7f73812c3b80 1 -- 172.21.15.131:0/2621992900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859ec0 msgr2=0x55a2d98607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:06.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.377+0000 7f73812c3b80 1 --2- 172.21.15.131:0/2621992900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859ec0 0x55a2d98607c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7354009a70 tx=0x7f7354009d80 comp rx=0 tx=0).stop 2024-05-10T05:44:06.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.377+0000 7f73812c3b80 1 -- 172.21.15.131:0/2621992900 shutdown_connections 2024-05-10T05:44:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.377+0000 7f73812c3b80 1 --2- 172.21.15.131:0/2621992900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859ec0 0x55a2d98607c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.377+0000 7f73812c3b80 1 --2- 172.21.15.131:0/2621992900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d98586a0 0x55a2d9858af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.377+0000 7f73812c3b80 1 --2- 172.21.15.131:0/2621992900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859570 0x55a2d9859980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.377+0000 7f73812c3b80 1 -- 172.21.15.131:0/2621992900 >> 172.21.15.131:0/2621992900 conn(0x55a2d97baa60 msgr2=0x55a2d9852fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.378+0000 7f73812c3b80 1 -- 172.21.15.131:0/2621992900 shutdown_connections 2024-05-10T05:44:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.378+0000 7f73812c3b80 1 -- 172.21.15.131:0/2621992900 wait complete. 2024-05-10T05:44:06.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.379+0000 7f73812c3b80 1 -- start start 2024-05-10T05:44:06.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.380+0000 7f73812c3b80 1 Processor -- start 2024-05-10T05:44:06.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.382+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d98735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.383+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9873b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.383+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859ec0 0x55a2d987aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.383+0000 7f7362800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d98735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.383+0000 7f7362800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d98735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56704/0 (socket says 172.21.15.131:56704) 2024-05-10T05:44:06.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.383+0000 7f7362800700 1 -- 172.21.15.131:0/3620509248 learned_addr learned my addr 172.21.15.131:0/3620509248 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:06.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.383+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d9714560 con 0x55a2d9859ec0 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.383+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d9714200 con 0x55a2d9859570 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.383+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d95fb6a0 con 0x55a2d98586a0 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.383+0000 7f73812c3b80 10 client.0 _ll_register_callbacks cb 0x55a2d971db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.383+0000 7f7361e00700 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9873b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.384+0000 7f7361e00700 1 -- 172.21.15.131:0/3620509248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 msgr2=0x55a2d98735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.384+0000 7f7361e00700 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d98735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.384+0000 7f7361e00700 1 -- 172.21.15.131:0/3620509248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859ec0 msgr2=0x55a2d987aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.384+0000 7f7361e00700 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859ec0 0x55a2d987aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.384+0000 7f7361e00700 1 -- 172.21.15.131:0/3620509248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73540096d0 con 0x55a2d9859570 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.388+0000 7f7361e00700 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9873b10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f73500c9970 tx=0x7f73500104e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcffdeb290eec7cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.390+0000 7f7343e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:44:06.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.392+0000 7f7343e00700 1 -- 172.21.15.131:0/3620509248 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350099350 con 0x55a2d9859570 2024-05-10T05:44:06.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.392+0000 7f7343e00700 1 -- 172.21.15.131:0/3620509248 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f73500ae070 con 0x55a2d9859570 2024-05-10T05:44:06.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.394+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2d96d1270 con 0x55a2d9859570 2024-05-10T05:44:06.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.396+0000 7f7343e00700 1 -- 172.21.15.131:0/3620509248 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f735009d180 con 0x55a2d9859570 2024-05-10T05:44:06.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.396+0000 7f73812c3b80 10 client.27332 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:06.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.396+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a2d95a5b60 con 0x55a2d9859570 2024-05-10T05:44:06.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.401+0000 7f7343e00700 1 -- 172.21.15.131:0/3620509248 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a2d96d1270 con 0x55a2d9859570 2024-05-10T05:44:06.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.401+0000 7f7343e00700 1 -- 172.21.15.131:0/3620509248 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f73500a12b0 con 0x55a2d9859570 2024-05-10T05:44:06.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.401+0000 7f73812c3b80 10 client.27332 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:06.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.401+0000 7f73812c3b80 10 client.27332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:06.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.401+0000 7f73812c3b80 20 client.27332 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:06.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.401+0000 7f73812c3b80 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55a2d987f020 0x55a2d9881470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.404+0000 7f73812c3b80 4 client.27332 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:06.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.404+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2d9827780 con 0x55a2d987f020 2024-05-10T05:44:06.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.406+0000 7f7362800700 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55a2d987f020 0x55a2d9881470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.408+0000 7f7362800700 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55a2d987f020 0x55a2d9881470 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8371d464be9924ec server_cookie=2a783c17cd821427 in_seq=0 out_seq=0 2024-05-10T05:44:06.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.409+0000 7f7343e00700 10 client.27332 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:06.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.409+0000 7f7343e00700 1 -- 172.21.15.131:0/3620509248 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a2d9827780 con 0x55a2d987f020 2024-05-10T05:44:06.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.409+0000 7f7343e00700 10 client.27332 handle_command_reply: tid=0 2024-05-10T05:44:06.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.410+0000 7f73812c3b80 1 client.27332 shutdown 2024-05-10T05:44:06.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.413+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 msgr2=0x55a2d9873b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:06.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.413+0000 7f73812c3b80 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9873b10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f73500c9970 tx=0x7f73500104e0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.413+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 shutdown_connections 2024-05-10T05:44:06.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.415+0000 7f73812c3b80 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55a2d987f020 0x55a2d9881470 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.416+0000 7f7343e00700 0 client.27332 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:06.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.417+0000 7f73812c3b80 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859ec0 0x55a2d987aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.417+0000 7f73812c3b80 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d98735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.417+0000 7f73812c3b80 1 --2- 172.21.15.131:0/3620509248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9873b10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.417+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 >> 172.21.15.131:0/3620509248 conn(0x55a2d97baa60 msgr2=0x55a2d9852fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:06.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.419+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 shutdown_connections 2024-05-10T05:44:06.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.419+0000 7f73812c3b80 1 -- 172.21.15.131:0/3620509248 wait complete. 2024-05-10T05:44:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.421+0000 7f73812c3b80 20 client.27332 trim_cache size 0 max 16384 2024-05-10T05:44:06.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.458+0000 7f73812c3b80 1 Processor -- start 2024-05-10T05:44:06.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.458+0000 7f73812c3b80 1 -- start start 2024-05-10T05:44:06.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.459+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d98586a0 0x55a2d9881470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.459+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859570 0x55a2d987aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.460+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859ec0 0x55a2d9872b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.460+0000 7f73812c3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d9860530 con 0x55a2d98586a0 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.460+0000 7f73812c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d9854770 con 0x55a2d9859ec0 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.460+0000 7f73812c3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d9858b90 con 0x55a2d9859570 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.460+0000 7f7361e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859570 0x55a2d987aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.460+0000 7f7361e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859570 0x55a2d987aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56714/0 (socket says 172.21.15.131:56714) 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.460+0000 7f7361e00700 1 -- 172.21.15.131:0/3293120894 learned_addr learned my addr 172.21.15.131:0/3293120894 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.461+0000 7f7361e00700 1 -- 172.21.15.131:0/3293120894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859ec0 msgr2=0x55a2d9872b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.461+0000 7f7361e00700 1 --2- 172.21.15.131:0/3293120894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859ec0 0x55a2d9872b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.461+0000 7f7361e00700 1 -- 172.21.15.131:0/3293120894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d98586a0 msgr2=0x55a2d9881470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.461+0000 7f7361e00700 1 --2- 172.21.15.131:0/3293120894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d98586a0 0x55a2d9881470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.461+0000 7f7361e00700 1 -- 172.21.15.131:0/3293120894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2d973e560 con 0x55a2d9859570 2024-05-10T05:44:06.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.461+0000 7f7361e00700 1 --2- 172.21.15.131:0/3293120894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859570 0x55a2d987aa30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7350095850 tx=0x7f7350010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3139ecb85ee18ca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:06.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.461+0000 7f734fe00700 1 -- 172.21.15.131:0/3293120894 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735009d180 con 0x55a2d9859570 2024-05-10T05:44:06.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.461+0000 7f734fe00700 1 -- 172.21.15.131:0/3293120894 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7350082cc0 con 0x55a2d9859570 2024-05-10T05:44:06.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.461+0000 7f734fe00700 1 -- 172.21.15.131:0/3293120894 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350095140 con 0x55a2d9859570 2024-05-10T05:44:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.467+0000 7f73812c3b80 1 -- 172.21.15.131:0/3293120894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859570 msgr2=0x55a2d987aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.467+0000 7f73812c3b80 1 --2- 172.21.15.131:0/3293120894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859570 0x55a2d987aa30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7350095850 tx=0x7f7350010a40 comp rx=0 tx=0).stop 2024-05-10T05:44:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.467+0000 7f73812c3b80 1 -- 172.21.15.131:0/3293120894 shutdown_connections 2024-05-10T05:44:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.467+0000 7f73812c3b80 1 --2- 172.21.15.131:0/3293120894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d98586a0 0x55a2d9881470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.467+0000 7f73812c3b80 1 --2- 172.21.15.131:0/3293120894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859ec0 0x55a2d9872b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.467+0000 7f73812c3b80 1 --2- 172.21.15.131:0/3293120894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d9859570 0x55a2d987aa30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.468+0000 7f73812c3b80 1 -- 172.21.15.131:0/3293120894 >> 172.21.15.131:0/3293120894 conn(0x55a2d97baa60 msgr2=0x55a2d985c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:06.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.471+0000 7f73812c3b80 1 -- 172.21.15.131:0/3293120894 shutdown_connections 2024-05-10T05:44:06.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.471+0000 7f73812c3b80 1 -- 172.21.15.131:0/3293120894 wait complete. 2024-05-10T05:44:06.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.471+0000 7f73812c3b80 1 -- start start 2024-05-10T05:44:06.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.472+0000 7f73812c3b80 1 Processor -- start 2024-05-10T05:44:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.476+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d9878000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.476+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9878540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.477+0000 7f7362800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d9878000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.477+0000 7f7362800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d9878000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56720/0 (socket says 172.21.15.131:56720) 2024-05-10T05:44:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.477+0000 7f73812c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859ec0 0x55a2d9879330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.477+0000 7f7361e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9878540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.477+0000 7f7361e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9878540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35316/0 (socket says 172.21.15.131:35316) 2024-05-10T05:44:06.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.477+0000 7f7361e00700 1 -- 172.21.15.131:0/2351791973 learned_addr learned my addr 172.21.15.131:0/2351791973 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:06.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.477+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d9879870 con 0x55a2d9859ec0 2024-05-10T05:44:06.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.477+0000 7f7363200700 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859ec0 0x55a2d9879330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.478+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d98799e0 con 0x55a2d9859570 2024-05-10T05:44:06.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.478+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2d9879ba0 con 0x55a2d98586a0 2024-05-10T05:44:06.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.479+0000 7f73812c3b80 10 client.0 _ll_register_callbacks cb 0x55a2d971db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:06.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.479+0000 7f7361e00700 1 -- 172.21.15.131:0/2351791973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 msgr2=0x55a2d9878000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:06.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.479+0000 7f7361e00700 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d9878000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.479+0000 7f7361e00700 1 -- 172.21.15.131:0/2351791973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859ec0 msgr2=0x55a2d9879330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:06.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.480+0000 7f7361e00700 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859ec0 0x55a2d9879330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.480+0000 7f7361e00700 1 -- 172.21.15.131:0/2351791973 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2d95a5b60 con 0x55a2d9859570 2024-05-10T05:44:06.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.480+0000 7f7362800700 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d9878000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:06.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.480+0000 7f7363200700 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859ec0 0x55a2d9879330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:06.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.481+0000 7f7343e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:44:06.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.481+0000 7f7361e00700 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9878540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f73500a15e0 tx=0x7f7350095370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9525e60e81b05039 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:06.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.481+0000 7f7343e00700 1 -- 172.21.15.131:0/2351791973 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735009d180 con 0x55a2d9859570 2024-05-10T05:44:06.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.481+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7354013470 con 0x55a2d9859570 2024-05-10T05:44:06.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.481+0000 7f7343e00700 1 -- 172.21.15.131:0/2351791973 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f73500ae070 con 0x55a2d9859570 2024-05-10T05:44:06.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.482+0000 7f7343e00700 1 -- 172.21.15.131:0/2351791973 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f735009d180 con 0x55a2d9859570 2024-05-10T05:44:06.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.482+0000 7f73812c3b80 10 client.27335 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:06.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.482+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a2d95fdcd0 con 0x55a2d9859570 2024-05-10T05:44:06.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.483+0000 7f7343e00700 1 -- 172.21.15.131:0/2351791973 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73500a5020 con 0x55a2d9859570 2024-05-10T05:44:06.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.483+0000 7f7343e00700 1 -- 172.21.15.131:0/2351791973 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f73500b72a0 con 0x55a2d9859570 2024-05-10T05:44:06.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.484+0000 7f73812c3b80 10 client.27335 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:06.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.484+0000 7f73812c3b80 10 client.27335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:06.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.484+0000 7f73812c3b80 20 client.27335 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:06.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.484+0000 7f73812c3b80 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55a2d987f020 0x55a2d98d7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:06.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.484+0000 7f73812c3b80 4 client.27335 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:44:06.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.484+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a2d9770ec0 con 0x55a2d987f020 2024-05-10T05:44:06.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.485+0000 7f7362800700 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55a2d987f020 0x55a2d98d7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:06.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.485+0000 7f7362800700 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55a2d987f020 0x55a2d98d7500 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfdce9ad4172cac5 server_cookie=de6051639180a44 in_seq=0 out_seq=0 2024-05-10T05:44:06.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.485+0000 7f7343e00700 10 client.27335 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:06.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.485+0000 7f7343e00700 1 -- 172.21.15.131:0/2351791973 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a2d9770ec0 con 0x55a2d987f020 2024-05-10T05:44:06.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.486+0000 7f7343e00700 10 client.27335 handle_command_reply: tid=0 2024-05-10T05:44:06.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.486+0000 7f73812c3b80 1 client.27335 shutdown 2024-05-10T05:44:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.486+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 msgr2=0x55a2d9878540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.486+0000 7f73812c3b80 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9878540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f73500a15e0 tx=0x7f7350095370 comp rx=0 tx=0).stop 2024-05-10T05:44:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.486+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 shutdown_connections 2024-05-10T05:44:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.487+0000 7f73812c3b80 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55a2d987f020 0x55a2d98d7500 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.487+0000 7f73812c3b80 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d9859ec0 0x55a2d9879330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.487+0000 7f73812c3b80 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2d98586a0 0x55a2d9878000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.487+0000 7f7343e00700 0 client.27335 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.487+0000 7f73812c3b80 1 --2- 172.21.15.131:0/2351791973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d9859570 0x55a2d9878540 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.487+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 >> 172.21.15.131:0/2351791973 conn(0x55a2d97baa60 msgr2=0x55a2d985c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:06.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.487+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 shutdown_connections 2024-05-10T05:44:06.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.488+0000 7f73812c3b80 1 -- 172.21.15.131:0/2351791973 wait complete. 2024-05-10T05:44:06.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.490+0000 7f73812c3b80 20 client.27335 trim_cache size 0 max 16384 2024-05-10T05:44:06.491 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:44:06.491 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:44:06.491 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:44:06.491 INFO:teuthology.orchestra.run.smithi131.stdout: "bc55f0f5-e3e4-4418-914e-d7165cbd75b2": { 2024-05-10T05:44:06.492 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:44:06.492 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "bc55f0f5-e3e4-4418-914e-d7165cbd75b2", 2024-05-10T05:44:06.492 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:44:06.492 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:44:06.492 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:44:06.492 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:44:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.491+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73480a1290 msgr2=0x7f73480a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.491+0000 7f7369400700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73480a1290 0x7f73480a37a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f735c009840 tx=0x7f735c00bb30 comp rx=0 tx=0).stop 2024-05-10T05:44:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.491+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73641092d0 msgr2=0x7f736419eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.491+0000 7f7369400700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73641092d0 0x7f736419eca0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f735000d9f0 tx=0x7f735000dd00 comp rx=0 tx=0).stop 2024-05-10T05:44:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.492+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 shutdown_connections 2024-05-10T05:44:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.492+0000 7f7369400700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f73480a1290 0x7f73480a37a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.492+0000 7f7369400700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7364109c70 0x7f73641a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.492+0000 7f7369400700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73641081b0 0x7f736419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.492+0000 7f7369400700 1 --2- 172.21.15.131:0/2633670377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73641092d0 0x7f736419eca0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.492+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 >> 172.21.15.131:0/2633670377 conn(0x7f73640771f0 msgr2=0x7f736410c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.494+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 shutdown_connections 2024-05-10T05:44:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:06.494+0000 7f7369400700 1 -- 172.21.15.131:0/2633670377 wait complete. 2024-05-10T05:44:06.557 INFO:tasks.cephfs.filesystem:scrub status for tag:bc55f0f5-e3e4-4418-914e-d7165cbd75b2 - {'path': '/', 'tag': 'bc55f0f5-e3e4-4418-914e-d7165cbd75b2', 'options': 'recursive,force'} 2024-05-10T05:44:06.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:06 smithi131 ceph-mon[37264]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 253 MiB/s wr, 80 op/s 2024-05-10T05:44:06.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:06 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:44:06.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:06 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:06.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:06 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:44:06.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:06 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:44:06.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:06 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:06.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:06 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:07.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:06 smithi157 ceph-mon[78427]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 253 MiB/s wr, 80 op/s 2024-05-10T05:44:07.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:06 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:44:07.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:06 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:07.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:06 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:44:07.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:06 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:44:07.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:06 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:07.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:06 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:07.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:06 smithi142 ceph-mon[77667]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 253 MiB/s wr, 80 op/s 2024-05-10T05:44:07.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:06 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:44:07.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:06 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:07.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:06 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:44:07.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:06 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:44:07.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:06 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:07.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:06 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:09.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:08 smithi157 ceph-mon[78427]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 208 MiB/s wr, 67 op/s 2024-05-10T05:44:09.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:08 smithi142 ceph-mon[77667]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 208 MiB/s wr, 67 op/s 2024-05-10T05:44:09.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:08 smithi131 ceph-mon[37264]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 208 MiB/s wr, 67 op/s 2024-05-10T05:44:10.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:09 smithi157 ceph-mon[78427]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 246 MiB/s wr, 76 op/s 2024-05-10T05:44:10.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:09 smithi142 ceph-mon[77667]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 246 MiB/s wr, 76 op/s 2024-05-10T05:44:10.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:09 smithi131 ceph-mon[37264]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 246 MiB/s wr, 76 op/s 2024-05-10T05:44:12.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:12 smithi131 ceph-mon[37264]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 237 MiB/s wr, 75 op/s 2024-05-10T05:44:13.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:12 smithi157 ceph-mon[78427]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 237 MiB/s wr, 75 op/s 2024-05-10T05:44:13.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:12 smithi142 ceph-mon[77667]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 237 MiB/s wr, 75 op/s 2024-05-10T05:44:14.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:14 smithi131 ceph-mon[37264]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 222 MiB/s wr, 68 op/s 2024-05-10T05:44:15.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:14 smithi157 ceph-mon[78427]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 222 MiB/s wr, 68 op/s 2024-05-10T05:44:15.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:14 smithi142 ceph-mon[77667]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 222 MiB/s wr, 68 op/s 2024-05-10T05:44:15.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:15 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:44:16.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:15 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:44:16.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:15 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:44:16.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:16 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:44:16.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:16 smithi131 ceph-mon[37264]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 255 MiB/s wr, 79 op/s 2024-05-10T05:44:17.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:16 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:44:17.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:16 smithi157 ceph-mon[78427]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 255 MiB/s wr, 79 op/s 2024-05-10T05:44:17.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:16 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:44:17.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:16 smithi142 ceph-mon[77667]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 255 MiB/s wr, 79 op/s 2024-05-10T05:44:18.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:18 smithi131 ceph-mon[37264]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 222 MiB/s wr, 69 op/s 2024-05-10T05:44:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:18 smithi157 ceph-mon[78427]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 222 MiB/s wr, 69 op/s 2024-05-10T05:44:19.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:18 smithi142 ceph-mon[77667]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 222 MiB/s wr, 69 op/s 2024-05-10T05:44:21.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:20 smithi157 ceph-mon[78427]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 264 MiB/s wr, 79 op/s 2024-05-10T05:44:21.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:20 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:44:21.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:20 smithi142 ceph-mon[77667]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 264 MiB/s wr, 79 op/s 2024-05-10T05:44:21.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:20 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:44:21.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:20 smithi131 ceph-mon[37264]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 264 MiB/s wr, 79 op/s 2024-05-10T05:44:21.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:20 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:44:22.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:22 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:22 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:22 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:22 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.461 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:22 smithi131 ceph-mon[37264]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 248 MiB/s wr, 77 op/s 2024-05-10T05:44:22.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:22 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:22 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:22 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:22 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:22 smithi157 ceph-mon[78427]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 248 MiB/s wr, 77 op/s 2024-05-10T05:44:22.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:22 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:22 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:22 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:22 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:22.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:22 smithi142 ceph-mon[77667]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 248 MiB/s wr, 77 op/s 2024-05-10T05:44:24.810 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:24 smithi157 ceph-mon[78427]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 70 op/s 2024-05-10T05:44:24.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:24 smithi142 ceph-mon[77667]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 70 op/s 2024-05-10T05:44:24.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:24 smithi131 ceph-mon[37264]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 70 op/s 2024-05-10T05:44:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:25 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:25 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:26.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:25 smithi157 ceph-mon[78427]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 276 MiB/s wr, 83 op/s 2024-05-10T05:44:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:25 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:25 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:26.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:25 smithi142 ceph-mon[77667]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 276 MiB/s wr, 83 op/s 2024-05-10T05:44:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:25 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:25 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:26.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:25 smithi131 ceph-mon[37264]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 276 MiB/s wr, 83 op/s 2024-05-10T05:44:27.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:27 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:44:27.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:27 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:44:27.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:27 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:27.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:27 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:44:27.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:27 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:44:27.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:27 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:27.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:27 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:44:27.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:27 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:44:27.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:27 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:44:28.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:28 smithi157 ceph-mon[78427]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:44:28.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:28 smithi142 ceph-mon[77667]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:44:28.462 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:28 smithi131 ceph-mon[37264]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 73 op/s 2024-05-10T05:44:30.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:30 smithi157 ceph-mon[78427]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 82 op/s 2024-05-10T05:44:30.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:30 smithi142 ceph-mon[77667]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 82 op/s 2024-05-10T05:44:30.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:30 smithi131 ceph-mon[37264]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 82 op/s 2024-05-10T05:44:32.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:32 smithi157 ceph-mon[78427]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 266 MiB/s wr, 81 op/s 2024-05-10T05:44:32.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:32 smithi142 ceph-mon[77667]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 266 MiB/s wr, 81 op/s 2024-05-10T05:44:32.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:32 smithi131 ceph-mon[37264]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 266 MiB/s wr, 81 op/s 2024-05-10T05:44:34.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:34 smithi142 ceph-mon[77667]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 73 op/s 2024-05-10T05:44:34.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:34 smithi131 ceph-mon[37264]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 73 op/s 2024-05-10T05:44:35.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:34 smithi157 ceph-mon[78427]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 73 op/s 2024-05-10T05:44:36.559 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:44:36.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:36 smithi142 ceph-mon[77667]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 271 MiB/s wr, 82 op/s 2024-05-10T05:44:36.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:36 smithi131 ceph-mon[37264]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 271 MiB/s wr, 82 op/s 2024-05-10T05:44:37.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:36 smithi157 ceph-mon[78427]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 271 MiB/s wr, 82 op/s 2024-05-10T05:44:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.123+0000 7f2a24c00700 1 -- 172.21.15.131:0/3479046772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a20102d00 msgr2=0x7f2a20103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.123+0000 7f2a24c00700 1 --2- 172.21.15.131:0/3479046772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a20102d00 0x7f2a20103110 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009990 tx=0x7f2a10009ca0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.125+0000 7f2a24c00700 1 -- 172.21.15.131:0/3479046772 shutdown_connections 2024-05-10T05:44:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.125+0000 7f2a24c00700 1 --2- 172.21.15.131:0/3479046772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a201036a0 0x7f2a20109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.125+0000 7f2a24c00700 1 --2- 172.21.15.131:0/3479046772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a20101be0 0x7f2a20101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.125+0000 7f2a24c00700 1 --2- 172.21.15.131:0/3479046772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a20102d00 0x7f2a20103110 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.125+0000 7f2a24c00700 1 -- 172.21.15.131:0/3479046772 >> 172.21.15.131:0/3479046772 conn(0x7f2a200fd210 msgr2=0x7f2a200ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.125+0000 7f2a24c00700 1 -- 172.21.15.131:0/3479046772 shutdown_connections 2024-05-10T05:44:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.125+0000 7f2a24c00700 1 -- 172.21.15.131:0/3479046772 wait complete. 2024-05-10T05:44:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.127+0000 7f2a24c00700 1 Processor -- start 2024-05-10T05:44:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.127+0000 7f2a24c00700 1 -- start start 2024-05-10T05:44:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.127+0000 7f2a24c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a20101be0 0x7f2a2019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.127+0000 7f2a24c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a20102d00 0x7f2a2019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.127+0000 7f2a24c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a201036a0 0x7f2a201a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.127+0000 7f2a24c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a201a60c0 con 0x7f2a20101be0 2024-05-10T05:44:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.127+0000 7f2a24c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a201a6230 con 0x7f2a20102d00 2024-05-10T05:44:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.127+0000 7f2a24c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a201a63f0 con 0x7f2a201036a0 2024-05-10T05:44:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.128+0000 7f2a1e800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a201036a0 0x7f2a201a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.128+0000 7f2a1e800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a201036a0 0x7f2a201a5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60950/0 (socket says 172.21.15.131:60950) 2024-05-10T05:44:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.128+0000 7f2a1e800700 1 -- 172.21.15.131:0/2241560037 learned_addr learned my addr 172.21.15.131:0/2241560037 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.129+0000 7f2a1e800700 1 -- 172.21.15.131:0/2241560037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a20102d00 msgr2=0x7f2a2019eb20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T05:44:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.129+0000 7f2a1e800700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a20102d00 0x7f2a2019eb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.129+0000 7f2a1e800700 1 -- 172.21.15.131:0/2241560037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a20101be0 msgr2=0x7f2a2019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.129+0000 7f2a1e800700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a20101be0 0x7f2a2019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.129+0000 7f2a1e800700 1 -- 172.21.15.131:0/2241560037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a100095f0 con 0x7f2a201036a0 2024-05-10T05:44:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.129+0000 7f2a1e800700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a201036a0 0x7f2a201a5ab0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2a18007ee0 tx=0x7f2a1800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.130+0000 7f2a16a00700 1 -- 172.21.15.131:0/2241560037 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18017070 con 0x7f2a201036a0 2024-05-10T05:44:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.130+0000 7f2a16a00700 1 -- 172.21.15.131:0/2241560037 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a1800f040 con 0x7f2a201036a0 2024-05-10T05:44:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.130+0000 7f2a16a00700 1 -- 172.21.15.131:0/2241560037 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a180057d0 con 0x7f2a201036a0 2024-05-10T05:44:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.130+0000 7f2a24c00700 1 -- 172.21.15.131:0/2241560037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a201a66d0 con 0x7f2a201036a0 2024-05-10T05:44:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.130+0000 7f2a24c00700 1 -- 172.21.15.131:0/2241560037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a201a6c10 con 0x7f2a201036a0 2024-05-10T05:44:37.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.135+0000 7f2a16a00700 1 -- 172.21.15.131:0/2241560037 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2a18005930 con 0x7f2a201036a0 2024-05-10T05:44:37.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.137+0000 7f2a16a00700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a040a13b0 0x7f2a040a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.138+0000 7f2a16a00700 1 -- 172.21.15.131:0/2241560037 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2a180ccf10 con 0x7f2a201036a0 2024-05-10T05:44:37.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.138+0000 7f2a3ca29b80 1 Processor -- start 2024-05-10T05:44:37.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.138+0000 7f2a1de00700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a040a13b0 0x7f2a040a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.138+0000 7f2a3ca29b80 1 -- start start 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.139+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 0x55c2ec190af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.139+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191570 0x55c2ec191980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.139+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec191ec0 0x55c2ec1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.139+0000 7f2a3ca29b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec0b7150 con 0x55c2ec191570 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.139+0000 7f2a3ca29b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec0b4b10 con 0x55c2ec191ec0 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.139+0000 7f2a3ca29b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec0b0400 con 0x55c2ec1906a0 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.140+0000 7f2a1d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191570 0x55c2ec191980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.140+0000 7f2a1d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191570 0x55c2ec191980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44784/0 (socket says 172.21.15.131:44784) 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.140+0000 7f2a1d400700 1 -- 172.21.15.131:0/2231714610 learned_addr learned my addr 172.21.15.131:0/2231714610 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.140+0000 7f2a1d400700 1 -- 172.21.15.131:0/2231714610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 msgr2=0x55c2ec190af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.140+0000 7f2a1d400700 1 --2- 172.21.15.131:0/2231714610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 0x55c2ec190af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.140+0000 7f2a1d400700 1 -- 172.21.15.131:0/2231714610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec191ec0 msgr2=0x55c2ec1987c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:44:37.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.140+0000 7f2a1d400700 1 --2- 172.21.15.131:0/2231714610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec191ec0 0x55c2ec1987c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.140+0000 7f2a1d400700 1 -- 172.21.15.131:0/2231714610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2ebf313b0 con 0x55c2ec191570 2024-05-10T05:44:37.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.141+0000 7f2a1d400700 1 --2- 172.21.15.131:0/2231714610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191570 0x55c2ec191980 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009530 tx=0x7f2a10009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ca02f129e7e065 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:37.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.142+0000 7f2a0b000700 1 -- 172.21.15.131:0/2231714610 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1001c070 con 0x55c2ec191570 2024-05-10T05:44:37.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.142+0000 7f2a0b000700 1 -- 172.21.15.131:0/2231714610 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a10003680 con 0x55c2ec191570 2024-05-10T05:44:37.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.142+0000 7f2a0b000700 1 -- 172.21.15.131:0/2231714610 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a100052d0 con 0x55c2ec191570 2024-05-10T05:44:37.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.144+0000 7f2a1de00700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a040a13b0 0x7f2a040a38c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2a20102c00 tx=0x7f2a0c006d20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:37.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.144+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/2231714610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191570 msgr2=0x55c2ec191980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:37.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.144+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/2231714610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191570 0x55c2ec191980 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009530 tx=0x7f2a10009560 comp rx=0 tx=0).stop 2024-05-10T05:44:37.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.144+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/2231714610 shutdown_connections 2024-05-10T05:44:37.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.144+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/2231714610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec191ec0 0x55c2ec1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.144+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/2231714610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 0x55c2ec190af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.144+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/2231714610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191570 0x55c2ec191980 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.144+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/2231714610 >> 172.21.15.131:0/2231714610 conn(0x55c2ec0f2a60 msgr2=0x55c2ec18afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:37.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.145+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/2231714610 shutdown_connections 2024-05-10T05:44:37.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.145+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/2231714610 wait complete. 2024-05-10T05:44:37.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.146+0000 7f2a3ca29b80 1 -- start start 2024-05-10T05:44:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.146+0000 7f2a3ca29b80 1 Processor -- start 2024-05-10T05:44:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.147+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1906a0 0x55c2ec1ab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.147+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 0x55c2ec1abe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.149+0000 7f2a1de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1906a0 0x55c2ec1ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.149+0000 7f2a1de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1906a0 0x55c2ec1ab850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41486/0 (socket says 172.21.15.131:41486) 2024-05-10T05:44:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.149+0000 7f2a1de00700 1 -- 172.21.15.131:0/816816239 learned_addr learned my addr 172.21.15.131:0/816816239 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.150+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.150+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec04c560 con 0x55c2ec191ec0 2024-05-10T05:44:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.150+0000 7f2a1e800700 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.150+0000 7f2a1d400700 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 0x55c2ec1abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.150+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec04c200 con 0x55c2ec1906a0 2024-05-10T05:44:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.150+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ebf336a0 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.151+0000 7f2a3ca29b80 10 client.0 _ll_register_callbacks cb 0x55c2ec055b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.151+0000 7f2a1e800700 1 -- 172.21.15.131:0/816816239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1906a0 msgr2=0x55c2ec1ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.151+0000 7f2a1e800700 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1906a0 0x55c2ec1ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.151+0000 7f2a1e800700 1 -- 172.21.15.131:0/816816239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 msgr2=0x55c2ec1abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.151+0000 7f2a1e800700 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 0x55c2ec1abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.151+0000 7f2a1e800700 1 -- 172.21.15.131:0/816816239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2010a9e0 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.151+0000 7f2a1d400700 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 0x55c2ec1abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.152+0000 7f2a1e800700 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1b3100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2a1809a290 tx=0x7f2a18009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295660c6267df8ef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.152+0000 7f29ff400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.152+0000 7f29ff400700 1 -- 172.21.15.131:0/816816239 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1809e130 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.152+0000 7f29ff400700 1 -- 172.21.15.131:0/816816239 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2a180a2780 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.152+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2ec15f780 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.153+0000 7f29ff400700 1 -- 172.21.15.131:0/816816239 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a180caac0 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.153+0000 7f2a3ca29b80 10 client.27355 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:37.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.153+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c2ebeddb60 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.153+0000 7f29ff400700 1 -- 172.21.15.131:0/816816239 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2ec15f780 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.153+0000 7f29ff400700 1 -- 172.21.15.131:0/816816239 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2a180998f0 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.153+0000 7f2a3ca29b80 10 client.27355 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:37.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.153+0000 7f2a3ca29b80 10 client.27355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:37.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.153+0000 7f2a3ca29b80 20 client.27355 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:37.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.153+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c2ec1b7950 0x55c2ec1b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.154+0000 7f2a3ca29b80 4 client.27355 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:37.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.154+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2ec0a8ec0 con 0x55c2ec1b7950 2024-05-10T05:44:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.154+0000 7f2a1de00700 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c2ec1b7950 0x55c2ec1b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.154+0000 7f29ff400700 10 client.27355 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.155+0000 7f2a1de00700 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c2ec1b7950 0x55c2ec1b9da0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd75bce5c3b5c023 server_cookie=b0fb637dcc96ff27 in_seq=0 out_seq=0 2024-05-10T05:44:37.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.156+0000 7f29ff400700 1 -- 172.21.15.131:0/816816239 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2ec0a8ec0 con 0x55c2ec1b7950 2024-05-10T05:44:37.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.156+0000 7f29ff400700 10 client.27355 handle_command_reply: tid=0 2024-05-10T05:44:37.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.156+0000 7f2a3ca29b80 1 client.27355 shutdown 2024-05-10T05:44:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.157+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1b0ce0 msgr2=0x55c2ec1b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:37.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.158+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1b3100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2a1809a290 tx=0x7f2a18009450 comp rx=0 tx=0).stop 2024-05-10T05:44:37.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.158+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 shutdown_connections 2024-05-10T05:44:37.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.158+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c2ec1b7950 0x55c2ec1b9da0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.158+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1b3100 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.158+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1906a0 0x55c2ec1ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.158+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/816816239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 0x55c2ec1abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.158+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 >> 172.21.15.131:0/816816239 conn(0x55c2ec0f2a60 msgr2=0x55c2ec18afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:37.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.159+0000 7f29ff400700 0 client.27355 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:37.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.159+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 shutdown_connections 2024-05-10T05:44:37.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.159+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/816816239 wait complete. 2024-05-10T05:44:37.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.164+0000 7f2a3ca29b80 20 client.27355 trim_cache size 0 max 16384 2024-05-10T05:44:37.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.210+0000 7f2a3ca29b80 1 Processor -- start 2024-05-10T05:44:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.210+0000 7f2a3ca29b80 1 -- start start 2024-05-10T05:44:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.210+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 0x55c2ec1b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.210+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 0x55c2ec1c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.210+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.210+0000 7f2a3ca29b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec18bc80 con 0x55c2ec191ec0 2024-05-10T05:44:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.210+0000 7f2a3ca29b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec1b4290 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.210+0000 7f2a3ca29b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec1ac660 con 0x55c2ec1906a0 2024-05-10T05:44:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.211+0000 7f2a1de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 0x55c2ec1b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.211+0000 7f2a1d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 0x55c2ec1c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.211+0000 7f2a1de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 0x55c2ec1b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60958/0 (socket says 172.21.15.131:60958) 2024-05-10T05:44:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.211+0000 7f2a1de00700 1 -- 172.21.15.131:0/2276104361 learned_addr learned my addr 172.21.15.131:0/2276104361 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:37.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.212+0000 7f2a1de00700 1 -- 172.21.15.131:0/2276104361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 msgr2=0x55c2ec1c81b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:44:37.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.212+0000 7f2a1de00700 1 --2- 172.21.15.131:0/2276104361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1c81b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.212+0000 7f2a1de00700 1 -- 172.21.15.131:0/2276104361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 msgr2=0x55c2ec1c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:37.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.212+0000 7f2a1de00700 1 --2- 172.21.15.131:0/2276104361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 0x55c2ec1c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.212+0000 7f2a1de00700 1 -- 172.21.15.131:0/2276104361 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2ec076560 con 0x55c2ec1906a0 2024-05-10T05:44:37.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.212+0000 7f2a1d400700 1 --2- 172.21.15.131:0/2276104361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 0x55c2ec1c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:37.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.213+0000 7f2a1de00700 1 --2- 172.21.15.131:0/2276104361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 0x55c2ec1b3100 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c015ab0 tx=0x7f2a0c006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8db52f06ada79f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:37.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.213+0000 7f2a0ba00700 1 -- 172.21.15.131:0/2276104361 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c01a040 con 0x55c2ec1906a0 2024-05-10T05:44:37.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.213+0000 7f2a0ba00700 1 -- 172.21.15.131:0/2276104361 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a0c01b040 con 0x55c2ec1906a0 2024-05-10T05:44:37.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.213+0000 7f2a0ba00700 1 -- 172.21.15.131:0/2276104361 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0244f0 con 0x55c2ec1906a0 2024-05-10T05:44:37.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.215+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/2276104361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 msgr2=0x55c2ec1b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:37.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.215+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/2276104361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 0x55c2ec1b3100 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c015ab0 tx=0x7f2a0c006f90 comp rx=0 tx=0).stop 2024-05-10T05:44:37.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.215+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/2276104361 shutdown_connections 2024-05-10T05:44:37.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.215+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/2276104361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.215+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/2276104361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec1906a0 0x55c2ec1b3100 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.215+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/2276104361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec191ec0 0x55c2ec1c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.215+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/2276104361 >> 172.21.15.131:0/2276104361 conn(0x55c2ec0f2a60 msgr2=0x55c2ec1947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:37.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.215+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/2276104361 shutdown_connections 2024-05-10T05:44:37.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.215+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/2276104361 wait complete. 2024-05-10T05:44:37.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.216+0000 7f2a3ca29b80 1 -- start start 2024-05-10T05:44:37.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.216+0000 7f2a3ca29b80 1 Processor -- start 2024-05-10T05:44:37.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.217+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec1906a0 0x55c2ec1c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.217+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec191ec0 0x55c2ec1c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.217+0000 7f2a3ca29b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.217+0000 7f2a3ca29b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec20c930 con 0x55c2ec1906a0 2024-05-10T05:44:37.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.217+0000 7f2a3ca29b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec20ca70 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.217+0000 7f2a3ca29b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec20cbb0 con 0x55c2ec191ec0 2024-05-10T05:44:37.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.217+0000 7f2a3ca29b80 10 client.0 _ll_register_callbacks cb 0x55c2ec055b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:37.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.218+0000 7f2a1e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.218+0000 7f2a1e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41502/0 (socket says 172.21.15.131:41502) 2024-05-10T05:44:37.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.218+0000 7f2a1e800700 1 -- 172.21.15.131:0/3144883963 learned_addr learned my addr 172.21.15.131:0/3144883963 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:37.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.219+0000 7f2a1e800700 1 -- 172.21.15.131:0/3144883963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec191ec0 msgr2=0x55c2ec1c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:37.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.219+0000 7f2a1d400700 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec191ec0 0x55c2ec1c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.220+0000 7f2a1e800700 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec191ec0 0x55c2ec1c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.220+0000 7f2a1e800700 1 -- 172.21.15.131:0/3144883963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec1906a0 msgr2=0x55c2ec1c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:37.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.220+0000 7f2a1e800700 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec1906a0 0x55c2ec1c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.220+0000 7f2a1e800700 1 -- 172.21.15.131:0/3144883963 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2ebf313b0 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.220+0000 7f2a1d400700 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec191ec0 0x55c2ec1c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:37.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.220+0000 7f2a1e800700 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1c74a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2a18096800 tx=0x7f2a18004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=952e05362bfac9d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:37.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.220+0000 7f29ff400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:44:37.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.221+0000 7f29ff400700 1 -- 172.21.15.131:0/3144883963 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a180a2780 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.221+0000 7f29ff400700 1 -- 172.21.15.131:0/3144883963 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2a1809e130 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.221+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/3144883963 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2ec009270 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.221+0000 7f29ff400700 1 -- 172.21.15.131:0/3144883963 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a1809a770 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.221+0000 7f2a3ca29b80 10 client.27344 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:37.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.221+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/3144883963 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f2a0c015710 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.222+0000 7f29ff400700 1 -- 172.21.15.131:0/3144883963 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a180997f0 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.222+0000 7f29ff400700 1 -- 172.21.15.131:0/3144883963 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2a18099260 con 0x55c2ec1b0ce0 2024-05-10T05:44:37.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.222+0000 7f2a3ca29b80 10 client.27344 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:37.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.222+0000 7f2a3ca29b80 10 client.27344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:37.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.222+0000 7f2a3ca29b80 20 client.27344 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:37.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.222+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c2ec20e060 0x55c2ec210480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:37.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.222+0000 7f2a3ca29b80 4 client.27344 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:44:37.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.222+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/3144883963 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c2ec009270 con 0x55c2ec20e060 2024-05-10T05:44:37.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.223+0000 7f2a1de00700 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c2ec20e060 0x55c2ec210480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.224+0000 7f2a1de00700 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c2ec20e060 0x55c2ec210480 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0070b44fcfb260 server_cookie=c5f6c0d27e02cfe6 in_seq=0 out_seq=0 2024-05-10T05:44:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.224+0000 7f29ff400700 10 client.27344 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.224+0000 7f29ff400700 1 -- 172.21.15.131:0/3144883963 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c2ec009270 con 0x55c2ec20e060 2024-05-10T05:44:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.224+0000 7f29ff400700 10 client.27344 handle_command_reply: tid=0 2024-05-10T05:44:37.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.224+0000 7f2a3ca29b80 1 client.27344 shutdown 2024-05-10T05:44:37.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/3144883963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 msgr2=0x55c2ec1c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:37.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1c74a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2a18096800 tx=0x7f2a18004d50 comp rx=0 tx=0).stop 2024-05-10T05:44:37.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/3144883963 shutdown_connections 2024-05-10T05:44:37.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55c2ec20e060 0x55c2ec210480 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2ec1906a0 0x55c2ec1c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2ec1b0ce0 0x55c2ec1c74a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f2a3ca29b80 1 --2- 172.21.15.131:0/3144883963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c2ec191ec0 0x55c2ec1c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/3144883963 >> 172.21.15.131:0/3144883963 conn(0x55c2ec0f2a60 msgr2=0x55c2ec195490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f29ff400700 0 client.27344 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/3144883963 shutdown_connections 2024-05-10T05:44:37.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.228+0000 7f2a3ca29b80 1 -- 172.21.15.131:0/3144883963 wait complete. 2024-05-10T05:44:37.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.230+0000 7f2a3ca29b80 20 client.27344 trim_cache size 0 max 16384 2024-05-10T05:44:37.232 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:44:37.232 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:44:37.232 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:44:37.233 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:44:37.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.233+0000 7f2a24c00700 1 -- 172.21.15.131:0/2241560037 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a040a13b0 msgr2=0x7f2a040a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:37.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.233+0000 7f2a24c00700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a040a13b0 0x7f2a040a38c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2a20102c00 tx=0x7f2a0c006d20 comp rx=0 tx=0).stop 2024-05-10T05:44:37.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.233+0000 7f2a24c00700 1 -- 172.21.15.131:0/2241560037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a201036a0 msgr2=0x7f2a201a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:37.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.233+0000 7f2a24c00700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a201036a0 0x7f2a201a5ab0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2a18007ee0 tx=0x7f2a1800d3b0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.234+0000 7f2a24c00700 1 -- 172.21.15.131:0/2241560037 shutdown_connections 2024-05-10T05:44:37.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.234+0000 7f2a24c00700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f2a040a13b0 0x7f2a040a38c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.234+0000 7f2a24c00700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a20101be0 0x7f2a2019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.235+0000 7f2a24c00700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a201036a0 0x7f2a201a5ab0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.235+0000 7f2a24c00700 1 --2- 172.21.15.131:0/2241560037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a20102d00 0x7f2a2019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.235+0000 7f2a24c00700 1 -- 172.21.15.131:0/2241560037 >> 172.21.15.131:0/2241560037 conn(0x7f2a200fd210 msgr2=0x7f2a20103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:37.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.236+0000 7f2a24c00700 1 -- 172.21.15.131:0/2241560037 shutdown_connections 2024-05-10T05:44:37.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.236+0000 7f2a24c00700 1 -- 172.21.15.131:0/2241560037 wait complete. 2024-05-10T05:44:37.293 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:44:37.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:44:37.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.988+0000 7f838be00700 1 -- 172.21.15.131:0/3082793294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83841092d0 msgr2=0x7f838410fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:37.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.988+0000 7f838be00700 1 --2- 172.21.15.131:0/3082793294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83841092d0 0x7f838410fde0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f837000ddd0 tx=0x7f83700084b0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.992+0000 7f838be00700 1 -- 172.21.15.131:0/3082793294 shutdown_connections 2024-05-10T05:44:37.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.992+0000 7f838be00700 1 --2- 172.21.15.131:0/3082793294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8384110320 0x7f8384112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.992+0000 7f838be00700 1 --2- 172.21.15.131:0/3082793294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83841088f0 0x7f8384108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.992+0000 7f838be00700 1 --2- 172.21.15.131:0/3082793294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83841092d0 0x7f838410fde0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:37.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.992+0000 7f838be00700 1 -- 172.21.15.131:0/3082793294 >> 172.21.15.131:0/3082793294 conn(0x7f838406ea00 msgr2=0x7f8384070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:37.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.992+0000 7f838be00700 1 -- 172.21.15.131:0/3082793294 shutdown_connections 2024-05-10T05:44:37.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.992+0000 7f838be00700 1 -- 172.21.15.131:0/3082793294 wait complete. 2024-05-10T05:44:37.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.994+0000 7f838be00700 1 Processor -- start 2024-05-10T05:44:38.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.997+0000 7f838be00700 1 -- start start 2024-05-10T05:44:38.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.997+0000 7f838be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83841088f0 0x7f83841af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.998+0000 7f838be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83841092d0 0x7f83841afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.998+0000 7f8389400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83841088f0 0x7f83841af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.998+0000 7f8389400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83841088f0 0x7f83841af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60970/0 (socket says 172.21.15.131:60970) 2024-05-10T05:44:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.998+0000 7f838be00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8384110320 0x7f83841b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.998+0000 7f838be00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83841b70a0 con 0x7f8384110320 2024-05-10T05:44:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.998+0000 7f838be00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83841b7210 con 0x7f83841092d0 2024-05-10T05:44:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.998+0000 7f838be00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83841b73d0 con 0x7f83841088f0 2024-05-10T05:44:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.998+0000 7f8389400700 1 -- 172.21.15.131:0/4046547543 learned_addr learned my addr 172.21.15.131:0/4046547543 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.999+0000 7f8388a00700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83841092d0 0x7f83841afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.999+0000 7f8389400700 1 -- 172.21.15.131:0/4046547543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83841092d0 msgr2=0x7f83841afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.999+0000 7f8389400700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83841092d0 0x7f83841afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.999+0000 7f8389400700 1 -- 172.21.15.131:0/4046547543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8384110320 msgr2=0x7f83841b6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.999+0000 7f8389400700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8384110320 0x7f83841b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:38.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.999+0000 7f8389400700 1 -- 172.21.15.131:0/4046547543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837000da00 con 0x7f83841088f0 2024-05-10T05:44:38.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.999+0000 7f8388a00700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83841092d0 0x7f83841afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:38.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:37.999+0000 7f8389400700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83841088f0 0x7f83841af5c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f837800e940 tx=0x7f837800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:38.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.000+0000 7f8382000700 1 -- 172.21.15.131:0/4046547543 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378009730 con 0x7f83841088f0 2024-05-10T05:44:38.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.000+0000 7f8382000700 1 -- 172.21.15.131:0/4046547543 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f837800cbd0 con 0x7f83841088f0 2024-05-10T05:44:38.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.002+0000 7f8382000700 1 -- 172.21.15.131:0/4046547543 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378018680 con 0x7f83841088f0 2024-05-10T05:44:38.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.003+0000 7f838be00700 1 -- 172.21.15.131:0/4046547543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83841b76b0 con 0x7f83841088f0 2024-05-10T05:44:38.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.003+0000 7f838be00700 1 -- 172.21.15.131:0/4046547543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83841b7b70 con 0x7f83841088f0 2024-05-10T05:44:38.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.006+0000 7f838be00700 1 -- 172.21.15.131:0/4046547543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838410ec80 con 0x7f83841088f0 2024-05-10T05:44:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.013+0000 7f8382000700 1 -- 172.21.15.131:0/4046547543 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f837800cd40 con 0x7f83841088f0 2024-05-10T05:44:38.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.015+0000 7f8382000700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f836c0a1260 0x7f836c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:38.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.015+0000 7f8382000700 1 -- 172.21.15.131:0/4046547543 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8378014070 con 0x7f83841088f0 2024-05-10T05:44:38.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.017+0000 7f8388a00700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f836c0a1260 0x7f836c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:38.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.019+0000 7f8382000700 1 -- 172.21.15.131:0/4046547543 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f83780963d0 con 0x7f83841088f0 2024-05-10T05:44:38.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.023+0000 7f8388a00700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f836c0a1260 0x7f836c0a3770 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8370000c00 tx=0x7f837000f040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:38.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.748+0000 7f838be00700 1 -- 172.21.15.131:0/4046547543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83840682e0 con 0x7f83841088f0 2024-05-10T05:44:38.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.749+0000 7f8382000700 1 -- 172.21.15.131:0/4046547543 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f83780999f0 con 0x7f83841088f0 2024-05-10T05:44:38.754 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:44:38.754 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:37:31.134382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:44:38.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.759+0000 7f838be00700 1 -- 172.21.15.131:0/4046547543 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f836c0a1260 msgr2=0x7f836c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:38.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.759+0000 7f838be00700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f836c0a1260 0x7f836c0a3770 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8370000c00 tx=0x7f837000f040 comp rx=0 tx=0).stop 2024-05-10T05:44:38.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.759+0000 7f838be00700 1 -- 172.21.15.131:0/4046547543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83841088f0 msgr2=0x7f83841af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:38.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.759+0000 7f838be00700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83841088f0 0x7f83841af5c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f837800e940 tx=0x7f837800ed00 comp rx=0 tx=0).stop 2024-05-10T05:44:38.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.759+0000 7f838be00700 1 -- 172.21.15.131:0/4046547543 shutdown_connections 2024-05-10T05:44:38.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.759+0000 7f838be00700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f836c0a1260 0x7f836c0a3770 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:38.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.759+0000 7f838be00700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8384110320 0x7f83841b6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:38.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.759+0000 7f838be00700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83841088f0 0x7f83841af5c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:38.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.759+0000 7f838be00700 1 --2- 172.21.15.131:0/4046547543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83841092d0 0x7f83841afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:38.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.759+0000 7f838be00700 1 -- 172.21.15.131:0/4046547543 >> 172.21.15.131:0/4046547543 conn(0x7f838406ea00 msgr2=0x7f838410bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:38.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.766+0000 7f838be00700 1 -- 172.21.15.131:0/4046547543 shutdown_connections 2024-05-10T05:44:38.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:38.766+0000 7f838be00700 1 -- 172.21.15.131:0/4046547543 wait complete. 2024-05-10T05:44:38.768 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-10T05:44:38.820 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:44:38.967 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:38 smithi131 ceph-mon[37264]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 221 MiB/s wr, 69 op/s 2024-05-10T05:44:39.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:38 smithi157 ceph-mon[78427]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 221 MiB/s wr, 69 op/s 2024-05-10T05:44:39.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:38 smithi142 ceph-mon[77667]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 221 MiB/s wr, 69 op/s 2024-05-10T05:44:39.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.540+0000 7f43eea00700 1 -- 172.21.15.131:0/3954072097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e80690b0 msgr2=0x7f43e8106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:39.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.540+0000 7f43eea00700 1 --2- 172.21.15.131:0/3954072097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e80690b0 0x7f43e8106b00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009990 tx=0x7f43d8009ca0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.541+0000 7f43eea00700 1 -- 172.21.15.131:0/3954072097 shutdown_connections 2024-05-10T05:44:39.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.541+0000 7f43eea00700 1 --2- 172.21.15.131:0/3954072097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e8109a80 0x7f43e810bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.541+0000 7f43eea00700 1 --2- 172.21.15.131:0/3954072097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e80690b0 0x7f43e8106b00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.541+0000 7f43eea00700 1 --2- 172.21.15.131:0/3954072097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e8107040 0x7f43e8109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.541+0000 7f43eea00700 1 -- 172.21.15.131:0/3954072097 >> 172.21.15.131:0/3954072097 conn(0x7f43e80fc0d0 msgr2=0x7f43e80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:39.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.541+0000 7f43eea00700 1 -- 172.21.15.131:0/3954072097 shutdown_connections 2024-05-10T05:44:39.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.541+0000 7f43eea00700 1 -- 172.21.15.131:0/3954072097 wait complete. 2024-05-10T05:44:39.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.543+0000 7f43eea00700 1 Processor -- start 2024-05-10T05:44:39.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.544+0000 7f43eea00700 1 -- start start 2024-05-10T05:44:39.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.544+0000 7f43eea00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e8107040 0x7f43e819e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.544+0000 7f43eea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e8109a80 0x7f43e819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.545+0000 7f43eea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e81a35a0 0x7f43e81a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.545+0000 7f43eca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e81a35a0 0x7f43e81a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.545+0000 7f43eca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e81a35a0 0x7f43e81a59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41534/0 (socket says 172.21.15.131:41534) 2024-05-10T05:44:39.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.545+0000 7f43e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e8107040 0x7f43e819e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.545+0000 7f43eca00700 1 -- 172.21.15.131:0/3894793141 learned_addr learned my addr 172.21.15.131:0/3894793141 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:39.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.545+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81a5fd0 con 0x7f43e8107040 2024-05-10T05:44:39.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.545+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81a6140 con 0x7f43e81a35a0 2024-05-10T05:44:39.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.545+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81a6300 con 0x7f43e8109a80 2024-05-10T05:44:39.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.546+0000 7f43e7400700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e8109a80 0x7f43e819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.546+0000 7f43eca00700 1 -- 172.21.15.131:0/3894793141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e8109a80 msgr2=0x7f43e819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:39.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.546+0000 7f43eca00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e8109a80 0x7f43e819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.546+0000 7f43eca00700 1 -- 172.21.15.131:0/3894793141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e8107040 msgr2=0x7f43e819e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:39.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.546+0000 7f43eca00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e8107040 0x7f43e819e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.546+0000 7f43eca00700 1 -- 172.21.15.131:0/3894793141 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0009520 con 0x7f43e81a35a0 2024-05-10T05:44:39.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.546+0000 7f43e7e00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e8107040 0x7f43e819e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:39.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.547+0000 7f43eca00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e81a35a0 0x7f43e81a59c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f43e0007ee0 tx=0x7f43e000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:39.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.547+0000 7f43e4c00700 1 -- 172.21.15.131:0/3894793141 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0017070 con 0x7f43e81a35a0 2024-05-10T05:44:39.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.547+0000 7f43e4c00700 1 -- 172.21.15.131:0/3894793141 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e000f040 con 0x7f43e81a35a0 2024-05-10T05:44:39.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.547+0000 7f43e4c00700 1 -- 172.21.15.131:0/3894793141 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0005740 con 0x7f43e81a35a0 2024-05-10T05:44:39.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.547+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d80095f0 con 0x7f43e81a35a0 2024-05-10T05:44:39.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.548+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e81a6910 con 0x7f43e81a35a0 2024-05-10T05:44:39.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.554+0000 7f43e4c00700 1 -- 172.21.15.131:0/3894793141 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f43e000dc60 con 0x7f43e81a35a0 2024-05-10T05:44:39.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.555+0000 7f44068f1b80 1 Processor -- start 2024-05-10T05:44:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.556+0000 7f44068f1b80 1 -- start start 2024-05-10T05:44:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.556+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x559722933af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.556+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722934570 0x559722934980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.556+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972293b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.556+0000 7f44068f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972285a150 con 0x559722934570 2024-05-10T05:44:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.556+0000 7f44068f1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722857b10 con 0x5597229336a0 2024-05-10T05:44:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.556+0000 7f44068f1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722853400 con 0x559722934ec0 2024-05-10T05:44:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.557+0000 7f43e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722934570 0x559722934980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.557+0000 7f43e7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722934570 0x559722934980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44856/0 (socket says 172.21.15.131:44856) 2024-05-10T05:44:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.557+0000 7f43e7e00700 1 -- 172.21.15.131:0/1082327114 learned_addr learned my addr 172.21.15.131:0/1082327114 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:39.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.557+0000 7f43e7400700 1 --2- 172.21.15.131:0/1082327114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x559722933af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.558+0000 7f43e7e00700 1 -- 172.21.15.131:0/1082327114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 msgr2=0x55972293b7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:44:39.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.558+0000 7f43e7400700 1 --2- 172.21.15.131:0/1082327114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972293b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.558+0000 7f43e7e00700 1 --2- 172.21.15.131:0/1082327114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972293b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.558+0000 7f43e7e00700 1 -- 172.21.15.131:0/1082327114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 msgr2=0x559722933af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:39.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.558+0000 7f43e7e00700 1 --2- 172.21.15.131:0/1082327114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x559722933af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.558+0000 7f43e7e00700 1 -- 172.21.15.131:0/1082327114 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597226d43b0 con 0x559722934570 2024-05-10T05:44:39.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.559+0000 7f43e7400700 1 --2- 172.21.15.131:0/1082327114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x559722933af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:39.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.559+0000 7f43e4c00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43cc0a12c0 0x7f43cc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.559+0000 7f43e4c00700 1 -- 172.21.15.131:0/3894793141 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f43e00cc040 con 0x7f43e81a35a0 2024-05-10T05:44:39.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.559+0000 7f43e7e00700 1 --2- 172.21.15.131:0/1082327114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722934570 0x559722934980 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009990 tx=0x7f43d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159d84ef5a8f538e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:39.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.560+0000 7f43e7400700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43cc0a12c0 0x7f43cc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.560+0000 7f43dd200700 1 -- 172.21.15.131:0/1082327114 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d80047e0 con 0x559722934570 2024-05-10T05:44:39.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.560+0000 7f43dd200700 1 -- 172.21.15.131:0/1082327114 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43d801e040 con 0x559722934570 2024-05-10T05:44:39.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.560+0000 7f43dd200700 1 -- 172.21.15.131:0/1082327114 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d800f460 con 0x559722934570 2024-05-10T05:44:39.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.562+0000 7f43e7400700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43cc0a12c0 0x7f43cc0a37d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x559722933380 tx=0x7f43d0015040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:39.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.562+0000 7f44068f1b80 1 -- 172.21.15.131:0/1082327114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722934570 msgr2=0x559722934980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:39.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.562+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1082327114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722934570 0x559722934980 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009990 tx=0x7f43d8005fd0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.562+0000 7f44068f1b80 1 -- 172.21.15.131:0/1082327114 shutdown_connections 2024-05-10T05:44:39.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.562+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1082327114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972293b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.562+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1082327114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x559722933af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.562+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1082327114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722934570 0x559722934980 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.562+0000 7f44068f1b80 1 -- 172.21.15.131:0/1082327114 >> 172.21.15.131:0/1082327114 conn(0x559722895a60 msgr2=0x55972292dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:39.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.562+0000 7f44068f1b80 1 -- 172.21.15.131:0/1082327114 shutdown_connections 2024-05-10T05:44:39.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.562+0000 7f44068f1b80 1 -- 172.21.15.131:0/1082327114 wait complete. 2024-05-10T05:44:39.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.563+0000 7f44068f1b80 1 -- start start 2024-05-10T05:44:39.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.563+0000 7f44068f1b80 1 Processor -- start 2024-05-10T05:44:39.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.564+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x55972293d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.564+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972293dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.564+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 0x559722945050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.564+0000 7f44068f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597227ef560 con 0x559722942c30 2024-05-10T05:44:39.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.564+0000 7f44068f1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597227ef200 con 0x5597229336a0 2024-05-10T05:44:39.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.565+0000 7f44068f1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597226d66a0 con 0x559722934ec0 2024-05-10T05:44:39.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.565+0000 7f44068f1b80 10 client.0 _ll_register_callbacks cb 0x5597227f8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:39.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.565+0000 7f43e7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972293dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.565+0000 7f43e7400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972293dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32782/0 (socket says 172.21.15.131:32782) 2024-05-10T05:44:39.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.565+0000 7f43e7400700 1 -- 172.21.15.131:0/870059888 learned_addr learned my addr 172.21.15.131:0/870059888 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:39.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.565+0000 7f43e7400700 1 -- 172.21.15.131:0/870059888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 msgr2=0x55972293d7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:44:39.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.565+0000 7f43e7400700 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x55972293d7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.565+0000 7f43e7400700 1 -- 172.21.15.131:0/870059888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 msgr2=0x559722945050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:39.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.565+0000 7f43e7400700 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 0x559722945050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.565+0000 7f43e7400700 1 -- 172.21.15.131:0/870059888 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e810cbb0 con 0x559722934ec0 2024-05-10T05:44:39.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.566+0000 7f43e7400700 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972293dda0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f43d0020c00 tx=0x7f43d0009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b5e4321c59a3bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:39.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.566+0000 7f43d5600700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:39.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.566+0000 7f43d5600700 1 -- 172.21.15.131:0/870059888 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d0013e70 con 0x559722934ec0 2024-05-10T05:44:39.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.567+0000 7f44068f1b80 1 -- 172.21.15.131:0/870059888 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597227ac270 con 0x559722934ec0 2024-05-10T05:44:39.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.567+0000 7f43d5600700 1 -- 172.21.15.131:0/870059888 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f43d0011af0 con 0x559722934ec0 2024-05-10T05:44:39.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.567+0000 7f43d5600700 1 -- 172.21.15.131:0/870059888 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43d0011cb0 con 0x559722934ec0 2024-05-10T05:44:39.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.567+0000 7f44068f1b80 10 client.27373 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:39.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.567+0000 7f44068f1b80 1 -- 172.21.15.131:0/870059888 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559722680b60 con 0x559722934ec0 2024-05-10T05:44:39.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.567+0000 7f43d5600700 1 -- 172.21.15.131:0/870059888 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5597227ac270 con 0x559722934ec0 2024-05-10T05:44:39.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.568+0000 7f43d5600700 1 -- 172.21.15.131:0/870059888 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f43d0003be0 con 0x559722934ec0 2024-05-10T05:44:39.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.568+0000 7f44068f1b80 10 client.27373 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.568+0000 7f44068f1b80 10 client.27373 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:44:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.568+0000 7f44068f1b80 20 client.27373 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.568+0000 7f44068f1b80 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55972295a940 0x55972295cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.568+0000 7f44068f1b80 4 client.27373 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.568+0000 7f44068f1b80 1 -- 172.21.15.131:0/870059888 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559722902780 con 0x55972295a940 2024-05-10T05:44:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.568+0000 7f43e7e00700 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55972295a940 0x55972295cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.569+0000 7f43e7e00700 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55972295a940 0x55972295cdd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ffbc1780ac69a6bf server_cookie=2877e6458d7ecd66 in_seq=0 out_seq=0 2024-05-10T05:44:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.569+0000 7f43d5600700 10 client.27373 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:44:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.570+0000 7f43d5600700 1 -- 172.21.15.131:0/870059888 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559722902780 con 0x55972295a940 2024-05-10T05:44:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.570+0000 7f43d5600700 10 client.27373 handle_command_reply: tid=0 2024-05-10T05:44:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.570+0000 7f44068f1b80 1 client.27373 shutdown 2024-05-10T05:44:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.571+0000 7f44068f1b80 1 -- 172.21.15.131:0/870059888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 msgr2=0x55972293dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.571+0000 7f44068f1b80 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972293dda0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f43d0020c00 tx=0x7f43d0009830 comp rx=0 tx=0).stop 2024-05-10T05:44:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.572+0000 7f44068f1b80 1 -- 172.21.15.131:0/870059888 shutdown_connections 2024-05-10T05:44:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.572+0000 7f44068f1b80 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55972295a940 0x55972295cdd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.572+0000 7f44068f1b80 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 0x559722945050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.572+0000 7f44068f1b80 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x55972293d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.572+0000 7f44068f1b80 1 --2- 172.21.15.131:0/870059888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972293dda0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.572+0000 7f44068f1b80 1 -- 172.21.15.131:0/870059888 >> 172.21.15.131:0/870059888 conn(0x559722895a60 msgr2=0x55972292dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.572+0000 7f43d5600700 0 client.27373 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:44:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.572+0000 7f44068f1b80 1 -- 172.21.15.131:0/870059888 shutdown_connections 2024-05-10T05:44:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.572+0000 7f44068f1b80 1 -- 172.21.15.131:0/870059888 wait complete. 2024-05-10T05:44:39.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.573+0000 7f44068f1b80 20 client.27373 trim_cache size 0 max 16384 2024-05-10T05:44:39.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.607+0000 7f44068f1b80 1 Processor -- start 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.607+0000 7f44068f1b80 1 -- start start 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.608+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5597229336a0 0x55972295cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.608+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559722934ec0 0x55972293cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.608+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722942c30 0x55972296ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.608+0000 7f44068f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972292f240 con 0x5597229336a0 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.608+0000 7f44068f1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722957e20 con 0x559722934ec0 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.608+0000 7f44068f1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972292ad30 con 0x559722942c30 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.608+0000 7f43eca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722942c30 0x55972296ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.608+0000 7f43eca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722942c30 0x55972296ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32792/0 (socket says 172.21.15.131:32792) 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.608+0000 7f43eca00700 1 -- 172.21.15.131:0/1480810201 learned_addr learned my addr 172.21.15.131:0/1480810201 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.608+0000 7f43e7400700 1 --2- 172.21.15.131:0/1480810201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559722934ec0 0x55972293cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.609+0000 7f43e7e00700 1 --2- 172.21.15.131:0/1480810201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5597229336a0 0x55972295cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.609+0000 7f43eca00700 1 -- 172.21.15.131:0/1480810201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559722934ec0 msgr2=0x55972293cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.609+0000 7f43eca00700 1 --2- 172.21.15.131:0/1480810201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559722934ec0 0x55972293cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.609+0000 7f43eca00700 1 -- 172.21.15.131:0/1480810201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5597229336a0 msgr2=0x55972295cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.609+0000 7f43eca00700 1 --2- 172.21.15.131:0/1480810201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5597229336a0 0x55972295cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.609+0000 7f43eca00700 1 -- 172.21.15.131:0/1480810201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559722819560 con 0x559722942c30 2024-05-10T05:44:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.609+0000 7f43e7400700 1 --2- 172.21.15.131:0/1480810201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559722934ec0 0x55972293cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.609+0000 7f43eca00700 1 --2- 172.21.15.131:0/1480810201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722942c30 0x55972296ad00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f43e0099130 tx=0x7f43e000df60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cb07032d46c4fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.610+0000 7f43ddc00700 1 -- 172.21.15.131:0/1480810201 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0099430 con 0x559722942c30 2024-05-10T05:44:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.610+0000 7f43ddc00700 1 -- 172.21.15.131:0/1480810201 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e009d260 con 0x559722942c30 2024-05-10T05:44:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.610+0000 7f43ddc00700 1 -- 172.21.15.131:0/1480810201 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0005aa0 con 0x559722942c30 2024-05-10T05:44:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.611+0000 7f44068f1b80 1 -- 172.21.15.131:0/1480810201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722942c30 msgr2=0x55972296ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.611+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1480810201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722942c30 0x55972296ad00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f43e0099130 tx=0x7f43e000df60 comp rx=0 tx=0).stop 2024-05-10T05:44:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.611+0000 7f44068f1b80 1 -- 172.21.15.131:0/1480810201 shutdown_connections 2024-05-10T05:44:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.611+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1480810201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5597229336a0 0x55972295cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.611+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1480810201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722942c30 0x55972296ad00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.611+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1480810201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559722934ec0 0x55972293cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.611+0000 7f44068f1b80 1 -- 172.21.15.131:0/1480810201 >> 172.21.15.131:0/1480810201 conn(0x559722895a60 msgr2=0x55972292a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.611+0000 7f44068f1b80 1 -- 172.21.15.131:0/1480810201 shutdown_connections 2024-05-10T05:44:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.611+0000 7f44068f1b80 1 -- 172.21.15.131:0/1480810201 wait complete. 2024-05-10T05:44:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.612+0000 7f44068f1b80 1 -- start start 2024-05-10T05:44:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.613+0000 7f44068f1b80 1 Processor -- start 2024-05-10T05:44:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.613+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x55972296a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.614+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972296a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.614+0000 7f44068f1b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 0x5597229af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.614+0000 7f44068f1b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597229aff00 con 0x559722942c30 2024-05-10T05:44:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.614+0000 7f44068f1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597229b0040 con 0x5597229336a0 2024-05-10T05:44:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.614+0000 7f44068f1b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597229b0180 con 0x559722934ec0 2024-05-10T05:44:39.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.615+0000 7f44068f1b80 10 client.0 _ll_register_callbacks cb 0x5597227f8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:39.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.615+0000 7f43eca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 0x5597229af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.615+0000 7f43eca00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 0x5597229af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44882/0 (socket says 172.21.15.131:44882) 2024-05-10T05:44:39.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.615+0000 7f43eca00700 1 -- 172.21.15.131:0/1556149057 learned_addr learned my addr 172.21.15.131:0/1556149057 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:39.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.615+0000 7f43e7e00700 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x55972296a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.615+0000 7f43e7400700 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972296a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.616+0000 7f43eca00700 1 -- 172.21.15.131:0/1556149057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 msgr2=0x55972296a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:39.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.616+0000 7f43eca00700 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972296a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.616+0000 7f43eca00700 1 -- 172.21.15.131:0/1556149057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 msgr2=0x55972296a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:39.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.616+0000 7f43eca00700 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x55972296a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.616+0000 7f43eca00700 1 -- 172.21.15.131:0/1556149057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559722680b60 con 0x559722942c30 2024-05-10T05:44:39.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.616+0000 7f43e7e00700 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x55972296a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:39.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.616+0000 7f43eca00700 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 0x5597229af9c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43e0015040 tx=0x7f43e0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff37eb6963d17fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:39.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.616+0000 7f43d5600700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:44:39.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.617+0000 7f43d5600700 1 -- 172.21.15.131:0/1556149057 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e00b5070 con 0x559722942c30 2024-05-10T05:44:39.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.617+0000 7f43d5600700 1 -- 172.21.15.131:0/1556149057 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f43e00a18b0 con 0x559722942c30 2024-05-10T05:44:39.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.617+0000 7f44068f1b80 1 -- 172.21.15.131:0/1556149057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43d801e040 con 0x559722942c30 2024-05-10T05:44:39.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.618+0000 7f43d5600700 1 -- 172.21.15.131:0/1556149057 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43e0099430 con 0x559722942c30 2024-05-10T05:44:39.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.618+0000 7f44068f1b80 10 client.16989 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:39.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.618+0000 7f44068f1b80 1 -- 172.21.15.131:0/1556149057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f43e81a6910 con 0x559722942c30 2024-05-10T05:44:39.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.618+0000 7f43d5600700 1 -- 172.21.15.131:0/1556149057 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43d801e040 con 0x559722942c30 2024-05-10T05:44:39.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.618+0000 7f43d5600700 1 -- 172.21.15.131:0/1556149057 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f43e00a1530 con 0x559722942c30 2024-05-10T05:44:39.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.618+0000 7f44068f1b80 10 client.16989 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:39.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.618+0000 7f44068f1b80 10 client.16989 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:44:39.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.618+0000 7f44068f1b80 20 client.16989 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:39.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.619+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5597229b17b0 0x5597229b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:39.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.619+0000 7f44068f1b80 4 client.16989 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:44:39.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.619+0000 7f44068f1b80 1 -- 172.21.15.131:0/1556149057 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55972284bec0 con 0x5597229b17b0 2024-05-10T05:44:39.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.619+0000 7f43e7e00700 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5597229b17b0 0x5597229b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:39.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.620+0000 7f43e7e00700 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5597229b17b0 0x5597229b3bd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7220c123404ebc13 server_cookie=c7530e821e721486 in_seq=0 out_seq=0 2024-05-10T05:44:39.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.620+0000 7f43d5600700 10 client.16989 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:44:39.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.620+0000 7f43d5600700 1 -- 172.21.15.131:0/1556149057 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55972284bec0 con 0x5597229b17b0 2024-05-10T05:44:39.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.620+0000 7f43d5600700 10 client.16989 handle_command_reply: tid=0 2024-05-10T05:44:39.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.620+0000 7f44068f1b80 1 client.16989 shutdown 2024-05-10T05:44:39.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.621+0000 7f44068f1b80 1 -- 172.21.15.131:0/1556149057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 msgr2=0x5597229af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:39.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.621+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 0x5597229af9c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43e0015040 tx=0x7f43e0005bd0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.621+0000 7f44068f1b80 1 -- 172.21.15.131:0/1556149057 shutdown_connections 2024-05-10T05:44:39.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.621+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x5597229b17b0 0x5597229b3bd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.621+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559722942c30 0x5597229af9c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.621+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597229336a0 0x55972296a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.621+0000 7f44068f1b80 1 --2- 172.21.15.131:0/1556149057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559722934ec0 0x55972296a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.621+0000 7f43d5600700 0 client.16989 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:44:39.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.621+0000 7f44068f1b80 1 -- 172.21.15.131:0/1556149057 >> 172.21.15.131:0/1556149057 conn(0x559722895a60 msgr2=0x55972292a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:39.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.623+0000 7f44068f1b80 1 -- 172.21.15.131:0/1556149057 shutdown_connections 2024-05-10T05:44:39.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.623+0000 7f44068f1b80 1 -- 172.21.15.131:0/1556149057 wait complete. 2024-05-10T05:44:39.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.623+0000 7f44068f1b80 20 client.16989 trim_cache size 0 max 16384 2024-05-10T05:44:39.628 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:44:39.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.624+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43cc0a12c0 msgr2=0x7f43cc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:39.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.624+0000 7f43eea00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43cc0a12c0 0x7f43cc0a37d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x559722933380 tx=0x7f43d0015040 comp rx=0 tx=0).stop 2024-05-10T05:44:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.624+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e81a35a0 msgr2=0x7f43e81a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.624+0000 7f43eea00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e81a35a0 0x7f43e81a59c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f43e0007ee0 tx=0x7f43e000d3b0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.625+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 shutdown_connections 2024-05-10T05:44:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.625+0000 7f43eea00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f43cc0a12c0 0x7f43cc0a37d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.625+0000 7f43eea00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e8107040 0x7f43e819e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.625+0000 7f43eea00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e81a35a0 0x7f43e81a59c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.625+0000 7f43eea00700 1 --2- 172.21.15.131:0/3894793141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43e8109a80 0x7f43e819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.625+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 >> 172.21.15.131:0/3894793141 conn(0x7f43e80fc0d0 msgr2=0x7f43e8108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:39.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.625+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 shutdown_connections 2024-05-10T05:44:39.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:39.625+0000 7f43eea00700 1 -- 172.21.15.131:0/3894793141 wait complete. 2024-05-10T05:44:39.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:44:39.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:39 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4046547543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:44:40.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:39 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4046547543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:44:40.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.133+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1705147578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aac109720 msgr2=0x7f8aac109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:40.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.133+0000 7f8ab3a00700 1 --2- 172.21.15.131:0/1705147578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aac109720 0x7f8aac109b30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009990 tx=0x7f8aa0009ca0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.134+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1705147578 shutdown_connections 2024-05-10T05:44:40.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.134+0000 7f8ab3a00700 1 --2- 172.21.15.131:0/1705147578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aac10a0c0 0x7f8aac110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.134+0000 7f8ab3a00700 1 --2- 172.21.15.131:0/1705147578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac108630 0x7f8aac108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.134+0000 7f8ab3a00700 1 --2- 172.21.15.131:0/1705147578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aac109720 0x7f8aac109b30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.134+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1705147578 >> 172.21.15.131:0/1705147578 conn(0x7f8aac06ee40 msgr2=0x7f8aac0712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:40.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.137+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1705147578 shutdown_connections 2024-05-10T05:44:40.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.137+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1705147578 wait complete. 2024-05-10T05:44:40.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.139+0000 7f8ab3a00700 1 Processor -- start 2024-05-10T05:44:40.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:39 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4046547543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:44:40.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.143+0000 7f8ab3a00700 1 -- start start 2024-05-10T05:44:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.144+0000 7f8ab3a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aac108630 0x7f8aac114360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.144+0000 7f8ab3a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aac109720 0x7f8aac112a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.144+0000 7f8ab3a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac10a0c0 0x7f8aac112f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.144+0000 7f8ab3a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac1a45a0 con 0x7f8aac108630 2024-05-10T05:44:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.144+0000 7f8ab3a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac1a4710 con 0x7f8aac109720 2024-05-10T05:44:40.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.144+0000 7f8ab3a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac1a48d0 con 0x7f8aac10a0c0 2024-05-10T05:44:40.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.145+0000 7f8aabe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aac109720 0x7f8aac112a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.146+0000 7f8ab1a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac10a0c0 0x7f8aac112f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.146+0000 7f8ab1a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac10a0c0 0x7f8aac112f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32818/0 (socket says 172.21.15.131:32818) 2024-05-10T05:44:40.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.146+0000 7f8ab1a00700 1 -- 172.21.15.131:0/1617295734 learned_addr learned my addr 172.21.15.131:0/1617295734 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:40.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.146+0000 7f8ab1a00700 1 -- 172.21.15.131:0/1617295734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aac109720 msgr2=0x7f8aac112a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:40.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.146+0000 7f8ab1a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aac109720 0x7f8aac112a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.146+0000 7f8ab1a00700 1 -- 172.21.15.131:0/1617295734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aac108630 msgr2=0x7f8aac114360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:40.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.146+0000 7f8ab1a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aac108630 0x7f8aac114360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.146+0000 7f8ab1a00700 1 -- 172.21.15.131:0/1617295734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00095f0 con 0x7f8aac10a0c0 2024-05-10T05:44:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.147+0000 7f8aabe00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aac109720 0x7f8aac112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.147+0000 7f8ab1a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac10a0c0 0x7f8aac112f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4007ee0 tx=0x7f8aa400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:40.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.147+0000 7f8aa9600700 1 -- 172.21.15.131:0/1617295734 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400ddd0 con 0x7f8aac10a0c0 2024-05-10T05:44:40.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.147+0000 7f8aa9600700 1 -- 172.21.15.131:0/1617295734 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa400f040 con 0x7f8aac10a0c0 2024-05-10T05:44:40.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.147+0000 7f8aa9600700 1 -- 172.21.15.131:0/1617295734 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4015400 con 0x7f8aac10a0c0 2024-05-10T05:44:40.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.148+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1617295734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac1a4bb0 con 0x7f8aac10a0c0 2024-05-10T05:44:40.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.148+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1617295734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac1a5070 con 0x7f8aac10a0c0 2024-05-10T05:44:40.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.157+0000 7f8acb9ccb80 1 Processor -- start 2024-05-10T05:44:40.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.167+0000 7f8aa9600700 1 -- 172.21.15.131:0/1617295734 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8aa4004000 con 0x7f8aac10a0c0 2024-05-10T05:44:40.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.168+0000 7f8acb9ccb80 1 -- start start 2024-05-10T05:44:40.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.169+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 0x561df1d80af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.169+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d81980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.169+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81ec0 0x561df1d887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.169+0000 7f8acb9ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1ca7150 con 0x561df1d81570 2024-05-10T05:44:40.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.169+0000 7f8acb9ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1ca4b10 con 0x561df1d81ec0 2024-05-10T05:44:40.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.169+0000 7f8acb9ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1ca0400 con 0x561df1d806a0 2024-05-10T05:44:40.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.172+0000 7f8aabe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81ec0 0x561df1d887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.172+0000 7f8aabe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81ec0 0x561df1d887c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41590/0 (socket says 172.21.15.131:41590) 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.172+0000 7f8aabe00700 1 -- 172.21.15.131:0/3382027184 learned_addr learned my addr 172.21.15.131:0/3382027184 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.173+0000 7f8aabe00700 1 --2- 172.21.15.131:0/3382027184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 0x561df1d80af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.174+0000 7f8aa9600700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a940a1240 0x7f8a940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.174+0000 7f8aa9600700 1 -- 172.21.15.131:0/1617295734 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8aa40cc260 con 0x7f8aac10a0c0 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.175+0000 7f8ab1000700 1 --2- 172.21.15.131:0/3382027184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d81980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.175+0000 7f8ab1a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a940a1240 0x7f8a940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.175+0000 7f8ab1000700 1 -- 172.21.15.131:0/3382027184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 msgr2=0x561df1d80af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.175+0000 7f8ab1000700 1 --2- 172.21.15.131:0/3382027184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 0x561df1d80af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.175+0000 7f8ab1000700 1 -- 172.21.15.131:0/3382027184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81ec0 msgr2=0x561df1d887c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.175+0000 7f8ab1000700 1 --2- 172.21.15.131:0/3382027184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81ec0 0x561df1d887c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.175+0000 7f8ab1000700 1 -- 172.21.15.131:0/3382027184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df1b213b0 con 0x561df1d81570 2024-05-10T05:44:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.176+0000 7f8ab1a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a940a1240 0x7f8a940a3750 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4095bb0 tx=0x7f8aa400df30 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:40.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.176+0000 7f8ab1000700 1 --2- 172.21.15.131:0/3382027184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d81980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8aac108320 tx=0x7f8a98009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f6b2b276c3540e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:40.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.181+0000 7f8a9e000700 1 -- 172.21.15.131:0/3382027184 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800eb00 con 0x561df1d81570 2024-05-10T05:44:40.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.181+0000 7f8a9e000700 1 -- 172.21.15.131:0/3382027184 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a98004cf0 con 0x561df1d81570 2024-05-10T05:44:40.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.181+0000 7f8a9e000700 1 -- 172.21.15.131:0/3382027184 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98005670 con 0x561df1d81570 2024-05-10T05:44:40.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.183+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3382027184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 msgr2=0x561df1d81980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:40.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.183+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/3382027184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d81980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8aac108320 tx=0x7f8a98009ea0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.197+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3382027184 shutdown_connections 2024-05-10T05:44:40.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.197+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/3382027184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81ec0 0x561df1d887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.197+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/3382027184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 0x561df1d80af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.197+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/3382027184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d81980 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.197+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3382027184 >> 172.21.15.131:0/3382027184 conn(0x561df1ce2a60 msgr2=0x561df1d7afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:40.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.201+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3382027184 shutdown_connections 2024-05-10T05:44:40.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.201+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3382027184 wait complete. 2024-05-10T05:44:40.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.205+0000 7f8acb9ccb80 1 -- start start 2024-05-10T05:44:40.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.206+0000 7f8acb9ccb80 1 Processor -- start 2024-05-10T05:44:40.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.213+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d806a0 0x561df1d8a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.213+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d8ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.213+0000 7f8aabe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d806a0 0x561df1d8a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.213+0000 7f8aabe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d806a0 0x561df1d8a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41604/0 (socket says 172.21.15.131:41604) 2024-05-10T05:44:40.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.214+0000 7f8aabe00700 1 -- 172.21.15.131:0/34917303 learned_addr learned my addr 172.21.15.131:0/34917303 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.214+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d81ec0 0x561df1d91bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.214+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1c3c560 con 0x561df1d81570 2024-05-10T05:44:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.214+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1c3c200 con 0x561df1d806a0 2024-05-10T05:44:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.214+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1b236a0 con 0x561df1d81ec0 2024-05-10T05:44:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.214+0000 7f8acb9ccb80 10 client.0 _ll_register_callbacks cb 0x561df1c45b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.217+0000 7f8aabe00700 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d81ec0 0x561df1d91bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.218+0000 7f8aabe00700 1 -- 172.21.15.131:0/34917303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d81ec0 msgr2=0x561df1d91bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.218+0000 7f8aabe00700 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d81ec0 0x561df1d91bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.218+0000 7f8aabe00700 1 -- 172.21.15.131:0/34917303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 msgr2=0x561df1d8ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.218+0000 7f8aabe00700 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d8ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.218+0000 7f8aabe00700 1 -- 172.21.15.131:0/34917303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a980098a0 con 0x561df1d806a0 2024-05-10T05:44:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.219+0000 7f8aabe00700 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d806a0 0x561df1d8a720 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009990 tx=0x7f8aa0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1255ed3d457aec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:40.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.219+0000 7f8a92a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:44:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.223+0000 7f8a92a00700 1 -- 172.21.15.131:0/34917303 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0010b10 con 0x561df1d806a0 2024-05-10T05:44:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.223+0000 7f8a92a00700 1 -- 172.21.15.131:0/34917303 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8aa0010c70 con 0x561df1d806a0 2024-05-10T05:44:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.223+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561df1bf9270 con 0x561df1d806a0 2024-05-10T05:44:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.224+0000 7f8a92a00700 1 -- 172.21.15.131:0/34917303 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aa0010e30 con 0x561df1d806a0 2024-05-10T05:44:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.224+0000 7f8acb9ccb80 10 client.27374 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:40.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.224+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561df1acdb60 con 0x561df1d806a0 2024-05-10T05:44:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.224+0000 7f8a92a00700 1 -- 172.21.15.131:0/34917303 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561df1bf9270 con 0x561df1d806a0 2024-05-10T05:44:40.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.226+0000 7f8a92a00700 1 -- 172.21.15.131:0/34917303 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8aa0004af0 con 0x561df1d806a0 2024-05-10T05:44:40.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.226+0000 7f8acb9ccb80 10 client.27374 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:40.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.226+0000 7f8acb9ccb80 10 client.27374 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:44:40.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.226+0000 7f8acb9ccb80 20 client.27374 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:40.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.226+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561df1d96330 0x561df1d98780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.226+0000 7f8acb9ccb80 4 client.27374 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:40.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.226+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561df1d4f780 con 0x561df1d96330 2024-05-10T05:44:40.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.227+0000 7f8ab1000700 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561df1d96330 0x561df1d98780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.228+0000 7f8a92a00700 10 client.27374 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:44:40.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.228+0000 7f8ab1000700 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561df1d96330 0x561df1d98780 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=738305e5efe62537 server_cookie=f657d03f24f63e08 in_seq=0 out_seq=0 2024-05-10T05:44:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.229+0000 7f8a92a00700 1 -- 172.21.15.131:0/34917303 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561df1d4f780 con 0x561df1d96330 2024-05-10T05:44:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.229+0000 7f8a92a00700 10 client.27374 handle_command_reply: tid=0 2024-05-10T05:44:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.229+0000 7f8acb9ccb80 1 client.27374 shutdown 2024-05-10T05:44:40.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.230+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d806a0 msgr2=0x561df1d8a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:40.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.230+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d806a0 0x561df1d8a720 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009990 tx=0x7f8aa0009310 comp rx=0 tx=0).stop 2024-05-10T05:44:40.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.230+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 shutdown_connections 2024-05-10T05:44:40.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.230+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561df1d96330 0x561df1d98780 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.230+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d81ec0 0x561df1d91bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.230+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d806a0 0x561df1d8a720 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.230+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/34917303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d8ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.230+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 >> 172.21.15.131:0/34917303 conn(0x561df1ce2a60 msgr2=0x561df1d7afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.230+0000 7f8a92a00700 0 client.27374 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:44:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.231+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 shutdown_connections 2024-05-10T05:44:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.231+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/34917303 wait complete. 2024-05-10T05:44:40.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.232+0000 7f8acb9ccb80 20 client.27374 trim_cache size 0 max 16384 2024-05-10T05:44:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.337+0000 7f8acb9ccb80 1 Processor -- start 2024-05-10T05:44:40.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.337+0000 7f8acb9ccb80 1 -- start start 2024-05-10T05:44:40.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.339+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d806a0 0x561df1d853b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.339+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81570 0x561df1d847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.339+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d81ec0 0x561df1d91b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.339+0000 7f8acb9ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1d98860 con 0x561df1d806a0 2024-05-10T05:44:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.339+0000 7f8acb9ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1d858f0 con 0x561df1d81570 2024-05-10T05:44:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.339+0000 7f8acb9ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1d85ab0 con 0x561df1d81ec0 2024-05-10T05:44:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.343+0000 7f8aabe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d806a0 0x561df1d853b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.343+0000 7f8aabe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d806a0 0x561df1d853b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44936/0 (socket says 172.21.15.131:44936) 2024-05-10T05:44:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.343+0000 7f8aabe00700 1 -- 172.21.15.131:0/1014226377 learned_addr learned my addr 172.21.15.131:0/1014226377 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.346+0000 7f8ab1000700 1 --2- 172.21.15.131:0/1014226377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81570 0x561df1d847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.346+0000 7f8aabe00700 1 --2- 172.21.15.131:0/1014226377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d81ec0 0x561df1d91b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.346+0000 7f8aabe00700 1 -- 172.21.15.131:0/1014226377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d81ec0 msgr2=0x561df1d91b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.346+0000 7f8aabe00700 1 --2- 172.21.15.131:0/1014226377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d81ec0 0x561df1d91b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.346+0000 7f8aabe00700 1 -- 172.21.15.131:0/1014226377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81570 msgr2=0x561df1d847a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.346+0000 7f8aabe00700 1 --2- 172.21.15.131:0/1014226377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81570 0x561df1d847a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.346+0000 7f8aabe00700 1 -- 172.21.15.131:0/1014226377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df1c66560 con 0x561df1d806a0 2024-05-10T05:44:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.347+0000 7f8aabe00700 1 --2- 172.21.15.131:0/1014226377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d806a0 0x561df1d853b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000b7b0 tx=0x7f8aa0004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0addcb06a339379 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.351+0000 7f8a9ea00700 1 -- 172.21.15.131:0/1014226377 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0010e30 con 0x561df1d806a0 2024-05-10T05:44:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.351+0000 7f8a9ea00700 1 -- 172.21.15.131:0/1014226377 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561df1bf9270 con 0x561df1d806a0 2024-05-10T05:44:40.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.351+0000 7f8a9ea00700 1 -- 172.21.15.131:0/1014226377 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00052b0 con 0x561df1d806a0 2024-05-10T05:44:40.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.358+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/1014226377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d806a0 msgr2=0x561df1d853b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:40.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.358+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/1014226377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d806a0 0x561df1d853b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000b7b0 tx=0x7f8aa0004820 comp rx=0 tx=0).stop 2024-05-10T05:44:40.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.361+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/1014226377 shutdown_connections 2024-05-10T05:44:40.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.361+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/1014226377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d806a0 0x561df1d853b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.361+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/1014226377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d81ec0 0x561df1d91b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.361+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/1014226377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81570 0x561df1d847a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.361+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/1014226377 >> 172.21.15.131:0/1014226377 conn(0x561df1ce2a60 msgr2=0x561df1d7af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:40.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.362+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/1014226377 shutdown_connections 2024-05-10T05:44:40.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.363+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/1014226377 wait complete. 2024-05-10T05:44:40.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.364+0000 7f8acb9ccb80 1 -- start start 2024-05-10T05:44:40.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.364+0000 7f8acb9ccb80 1 Processor -- start 2024-05-10T05:44:40.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.375+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 0x561df1d8fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.375+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d90190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.376+0000 7f8acb9ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81ec0 0x561df1d910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.376+0000 7f8acb9ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1d91600 con 0x561df1d81570 2024-05-10T05:44:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.376+0000 7f8acb9ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1d91770 con 0x561df1d81ec0 2024-05-10T05:44:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.376+0000 7f8acb9ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1d91930 con 0x561df1d806a0 2024-05-10T05:44:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.376+0000 7f8ab1000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d90190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.376+0000 7f8acb9ccb80 10 client.0 _ll_register_callbacks cb 0x561df1c45b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.376+0000 7f8ab1000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d90190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44950/0 (socket says 172.21.15.131:44950) 2024-05-10T05:44:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.376+0000 7f8ab1000700 1 -- 172.21.15.131:0/3899256973 learned_addr learned my addr 172.21.15.131:0/3899256973 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.377+0000 7f8aabe00700 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 0x561df1d8fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.377+0000 7f8ab1000700 1 -- 172.21.15.131:0/3899256973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 msgr2=0x561df1d8fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.377+0000 7f8ab1000700 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 0x561df1d8fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.377+0000 7f8ab1000700 1 -- 172.21.15.131:0/3899256973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81ec0 msgr2=0x561df1d910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:40.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.379+0000 7f8ab1000700 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81ec0 0x561df1d910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.379+0000 7f8ab1000700 1 -- 172.21.15.131:0/3899256973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561df1acdb60 con 0x561df1d81570 2024-05-10T05:44:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.379+0000 7f8ab1000700 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d90190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009710 tx=0x7f8a98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ea3c2bd481445e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.380+0000 7f8aabe00700 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 0x561df1d8fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.380+0000 7f8a92a00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:44:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.380+0000 7f8a92a00700 1 -- 172.21.15.131:0/3899256973 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98004ab0 con 0x561df1d81570 2024-05-10T05:44:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.380+0000 7f8a92a00700 1 -- 172.21.15.131:0/3899256973 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f8a98014070 con 0x561df1d81570 2024-05-10T05:44:40.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.385+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3899256973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a98004cf0 con 0x561df1d81570 2024-05-10T05:44:40.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.385+0000 7f8a92a00700 1 -- 172.21.15.131:0/3899256973 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a9800e900 con 0x561df1d81570 2024-05-10T05:44:40.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.385+0000 7f8acb9ccb80 10 client.17001 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:40.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.385+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3899256973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561df1b25cd0 con 0x561df1d81570 2024-05-10T05:44:40.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.389+0000 7f8a92a00700 1 -- 172.21.15.131:0/3899256973 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a98004cf0 con 0x561df1d81570 2024-05-10T05:44:40.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.389+0000 7f8a92a00700 1 -- 172.21.15.131:0/3899256973 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8a98027050 con 0x561df1d81570 2024-05-10T05:44:40.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.389+0000 7f8acb9ccb80 10 client.17001 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:40.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.389+0000 7f8acb9ccb80 10 client.17001 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:44:40.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.389+0000 7f8acb9ccb80 20 client.17001 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:40.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.389+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561df1ded100 0x561df1def520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:40.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.389+0000 7f8acb9ccb80 4 client.17001 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:44:40.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.389+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3899256973 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561df1c98ec0 con 0x561df1ded100 2024-05-10T05:44:40.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.389+0000 7f8aabe00700 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561df1ded100 0x561df1def520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:40.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.390+0000 7f8a92a00700 10 client.17001 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:44:40.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.390+0000 7f8aabe00700 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561df1ded100 0x561df1def520 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=832d55569dc8409a server_cookie=5b786a760f4a8c88 in_seq=0 out_seq=0 2024-05-10T05:44:40.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.390+0000 7f8a92a00700 1 -- 172.21.15.131:0/3899256973 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561df1c98ec0 con 0x561df1ded100 2024-05-10T05:44:40.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.390+0000 7f8a92a00700 10 client.17001 handle_command_reply: tid=0 2024-05-10T05:44:40.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.390+0000 7f8acb9ccb80 1 client.17001 shutdown 2024-05-10T05:44:40.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.393+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3899256973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 msgr2=0x561df1d90190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:40.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.393+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d90190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009710 tx=0x7f8a98004750 comp rx=0 tx=0).stop 2024-05-10T05:44:40.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.393+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3899256973 shutdown_connections 2024-05-10T05:44:40.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.393+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x561df1ded100 0x561df1def520 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.394+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561df1d81ec0 0x561df1d910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.394+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561df1d806a0 0x561df1d8fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.394+0000 7f8acb9ccb80 1 --2- 172.21.15.131:0/3899256973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561df1d81570 0x561df1d90190 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.394+0000 7f8a92a00700 0 client.17001 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:44:40.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.394+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3899256973 >> 172.21.15.131:0/3899256973 conn(0x561df1ce2a60 msgr2=0x561df1d7af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:40.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.394+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3899256973 shutdown_connections 2024-05-10T05:44:40.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.394+0000 7f8acb9ccb80 1 -- 172.21.15.131:0/3899256973 wait complete. 2024-05-10T05:44:40.397 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:44:40.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.395+0000 7f8acb9ccb80 20 client.17001 trim_cache size 0 max 16384 2024-05-10T05:44:40.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.397+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1617295734 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a940a1240 msgr2=0x7f8a940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:40.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.397+0000 7f8ab3a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a940a1240 0x7f8a940a3750 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4095bb0 tx=0x7f8aa400df30 comp rx=0 tx=0).stop 2024-05-10T05:44:40.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.397+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1617295734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac10a0c0 msgr2=0x7f8aac112f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:40.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.397+0000 7f8ab3a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac10a0c0 0x7f8aac112f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4007ee0 tx=0x7f8aa400d3b0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.397+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1617295734 shutdown_connections 2024-05-10T05:44:40.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.397+0000 7f8ab3a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f8a940a1240 0x7f8a940a3750 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.397+0000 7f8ab3a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8aac108630 0x7f8aac114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.397+0000 7f8ab3a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac10a0c0 0x7f8aac112f40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.397+0000 7f8ab3a00700 1 --2- 172.21.15.131:0/1617295734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aac109720 0x7f8aac112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.397+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1617295734 >> 172.21.15.131:0/1617295734 conn(0x7f8aac06ee40 msgr2=0x7f8aac10c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.398+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1617295734 shutdown_connections 2024-05-10T05:44:40.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:40.398+0000 7f8ab3a00700 1 -- 172.21.15.131:0/1617295734 wait complete. 2024-05-10T05:44:40.455 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:44:41.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:40 smithi157 ceph-mon[78427]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 249 MiB/s wr, 76 op/s 2024-05-10T05:44:41.147 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:40 smithi142 ceph-mon[77667]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 249 MiB/s wr, 76 op/s 2024-05-10T05:44:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.150+0000 7f0b5d400700 1 -- 172.21.15.131:0/2892537080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b58102d00 msgr2=0x7f0b58103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.150+0000 7f0b5d400700 1 --2- 172.21.15.131:0/2892537080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b58102d00 0x7f0b58103110 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009920 tx=0x7f0b44009c30 comp rx=0 tx=0).stop 2024-05-10T05:44:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.150+0000 7f0b5d400700 1 -- 172.21.15.131:0/2892537080 shutdown_connections 2024-05-10T05:44:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.150+0000 7f0b5d400700 1 --2- 172.21.15.131:0/2892537080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b581036a0 0x7f0b58109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.150+0000 7f0b5d400700 1 --2- 172.21.15.131:0/2892537080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58101be0 0x7f0b58101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.150+0000 7f0b5d400700 1 --2- 172.21.15.131:0/2892537080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b58102d00 0x7f0b58103110 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.150+0000 7f0b5d400700 1 -- 172.21.15.131:0/2892537080 >> 172.21.15.131:0/2892537080 conn(0x7f0b580fd1d0 msgr2=0x7f0b580ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.151+0000 7f0b5d400700 1 -- 172.21.15.131:0/2892537080 shutdown_connections 2024-05-10T05:44:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.151+0000 7f0b5d400700 1 -- 172.21.15.131:0/2892537080 wait complete. 2024-05-10T05:44:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.153+0000 7f0b5d400700 1 Processor -- start 2024-05-10T05:44:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.153+0000 7f0b5d400700 1 -- start start 2024-05-10T05:44:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.154+0000 7f0b5d400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b58101be0 0x7f0b58197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.154+0000 7f0b5d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58102d00 0x7f0b581984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.154+0000 7f0b5d400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b581036a0 0x7f0b5819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.154+0000 7f0b5d400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5819fa80 con 0x7f0b58101be0 2024-05-10T05:44:41.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.154+0000 7f0b5d400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5819fbf0 con 0x7f0b58102d00 2024-05-10T05:44:41.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.154+0000 7f0b5d400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5819fdb0 con 0x7f0b581036a0 2024-05-10T05:44:41.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.154+0000 7f0b55e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58102d00 0x7f0b581984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.154+0000 7f0b55e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58102d00 0x7f0b581984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41640/0 (socket says 172.21.15.131:41640) 2024-05-10T05:44:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.154+0000 7f0b55e00700 1 -- 172.21.15.131:0/59409360 learned_addr learned my addr 172.21.15.131:0/59409360 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.154+0000 7f0b56800700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b58101be0 0x7f0b58197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.155+0000 7f0b57200700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b581036a0 0x7f0b5819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.155+0000 7f0b55e00700 1 -- 172.21.15.131:0/59409360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b581036a0 msgr2=0x7f0b5819f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.155+0000 7f0b55e00700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b581036a0 0x7f0b5819f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.155+0000 7f0b55e00700 1 -- 172.21.15.131:0/59409360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b58101be0 msgr2=0x7f0b58197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.155+0000 7f0b55e00700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b58101be0 0x7f0b58197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.155+0000 7f0b55e00700 1 -- 172.21.15.131:0/59409360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b44009520 con 0x7f0b58102d00 2024-05-10T05:44:41.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.156+0000 7f0b55e00700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58102d00 0x7f0b581984e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009230 tx=0x7f0b4400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.156+0000 7f0b4f400700 1 -- 172.21.15.131:0/59409360 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4401c070 con 0x7f0b58102d00 2024-05-10T05:44:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.156+0000 7f0b4f400700 1 -- 172.21.15.131:0/59409360 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4400f7d0 con 0x7f0b58102d00 2024-05-10T05:44:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.156+0000 7f0b5d400700 1 -- 172.21.15.131:0/59409360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b58073440 con 0x7f0b58102d00 2024-05-10T05:44:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.156+0000 7f0b5d400700 1 -- 172.21.15.131:0/59409360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b580738a0 con 0x7f0b58102d00 2024-05-10T05:44:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.156+0000 7f0b4f400700 1 -- 172.21.15.131:0/59409360 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b440038e0 con 0x7f0b58102d00 2024-05-10T05:44:41.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.161+0000 7f0b4f400700 1 -- 172.21.15.131:0/59409360 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0b44020030 con 0x7f0b58102d00 2024-05-10T05:44:41.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.162+0000 7f0b753f0b80 1 Processor -- start 2024-05-10T05:44:41.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.164+0000 7f0b753f0b80 1 -- start start 2024-05-10T05:44:41.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.164+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e03a800 0x55f87e03ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.164+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 0x55f87e039790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.164+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e055ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.164+0000 7f0b753f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87df63150 con 0x55f87e039cd0 2024-05-10T05:44:41.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.164+0000 7f0b753f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87df60b10 con 0x55f87e03a800 2024-05-10T05:44:41.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.164+0000 7f0b753f0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87df5c400 con 0x55f87e039380 2024-05-10T05:44:41.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.166+0000 7f0b55e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e055ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.167+0000 7f0b55e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e055ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44974/0 (socket says 172.21.15.131:44974) 2024-05-10T05:44:41.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.167+0000 7f0b55e00700 1 -- 172.21.15.131:0/1043387564 learned_addr learned my addr 172.21.15.131:0/1043387564 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:41.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.167+0000 7f0b4f400700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0b3c0a1240 0x7f0b3c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.167+0000 7f0b55e00700 1 -- 172.21.15.131:0/1043387564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 msgr2=0x55f87e039790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:44:41.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.167+0000 7f0b55e00700 1 --2- 172.21.15.131:0/1043387564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 0x55f87e039790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.167+0000 7f0b55e00700 1 -- 172.21.15.131:0/1043387564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e03a800 msgr2=0x55f87e03ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:41.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.167+0000 7f0b55e00700 1 --2- 172.21.15.131:0/1043387564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e03a800 0x55f87e03ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.167+0000 7f0b55e00700 1 -- 172.21.15.131:0/1043387564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f87dddd3b0 con 0x55f87e039cd0 2024-05-10T05:44:41.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.168+0000 7f0b55e00700 1 --2- 172.21.15.131:0/1043387564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e055ae0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0b44021c00 tx=0x7f0b4402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca7398edad00ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:41.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.168+0000 7f0b4f400700 1 -- 172.21.15.131:0/59409360 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0b440cd770 con 0x7f0b58102d00 2024-05-10T05:44:41.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.168+0000 7f0b43400700 1 -- 172.21.15.131:0/1043387564 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4409a730 con 0x55f87e039cd0 2024-05-10T05:44:41.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.169+0000 7f0b43400700 1 -- 172.21.15.131:0/1043387564 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4409a550 con 0x55f87e039cd0 2024-05-10T05:44:41.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.169+0000 7f0b43400700 1 -- 172.21.15.131:0/1043387564 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400ed90 con 0x55f87e039cd0 2024-05-10T05:44:41.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.170+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1043387564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 msgr2=0x55f87e055ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:41.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.170+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1043387564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e055ae0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0b44021c00 tx=0x7f0b4402c000 comp rx=0 tx=0).stop 2024-05-10T05:44:41.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.171+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1043387564 shutdown_connections 2024-05-10T05:44:41.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.171+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1043387564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e055ae0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.171+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1043387564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e03a800 0x55f87e03ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.171+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1043387564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 0x55f87e039790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.171+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1043387564 >> 172.21.15.131:0/1043387564 conn(0x55f87df9ea60 msgr2=0x55f87e036fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:41.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.173+0000 7f0b57200700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0b3c0a1240 0x7f0b3c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.175+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1043387564 shutdown_connections 2024-05-10T05:44:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.175+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1043387564 wait complete. 2024-05-10T05:44:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.176+0000 7f0b753f0b80 1 -- start start 2024-05-10T05:44:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.176+0000 7f0b753f0b80 1 Processor -- start 2024-05-10T05:44:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.177+0000 7f0b57200700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0b3c0a1240 0x7f0b3c0a3750 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009ec0 tx=0x7f0b50009920 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.178+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039380 0x55f87e0572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.178+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 0x55f87e057810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.179+0000 7f0b56800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039380 0x55f87e0572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.179+0000 7f0b56800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039380 0x55f87e0572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44978/0 (socket says 172.21.15.131:44978) 2024-05-10T05:44:41.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.179+0000 7f0b56800700 1 -- 172.21.15.131:0/1963782813 learned_addr learned my addr 172.21.15.131:0/1963782813 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:41.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.179+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 0x55f87e05e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.179+0000 7f0b55e00700 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 0x55f87e057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.179+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87def8560 con 0x55f87e039380 2024-05-10T05:44:41.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.179+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87def8200 con 0x55f87e039cd0 2024-05-10T05:44:41.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.179+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87dddf6a0 con 0x55f87e03a800 2024-05-10T05:44:41.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.179+0000 7f0b753f0b80 10 client.0 _ll_register_callbacks cb 0x55f87df01b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:41.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.179+0000 7f0b57200700 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 0x55f87e05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.180+0000 7f0b57200700 1 -- 172.21.15.131:0/1963782813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 msgr2=0x55f87e057810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.180+0000 7f0b57200700 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 0x55f87e057810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.180+0000 7f0b57200700 1 -- 172.21.15.131:0/1963782813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039380 msgr2=0x55f87e0572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.180+0000 7f0b57200700 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039380 0x55f87e0572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.180+0000 7f0b57200700 1 -- 172.21.15.131:0/1963782813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b580738a0 con 0x55f87e03a800 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.180+0000 7f0b56800700 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039380 0x55f87e0572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.181+0000 7f0b57200700 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 0x55f87e05e770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009fd0 tx=0x7f0b5000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2aa1a7990bb8e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.181+0000 7f0b37e00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.181+0000 7f0b37e00700 1 -- 172.21.15.131:0/1963782813 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50003e20 con 0x55f87e03a800 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.181+0000 7f0b37e00700 1 -- 172.21.15.131:0/1963782813 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0b5001f410 con 0x55f87e03a800 2024-05-10T05:44:41.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.181+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f87deb5270 con 0x55f87e03a800 2024-05-10T05:44:41.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.182+0000 7f0b37e00700 1 -- 172.21.15.131:0/1963782813 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b5001f5d0 con 0x55f87e03a800 2024-05-10T05:44:41.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.182+0000 7f0b753f0b80 10 client.27400 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:41.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.182+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f87dd89b60 con 0x55f87e03a800 2024-05-10T05:44:41.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.182+0000 7f0b37e00700 1 -- 172.21.15.131:0/1963782813 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f87deb5270 con 0x55f87e03a800 2024-05-10T05:44:41.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.182+0000 7f0b37e00700 1 -- 172.21.15.131:0/1963782813 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0b5001b760 con 0x55f87e03a800 2024-05-10T05:44:41.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.183+0000 7f0b753f0b80 10 client.27400 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:41.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.183+0000 7f0b753f0b80 10 client.27400 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:44:41.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.183+0000 7f0b753f0b80 20 client.27400 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:41.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.183+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55f87e062fb0 0x55f87e065400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.183+0000 7f0b753f0b80 4 client.27400 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:41.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.183+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f87e00b780 con 0x55f87e062fb0 2024-05-10T05:44:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.184+0000 7f0b56800700 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55f87e062fb0 0x55f87e065400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.185+0000 7f0b37e00700 10 client.27400 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:44:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.185+0000 7f0b56800700 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55f87e062fb0 0x55f87e065400 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=721c2e9667fe384c server_cookie=eeff5a2240f33b1d in_seq=0 out_seq=0 2024-05-10T05:44:41.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.187+0000 7f0b37e00700 1 -- 172.21.15.131:0/1963782813 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f87e00b780 con 0x55f87e062fb0 2024-05-10T05:44:41.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.187+0000 7f0b37e00700 10 client.27400 handle_command_reply: tid=0 2024-05-10T05:44:41.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.187+0000 7f0b753f0b80 1 client.27400 shutdown 2024-05-10T05:44:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.187+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 msgr2=0x55f87e05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.187+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 0x55f87e05e770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009fd0 tx=0x7f0b5000b040 comp rx=0 tx=0).stop 2024-05-10T05:44:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.188+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 shutdown_connections 2024-05-10T05:44:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.188+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55f87e062fb0 0x55f87e065400 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.188+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039380 0x55f87e0572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.188+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 0x55f87e05e770 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.188+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1963782813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 0x55f87e057810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.188+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 >> 172.21.15.131:0/1963782813 conn(0x55f87df9ea60 msgr2=0x55f87e036fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.188+0000 7f0b37e00700 0 client.27400 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:44:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.188+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 shutdown_connections 2024-05-10T05:44:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.188+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1963782813 wait complete. 2024-05-10T05:44:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.189+0000 7f0b753f0b80 20 client.27400 trim_cache size 0 max 16384 2024-05-10T05:44:41.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:40 smithi131 ceph-mon[37264]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 249 MiB/s wr, 76 op/s 2024-05-10T05:44:41.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.229+0000 7f0b753f0b80 1 Processor -- start 2024-05-10T05:44:41.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.230+0000 7f0b753f0b80 1 -- start start 2024-05-10T05:44:41.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.230+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039380 0x55f87e0411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.230+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e0416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.230+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 0x55f87e064fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.230+0000 7f0b753f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e057ed0 con 0x55f87e039cd0 2024-05-10T05:44:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.230+0000 7f0b753f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e05e730 con 0x55f87e039380 2024-05-10T05:44:41.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.230+0000 7f0b753f0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e03ac10 con 0x55f87e03a800 2024-05-10T05:44:41.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.231+0000 7f0b55e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e0416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.231+0000 7f0b57200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 0x55f87e064fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.231+0000 7f0b55e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e0416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44986/0 (socket says 172.21.15.131:44986) 2024-05-10T05:44:41.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.232+0000 7f0b55e00700 1 -- 172.21.15.131:0/1822561719 learned_addr learned my addr 172.21.15.131:0/1822561719 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:41.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.233+0000 7f0b56800700 1 --2- 172.21.15.131:0/1822561719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039380 0x55f87e0411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.234+0000 7f0b55e00700 1 -- 172.21.15.131:0/1822561719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 msgr2=0x55f87e064fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:41.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.234+0000 7f0b55e00700 1 --2- 172.21.15.131:0/1822561719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 0x55f87e064fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.234+0000 7f0b55e00700 1 -- 172.21.15.131:0/1822561719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039380 msgr2=0x55f87e0411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:41.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.234+0000 7f0b55e00700 1 --2- 172.21.15.131:0/1822561719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039380 0x55f87e0411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.234+0000 7f0b55e00700 1 -- 172.21.15.131:0/1822561719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f87df22560 con 0x55f87e039cd0 2024-05-10T05:44:41.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.236+0000 7f0b55e00700 1 --2- 172.21.15.131:0/1822561719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e0416e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0b4409a8f0 tx=0x7f0b44009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d908f49e51d9f8d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:41.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.237+0000 7f0b43e00700 1 -- 172.21.15.131:0/1822561719 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4409a730 con 0x55f87e039cd0 2024-05-10T05:44:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.238+0000 7f0b43e00700 1 -- 172.21.15.131:0/1822561719 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4409b2f0 con 0x55f87e039cd0 2024-05-10T05:44:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.238+0000 7f0b43e00700 1 -- 172.21.15.131:0/1822561719 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44099020 con 0x55f87e039cd0 2024-05-10T05:44:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.239+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1822561719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 msgr2=0x55f87e0416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.239+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1822561719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e0416e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0b4409a8f0 tx=0x7f0b44009c30 comp rx=0 tx=0).stop 2024-05-10T05:44:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.239+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1822561719 shutdown_connections 2024-05-10T05:44:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.239+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1822561719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e03a800 0x55f87e064fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.240+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1822561719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039380 0x55f87e0411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.240+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/1822561719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e039cd0 0x55f87e0416e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.240+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1822561719 >> 172.21.15.131:0/1822561719 conn(0x55f87df9ea60 msgr2=0x55f87e055ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:41.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.240+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1822561719 shutdown_connections 2024-05-10T05:44:41.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.240+0000 7f0b753f0b80 1 -- 172.21.15.131:0/1822561719 wait complete. 2024-05-10T05:44:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.241+0000 7f0b753f0b80 1 -- start start 2024-05-10T05:44:41.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.241+0000 7f0b753f0b80 1 Processor -- start 2024-05-10T05:44:41.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.242+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 0x55f87e03fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.242+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 0x55f87e0401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.242+0000 7f0b753f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e03a800 0x55f87e0b7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.243+0000 7f0b55e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 0x55f87e0401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.244+0000 7f0b753f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e0b7bc0 con 0x55f87e03a800 2024-05-10T05:44:41.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.244+0000 7f0b753f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e0b7d30 con 0x55f87e039cd0 2024-05-10T05:44:41.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.244+0000 7f0b753f0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87e0b7ef0 con 0x55f87e039380 2024-05-10T05:44:41.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.244+0000 7f0b753f0b80 10 client.0 _ll_register_callbacks cb 0x55f87df01b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:41.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.244+0000 7f0b56800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 0x55f87e03fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.244+0000 7f0b56800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 0x55f87e03fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32922/0 (socket says 172.21.15.131:32922) 2024-05-10T05:44:41.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.244+0000 7f0b56800700 1 -- 172.21.15.131:0/701913839 learned_addr learned my addr 172.21.15.131:0/701913839 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:41.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.243+0000 7f0b57200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e03a800 0x55f87e0b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.245+0000 7f0b57200700 1 -- 172.21.15.131:0/701913839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 msgr2=0x55f87e03fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.245+0000 7f0b57200700 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 0x55f87e03fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.245+0000 7f0b57200700 1 -- 172.21.15.131:0/701913839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 msgr2=0x55f87e0401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.245+0000 7f0b57200700 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 0x55f87e0401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.245+0000 7f0b57200700 1 -- 172.21.15.131:0/701913839 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f87dddd3b0 con 0x55f87e03a800 2024-05-10T05:44:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.245+0000 7f0b56800700 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 0x55f87e03fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.246+0000 7f0b55e00700 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 0x55f87e0401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:41.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.246+0000 7f0b57200700 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e03a800 0x55f87e0b7680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0b500060b0 tx=0x7f0b5001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972fa0d11311d609 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:41.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.247+0000 7f0b37e00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:44:41.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.247+0000 7f0b37e00700 1 -- 172.21.15.131:0/701913839 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50034440 con 0x55f87e03a800 2024-05-10T05:44:41.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.247+0000 7f0b37e00700 1 -- 172.21.15.131:0/701913839 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f0b500345a0 con 0x55f87e03a800 2024-05-10T05:44:41.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.248+0000 7f0b753f0b80 1 -- 172.21.15.131:0/701913839 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b4409a550 con 0x55f87e03a800 2024-05-10T05:44:41.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.248+0000 7f0b37e00700 1 -- 172.21.15.131:0/701913839 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b50034760 con 0x55f87e03a800 2024-05-10T05:44:41.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.249+0000 7f0b753f0b80 10 client.17019 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:41.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.249+0000 7f0b753f0b80 1 -- 172.21.15.131:0/701913839 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f87dde1cd0 con 0x55f87e03a800 2024-05-10T05:44:41.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.250+0000 7f0b37e00700 1 -- 172.21.15.131:0/701913839 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b50041030 con 0x55f87e03a800 2024-05-10T05:44:41.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.250+0000 7f0b37e00700 1 -- 172.21.15.131:0/701913839 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0b5003b060 con 0x55f87e03a800 2024-05-10T05:44:41.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.250+0000 7f0b753f0b80 10 client.17019 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:41.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.250+0000 7f0b753f0b80 10 client.17019 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:44:41.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.250+0000 7f0b753f0b80 20 client.17019 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:41.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.250+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55f87e0b97f0 0x55f87e0bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:41.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.250+0000 7f0b753f0b80 4 client.17019 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:44:41.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.250+0000 7f0b753f0b80 1 -- 172.21.15.131:0/701913839 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f87df54ec0 con 0x55f87e0b97f0 2024-05-10T05:44:41.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.251+0000 7f0b56800700 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55f87e0b97f0 0x55f87e0bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:41.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.251+0000 7f0b37e00700 10 client.17019 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:44:41.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.252+0000 7f0b56800700 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55f87e0b97f0 0x55f87e0bbc10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aeca2d00fb7b45c5 server_cookie=5dd8ece6a1f016a3 in_seq=0 out_seq=0 2024-05-10T05:44:41.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.253+0000 7f0b37e00700 1 -- 172.21.15.131:0/701913839 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f87df54ec0 con 0x55f87e0b97f0 2024-05-10T05:44:41.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.253+0000 7f0b37e00700 10 client.17019 handle_command_reply: tid=0 2024-05-10T05:44:41.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.253+0000 7f0b753f0b80 1 client.17019 shutdown 2024-05-10T05:44:41.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.253+0000 7f0b753f0b80 1 -- 172.21.15.131:0/701913839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e03a800 msgr2=0x55f87e0b7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:41.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.253+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e03a800 0x55f87e0b7680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0b500060b0 tx=0x7f0b5001ba30 comp rx=0 tx=0).stop 2024-05-10T05:44:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.254+0000 7f0b753f0b80 1 -- 172.21.15.131:0/701913839 shutdown_connections 2024-05-10T05:44:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.254+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55f87e0b97f0 0x55f87e0bbc10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.254+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f87e03a800 0x55f87e0b7680 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.254+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f87e039380 0x55f87e03fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.254+0000 7f0b753f0b80 1 --2- 172.21.15.131:0/701913839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f87e039cd0 0x55f87e0401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.254+0000 7f0b37e00700 0 client.17019 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:44:41.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.254+0000 7f0b753f0b80 1 -- 172.21.15.131:0/701913839 >> 172.21.15.131:0/701913839 conn(0x55f87df9ea60 msgr2=0x55f87e0411a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:41.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.254+0000 7f0b753f0b80 1 -- 172.21.15.131:0/701913839 shutdown_connections 2024-05-10T05:44:41.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.254+0000 7f0b753f0b80 1 -- 172.21.15.131:0/701913839 wait complete. 2024-05-10T05:44:41.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.255+0000 7f0b753f0b80 20 client.17019 trim_cache size 0 max 16384 2024-05-10T05:44:41.256 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:44:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.256+0000 7f0b5d400700 1 -- 172.21.15.131:0/59409360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0b3c0a1240 msgr2=0x7f0b3c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.256+0000 7f0b5d400700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0b3c0a1240 0x7f0b3c0a3750 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009ec0 tx=0x7f0b50009920 comp rx=0 tx=0).stop 2024-05-10T05:44:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.256+0000 7f0b5d400700 1 -- 172.21.15.131:0/59409360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58102d00 msgr2=0x7f0b581984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:41.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.256+0000 7f0b5d400700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58102d00 0x7f0b581984e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009230 tx=0x7f0b4400b790 comp rx=0 tx=0).stop 2024-05-10T05:44:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.256+0000 7f0b5d400700 1 -- 172.21.15.131:0/59409360 shutdown_connections 2024-05-10T05:44:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.256+0000 7f0b5d400700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f0b3c0a1240 0x7f0b3c0a3750 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.256+0000 7f0b5d400700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b58101be0 0x7f0b58197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.256+0000 7f0b5d400700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b581036a0 0x7f0b5819f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.256+0000 7f0b5d400700 1 --2- 172.21.15.131:0/59409360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b58102d00 0x7f0b581984e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.256+0000 7f0b5d400700 1 -- 172.21.15.131:0/59409360 >> 172.21.15.131:0/59409360 conn(0x7f0b580fd1d0 msgr2=0x7f0b58105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:41.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.257+0000 7f0b5d400700 1 -- 172.21.15.131:0/59409360 shutdown_connections 2024-05-10T05:44:41.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:41.258+0000 7f0b5d400700 1 -- 172.21.15.131:0/59409360 wait complete. 2024-05-10T05:44:41.327 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:44:42.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.027+0000 7fdf84a00700 1 -- 172.21.15.131:0/405906955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf8010c510 msgr2=0x7fdf8010e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.027+0000 7fdf84a00700 1 --2- 172.21.15.131:0/405906955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf8010c510 0x7fdf8010e930 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009990 tx=0x7fdf6c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.031+0000 7fdf84a00700 1 -- 172.21.15.131:0/405906955 shutdown_connections 2024-05-10T05:44:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.031+0000 7fdf84a00700 1 --2- 172.21.15.131:0/405906955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf8010ee70 0x7fdf80111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.031+0000 7fdf84a00700 1 --2- 172.21.15.131:0/405906955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf80109bb0 0x7fdf8010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.031+0000 7fdf84a00700 1 --2- 172.21.15.131:0/405906955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf8010c510 0x7fdf8010e930 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.031+0000 7fdf84a00700 1 -- 172.21.15.131:0/405906955 >> 172.21.15.131:0/405906955 conn(0x7fdf8006e800 msgr2=0x7fdf80070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.033+0000 7fdf84a00700 1 -- 172.21.15.131:0/405906955 shutdown_connections 2024-05-10T05:44:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.034+0000 7fdf84a00700 1 -- 172.21.15.131:0/405906955 wait complete. 2024-05-10T05:44:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.035+0000 7fdf84a00700 1 Processor -- start 2024-05-10T05:44:42.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.040+0000 7fdf84a00700 1 -- start start 2024-05-10T05:44:42.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.043+0000 7fdf84a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80109bb0 0x7fdf801198b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.043+0000 7fdf84a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf8010c510 0x7fdf80119df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.043+0000 7fdf84a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf8010ee70 0x7fdf801b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.045+0000 7fdf84a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80120e10 con 0x7fdf8010c510 2024-05-10T05:44:42.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.045+0000 7fdf7f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf8010ee70 0x7fdf801b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.045+0000 7fdf7f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf8010ee70 0x7fdf801b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32940/0 (socket says 172.21.15.131:32940) 2024-05-10T05:44:42.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.045+0000 7fdf7f200700 1 -- 172.21.15.131:0/4156466080 learned_addr learned my addr 172.21.15.131:0/4156466080 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:42.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.045+0000 7fdf7de00700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf8010c510 0x7fdf80119df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.046+0000 7fdf84a00700 1 -- 172.21.15.131:0/4156466080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf801b7240 con 0x7fdf80109bb0 2024-05-10T05:44:42.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.046+0000 7fdf84a00700 1 -- 172.21.15.131:0/4156466080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf801b7380 con 0x7fdf8010ee70 2024-05-10T05:44:42.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.045+0000 7fdf7e800700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80109bb0 0x7fdf801198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.046+0000 7fdf7f200700 1 -- 172.21.15.131:0/4156466080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80109bb0 msgr2=0x7fdf801198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.046+0000 7fdf7f200700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80109bb0 0x7fdf801198b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.047+0000 7fdf7f200700 1 -- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf8010c510 msgr2=0x7fdf80119df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.048+0000 7fdf7f200700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf8010c510 0x7fdf80119df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.048+0000 7fdf7f200700 1 -- 172.21.15.131:0/4156466080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c0095f0 con 0x7fdf8010ee70 2024-05-10T05:44:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.049+0000 7fdf7f200700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf8010ee70 0x7fdf801b6d00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b550 tx=0x7fdf7000b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.049+0000 7fdf77400700 1 -- 172.21.15.131:0/4156466080 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000bea0 con 0x7fdf8010ee70 2024-05-10T05:44:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.050+0000 7fdf84a00700 1 -- 172.21.15.131:0/4156466080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf801b7630 con 0x7fdf8010ee70 2024-05-10T05:44:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.050+0000 7fdf84a00700 1 -- 172.21.15.131:0/4156466080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf801b7b40 con 0x7fdf8010ee70 2024-05-10T05:44:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.051+0000 7fdf7de00700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf8010c510 0x7fdf80119df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.052+0000 7fdf77400700 1 -- 172.21.15.131:0/4156466080 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf70011cf0 con 0x7fdf8010ee70 2024-05-10T05:44:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.052+0000 7fdf77400700 1 -- 172.21.15.131:0/4156466080 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70010e10 con 0x7fdf8010ee70 2024-05-10T05:44:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.060+0000 7fdf9d351b80 1 Processor -- start 2024-05-10T05:44:42.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.063+0000 7fdf9d351b80 1 -- start start 2024-05-10T05:44:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.066+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 0x55bf302c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.067+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.067+0000 7fdf77400700 1 -- 172.21.15.131:0/4156466080 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fdf70011830 con 0x7fdf8010ee70 2024-05-10T05:44:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.067+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.067+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf301eb150 con 0x55bf302c4530 2024-05-10T05:44:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.067+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf301e8b10 con 0x55bf302c57a0 2024-05-10T05:44:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.067+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf301e4400 con 0x55bf302c60f0 2024-05-10T05:44:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.067+0000 7fdf7e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.067+0000 7fdf7e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41694/0 (socket says 172.21.15.131:41694) 2024-05-10T05:44:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.067+0000 7fdf7e800700 1 -- 172.21.15.131:0/2816555848 learned_addr learned my addr 172.21.15.131:0/2816555848 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.068+0000 7fdf7de00700 1 --2- 172.21.15.131:0/2816555848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 0x55bf302c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.069+0000 7fdf77400700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdf640a1320 0x7fdf640a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.069+0000 7fdf7e800700 1 -- 172.21.15.131:0/2816555848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 msgr2=0x55bf302cc9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:44:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.069+0000 7fdf7e800700 1 --2- 172.21.15.131:0/2816555848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302cc9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.069+0000 7fdf7e800700 1 -- 172.21.15.131:0/2816555848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 msgr2=0x55bf302c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.069+0000 7fdf77400700 1 -- 172.21.15.131:0/4156466080 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fdf700cc0d0 con 0x7fdf8010ee70 2024-05-10T05:44:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.069+0000 7fdf7e800700 1 --2- 172.21.15.131:0/2816555848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 0x55bf302c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.069+0000 7fdf7e800700 1 -- 172.21.15.131:0/2816555848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf300653b0 con 0x55bf302c57a0 2024-05-10T05:44:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.069+0000 7fdf7f200700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdf640a1320 0x7fdf640a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.070+0000 7fdf7de00700 1 --2- 172.21.15.131:0/2816555848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 0x55bf302c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.070+0000 7fdf7e800700 1 --2- 172.21.15.131:0/2816555848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302c5bb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009fd0 tx=0x7fdf780096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9077fd3ea9488f4c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.071+0000 7fdf6b400700 1 -- 172.21.15.131:0/2816555848 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78010040 con 0x55bf302c57a0 2024-05-10T05:44:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.071+0000 7fdf6b400700 1 -- 172.21.15.131:0/2816555848 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf78014ae0 con 0x55bf302c57a0 2024-05-10T05:44:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.072+0000 7fdf6b400700 1 -- 172.21.15.131:0/2816555848 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78013a60 con 0x55bf302c57a0 2024-05-10T05:44:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.073+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2816555848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 msgr2=0x55bf302c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.073+0000 7fdf7f200700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdf640a1320 0x7fdf640a3830 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf70099450 tx=0x7fdf70004860 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.073+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/2816555848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302c5bb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009fd0 tx=0x7fdf780096d0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.073+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2816555848 shutdown_connections 2024-05-10T05:44:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.073+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/2816555848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 0x55bf302c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.073+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/2816555848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.073+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/2816555848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302c5bb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.073+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2816555848 >> 172.21.15.131:0/2816555848 conn(0x55bf30226a60 msgr2=0x55bf302bf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.073+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2816555848 shutdown_connections 2024-05-10T05:44:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.073+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2816555848 wait complete. 2024-05-10T05:44:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.074+0000 7fdf9d351b80 1 -- start start 2024-05-10T05:44:42.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.075+0000 7fdf9d351b80 1 Processor -- start 2024-05-10T05:44:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.076+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 0x55bf302ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.076+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.077+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.077+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf30180560 con 0x55bf302c4530 2024-05-10T05:44:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.077+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf30180200 con 0x55bf302c57a0 2024-05-10T05:44:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.077+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf300676a0 con 0x55bf302c60f0 2024-05-10T05:44:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.077+0000 7fdf7e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.077+0000 7fdf7e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41696/0 (socket says 172.21.15.131:41696) 2024-05-10T05:44:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.078+0000 7fdf9d351b80 10 client.0 _ll_register_callbacks cb 0x55bf30189b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.077+0000 7fdf7de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 0x55bf302ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.078+0000 7fdf7e800700 1 -- 172.21.15.131:0/3691755853 learned_addr learned my addr 172.21.15.131:0/3691755853 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.079+0000 7fdf7de00700 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.079+0000 7fdf7e800700 1 -- 172.21.15.131:0/3691755853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 msgr2=0x55bf302d5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:42.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.079+0000 7fdf7e800700 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302d5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.079+0000 7fdf7e800700 1 -- 172.21.15.131:0/3691755853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 msgr2=0x55bf302ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:42.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.079+0000 7fdf7e800700 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 0x55bf302ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.079+0000 7fdf7e800700 1 -- 172.21.15.131:0/3691755853 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf80111eb0 con 0x55bf302c57a0 2024-05-10T05:44:42.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.079+0000 7fdf7de00700 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302d5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.080+0000 7fdf7e800700 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302cee30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009540 tx=0x7fdf78014650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2960b7e274434eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.080+0000 7fdf5fe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:44:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.080+0000 7fdf5fe00700 1 -- 172.21.15.131:0/3691755853 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78010030 con 0x55bf302c57a0 2024-05-10T05:44:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.080+0000 7fdf5fe00700 1 -- 172.21.15.131:0/3691755853 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fdf78014920 con 0x55bf302c57a0 2024-05-10T05:44:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.080+0000 7fdf9d351b80 1 -- 172.21.15.131:0/3691755853 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf3013d270 con 0x55bf302c57a0 2024-05-10T05:44:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.080+0000 7fdf5fe00700 1 -- 172.21.15.131:0/3691755853 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf78013bf0 con 0x55bf302c57a0 2024-05-10T05:44:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.081+0000 7fdf9d351b80 10 client.27401 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.081+0000 7fdf9d351b80 1 -- 172.21.15.131:0/3691755853 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fdf78009d00 con 0x55bf302c57a0 2024-05-10T05:44:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.081+0000 7fdf5fe00700 1 -- 172.21.15.131:0/3691755853 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bf3013d270 con 0x55bf302c57a0 2024-05-10T05:44:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.082+0000 7fdf5fe00700 1 -- 172.21.15.131:0/3691755853 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdf78024020 con 0x55bf302c57a0 2024-05-10T05:44:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.082+0000 7fdf9d351b80 10 client.27401 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.083+0000 7fdf9d351b80 10 client.27401 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:44:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.083+0000 7fdf9d351b80 20 client.27401 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.083+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bf302da440 0x55bf302dc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.083+0000 7fdf9d351b80 4 client.27401 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:42.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.083+0000 7fdf9d351b80 1 -- 172.21.15.131:0/3691755853 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf30293780 con 0x55bf302da440 2024-05-10T05:44:42.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.083+0000 7fdf7de00700 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bf302da440 0x55bf302dc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.084+0000 7fdf7de00700 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bf302da440 0x55bf302dc890 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40b9336b7ad9e6ea server_cookie=6d53c899f9e082e7 in_seq=0 out_seq=0 2024-05-10T05:44:42.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.084+0000 7fdf5fe00700 10 client.27401 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:44:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.085+0000 7fdf5fe00700 1 -- 172.21.15.131:0/3691755853 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf30293780 con 0x55bf302da440 2024-05-10T05:44:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.085+0000 7fdf5fe00700 10 client.27401 handle_command_reply: tid=0 2024-05-10T05:44:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.085+0000 7fdf9d351b80 1 client.27401 shutdown 2024-05-10T05:44:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.086+0000 7fdf9d351b80 1 -- 172.21.15.131:0/3691755853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 msgr2=0x55bf302cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.086+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302cee30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009540 tx=0x7fdf78014650 comp rx=0 tx=0).stop 2024-05-10T05:44:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.086+0000 7fdf9d351b80 1 -- 172.21.15.131:0/3691755853 shutdown_connections 2024-05-10T05:44:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.086+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bf302da440 0x55bf302dc890 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.086+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c4530 0x55bf302ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.086+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302d5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.086+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/3691755853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302cee30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.086+0000 7fdf9d351b80 1 -- 172.21.15.131:0/3691755853 >> 172.21.15.131:0/3691755853 conn(0x55bf30226a60 msgr2=0x55bf302c3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.086+0000 7fdf5fe00700 0 client.27401 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:44:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.087+0000 7fdf9d351b80 1 -- 172.21.15.131:0/3691755853 shutdown_connections 2024-05-10T05:44:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.087+0000 7fdf9d351b80 1 -- 172.21.15.131:0/3691755853 wait complete. 2024-05-10T05:44:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.087+0000 7fdf9d351b80 20 client.27401 trim_cache size 0 max 16384 2024-05-10T05:44:42.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.118+0000 7fdf9d351b80 1 Processor -- start 2024-05-10T05:44:42.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.119+0000 7fdf9d351b80 1 -- start start 2024-05-10T05:44:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.120+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c4530 0x55bf302cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.121+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c57a0 0x55bf302d59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.121+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302dc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.121+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf302b92f0 con 0x55bf302c57a0 2024-05-10T05:44:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.121+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf302d6e90 con 0x55bf302c4530 2024-05-10T05:44:42.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.121+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf302b98d0 con 0x55bf302c60f0 2024-05-10T05:44:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c4530 0x55bf302cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c4530 0x55bf302cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:41698/0 (socket says 172.21.15.131:41698) 2024-05-10T05:44:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7de00700 1 -- 172.21.15.131:0/414443342 learned_addr learned my addr 172.21.15.131:0/414443342 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7e800700 1 --2- 172.21.15.131:0/414443342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c57a0 0x55bf302d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7de00700 1 --2- 172.21.15.131:0/414443342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302dc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7de00700 1 -- 172.21.15.131:0/414443342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 msgr2=0x55bf302dc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7de00700 1 --2- 172.21.15.131:0/414443342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302dc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7de00700 1 -- 172.21.15.131:0/414443342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c57a0 msgr2=0x55bf302d59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7de00700 1 --2- 172.21.15.131:0/414443342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c57a0 0x55bf302d59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7de00700 1 -- 172.21.15.131:0/414443342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf30011b60 con 0x55bf302c4530 2024-05-10T05:44:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.122+0000 7fdf7e800700 1 --2- 172.21.15.131:0/414443342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c57a0 0x55bf302d59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:42.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.123+0000 7fdf7de00700 1 --2- 172.21.15.131:0/414443342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c4530 0x55bf302cde60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009990 tx=0x7fdf6c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5efa0bab6b1c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:42.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.125+0000 7fdf6be00700 1 -- 172.21.15.131:0/414443342 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0050b0 con 0x55bf302c4530 2024-05-10T05:44:42.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.125+0000 7fdf6be00700 1 -- 172.21.15.131:0/414443342 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf6c01e040 con 0x55bf302c4530 2024-05-10T05:44:42.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.125+0000 7fdf6be00700 1 -- 172.21.15.131:0/414443342 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c00f460 con 0x55bf302c4530 2024-05-10T05:44:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.126+0000 7fdf9d351b80 1 -- 172.21.15.131:0/414443342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c4530 msgr2=0x55bf302cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.126+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/414443342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c4530 0x55bf302cde60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009990 tx=0x7fdf6c01a040 comp rx=0 tx=0).stop 2024-05-10T05:44:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.126+0000 7fdf9d351b80 1 -- 172.21.15.131:0/414443342 shutdown_connections 2024-05-10T05:44:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.126+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/414443342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c60f0 0x55bf302dc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.126+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/414443342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c4530 0x55bf302cde60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.126+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/414443342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c57a0 0x55bf302d59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.126+0000 7fdf9d351b80 1 -- 172.21.15.131:0/414443342 >> 172.21.15.131:0/414443342 conn(0x55bf30226a60 msgr2=0x55bf302c9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.127+0000 7fdf9d351b80 1 -- 172.21.15.131:0/414443342 shutdown_connections 2024-05-10T05:44:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.127+0000 7fdf9d351b80 1 -- 172.21.15.131:0/414443342 wait complete. 2024-05-10T05:44:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.127+0000 7fdf9d351b80 1 -- start start 2024-05-10T05:44:42.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.128+0000 7fdf9d351b80 1 Processor -- start 2024-05-10T05:44:42.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.128+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c4530 0x55bf302d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.128+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.129+0000 7fdf9d351b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c60f0 0x55bf302d4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.129+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf302d48d0 con 0x55bf302c60f0 2024-05-10T05:44:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.129+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf302d4a40 con 0x55bf302c57a0 2024-05-10T05:44:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.129+0000 7fdf9d351b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf302d4c00 con 0x55bf302c4530 2024-05-10T05:44:42.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.129+0000 7fdf9d351b80 10 client.0 _ll_register_callbacks cb 0x55bf30189b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:42.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.129+0000 7fdf7de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c60f0 0x55bf302d4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.130+0000 7fdf7de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c4530 0x55bf302d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.130+0000 7fdf7e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.130+0000 7fdf7de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c4530 0x55bf302d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:32964/0 (socket says 172.21.15.131:32964) 2024-05-10T05:44:42.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.130+0000 7fdf7de00700 1 -- 172.21.15.131:0/2984354346 learned_addr learned my addr 172.21.15.131:0/2984354346 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:42.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.131+0000 7fdf7e800700 1 -- 172.21.15.131:0/2984354346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c4530 msgr2=0x55bf302d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:42.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.131+0000 7fdf7e800700 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c4530 0x55bf302d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.131+0000 7fdf7e800700 1 -- 172.21.15.131:0/2984354346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c60f0 msgr2=0x55bf302d4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:42.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.131+0000 7fdf7e800700 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c60f0 0x55bf302d4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.131+0000 7fdf7e800700 1 -- 172.21.15.131:0/2984354346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c020040 con 0x55bf302c57a0 2024-05-10T05:44:42.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.131+0000 7fdf7e800700 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302d34b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800a420 tx=0x7fdf7800ab40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9234a5d1b4148492 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:42.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.132+0000 7fdf7de00700 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c4530 0x55bf302d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:42.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.132+0000 7fdf5fe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:44:42.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.132+0000 7fdf5fe00700 1 -- 172.21.15.131:0/2984354346 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78010070 con 0x55bf302c57a0 2024-05-10T05:44:42.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.132+0000 7fdf5fe00700 1 -- 172.21.15.131:0/2984354346 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fdf780139f0 con 0x55bf302c57a0 2024-05-10T05:44:42.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.132+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2984354346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf78014ae0 con 0x55bf302c57a0 2024-05-10T05:44:42.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.134+0000 7fdf5fe00700 1 -- 172.21.15.131:0/2984354346 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf78031620 con 0x55bf302c57a0 2024-05-10T05:44:42.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.134+0000 7fdf9d351b80 10 client.27407 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:42.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.134+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2984354346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bf301aa560 con 0x55bf302c57a0 2024-05-10T05:44:42.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.136+0000 7fdf5fe00700 1 -- 172.21.15.131:0/2984354346 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf78014ae0 con 0x55bf302c57a0 2024-05-10T05:44:42.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.136+0000 7fdf5fe00700 1 -- 172.21.15.131:0/2984354346 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdf7800a7b0 con 0x55bf302c57a0 2024-05-10T05:44:42.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.136+0000 7fdf9d351b80 10 client.27407 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:42.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.136+0000 7fdf9d351b80 10 client.27407 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:44:42.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.136+0000 7fdf9d351b80 20 client.27407 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:42.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.137+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bf303308f0 0x55bf30332d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.137+0000 7fdf9d351b80 4 client.27407 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:44:42.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.137+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2984354346 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bf301dcec0 con 0x55bf303308f0 2024-05-10T05:44:42.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.137+0000 7fdf7de00700 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bf303308f0 0x55bf30332d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.138+0000 7fdf7de00700 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bf303308f0 0x55bf30332d10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=444f4c7ad3101793 server_cookie=e1496a8230bd08cd in_seq=0 out_seq=0 2024-05-10T05:44:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.139+0000 7fdf5fe00700 10 client.27407 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:44:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.139+0000 7fdf5fe00700 1 -- 172.21.15.131:0/2984354346 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bf301dcec0 con 0x55bf303308f0 2024-05-10T05:44:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.140+0000 7fdf5fe00700 10 client.27407 handle_command_reply: tid=0 2024-05-10T05:44:42.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.140+0000 7fdf9d351b80 1 client.27407 shutdown 2024-05-10T05:44:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.141+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2984354346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 msgr2=0x55bf302d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.141+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302d34b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800a420 tx=0x7fdf7800ab40 comp rx=0 tx=0).stop 2024-05-10T05:44:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.141+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2984354346 shutdown_connections 2024-05-10T05:44:42.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.141+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x55bf303308f0 0x55bf30332d10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.141+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bf302c60f0 0x55bf302d4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.141+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf302c4530 0x55bf302d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.141+0000 7fdf9d351b80 1 --2- 172.21.15.131:0/2984354346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf302c57a0 0x55bf302d34b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.141+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2984354346 >> 172.21.15.131:0/2984354346 conn(0x55bf30226a60 msgr2=0x55bf302c9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:42.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.141+0000 7fdf5fe00700 0 client.27407 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:44:42.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.142+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2984354346 shutdown_connections 2024-05-10T05:44:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.142+0000 7fdf9d351b80 1 -- 172.21.15.131:0/2984354346 wait complete. 2024-05-10T05:44:42.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.143+0000 7fdf9d351b80 20 client.27407 trim_cache size 0 max 16384 2024-05-10T05:44:42.145 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:44:42.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.144+0000 7fdf84a00700 1 -- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdf640a1320 msgr2=0x7fdf640a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.144+0000 7fdf84a00700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdf640a1320 0x7fdf640a3830 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf70099450 tx=0x7fdf70004860 comp rx=0 tx=0).stop 2024-05-10T05:44:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.144+0000 7fdf84a00700 1 -- 172.21.15.131:0/4156466080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf8010ee70 msgr2=0x7fdf801b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.144+0000 7fdf84a00700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf8010ee70 0x7fdf801b6d00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b550 tx=0x7fdf7000b860 comp rx=0 tx=0).stop 2024-05-10T05:44:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.145+0000 7fdf84a00700 1 -- 172.21.15.131:0/4156466080 shutdown_connections 2024-05-10T05:44:42.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.145+0000 7fdf84a00700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fdf640a1320 0x7fdf640a3830 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.145+0000 7fdf84a00700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf8010ee70 0x7fdf801b6d00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.145+0000 7fdf84a00700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80109bb0 0x7fdf801198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.145+0000 7fdf84a00700 1 --2- 172.21.15.131:0/4156466080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf8010c510 0x7fdf80119df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.145+0000 7fdf84a00700 1 -- 172.21.15.131:0/4156466080 >> 172.21.15.131:0/4156466080 conn(0x7fdf8006e800 msgr2=0x7fdf8010bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:42.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.145+0000 7fdf84a00700 1 -- 172.21.15.131:0/4156466080 shutdown_connections 2024-05-10T05:44:42.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.145+0000 7fdf84a00700 1 -- 172.21.15.131:0/4156466080 wait complete. 2024-05-10T05:44:42.212 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:44:42.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.923+0000 7fa4a7c00700 1 -- 172.21.15.131:0/486266214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a0102c30 msgr2=0x7fa4a0105050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.923+0000 7fa4a7c00700 1 --2- 172.21.15.131:0/486266214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a0102c30 0x7fa4a0105050 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa488009990 tx=0x7fa488009ca0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.924+0000 7fa4a7c00700 1 -- 172.21.15.131:0/486266214 shutdown_connections 2024-05-10T05:44:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.924+0000 7fa4a7c00700 1 --2- 172.21.15.131:0/486266214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a01002d0 0x7fa4a01026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.924+0000 7fa4a7c00700 1 --2- 172.21.15.131:0/486266214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a0105590 0x7fa4a01059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.924+0000 7fa4a7c00700 1 --2- 172.21.15.131:0/486266214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a0102c30 0x7fa4a0105050 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.924+0000 7fa4a7c00700 1 -- 172.21.15.131:0/486266214 >> 172.21.15.131:0/486266214 conn(0x7fa4a00fbf20 msgr2=0x7fa4a00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.924+0000 7fa4a7c00700 1 -- 172.21.15.131:0/486266214 shutdown_connections 2024-05-10T05:44:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.925+0000 7fa4a7c00700 1 -- 172.21.15.131:0/486266214 wait complete. 2024-05-10T05:44:42.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.928+0000 7fa4a7c00700 1 Processor -- start 2024-05-10T05:44:42.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.929+0000 7fa4a7c00700 1 -- start start 2024-05-10T05:44:42.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.930+0000 7fa4a7c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a01002d0 0x7fa4a010bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.930+0000 7fa4a7c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102c30 0x7fa4a010c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.930+0000 7fa4a7c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a0105590 0x7fa4a010d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.930+0000 7fa4a5200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a01002d0 0x7fa4a010bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.930+0000 7fa4a5200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a01002d0 0x7fa4a010bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54708/0 (socket says 172.21.15.131:54708) 2024-05-10T05:44:42.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.930+0000 7fa4a7c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a010d660 con 0x7fa4a0102c30 2024-05-10T05:44:42.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.931+0000 7fa4a7c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a010d7d0 con 0x7fa4a0105590 2024-05-10T05:44:42.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.931+0000 7fa4a7c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a010d990 con 0x7fa4a01002d0 2024-05-10T05:44:42.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.931+0000 7fa4a5c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a0105590 0x7fa4a010d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.931+0000 7fa4a5c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a0105590 0x7fa4a010d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:50066/0 (socket says 172.21.15.131:50066) 2024-05-10T05:44:42.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.931+0000 7fa4a5c00700 1 -- 172.21.15.131:0/1698269443 learned_addr learned my addr 172.21.15.131:0/1698269443 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:42.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.932+0000 7fa4a5c00700 1 -- 172.21.15.131:0/1698269443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a01002d0 msgr2=0x7fa4a010bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.932+0000 7fa4a5c00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a01002d0 0x7fa4a010bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.932+0000 7fa4a5c00700 1 -- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102c30 msgr2=0x7fa4a010c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.931+0000 7fa497e00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102c30 0x7fa4a010c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.932+0000 7fa4a5c00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102c30 0x7fa4a010c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.932+0000 7fa4a5c00700 1 -- 172.21.15.131:0/1698269443 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4880095f0 con 0x7fa4a0105590 2024-05-10T05:44:42.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.934+0000 7fa497e00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102c30 0x7fa4a010c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:42.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.934+0000 7fa4a5c00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a0105590 0x7fa4a010d120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa498007d40 tx=0x7fa49800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:42.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.935+0000 7fa49e000700 1 -- 172.21.15.131:0/1698269443 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49800ad40 con 0x7fa4a0105590 2024-05-10T05:44:42.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.935+0000 7fa49e000700 1 -- 172.21.15.131:0/1698269443 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4980128e0 con 0x7fa4a0105590 2024-05-10T05:44:42.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.935+0000 7fa49e000700 1 -- 172.21.15.131:0/1698269443 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011820 con 0x7fa4a0105590 2024-05-10T05:44:42.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.937+0000 7fa4a7c00700 1 -- 172.21.15.131:0/1698269443 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a010dc70 con 0x7fa4a0105590 2024-05-10T05:44:42.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.937+0000 7fa4a7c00700 1 -- 172.21.15.131:0/1698269443 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a010e180 con 0x7fa4a0105590 2024-05-10T05:44:42.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.944+0000 7fa4bfb8ab80 1 Processor -- start 2024-05-10T05:44:42.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.946+0000 7fa4bfb8ab80 1 -- start start 2024-05-10T05:44:42.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.946+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf4a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.946+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf4bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.946+0000 7fa49e000700 1 -- 172.21.15.131:0/1698269443 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa498012420 con 0x7fa4a0105590 2024-05-10T05:44:42.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.946+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf529f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.946+0000 7fa4bfb8ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4ce71150 con 0x55ad4cf4c0f0 2024-05-10T05:44:42.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.946+0000 7fa4bfb8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4ce6eb10 con 0x55ad4cf4b7a0 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.946+0000 7fa4bfb8ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4ce6a400 con 0x55ad4cf4a530 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.947+0000 7fa497e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf4a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.947+0000 7fa497e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf4a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54718/0 (socket says 172.21.15.131:54718) 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.947+0000 7fa497e00700 1 -- 172.21.15.131:0/2173088120 learned_addr learned my addr 172.21.15.131:0/2173088120 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.947+0000 7fa497e00700 1 --2- 172.21.15.131:0/2173088120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.948+0000 7fa497e00700 1 -- 172.21.15.131:0/2173088120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 msgr2=0x55ad4cf4a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.948+0000 7fa497e00700 1 --2- 172.21.15.131:0/2173088120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf4a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.948+0000 7fa497e00700 1 -- 172.21.15.131:0/2173088120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4b7a0 msgr2=0x55ad4cf4bbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.948+0000 7fa497e00700 1 --2- 172.21.15.131:0/2173088120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf4bbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.948+0000 7fa497e00700 1 -- 172.21.15.131:0/2173088120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad4cceb3b0 con 0x55ad4cf4c0f0 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.949+0000 7fa497e00700 1 --2- 172.21.15.131:0/2173088120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf529f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa4880038c0 tx=0x7fa488005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372e03547f03382 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.949+0000 7fa496000700 1 -- 172.21.15.131:0/2173088120 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488005d40 con 0x55ad4cf4c0f0 2024-05-10T05:44:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.950+0000 7fa49e000700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4840a1220 0x7fa4840a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.951+0000 7fa4a5200700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4840a1220 0x7fa4840a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.951+0000 7fa49e000700 1 -- 172.21.15.131:0/1698269443 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa4980ccc60 con 0x7fa4a0105590 2024-05-10T05:44:42.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.952+0000 7fa496000700 1 -- 172.21.15.131:0/2173088120 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa488003e20 con 0x55ad4cf4c0f0 2024-05-10T05:44:42.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.952+0000 7fa496000700 1 -- 172.21.15.131:0/2173088120 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48800f660 con 0x55ad4cf4c0f0 2024-05-10T05:44:42.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.953+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/2173088120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4c0f0 msgr2=0x55ad4cf529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.953+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/2173088120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf529f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa4880038c0 tx=0x7fa488005fb0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.953+0000 7fa4a5200700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4840a1220 0x7fa4840a3730 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa490006fd0 tx=0x7fa490008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:42.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.955+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/2173088120 shutdown_connections 2024-05-10T05:44:42.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.955+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/2173088120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf529f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.955+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/2173088120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf4a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.955+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/2173088120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf4bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.955+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/2173088120 >> 172.21.15.131:0/2173088120 conn(0x55ad4ceaca60 msgr2=0x55ad4cf45330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:42.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.956+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/2173088120 shutdown_connections 2024-05-10T05:44:42.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.956+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/2173088120 wait complete. 2024-05-10T05:44:42.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.957+0000 7fa4bfb8ab80 1 -- start start 2024-05-10T05:44:42.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.958+0000 7fa4bfb8ab80 1 Processor -- start 2024-05-10T05:44:42.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.959+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4a530 0x55ad4cf54870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.960+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf54db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.960+0000 7fa4a5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4a530 0x55ad4cf54870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.960+0000 7fa4a5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4a530 0x55ad4cf54870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:50072/0 (socket says 172.21.15.131:50072) 2024-05-10T05:44:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.960+0000 7fa4a5200700 1 -- 172.21.15.131:0/1153557556 learned_addr learned my addr 172.21.15.131:0/1153557556 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.961+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.961+0000 7fa497e00700 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf54db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.961+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4ce06560 con 0x55ad4cf4b7a0 2024-05-10T05:44:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.961+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4ce06200 con 0x55ad4cf4a530 2024-05-10T05:44:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.961+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4cced6a0 con 0x55ad4cf4c0f0 2024-05-10T05:44:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.961+0000 7fa4a5c00700 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.961+0000 7fa4bfb8ab80 10 client.0 _ll_register_callbacks cb 0x55ad4ce0fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.962+0000 7fa4a5200700 1 -- 172.21.15.131:0/1153557556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4c0f0 msgr2=0x55ad4cf5bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.962+0000 7fa4a5200700 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.962+0000 7fa4a5200700 1 -- 172.21.15.131:0/1153557556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4b7a0 msgr2=0x55ad4cf54db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.962+0000 7fa4a5200700 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf54db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.962+0000 7fa4a5200700 1 -- 172.21.15.131:0/1153557556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a01065c0 con 0x55ad4cf4a530 2024-05-10T05:44:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.962+0000 7fa4a5200700 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4a530 0x55ad4cf54870 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa490014cf0 tx=0x7fa490004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f838e6e7de9f47d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.962+0000 7fa48ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:44:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.962+0000 7fa48ea00700 1 -- 172.21.15.131:0/1153557556 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490012ab0 con 0x55ad4cf4a530 2024-05-10T05:44:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.963+0000 7fa48ea00700 1 -- 172.21.15.131:0/1153557556 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa490012c10 con 0x55ad4cf4a530 2024-05-10T05:44:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.963+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad4cdc3270 con 0x55ad4cf4a530 2024-05-10T05:44:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.963+0000 7fa48ea00700 1 -- 172.21.15.131:0/1153557556 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa490012dd0 con 0x55ad4cf4a530 2024-05-10T05:44:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.963+0000 7fa4bfb8ab80 10 client.27416 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.964+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ad4cc97b60 con 0x55ad4cf4a530 2024-05-10T05:44:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.964+0000 7fa48ea00700 1 -- 172.21.15.131:0/1153557556 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad4cdc3270 con 0x55ad4cf4a530 2024-05-10T05:44:42.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.964+0000 7fa48ea00700 1 -- 172.21.15.131:0/1153557556 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa490027070 con 0x55ad4cf4a530 2024-05-10T05:44:42.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.964+0000 7fa4bfb8ab80 10 client.27416 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:42.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.965+0000 7fa4bfb8ab80 10 client.27416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:42.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.965+0000 7fa4bfb8ab80 20 client.27416 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:42.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.965+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad4cf60500 0x55ad4cf62950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:42.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.965+0000 7fa4bfb8ab80 4 client.27416 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:42.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.965+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad4cf19780 con 0x55ad4cf60500 2024-05-10T05:44:42.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.965+0000 7fa497e00700 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad4cf60500 0x55ad4cf62950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:42.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.966+0000 7fa497e00700 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad4cf60500 0x55ad4cf62950 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33dd9a64e41108b2 server_cookie=175be36845ee8f52 in_seq=0 out_seq=0 2024-05-10T05:44:42.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.966+0000 7fa48ea00700 10 client.27416 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:42.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.967+0000 7fa48ea00700 1 -- 172.21.15.131:0/1153557556 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad4cf19780 con 0x55ad4cf60500 2024-05-10T05:44:42.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.968+0000 7fa48ea00700 10 client.27416 handle_command_reply: tid=0 2024-05-10T05:44:42.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.968+0000 7fa4bfb8ab80 1 client.27416 shutdown 2024-05-10T05:44:42.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.968+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4a530 msgr2=0x55ad4cf54870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:42.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.969+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4a530 0x55ad4cf54870 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa490014cf0 tx=0x7fa490004860 comp rx=0 tx=0).stop 2024-05-10T05:44:42.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.969+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 shutdown_connections 2024-05-10T05:44:42.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.969+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad4cf60500 0x55ad4cf62950 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.969+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.969+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4a530 0x55ad4cf54870 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.969+0000 7fa48ea00700 0 client.27416 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:42.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.969+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/1153557556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf54db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:42.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.969+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 >> 172.21.15.131:0/1153557556 conn(0x55ad4ceaca60 msgr2=0x55ad4cf49060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:42.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.969+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 shutdown_connections 2024-05-10T05:44:42.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.970+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/1153557556 wait complete. 2024-05-10T05:44:42.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:42.971+0000 7fa4bfb8ab80 20 client.27416 trim_cache size 0 max 16384 2024-05-10T05:44:43.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.003+0000 7fa4bfb8ab80 1 Processor -- start 2024-05-10T05:44:43.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.004+0000 7fa4bfb8ab80 1 -- start start 2024-05-10T05:44:43.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.005+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf62920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:43.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.005+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf5bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:43.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.007+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:43.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.007+0000 7fa4bfb8ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4cf556c0 con 0x55ad4cf4b7a0 2024-05-10T05:44:43.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.007+0000 7fa4bfb8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4cf67b30 con 0x55ad4cf4c0f0 2024-05-10T05:44:43.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.007+0000 7fa4bfb8ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4cf4ffa0 con 0x55ad4cf4a530 2024-05-10T05:44:43.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.007+0000 7fa4a5200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf62920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:43.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.008+0000 7fa4a5c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:43.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.008+0000 7fa497e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf5bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:43.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.008+0000 7fa4a5200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf62920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54730/0 (socket says 172.21.15.131:54730) 2024-05-10T05:44:43.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.008+0000 7fa4a5200700 1 -- 172.21.15.131:0/559815587 learned_addr learned my addr 172.21.15.131:0/559815587 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:43.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.009+0000 7fa4a5200700 1 -- 172.21.15.131:0/559815587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4c0f0 msgr2=0x55ad4cf5c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:43.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.009+0000 7fa4a5200700 1 --2- 172.21.15.131:0/559815587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.009+0000 7fa4a5200700 1 -- 172.21.15.131:0/559815587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4b7a0 msgr2=0x55ad4cf5bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.009+0000 7fa4a5200700 1 --2- 172.21.15.131:0/559815587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf5bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.009+0000 7fa4a5200700 1 -- 172.21.15.131:0/559815587 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad4ce30560 con 0x55ad4cf4a530 2024-05-10T05:44:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.010+0000 7fa4a5200700 1 --2- 172.21.15.131:0/559815587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf62920 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa4900127a0 tx=0x7fa4900127d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b0565fe52c176e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:43.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.011+0000 7fa496a00700 1 -- 172.21.15.131:0/559815587 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490012ab0 con 0x55ad4cf4a530 2024-05-10T05:44:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.014+0000 7fa496a00700 1 -- 172.21.15.131:0/559815587 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa490012c10 con 0x55ad4cf4a530 2024-05-10T05:44:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.014+0000 7fa496a00700 1 -- 172.21.15.131:0/559815587 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4900062a0 con 0x55ad4cf4a530 2024-05-10T05:44:43.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.015+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/559815587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 msgr2=0x55ad4cf62920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:43.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.015+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/559815587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf62920 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa4900127a0 tx=0x7fa4900127d0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.016+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/559815587 shutdown_connections 2024-05-10T05:44:43.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.016+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/559815587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.016+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/559815587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4a530 0x55ad4cf62920 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.016+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/559815587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf5bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.016+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/559815587 >> 172.21.15.131:0/559815587 conn(0x55ad4ceaca60 msgr2=0x55ad4cf44bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:43.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.016+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/559815587 shutdown_connections 2024-05-10T05:44:43.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.016+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/559815587 wait complete. 2024-05-10T05:44:43.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.017+0000 7fa4bfb8ab80 1 -- start start 2024-05-10T05:44:43.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.019+0000 7fa4bfb8ab80 1 Processor -- start 2024-05-10T05:44:43.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.022+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4a530 0x55ad4cf59960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:43.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.022+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf59ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:43.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.022+0000 7fa4bfb8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:43.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.022+0000 7fa4bfb8ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4cf5b270 con 0x55ad4cf4a530 2024-05-10T05:44:43.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.022+0000 7fa4bfb8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4cf5b3e0 con 0x55ad4cf4c0f0 2024-05-10T05:44:43.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.022+0000 7fa4bfb8ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4cf5b5a0 con 0x55ad4cf4b7a0 2024-05-10T05:44:43.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.022+0000 7fa4bfb8ab80 10 client.0 _ll_register_callbacks cb 0x55ad4ce0fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:43.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.023+0000 7fa497e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf59ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:43.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.024+0000 7fa497e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf59ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54740/0 (socket says 172.21.15.131:54740) 2024-05-10T05:44:43.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.024+0000 7fa497e00700 1 -- 172.21.15.131:0/872870817 learned_addr learned my addr 172.21.15.131:0/872870817 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:43.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.024+0000 7fa4a5c00700 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:43.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.024+0000 7fa4a5200700 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4a530 0x55ad4cf59960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:43.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.025+0000 7fa497e00700 1 -- 172.21.15.131:0/872870817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4c0f0 msgr2=0x55ad4cf5ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:43.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.025+0000 7fa497e00700 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.025+0000 7fa497e00700 1 -- 172.21.15.131:0/872870817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4a530 msgr2=0x55ad4cf59960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:43.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.025+0000 7fa497e00700 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4a530 0x55ad4cf59960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.025+0000 7fa497e00700 1 -- 172.21.15.131:0/872870817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad4cc97b60 con 0x55ad4cf4b7a0 2024-05-10T05:44:43.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.025+0000 7fa4a5200700 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4a530 0x55ad4cf59960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:43.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.026+0000 7fa497e00700 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf59ea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa48800b5c0 tx=0x7fa488003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38659046c630c6ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:43.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.026+0000 7fa48ea00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:43.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.027+0000 7fa48ea00700 1 -- 172.21.15.131:0/872870817 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48803c040 con 0x55ad4cf4b7a0 2024-05-10T05:44:43.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.027+0000 7fa48ea00700 1 -- 172.21.15.131:0/872870817 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fa488000e40 con 0x55ad4cf4b7a0 2024-05-10T05:44:43.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.028+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/872870817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa488003e20 con 0x55ad4cf4b7a0 2024-05-10T05:44:43.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.029+0000 7fa48ea00700 1 -- 172.21.15.131:0/872870817 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa488003a20 con 0x55ad4cf4b7a0 2024-05-10T05:44:43.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.031+0000 7fa4bfb8ab80 10 client.27427 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:43.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.031+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/872870817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ad4ccefcd0 con 0x55ad4cf4b7a0 2024-05-10T05:44:43.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.031+0000 7fa48ea00700 1 -- 172.21.15.131:0/872870817 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa488003e20 con 0x55ad4cf4b7a0 2024-05-10T05:44:43.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.031+0000 7fa48ea00700 1 -- 172.21.15.131:0/872870817 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa4880053c0 con 0x55ad4cf4b7a0 2024-05-10T05:44:43.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.031+0000 7fa4bfb8ab80 10 client.27427 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:43.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.031+0000 7fa4bfb8ab80 10 client.27427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:43.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.031+0000 7fa4bfb8ab80 20 client.27427 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:43.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.031+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad4cfb6c70 0x55ad4cfb9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:43.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.031+0000 7fa4bfb8ab80 4 client.27427 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:44:43.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.031+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/872870817 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ad4ce62ec0 con 0x55ad4cfb6c70 2024-05-10T05:44:43.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.034+0000 7fa4a5200700 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad4cfb6c70 0x55ad4cfb9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:43.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.039+0000 7fa4a5200700 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad4cfb6c70 0x55ad4cfb9090 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e585d7ddd62d0ca server_cookie=3c46dcb9af724d7 in_seq=0 out_seq=0 2024-05-10T05:44:43.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.041+0000 7fa48ea00700 10 client.27427 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:43.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.042+0000 7fa48ea00700 1 -- 172.21.15.131:0/872870817 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ad4ce62ec0 con 0x55ad4cfb6c70 2024-05-10T05:44:43.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.042+0000 7fa48ea00700 10 client.27427 handle_command_reply: tid=0 2024-05-10T05:44:43.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.042+0000 7fa4bfb8ab80 1 client.27427 shutdown 2024-05-10T05:44:43.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.043+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/872870817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4b7a0 msgr2=0x55ad4cf59ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:43.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.043+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf59ea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa48800b5c0 tx=0x7fa488003f90 comp rx=0 tx=0).stop 2024-05-10T05:44:43.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.044+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/872870817 shutdown_connections 2024-05-10T05:44:43.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.044+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55ad4cfb6c70 0x55ad4cfb9090 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.044+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad4cf4a530 0x55ad4cf59960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.044+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad4cf4c0f0 0x55ad4cf5ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.044+0000 7fa4bfb8ab80 1 --2- 172.21.15.131:0/872870817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad4cf4b7a0 0x55ad4cf59ea0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.045+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/872870817 >> 172.21.15.131:0/872870817 conn(0x55ad4ceaca60 msgr2=0x55ad4cf52f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.045+0000 7fa48ea00700 0 client.27427 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:43.051 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:44:43.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.047+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/872870817 shutdown_connections 2024-05-10T05:44:43.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.047+0000 7fa4bfb8ab80 1 -- 172.21.15.131:0/872870817 wait complete. 2024-05-10T05:44:43.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.049+0000 7fa4bfb8ab80 20 client.27427 trim_cache size 0 max 16384 2024-05-10T05:44:43.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:42 smithi157 ceph-mon[78427]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 226 MiB/s wr, 71 op/s 2024-05-10T05:44:43.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.055+0000 7fa4a7c00700 1 -- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4840a1220 msgr2=0x7fa4840a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:43.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.055+0000 7fa4a7c00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4840a1220 0x7fa4840a3730 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa490006fd0 tx=0x7fa490008040 comp rx=0 tx=0).stop 2024-05-10T05:44:43.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.055+0000 7fa4a7c00700 1 -- 172.21.15.131:0/1698269443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a0105590 msgr2=0x7fa4a010d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:43.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.055+0000 7fa4a7c00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a0105590 0x7fa4a010d120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa498007d40 tx=0x7fa49800a340 comp rx=0 tx=0).stop 2024-05-10T05:44:43.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.059+0000 7fa4a7c00700 1 -- 172.21.15.131:0/1698269443 shutdown_connections 2024-05-10T05:44:43.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.059+0000 7fa4a7c00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fa4840a1220 0x7fa4840a3730 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.059+0000 7fa4a7c00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a0105590 0x7fa4a010d120 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.059+0000 7fa4a7c00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4a01002d0 0x7fa4a010bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.059+0000 7fa4a7c00700 1 --2- 172.21.15.131:0/1698269443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102c30 0x7fa4a010c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:43.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.059+0000 7fa4a7c00700 1 -- 172.21.15.131:0/1698269443 >> 172.21.15.131:0/1698269443 conn(0x7fa4a00fbf20 msgr2=0x7fa4a0101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:43.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.059+0000 7fa4a7c00700 1 -- 172.21.15.131:0/1698269443 shutdown_connections 2024-05-10T05:44:43.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:43.059+0000 7fa4a7c00700 1 -- 172.21.15.131:0/1698269443 wait complete. 2024-05-10T05:44:43.138 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:42 smithi131 ceph-mon[37264]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 226 MiB/s wr, 71 op/s 2024-05-10T05:44:43.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:42 smithi142 ceph-mon[77667]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 226 MiB/s wr, 71 op/s 2024-05-10T05:44:44.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:43 smithi142 ceph-mon[77667]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 210 MiB/s wr, 65 op/s 2024-05-10T05:44:44.146 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T05:44:44.147 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T05:44:44.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:43 smithi131 ceph-mon[37264]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 210 MiB/s wr, 65 op/s 2024-05-10T05:44:44.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:43 smithi157 ceph-mon[78427]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 210 MiB/s wr, 65 op/s 2024-05-10T05:44:44.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.823+0000 7f09b0800700 1 -- 172.21.15.131:0/4071639178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a8110320 msgr2=0x7f09a8112740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:44.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.823+0000 7f09b0800700 1 --2- 172.21.15.131:0/4071639178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a8110320 0x7f09a8112740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f099c0099c0 tx=0x7f099c009cd0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.823+0000 7f09b0800700 1 -- 172.21.15.131:0/4071639178 shutdown_connections 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.823+0000 7f09b0800700 1 --2- 172.21.15.131:0/4071639178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a8110320 0x7f09a8112740 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.823+0000 7f09b0800700 1 --2- 172.21.15.131:0/4071639178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09a8108210 0x7f09a8108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.823+0000 7f09b0800700 1 --2- 172.21.15.131:0/4071639178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09a8108b60 0x7f09a810fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.823+0000 7f09b0800700 1 -- 172.21.15.131:0/4071639178 >> 172.21.15.131:0/4071639178 conn(0x7f09a806ea00 msgr2=0x7f09a8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.824+0000 7f09b0800700 1 -- 172.21.15.131:0/4071639178 shutdown_connections 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.824+0000 7f09b0800700 1 -- 172.21.15.131:0/4071639178 wait complete. 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.826+0000 7f09b0800700 1 Processor -- start 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.826+0000 7f09b0800700 1 -- start start 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.826+0000 7f09b0800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a8108210 0x7f09a81a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.827+0000 7f09b0800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09a8108b60 0x7f09a81a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.827+0000 7f09b0800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09a8110320 0x7f09a81a7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.827+0000 7f09b0800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81a8260 con 0x7f09a8108b60 2024-05-10T05:44:44.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.827+0000 7f09b0800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81a83d0 con 0x7f09a8110320 2024-05-10T05:44:44.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.827+0000 7f09b0800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81a8590 con 0x7f09a8108210 2024-05-10T05:44:44.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.827+0000 7f09ade00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a8108210 0x7f09a81a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.827+0000 7f09ad400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09a8108b60 0x7f09a81a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.827+0000 7f09ade00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a8108210 0x7f09a81a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54750/0 (socket says 172.21.15.131:54750) 2024-05-10T05:44:44.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.827+0000 7f09ade00700 1 -- 172.21.15.131:0/1477026971 learned_addr learned my addr 172.21.15.131:0/1477026971 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.828+0000 7f09ae800700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09a8110320 0x7f09a81a7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.828+0000 7f09ad400700 1 -- 172.21.15.131:0/1477026971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a8108210 msgr2=0x7f09a81a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.828+0000 7f09ad400700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a8108210 0x7f09a81a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.828+0000 7f09ad400700 1 -- 172.21.15.131:0/1477026971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09a8110320 msgr2=0x7f09a81a7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.828+0000 7f09ad400700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09a8110320 0x7f09a81a7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.828+0000 7f09ad400700 1 -- 172.21.15.131:0/1477026971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c0095f0 con 0x7f09a8108b60 2024-05-10T05:44:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.829+0000 7f09ad400700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09a8108b60 0x7f09a81a0cc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f099800b450 tx=0x7f099800b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.830+0000 7f09a2a00700 1 -- 172.21.15.131:0/1477026971 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998011830 con 0x7f09a8108b60 2024-05-10T05:44:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.830+0000 7f09a2a00700 1 -- 172.21.15.131:0/1477026971 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0998011e50 con 0x7f09a8108b60 2024-05-10T05:44:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.830+0000 7f09a2a00700 1 -- 172.21.15.131:0/1477026971 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998010de0 con 0x7f09a8108b60 2024-05-10T05:44:44.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.830+0000 7f09b0800700 1 -- 172.21.15.131:0/1477026971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a8109d50 con 0x7f09a8108b60 2024-05-10T05:44:44.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.830+0000 7f09b0800700 1 -- 172.21.15.131:0/1477026971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a810a290 con 0x7f09a8108b60 2024-05-10T05:44:44.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.837+0000 7f09a2a00700 1 -- 172.21.15.131:0/1477026971 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f099801a400 con 0x7f09a8108b60 2024-05-10T05:44:44.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.839+0000 7f09a2a00700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09900a1290 0x7f09900a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.840+0000 7f09a2a00700 1 -- 172.21.15.131:0/1477026971 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f099809ad30 con 0x7f09a8108b60 2024-05-10T05:44:44.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.840+0000 7f09ade00700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09900a1290 0x7f09900a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.841+0000 7f09ade00700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09900a1290 0x7f09900a37a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f09a40148c0 tx=0x7f09a4016040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:44.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.845+0000 7f09c866cb80 1 Processor -- start 2024-05-10T05:44:44.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.847+0000 7f09c866cb80 1 -- start start 2024-05-10T05:44:44.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.847+0000 7f09c866cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c961b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.847+0000 7f09c866cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c962a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.848+0000 7f09c866cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c969840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.848+0000 7f09ade00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c961b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.848+0000 7f09ade00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c961b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54764/0 (socket says 172.21.15.131:54764) 2024-05-10T05:44:44.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.848+0000 7f09c866cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c8881d0 con 0x555e5c9625f0 2024-05-10T05:44:44.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.848+0000 7f09c866cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c885b90 con 0x555e5c962f40 2024-05-10T05:44:44.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.848+0000 7f09c866cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c881480 con 0x555e5c961720 2024-05-10T05:44:44.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.848+0000 7f09ade00700 1 -- 172.21.15.131:0/3887731184 learned_addr learned my addr 172.21.15.131:0/3887731184 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:44.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.849+0000 7f09ae800700 1 --2- 172.21.15.131:0/3887731184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c969840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.849+0000 7f09ade00700 1 -- 172.21.15.131:0/3887731184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 msgr2=0x555e5c969840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:44.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.849+0000 7f09ade00700 1 --2- 172.21.15.131:0/3887731184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c969840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.850+0000 7f09ade00700 1 -- 172.21.15.131:0/3887731184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 msgr2=0x555e5c962a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:44.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.850+0000 7f09ade00700 1 --2- 172.21.15.131:0/3887731184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c962a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.850+0000 7f09ade00700 1 -- 172.21.15.131:0/3887731184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e5c702430 con 0x555e5c961720 2024-05-10T05:44:44.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.851+0000 7f09ade00700 1 --2- 172.21.15.131:0/3887731184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c961b70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f09a40120f0 tx=0x7f09a401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed05af926cb7e13 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.855+0000 7f0997000700 1 -- 172.21.15.131:0/3887731184 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4019040 con 0x555e5c961720 2024-05-10T05:44:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.855+0000 7f0997000700 1 -- 172.21.15.131:0/3887731184 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09a4025040 con 0x555e5c961720 2024-05-10T05:44:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.855+0000 7f0997000700 1 -- 172.21.15.131:0/3887731184 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4029df0 con 0x555e5c961720 2024-05-10T05:44:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.856+0000 7f09c866cb80 1 -- 172.21.15.131:0/3887731184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 msgr2=0x555e5c961b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.856+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3887731184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c961b70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f09a40120f0 tx=0x7f09a401a000 comp rx=0 tx=0).stop 2024-05-10T05:44:44.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.860+0000 7f09c866cb80 1 -- 172.21.15.131:0/3887731184 shutdown_connections 2024-05-10T05:44:44.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.860+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3887731184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c969840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.860+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3887731184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c961b70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.860+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3887731184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c962a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.860+0000 7f09c866cb80 1 -- 172.21.15.131:0/3887731184 >> 172.21.15.131:0/3887731184 conn(0x555e5c8c3ae0 msgr2=0x555e5c95c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:44.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.864+0000 7f09c866cb80 1 -- 172.21.15.131:0/3887731184 shutdown_connections 2024-05-10T05:44:44.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.864+0000 7f09c866cb80 1 -- 172.21.15.131:0/3887731184 wait complete. 2024-05-10T05:44:44.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.865+0000 7f09c866cb80 1 -- start start 2024-05-10T05:44:44.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.867+0000 7f09c866cb80 1 Processor -- start 2024-05-10T05:44:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.869+0000 7f09c866cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c96b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.869+0000 7f09ade00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c96b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.869+0000 7f09ade00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c96b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54772/0 (socket says 172.21.15.131:54772) 2024-05-10T05:44:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.869+0000 7f09ade00700 1 -- 172.21.15.131:0/3952190541 learned_addr learned my addr 172.21.15.131:0/3952190541 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.869+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c96bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.869+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c972c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.870+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c81d5e0 con 0x555e5c9625f0 2024-05-10T05:44:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.870+0000 7f09ae800700 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c972c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.870+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c81d280 con 0x555e5c962f40 2024-05-10T05:44:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.870+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c704720 con 0x555e5c961720 2024-05-10T05:44:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.870+0000 7f09ad400700 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c96bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.871+0000 7f09ade00700 1 -- 172.21.15.131:0/3952190541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 msgr2=0x555e5c972c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.871+0000 7f09ade00700 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c972c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.871+0000 7f09ade00700 1 -- 172.21.15.131:0/3952190541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 msgr2=0x555e5c96bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.871+0000 7f09ade00700 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c96bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.871+0000 7f09ade00700 1 -- 172.21.15.131:0/3952190541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a4014e10 con 0x555e5c961720 2024-05-10T05:44:44.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.871+0000 7f098b400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:44.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.871+0000 7f09ade00700 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c96b7f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f09a401fea0 tx=0x7f09a402adb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb50b89d13f4fc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:44.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.872+0000 7f098b400700 1 -- 172.21.15.131:0/3952190541 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4003ac0 con 0x555e5c961720 2024-05-10T05:44:44.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.873+0000 7f098b400700 1 -- 172.21.15.131:0/3952190541 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f09a4003c20 con 0x555e5c961720 2024-05-10T05:44:44.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.873+0000 7f098b400700 10 client.0 inactive, discarding osd_map(82..82 src has 1..82) v4 2024-05-10T05:44:44.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.873+0000 7f09c866cb80 10 client.0 _ll_register_callbacks cb 0x555e5c826bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:44.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.873+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e5c7da2f0 con 0x555e5c961720 2024-05-10T05:44:44.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.874+0000 7f098b400700 1 -- 172.21.15.131:0/3952190541 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09a4003e20 con 0x555e5c961720 2024-05-10T05:44:44.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.874+0000 7f09c866cb80 10 client.27439 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:44.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.874+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x555e5c6aebe0 con 0x555e5c961720 2024-05-10T05:44:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.874+0000 7f098b400700 1 -- 172.21.15.131:0/3952190541 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f09a4028070 con 0x555e5c961720 2024-05-10T05:44:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.874+0000 7f09c866cb80 10 client.27439 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.874+0000 7f09c866cb80 10 client.27439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.874+0000 7f09c866cb80 20 client.27439 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.875+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555e5c977030 0x555e5c979540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.875+0000 7f09c866cb80 4 client.27439 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.875+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e5c930800 con 0x555e5c977030 2024-05-10T05:44:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.875+0000 7f09ad400700 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555e5c977030 0x555e5c979540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.876+0000 7f09ad400700 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555e5c977030 0x555e5c979540 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df9f30af1a13f5b server_cookie=6af2dc2d30e061f4 in_seq=0 out_seq=0 2024-05-10T05:44:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.876+0000 7f098b400700 10 client.27439 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.876+0000 7f098b400700 1 -- 172.21.15.131:0/3952190541 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e5c930800 con 0x555e5c977030 2024-05-10T05:44:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.876+0000 7f098b400700 10 client.27439 handle_command_reply: tid=0 2024-05-10T05:44:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.876+0000 7f09c866cb80 1 client.27439 shutdown 2024-05-10T05:44:44.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.877+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 msgr2=0x555e5c96b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:44.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.877+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c96b7f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f09a401fea0 tx=0x7f09a402adb0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.877+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 shutdown_connections 2024-05-10T05:44:44.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.877+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555e5c977030 0x555e5c979540 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.877+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c972c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.878+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c961720 0x555e5c96b7f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.878+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3952190541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c96bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.878+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 >> 172.21.15.131:0/3952190541 conn(0x555e5c8c3ae0 msgr2=0x555e5c95c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:44.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.877+0000 7f098b400700 0 client.27439 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:44.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.878+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 shutdown_connections 2024-05-10T05:44:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.878+0000 7f09c866cb80 1 -- 172.21.15.131:0/3952190541 wait complete. 2024-05-10T05:44:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.879+0000 7f09c866cb80 20 client.27439 trim_cache size 0 max 16384 2024-05-10T05:44:44.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.925+0000 7f09c866cb80 1 Processor -- start 2024-05-10T05:44:44.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.931+0000 7f09c866cb80 1 -- start start 2024-05-10T05:44:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.932+0000 7f09c866cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c961720 0x555e5c979470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.933+0000 7f09c866cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c9625f0 0x555e5c972c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.933+0000 7f09c866cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c966390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.933+0000 7f09ad400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c9625f0 0x555e5c972c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.933+0000 7f09ad400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c9625f0 0x555e5c972c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54786/0 (socket says 172.21.15.131:54786) 2024-05-10T05:44:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.934+0000 7f09ad400700 1 -- 172.21.15.131:0/1569382587 learned_addr learned my addr 172.21.15.131:0/1569382587 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.934+0000 7f09c866cb80 1 -- 172.21.15.131:0/1569382587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c95b450 con 0x555e5c961720 2024-05-10T05:44:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.934+0000 7f09c866cb80 1 -- 172.21.15.131:0/1569382587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c95d820 con 0x555e5c962f40 2024-05-10T05:44:44.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.934+0000 7f09c866cb80 1 -- 172.21.15.131:0/1569382587 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c9772b0 con 0x555e5c9625f0 2024-05-10T05:44:44.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.934+0000 7f09ade00700 1 --2- 172.21.15.131:0/1569382587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c961720 0x555e5c979470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.934+0000 7f09ae800700 1 --2- 172.21.15.131:0/1569382587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c966390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.935+0000 7f09ad400700 1 -- 172.21.15.131:0/1569382587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 msgr2=0x555e5c966390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:44.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.935+0000 7f09ad400700 1 --2- 172.21.15.131:0/1569382587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c966390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.936+0000 7f09ad400700 1 -- 172.21.15.131:0/1569382587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c961720 msgr2=0x555e5c979470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:44.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.936+0000 7f09ad400700 1 --2- 172.21.15.131:0/1569382587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c961720 0x555e5c979470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.936+0000 7f09ad400700 1 -- 172.21.15.131:0/1569382587 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e5c8475e0 con 0x555e5c9625f0 2024-05-10T05:44:44.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.936+0000 7f09ade00700 1 --2- 172.21.15.131:0/1569382587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c961720 0x555e5c979470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.937+0000 7f09ad400700 1 --2- 172.21.15.131:0/1569382587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c9625f0 0x555e5c972c20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f09980048a0 tx=0x7f099809f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebad83daeaa4f230 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.937+0000 7f0997a00700 1 -- 172.21.15.131:0/1569382587 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998009280 con 0x555e5c9625f0 2024-05-10T05:44:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.938+0000 7f0997a00700 1 -- 172.21.15.131:0/1569382587 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09980ba0b0 con 0x555e5c9625f0 2024-05-10T05:44:44.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.940+0000 7f0997a00700 1 -- 172.21.15.131:0/1569382587 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099809e270 con 0x555e5c9625f0 2024-05-10T05:44:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.941+0000 7f09c866cb80 1 -- 172.21.15.131:0/1569382587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c9625f0 msgr2=0x555e5c972c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.942+0000 7f09c866cb80 1 --2- 172.21.15.131:0/1569382587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c9625f0 0x555e5c972c20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f09980048a0 tx=0x7f099809f310 comp rx=0 tx=0).stop 2024-05-10T05:44:44.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.942+0000 7f09c866cb80 1 -- 172.21.15.131:0/1569382587 shutdown_connections 2024-05-10T05:44:44.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.942+0000 7f09c866cb80 1 --2- 172.21.15.131:0/1569382587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c961720 0x555e5c979470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.942+0000 7f09c866cb80 1 --2- 172.21.15.131:0/1569382587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c962f40 0x555e5c966390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.942+0000 7f09c866cb80 1 --2- 172.21.15.131:0/1569382587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c9625f0 0x555e5c972c20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.942+0000 7f09c866cb80 1 -- 172.21.15.131:0/1569382587 >> 172.21.15.131:0/1569382587 conn(0x555e5c8c3ae0 msgr2=0x555e5c95ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:44.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.946+0000 7f09c866cb80 1 -- 172.21.15.131:0/1569382587 shutdown_connections 2024-05-10T05:44:44.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.947+0000 7f09c866cb80 1 -- 172.21.15.131:0/1569382587 wait complete. 2024-05-10T05:44:44.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.947+0000 7f09c866cb80 1 -- start start 2024-05-10T05:44:44.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.948+0000 7f09c866cb80 1 Processor -- start 2024-05-10T05:44:44.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.951+0000 7f09c866cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c961720 0x555e5c9c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.951+0000 7f09c866cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c9c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.952+0000 7f09c866cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c962f40 0x555e5c9ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.952+0000 7f09c866cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c9cd310 con 0x555e5c9625f0 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.952+0000 7f09c866cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c9cd450 con 0x555e5c961720 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.952+0000 7f09c866cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5c9de5f0 con 0x555e5c962f40 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.952+0000 7f09c866cb80 10 client.0 _ll_register_callbacks cb 0x555e5c826bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.953+0000 7f09ae800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c962f40 0x555e5c9ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.953+0000 7f09ae800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c962f40 0x555e5c9ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54798/0 (socket says 172.21.15.131:54798) 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.953+0000 7f09ae800700 1 -- 172.21.15.131:0/3787684058 learned_addr learned my addr 172.21.15.131:0/3787684058 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.955+0000 7f09ade00700 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c961720 0x555e5c9c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.955+0000 7f09ad400700 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c9c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.955+0000 7f09ae800700 1 -- 172.21.15.131:0/3787684058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c961720 msgr2=0x555e5c9c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.955+0000 7f09ae800700 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c961720 0x555e5c9c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.955+0000 7f09ae800700 1 -- 172.21.15.131:0/3787684058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 msgr2=0x555e5c9c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.956+0000 7f09ae800700 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c9c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.956+0000 7f09ae800700 1 -- 172.21.15.131:0/3787684058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e5c6aebe0 con 0x555e5c962f40 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.959+0000 7f09ad400700 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c9c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.959+0000 7f09ae800700 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c962f40 0x555e5c9ccdd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f099c0099c0 tx=0x7f099c00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf8b36b81b410c80 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.959+0000 7f098b400700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.960+0000 7f098b400700 1 -- 172.21.15.131:0/3787684058 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c01d040 con 0x555e5c962f40 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.960+0000 7f098b400700 1 -- 172.21.15.131:0/3787684058 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f099c003da0 con 0x555e5c962f40 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.960+0000 7f09c866cb80 1 -- 172.21.15.131:0/3787684058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09a4025040 con 0x555e5c962f40 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.961+0000 7f098b400700 1 -- 172.21.15.131:0/3787684058 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f099c003f60 con 0x555e5c962f40 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.961+0000 7f09c866cb80 10 client.27445 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.961+0000 7f09c866cb80 1 -- 172.21.15.131:0/3787684058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f09a810a290 con 0x555e5c962f40 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.963+0000 7f098b400700 1 -- 172.21.15.131:0/3787684058 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f099c0040c0 con 0x555e5c962f40 2024-05-10T05:44:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.963+0000 7f098b400700 1 -- 172.21.15.131:0/3787684058 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f099c02e430 con 0x555e5c962f40 2024-05-10T05:44:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.963+0000 7f09c866cb80 10 client.27445 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.963+0000 7f09c866cb80 10 client.27445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.963+0000 7f09c866cb80 20 client.27445 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.963+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555e5c9e0100 0x555e5c9e2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.963+0000 7f09c866cb80 4 client.27445 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T05:44:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.963+0000 7f09c866cb80 1 -- 172.21.15.131:0/3787684058 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f09a4025040 con 0x555e5c9e0100 2024-05-10T05:44:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.964+0000 7f09ade00700 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555e5c9e0100 0x555e5c9e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.964+0000 7f09ade00700 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555e5c9e0100 0x555e5c9e2520 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74d4cf84abed6ab server_cookie=1a90ab093557aa26 in_seq=0 out_seq=0 2024-05-10T05:44:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f098b400700 10 client.27445 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f098b400700 1 -- 172.21.15.131:0/3787684058 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f09a4025040 con 0x555e5c9e0100 2024-05-10T05:44:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f098b400700 10 client.27445 handle_command_reply: tid=0 2024-05-10T05:44:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f09c866cb80 1 client.27445 shutdown 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f09c866cb80 1 -- 172.21.15.131:0/3787684058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c962f40 msgr2=0x555e5c9ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c962f40 0x555e5c9ccdd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f099c0099c0 tx=0x7f099c00bf90 comp rx=0 tx=0).stop 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f09c866cb80 1 -- 172.21.15.131:0/3787684058 shutdown_connections 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x555e5c9e0100 0x555e5c9e2520 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e5c962f40 0x555e5c9ccdd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e5c961720 0x555e5c9c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f098b400700 0 client.27445 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f09c866cb80 1 --2- 172.21.15.131:0/3787684058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e5c9625f0 0x555e5c9c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.967+0000 7f09c866cb80 1 -- 172.21.15.131:0/3787684058 >> 172.21.15.131:0/3787684058 conn(0x555e5c8c3ae0 msgr2=0x555e5c95ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.968+0000 7f09c866cb80 1 -- 172.21.15.131:0/3787684058 shutdown_connections 2024-05-10T05:44:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.968+0000 7f09c866cb80 1 -- 172.21.15.131:0/3787684058 wait complete. 2024-05-10T05:44:44.972 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:44:44.972 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-05-10T05:44:44.972 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "e4a65dac-e5d1-4a1c-ad57-1e855231a569", 2024-05-10T05:44:44.972 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-05-10T05:44:44.972 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:44:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.968+0000 7f09c866cb80 20 client.27445 trim_cache size 0 max 16384 2024-05-10T05:44:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.970+0000 7f09b0800700 1 -- 172.21.15.131:0/1477026971 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09900a1290 msgr2=0x7f09900a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.970+0000 7f09b0800700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09900a1290 0x7f09900a37a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f09a40148c0 tx=0x7f09a4016040 comp rx=0 tx=0).stop 2024-05-10T05:44:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.970+0000 7f09b0800700 1 -- 172.21.15.131:0/1477026971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09a8108b60 msgr2=0x7f09a81a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.970+0000 7f09b0800700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09a8108b60 0x7f09a81a0cc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f099800b450 tx=0x7f099800b760 comp rx=0 tx=0).stop 2024-05-10T05:44:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.971+0000 7f09b0800700 1 -- 172.21.15.131:0/1477026971 shutdown_connections 2024-05-10T05:44:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.971+0000 7f09b0800700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f09900a1290 0x7f09900a37a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.971+0000 7f09b0800700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09a8110320 0x7f09a81a7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.971+0000 7f09b0800700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09a8108210 0x7f09a81a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.971+0000 7f09b0800700 1 --2- 172.21.15.131:0/1477026971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09a8108b60 0x7f09a81a0cc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.971+0000 7f09b0800700 1 -- 172.21.15.131:0/1477026971 >> 172.21.15.131:0/1477026971 conn(0x7f09a806ea00 msgr2=0x7f09a810bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.971+0000 7f09b0800700 1 -- 172.21.15.131:0/1477026971 shutdown_connections 2024-05-10T05:44:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:44.972+0000 7f09b0800700 1 -- 172.21.15.131:0/1477026971 wait complete. 2024-05-10T05:44:45.036 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:44:45.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.649+0000 7f212de00700 1 -- 172.21.15.131:0/2887684847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128100ac0 msgr2=0x7f2128100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:45.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.649+0000 7f212de00700 1 --2- 172.21.15.131:0/2887684847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128100ac0 0x7f2128100ed0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f21100099c0 tx=0x7f2110009cd0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.649+0000 7f212de00700 1 -- 172.21.15.131:0/2887684847 shutdown_connections 2024-05-10T05:44:45.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.649+0000 7f212de00700 1 --2- 172.21.15.131:0/2887684847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128102580 0x7f2128108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.649+0000 7f212de00700 1 --2- 172.21.15.131:0/2887684847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128100ac0 0x7f2128100ed0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.649+0000 7f212de00700 1 --2- 172.21.15.131:0/2887684847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2128101be0 0x7f2128101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.649+0000 7f212de00700 1 -- 172.21.15.131:0/2887684847 >> 172.21.15.131:0/2887684847 conn(0x7f21280fc0d0 msgr2=0x7f21280fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.649+0000 7f212de00700 1 -- 172.21.15.131:0/2887684847 shutdown_connections 2024-05-10T05:44:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.649+0000 7f212de00700 1 -- 172.21.15.131:0/2887684847 wait complete. 2024-05-10T05:44:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.651+0000 7f212de00700 1 Processor -- start 2024-05-10T05:44:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.651+0000 7f212de00700 1 -- start start 2024-05-10T05:44:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.652+0000 7f212de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128100ac0 0x7f212819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.652+0000 7f212de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128101be0 0x7f212819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.652+0000 7f212de00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2128102580 0x7f21281a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.652+0000 7f212de00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281a1c60 con 0x7f2128102580 2024-05-10T05:44:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.652+0000 7f212de00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281a1dd0 con 0x7f2128101be0 2024-05-10T05:44:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.652+0000 7f212de00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281a1f90 con 0x7f2128100ac0 2024-05-10T05:44:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.652+0000 7f2126a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128100ac0 0x7f212819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.652+0000 7f2126000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128101be0 0x7f212819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.652+0000 7f2126000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128101be0 0x7f212819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:50182/0 (socket says 172.21.15.131:50182) 2024-05-10T05:44:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.652+0000 7f2126000700 1 -- 172.21.15.131:0/2036532327 learned_addr learned my addr 172.21.15.131:0/2036532327 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.653+0000 7f2127400700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2128102580 0x7f21281a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.653+0000 7f2126000700 1 -- 172.21.15.131:0/2036532327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128100ac0 msgr2=0x7f212819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.653+0000 7f2126000700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128100ac0 0x7f212819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.653+0000 7f2126000700 1 -- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2128102580 msgr2=0x7f21281a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.653+0000 7f2126000700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2128102580 0x7f21281a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.653+0000 7f2126000700 1 -- 172.21.15.131:0/2036532327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21100095f0 con 0x7f2128101be0 2024-05-10T05:44:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.653+0000 7f2127400700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2128102580 0x7f21281a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:44:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.653+0000 7f2126000700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128101be0 0x7f212819a6c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f211800d950 tx=0x7f211800dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:45.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.653+0000 7f212ca00700 1 -- 172.21.15.131:0/2036532327 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118009b40 con 0x7f2128101be0 2024-05-10T05:44:45.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.654+0000 7f212ca00700 1 -- 172.21.15.131:0/2036532327 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2118009ca0 con 0x7f2128101be0 2024-05-10T05:44:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.654+0000 7f212de00700 1 -- 172.21.15.131:0/2036532327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21281a2270 con 0x7f2128101be0 2024-05-10T05:44:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.654+0000 7f212de00700 1 -- 172.21.15.131:0/2036532327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21281a27d0 con 0x7f2128101be0 2024-05-10T05:44:45.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.655+0000 7f212ca00700 1 -- 172.21.15.131:0/2036532327 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211800f620 con 0x7f2128101be0 2024-05-10T05:44:45.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.661+0000 7f212ca00700 1 -- 172.21.15.131:0/2036532327 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f211800f780 con 0x7f2128101be0 2024-05-10T05:44:45.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.664+0000 7f212ca00700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f210c0a12c0 0x7f210c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.664+0000 7f212ca00700 1 -- 172.21.15.131:0/2036532327 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f21180ccb70 con 0x7f2128101be0 2024-05-10T05:44:45.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.664+0000 7f2126a00700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f210c0a12c0 0x7f210c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.666+0000 7f2126a00700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f210c0a12c0 0x7f210c0a37d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2110006010 tx=0x7f2110000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:45.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.667+0000 7f2145d8ab80 1 Processor -- start 2024-05-10T05:44:45.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.668+0000 7f2145d8ab80 1 -- start start 2024-05-10T05:44:45.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.669+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d76a0 0x559ec91d7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.669+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8570 0x559ec91d8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.669+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 0x559ec91df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.669+0000 7f2145d8ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec90fe150 con 0x559ec91d8570 2024-05-10T05:44:45.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.669+0000 7f2145d8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec90fbb10 con 0x559ec91d76a0 2024-05-10T05:44:45.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.670+0000 7f2127400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 0x559ec91df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.670+0000 7f2126a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d76a0 0x559ec91d7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.670+0000 7f2127400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 0x559ec91df7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54830/0 (socket says 172.21.15.131:54830) 2024-05-10T05:44:45.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.670+0000 7f2127400700 1 -- 172.21.15.131:0/1907286884 learned_addr learned my addr 172.21.15.131:0/1907286884 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:45.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.670+0000 7f2145d8ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec90f7400 con 0x559ec91d8ec0 2024-05-10T05:44:45.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.670+0000 7f2126000700 1 --2- 172.21.15.131:0/1907286884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8570 0x559ec91d8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.670+0000 7f2127400700 1 -- 172.21.15.131:0/1907286884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d76a0 msgr2=0x559ec91d7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:45.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.670+0000 7f2127400700 1 --2- 172.21.15.131:0/1907286884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d76a0 0x559ec91d7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.670+0000 7f2127400700 1 -- 172.21.15.131:0/1907286884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8570 msgr2=0x559ec91d8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:45.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.671+0000 7f2127400700 1 --2- 172.21.15.131:0/1907286884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8570 0x559ec91d8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.671+0000 7f2127400700 1 -- 172.21.15.131:0/1907286884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ec8f783b0 con 0x559ec91d8ec0 2024-05-10T05:44:45.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.672+0000 7f2127400700 1 --2- 172.21.15.131:0/1907286884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 0x559ec91df7c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2120009dd0 tx=0x7f21200077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b300ea72b63f630c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:45.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.673+0000 7f2117e00700 1 -- 172.21.15.131:0/1907286884 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120014610 con 0x559ec91d8ec0 2024-05-10T05:44:45.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.673+0000 7f2117e00700 1 -- 172.21.15.131:0/1907286884 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f212000f050 con 0x559ec91d8ec0 2024-05-10T05:44:45.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.674+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1907286884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 msgr2=0x559ec91df7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:45.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.674+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/1907286884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 0x559ec91df7c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2120009dd0 tx=0x7f21200077e0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.674+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1907286884 shutdown_connections 2024-05-10T05:44:45.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.674+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/1907286884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 0x559ec91df7c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.674+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/1907286884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d76a0 0x559ec91d7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.674+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/1907286884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8570 0x559ec91d8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.674+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1907286884 >> 172.21.15.131:0/1907286884 conn(0x559ec9139a60 msgr2=0x559ec91d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:45.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.674+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1907286884 shutdown_connections 2024-05-10T05:44:45.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.674+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1907286884 wait complete. 2024-05-10T05:44:45.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.675+0000 7f2145d8ab80 1 -- start start 2024-05-10T05:44:45.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.676+0000 7f2145d8ab80 1 Processor -- start 2024-05-10T05:44:45.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.678+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d76a0 0x559ec91e9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.678+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.678+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 0x559ec91f1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.678+0000 7f2145d8ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec9093560 con 0x559ec91d76a0 2024-05-10T05:44:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.678+0000 7f2145d8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec9093200 con 0x559ec91d8570 2024-05-10T05:44:45.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.678+0000 7f2145d8ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec8f7a6a0 con 0x559ec91d8ec0 2024-05-10T05:44:45.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.678+0000 7f2145d8ab80 10 client.0 _ll_register_callbacks cb 0x559ec909cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:45.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2126a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d76a0 0x559ec91e9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2127400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 0x559ec91f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2126a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d76a0 0x559ec91e9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54892/0 (socket says 172.21.15.131:54892) 2024-05-10T05:44:45.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2126a00700 1 -- 172.21.15.131:0/4277723702 learned_addr learned my addr 172.21.15.131:0/4277723702 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:45.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2126000700 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2126a00700 1 -- 172.21.15.131:0/4277723702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 msgr2=0x559ec91f1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:45.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2126a00700 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 0x559ec91f1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2126a00700 1 -- 172.21.15.131:0/4277723702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 msgr2=0x559ec91ea480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:45.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2126a00700 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ea480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2126a00700 1 -- 172.21.15.131:0/4277723702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2120009a00 con 0x559ec91d76a0 2024-05-10T05:44:45.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.679+0000 7f2126000700 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ea480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:44:45.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.680+0000 7f2126a00700 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d76a0 0x559ec91e9f40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2110015000 tx=0x7f2110009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea49bd817584af3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:45.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.680+0000 7f2114c00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:44:45.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.680+0000 7f2114c00700 1 -- 172.21.15.131:0/4277723702 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211002b040 con 0x559ec91d76a0 2024-05-10T05:44:45.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.681+0000 7f2114c00700 1 -- 172.21.15.131:0/4277723702 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f2110003f60 con 0x559ec91d76a0 2024-05-10T05:44:45.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.681+0000 7f2145d8ab80 1 -- 172.21.15.131:0/4277723702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ec9050270 con 0x559ec91d76a0 2024-05-10T05:44:45.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.682+0000 7f2114c00700 1 -- 172.21.15.131:0/4277723702 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f211001d070 con 0x559ec91d76a0 2024-05-10T05:44:45.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.682+0000 7f2145d8ab80 10 client.17061 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:45.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.682+0000 7f2145d8ab80 1 -- 172.21.15.131:0/4277723702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559ec8f24b60 con 0x559ec91d76a0 2024-05-10T05:44:45.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.682+0000 7f2114c00700 1 -- 172.21.15.131:0/4277723702 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ec9050270 con 0x559ec91d76a0 2024-05-10T05:44:45.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.683+0000 7f2114c00700 1 -- 172.21.15.131:0/4277723702 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2110030430 con 0x559ec91d76a0 2024-05-10T05:44:45.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.684+0000 7f2145d8ab80 10 client.17061 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:45.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.684+0000 7f2145d8ab80 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:45.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.684+0000 7f2145d8ab80 20 client.17061 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:45.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.684+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559ec91f5b90 0x559ec91f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.684+0000 7f2145d8ab80 4 client.17061 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:44:45.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.684+0000 7f2145d8ab80 1 -- 172.21.15.131:0/4277723702 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ec91a6780 con 0x559ec91f5b90 2024-05-10T05:44:45.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.685+0000 7f2126000700 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559ec91f5b90 0x559ec91f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.687+0000 7f2114c00700 10 client.17061 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:45.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.687+0000 7f2126000700 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559ec91f5b90 0x559ec91f7fe0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d38069afd959634 server_cookie=cfc4526f04f206b1 in_seq=0 out_seq=0 2024-05-10T05:44:45.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.689+0000 7f2114c00700 1 -- 172.21.15.131:0/4277723702 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559ec91a6780 con 0x559ec91f5b90 2024-05-10T05:44:45.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.689+0000 7f2114c00700 10 client.17061 handle_command_reply: tid=0 2024-05-10T05:44:45.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.690+0000 7f2145d8ab80 1 client.17061 shutdown 2024-05-10T05:44:45.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.695+0000 7f2145d8ab80 1 -- 172.21.15.131:0/4277723702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d76a0 msgr2=0x559ec91e9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:45.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.695+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d76a0 0x559ec91e9f40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2110015000 tx=0x7f2110009cd0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.696+0000 7f2145d8ab80 1 -- 172.21.15.131:0/4277723702 shutdown_connections 2024-05-10T05:44:45.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.696+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559ec91f5b90 0x559ec91f7fe0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.696+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d76a0 0x559ec91e9f40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.696+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d8ec0 0x559ec91f1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.696+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/4277723702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ea480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.696+0000 7f2145d8ab80 1 -- 172.21.15.131:0/4277723702 >> 172.21.15.131:0/4277723702 conn(0x559ec9139a60 msgr2=0x559ec91d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:45.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.696+0000 7f2114c00700 0 client.17061 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:45.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.698+0000 7f2145d8ab80 1 -- 172.21.15.131:0/4277723702 shutdown_connections 2024-05-10T05:44:45.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.698+0000 7f2145d8ab80 1 -- 172.21.15.131:0/4277723702 wait complete. 2024-05-10T05:44:45.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.705+0000 7f2145d8ab80 20 client.17061 trim_cache size 0 max 16384 2024-05-10T05:44:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:45 smithi157 ceph-mon[78427]: scrub queued for path: / 2024-05-10T05:44:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:45 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:45 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:44:45.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:45 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.800+0000 7f2145d8ab80 1 Processor -- start 2024-05-10T05:44:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.800+0000 7f2145d8ab80 1 -- start start 2024-05-10T05:44:45.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.801+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91e8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.801+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8570 0x559ec91f1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.801+0000 7f2126a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.801+0000 7f2126a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91e8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54860/0 (socket says 172.21.15.131:54860) 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.802+0000 7f2126a00700 1 -- 172.21.15.131:0/3815993553 learned_addr learned my addr 172.21.15.131:0/3815993553 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.802+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/3815993553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8ec0 0x559ec91e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.802+0000 7f2145d8ab80 1 -- 172.21.15.131:0/3815993553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec91eaf90 con 0x559ec91d8570 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.802+0000 7f2145d8ab80 1 -- 172.21.15.131:0/3815993553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec91eaa80 con 0x559ec91d8ec0 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.802+0000 7f2145d8ab80 1 -- 172.21.15.131:0/3815993553 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec91cc7c0 con 0x559ec91d76a0 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.802+0000 7f2127400700 1 --2- 172.21.15.131:0/3815993553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8ec0 0x559ec91e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.803+0000 7f2126a00700 1 -- 172.21.15.131:0/3815993553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8ec0 msgr2=0x559ec91e8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.803+0000 7f2126a00700 1 --2- 172.21.15.131:0/3815993553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8ec0 0x559ec91e8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.803+0000 7f2126a00700 1 -- 172.21.15.131:0/3815993553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8570 msgr2=0x559ec91f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:45.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.803+0000 7f2126a00700 1 --2- 172.21.15.131:0/3815993553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8570 0x559ec91f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.803+0000 7f2126a00700 1 -- 172.21.15.131:0/3815993553 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ec90bd560 con 0x559ec91d76a0 2024-05-10T05:44:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.804+0000 7f2126a00700 1 --2- 172.21.15.131:0/3815993553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91e8880 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2110005d80 tx=0x7f2110005630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af45526323353a97 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.810+0000 7f211d200700 1 -- 172.21.15.131:0/3815993553 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211002b040 con 0x559ec91d76a0 2024-05-10T05:44:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.810+0000 7f211d200700 1 -- 172.21.15.131:0/3815993553 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21100304f0 con 0x559ec91d76a0 2024-05-10T05:44:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.810+0000 7f211d200700 1 -- 172.21.15.131:0/3815993553 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211000fb80 con 0x559ec91d76a0 2024-05-10T05:44:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.816+0000 7f2145d8ab80 1 -- 172.21.15.131:0/3815993553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 msgr2=0x559ec91e8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.816+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/3815993553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91e8880 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2110005d80 tx=0x7f2110005630 comp rx=0 tx=0).stop 2024-05-10T05:44:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.816+0000 7f2145d8ab80 1 -- 172.21.15.131:0/3815993553 shutdown_connections 2024-05-10T05:44:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.816+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/3815993553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8ec0 0x559ec91e8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.816+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/3815993553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91e8880 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.816+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/3815993553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8570 0x559ec91f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.816+0000 7f2145d8ab80 1 -- 172.21.15.131:0/3815993553 >> 172.21.15.131:0/3815993553 conn(0x559ec9139a60 msgr2=0x559ec91db7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:45.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.821+0000 7f2145d8ab80 1 -- 172.21.15.131:0/3815993553 shutdown_connections 2024-05-10T05:44:45.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.827+0000 7f2145d8ab80 1 -- 172.21.15.131:0/3815993553 wait complete. 2024-05-10T05:44:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.828+0000 7f2145d8ab80 1 -- start start 2024-05-10T05:44:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.828+0000 7f2145d8ab80 1 Processor -- start 2024-05-10T05:44:45.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.832+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91eee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.832+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.833+0000 7f2145d8ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8ec0 0x559ec91f0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.833+0000 7f2126a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.833+0000 7f2126a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91eee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54870/0 (socket says 172.21.15.131:54870) 2024-05-10T05:44:45.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.833+0000 7f2145d8ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec91f07c0 con 0x559ec91d8ec0 2024-05-10T05:44:45.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.833+0000 7f2145d8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec91f0930 con 0x559ec91d8570 2024-05-10T05:44:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.833+0000 7f2145d8ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec91f0af0 con 0x559ec91d76a0 2024-05-10T05:44:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.833+0000 7f2126000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.834+0000 7f2126000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ef3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:50220/0 (socket says 172.21.15.131:50220) 2024-05-10T05:44:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.834+0000 7f2126000700 1 -- 172.21.15.131:0/1323185887 learned_addr learned my addr 172.21.15.131:0/1323185887 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:44:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.834+0000 7f2145d8ab80 10 client.0 _ll_register_callbacks cb 0x559ec909cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:44:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.834+0000 7f2127400700 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8ec0 0x559ec91f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.835+0000 7f2126000700 1 -- 172.21.15.131:0/1323185887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 msgr2=0x559ec91eee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:45.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.835+0000 7f2126000700 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91eee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.835+0000 7f2126000700 1 -- 172.21.15.131:0/1323185887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8ec0 msgr2=0x559ec91f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:44:45.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.835+0000 7f2126000700 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8ec0 0x559ec91f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.835+0000 7f2126000700 1 -- 172.21.15.131:0/1323185887 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ec8f24b60 con 0x559ec91d8570 2024-05-10T05:44:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.836+0000 7f2126000700 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ef3a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2118005e10 tx=0x7f21180ca610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60af6401b31ac9a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:44:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.836+0000 7f2114c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:44:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.837+0000 7f2114c00700 1 -- 172.21.15.131:0/1323185887 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21180b3260 con 0x559ec91d8570 2024-05-10T05:44:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.837+0000 7f2114c00700 1 -- 172.21.15.131:0/1323185887 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f211809b160 con 0x559ec91d8570 2024-05-10T05:44:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.838+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1323185887 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f212000f050 con 0x559ec91d8570 2024-05-10T05:44:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.838+0000 7f2114c00700 1 -- 172.21.15.131:0/1323185887 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21180b33c0 con 0x559ec91d8570 2024-05-10T05:44:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.840+0000 7f2145d8ab80 10 client.27455 fetch_fsmap learned FSMap version 29 2024-05-10T05:44:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.840+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1323185887 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559ec8f7ccd0 con 0x559ec91d8570 2024-05-10T05:44:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.841+0000 7f2114c00700 1 -- 172.21.15.131:0/1323185887 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21180b7080 con 0x559ec91d8570 2024-05-10T05:44:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.841+0000 7f2114c00700 1 -- 172.21.15.131:0/1323185887 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f21180a3010 con 0x559ec91d8570 2024-05-10T05:44:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.841+0000 7f2145d8ab80 10 client.27455 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T05:44:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.841+0000 7f2145d8ab80 10 client.27455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:44:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.841+0000 7f2145d8ab80 20 client.27455 populate_metadata read hostname 'smithi131' 2024-05-10T05:44:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.841+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559ec924cbb0 0x559ec924efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:44:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.841+0000 7f2145d8ab80 4 client.27455 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:44:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.841+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1323185887 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559ec90efec0 con 0x559ec924cbb0 2024-05-10T05:44:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.841+0000 7f2126a00700 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559ec924cbb0 0x559ec924efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:44:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.842+0000 7f2114c00700 10 client.27455 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.842+0000 7f2126a00700 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559ec924cbb0 0x559ec924efd0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f1ea3500c6e7ce1 server_cookie=92b93565afdec1e2 in_seq=0 out_seq=0 2024-05-10T05:44:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.842+0000 7f2114c00700 1 -- 172.21.15.131:0/1323185887 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559ec90efec0 con 0x559ec924cbb0 2024-05-10T05:44:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.843+0000 7f2114c00700 10 client.27455 handle_command_reply: tid=0 2024-05-10T05:44:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.843+0000 7f2145d8ab80 1 client.27455 shutdown 2024-05-10T05:44:45.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.849+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1323185887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 msgr2=0x559ec91ef3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:45.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.849+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ef3a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2118005e10 tx=0x7f21180ca610 comp rx=0 tx=0).stop 2024-05-10T05:44:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.853+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1323185887 shutdown_connections 2024-05-10T05:44:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.853+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x559ec924cbb0 0x559ec924efd0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.853+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559ec91d8ec0 0x559ec91f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.853+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ec91d76a0 0x559ec91eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.853+0000 7f2145d8ab80 1 --2- 172.21.15.131:0/1323185887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ec91d8570 0x559ec91ef3a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.853+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1323185887 >> 172.21.15.131:0/1323185887 conn(0x559ec9139a60 msgr2=0x559ec91db7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.853+0000 7f2114c00700 0 client.27455 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:44:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.853+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1323185887 shutdown_connections 2024-05-10T05:44:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.853+0000 7f2145d8ab80 1 -- 172.21.15.131:0/1323185887 wait complete. 2024-05-10T05:44:45.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.853+0000 7f2145d8ab80 20 client.27455 trim_cache size 0 max 16384 2024-05-10T05:44:45.855 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:44:45.855 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T05:44:45.855 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-05-10T05:44:45.855 INFO:teuthology.orchestra.run.smithi131.stdout: "e4a65dac-e5d1-4a1c-ad57-1e855231a569": { 2024-05-10T05:44:45.855 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-05-10T05:44:45.856 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "e4a65dac-e5d1-4a1c-ad57-1e855231a569", 2024-05-10T05:44:45.856 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,force" 2024-05-10T05:44:45.856 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:44:45.856 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:44:45.856 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:44:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.857+0000 7f212de00700 1 -- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f210c0a12c0 msgr2=0x7f210c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.858+0000 7f212de00700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f210c0a12c0 0x7f210c0a37d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2110006010 tx=0x7f2110000bc0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.858+0000 7f212de00700 1 -- 172.21.15.131:0/2036532327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128101be0 msgr2=0x7f212819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:44:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.858+0000 7f212de00700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128101be0 0x7f212819a6c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f211800d950 tx=0x7f211800dd10 comp rx=0 tx=0).stop 2024-05-10T05:44:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.858+0000 7f212de00700 1 -- 172.21.15.131:0/2036532327 shutdown_connections 2024-05-10T05:44:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.858+0000 7f212de00700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f210c0a12c0 0x7f210c0a37d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.858+0000 7f212de00700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2128102580 0x7f21281a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.858+0000 7f212de00700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2128100ac0 0x7f212819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.858+0000 7f212de00700 1 --2- 172.21.15.131:0/2036532327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2128101be0 0x7f212819a6c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:44:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.858+0000 7f212de00700 1 -- 172.21.15.131:0/2036532327 >> 172.21.15.131:0/2036532327 conn(0x7f21280fc0d0 msgr2=0x7f21281098c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:44:45.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.859+0000 7f212de00700 1 -- 172.21.15.131:0/2036532327 shutdown_connections 2024-05-10T05:44:45.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:44:45.859+0000 7f212de00700 1 -- 172.21.15.131:0/2036532327 wait complete. 2024-05-10T05:44:45.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:45 smithi142 ceph-mon[77667]: scrub queued for path: / 2024-05-10T05:44:45.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:45 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:45.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:45 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:44:45.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:45 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:45.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:45 smithi131 ceph-mon[37264]: scrub queued for path: / 2024-05-10T05:44:45.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:45 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:45.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:45 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:44:45.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:45 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:45.972 INFO:tasks.cephfs.filesystem:scrub status for tag:e4a65dac-e5d1-4a1c-ad57-1e855231a569 - {'path': '/', 'tag': 'e4a65dac-e5d1-4a1c-ad57-1e855231a569', 'options': 'recursive,force'} 2024-05-10T05:44:46.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:46 smithi157 ceph-mon[78427]: scrub summary: active paths [/] 2024-05-10T05:44:46.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:46 smithi157 ceph-mon[78427]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:46.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:46 smithi157 ceph-mon[78427]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:44:46.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:46 smithi142 ceph-mon[77667]: scrub summary: active paths [/] 2024-05-10T05:44:46.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:46 smithi142 ceph-mon[77667]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:46.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:46 smithi142 ceph-mon[77667]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:44:46.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:46 smithi131 ceph-mon[37264]: scrub summary: active paths [/] 2024-05-10T05:44:46.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:46 smithi131 ceph-mon[37264]: scrub summary: idle+waiting paths [/] 2024-05-10T05:44:46.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:46 smithi131 ceph-mon[37264]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 240 MiB/s wr, 74 op/s 2024-05-10T05:44:48.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:48 smithi142 ceph-mon[77667]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 210 MiB/s wr, 65 op/s 2024-05-10T05:44:48.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:48 smithi131 ceph-mon[37264]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 210 MiB/s wr, 65 op/s 2024-05-10T05:44:49.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:48 smithi157 ceph-mon[78427]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 210 MiB/s wr, 65 op/s 2024-05-10T05:44:50.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:50 smithi142 ceph-mon[77667]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 255 MiB/s wr, 77 op/s 2024-05-10T05:44:50.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:50 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:44:50.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:50 smithi131 ceph-mon[37264]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 255 MiB/s wr, 77 op/s 2024-05-10T05:44:50.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:50 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:44:51.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:50 smithi157 ceph-mon[78427]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 255 MiB/s wr, 77 op/s 2024-05-10T05:44:51.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:50 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:44:51.968 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:51 smithi131 ceph-mon[37264]: scrub summary: idle 2024-05-10T05:44:52.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:51 smithi157 ceph-mon[78427]: scrub summary: idle 2024-05-10T05:44:52.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:51 smithi142 ceph-mon[77667]: scrub summary: idle 2024-05-10T05:44:53.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:52 smithi157 ceph-mon[78427]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 254 MiB/s wr, 79 op/s 2024-05-10T05:44:53.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:52 smithi142 ceph-mon[77667]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 254 MiB/s wr, 79 op/s 2024-05-10T05:44:53.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:52 smithi131 ceph-mon[37264]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 254 MiB/s wr, 79 op/s 2024-05-10T05:44:55.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:54 smithi157 ceph-mon[78427]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 257 MiB/s wr, 78 op/s 2024-05-10T05:44:55.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:54 smithi142 ceph-mon[77667]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 257 MiB/s wr, 78 op/s 2024-05-10T05:44:55.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:54 smithi131 ceph-mon[37264]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 257 MiB/s wr, 78 op/s 2024-05-10T05:44:57.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:56 smithi157 ceph-mon[78427]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 270 MiB/s wr, 83 op/s 2024-05-10T05:44:57.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:56 smithi142 ceph-mon[77667]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 270 MiB/s wr, 83 op/s 2024-05-10T05:44:57.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:56 smithi131 ceph-mon[37264]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 270 MiB/s wr, 83 op/s 2024-05-10T05:44:59.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:58 smithi157 ceph-mon[78427]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 239 MiB/s wr, 74 op/s 2024-05-10T05:44:59.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:58 smithi142 ceph-mon[77667]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 239 MiB/s wr, 74 op/s 2024-05-10T05:44:59.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:58 smithi131 ceph-mon[37264]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 239 MiB/s wr, 74 op/s 2024-05-10T05:45:00.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:44:59 smithi142 ceph-mon[77667]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 84 op/s 2024-05-10T05:45:00.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:44:59 smithi131 ceph-mon[37264]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 84 op/s 2024-05-10T05:45:00.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:44:59 smithi157 ceph-mon[78427]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 84 op/s 2024-05-10T05:45:02.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:02 smithi142 ceph-mon[77667]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 255 MiB/s wr, 80 op/s 2024-05-10T05:45:02.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:02 smithi131 ceph-mon[37264]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 255 MiB/s wr, 80 op/s 2024-05-10T05:45:03.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:02 smithi157 ceph-mon[78427]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 255 MiB/s wr, 80 op/s 2024-05-10T05:45:04.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:04 smithi142 ceph-mon[77667]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:45:04.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:04 smithi131 ceph-mon[37264]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:45:05.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:04 smithi157 ceph-mon[78427]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 248 MiB/s wr, 75 op/s 2024-05-10T05:45:05.707 INFO:tasks.workunit.client.0.smithi131.stdout:stopping iogen 2024-05-10T05:45:05.718 INFO:tasks.workunit.client.0.smithi131.stdout:OK 2024-05-10T05:45:06.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:06 smithi142 ceph-mon[77667]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 259 MiB/s wr, 80 op/s 2024-05-10T05:45:06.968 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:06 smithi131 ceph-mon[37264]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 259 MiB/s wr, 80 op/s 2024-05-10T05:45:07.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:06 smithi157 ceph-mon[78427]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 259 MiB/s wr, 80 op/s 2024-05-10T05:45:07.782 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-10T05:45:07.783 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-10T05:45:08.032 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-05-10T05:45:08.032 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-10T05:45:08.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:08 smithi131 ceph-mon[37264]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 72 op/s 2024-05-10T05:45:09.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:08 smithi157 ceph-mon[78427]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 72 op/s 2024-05-10T05:45:09.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:08 smithi142 ceph-mon[77667]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 72 op/s 2024-05-10T05:45:09.830 DEBUG:teuthology.parallel:result is None 2024-05-10T05:45:09.831 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T05:45:09.959 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T05:45:09.959 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-10T05:45:09.970 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-10T05:45:10.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:09 smithi157 ceph-mon[78427]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:45:10.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:09 smithi142 ceph-mon[77667]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:45:10.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:09 smithi131 ceph-mon[37264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T05:45:11.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:10 smithi157 ceph-mon[78427]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 281 MiB/s wr, 84 op/s 2024-05-10T05:45:11.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:10 smithi142 ceph-mon[77667]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 281 MiB/s wr, 84 op/s 2024-05-10T05:45:11.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:10 smithi131 ceph-mon[37264]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 281 MiB/s wr, 84 op/s 2024-05-10T05:45:13.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:12 smithi157 ceph-mon[78427]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 286 MiB/s wr, 89 op/s 2024-05-10T05:45:13.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:12 smithi142 ceph-mon[77667]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 286 MiB/s wr, 89 op/s 2024-05-10T05:45:13.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:12 smithi131 ceph-mon[37264]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 286 MiB/s wr, 89 op/s 2024-05-10T05:45:15.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:14 smithi157 ceph-mon[78427]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 293 MiB/s wr, 91 op/s 2024-05-10T05:45:15.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:14 smithi142 ceph-mon[77667]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 293 MiB/s wr, 91 op/s 2024-05-10T05:45:15.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:14 smithi131 ceph-mon[37264]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 293 MiB/s wr, 91 op/s 2024-05-10T05:45:15.974 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T05:45:16.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.411+0000 7fd724a00700 1 -- 172.21.15.131:0/602711820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd720102b00 msgr2=0x7fd720102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:16.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.411+0000 7fd724a00700 1 --2- 172.21.15.131:0/602711820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd720102b00 0x7fd720102f10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0099c0 tx=0x7fd70c009cd0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.411+0000 7fd724a00700 1 -- 172.21.15.131:0/602711820 shutdown_connections 2024-05-10T05:45:16.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.411+0000 7fd724a00700 1 --2- 172.21.15.131:0/602711820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7201019e0 0x7fd720101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.411+0000 7fd724a00700 1 --2- 172.21.15.131:0/602711820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7201034a0 0x7fd720109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.411+0000 7fd724a00700 1 --2- 172.21.15.131:0/602711820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd720102b00 0x7fd720102f10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.411+0000 7fd724a00700 1 -- 172.21.15.131:0/602711820 >> 172.21.15.131:0/602711820 conn(0x7fd7200fcfd0 msgr2=0x7fd7200ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:16.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.411+0000 7fd724a00700 1 -- 172.21.15.131:0/602711820 shutdown_connections 2024-05-10T05:45:16.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.411+0000 7fd724a00700 1 -- 172.21.15.131:0/602711820 wait complete. 2024-05-10T05:45:16.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.413+0000 7fd724a00700 1 Processor -- start 2024-05-10T05:45:16.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.413+0000 7fd724a00700 1 -- start start 2024-05-10T05:45:16.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.413+0000 7fd724a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7201019e0 0x7fd720107550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.413+0000 7fd724a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd720102b00 0x7fd720107a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.413+0000 7fd724a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7201034a0 0x7fd7201040e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.413+0000 7fd724a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7201047e0 con 0x7fd7201019e0 2024-05-10T05:45:16.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.413+0000 7fd724a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd720104950 con 0x7fd7201034a0 2024-05-10T05:45:16.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.413+0000 7fd724a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd720104b10 con 0x7fd720102b00 2024-05-10T05:45:16.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7201019e0 0x7fd720107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7201034a0 0x7fd7201040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd720102b00 0x7fd720107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71f400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7201019e0 0x7fd720107550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41632/0 (socket says 172.21.15.131:41632) 2024-05-10T05:45:16.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71f400700 1 -- 172.21.15.131:0/3947760602 learned_addr learned my addr 172.21.15.131:0/3947760602 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:16.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71fe00700 1 -- 172.21.15.131:0/3947760602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd720102b00 msgr2=0x7fd720107a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:16.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71fe00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd720102b00 0x7fd720107a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71fe00700 1 -- 172.21.15.131:0/3947760602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7201019e0 msgr2=0x7fd720107550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:16.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71fe00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7201019e0 0x7fd720107550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71fe00700 1 -- 172.21.15.131:0/3947760602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710009520 con 0x7fd7201034a0 2024-05-10T05:45:16.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71f400700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7201019e0 0x7fd720107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:16.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71ea00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd720102b00 0x7fd720107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:16.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.414+0000 7fd71fe00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7201034a0 0x7fd7201040e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd71800efc0 tx=0x7fd71800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:16.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.415+0000 7fd717e00700 1 -- 172.21.15.131:0/3947760602 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718018070 con 0x7fd7201034a0 2024-05-10T05:45:16.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.415+0000 7fd717e00700 1 -- 172.21.15.131:0/3947760602 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd71800f040 con 0x7fd7201034a0 2024-05-10T05:45:16.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.415+0000 7fd717e00700 1 -- 172.21.15.131:0/3947760602 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800cd40 con 0x7fd7201034a0 2024-05-10T05:45:16.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.415+0000 7fd724a00700 1 -- 172.21.15.131:0/3947760602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70c0095f0 con 0x7fd7201034a0 2024-05-10T05:45:16.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.415+0000 7fd724a00700 1 -- 172.21.15.131:0/3947760602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7201a6700 con 0x7fd7201034a0 2024-05-10T05:45:16.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.419+0000 7fd73de8cb80 1 Processor -- start 2024-05-10T05:45:16.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.420+0000 7fd717e00700 1 -- 172.21.15.131:0/3947760602 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd718003a00 con 0x7fd7201034a0 2024-05-10T05:45:16.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.420+0000 7fd73de8cb80 1 -- start start 2024-05-10T05:45:16.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.420+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 0x55645d5b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.420+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b77a0 0x55645d5b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.420+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 0x55645d5be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.420+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d4dd150 con 0x55645d5b80f0 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.420+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d4dab10 con 0x55645d5b77a0 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.420+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d4d6400 con 0x55645d5b6530 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.420+0000 7fd71f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b77a0 0x55645d5b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b77a0 0x55645d5b7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34872/0 (socket says 172.21.15.131:34872) 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71f400700 1 -- 172.21.15.131:0/3317259353 learned_addr learned my addr 172.21.15.131:0/3317259353 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71ea00700 1 --2- 172.21.15.131:0/3317259353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 0x55645d5b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71ea00700 1 --2- 172.21.15.131:0/3317259353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 0x55645d5be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71f400700 1 -- 172.21.15.131:0/3317259353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 msgr2=0x55645d5b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71f400700 1 --2- 172.21.15.131:0/3317259353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 0x55645d5b6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71f400700 1 -- 172.21.15.131:0/3317259353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 msgr2=0x55645d5be9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71f400700 1 --2- 172.21.15.131:0/3317259353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 0x55645d5be9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71f400700 1 -- 172.21.15.131:0/3317259353 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645d3573b0 con 0x55645d5b77a0 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71ea00700 1 --2- 172.21.15.131:0/3317259353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 0x55645d5be9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd71f400700 1 --2- 172.21.15.131:0/3317259353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b77a0 0x55645d5b7bb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd710011810 tx=0x7fd710011b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58801e2a2479d92a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd717e00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7040a11a0 0x7fd7040a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd70be00700 1 -- 172.21.15.131:0/3317259353 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710009e80 con 0x55645d5b77a0 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd70be00700 1 -- 172.21.15.131:0/3317259353 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd71000fb00 con 0x55645d5b77a0 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.421+0000 7fd70be00700 1 -- 172.21.15.131:0/3317259353 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710019cf0 con 0x55645d5b77a0 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.422+0000 7fd717e00700 1 -- 172.21.15.131:0/3947760602 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd7180cc400 con 0x7fd7201034a0 2024-05-10T05:45:16.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.422+0000 7fd71ea00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7040a11a0 0x7fd7040a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.422+0000 7fd73de8cb80 1 -- 172.21.15.131:0/3317259353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b77a0 msgr2=0x55645d5b7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:16.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.422+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/3317259353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b77a0 0x55645d5b7bb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd710011810 tx=0x7fd710011b20 comp rx=0 tx=0).stop 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.422+0000 7fd73de8cb80 1 -- 172.21.15.131:0/3317259353 shutdown_connections 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.422+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/3317259353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 0x55645d5be9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.422+0000 7fd71ea00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7040a11a0 0x7fd7040a36b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009990 tx=0x7fd70c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.422+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/3317259353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 0x55645d5b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.422+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/3317259353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b77a0 0x55645d5b7bb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.422+0000 7fd73de8cb80 1 -- 172.21.15.131:0/3317259353 >> 172.21.15.131:0/3317259353 conn(0x55645d518a60 msgr2=0x55645d5b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.423+0000 7fd73de8cb80 1 -- 172.21.15.131:0/3317259353 shutdown_connections 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.423+0000 7fd73de8cb80 1 -- 172.21.15.131:0/3317259353 wait complete. 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.423+0000 7fd73de8cb80 1 -- start start 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.423+0000 7fd73de8cb80 1 Processor -- start 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b6530 0x55645d5c0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 0x55645d5c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 0x55645d5c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d472560 con 0x55645d5b80f0 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d472200 con 0x55645d5b6530 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d3596a0 con 0x55645d5b77a0 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd73de8cb80 10 client.0 _ll_register_callbacks cb 0x55645d47bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd71ea00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 0x55645d5c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd71f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b6530 0x55645d5c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd71f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b6530 0x55645d5c0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34878/0 (socket says 172.21.15.131:34878) 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd71f400700 1 -- 172.21.15.131:0/4115424613 learned_addr learned my addr 172.21.15.131:0/4115424613 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.424+0000 7fd71fe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 0x55645d5c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.425+0000 7fd71f400700 1 -- 172.21.15.131:0/4115424613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 msgr2=0x55645d5c0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.425+0000 7fd71f400700 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 0x55645d5c0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.425+0000 7fd71f400700 1 -- 172.21.15.131:0/4115424613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 msgr2=0x55645d5c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.425+0000 7fd71f400700 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 0x55645d5c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.425+0000 7fd71f400700 1 -- 172.21.15.131:0/4115424613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd710011440 con 0x55645d5b6530 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.425+0000 7fd71f400700 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b6530 0x55645d5c0850 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd710005950 tx=0x7fd71000f870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64353ba0b4e80ced server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.425+0000 7fd708c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.425+0000 7fd708c00700 1 -- 172.21.15.131:0/4115424613 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710026070 con 0x55645d5b6530 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.425+0000 7fd708c00700 1 -- 172.21.15.131:0/4115424613 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd710009e80 con 0x55645d5b6530 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.425+0000 7fd73de8cb80 1 -- 172.21.15.131:0/4115424613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55645d42f270 con 0x55645d5b6530 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.426+0000 7fd708c00700 1 -- 172.21.15.131:0/4115424613 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd710019c20 con 0x55645d5b6530 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.426+0000 7fd73de8cb80 10 client.27467 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.426+0000 7fd73de8cb80 1 -- 172.21.15.131:0/4115424613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55645d303b60 con 0x55645d5b6530 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.426+0000 7fd708c00700 1 -- 172.21.15.131:0/4115424613 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55645d42f270 con 0x55645d5b6530 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.426+0000 7fd708c00700 1 -- 172.21.15.131:0/4115424613 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd710029030 con 0x55645d5b6530 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.426+0000 7fd73de8cb80 10 client.27467 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.426+0000 7fd73de8cb80 10 client.27467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:45:16.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.427+0000 7fd73de8cb80 20 client.27467 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:16.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.427+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55645d5cc330 0x55645d5ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.427+0000 7fd73de8cb80 4 client.27467 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:45:16.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.427+0000 7fd73de8cb80 1 -- 172.21.15.131:0/4115424613 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55645d585780 con 0x55645d5cc330 2024-05-10T05:45:16.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.427+0000 7fd71ea00700 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55645d5cc330 0x55645d5ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.427+0000 7fd708c00700 10 client.27467 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:45:16.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.427+0000 7fd71ea00700 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55645d5cc330 0x55645d5ce780 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e48b54335fedc4 server_cookie=8bcd44f7f3a332a2 in_seq=0 out_seq=0 2024-05-10T05:45:16.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.428+0000 7fd708c00700 1 -- 172.21.15.131:0/4115424613 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55645d585780 con 0x55645d5cc330 2024-05-10T05:45:16.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.428+0000 7fd708c00700 10 client.27467 handle_command_reply: tid=0 2024-05-10T05:45:16.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.428+0000 7fd73de8cb80 1 client.27467 shutdown 2024-05-10T05:45:16.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.428+0000 7fd73de8cb80 1 -- 172.21.15.131:0/4115424613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b6530 msgr2=0x55645d5c0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.428+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b6530 0x55645d5c0850 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd710005950 tx=0x7fd71000f870 comp rx=0 tx=0).stop 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.429+0000 7fd73de8cb80 1 -- 172.21.15.131:0/4115424613 shutdown_connections 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.429+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55645d5cc330 0x55645d5ce780 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.429+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b80f0 0x55645d5c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.429+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b6530 0x55645d5c0850 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.429+0000 7fd708c00700 0 client.27467 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.429+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/4115424613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 0x55645d5c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.429+0000 7fd73de8cb80 1 -- 172.21.15.131:0/4115424613 >> 172.21.15.131:0/4115424613 conn(0x55645d518a60 msgr2=0x55645d5b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.429+0000 7fd73de8cb80 1 -- 172.21.15.131:0/4115424613 shutdown_connections 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.429+0000 7fd73de8cb80 1 -- 172.21.15.131:0/4115424613 wait complete. 2024-05-10T05:45:16.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.429+0000 7fd73de8cb80 20 client.27467 trim_cache size 0 max 16384 2024-05-10T05:45:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.449+0000 7fd73de8cb80 1 Processor -- start 2024-05-10T05:45:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.450+0000 7fd73de8cb80 1 -- start start 2024-05-10T05:45:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.450+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b6530 0x55645d5b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.450+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 0x55645d5c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.450+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 0x55645d5c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.450+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d5ab2f0 con 0x55645d5b6530 2024-05-10T05:45:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.450+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d5ab8d0 con 0x55645d5b80f0 2024-05-10T05:45:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.450+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d5b7060 con 0x55645d5b77a0 2024-05-10T05:45:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.451+0000 7fd71fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 0x55645d5c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.451+0000 7fd71fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 0x55645d5c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34894/0 (socket says 172.21.15.131:34894) 2024-05-10T05:45:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.451+0000 7fd71fe00700 1 -- 172.21.15.131:0/2116594306 learned_addr learned my addr 172.21.15.131:0/2116594306 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.451+0000 7fd71ea00700 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 0x55645d5c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.451+0000 7fd71ea00700 1 -- 172.21.15.131:0/2116594306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 msgr2=0x55645d5c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.451+0000 7fd71ea00700 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 0x55645d5c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.451+0000 7fd71ea00700 1 -- 172.21.15.131:0/2116594306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b6530 msgr2=0x55645d5b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.451+0000 7fd71f400700 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b6530 0x55645d5b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.452+0000 7fd71ea00700 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b6530 0x55645d5b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.452+0000 7fd71ea00700 1 -- 172.21.15.131:0/2116594306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645d49c560 con 0x55645d5b77a0 2024-05-10T05:45:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.452+0000 7fd71fe00700 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 0x55645d5c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.452+0000 7fd71f400700 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b6530 0x55645d5b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:45:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.452+0000 7fd71ea00700 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 0x55645d5c5ca0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0058e0 tx=0x7fd70c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5b56c45bcdb93c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.453+0000 7fd715200700 1 -- 172.21.15.131:0/2116594306 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0037d0 con 0x55645d5b77a0 2024-05-10T05:45:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.453+0000 7fd715200700 1 -- 172.21.15.131:0/2116594306 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70c01e040 con 0x55645d5b77a0 2024-05-10T05:45:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.453+0000 7fd715200700 1 -- 172.21.15.131:0/2116594306 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0337e0 con 0x55645d5b77a0 2024-05-10T05:45:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.454+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2116594306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 msgr2=0x55645d5c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.454+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 0x55645d5c5ca0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0058e0 tx=0x7fd70c009310 comp rx=0 tx=0).stop 2024-05-10T05:45:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.454+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2116594306 shutdown_connections 2024-05-10T05:45:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.454+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b6530 0x55645d5b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.454+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 0x55645d5c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.454+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/2116594306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b77a0 0x55645d5c5ca0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.454+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2116594306 >> 172.21.15.131:0/2116594306 conn(0x55645d518a60 msgr2=0x55645d5b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.454+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2116594306 shutdown_connections 2024-05-10T05:45:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.454+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2116594306 wait complete. 2024-05-10T05:45:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.455+0000 7fd73de8cb80 1 -- start start 2024-05-10T05:45:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.455+0000 7fd73de8cb80 1 Processor -- start 2024-05-10T05:45:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.456+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 0x55645d61c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b77a0 0x55645d61ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd73de8cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 0x55645d621270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d6217b0 con 0x55645d5b77a0 2024-05-10T05:45:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d6218f0 con 0x55645d5b80f0 2024-05-10T05:45:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd73de8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645d621a30 con 0x55645d5b6530 2024-05-10T05:45:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd73de8cb80 10 client.0 _ll_register_callbacks cb 0x55645d47bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd71f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 0x55645d61c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd71f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 0x55645d61c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40318/0 (socket says 172.21.15.131:40318) 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd71f400700 1 -- 172.21.15.131:0/2640640834 learned_addr learned my addr 172.21.15.131:0/2640640834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd71fe00700 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 0x55645d621270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.457+0000 7fd71ea00700 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b77a0 0x55645d61ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.458+0000 7fd71f400700 1 -- 172.21.15.131:0/2640640834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 msgr2=0x55645d621270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.458+0000 7fd71f400700 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 0x55645d621270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.458+0000 7fd71f400700 1 -- 172.21.15.131:0/2640640834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b77a0 msgr2=0x55645d61ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.458+0000 7fd71f400700 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b77a0 0x55645d61ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.458+0000 7fd71f400700 1 -- 172.21.15.131:0/2640640834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7201a6700 con 0x55645d5b6530 2024-05-10T05:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.458+0000 7fd71ea00700 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b77a0 0x55645d61ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:45:16.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.458+0000 7fd71f400700 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 0x55645d61c7d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd7100049f0 tx=0x7fd710005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90076106e8cb004e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:16.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.458+0000 7fd708c00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:45:16.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.459+0000 7fd708c00700 1 -- 172.21.15.131:0/2640640834 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710026070 con 0x55645d5b6530 2024-05-10T05:45:16.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.459+0000 7fd708c00700 1 -- 172.21.15.131:0/2640640834 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fd710000e40 con 0x55645d5b6530 2024-05-10T05:45:16.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.460+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2640640834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd71000fb00 con 0x55645d5b6530 2024-05-10T05:45:16.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.461+0000 7fd708c00700 1 -- 172.21.15.131:0/2640640834 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd710004c60 con 0x55645d5b6530 2024-05-10T05:45:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.461+0000 7fd73de8cb80 10 client.27475 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.461+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2640640834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fd72010a7e0 con 0x55645d5b6530 2024-05-10T05:45:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.462+0000 7fd708c00700 1 -- 172.21.15.131:0/2640640834 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd71000fb00 con 0x55645d5b6530 2024-05-10T05:45:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.462+0000 7fd708c00700 1 -- 172.21.15.131:0/2640640834 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd710029050 con 0x55645d5b6530 2024-05-10T05:45:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.462+0000 7fd73de8cb80 10 client.27475 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.462+0000 7fd73de8cb80 10 client.27475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:45:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.462+0000 7fd73de8cb80 20 client.27475 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.462+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55645d623100 0x55645d625520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.462+0000 7fd73de8cb80 4 client.27475 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T05:45:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.462+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2640640834 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55645d4ceec0 con 0x55645d623100 2024-05-10T05:45:16.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.463+0000 7fd71ea00700 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55645d623100 0x55645d625520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:16.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.465+0000 7fd708c00700 10 client.27475 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:45:16.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.466+0000 7fd71ea00700 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55645d623100 0x55645d625520 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514a86a8a2748030 server_cookie=74c3832acca4ab72 in_seq=0 out_seq=0 2024-05-10T05:45:16.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.466+0000 7fd708c00700 1 -- 172.21.15.131:0/2640640834 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55645d4ceec0 con 0x55645d623100 2024-05-10T05:45:16.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.466+0000 7fd708c00700 10 client.27475 handle_command_reply: tid=0 2024-05-10T05:45:16.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.466+0000 7fd73de8cb80 1 client.27475 shutdown 2024-05-10T05:45:16.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.467+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2640640834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 msgr2=0x55645d61c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:16.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.467+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 0x55645d61c7d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd7100049f0 tx=0x7fd710005e50 comp rx=0 tx=0).stop 2024-05-10T05:45:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.469+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2640640834 shutdown_connections 2024-05-10T05:45:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.469+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x55645d623100 0x55645d625520 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.469+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645d5b80f0 0x55645d621270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.469+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55645d5b6530 0x55645d61c7d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.469+0000 7fd73de8cb80 1 --2- 172.21.15.131:0/2640640834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55645d5b77a0 0x55645d61ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.469+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2640640834 >> 172.21.15.131:0/2640640834 conn(0x55645d518a60 msgr2=0x55645d5bead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.469+0000 7fd708c00700 0 client.27475 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:45:16.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.470+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2640640834 shutdown_connections 2024-05-10T05:45:16.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.470+0000 7fd73de8cb80 1 -- 172.21.15.131:0/2640640834 wait complete. 2024-05-10T05:45:16.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.470+0000 7fd73de8cb80 20 client.27475 trim_cache size 0 max 16384 2024-05-10T05:45:16.472 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:45:16.472 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-05-10T05:45:16.472 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {} 2024-05-10T05:45:16.472 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:45:16.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.471+0000 7fd724a00700 1 -- 172.21.15.131:0/3947760602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7040a11a0 msgr2=0x7fd7040a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:16.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.471+0000 7fd724a00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7040a11a0 0x7fd7040a36b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009990 tx=0x7fd70c005fb0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.471+0000 7fd724a00700 1 -- 172.21.15.131:0/3947760602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7201034a0 msgr2=0x7fd7201040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.471+0000 7fd724a00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7201034a0 0x7fd7201040e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd71800efc0 tx=0x7fd71800c3a0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.472+0000 7fd724a00700 1 -- 172.21.15.131:0/3947760602 shutdown_connections 2024-05-10T05:45:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.472+0000 7fd724a00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fd7040a11a0 0x7fd7040a36b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.472+0000 7fd724a00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7201019e0 0x7fd720107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.472+0000 7fd724a00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7201034a0 0x7fd7201040e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.472+0000 7fd724a00700 1 --2- 172.21.15.131:0/3947760602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd720102b00 0x7fd720107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.472+0000 7fd724a00700 1 -- 172.21.15.131:0/3947760602 >> 172.21.15.131:0/3947760602 conn(0x7fd7200fcfd0 msgr2=0x7fd720073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:16.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.473+0000 7fd724a00700 1 -- 172.21.15.131:0/3947760602 shutdown_connections 2024-05-10T05:45:16.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:16.473+0000 7fd724a00700 1 -- 172.21.15.131:0/3947760602 wait complete. 2024-05-10T05:45:16.525 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T05:45:16.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:45:16.968 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:16 smithi131 ceph-mon[37264]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 332 MiB/s wr, 105 op/s 2024-05-10T05:45:17.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.035+0000 7f5133200700 1 -- 172.21.15.131:0/1750264815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5134109150 msgr2=0x7f5134109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:17.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.035+0000 7f5133200700 1 --2- 172.21.15.131:0/1750264815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5134109150 0x7f5134109560 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f512c00b0a0 tx=0x7f512c00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:45:17.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.036+0000 7f5133200700 1 -- 172.21.15.131:0/1750264815 shutdown_connections 2024-05-10T05:45:17.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.036+0000 7f5133200700 1 --2- 172.21.15.131:0/1750264815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134109af0 0x7f51341103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:17.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.036+0000 7f5133200700 1 --2- 172.21.15.131:0/1750264815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5134108030 0x7f5134108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:17.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.036+0000 7f5133200700 1 --2- 172.21.15.131:0/1750264815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5134109150 0x7f5134109560 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:17.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.036+0000 7f5133200700 1 -- 172.21.15.131:0/1750264815 >> 172.21.15.131:0/1750264815 conn(0x7f5134077080 msgr2=0x7f5134106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:17.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.039+0000 7f5133200700 1 -- 172.21.15.131:0/1750264815 shutdown_connections 2024-05-10T05:45:17.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.040+0000 7f5133200700 1 -- 172.21.15.131:0/1750264815 wait complete. 2024-05-10T05:45:17.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.041+0000 7f5133200700 1 Processor -- start 2024-05-10T05:45:17.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.042+0000 7f5133200700 1 -- start start 2024-05-10T05:45:17.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.044+0000 7f5133200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5134108030 0x7f513407ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.044+0000 7f5133200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134109af0 0x7f513407f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.044+0000 7f5133200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51341ac150 0x7f51341ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.045+0000 7f5133200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51341aeb40 con 0x7f5134108030 2024-05-10T05:45:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.045+0000 7f5133200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51341aecb0 con 0x7f51341ac150 2024-05-10T05:45:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.045+0000 7f5133200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51341aee70 con 0x7f5134109af0 2024-05-10T05:45:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.045+0000 7f5132800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51341ac150 0x7f51341ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.045+0000 7f5132800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51341ac150 0x7f51341ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34926/0 (socket says 172.21.15.131:34926) 2024-05-10T05:45:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.045+0000 7f5132800700 1 -- 172.21.15.131:0/376814606 learned_addr learned my addr 172.21.15.131:0/376814606 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:17.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.045+0000 7f5131400700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134109af0 0x7f513407f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:17.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.046+0000 7f5132800700 1 -- 172.21.15.131:0/376814606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134109af0 msgr2=0x7f513407f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:17.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.046+0000 7f5132800700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134109af0 0x7f513407f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:17.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.046+0000 7f5132800700 1 -- 172.21.15.131:0/376814606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5134108030 msgr2=0x7f513407ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:17.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.046+0000 7f5132800700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5134108030 0x7f513407ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:17.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.046+0000 7f5132800700 1 -- 172.21.15.131:0/376814606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c009d00 con 0x7f51341ac150 2024-05-10T05:45:17.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.046+0000 7f5132800700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51341ac150 0x7f51341ae570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f512000e940 tx=0x7f512000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.047+0000 7f512aa00700 1 -- 172.21.15.131:0/376814606 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512000cbd0 con 0x7f51341ac150 2024-05-10T05:45:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.047+0000 7f512aa00700 1 -- 172.21.15.131:0/376814606 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f512000ee90 con 0x7f51341ac150 2024-05-10T05:45:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.047+0000 7f5133200700 1 -- 172.21.15.131:0/376814606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51341af150 con 0x7f51341ac150 2024-05-10T05:45:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.047+0000 7f512aa00700 1 -- 172.21.15.131:0/376814606 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51200055c0 con 0x7f51341ac150 2024-05-10T05:45:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.047+0000 7f5133200700 1 -- 172.21.15.131:0/376814606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51341af6b0 con 0x7f51341ac150 2024-05-10T05:45:17.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.050+0000 7f5133200700 1 -- 172.21.15.131:0/376814606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513410f240 con 0x7f51341ac150 2024-05-10T05:45:17.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:16 smithi157 ceph-mon[78427]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 332 MiB/s wr, 105 op/s 2024-05-10T05:45:17.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.054+0000 7f512aa00700 1 -- 172.21.15.131:0/376814606 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5120004750 con 0x7f51341ac150 2024-05-10T05:45:17.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.057+0000 7f512aa00700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f51180a12c0 0x7f51180a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:17.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.058+0000 7f512aa00700 1 -- 172.21.15.131:0/376814606 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f5120014070 con 0x7f51341ac150 2024-05-10T05:45:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.058+0000 7f512aa00700 1 -- 172.21.15.131:0/376814606 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f51200cd4e0 con 0x7f51341ac150 2024-05-10T05:45:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.059+0000 7f5131e00700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f51180a12c0 0x7f51180a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.060+0000 7f5131e00700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f51180a12c0 0x7f51180a37d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f513407fde0 tx=0x7f5124009470 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:17.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:16 smithi142 ceph-mon[77667]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 332 MiB/s wr, 105 op/s 2024-05-10T05:45:17.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.481+0000 7f5133200700 1 -- 172.21.15.131:0/376814606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5134113b80 con 0x7f51341ac150 2024-05-10T05:45:17.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.483+0000 7f512aa00700 1 -- 172.21.15.131:0/376814606 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10931 (secure 0 0 0) 0x7f5120096950 con 0x7f51341ac150 2024-05-10T05:45:17.484 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:45:17.484 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:45:08.699705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:45:17.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 -- 172.21.15.131:0/376814606 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f51180a12c0 msgr2=0x7f51180a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:17.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f51180a12c0 0x7f51180a37d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f513407fde0 tx=0x7f5124009470 comp rx=0 tx=0).stop 2024-05-10T05:45:17.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 -- 172.21.15.131:0/376814606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51341ac150 msgr2=0x7f51341ae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:17.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51341ac150 0x7f51341ae570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f512000e940 tx=0x7f512000ec50 comp rx=0 tx=0).stop 2024-05-10T05:45:17.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 -- 172.21.15.131:0/376814606 shutdown_connections 2024-05-10T05:45:17.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f51180a12c0 0x7f51180a37d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:17.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5134108030 0x7f513407ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:17.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51341ac150 0x7f51341ae570 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:17.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 --2- 172.21.15.131:0/376814606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5134109af0 0x7f513407f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:17.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 -- 172.21.15.131:0/376814606 >> 172.21.15.131:0/376814606 conn(0x7f5134077080 msgr2=0x7f513410c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:17.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.497+0000 7f5133200700 1 -- 172.21.15.131:0/376814606 shutdown_connections 2024-05-10T05:45:17.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.498+0000 7f5133200700 1 -- 172.21.15.131:0/376814606 wait complete. 2024-05-10T05:45:17.499 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 30 2024-05-10T05:45:17.546 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T05:45:17.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:17 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/376814606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:45:17.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.999+0000 7fafc7400700 1 -- 172.21.15.131:0/2642610825 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc003b50 con 0x7fafc8108630 2024-05-10T05:45:18.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.999+0000 7fafcfc00700 1 -- 172.21.15.131:0/2642610825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc8108630 msgr2=0x7fafc8108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:17.999+0000 7fafcfc00700 1 --2- 172.21.15.131:0/2642610825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc8108630 0x7fafc8108a40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009990 tx=0x7fafbc009ca0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.000+0000 7fafcfc00700 1 -- 172.21.15.131:0/2642610825 shutdown_connections 2024-05-10T05:45:18.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.000+0000 7fafcfc00700 1 --2- 172.21.15.131:0/2642610825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc8108630 0x7fafc8108a40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.000+0000 7fafcfc00700 1 --2- 172.21.15.131:0/2642610825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc8109c50 0x7fafc810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.000+0000 7fafcfc00700 1 --2- 172.21.15.131:0/2642610825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc8109300 0x7fafc8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.000+0000 7fafcfc00700 1 -- 172.21.15.131:0/2642610825 >> 172.21.15.131:0/2642610825 conn(0x7fafc806ea00 msgr2=0x7fafc8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.001+0000 7fafcfc00700 1 -- 172.21.15.131:0/2642610825 shutdown_connections 2024-05-10T05:45:18.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.001+0000 7fafcfc00700 1 -- 172.21.15.131:0/2642610825 wait complete. 2024-05-10T05:45:18.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.002+0000 7fafcfc00700 1 Processor -- start 2024-05-10T05:45:18.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.002+0000 7fafcfc00700 1 -- start start 2024-05-10T05:45:18.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.003+0000 7fafcfc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc8108630 0x7fafc81a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.003+0000 7fafcfc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc8109300 0x7fafc81a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.003+0000 7fafcfc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc8109c50 0x7fafc81ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.003+0000 7fafcfc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81ae840 con 0x7fafc8109c50 2024-05-10T05:45:18.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.003+0000 7fafcfc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81ae9b0 con 0x7fafc8109300 2024-05-10T05:45:18.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.003+0000 7fafcfc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81aeb70 con 0x7fafc8108630 2024-05-10T05:45:18.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafcd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc8108630 0x7fafc81a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafcd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc8108630 0x7fafc81a6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40334/0 (socket says 172.21.15.131:40334) 2024-05-10T05:45:18.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafcd200700 1 -- 172.21.15.131:0/3850317286 learned_addr learned my addr 172.21.15.131:0/3850317286 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:18.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafc7e00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc8109300 0x7fafc81a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafcdc00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc8109c50 0x7fafc81ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafcd200700 1 -- 172.21.15.131:0/3850317286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc8109300 msgr2=0x7fafc81a72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafcd200700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc8109300 0x7fafc81a72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafcd200700 1 -- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc8109c50 msgr2=0x7fafc81ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafcd200700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc8109c50 0x7fafc81ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafcd200700 1 -- 172.21.15.131:0/3850317286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc0095f0 con 0x7fafc8108630 2024-05-10T05:45:18.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.004+0000 7fafc7e00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc8109300 0x7fafc81a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:18.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.005+0000 7fafcdc00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc8109c50 0x7fafc81ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:45:18.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.005+0000 7fafcd200700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc8108630 0x7fafc81a6d70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fafbc005b40 tx=0x7fafbc00f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.005+0000 7fafc5600700 1 -- 172.21.15.131:0/3850317286 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc01d070 con 0x7fafc8108630 2024-05-10T05:45:18.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.005+0000 7fafcfc00700 1 -- 172.21.15.131:0/3850317286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc81aedf0 con 0x7fafc8108630 2024-05-10T05:45:18.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.005+0000 7fafc5600700 1 -- 172.21.15.131:0/3850317286 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafbc0054f0 con 0x7fafc8108630 2024-05-10T05:45:18.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.005+0000 7fafc5600700 1 -- 172.21.15.131:0/3850317286 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc004a10 con 0x7fafc8108630 2024-05-10T05:45:18.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.006+0000 7fafcfc00700 1 -- 172.21.15.131:0/3850317286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc81af1b0 con 0x7fafc8108630 2024-05-10T05:45:18.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.010+0000 7fafc5600700 1 -- 172.21.15.131:0/3850317286 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fafbc004c50 con 0x7fafc8108630 2024-05-10T05:45:18.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.013+0000 7fafc5600700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fafb00a1240 0x7fafb00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.013+0000 7fafc7e00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fafb00a1240 0x7fafb00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.013+0000 7fafc5600700 1 -- 172.21.15.131:0/3850317286 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fafbc0cd980 con 0x7fafc8108630 2024-05-10T05:45:18.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.014+0000 7fafe7a89b80 1 Processor -- start 2024-05-10T05:45:18.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.014+0000 7fafc7e00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fafb00a1240 0x7fafb00a3750 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005950 tx=0x7fafb400b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.015+0000 7fafe7a89b80 1 -- start start 2024-05-10T05:45:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.015+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead0976a0 0x55bead097af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.015+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead098980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.015+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 0x55bead09f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.015+0000 7fafe7a89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beacfbe150 con 0x55bead0976a0 2024-05-10T05:45:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.015+0000 7fafe7a89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beacfbbb10 con 0x55bead098ec0 2024-05-10T05:45:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.015+0000 7fafe7a89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beacfb7400 con 0x55bead098570 2024-05-10T05:45:18.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafcd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead0976a0 0x55bead097af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafcd200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead0976a0 0x55bead097af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41700/0 (socket says 172.21.15.131:41700) 2024-05-10T05:45:18.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafcd200700 1 -- 172.21.15.131:0/3559622539 learned_addr learned my addr 172.21.15.131:0/3559622539 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:18.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafcdc00700 1 --2- 172.21.15.131:0/3559622539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 0x55bead09f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafc7e00700 1 --2- 172.21.15.131:0/3559622539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead098980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafc7e00700 1 -- 172.21.15.131:0/3559622539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 msgr2=0x55bead09f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafc7e00700 1 --2- 172.21.15.131:0/3559622539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 0x55bead09f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafc7e00700 1 -- 172.21.15.131:0/3559622539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead0976a0 msgr2=0x55bead097af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafc7e00700 1 --2- 172.21.15.131:0/3559622539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead0976a0 0x55bead097af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafc7e00700 1 -- 172.21.15.131:0/3559622539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beace383b0 con 0x55bead098570 2024-05-10T05:45:18.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.016+0000 7fafcd200700 1 --2- 172.21.15.131:0/3559622539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead0976a0 0x55bead097af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:18.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.017+0000 7fafc7e00700 1 --2- 172.21.15.131:0/3559622539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead098980 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fafb4010c00 tx=0x7fafb4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6540f539826e070 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.017+0000 7fafba000700 1 -- 172.21.15.131:0/3559622539 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb401e420 con 0x55bead098570 2024-05-10T05:45:18.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.017+0000 7fafba000700 1 -- 172.21.15.131:0/3559622539 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafb4015070 con 0x55bead098570 2024-05-10T05:45:18.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.017+0000 7fafba000700 1 -- 172.21.15.131:0/3559622539 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb401c920 con 0x55bead098570 2024-05-10T05:45:18.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.018+0000 7fafe7a89b80 1 -- 172.21.15.131:0/3559622539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 msgr2=0x55bead098980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.018+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/3559622539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead098980 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fafb4010c00 tx=0x7fafb4013000 comp rx=0 tx=0).stop 2024-05-10T05:45:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.018+0000 7fafe7a89b80 1 -- 172.21.15.131:0/3559622539 shutdown_connections 2024-05-10T05:45:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.018+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/3559622539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead0976a0 0x55bead097af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.018+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/3559622539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 0x55bead09f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.018+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/3559622539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead098980 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.018+0000 7fafe7a89b80 1 -- 172.21.15.131:0/3559622539 >> 172.21.15.131:0/3559622539 conn(0x55beacff9a60 msgr2=0x55bead091fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.019+0000 7fafe7a89b80 1 -- 172.21.15.131:0/3559622539 shutdown_connections 2024-05-10T05:45:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.019+0000 7fafe7a89b80 1 -- 172.21.15.131:0/3559622539 wait complete. 2024-05-10T05:45:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.019+0000 7fafe7a89b80 1 -- start start 2024-05-10T05:45:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.020+0000 7fafe7a89b80 1 Processor -- start 2024-05-10T05:45:18.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.021+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.021+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 0x55bead0a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.021+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098ec0 0x55bead0a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.023+0000 7fafe7a89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beacf53560 con 0x55bead098570 2024-05-10T05:45:18.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.023+0000 7fafe7a89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beacf53200 con 0x55bead0976a0 2024-05-10T05:45:18.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.023+0000 7fafe7a89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beace3a6a0 con 0x55bead098ec0 2024-05-10T05:45:18.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.023+0000 7fafcd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.023+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 0x55bead0a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.023+0000 7fafe7a89b80 10 client.0 _ll_register_callbacks cb 0x55beacf5cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:18.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.024+0000 7fafcdc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098ec0 0x55bead0a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.024+0000 7fafcd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34958/0 (socket says 172.21.15.131:34958) 2024-05-10T05:45:18.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.024+0000 7fafcd200700 1 -- 172.21.15.131:0/873023754 learned_addr learned my addr 172.21.15.131:0/873023754 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:18.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.026+0000 7fafc7e00700 1 -- 172.21.15.131:0/873023754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098ec0 msgr2=0x55bead0a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.026+0000 7fafc7e00700 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098ec0 0x55bead0a8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.026+0000 7fafc7e00700 1 -- 172.21.15.131:0/873023754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 msgr2=0x55bead0a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.026+0000 7fafc7e00700 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.026+0000 7fafc7e00700 1 -- 172.21.15.131:0/873023754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc81af1b0 con 0x55bead098570 2024-05-10T05:45:18.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.027+0000 7fafc7e00700 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 0x55bead0a1c00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fafb4024000 tx=0x7fafb401ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5934caf04d1d1d1f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.027+0000 7fafcd200700 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:18.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.027+0000 7fafaea00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:45:18.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.027+0000 7fafaea00700 1 -- 172.21.15.131:0/873023754 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb402b070 con 0x55bead098570 2024-05-10T05:45:18.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.027+0000 7fafaea00700 1 -- 172.21.15.131:0/873023754 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fafb4017070 con 0x55bead098570 2024-05-10T05:45:18.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.027+0000 7fafe7a89b80 1 -- 172.21.15.131:0/873023754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55beacf10270 con 0x55bead098570 2024-05-10T05:45:18.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.028+0000 7fafaea00700 1 -- 172.21.15.131:0/873023754 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafb402b070 con 0x55bead098570 2024-05-10T05:45:18.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.029+0000 7fafe7a89b80 10 client.17091 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:18.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.029+0000 7fafe7a89b80 1 -- 172.21.15.131:0/873023754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fafb4011440 con 0x55bead098570 2024-05-10T05:45:18.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.030+0000 7fafaea00700 1 -- 172.21.15.131:0/873023754 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55beacf10270 con 0x55bead098570 2024-05-10T05:45:18.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.030+0000 7fafaea00700 1 -- 172.21.15.131:0/873023754 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fafb401ea50 con 0x55bead098570 2024-05-10T05:45:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.031+0000 7fafe7a89b80 10 client.17091 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.031+0000 7fafe7a89b80 10 client.17091 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:45:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.031+0000 7fafe7a89b80 20 client.17091 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.031+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55bead0ad2a0 0x55bead0af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.031+0000 7fafe7a89b80 4 client.17091 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:45:18.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.031+0000 7fafe7a89b80 1 -- 172.21.15.131:0/873023754 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bead066780 con 0x55bead0ad2a0 2024-05-10T05:45:18.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.032+0000 7fafcd200700 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55bead0ad2a0 0x55bead0af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.037+0000 7fafcd200700 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55bead0ad2a0 0x55bead0af6f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5bf2978237a8f823 server_cookie=9c475538ad3d3953 in_seq=0 out_seq=0 2024-05-10T05:45:18.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.037+0000 7fafaea00700 10 client.17091 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:45:18.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.038+0000 7fafaea00700 1 -- 172.21.15.131:0/873023754 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bead066780 con 0x55bead0ad2a0 2024-05-10T05:45:18.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.038+0000 7fafaea00700 10 client.17091 handle_command_reply: tid=0 2024-05-10T05:45:18.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.039+0000 7fafe7a89b80 1 client.17091 shutdown 2024-05-10T05:45:18.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.044+0000 7fafe7a89b80 1 -- 172.21.15.131:0/873023754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 msgr2=0x55bead0a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.044+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 0x55bead0a1c00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fafb4024000 tx=0x7fafb401ef40 comp rx=0 tx=0).stop 2024-05-10T05:45:18.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.044+0000 7fafe7a89b80 1 -- 172.21.15.131:0/873023754 shutdown_connections 2024-05-10T05:45:18.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.044+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55bead0ad2a0 0x55bead0af6f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.044+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098ec0 0x55bead0a8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.044+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.045+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/873023754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 0x55bead0a1c00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.045+0000 7fafe7a89b80 1 -- 172.21.15.131:0/873023754 >> 172.21.15.131:0/873023754 conn(0x55beacff9a60 msgr2=0x55bead091fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.045+0000 7fafaea00700 0 client.17091 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:45:18.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.048+0000 7fafe7a89b80 1 -- 172.21.15.131:0/873023754 shutdown_connections 2024-05-10T05:45:18.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.048+0000 7fafe7a89b80 1 -- 172.21.15.131:0/873023754 wait complete. 2024-05-10T05:45:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.048+0000 7fafe7a89b80 20 client.17091 trim_cache size 0 max 16384 2024-05-10T05:45:18.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:17 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/376814606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:45:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.092+0000 7fafe7a89b80 1 Processor -- start 2024-05-10T05:45:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.092+0000 7fafe7a89b80 1 -- start start 2024-05-10T05:45:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.093+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.094+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 0x55bead0a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.094+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 0x55bead0a8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.095+0000 7fafe7a89b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bead0a9c90 con 0x55bead098570 2024-05-10T05:45:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.095+0000 7fafe7a89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bead0983d0 con 0x55bead098ec0 2024-05-10T05:45:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.095+0000 7fafe7a89b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bead092b00 con 0x55bead0976a0 2024-05-10T05:45:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.095+0000 7fafcd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.095+0000 7fafcdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 0x55bead0a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.095+0000 7fafcdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 0x55bead0a8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34972/0 (socket says 172.21.15.131:34972) 2024-05-10T05:45:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.095+0000 7fafcdc00700 1 -- 172.21.15.131:0/315810709 learned_addr learned my addr 172.21.15.131:0/315810709 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.095+0000 7fafcd200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40366/0 (socket says 172.21.15.131:40366) 2024-05-10T05:45:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.096+0000 7fafc7e00700 1 --2- 172.21.15.131:0/315810709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 0x55bead0a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.096+0000 7fafcdc00700 1 -- 172.21.15.131:0/315810709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead0976a0 msgr2=0x55bead0a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.096+0000 7fafcdc00700 1 --2- 172.21.15.131:0/315810709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.096+0000 7fafcdc00700 1 -- 172.21.15.131:0/315810709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 msgr2=0x55bead0a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.096+0000 7fafcdc00700 1 --2- 172.21.15.131:0/315810709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 0x55bead0a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.097+0000 7fafcdc00700 1 -- 172.21.15.131:0/315810709 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beacde4b60 con 0x55bead098ec0 2024-05-10T05:45:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.097+0000 7fafcd200700 1 --2- 172.21.15.131:0/315810709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:45:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.097+0000 7fafc7e00700 1 --2- 172.21.15.131:0/315810709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 0x55bead0a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.097+0000 7fafcdc00700 1 --2- 172.21.15.131:0/315810709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 0x55bead0a8790 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c770 tx=0x7fafc00097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39ce539d4a6f47a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.097+0000 7fafbaa00700 1 -- 172.21.15.131:0/315810709 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0011040 con 0x55bead098ec0 2024-05-10T05:45:18.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.098+0000 7fafbaa00700 1 -- 172.21.15.131:0/315810709 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafc0012040 con 0x55bead098ec0 2024-05-10T05:45:18.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.098+0000 7fafbaa00700 1 -- 172.21.15.131:0/315810709 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000a660 con 0x55bead098ec0 2024-05-10T05:45:18.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.103+0000 7fafe7a89b80 1 -- 172.21.15.131:0/315810709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 msgr2=0x55bead0a8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.103+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/315810709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 0x55bead0a8790 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c770 tx=0x7fafc00097b0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.103+0000 7fafe7a89b80 1 -- 172.21.15.131:0/315810709 shutdown_connections 2024-05-10T05:45:18.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.103+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/315810709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead098ec0 0x55bead0a8790 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.103+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/315810709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead0976a0 0x55bead0a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.104+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/315810709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098570 0x55bead0a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.104+0000 7fafe7a89b80 1 -- 172.21.15.131:0/315810709 >> 172.21.15.131:0/315810709 conn(0x55beacff9a60 msgr2=0x55bead09b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.104+0000 7fafe7a89b80 1 -- 172.21.15.131:0/315810709 shutdown_connections 2024-05-10T05:45:18.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.104+0000 7fafe7a89b80 1 -- 172.21.15.131:0/315810709 wait complete. 2024-05-10T05:45:18.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.105+0000 7fafe7a89b80 1 -- start start 2024-05-10T05:45:18.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.106+0000 7fafe7a89b80 1 Processor -- start 2024-05-10T05:45:18.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.108+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.109+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead0a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.109+0000 7fafe7a89b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098ec0 0x55bead0a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.110+0000 7fafcd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.110+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead0a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.110+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead0a6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40380/0 (socket says 172.21.15.131:40380) 2024-05-10T05:45:18.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.110+0000 7fafcd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:34982/0 (socket says 172.21.15.131:34982) 2024-05-10T05:45:18.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.110+0000 7fafc7e00700 1 -- 172.21.15.131:0/1559371596 learned_addr learned my addr 172.21.15.131:0/1559371596 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:18.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.110+0000 7fafcdc00700 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098ec0 0x55bead0a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.110+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bead0a7fb0 con 0x55bead098ec0 2024-05-10T05:45:18.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.111+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bead0a8120 con 0x55bead0976a0 2024-05-10T05:45:18.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.111+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bead0a82e0 con 0x55bead098570 2024-05-10T05:45:18.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.111+0000 7fafe7a89b80 10 client.0 _ll_register_callbacks cb 0x55beacf5cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:18.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.112+0000 7fafcd200700 1 -- 172.21.15.131:0/1559371596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 msgr2=0x55bead0a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.112+0000 7fafcd200700 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead0a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.112+0000 7fafcd200700 1 -- 172.21.15.131:0/1559371596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098ec0 msgr2=0x55bead0a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.112+0000 7fafcd200700 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098ec0 0x55bead0a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.112+0000 7fafcd200700 1 -- 172.21.15.131:0/1559371596 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb4011440 con 0x55bead0976a0 2024-05-10T05:45:18.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.112+0000 7fafcdc00700 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098ec0 0x55bead0a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:18.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.112+0000 7fafc7e00700 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead0a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:18.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.113+0000 7fafcd200700 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a6650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0098e0 tx=0x7fafbc003790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a8ff9a0ba97982 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.113+0000 7fafaea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:45:18.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.113+0000 7fafaea00700 1 -- 172.21.15.131:0/1559371596 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc09c3d0 con 0x55bead0976a0 2024-05-10T05:45:18.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.113+0000 7fafaea00700 1 -- 172.21.15.131:0/1559371596 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fafbc09bcd0 con 0x55bead0976a0 2024-05-10T05:45:18.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.113+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafb4015070 con 0x55bead0976a0 2024-05-10T05:45:18.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.113+0000 7fafaea00700 1 -- 172.21.15.131:0/1559371596 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafbc09fb00 con 0x55bead0976a0 2024-05-10T05:45:18.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.115+0000 7fafe7a89b80 10 client.27494 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:18.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.115+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fafc000c420 con 0x55bead0976a0 2024-05-10T05:45:18.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.116+0000 7fafaea00700 1 -- 172.21.15.131:0/1559371596 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafb4015070 con 0x55bead0976a0 2024-05-10T05:45:18.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.116+0000 7fafaea00700 1 -- 172.21.15.131:0/1559371596 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fafbc085170 con 0x55bead0976a0 2024-05-10T05:45:18.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.116+0000 7fafe7a89b80 10 client.27494 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:18.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.116+0000 7fafe7a89b80 10 client.27494 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-10T05:45:18.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.116+0000 7fafe7a89b80 20 client.27494 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:18.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.116+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55bead103ea0 0x55bead1062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.116+0000 7fafe7a89b80 4 client.27494 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:45:18.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.116+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 --> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55beacfafec0 con 0x55bead103ea0 2024-05-10T05:45:18.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.116+0000 7fafc7e00700 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55bead103ea0 0x55bead1062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.117+0000 7fafc7e00700 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55bead103ea0 0x55bead1062c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d691df80aba9c38 server_cookie=ba1127361008c3a8 in_seq=0 out_seq=0 2024-05-10T05:45:18.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.117+0000 7fafaea00700 10 client.27494 ms_handle_connect on v2:172.21.15.157:6836/707351222 2024-05-10T05:45:18.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.117+0000 7fafaea00700 1 -- 172.21.15.131:0/1559371596 <== mds.4 v2:172.21.15.157:6836/707351222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55beacfafec0 con 0x55bead103ea0 2024-05-10T05:45:18.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.117+0000 7fafaea00700 10 client.27494 handle_command_reply: tid=0 2024-05-10T05:45:18.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.117+0000 7fafe7a89b80 1 client.27494 shutdown 2024-05-10T05:45:18.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 msgr2=0x55bead0a6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a6650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0098e0 tx=0x7fafbc003790 comp rx=0 tx=0).stop 2024-05-10T05:45:18.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 shutdown_connections 2024-05-10T05:45:18.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.157:6836/707351222,v1:172.21.15.157:6837/707351222] conn(0x55bead103ea0 0x55bead1062c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bead098ec0 0x55bead0a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bead0976a0 0x55bead0a6650 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafe7a89b80 1 --2- 172.21.15.131:0/1559371596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bead098570 0x55bead0a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafaea00700 0 client.27494 ms_handle_reset on v2:172.21.15.157:6836/707351222 2024-05-10T05:45:18.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 >> 172.21.15.131:0/1559371596 conn(0x55beacff9a60 msgr2=0x55bead0a1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 shutdown_connections 2024-05-10T05:45:18.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.118+0000 7fafe7a89b80 1 -- 172.21.15.131:0/1559371596 wait complete. 2024-05-10T05:45:18.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.119+0000 7fafe7a89b80 20 client.27494 trim_cache size 0 max 16384 2024-05-10T05:45:18.121 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:45:18.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.119+0000 7fafcfc00700 1 -- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fafb00a1240 msgr2=0x7fafb00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.119+0000 7fafcfc00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fafb00a1240 0x7fafb00a3750 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005950 tx=0x7fafb400b410 comp rx=0 tx=0).stop 2024-05-10T05:45:18.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.120+0000 7fafcfc00700 1 -- 172.21.15.131:0/3850317286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc8108630 msgr2=0x7fafc81a6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.120+0000 7fafcfc00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc8108630 0x7fafc81a6d70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fafbc005b40 tx=0x7fafbc00f710 comp rx=0 tx=0).stop 2024-05-10T05:45:18.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.120+0000 7fafcfc00700 1 -- 172.21.15.131:0/3850317286 shutdown_connections 2024-05-10T05:45:18.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.120+0000 7fafcfc00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fafb00a1240 0x7fafb00a3750 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.120+0000 7fafcfc00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc8109c50 0x7fafc81ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.120+0000 7fafcfc00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafc8108630 0x7fafc81a6d70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.120+0000 7fafcfc00700 1 --2- 172.21.15.131:0/3850317286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafc8109300 0x7fafc81a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.120+0000 7fafcfc00700 1 -- 172.21.15.131:0/3850317286 >> 172.21.15.131:0/3850317286 conn(0x7fafc806ea00 msgr2=0x7fafc810a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.120+0000 7fafcfc00700 1 -- 172.21.15.131:0/3850317286 shutdown_connections 2024-05-10T05:45:18.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.120+0000 7fafcfc00700 1 -- 172.21.15.131:0/3850317286 wait complete. 2024-05-10T05:45:18.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:17 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/376814606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:45:18.174 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T05:45:18.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.742+0000 7f1efde00700 1 -- 172.21.15.131:0/591007877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef8102b00 msgr2=0x7f1ef8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.742+0000 7f1efde00700 1 --2- 172.21.15.131:0/591007877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef8102b00 0x7f1ef8102f10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40099c0 tx=0x7f1ee4009cd0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.742+0000 7f1efde00700 1 -- 172.21.15.131:0/591007877 shutdown_connections 2024-05-10T05:45:18.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.742+0000 7f1efde00700 1 --2- 172.21.15.131:0/591007877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef81034a0 0x7f1ef8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.742+0000 7f1efde00700 1 --2- 172.21.15.131:0/591007877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef81019e0 0x7f1ef8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.742+0000 7f1efde00700 1 --2- 172.21.15.131:0/591007877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef8102b00 0x7f1ef8102f10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.742+0000 7f1efde00700 1 -- 172.21.15.131:0/591007877 >> 172.21.15.131:0/591007877 conn(0x7f1ef80fcfd0 msgr2=0x7f1ef80ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.742+0000 7f1efde00700 1 -- 172.21.15.131:0/591007877 shutdown_connections 2024-05-10T05:45:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.742+0000 7f1efde00700 1 -- 172.21.15.131:0/591007877 wait complete. 2024-05-10T05:45:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.744+0000 7f1efde00700 1 Processor -- start 2024-05-10T05:45:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.744+0000 7f1efde00700 1 -- start start 2024-05-10T05:45:18.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1efde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef81019e0 0x7f1ef8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1efde00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8102b00 0x7f1ef8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1efde00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef81034a0 0x7f1ef819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1efde00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef819f910 con 0x7f1ef81034a0 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1efde00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef819fa50 con 0x7f1ef81019e0 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1efde00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef819fbe0 con 0x7f1ef8102b00 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1efca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef81019e0 0x7f1ef8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1efca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef81019e0 0x7f1ef8197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35000/0 (socket says 172.21.15.131:35000) 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1efca00700 1 -- 172.21.15.131:0/546072708 learned_addr learned my addr 172.21.15.131:0/546072708 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1ef7e00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8102b00 0x7f1ef8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.745+0000 7f1efd400700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef81034a0 0x7f1ef819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1efca00700 1 -- 172.21.15.131:0/546072708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8102b00 msgr2=0x7f1ef8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1efca00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8102b00 0x7f1ef8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1efca00700 1 -- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef81034a0 msgr2=0x7f1ef819f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1efca00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef81034a0 0x7f1ef819f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1efca00700 1 -- 172.21.15.131:0/546072708 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40095f0 con 0x7f1ef81019e0 2024-05-10T05:45:18.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1efd400700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef81034a0 0x7f1ef819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:45:18.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1efca00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef81019e0 0x7f1ef8197f50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00b4d0 tx=0x7f1eec00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1ef5600700 1 -- 172.21.15.131:0/546072708 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec011830 con 0x7f1ef81019e0 2024-05-10T05:45:18.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1ef5600700 1 -- 172.21.15.131:0/546072708 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eec011e50 con 0x7f1ef81019e0 2024-05-10T05:45:18.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1efde00700 1 -- 172.21.15.131:0/546072708 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef819fec0 con 0x7f1ef81019e0 2024-05-10T05:45:18.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.746+0000 7f1ef5600700 1 -- 172.21.15.131:0/546072708 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec010ea0 con 0x7f1ef81019e0 2024-05-10T05:45:18.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.747+0000 7f1efde00700 1 -- 172.21.15.131:0/546072708 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef81a0340 con 0x7f1ef81019e0 2024-05-10T05:45:18.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.754+0000 7f1ef5600700 1 -- 172.21.15.131:0/546072708 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1eec010510 con 0x7f1ef81019e0 2024-05-10T05:45:18.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.758+0000 7f1ef5600700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ee00a11a0 0x7f1ee00a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.758+0000 7f1f17413b80 1 Processor -- start 2024-05-10T05:45:18.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.758+0000 7f1ef5600700 1 -- 172.21.15.131:0/546072708 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1eec0cc280 con 0x7f1ef81019e0 2024-05-10T05:45:18.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.759+0000 7f1f17413b80 1 -- start start 2024-05-10T05:45:18.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.760+0000 7f1ef7e00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ee00a11a0 0x7f1ee00a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.760+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d158d580 0x55f9d158d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.760+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d1596f60 0x55f9d1597370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.760+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d15978b0 0x55f9d159e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1f17413b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d14b4150 con 0x55f9d1596f60 2024-05-10T05:45:18.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1f17413b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d14b1b10 con 0x55f9d15978b0 2024-05-10T05:45:18.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1f17413b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d14ad400 con 0x55f9d158d580 2024-05-10T05:45:18.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1efd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d15978b0 0x55f9d159e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1efd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d15978b0 0x55f9d159e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35002/0 (socket says 172.21.15.131:35002) 2024-05-10T05:45:18.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1efd400700 1 -- 172.21.15.131:0/3597294050 learned_addr learned my addr 172.21.15.131:0/3597294050 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:18.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.760+0000 7f1efca00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d158d580 0x55f9d158d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1efd400700 1 -- 172.21.15.131:0/3597294050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d158d580 msgr2=0x55f9d158d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1efd400700 1 --2- 172.21.15.131:0/3597294050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d158d580 0x55f9d158d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1efd400700 1 -- 172.21.15.131:0/3597294050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d1596f60 msgr2=0x55f9d1597370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T05:45:18.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1efd400700 1 --2- 172.21.15.131:0/3597294050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d1596f60 0x55f9d1597370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.761+0000 7f1efd400700 1 -- 172.21.15.131:0/3597294050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d132e3b0 con 0x55f9d15978b0 2024-05-10T05:45:18.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.762+0000 7f1efd400700 1 --2- 172.21.15.131:0/3597294050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d15978b0 0x55f9d159e1b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000e260 tx=0x7f1ef000e570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21bef31cf79a81a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.762+0000 7f1ef7e00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ee00a11a0 0x7f1ee00a36b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8198fa0 tx=0x7f1ee401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.766+0000 7f1ee9c00700 1 -- 172.21.15.131:0/3597294050 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0018070 con 0x55f9d15978b0 2024-05-10T05:45:18.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.766+0000 7f1ee9c00700 1 -- 172.21.15.131:0/3597294050 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef000ee00 con 0x55f9d15978b0 2024-05-10T05:45:18.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.767+0000 7f1f17413b80 1 -- 172.21.15.131:0/3597294050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d15978b0 msgr2=0x55f9d159e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.767+0000 7f1f17413b80 1 --2- 172.21.15.131:0/3597294050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d15978b0 0x55f9d159e1b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000e260 tx=0x7f1ef000e570 comp rx=0 tx=0).stop 2024-05-10T05:45:18.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.767+0000 7f1f17413b80 1 -- 172.21.15.131:0/3597294050 shutdown_connections 2024-05-10T05:45:18.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.767+0000 7f1f17413b80 1 --2- 172.21.15.131:0/3597294050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d15978b0 0x55f9d159e1b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.767+0000 7f1f17413b80 1 --2- 172.21.15.131:0/3597294050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d158d580 0x55f9d158d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.767+0000 7f1f17413b80 1 --2- 172.21.15.131:0/3597294050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d1596f60 0x55f9d1597370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.768+0000 7f1f17413b80 1 -- 172.21.15.131:0/3597294050 >> 172.21.15.131:0/3597294050 conn(0x55f9d14efa60 msgr2=0x55f9d1588330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.769+0000 7f1f17413b80 1 -- 172.21.15.131:0/3597294050 shutdown_connections 2024-05-10T05:45:18.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.769+0000 7f1f17413b80 1 -- 172.21.15.131:0/3597294050 wait complete. 2024-05-10T05:45:18.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.770+0000 7f1f17413b80 1 -- start start 2024-05-10T05:45:18.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.770+0000 7f1f17413b80 1 Processor -- start 2024-05-10T05:45:18.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.772+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d158d580 0x55f9d15a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.773+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d1596f60 0x55f9d15a0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.773+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d15a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.774+0000 7f1f17413b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d1449560 con 0x55f9d158d580 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.774+0000 7f1f17413b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d1449200 con 0x55f9d1596f60 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.774+0000 7f1f17413b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d13306a0 con 0x55f9d15978b0 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.774+0000 7f1f17413b80 10 client.0 _ll_register_callbacks cb 0x55f9d1452b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.775+0000 7f1efd400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d15a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.775+0000 7f1efd400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d15a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40420/0 (socket says 172.21.15.131:40420) 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.775+0000 7f1efd400700 1 -- 172.21.15.131:0/1966227091 learned_addr learned my addr 172.21.15.131:0/1966227091 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.775+0000 7f1efd400700 1 -- 172.21.15.131:0/1966227091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d1596f60 msgr2=0x55f9d15a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.775+0000 7f1efd400700 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d1596f60 0x55f9d15a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.775+0000 7f1efd400700 1 -- 172.21.15.131:0/1966227091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d158d580 msgr2=0x55f9d15a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.775+0000 7f1efd400700 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d158d580 0x55f9d15a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.775+0000 7f1efd400700 1 -- 172.21.15.131:0/1966227091 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef000f040 con 0x55f9d15978b0 2024-05-10T05:45:18.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.779+0000 7f1efd400700 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d15a75c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009010 tx=0x7f1ef0004d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c49a1d87b1c4967 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.779+0000 7f1ede000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:45:18.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.780+0000 7f1ede000700 1 -- 172.21.15.131:0/1966227091 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0014850 con 0x55f9d15978b0 2024-05-10T05:45:18.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.780+0000 7f1ede000700 1 -- 172.21.15.131:0/1966227091 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1ef0014e40 con 0x55f9d15978b0 2024-05-10T05:45:18.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.780+0000 7f1f17413b80 1 -- 172.21.15.131:0/1966227091 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9d1406270 con 0x55f9d15978b0 2024-05-10T05:45:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.807+0000 7f1ede000700 1 -- 172.21.15.131:0/1966227091 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ef001e420 con 0x55f9d15978b0 2024-05-10T05:45:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.807+0000 7f1f17413b80 10 client.27508 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.807+0000 7f1f17413b80 1 -- 172.21.15.131:0/1966227091 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f9d12dab60 con 0x55f9d15978b0 2024-05-10T05:45:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.807+0000 7f1ede000700 1 -- 172.21.15.131:0/1966227091 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9d1406270 con 0x55f9d15978b0 2024-05-10T05:45:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.809+0000 7f1ede000700 1 -- 172.21.15.131:0/1966227091 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1ef00325e0 con 0x55f9d15978b0 2024-05-10T05:45:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.809+0000 7f1f17413b80 10 client.27508 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.809+0000 7f1f17413b80 10 client.27508 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:45:18.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.809+0000 7f1f17413b80 20 client.27508 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:18.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.809+0000 7f1f17413b80 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55f9d15abc60 0x55f9d15ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.809+0000 7f1f17413b80 4 client.27508 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:45:18.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.809+0000 7f1f17413b80 1 -- 172.21.15.131:0/1966227091 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9d155c780 con 0x55f9d15abc60 2024-05-10T05:45:18.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.809+0000 7f1efca00700 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55f9d15abc60 0x55f9d15ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.812+0000 7f1efca00700 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55f9d15abc60 0x55f9d15ae0b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dbaf6d35c59ba571 server_cookie=d1807138c9144cdd in_seq=0 out_seq=0 2024-05-10T05:45:18.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.812+0000 7f1ede000700 10 client.27508 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:45:18.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.814+0000 7f1ede000700 1 -- 172.21.15.131:0/1966227091 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9d155c780 con 0x55f9d15abc60 2024-05-10T05:45:18.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.814+0000 7f1ede000700 10 client.27508 handle_command_reply: tid=0 2024-05-10T05:45:18.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.814+0000 7f1f17413b80 1 client.27508 shutdown 2024-05-10T05:45:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.816+0000 7f1f17413b80 1 -- 172.21.15.131:0/1966227091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 msgr2=0x55f9d15a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.816+0000 7f1f17413b80 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d15a75c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009010 tx=0x7f1ef0004d60 comp rx=0 tx=0).stop 2024-05-10T05:45:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.817+0000 7f1f17413b80 1 -- 172.21.15.131:0/1966227091 shutdown_connections 2024-05-10T05:45:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.817+0000 7f1f17413b80 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55f9d15abc60 0x55f9d15ae0b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.817+0000 7f1f17413b80 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d158d580 0x55f9d15a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.817+0000 7f1f17413b80 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d15a75c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.817+0000 7f1f17413b80 1 --2- 172.21.15.131:0/1966227091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d1596f60 0x55f9d15a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.817+0000 7f1f17413b80 1 -- 172.21.15.131:0/1966227091 >> 172.21.15.131:0/1966227091 conn(0x55f9d14efa60 msgr2=0x55f9d158c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.817+0000 7f1ede000700 0 client.27508 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:45:18.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.818+0000 7f1f17413b80 1 -- 172.21.15.131:0/1966227091 shutdown_connections 2024-05-10T05:45:18.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.818+0000 7f1f17413b80 1 -- 172.21.15.131:0/1966227091 wait complete. 2024-05-10T05:45:18.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.821+0000 7f1f17413b80 20 client.27508 trim_cache size 0 max 16384 2024-05-10T05:45:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.856+0000 7f1f17413b80 1 Processor -- start 2024-05-10T05:45:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.857+0000 7f1f17413b80 1 -- start start 2024-05-10T05:45:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.857+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d158d580 0x55f9d159acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.857+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d1596f60 0x55f9d15a7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.858+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d15978b0 0x55f9d159b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.858+0000 7f1f17413b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d15828d0 con 0x55f9d15978b0 2024-05-10T05:45:18.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.858+0000 7f1f17413b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d159a810 con 0x55f9d158d580 2024-05-10T05:45:18.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.858+0000 7f1f17413b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d15a0830 con 0x55f9d1596f60 2024-05-10T05:45:18.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.858+0000 7f1efca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d158d580 0x55f9d159acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.858+0000 7f1ef7e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d1596f60 0x55f9d15a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.858+0000 7f1efca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d158d580 0x55f9d159acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35030/0 (socket says 172.21.15.131:35030) 2024-05-10T05:45:18.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.858+0000 7f1efca00700 1 -- 172.21.15.131:0/2822696892 learned_addr learned my addr 172.21.15.131:0/2822696892 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.858+0000 7f1efd400700 1 --2- 172.21.15.131:0/2822696892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d15978b0 0x55f9d159b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.859+0000 7f1efd400700 1 -- 172.21.15.131:0/2822696892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d1596f60 msgr2=0x55f9d15a7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.859+0000 7f1efd400700 1 --2- 172.21.15.131:0/2822696892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d1596f60 0x55f9d15a7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.859+0000 7f1efd400700 1 -- 172.21.15.131:0/2822696892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d158d580 msgr2=0x55f9d159acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.859+0000 7f1efd400700 1 --2- 172.21.15.131:0/2822696892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d158d580 0x55f9d159acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.859+0000 7f1efd400700 1 -- 172.21.15.131:0/2822696892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d1473560 con 0x55f9d15978b0 2024-05-10T05:45:18.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.860+0000 7f1efd400700 1 --2- 172.21.15.131:0/2822696892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d15978b0 0x55f9d159b1f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0013b50 tx=0x7f1ef0004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2432d908fbc30008 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.860+0000 7f1eea600700 1 -- 172.21.15.131:0/2822696892 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000b070 con 0x55f9d15978b0 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.860+0000 7f1eea600700 1 -- 172.21.15.131:0/2822696892 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef0007be0 con 0x55f9d15978b0 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.860+0000 7f1eea600700 1 -- 172.21.15.131:0/2822696892 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0013f00 con 0x55f9d15978b0 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.861+0000 7f1f17413b80 1 -- 172.21.15.131:0/2822696892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d15978b0 msgr2=0x55f9d159b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.861+0000 7f1f17413b80 1 --2- 172.21.15.131:0/2822696892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d15978b0 0x55f9d159b1f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0013b50 tx=0x7f1ef0004f00 comp rx=0 tx=0).stop 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.861+0000 7f1f17413b80 1 -- 172.21.15.131:0/2822696892 shutdown_connections 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.861+0000 7f1f17413b80 1 --2- 172.21.15.131:0/2822696892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d15978b0 0x55f9d159b1f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.861+0000 7f1f17413b80 1 --2- 172.21.15.131:0/2822696892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d158d580 0x55f9d159acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.861+0000 7f1f17413b80 1 --2- 172.21.15.131:0/2822696892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d1596f60 0x55f9d15a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.861+0000 7f1f17413b80 1 -- 172.21.15.131:0/2822696892 >> 172.21.15.131:0/2822696892 conn(0x55f9d14efa60 msgr2=0x55f9d15a0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.861+0000 7f1f17413b80 1 -- 172.21.15.131:0/2822696892 shutdown_connections 2024-05-10T05:45:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.861+0000 7f1f17413b80 1 -- 172.21.15.131:0/2822696892 wait complete. 2024-05-10T05:45:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.862+0000 7f1f17413b80 1 -- start start 2024-05-10T05:45:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.862+0000 7f1f17413b80 1 Processor -- start 2024-05-10T05:45:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d158d580 0x55f9d15a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d1596f60 0x55f9d15a4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1f17413b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d1600140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1f17413b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d15a7660 con 0x55f9d158d580 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1f17413b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d15a77d0 con 0x55f9d1596f60 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1f17413b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d1600680 con 0x55f9d15978b0 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1f17413b80 10 client.0 _ll_register_callbacks cb 0x55f9d1452b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1efd400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d1600140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1efd400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d1600140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40442/0 (socket says 172.21.15.131:40442) 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1efd400700 1 -- 172.21.15.131:0/3111475644 learned_addr learned my addr 172.21.15.131:0/3111475644 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.863+0000 7f1efca00700 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d158d580 0x55f9d15a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.864+0000 7f1efd400700 1 -- 172.21.15.131:0/3111475644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d1596f60 msgr2=0x55f9d15a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.864+0000 7f1efd400700 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d1596f60 0x55f9d15a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.864+0000 7f1efd400700 1 -- 172.21.15.131:0/3111475644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d158d580 msgr2=0x55f9d15a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.864+0000 7f1efd400700 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d158d580 0x55f9d15a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.864+0000 7f1efd400700 1 -- 172.21.15.131:0/3111475644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9d12dab60 con 0x55f9d15978b0 2024-05-10T05:45:18.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.864+0000 7f1efd400700 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d1600140 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000e180 tx=0x7f1ef000e980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5238b498cb3d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:18.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.864+0000 7f1ede000700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:45:18.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.865+0000 7f1ede000700 1 -- 172.21.15.131:0/3111475644 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000b040 con 0x55f9d15978b0 2024-05-10T05:45:18.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.865+0000 7f1ede000700 1 -- 172.21.15.131:0/3111475644 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1ef001fe70 con 0x55f9d15978b0 2024-05-10T05:45:18.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.865+0000 7f1f17413b80 1 -- 172.21.15.131:0/3111475644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ef000ee00 con 0x55f9d15978b0 2024-05-10T05:45:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.865+0000 7f1ede000700 1 -- 172.21.15.131:0/3111475644 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ef0013e80 con 0x55f9d15978b0 2024-05-10T05:45:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.865+0000 7f1f17413b80 10 client.27514 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.865+0000 7f1f17413b80 1 -- 172.21.15.131:0/3111475644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f9d1332cd0 con 0x55f9d15978b0 2024-05-10T05:45:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.865+0000 7f1ede000700 1 -- 172.21.15.131:0/3111475644 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef000ee00 con 0x55f9d15978b0 2024-05-10T05:45:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.866+0000 7f1ede000700 1 -- 172.21.15.131:0/3111475644 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1ef00133e0 con 0x55f9d15978b0 2024-05-10T05:45:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.866+0000 7f1f17413b80 10 client.27514 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.866+0000 7f1f17413b80 10 client.27514 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-05-10T05:45:18.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.866+0000 7f1f17413b80 20 client.27514 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.866+0000 7f1f17413b80 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55f9d1601a00 0x55f9d1603e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.866+0000 7f1f17413b80 4 client.27514 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.866+0000 7f1f17413b80 1 -- 172.21.15.131:0/3111475644 --> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f9d14a5ec0 con 0x55f9d1601a00 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.866+0000 7f1efca00700 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55f9d1601a00 0x55f9d1603e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.867+0000 7f1ede000700 10 client.27514 ms_handle_connect on v2:172.21.15.157:6838/1794088540 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.867+0000 7f1efca00700 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55f9d1601a00 0x55f9d1603e20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8c99e5ffe530d7c4 server_cookie=88a86f2731936dfe in_seq=0 out_seq=0 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.867+0000 7f1ede000700 1 -- 172.21.15.131:0/3111475644 <== mds.3 v2:172.21.15.157:6838/1794088540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f9d14a5ec0 con 0x55f9d1601a00 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.867+0000 7f1ede000700 10 client.27514 handle_command_reply: tid=0 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.867+0000 7f1f17413b80 1 client.27514 shutdown 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 1 -- 172.21.15.131:0/3111475644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 msgr2=0x55f9d1600140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d1600140 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000e180 tx=0x7f1ef000e980 comp rx=0 tx=0).stop 2024-05-10T05:45:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 1 -- 172.21.15.131:0/3111475644 shutdown_connections 2024-05-10T05:45:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.157:6838/1794088540,v1:172.21.15.157:6839/1794088540] conn(0x55f9d1601a00 0x55f9d1603e20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9d158d580 0x55f9d15a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9d15978b0 0x55f9d1600140 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 1 --2- 172.21.15.131:0/3111475644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9d1596f60 0x55f9d15a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 1 -- 172.21.15.131:0/3111475644 >> 172.21.15.131:0/3111475644 conn(0x55f9d14efa60 msgr2=0x55f9d159acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1ede000700 0 client.27514 ms_handle_reset on v2:172.21.15.157:6838/1794088540 2024-05-10T05:45:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 1 -- 172.21.15.131:0/3111475644 shutdown_connections 2024-05-10T05:45:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 1 -- 172.21.15.131:0/3111475644 wait complete. 2024-05-10T05:45:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.868+0000 7f1f17413b80 20 client.27514 trim_cache size 0 max 16384 2024-05-10T05:45:18.871 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:45:18.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.869+0000 7f1efde00700 1 -- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ee00a11a0 msgr2=0x7f1ee00a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.869+0000 7f1efde00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ee00a11a0 0x7f1ee00a36b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8198fa0 tx=0x7f1ee401a040 comp rx=0 tx=0).stop 2024-05-10T05:45:18.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.869+0000 7f1efde00700 1 -- 172.21.15.131:0/546072708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef81019e0 msgr2=0x7f1ef8197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:18.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.869+0000 7f1efde00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef81019e0 0x7f1ef8197f50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00b4d0 tx=0x7f1eec00b7e0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.869+0000 7f1efde00700 1 -- 172.21.15.131:0/546072708 shutdown_connections 2024-05-10T05:45:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.869+0000 7f1efde00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1ee00a11a0 0x7f1ee00a36b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.869+0000 7f1efde00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ef81034a0 0x7f1ef819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.869+0000 7f1efde00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ef81019e0 0x7f1ef8197f50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.869+0000 7f1efde00700 1 --2- 172.21.15.131:0/546072708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8102b00 0x7f1ef8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.869+0000 7f1efde00700 1 -- 172.21.15.131:0/546072708 >> 172.21.15.131:0/546072708 conn(0x7f1ef80fcfd0 msgr2=0x7f1ef8105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.870+0000 7f1efde00700 1 -- 172.21.15.131:0/546072708 shutdown_connections 2024-05-10T05:45:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:18.870+0000 7f1efde00700 1 -- 172.21.15.131:0/546072708 wait complete. 2024-05-10T05:45:18.909 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T05:45:19.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:18 smithi157 ceph-mon[78427]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 312 MiB/s wr, 101 op/s 2024-05-10T05:45:19.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:18 smithi142 ceph-mon[77667]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 312 MiB/s wr, 101 op/s 2024-05-10T05:45:19.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:18 smithi131 ceph-mon[37264]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 312 MiB/s wr, 101 op/s 2024-05-10T05:45:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.393+0000 7f1e18e00700 1 -- 172.21.15.131:0/1980652645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e14107040 msgr2=0x7f1e14109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.393+0000 7f1e18e00700 1 --2- 172.21.15.131:0/1980652645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e14107040 0x7f1e14109460 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00b0a0 tx=0x7f1dfc00b3b0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.393+0000 7f1e18e00700 1 -- 172.21.15.131:0/1980652645 shutdown_connections 2024-05-10T05:45:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.393+0000 7f1e18e00700 1 --2- 172.21.15.131:0/1980652645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e140690b0 0x7f1e14106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.393+0000 7f1e18e00700 1 --2- 172.21.15.131:0/1980652645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e14109a80 0x7f1e1410bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.393+0000 7f1e18e00700 1 --2- 172.21.15.131:0/1980652645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e14107040 0x7f1e14109460 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.393+0000 7f1e18e00700 1 -- 172.21.15.131:0/1980652645 >> 172.21.15.131:0/1980652645 conn(0x7f1e140fc0d0 msgr2=0x7f1e140fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.393+0000 7f1e18e00700 1 -- 172.21.15.131:0/1980652645 shutdown_connections 2024-05-10T05:45:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.393+0000 7f1e18e00700 1 -- 172.21.15.131:0/1980652645 wait complete. 2024-05-10T05:45:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.395+0000 7f1e18e00700 1 Processor -- start 2024-05-10T05:45:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.395+0000 7f1e18e00700 1 -- start start 2024-05-10T05:45:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.395+0000 7f1e18e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e14107040 0x7f1e1419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.395+0000 7f1e18e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e14109a80 0x7f1e1419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.396+0000 7f1e11400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e14109a80 0x7f1e1419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.396+0000 7f1e11400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e14109a80 0x7f1e1419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40472/0 (socket says 172.21.15.131:40472) 2024-05-10T05:45:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.396+0000 7f1e11e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e14107040 0x7f1e1419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.396+0000 7f1e11e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e14107040 0x7f1e1419e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35050/0 (socket says 172.21.15.131:35050) 2024-05-10T05:45:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.396+0000 7f1e18e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e141a3730 0x7f1e141a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.396+0000 7f1e18e00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e141a6120 con 0x7f1e141a3730 2024-05-10T05:45:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.396+0000 7f1e18e00700 1 -- 172.21.15.131:0/865855234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e141a6260 con 0x7f1e14107040 2024-05-10T05:45:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.396+0000 7f1e18e00700 1 -- 172.21.15.131:0/865855234 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e141a63f0 con 0x7f1e14109a80 2024-05-10T05:45:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.396+0000 7f1e11400700 1 -- 172.21.15.131:0/865855234 learned_addr learned my addr 172.21.15.131:0/865855234 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:19.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.397+0000 7f1e12800700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e141a3730 0x7f1e141a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.397+0000 7f1e11400700 1 -- 172.21.15.131:0/865855234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e14107040 msgr2=0x7f1e1419e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.397+0000 7f1e11400700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e14107040 0x7f1e1419e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.397+0000 7f1e11400700 1 -- 172.21.15.131:0/865855234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e141a3730 msgr2=0x7f1e141a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.397+0000 7f1e11400700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e141a3730 0x7f1e141a5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.397+0000 7f1e11400700 1 -- 172.21.15.131:0/865855234 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc009d00 con 0x7f1e14109a80 2024-05-10T05:45:19.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.397+0000 7f1e11400700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e14109a80 0x7f1e1419e7b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0095d0 tx=0x7f1dfc00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:19.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.397+0000 7f1e0aa00700 1 -- 172.21.15.131:0/865855234 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc00e040 con 0x7f1e14109a80 2024-05-10T05:45:19.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.397+0000 7f1e0aa00700 1 -- 172.21.15.131:0/865855234 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dfc004c50 con 0x7f1e14109a80 2024-05-10T05:45:19.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.397+0000 7f1e0aa00700 1 -- 172.21.15.131:0/865855234 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0052a0 con 0x7f1e14109a80 2024-05-10T05:45:19.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.398+0000 7f1e18e00700 1 -- 172.21.15.131:0/865855234 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e141a6670 con 0x7f1e14109a80 2024-05-10T05:45:19.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.398+0000 7f1e18e00700 1 -- 172.21.15.131:0/865855234 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e141a6aa0 con 0x7f1e14109a80 2024-05-10T05:45:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.402+0000 7f1e0aa00700 1 -- 172.21.15.131:0/865855234 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1dfc01d470 con 0x7f1e14109a80 2024-05-10T05:45:19.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.403+0000 7f1e0aa00700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1df40a1240 0x7f1df40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.404+0000 7f1e0aa00700 1 -- 172.21.15.131:0/865855234 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1dfc0cd5c0 con 0x7f1e14109a80 2024-05-10T05:45:19.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.404+0000 7f1e11e00700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1df40a1240 0x7f1df40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.405+0000 7f1e30dfeb80 1 Processor -- start 2024-05-10T05:45:19.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.405+0000 7f1e30dfeb80 1 -- start start 2024-05-10T05:45:19.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.405+0000 7f1e11e00700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1df40a1240 0x7f1df40a3750 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1e14068da0 tx=0x7f1e04009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:19.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.405+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57da6a0 0x55bad57daaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.405+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 0x55bad57db980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57dbec0 0x55bad57e27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad5701150 con 0x55bad57dbec0 2024-05-10T05:45:19.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad56feb10 con 0x55bad57da6a0 2024-05-10T05:45:19.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad56fa400 con 0x55bad57db570 2024-05-10T05:45:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e11400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 0x55bad57db980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e11e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57da6a0 0x55bad57daaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e11400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 0x55bad57db980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40488/0 (socket says 172.21.15.131:40488) 2024-05-10T05:45:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e11400700 1 -- 172.21.15.131:0/3659721881 learned_addr learned my addr 172.21.15.131:0/3659721881 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e12800700 1 --2- 172.21.15.131:0/3659721881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57dbec0 0x55bad57e27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e11400700 1 -- 172.21.15.131:0/3659721881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57da6a0 msgr2=0x55bad57daaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e11400700 1 --2- 172.21.15.131:0/3659721881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57da6a0 0x55bad57daaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e11400700 1 -- 172.21.15.131:0/3659721881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57dbec0 msgr2=0x55bad57e27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e11400700 1 --2- 172.21.15.131:0/3659721881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57dbec0 0x55bad57e27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e11400700 1 -- 172.21.15.131:0/3659721881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bad557b3b0 con 0x55bad57db570 2024-05-10T05:45:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.406+0000 7f1e12800700 1 --2- 172.21.15.131:0/3659721881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57dbec0 0x55bad57e27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:45:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.407+0000 7f1e11400700 1 --2- 172.21.15.131:0/3659721881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 0x55bad57db980 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc096fd0 tx=0x7f1dfc006020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8a6708a6bb72850 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.407+0000 7f1e03400700 1 -- 172.21.15.131:0/3659721881 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc09b1d0 con 0x55bad57db570 2024-05-10T05:45:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.407+0000 7f1e03400700 1 -- 172.21.15.131:0/3659721881 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dfc09aad0 con 0x55bad57db570 2024-05-10T05:45:19.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.407+0000 7f1e03400700 1 -- 172.21.15.131:0/3659721881 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc099fc0 con 0x55bad57db570 2024-05-10T05:45:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.408+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/3659721881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 msgr2=0x55bad57db980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.408+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/3659721881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 0x55bad57db980 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc096fd0 tx=0x7f1dfc006020 comp rx=0 tx=0).stop 2024-05-10T05:45:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.409+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/3659721881 shutdown_connections 2024-05-10T05:45:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.409+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/3659721881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57dbec0 0x55bad57e27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.409+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/3659721881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57da6a0 0x55bad57daaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.409+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/3659721881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 0x55bad57db980 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.409+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/3659721881 >> 172.21.15.131:0/3659721881 conn(0x55bad573ca60 msgr2=0x55bad57d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.409+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/3659721881 shutdown_connections 2024-05-10T05:45:19.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.409+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/3659721881 wait complete. 2024-05-10T05:45:19.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.409+0000 7f1e30dfeb80 1 -- start start 2024-05-10T05:45:19.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.410+0000 7f1e30dfeb80 1 Processor -- start 2024-05-10T05:45:19.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.410+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57da6a0 0x55bad57e4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.410+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 0x55bad57e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.411+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57dbec0 0x55bad57eb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.411+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad5696560 con 0x55bad57da6a0 2024-05-10T05:45:19.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.411+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad5696200 con 0x55bad57dbec0 2024-05-10T05:45:19.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.411+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad557d6a0 con 0x55bad57db570 2024-05-10T05:45:19.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.411+0000 7f1e30dfeb80 10 client.0 _ll_register_callbacks cb 0x55bad569fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:19.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.411+0000 7f1e12800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57dbec0 0x55bad57eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.411+0000 7f1e12800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57dbec0 0x55bad57eb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35062/0 (socket says 172.21.15.131:35062) 2024-05-10T05:45:19.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.411+0000 7f1e12800700 1 -- 172.21.15.131:0/1654425892 learned_addr learned my addr 172.21.15.131:0/1654425892 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:19.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.411+0000 7f1e11e00700 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57da6a0 0x55bad57e4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.412+0000 7f1e12800700 1 -- 172.21.15.131:0/1654425892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 msgr2=0x55bad57e4a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T05:45:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.412+0000 7f1e12800700 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 0x55bad57e4a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.412+0000 7f1e12800700 1 -- 172.21.15.131:0/1654425892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57da6a0 msgr2=0x55bad57e4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.412+0000 7f1e12800700 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57da6a0 0x55bad57e4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.412+0000 7f1e12800700 1 -- 172.21.15.131:0/1654425892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e141a6aa0 con 0x55bad57dbec0 2024-05-10T05:45:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.412+0000 7f1e12800700 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57dbec0 0x55bad57eb980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00f760 tx=0x7f1e0c00fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9c3562c87c8c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.412+0000 7f1dfbe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:45:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.413+0000 7f1dfbe00700 1 -- 172.21.15.131:0/1654425892 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c004a30 con 0x55bad57dbec0 2024-05-10T05:45:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.413+0000 7f1dfbe00700 1 -- 172.21.15.131:0/1654425892 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1e0c004b90 con 0x55bad57dbec0 2024-05-10T05:45:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.413+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/1654425892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bad5653270 con 0x55bad57dbec0 2024-05-10T05:45:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.413+0000 7f1dfbe00700 1 -- 172.21.15.131:0/1654425892 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e0c004d50 con 0x55bad57dbec0 2024-05-10T05:45:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.413+0000 7f1e30dfeb80 10 client.27515 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.413+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/1654425892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bad5527b60 con 0x55bad57dbec0 2024-05-10T05:45:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.414+0000 7f1dfbe00700 1 -- 172.21.15.131:0/1654425892 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bad5653270 con 0x55bad57dbec0 2024-05-10T05:45:19.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.414+0000 7f1dfbe00700 1 -- 172.21.15.131:0/1654425892 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1e0c018030 con 0x55bad57dbec0 2024-05-10T05:45:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.414+0000 7f1e30dfeb80 10 client.27515 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.415+0000 7f1e30dfeb80 10 client.27515 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:45:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.415+0000 7f1e30dfeb80 20 client.27515 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.415+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bad57f0220 0x55bad57f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.415+0000 7f1e30dfeb80 4 client.27515 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:45:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.415+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/1654425892 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bad57a9780 con 0x55bad57f0220 2024-05-10T05:45:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.415+0000 7f1e11e00700 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bad57f0220 0x55bad57f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.416+0000 7f1e11e00700 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bad57f0220 0x55bad57f2670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c9201e0392b2dd3 server_cookie=8d85ba08d6dcbe50 in_seq=0 out_seq=0 2024-05-10T05:45:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.416+0000 7f1dfbe00700 10 client.27515 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:45:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.417+0000 7f1dfbe00700 1 -- 172.21.15.131:0/1654425892 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bad57a9780 con 0x55bad57f0220 2024-05-10T05:45:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.417+0000 7f1dfbe00700 10 client.27515 handle_command_reply: tid=0 2024-05-10T05:45:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.417+0000 7f1e30dfeb80 1 client.27515 shutdown 2024-05-10T05:45:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/1654425892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57dbec0 msgr2=0x55bad57eb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57dbec0 0x55bad57eb980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00f760 tx=0x7f1e0c00fb20 comp rx=0 tx=0).stop 2024-05-10T05:45:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/1654425892 shutdown_connections 2024-05-10T05:45:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bad57f0220 0x55bad57f2670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad57da6a0 0x55bad57e4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57dbec0 0x55bad57eb980 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/1654425892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57db570 0x55bad57e4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/1654425892 >> 172.21.15.131:0/1654425892 conn(0x55bad573ca60 msgr2=0x55bad57d4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:19.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1dfbe00700 0 client.27515 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:45:19.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/1654425892 shutdown_connections 2024-05-10T05:45:19.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.419+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/1654425892 wait complete. 2024-05-10T05:45:19.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.420+0000 7f1e30dfeb80 20 client.27515 trim_cache size 0 max 16384 2024-05-10T05:45:19.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.439+0000 7f1e30dfeb80 1 Processor -- start 2024-05-10T05:45:19.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.439+0000 7f1e30dfeb80 1 -- start start 2024-05-10T05:45:19.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.440+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 0x55bad57df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.440+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57df950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.440+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57db570 0x55bad57eb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e11400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57df950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad57d5500 con 0x55bad573ca60 2024-05-10T05:45:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e11e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 0x55bad57df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad57ebd70 con 0x55bad57db570 2024-05-10T05:45:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad57e26e0 con 0x55bad57da6a0 2024-05-10T05:45:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e12800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57db570 0x55bad57eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e11400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57df950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40504/0 (socket says 172.21.15.131:40504) 2024-05-10T05:45:19.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e11400700 1 -- 172.21.15.131:0/3423069049 learned_addr learned my addr 172.21.15.131:0/3423069049 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:19.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e11400700 1 -- 172.21.15.131:0/3423069049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57db570 msgr2=0x55bad57eb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:19.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e11400700 1 --2- 172.21.15.131:0/3423069049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57db570 0x55bad57eb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e11400700 1 -- 172.21.15.131:0/3423069049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 msgr2=0x55bad57df410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:19.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e11400700 1 --2- 172.21.15.131:0/3423069049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 0x55bad57df410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.441+0000 7f1e11400700 1 -- 172.21.15.131:0/3423069049 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bad56c0560 con 0x55bad57da6a0 2024-05-10T05:45:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.442+0000 7f1e11400700 1 --2- 172.21.15.131:0/3423069049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57df950 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc09aa60 tx=0x7f1dfc096f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4336e8da873c78e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.442+0000 7f1e03e00700 1 -- 172.21.15.131:0/3423069049 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc09b1d0 con 0x55bad57da6a0 2024-05-10T05:45:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.442+0000 7f1e03e00700 1 -- 172.21.15.131:0/3423069049 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dfc004dc0 con 0x55bad57da6a0 2024-05-10T05:45:19.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.442+0000 7f1e03e00700 1 -- 172.21.15.131:0/3423069049 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0b33d0 con 0x55bad57da6a0 2024-05-10T05:45:19.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.444+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/3423069049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 msgr2=0x55bad57df950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.444+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/3423069049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57df950 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc09aa60 tx=0x7f1dfc096f40 comp rx=0 tx=0).stop 2024-05-10T05:45:19.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.444+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/3423069049 shutdown_connections 2024-05-10T05:45:19.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.444+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/3423069049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 0x55bad57df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.444+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/3423069049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57db570 0x55bad57eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.444+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/3423069049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57df950 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.444+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/3423069049 >> 172.21.15.131:0/3423069049 conn(0x55bad57d1450 msgr2=0x55bad57de7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:19.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.444+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/3423069049 shutdown_connections 2024-05-10T05:45:19.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.444+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/3423069049 wait complete. 2024-05-10T05:45:19.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.445+0000 7f1e30dfeb80 1 -- start start 2024-05-10T05:45:19.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.445+0000 7f1e30dfeb80 1 Processor -- start 2024-05-10T05:45:19.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.446+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 0x55bad57e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.446+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57e9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.446+0000 7f1e30dfeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57db570 0x55bad57ea220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.446+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad57ea760 con 0x55bad573ca60 2024-05-10T05:45:19.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.446+0000 7f1e11400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57e9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.446+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad57ea8d0 con 0x55bad57db570 2024-05-10T05:45:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.446+0000 7f1e30dfeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad57eaa90 con 0x55bad57da6a0 2024-05-10T05:45:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.446+0000 7f1e11400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57e9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40510/0 (socket says 172.21.15.131:40510) 2024-05-10T05:45:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.446+0000 7f1e11400700 1 -- 172.21.15.131:0/2334350110 learned_addr learned my addr 172.21.15.131:0/2334350110 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:19.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.447+0000 7f1e11e00700 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 0x55bad57e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.447+0000 7f1e30dfeb80 10 client.0 _ll_register_callbacks cb 0x55bad569fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.447+0000 7f1e12800700 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57db570 0x55bad57ea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.447+0000 7f1e11e00700 1 -- 172.21.15.131:0/2334350110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 msgr2=0x55bad57e9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.447+0000 7f1e11e00700 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57e9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.447+0000 7f1e11e00700 1 -- 172.21.15.131:0/2334350110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57db570 msgr2=0x55bad57ea220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.447+0000 7f1e11e00700 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57db570 0x55bad57ea220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.447+0000 7f1e11e00700 1 -- 172.21.15.131:0/2334350110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bad557b3b0 con 0x55bad573ca60 2024-05-10T05:45:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.448+0000 7f1e11e00700 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 0x55bad57e8e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55bad57f2ed0 tx=0x7f1e040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc0915290089e0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.448+0000 7f1dfbe00700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-10T05:45:19.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.448+0000 7f1dfbe00700 1 -- 172.21.15.131:0/2334350110 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040298c0 con 0x55bad573ca60 2024-05-10T05:45:19.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.448+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/2334350110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dfc09aad0 con 0x55bad573ca60 2024-05-10T05:45:19.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.448+0000 7f1dfbe00700 1 -- 172.21.15.131:0/2334350110 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1e04003800 con 0x55bad573ca60 2024-05-10T05:45:19.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1dfbe00700 1 -- 172.21.15.131:0/2334350110 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e040039c0 con 0x55bad573ca60 2024-05-10T05:45:19.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1e30dfeb80 10 client.17121 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:19.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/2334350110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bad557fcd0 con 0x55bad573ca60 2024-05-10T05:45:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1dfbe00700 1 -- 172.21.15.131:0/2334350110 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dfc09aad0 con 0x55bad573ca60 2024-05-10T05:45:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1dfbe00700 1 -- 172.21.15.131:0/2334350110 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1e04032c80 con 0x55bad573ca60 2024-05-10T05:45:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1e30dfeb80 10 client.17121 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1e30dfeb80 10 client.17121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-05-10T05:45:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1e30dfeb80 20 client.17121 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bad57dbec0 0x55bad5848db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1e30dfeb80 4 client.17121 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:45:19.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.449+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/2334350110 --> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bad56f2ec0 con 0x55bad57dbec0 2024-05-10T05:45:19.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.450+0000 7f1e11400700 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bad57dbec0 0x55bad5848db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.451+0000 7f1e11400700 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bad57dbec0 0x55bad5848db0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=203cc0bfe8325a38 server_cookie=e939fa6715e8d824 in_seq=0 out_seq=0 2024-05-10T05:45:19.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.451+0000 7f1dfbe00700 10 client.17121 ms_handle_connect on v2:172.21.15.131:6840/2396760252 2024-05-10T05:45:19.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.451+0000 7f1dfbe00700 1 -- 172.21.15.131:0/2334350110 <== mds.2 v2:172.21.15.131:6840/2396760252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bad56f2ec0 con 0x55bad57dbec0 2024-05-10T05:45:19.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1dfbe00700 10 client.17121 handle_command_reply: tid=0 2024-05-10T05:45:19.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1e30dfeb80 1 client.17121 shutdown 2024-05-10T05:45:19.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/2334350110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 msgr2=0x55bad57e8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 0x55bad57e8e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55bad57f2ed0 tx=0x7f1e040047c0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/2334350110 shutdown_connections 2024-05-10T05:45:19.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.131:6840/2396760252,v1:172.21.15.131:6841/2396760252] conn(0x55bad57dbec0 0x55bad5848db0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad573ca60 0x55bad57e8e00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bad57db570 0x55bad57ea220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1e30dfeb80 1 --2- 172.21.15.131:0/2334350110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bad57da6a0 0x55bad57e9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/2334350110 >> 172.21.15.131:0/2334350110 conn(0x55bad57d1450 msgr2=0x55bad57df410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:19.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.452+0000 7f1dfbe00700 0 client.17121 ms_handle_reset on v2:172.21.15.131:6840/2396760252 2024-05-10T05:45:19.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.453+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/2334350110 shutdown_connections 2024-05-10T05:45:19.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.453+0000 7f1e30dfeb80 1 -- 172.21.15.131:0/2334350110 wait complete. 2024-05-10T05:45:19.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.453+0000 7f1e30dfeb80 20 client.17121 trim_cache size 0 max 16384 2024-05-10T05:45:19.455 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 -- 172.21.15.131:0/865855234 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1df40a1240 msgr2=0x7f1df40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1df40a1240 0x7f1df40a3750 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1e14068da0 tx=0x7f1e04009260 comp rx=0 tx=0).stop 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 -- 172.21.15.131:0/865855234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e14109a80 msgr2=0x7f1e1419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e14109a80 0x7f1e1419e7b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0095d0 tx=0x7f1dfc00bab0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 -- 172.21.15.131:0/865855234 shutdown_connections 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1df40a1240 0x7f1df40a3750 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e141a3730 0x7f1e141a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e14107040 0x7f1e1419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 --2- 172.21.15.131:0/865855234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e14109a80 0x7f1e1419e7b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 -- 172.21.15.131:0/865855234 >> 172.21.15.131:0/865855234 conn(0x7f1e140fc0d0 msgr2=0x7f1e14107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.454+0000 7f1e18e00700 1 -- 172.21.15.131:0/865855234 shutdown_connections 2024-05-10T05:45:19.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.455+0000 7f1e18e00700 1 -- 172.21.15.131:0/865855234 wait complete. 2024-05-10T05:45:19.513 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T05:45:19.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.954+0000 7f1b4c000700 1 -- 172.21.15.131:0/1038102105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b4410d6e0 msgr2=0x7f1b4410daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.954+0000 7f1b4c000700 1 --2- 172.21.15.131:0/1038102105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b4410d6e0 0x7f1b4410daf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1b380098f0 tx=0x7f1b38009c00 comp rx=0 tx=0).stop 2024-05-10T05:45:19.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.954+0000 7f1b4c000700 1 -- 172.21.15.131:0/1038102105 shutdown_connections 2024-05-10T05:45:19.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.954+0000 7f1b4c000700 1 --2- 172.21.15.131:0/1038102105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4410e080 0x7f1b44114930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.954+0000 7f1b4c000700 1 --2- 172.21.15.131:0/1038102105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b4410c610 0x7f1b4410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.954+0000 7f1b4c000700 1 --2- 172.21.15.131:0/1038102105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b4410d6e0 0x7f1b4410daf0 secure :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1b380098f0 tx=0x7f1b38009c00 comp rx=0 tx=0).stop 2024-05-10T05:45:19.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.954+0000 7f1b4c000700 1 -- 172.21.15.131:0/1038102105 >> 172.21.15.131:0/1038102105 conn(0x7f1b4406c860 msgr2=0x7f1b4410a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:19.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.955+0000 7f1b4c000700 1 -- 172.21.15.131:0/1038102105 shutdown_connections 2024-05-10T05:45:19.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.955+0000 7f1b4c000700 1 -- 172.21.15.131:0/1038102105 wait complete. 2024-05-10T05:45:19.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.957+0000 7f1b4c000700 1 Processor -- start 2024-05-10T05:45:19.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.958+0000 7f1b4c000700 1 -- start start 2024-05-10T05:45:19.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.959+0000 7f1b4c000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b4410c610 0x7f1b440706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.959+0000 7f1b4c000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b4410e080 0x7f1b44070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.959+0000 7f1b4c000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4406d2e0 0x7f1b4406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.959+0000 7f1b4c000700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b441aefd0 con 0x7f1b4410c610 2024-05-10T05:45:19.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.959+0000 7f1b4c000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b441a27a0 con 0x7f1b4406d2e0 2024-05-10T05:45:19.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.959+0000 7f1b4c000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b441a2960 con 0x7f1b4410e080 2024-05-10T05:45:19.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.960+0000 7f1b49600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b4410c610 0x7f1b440706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.961+0000 7f1b49600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b4410c610 0x7f1b440706b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41828/0 (socket says 172.21.15.131:41828) 2024-05-10T05:45:19.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.961+0000 7f1b49600700 1 -- 172.21.15.131:0/3805701823 learned_addr learned my addr 172.21.15.131:0/3805701823 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:19.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.961+0000 7f1b4a000700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4406d2e0 0x7f1b4406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.961+0000 7f1b48c00700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b4410e080 0x7f1b44070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.961+0000 7f1b49600700 1 -- 172.21.15.131:0/3805701823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b4410e080 msgr2=0x7f1b44070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.961+0000 7f1b49600700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b4410e080 0x7f1b44070cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.961+0000 7f1b49600700 1 -- 172.21.15.131:0/3805701823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4406d2e0 msgr2=0x7f1b4406d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.961+0000 7f1b49600700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4406d2e0 0x7f1b4406d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.961+0000 7f1b49600700 1 -- 172.21.15.131:0/3805701823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b38009520 con 0x7f1b4410c610 2024-05-10T05:45:19.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.963+0000 7f1b49600700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b4410c610 0x7f1b440706b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400e940 tx=0x7f1b3400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:19.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.964+0000 7f1b42000700 1 -- 172.21.15.131:0/3805701823 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3400cbd0 con 0x7f1b4410c610 2024-05-10T05:45:19.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.964+0000 7f1b42000700 1 -- 172.21.15.131:0/3805701823 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b340044e0 con 0x7f1b4410c610 2024-05-10T05:45:19.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.964+0000 7f1b42000700 1 -- 172.21.15.131:0/3805701823 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34003d90 con 0x7f1b4410c610 2024-05-10T05:45:19.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.964+0000 7f1b4c000700 1 -- 172.21.15.131:0/3805701823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b441a2c40 con 0x7f1b4410c610 2024-05-10T05:45:19.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.964+0000 7f1b4c000700 1 -- 172.21.15.131:0/3805701823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b441a3100 con 0x7f1b4410c610 2024-05-10T05:45:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.975+0000 7f1b42000700 1 -- 172.21.15.131:0/3805701823 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1b34003f10 con 0x7f1b4410c610 2024-05-10T05:45:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.976+0000 7f1b63f57b80 1 Processor -- start 2024-05-10T05:45:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.976+0000 7f1b63f57b80 1 -- start start 2024-05-10T05:45:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.978+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167aa2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.978+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 0x565167aa3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.978+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167aaa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.978+0000 7f1b63f57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651679c9150 con 0x565167aa26a0 2024-05-10T05:45:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.978+0000 7f1b63f57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651679c6b10 con 0x565167aa3570 2024-05-10T05:45:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.978+0000 7f1b63f57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651679c2400 con 0x565167aa3ec0 2024-05-10T05:45:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.979+0000 7f1b4a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167aa2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.979+0000 7f1b4a000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167aa2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41832/0 (socket says 172.21.15.131:41832) 2024-05-10T05:45:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.979+0000 7f1b4a000700 1 -- 172.21.15.131:0/3896955013 learned_addr learned my addr 172.21.15.131:0/3896955013 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.979+0000 7f1b48c00700 1 --2- 172.21.15.131:0/3896955013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167aaa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.980+0000 7f1b49600700 1 --2- 172.21.15.131:0/3896955013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 0x565167aa3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.981+0000 7f1b4a000700 1 -- 172.21.15.131:0/3896955013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 msgr2=0x565167aaa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.981+0000 7f1b4a000700 1 --2- 172.21.15.131:0/3896955013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167aaa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.981+0000 7f1b4a000700 1 -- 172.21.15.131:0/3896955013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 msgr2=0x565167aa3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.984+0000 7f1b42000700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1b2c0a1330 0x7f1b2c0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.984+0000 7f1b4a000700 1 --2- 172.21.15.131:0/3896955013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 0x565167aa3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.984+0000 7f1b4a000700 1 -- 172.21.15.131:0/3896955013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651678433b0 con 0x565167aa26a0 2024-05-10T05:45:19.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.985+0000 7f1b42000700 1 -- 172.21.15.131:0/3805701823 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1b34014070 con 0x7f1b4410c610 2024-05-10T05:45:19.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.985+0000 7f1b49600700 1 --2- 172.21.15.131:0/3896955013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 0x565167aa3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:45:19.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.986+0000 7f1b4a000700 1 --2- 172.21.15.131:0/3896955013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167aa2af0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009410 tx=0x7f1b3c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ce3e05e15c804 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:19.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.988+0000 7f1b32600700 1 -- 172.21.15.131:0/3896955013 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00e070 con 0x565167aa26a0 2024-05-10T05:45:19.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.988+0000 7f1b32600700 1 -- 172.21.15.131:0/3896955013 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b3c00f040 con 0x565167aa26a0 2024-05-10T05:45:19.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.988+0000 7f1b32600700 1 -- 172.21.15.131:0/3896955013 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c015b00 con 0x565167aa26a0 2024-05-10T05:45:19.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.990+0000 7f1b63f57b80 1 -- 172.21.15.131:0/3896955013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 msgr2=0x565167aa2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:19.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.990+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/3896955013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167aa2af0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009410 tx=0x7f1b3c0099e0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.992+0000 7f1b63f57b80 1 -- 172.21.15.131:0/3896955013 shutdown_connections 2024-05-10T05:45:19.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.992+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/3896955013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167aa2af0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.992+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/3896955013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167aaa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.992+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/3896955013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 0x565167aa3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:19.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.992+0000 7f1b63f57b80 1 -- 172.21.15.131:0/3896955013 >> 172.21.15.131:0/3896955013 conn(0x565167a04a60 msgr2=0x565167a9cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:19.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.992+0000 7f1b48c00700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1b2c0a1330 0x7f1b2c0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.993+0000 7f1b63f57b80 1 -- 172.21.15.131:0/3896955013 shutdown_connections 2024-05-10T05:45:19.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.993+0000 7f1b48c00700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1b2c0a1330 0x7f1b2c0a3840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800fba0 tx=0x7f1b38019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:19.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.994+0000 7f1b63f57b80 1 -- 172.21.15.131:0/3896955013 wait complete. 2024-05-10T05:45:19.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.995+0000 7f1b63f57b80 1 -- start start 2024-05-10T05:45:19.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.995+0000 7f1b63f57b80 1 Processor -- start 2024-05-10T05:45:19.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.996+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167aac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.996+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3570 0x565167aacc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.996+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3ec0 0x565167ab3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:19.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.997+0000 7f1b48c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3570 0x565167aacc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.997+0000 7f1b48c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3570 0x565167aacc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40552/0 (socket says 172.21.15.131:40552) 2024-05-10T05:45:19.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.997+0000 7f1b4a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3ec0 0x565167ab3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:19.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.997+0000 7f1b4a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3ec0 0x565167ab3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35106/0 (socket says 172.21.15.131:35106) 2024-05-10T05:45:19.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.997+0000 7f1b63f57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516795e560 con 0x565167aa26a0 2024-05-10T05:45:19.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.997+0000 7f1b48c00700 1 -- 172.21.15.131:0/4006681263 learned_addr learned my addr 172.21.15.131:0/4006681263 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:19.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.998+0000 7f1b63f57b80 1 -- 172.21.15.131:0/4006681263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516795e200 con 0x565167aa3ec0 2024-05-10T05:45:19.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.998+0000 7f1b63f57b80 1 -- 172.21.15.131:0/4006681263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651678456a0 con 0x565167aa3570 2024-05-10T05:45:19.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.998+0000 7f1b49600700 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167aac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.998+0000 7f1b63f57b80 10 client.0 _ll_register_callbacks cb 0x565167967b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:20.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.999+0000 7f1b4a000700 1 -- 172.21.15.131:0/4006681263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3570 msgr2=0x565167aacc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.999+0000 7f1b4a000700 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3570 0x565167aacc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.999+0000 7f1b4a000700 1 -- 172.21.15.131:0/4006681263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 msgr2=0x565167aac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.999+0000 7f1b4a000700 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167aac720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.999+0000 7f1b4a000700 1 -- 172.21.15.131:0/4006681263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3c0090f0 con 0x565167aa3ec0 2024-05-10T05:45:20.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.999+0000 7f1b4a000700 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3ec0 0x565167ab3bc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c000f80 tx=0x7f1b3c016f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae026c0389fbc331 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:20.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:19.999+0000 7f1b26a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:45:20.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.000+0000 7f1b26a00700 1 -- 172.21.15.131:0/4006681263 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00e040 con 0x565167aa3ec0 2024-05-10T05:45:20.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.000+0000 7f1b26a00700 1 -- 172.21.15.131:0/4006681263 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1b3c013960 con 0x565167aa3ec0 2024-05-10T05:45:20.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.000+0000 7f1b63f57b80 1 -- 172.21.15.131:0/4006681263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56516791b270 con 0x565167aa3ec0 2024-05-10T05:45:20.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b26a00700 1 -- 172.21.15.131:0/4006681263 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b3c013b20 con 0x565167aa3ec0 2024-05-10T05:45:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b63f57b80 10 client.27530 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b63f57b80 1 -- 172.21.15.131:0/4006681263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5651677efb60 con 0x565167aa3ec0 2024-05-10T05:45:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b26a00700 1 -- 172.21.15.131:0/4006681263 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56516791b270 con 0x565167aa3ec0 2024-05-10T05:45:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b26a00700 1 -- 172.21.15.131:0/4006681263 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1b3c01c050 con 0x565167aa3ec0 2024-05-10T05:45:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b63f57b80 10 client.27530 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b63f57b80 10 client.27530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:45:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b63f57b80 20 client.27530 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x565167ab8150 0x565167aba5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b63f57b80 4 client.27530 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:45:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.001+0000 7f1b63f57b80 1 -- 172.21.15.131:0/4006681263 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565167a71780 con 0x565167ab8150 2024-05-10T05:45:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.002+0000 7f1b49600700 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x565167ab8150 0x565167aba5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.006+0000 7f1b49600700 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x565167ab8150 0x565167aba5a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=756752eb4f4e3b3f server_cookie=abe52debf6c1c7bb in_seq=0 out_seq=0 2024-05-10T05:45:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.006+0000 7f1b26a00700 10 client.27530 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:45:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.007+0000 7f1b26a00700 1 -- 172.21.15.131:0/4006681263 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565167a71780 con 0x565167ab8150 2024-05-10T05:45:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.007+0000 7f1b26a00700 10 client.27530 handle_command_reply: tid=0 2024-05-10T05:45:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.007+0000 7f1b63f57b80 1 client.27530 shutdown 2024-05-10T05:45:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.007+0000 7f1b63f57b80 1 -- 172.21.15.131:0/4006681263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3ec0 msgr2=0x565167ab3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.007+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3ec0 0x565167ab3bc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c000f80 tx=0x7f1b3c016f00 comp rx=0 tx=0).stop 2024-05-10T05:45:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.008+0000 7f1b63f57b80 1 -- 172.21.15.131:0/4006681263 shutdown_connections 2024-05-10T05:45:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.008+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x565167ab8150 0x565167aba5a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.008+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167aac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.008+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3ec0 0x565167ab3bc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.008+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/4006681263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3570 0x565167aacc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.008+0000 7f1b63f57b80 1 -- 172.21.15.131:0/4006681263 >> 172.21.15.131:0/4006681263 conn(0x565167a04a60 msgr2=0x565167a9ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.008+0000 7f1b26a00700 0 client.27530 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:45:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.009+0000 7f1b63f57b80 1 -- 172.21.15.131:0/4006681263 shutdown_connections 2024-05-10T05:45:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.009+0000 7f1b63f57b80 1 -- 172.21.15.131:0/4006681263 wait complete. 2024-05-10T05:45:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.010+0000 7f1b63f57b80 20 client.27530 trim_cache size 0 max 16384 2024-05-10T05:45:20.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.038+0000 7f1b63f57b80 1 Processor -- start 2024-05-10T05:45:20.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.038+0000 7f1b63f57b80 1 -- start start 2024-05-10T05:45:20.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.038+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa26a0 0x565167aab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.038+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa3570 0x565167ab1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167ab20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b63f57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167a9e770 con 0x565167aa3570 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b63f57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167abf780 con 0x565167aa26a0 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b63f57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167aa7d60 con 0x565167aa3ec0 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b4a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167ab20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b49600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa26a0 0x565167aab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b49600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa26a0 0x565167aab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35118/0 (socket says 172.21.15.131:35118) 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b49600700 1 -- 172.21.15.131:0/2995958093 learned_addr learned my addr 172.21.15.131:0/2995958093 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b48c00700 1 --2- 172.21.15.131:0/2995958093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa3570 0x565167ab1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b4a000700 1 -- 172.21.15.131:0/2995958093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa26a0 msgr2=0x565167aab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b4a000700 1 --2- 172.21.15.131:0/2995958093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa26a0 0x565167aab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b4a000700 1 -- 172.21.15.131:0/2995958093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa3570 msgr2=0x565167ab1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b4a000700 1 --2- 172.21.15.131:0/2995958093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa3570 0x565167ab1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.039+0000 7f1b4a000700 1 -- 172.21.15.131:0/2995958093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565167988560 con 0x565167aa3ec0 2024-05-10T05:45:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.040+0000 7f1b4a000700 1 --2- 172.21.15.131:0/2995958093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167ab20b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c013fd0 tx=0x7f1b3c01f560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345dd5a3a77324c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:20.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.040+0000 7f1b33000700 1 -- 172.21.15.131:0/2995958093 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c004c90 con 0x565167aa3ec0 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.040+0000 7f1b33000700 1 -- 172.21.15.131:0/2995958093 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b3c01f630 con 0x565167aa3ec0 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.040+0000 7f1b33000700 1 -- 172.21.15.131:0/2995958093 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c01f870 con 0x565167aa3ec0 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.041+0000 7f1b63f57b80 1 -- 172.21.15.131:0/2995958093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 msgr2=0x565167ab20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.041+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/2995958093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167ab20b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c013fd0 tx=0x7f1b3c01f560 comp rx=0 tx=0).stop 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.041+0000 7f1b63f57b80 1 -- 172.21.15.131:0/2995958093 shutdown_connections 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.041+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/2995958093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167ab20b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.041+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/2995958093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa26a0 0x565167aab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.041+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/2995958093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa3570 0x565167ab1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.041+0000 7f1b63f57b80 1 -- 172.21.15.131:0/2995958093 >> 172.21.15.131:0/2995958093 conn(0x565167a04a60 msgr2=0x565167aaaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.041+0000 7f1b63f57b80 1 -- 172.21.15.131:0/2995958093 shutdown_connections 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.042+0000 7f1b63f57b80 1 -- 172.21.15.131:0/2995958093 wait complete. 2024-05-10T05:45:20.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.042+0000 7f1b63f57b80 1 -- start start 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.042+0000 7f1b63f57b80 1 Processor -- start 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.043+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167ab1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.043+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 0x565167ab1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.043+0000 7f1b63f57b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167b0c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.043+0000 7f1b63f57b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167b0cde0 con 0x565167aa26a0 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.043+0000 7f1b63f57b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167b0cf20 con 0x565167aa3570 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.043+0000 7f1b63f57b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565167b0d0e0 con 0x565167aa3ec0 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.043+0000 7f1b63f57b80 10 client.0 _ll_register_callbacks cb 0x565167967b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.043+0000 7f1b4a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167b0c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.043+0000 7f1b4a000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167b0c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40568/0 (socket says 172.21.15.131:40568) 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b4a000700 1 -- 172.21.15.131:0/1214633530 learned_addr learned my addr 172.21.15.131:0/1214633530 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:20.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b48c00700 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 0x565167ab1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b49600700 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167ab1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b4a000700 1 -- 172.21.15.131:0/1214633530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 msgr2=0x565167ab1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b4a000700 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 0x565167ab1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b4a000700 1 -- 172.21.15.131:0/1214633530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 msgr2=0x565167ab1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b4a000700 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167ab1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b4a000700 1 -- 172.21.15.131:0/1214633530 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651677efb60 con 0x565167aa3ec0 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b49600700 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167ab1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b4a000700 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167b0c8a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c014040 tx=0x7f1b3c0328d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45001382fc04843 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b26a00700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.044+0000 7f1b26a00700 1 -- 172.21.15.131:0/1214633530 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00e070 con 0x565167aa3ec0 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b26a00700 1 -- 172.21.15.131:0/1214633530 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1b3c032c20 con 0x565167aa3ec0 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b63f57b80 1 -- 172.21.15.131:0/1214633530 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b3c00f040 con 0x565167aa3ec0 2024-05-10T05:45:20.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b26a00700 1 -- 172.21.15.131:0/1214633530 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b3c032e20 con 0x565167aa3ec0 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b63f57b80 10 client.27547 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b63f57b80 1 -- 172.21.15.131:0/1214633530 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x565167847cd0 con 0x565167aa3ec0 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b26a00700 1 -- 172.21.15.131:0/1214633530 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b3c00f040 con 0x565167aa3ec0 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b26a00700 1 -- 172.21.15.131:0/1214633530 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1b3c03a070 con 0x565167aa3ec0 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b63f57b80 10 client.27547 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b63f57b80 10 client.27547 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b63f57b80 20 client.27547 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x565167b0e5d0 0x565167b109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b63f57b80 4 client.27547 mds_command: new command op to 24571 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.045+0000 7f1b63f57b80 1 -- 172.21.15.131:0/1214633530 --> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5651679baec0 con 0x565167b0e5d0 2024-05-10T05:45:20.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.046+0000 7f1b49600700 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x565167b0e5d0 0x565167b109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.046+0000 7f1b49600700 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x565167b0e5d0 0x565167b109f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48a059950c32752a server_cookie=84bc686fc7e9926d in_seq=0 out_seq=0 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.046+0000 7f1b26a00700 10 client.27547 ms_handle_connect on v2:172.21.15.131:6838/2980869223 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b26a00700 1 -- 172.21.15.131:0/1214633530 <== mds.1 v2:172.21.15.131:6838/2980869223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5651679baec0 con 0x565167b0e5d0 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b26a00700 10 client.27547 handle_command_reply: tid=0 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 client.27547 shutdown 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 -- 172.21.15.131:0/1214633530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 msgr2=0x565167b0c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167b0c8a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c014040 tx=0x7f1b3c0328d0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 -- 172.21.15.131:0/1214633530 shutdown_connections 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.131:6838/2980869223,v1:172.21.15.131:6839/2980869223] conn(0x565167b0e5d0 0x565167b109f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565167aa26a0 0x565167ab1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b26a00700 0 client.27547 ms_handle_reset on v2:172.21.15.131:6838/2980869223 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565167aa3ec0 0x565167b0c8a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 --2- 172.21.15.131:0/1214633530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565167aa3570 0x565167ab1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 -- 172.21.15.131:0/1214633530 >> 172.21.15.131:0/1214633530 conn(0x565167a04a60 msgr2=0x565167aa12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:20.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 -- 172.21.15.131:0/1214633530 shutdown_connections 2024-05-10T05:45:20.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.047+0000 7f1b63f57b80 1 -- 172.21.15.131:0/1214633530 wait complete. 2024-05-10T05:45:20.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.048+0000 7f1b63f57b80 20 client.27547 trim_cache size 0 max 16384 2024-05-10T05:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:45:20.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.048+0000 7f1b4c000700 1 -- 172.21.15.131:0/3805701823 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1b2c0a1330 msgr2=0x7f1b2c0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.048+0000 7f1b4c000700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1b2c0a1330 0x7f1b2c0a3840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800fba0 tx=0x7f1b38019040 comp rx=0 tx=0).stop 2024-05-10T05:45:20.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.049+0000 7f1b4c000700 1 -- 172.21.15.131:0/3805701823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b4410c610 msgr2=0x7f1b440706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.049+0000 7f1b4c000700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b4410c610 0x7f1b440706b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400e940 tx=0x7f1b3400ed00 comp rx=0 tx=0).stop 2024-05-10T05:45:20.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.049+0000 7f1b4c000700 1 -- 172.21.15.131:0/3805701823 shutdown_connections 2024-05-10T05:45:20.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.049+0000 7f1b4c000700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1b2c0a1330 0x7f1b2c0a3840 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.049+0000 7f1b4c000700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b4410c610 0x7f1b440706b0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.049+0000 7f1b4c000700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4406d2e0 0x7f1b4406d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.049+0000 7f1b4c000700 1 --2- 172.21.15.131:0/3805701823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b4410e080 0x7f1b44070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.049+0000 7f1b4c000700 1 -- 172.21.15.131:0/3805701823 >> 172.21.15.131:0/3805701823 conn(0x7f1b4406c860 msgr2=0x7f1b44110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:20.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.049+0000 7f1b4c000700 1 -- 172.21.15.131:0/3805701823 shutdown_connections 2024-05-10T05:45:20.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.049+0000 7f1b4c000700 1 -- 172.21.15.131:0/3805701823 wait complete. 2024-05-10T05:45:20.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T05:45:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.587+0000 7ff7bf800700 1 -- 172.21.15.131:0/42882293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b8101be0 msgr2=0x7ff7b8101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.587+0000 7ff7bf800700 1 --2- 172.21.15.131:0/42882293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b8101be0 0x7ff7b8101ff0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009990 tx=0x7ff7a4009ca0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.587+0000 7ff7bf800700 1 -- 172.21.15.131:0/42882293 shutdown_connections 2024-05-10T05:45:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.587+0000 7ff7bf800700 1 --2- 172.21.15.131:0/42882293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7b8102580 0x7ff7b8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.587+0000 7ff7bf800700 1 --2- 172.21.15.131:0/42882293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b8100ac0 0x7ff7b8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.588+0000 7ff7bf800700 1 --2- 172.21.15.131:0/42882293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b8101be0 0x7ff7b8101ff0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.588+0000 7ff7bf800700 1 -- 172.21.15.131:0/42882293 >> 172.21.15.131:0/42882293 conn(0x7ff7b80fc0d0 msgr2=0x7ff7b80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.588+0000 7ff7bf800700 1 -- 172.21.15.131:0/42882293 shutdown_connections 2024-05-10T05:45:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.589+0000 7ff7bf800700 1 -- 172.21.15.131:0/42882293 wait complete. 2024-05-10T05:45:20.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.591+0000 7ff7bf800700 1 Processor -- start 2024-05-10T05:45:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.592+0000 7ff7bf800700 1 -- start start 2024-05-10T05:45:20.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.593+0000 7ff7bf800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7b8100ac0 0x7ff7b8195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.593+0000 7ff7bf800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b8102580 0x7ff7b8196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.593+0000 7ff7bf800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b81ac490 0x7ff7b81ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.593+0000 7ff7bf800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81aedf0 con 0x7ff7b8102580 2024-05-10T05:45:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.593+0000 7ff7bf800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81aef60 con 0x7ff7b81ac490 2024-05-10T05:45:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.593+0000 7ff7bf800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81af120 con 0x7ff7b8100ac0 2024-05-10T05:45:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.594+0000 7ff7bce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7b8100ac0 0x7ff7b8195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.594+0000 7ff7bce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7b8100ac0 0x7ff7b8195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40588/0 (socket says 172.21.15.131:40588) 2024-05-10T05:45:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.594+0000 7ff7bce00700 1 -- 172.21.15.131:0/2663421558 learned_addr learned my addr 172.21.15.131:0/2663421558 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.594+0000 7ff7bd800700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b81ac490 0x7ff7b81ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.594+0000 7ff7afe00700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b8102580 0x7ff7b8196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.595+0000 7ff7bce00700 1 -- 172.21.15.131:0/2663421558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b81ac490 msgr2=0x7ff7b81ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.595+0000 7ff7bce00700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b81ac490 0x7ff7b81ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.595+0000 7ff7bce00700 1 -- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b8102580 msgr2=0x7ff7b8196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.596+0000 7ff7bce00700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b8102580 0x7ff7b8196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.596+0000 7ff7bce00700 1 -- 172.21.15.131:0/2663421558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a40095f0 con 0x7ff7b8100ac0 2024-05-10T05:45:20.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.596+0000 7ff7afe00700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b8102580 0x7ff7b8196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:45:20.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.596+0000 7ff7bce00700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7b8100ac0 0x7ff7b8195e30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800e940 tx=0x7ff7a800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:20.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.597+0000 7ff7b6000700 1 -- 172.21.15.131:0/2663421558 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a800cbd0 con 0x7ff7b8100ac0 2024-05-10T05:45:20.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.597+0000 7ff7b6000700 1 -- 172.21.15.131:0/2663421558 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7a800ee90 con 0x7ff7b8100ac0 2024-05-10T05:45:20.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.597+0000 7ff7b6000700 1 -- 172.21.15.131:0/2663421558 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8003cb0 con 0x7ff7b8100ac0 2024-05-10T05:45:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.600+0000 7ff7bf800700 1 -- 172.21.15.131:0/2663421558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b81af400 con 0x7ff7b8100ac0 2024-05-10T05:45:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.601+0000 7ff7bf800700 1 -- 172.21.15.131:0/2663421558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b81af890 con 0x7ff7b8100ac0 2024-05-10T05:45:20.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.611+0000 7ff7b6000700 1 -- 172.21.15.131:0/2663421558 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff7a800cd30 con 0x7ff7b8100ac0 2024-05-10T05:45:20.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.611+0000 7ff7d780fb80 1 Processor -- start 2024-05-10T05:45:20.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.613+0000 7ff7b6000700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff79c0a1240 0x7ff79c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.613+0000 7ff7b6000700 1 -- 172.21.15.131:0/2663421558 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff7a8014070 con 0x7ff7b8100ac0 2024-05-10T05:45:20.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.615+0000 7ff7d780fb80 1 -- start start 2024-05-10T05:45:20.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.615+0000 7ff7bd800700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff79c0a1240 0x7ff79c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.615+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045286a0 0x563404528af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.615+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 0x563404529980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.615+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x5634045307c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.615+0000 7ff7d780fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340444f150 con 0x563404529570 2024-05-10T05:45:20.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.615+0000 7ff7d780fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340444cb10 con 0x5634045286a0 2024-05-10T05:45:20.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.615+0000 7ff7d780fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563404448400 con 0x563404529ec0 2024-05-10T05:45:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.616+0000 7ff7bce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045286a0 0x563404528af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.616+0000 7ff7bce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045286a0 0x563404528af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35136/0 (socket says 172.21.15.131:35136) 2024-05-10T05:45:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.616+0000 7ff7bce00700 1 -- 172.21.15.131:0/3257998869 learned_addr learned my addr 172.21.15.131:0/3257998869 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.616+0000 7ff7afe00700 1 --2- 172.21.15.131:0/3257998869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 0x563404529980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.617+0000 7ff7bce00700 1 -- 172.21.15.131:0/3257998869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 msgr2=0x5634045307c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.617+0000 7ff7bd800700 1 --2- 172.21.15.131:0/3257998869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x5634045307c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.617+0000 7ff7bce00700 1 --2- 172.21.15.131:0/3257998869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x5634045307c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.617+0000 7ff7bce00700 1 -- 172.21.15.131:0/3257998869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 msgr2=0x563404529980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.617+0000 7ff7bce00700 1 --2- 172.21.15.131:0/3257998869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 0x563404529980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.617+0000 7ff7bce00700 1 -- 172.21.15.131:0/3257998869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634042c93b0 con 0x5634045286a0 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.617+0000 7ff7bd800700 1 --2- 172.21.15.131:0/3257998869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x5634045307c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.617+0000 7ff7bd800700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff79c0a1240 0x7ff79c0a3750 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009ec0 tx=0x7ff7b0009920 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.617+0000 7ff7bce00700 1 --2- 172.21.15.131:0/3257998869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045286a0 0x563404528af0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80a1f10 tx=0x7ff7a8099b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea584c5b6c9ca777 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.618+0000 7ff7ae000700 1 -- 172.21.15.131:0/3257998869 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80b1070 con 0x5634045286a0 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.618+0000 7ff7ae000700 1 -- 172.21.15.131:0/3257998869 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7a80ade90 con 0x5634045286a0 2024-05-10T05:45:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.618+0000 7ff7ae000700 1 -- 172.21.15.131:0/3257998869 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8098360 con 0x5634045286a0 2024-05-10T05:45:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.619+0000 7ff7d780fb80 1 -- 172.21.15.131:0/3257998869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045286a0 msgr2=0x563404528af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.620+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/3257998869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045286a0 0x563404528af0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80a1f10 tx=0x7ff7a8099b00 comp rx=0 tx=0).stop 2024-05-10T05:45:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.620+0000 7ff7d780fb80 1 -- 172.21.15.131:0/3257998869 shutdown_connections 2024-05-10T05:45:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.620+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/3257998869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x5634045307c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.620+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/3257998869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045286a0 0x563404528af0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.620+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/3257998869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 0x563404529980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.620+0000 7ff7d780fb80 1 -- 172.21.15.131:0/3257998869 >> 172.21.15.131:0/3257998869 conn(0x56340448aa60 msgr2=0x563404522fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.621+0000 7ff7d780fb80 1 -- 172.21.15.131:0/3257998869 shutdown_connections 2024-05-10T05:45:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.621+0000 7ff7d780fb80 1 -- 172.21.15.131:0/3257998869 wait complete. 2024-05-10T05:45:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.621+0000 7ff7d780fb80 1 -- start start 2024-05-10T05:45:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.622+0000 7ff7d780fb80 1 Processor -- start 2024-05-10T05:45:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.623+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 0x5634045324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.623+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529570 0x563404532a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.623+0000 7ff7bce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 0x5634045324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.623+0000 7ff7bce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 0x5634045324d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40600/0 (socket says 172.21.15.131:40600) 2024-05-10T05:45:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.623+0000 7ff7bce00700 1 -- 172.21.15.131:0/117112991 learned_addr learned my addr 172.21.15.131:0/117112991 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.624+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529ec0 0x563404539970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.624+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634043e4560 con 0x563404529ec0 2024-05-10T05:45:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.624+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634043e4200 con 0x563404529570 2024-05-10T05:45:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.624+0000 7ff7afe00700 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529570 0x563404532a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.624+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634042cb6a0 con 0x5634045286a0 2024-05-10T05:45:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.625+0000 7ff7bce00700 1 -- 172.21.15.131:0/117112991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529570 msgr2=0x563404532a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.625+0000 7ff7bce00700 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529570 0x563404532a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.625+0000 7ff7bce00700 1 -- 172.21.15.131:0/117112991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529ec0 msgr2=0x563404539970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.625+0000 7ff7bce00700 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529ec0 0x563404539970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.625+0000 7ff7bce00700 1 -- 172.21.15.131:0/117112991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b81af890 con 0x5634045286a0 2024-05-10T05:45:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.625+0000 7ff7d780fb80 10 client.0 _ll_register_callbacks cb 0x5634043edb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.625+0000 7ff7bce00700 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 0x5634045324d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff7a809afd0 tx=0x7ff7a801ff50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2626d8d2e82f506d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.625+0000 7ff7a2a00700 10 client.27556 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.625+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634043a1270 con 0x5634045286a0 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.626+0000 7ff7a2a00700 1 -- 172.21.15.131:0/117112991 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80b1030 con 0x5634045286a0 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.626+0000 7ff7a2a00700 1 -- 172.21.15.131:0/117112991 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff7a807b020 con 0x5634045286a0 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.626+0000 7ff7a2a00700 1 -- 172.21.15.131:0/117112991 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7a809a6c0 con 0x5634045286a0 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.626+0000 7ff7d780fb80 10 client.27556 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.626+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x563404275b60 con 0x5634045286a0 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.626+0000 7ff7a2a00700 1 -- 172.21.15.131:0/117112991 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff7a80b8180 con 0x5634045286a0 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.626+0000 7ff7d780fb80 10 client.27556 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.627+0000 7ff7d780fb80 10 client.27556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.627+0000 7ff7d780fb80 20 client.27556 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.627+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56340453da40 0x56340453fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.627+0000 7ff7d780fb80 4 client.27556 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T05:45:20.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.627+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634044f7780 con 0x56340453da40 2024-05-10T05:45:20.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.627+0000 7ff7afe00700 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56340453da40 0x56340453fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.628+0000 7ff7a2a00700 10 client.27556 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:45:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.628+0000 7ff7afe00700 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56340453da40 0x56340453fe90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01d1fd56fb8881e server_cookie=39d73261d700058 in_seq=0 out_seq=0 2024-05-10T05:45:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.628+0000 7ff7a2a00700 1 -- 172.21.15.131:0/117112991 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634044f7780 con 0x56340453da40 2024-05-10T05:45:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.628+0000 7ff7a2a00700 10 client.27556 handle_command_reply: tid=0 2024-05-10T05:45:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.628+0000 7ff7d780fb80 1 client.27556 shutdown 2024-05-10T05:45:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 msgr2=0x5634045324d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 0x5634045324d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff7a809afd0 tx=0x7ff7a801ff50 comp rx=0 tx=0).stop 2024-05-10T05:45:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 shutdown_connections 2024-05-10T05:45:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x56340453da40 0x56340453fe90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529ec0 0x563404539970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 0x5634045324d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/117112991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529570 0x563404532a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 >> 172.21.15.131:0/117112991 conn(0x56340448aa60 msgr2=0x563404522fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7a2a00700 0 client.27556 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:45:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 shutdown_connections 2024-05-10T05:45:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 1 -- 172.21.15.131:0/117112991 wait complete. 2024-05-10T05:45:20.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.629+0000 7ff7d780fb80 20 client.27556 trim_cache size 0 max 16384 2024-05-10T05:45:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.669+0000 7ff7d780fb80 1 Processor -- start 2024-05-10T05:45:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.669+0000 7ff7d780fb80 1 -- start start 2024-05-10T05:45:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.669+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 0x56340451f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.669+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 0x563404539930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.669+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529ec0 0x563404523b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.670+0000 7ff7d780fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340451f700 con 0x563404529570 2024-05-10T05:45:20.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.670+0000 7ff7d780fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340453aa60 con 0x563404529ec0 2024-05-10T05:45:20.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.670+0000 7ff7d780fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340452a440 con 0x5634045286a0 2024-05-10T05:45:20.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.670+0000 7ff7afe00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 0x563404539930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.670+0000 7ff7bd800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529ec0 0x563404523b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.670+0000 7ff7bd800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529ec0 0x563404523b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:35148/0 (socket says 172.21.15.131:35148) 2024-05-10T05:45:20.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.670+0000 7ff7bd800700 1 -- 172.21.15.131:0/3836626389 learned_addr learned my addr 172.21.15.131:0/3836626389 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:20.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.671+0000 7ff7bce00700 1 --2- 172.21.15.131:0/3836626389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 0x56340451f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.672+0000 7ff7afe00700 1 -- 172.21.15.131:0/3836626389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 msgr2=0x56340451f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.672+0000 7ff7afe00700 1 --2- 172.21.15.131:0/3836626389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 0x56340451f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.672+0000 7ff7afe00700 1 -- 172.21.15.131:0/3836626389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529ec0 msgr2=0x563404523b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.672+0000 7ff7afe00700 1 --2- 172.21.15.131:0/3836626389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529ec0 0x563404523b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.672+0000 7ff7afe00700 1 -- 172.21.15.131:0/3836626389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56340440e560 con 0x563404529570 2024-05-10T05:45:20.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.673+0000 7ff7afe00700 1 --2- 172.21.15.131:0/3836626389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 0x563404539930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4008000 tx=0x7ff7a4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250d2f01f4de45e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:20.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.673+0000 7ff7aea00700 1 -- 172.21.15.131:0/3836626389 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400bb70 con 0x563404529570 2024-05-10T05:45:20.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.673+0000 7ff7aea00700 1 -- 172.21.15.131:0/3836626389 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7a401e040 con 0x563404529570 2024-05-10T05:45:20.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.674+0000 7ff7aea00700 1 -- 172.21.15.131:0/3836626389 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40262d0 con 0x563404529570 2024-05-10T05:45:20.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.677+0000 7ff7d780fb80 1 -- 172.21.15.131:0/3836626389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 msgr2=0x563404539930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.677+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/3836626389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 0x563404539930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4008000 tx=0x7ff7a4009310 comp rx=0 tx=0).stop 2024-05-10T05:45:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.677+0000 7ff7d780fb80 1 -- 172.21.15.131:0/3836626389 shutdown_connections 2024-05-10T05:45:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.677+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/3836626389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563404529ec0 0x563404523b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.677+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/3836626389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634045286a0 0x56340451f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.677+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/3836626389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563404529570 0x563404539930 secure :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4008000 tx=0x7ff7a4009310 comp rx=0 tx=0).stop 2024-05-10T05:45:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.677+0000 7ff7d780fb80 1 -- 172.21.15.131:0/3836626389 >> 172.21.15.131:0/3836626389 conn(0x56340448aa60 msgr2=0x563404522f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:20.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.679+0000 7ff7d780fb80 1 -- 172.21.15.131:0/3836626389 shutdown_connections 2024-05-10T05:45:20.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.679+0000 7ff7d780fb80 1 -- 172.21.15.131:0/3836626389 wait complete. 2024-05-10T05:45:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.680+0000 7ff7d780fb80 1 -- start start 2024-05-10T05:45:20.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.680+0000 7ff7d780fb80 1 Processor -- start 2024-05-10T05:45:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.681+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634045286a0 0x563404537450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.681+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x563404537990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.681+0000 7ff7d780fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045388c0 0x563404538cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.682+0000 7ff7afe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x563404537990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.682+0000 7ff7afe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x563404537990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:40618/0 (socket says 172.21.15.131:40618) 2024-05-10T05:45:20.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.682+0000 7ff7bce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634045286a0 0x563404537450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.682+0000 7ff7bce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634045286a0 0x563404537450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41916/0 (socket says 172.21.15.131:41916) 2024-05-10T05:45:20.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.682+0000 7ff7bd800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045388c0 0x563404538cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.682+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x563404539210 con 0x5634045286a0 2024-05-10T05:45:20.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.682+0000 7ff7afe00700 1 -- 172.21.15.131:0/255738584 learned_addr learned my addr 172.21.15.131:0/255738584 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:20.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.682+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563404539380 con 0x5634045388c0 2024-05-10T05:45:20.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.682+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563404539540 con 0x563404529ec0 2024-05-10T05:45:20.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.683+0000 7ff7d780fb80 10 client.0 _ll_register_callbacks cb 0x5634043edb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T05:45:20.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.683+0000 7ff7bd800700 1 -- 172.21.15.131:0/255738584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 msgr2=0x563404537990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.683+0000 7ff7bd800700 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x563404537990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.683+0000 7ff7bd800700 1 -- 172.21.15.131:0/255738584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634045286a0 msgr2=0x563404537450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T05:45:20.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.683+0000 7ff7bd800700 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634045286a0 0x563404537450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.683+0000 7ff7bd800700 1 -- 172.21.15.131:0/255738584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b80683c0 con 0x5634045388c0 2024-05-10T05:45:20.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.683+0000 7ff7afe00700 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x563404537990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:45:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.683+0000 7ff7bce00700 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634045286a0 0x563404537450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.684+0000 7ff7bd800700 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045388c0 0x563404538cd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009fd0 tx=0x7ff7b0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59b95ec34b14d1a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.684+0000 7ff7a2a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T05:45:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.684+0000 7ff7a2a00700 1 -- 172.21.15.131:0/255738584 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000e040 con 0x5634045388c0 2024-05-10T05:45:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.684+0000 7ff7a2a00700 1 -- 172.21.15.131:0/255738584 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff7b0003c10 con 0x5634045388c0 2024-05-10T05:45:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.684+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7a80ade90 con 0x5634045388c0 2024-05-10T05:45:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.685+0000 7ff7a2a00700 1 -- 172.21.15.131:0/255738584 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b0003dd0 con 0x5634045388c0 2024-05-10T05:45:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.685+0000 7ff7d780fb80 10 client.27554 fetch_fsmap learned FSMap version 30 2024-05-10T05:45:20.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.685+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5634042cdcd0 con 0x5634045388c0 2024-05-10T05:45:20.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.685+0000 7ff7a2a00700 1 -- 172.21.15.131:0/255738584 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7a80ade90 con 0x5634045388c0 2024-05-10T05:45:20.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.685+0000 7ff7a2a00700 1 -- 172.21.15.131:0/255738584 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff7b0004590 con 0x5634045388c0 2024-05-10T05:45:20.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.686+0000 7ff7d780fb80 10 client.27554 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T05:45:20.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.686+0000 7ff7d780fb80 10 client.27554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T05:45:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.686+0000 7ff7d780fb80 20 client.27554 populate_metadata read hostname 'smithi131' 2024-05-10T05:45:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.686+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x563404594f50 0x563404597370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.686+0000 7ff7d780fb80 4 client.27554 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T05:45:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.686+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 --> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563404440ec0 con 0x563404594f50 2024-05-10T05:45:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.687+0000 7ff7bce00700 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x563404594f50 0x563404597370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:20.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.689+0000 7ff7a2a00700 10 client.27554 ms_handle_connect on v2:172.21.15.142:6838/2348955278 2024-05-10T05:45:20.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.689+0000 7ff7bce00700 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x563404594f50 0x563404597370 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5356540543f4b7e server_cookie=a9908ba8fb4294f9 in_seq=0 out_seq=0 2024-05-10T05:45:20.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.689+0000 7ff7a2a00700 1 -- 172.21.15.131:0/255738584 <== mds.0 v2:172.21.15.142:6838/2348955278 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563404440ec0 con 0x563404594f50 2024-05-10T05:45:20.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.689+0000 7ff7a2a00700 10 client.27554 handle_command_reply: tid=0 2024-05-10T05:45:20.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.689+0000 7ff7d780fb80 1 client.27554 shutdown 2024-05-10T05:45:20.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.691+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045388c0 msgr2=0x563404538cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.691+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045388c0 0x563404538cd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009fd0 tx=0x7ff7b0004ad0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.692+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 shutdown_connections 2024-05-10T05:45:20.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.692+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.142:6838/2348955278,v1:172.21.15.142:6839/2348955278] conn(0x563404594f50 0x563404597370 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.692+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634045286a0 0x563404537450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.692+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634045388c0 0x563404538cd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.692+0000 7ff7d780fb80 1 --2- 172.21.15.131:0/255738584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563404529ec0 0x563404537990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.692+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 >> 172.21.15.131:0/255738584 conn(0x56340448aa60 msgr2=0x56340452d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:20.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.692+0000 7ff7a2a00700 0 client.27554 ms_handle_reset on v2:172.21.15.142:6838/2348955278 2024-05-10T05:45:20.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.692+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 shutdown_connections 2024-05-10T05:45:20.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.692+0000 7ff7d780fb80 1 -- 172.21.15.131:0/255738584 wait complete. 2024-05-10T05:45:20.701 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-05-10T05:45:20.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.693+0000 7ff7d780fb80 20 client.27554 trim_cache size 0 max 16384 2024-05-10T05:45:20.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.693+0000 7ff7bf800700 1 -- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff79c0a1240 msgr2=0x7ff79c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.693+0000 7ff7bf800700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff79c0a1240 0x7ff79c0a3750 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009ec0 tx=0x7ff7b0009920 comp rx=0 tx=0).stop 2024-05-10T05:45:20.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.694+0000 7ff7bf800700 1 -- 172.21.15.131:0/2663421558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7b8100ac0 msgr2=0x7ff7b8195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:20.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.694+0000 7ff7bf800700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7b8100ac0 0x7ff7b8195e30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800e940 tx=0x7ff7a800ec50 comp rx=0 tx=0).stop 2024-05-10T05:45:20.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.694+0000 7ff7bf800700 1 -- 172.21.15.131:0/2663421558 shutdown_connections 2024-05-10T05:45:20.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.694+0000 7ff7bf800700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff79c0a1240 0x7ff79c0a3750 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.694+0000 7ff7bf800700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b81ac490 0x7ff7b81ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.694+0000 7ff7bf800700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7b8100ac0 0x7ff7b8195e30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.694+0000 7ff7bf800700 1 --2- 172.21.15.131:0/2663421558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b8102580 0x7ff7b8196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.694+0000 7ff7bf800700 1 -- 172.21.15.131:0/2663421558 >> 172.21.15.131:0/2663421558 conn(0x7ff7b80fc0d0 msgr2=0x7ff7b8102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.695+0000 7ff7bf800700 1 -- 172.21.15.131:0/2663421558 shutdown_connections 2024-05-10T05:45:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:20.695+0000 7ff7bf800700 1 -- 172.21.15.131:0/2663421558 wait complete. 2024-05-10T05:45:21.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:20 smithi157 ceph-mon[78427]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 384 MiB/s wr, 131 op/s 2024-05-10T05:45:21.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:20 smithi142 ceph-mon[77667]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 384 MiB/s wr, 131 op/s 2024-05-10T05:45:21.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:20 smithi131 ceph-mon[37264]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 384 MiB/s wr, 131 op/s 2024-05-10T05:45:21.729 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-10T05:45:21.729 INFO:tasks.fwd_scrub:done joining 2024-05-10T05:45:21.729 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-10T05:45:21.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:45:22.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.268+0000 7ff7fe400700 1 -- 172.21.15.131:0/1546005333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f81019f0 msgr2=0x7ff7f8101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:22.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.268+0000 7ff7fe400700 1 --2- 172.21.15.131:0/1546005333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f81019f0 0x7ff7f8101e00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0099c0 tx=0x7ff7ec009cd0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.268+0000 7ff7fe400700 1 -- 172.21.15.131:0/1546005333 shutdown_connections 2024-05-10T05:45:22.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.268+0000 7ff7fe400700 1 --2- 172.21.15.131:0/1546005333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81034b0 0x7ff7f8109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.268+0000 7ff7fe400700 1 --2- 172.21.15.131:0/1546005333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f81019f0 0x7ff7f8101e00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.268+0000 7ff7fe400700 1 --2- 172.21.15.131:0/1546005333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f8102b10 0x7ff7f8102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.268+0000 7ff7fe400700 1 -- 172.21.15.131:0/1546005333 >> 172.21.15.131:0/1546005333 conn(0x7ff7f80fcfe0 msgr2=0x7ff7f80ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:22.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.268+0000 7ff7fe400700 1 -- 172.21.15.131:0/1546005333 shutdown_connections 2024-05-10T05:45:22.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.268+0000 7ff7fe400700 1 -- 172.21.15.131:0/1546005333 wait complete. 2024-05-10T05:45:22.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.270+0000 7ff7fe400700 1 Processor -- start 2024-05-10T05:45:22.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.270+0000 7ff7fe400700 1 -- start start 2024-05-10T05:45:22.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.271+0000 7ff7fe400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81019f0 0x7ff7f8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:22.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.271+0000 7ff7fe400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f8102b10 0x7ff7f8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:22.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.271+0000 7ff7fe400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f81034b0 0x7ff7f819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:22.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.271+0000 7ff7fe400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f819f7b0 con 0x7ff7f81019f0 2024-05-10T05:45:22.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.271+0000 7ff7fe400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f819f8f0 con 0x7ff7f8102b10 2024-05-10T05:45:22.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.271+0000 7ff7fe400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f819fab0 con 0x7ff7f81034b0 2024-05-10T05:45:22.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7fd000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81019f0 0x7ff7f8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:22.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f8102b10 0x7ff7f8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:22.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7fd000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81019f0 0x7ff7f8197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41926/0 (socket says 172.21.15.131:41926) 2024-05-10T05:45:22.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7fd000700 1 -- 172.21.15.131:0/430895642 learned_addr learned my addr 172.21.15.131:0/430895642 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:22.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7fda00700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f81034b0 0x7ff7f819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:22.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7fda00700 1 -- 172.21.15.131:0/430895642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f8102b10 msgr2=0x7ff7f8198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:22.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7fda00700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f8102b10 0x7ff7f8198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7fda00700 1 -- 172.21.15.131:0/430895642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81019f0 msgr2=0x7ff7f8197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:22.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7fda00700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81019f0 0x7ff7f8197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7fda00700 1 -- 172.21.15.131:0/430895642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec0095f0 con 0x7ff7f81034b0 2024-05-10T05:45:22.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.272+0000 7ff7fd000700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81019f0 0x7ff7f8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:22.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.273+0000 7ff7fda00700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f81034b0 0x7ff7f819f1e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0007ec0 tx=0x7ff7f000f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:22.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.273+0000 7ff7f5600700 1 -- 172.21.15.131:0/430895642 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0014070 con 0x7ff7f81034b0 2024-05-10T05:45:22.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.273+0000 7ff7fe400700 1 -- 172.21.15.131:0/430895642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f80734a0 con 0x7ff7f81034b0 2024-05-10T05:45:22.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.273+0000 7ff7f5600700 1 -- 172.21.15.131:0/430895642 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f0009390 con 0x7ff7f81034b0 2024-05-10T05:45:22.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.273+0000 7ff7f5600700 1 -- 172.21.15.131:0/430895642 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0004260 con 0x7ff7f81034b0 2024-05-10T05:45:22.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.273+0000 7ff7fe400700 1 -- 172.21.15.131:0/430895642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f8073960 con 0x7ff7f81034b0 2024-05-10T05:45:22.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.275+0000 7ff7fe400700 1 -- 172.21.15.131:0/430895642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f8108c00 con 0x7ff7f81034b0 2024-05-10T05:45:22.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.276+0000 7ff7f5600700 1 -- 172.21.15.131:0/430895642 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff7f0018070 con 0x7ff7f81034b0 2024-05-10T05:45:22.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.278+0000 7ff7f5600700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7e00a11c0 0x7ff7e00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:22.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.278+0000 7ff7f5600700 1 -- 172.21.15.131:0/430895642 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7ff7f00cfe40 con 0x7ff7f81034b0 2024-05-10T05:45:22.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.279+0000 7ff7fd000700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7e00a11c0 0x7ff7e00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:22.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.279+0000 7ff7fd000700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7e00a11c0 0x7ff7e00a36d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec000c00 tx=0x7ff7ec005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.283+0000 7ff7f5600700 1 -- 172.21.15.131:0/430895642 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff7f0099570 con 0x7ff7f81034b0 2024-05-10T05:45:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.716+0000 7ff7fe400700 1 -- 172.21.15.131:0/430895642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7f81085e0 con 0x7ff7f81034b0 2024-05-10T05:45:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.720+0000 7ff7f5600700 1 -- 172.21.15.131:0/430895642 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7ff7f001e090 con 0x7ff7f81034b0 2024-05-10T05:45:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:45:22.724 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:45:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.724+0000 7ff7fe400700 1 -- 172.21.15.131:0/430895642 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7e00a11c0 msgr2=0x7ff7e00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.724+0000 7ff7fe400700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7e00a11c0 0x7ff7e00a36d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec000c00 tx=0x7ff7ec005fb0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.724+0000 7ff7fe400700 1 -- 172.21.15.131:0/430895642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f81034b0 msgr2=0x7ff7f819f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.724+0000 7ff7fe400700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f81034b0 0x7ff7f819f1e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0007ec0 tx=0x7ff7f000f740 comp rx=0 tx=0).stop 2024-05-10T05:45:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.725+0000 7ff7fe400700 1 -- 172.21.15.131:0/430895642 shutdown_connections 2024-05-10T05:45:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.725+0000 7ff7fe400700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7ff7e00a11c0 0x7ff7e00a36d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.725+0000 7ff7fe400700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7f81019f0 0x7ff7f8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.725+0000 7ff7fe400700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7f81034b0 0x7ff7f819f1e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.725+0000 7ff7fe400700 1 --2- 172.21.15.131:0/430895642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f8102b10 0x7ff7f8198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:22.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.725+0000 7ff7fe400700 1 -- 172.21.15.131:0/430895642 >> 172.21.15.131:0/430895642 conn(0x7ff7f80fcfe0 msgr2=0x7ff7f8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:22.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.725+0000 7ff7fe400700 1 -- 172.21.15.131:0/430895642 shutdown_connections 2024-05-10T05:45:22.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:22.725+0000 7ff7fe400700 1 -- 172.21.15.131:0/430895642 wait complete. 2024-05-10T05:45:22.775 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T05:31:49.481667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T05:31:50.960751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:45:22.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:45:23.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:22 smithi157 ceph-mon[78427]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 376 MiB/s wr, 145 op/s 2024-05-10T05:45:23.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:22 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/430895642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:45:23.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:22 smithi142 ceph-mon[77667]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 376 MiB/s wr, 145 op/s 2024-05-10T05:45:23.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:22 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/430895642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:45:23.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.190+0000 7f2001600700 1 -- 172.21.15.131:0/1317228938 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4017400 con 0x7f2004101be0 2024-05-10T05:45:23.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.191+0000 7f2009c00700 1 -- 172.21.15.131:0/1317228938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2004101be0 msgr2=0x7f2004101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:23.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.191+0000 7f2009c00700 1 --2- 172.21.15.131:0/1317228938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2004101be0 0x7f2004101ff0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40098f0 tx=0x7f1ff4009c00 comp rx=0 tx=0).stop 2024-05-10T05:45:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.191+0000 7f2009c00700 1 -- 172.21.15.131:0/1317228938 shutdown_connections 2024-05-10T05:45:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.191+0000 7f2009c00700 1 --2- 172.21.15.131:0/1317228938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2004101be0 0x7f2004101ff0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.191+0000 7f2009c00700 1 --2- 172.21.15.131:0/1317228938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20041036a0 0x7f2004109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.191+0000 7f2009c00700 1 --2- 172.21.15.131:0/1317228938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2004102d00 0x7f2004103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.191+0000 7f2009c00700 1 -- 172.21.15.131:0/1317228938 >> 172.21.15.131:0/1317228938 conn(0x7f20040fd1d0 msgr2=0x7f20040ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:23.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.191+0000 7f2009c00700 1 -- 172.21.15.131:0/1317228938 shutdown_connections 2024-05-10T05:45:23.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.191+0000 7f2009c00700 1 -- 172.21.15.131:0/1317228938 wait complete. 2024-05-10T05:45:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.192+0000 7f2009c00700 1 Processor -- start 2024-05-10T05:45:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.192+0000 7f2009c00700 1 -- start start 2024-05-10T05:45:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2009c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2004102d00 0x7f2004198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2009c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041036a0 0x7f2004198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2009c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200419d4d0 0x7f200419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2009c00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200419ffd0 con 0x7f20041036a0 2024-05-10T05:45:23.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2009c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20041a0140 con 0x7f2004102d00 2024-05-10T05:45:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2009c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20041a0300 con 0x7f200419d4d0 2024-05-10T05:45:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2002000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041036a0 0x7f2004198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2002000700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041036a0 0x7f2004198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57742/0 (socket says 172.21.15.131:57742) 2024-05-10T05:45:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2002000700 1 -- 172.21.15.131:0/884014747 learned_addr learned my addr 172.21.15.131:0/884014747 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:23.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2002a00700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2004102d00 0x7f2004198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:23.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.193+0000 7f2003400700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200419d4d0 0x7f200419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:23.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.194+0000 7f2002000700 1 -- 172.21.15.131:0/884014747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200419d4d0 msgr2=0x7f200419f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:23.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.194+0000 7f2002000700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200419d4d0 0x7f200419f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.194+0000 7f2002000700 1 -- 172.21.15.131:0/884014747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2004102d00 msgr2=0x7f2004198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.194+0000 7f2002000700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2004102d00 0x7f2004198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.194+0000 7f2002000700 1 -- 172.21.15.131:0/884014747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff4009520 con 0x7f20041036a0 2024-05-10T05:45:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.194+0000 7f2002a00700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2004102d00 0x7f2004198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T05:45:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.194+0000 7f2002000700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041036a0 0x7f2004198660 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000e9a0 tx=0x7f1ff000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.194+0000 7f1ffb400700 1 -- 172.21.15.131:0/884014747 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0009790 con 0x7f20041036a0 2024-05-10T05:45:23.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.194+0000 7f1ffb400700 1 -- 172.21.15.131:0/884014747 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ff000cbd0 con 0x7f20041036a0 2024-05-10T05:45:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.195+0000 7f2009c00700 1 -- 172.21.15.131:0/884014747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20041a05e0 con 0x7f20041036a0 2024-05-10T05:45:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.195+0000 7f1ffb400700 1 -- 172.21.15.131:0/884014747 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0018680 con 0x7f20041036a0 2024-05-10T05:45:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.195+0000 7f2009c00700 1 -- 172.21.15.131:0/884014747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20041a0b10 con 0x7f20041036a0 2024-05-10T05:45:23.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.196+0000 7f2009c00700 1 -- 172.21.15.131:0/884014747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2004108df0 con 0x7f20041036a0 2024-05-10T05:45:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.200+0000 7f1ffb400700 1 -- 172.21.15.131:0/884014747 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1ff000cd40 con 0x7f20041036a0 2024-05-10T05:45:23.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.201+0000 7f1ffb400700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1fe80a1240 0x7f1fe80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:23.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.201+0000 7f1ffb400700 1 -- 172.21.15.131:0/884014747 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1ff0014070 con 0x7f20041036a0 2024-05-10T05:45:23.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.201+0000 7f2002a00700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1fe80a1240 0x7f1fe80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:23.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.202+0000 7f2002a00700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1fe80a1240 0x7f1fe80a3750 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4008000 tx=0x7f1ff4009f50 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.204+0000 7f1ffb400700 1 -- 172.21.15.131:0/884014747 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ff0096860 con 0x7f20041036a0 2024-05-10T05:45:23.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:22 smithi131 ceph-mon[37264]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 376 MiB/s wr, 145 op/s 2024-05-10T05:45:23.210 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:22 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/430895642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:45:23.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.523+0000 7f2009c00700 1 -- 172.21.15.131:0/884014747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f200410d3c0 con 0x7f20041036a0 2024-05-10T05:45:23.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.523+0000 7f1ffb400700 1 -- 172.21.15.131:0/884014747 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ff0099e80 con 0x7f20041036a0 2024-05-10T05:45:23.525 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:45:23.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.528+0000 7f2009c00700 1 -- 172.21.15.131:0/884014747 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1fe80a1240 msgr2=0x7f1fe80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:23.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.528+0000 7f2009c00700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1fe80a1240 0x7f1fe80a3750 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4008000 tx=0x7f1ff4009f50 comp rx=0 tx=0).stop 2024-05-10T05:45:23.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.528+0000 7f2009c00700 1 -- 172.21.15.131:0/884014747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041036a0 msgr2=0x7f2004198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:23.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.528+0000 7f2009c00700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041036a0 0x7f2004198660 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000e9a0 tx=0x7f1ff000ed60 comp rx=0 tx=0).stop 2024-05-10T05:45:23.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.529+0000 7f2009c00700 1 -- 172.21.15.131:0/884014747 shutdown_connections 2024-05-10T05:45:23.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.529+0000 7f2009c00700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1fe80a1240 0x7f1fe80a3750 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:23.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.529+0000 7f2009c00700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f200419d4d0 0x7f200419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:23.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.529+0000 7f2009c00700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2004102d00 0x7f2004198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:23.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.529+0000 7f2009c00700 1 --2- 172.21.15.131:0/884014747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041036a0 0x7f2004198660 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:23.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.529+0000 7f2009c00700 1 -- 172.21.15.131:0/884014747 >> 172.21.15.131:0/884014747 conn(0x7f20040fd1d0 msgr2=0x7f2004105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:23.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.529+0000 7f2009c00700 1 -- 172.21.15.131:0/884014747 shutdown_connections 2024-05-10T05:45:23.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:23.529+0000 7f2009c00700 1 -- 172.21.15.131:0/884014747 wait complete. 2024-05-10T05:45:23.567 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T05:45:23.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:23 smithi131 ceph-mon[37264]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 365 MiB/s wr, 162 op/s 2024-05-10T05:45:23.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:23 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/884014747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:45:24.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.011+0000 7f195b600700 1 -- 172.21.15.131:0/384901988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1954109ac0 msgr2=0x7f195410bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:24.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.011+0000 7f195b600700 1 --2- 172.21.15.131:0/384901988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1954109ac0 0x7f195410bfa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f194c009990 tx=0x7f194c009ca0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.011+0000 7f195b600700 1 -- 172.21.15.131:0/384901988 shutdown_connections 2024-05-10T05:45:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.011+0000 7f195b600700 1 --2- 172.21.15.131:0/384901988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1954075140 0x7f1954073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.011+0000 7f195b600700 1 --2- 172.21.15.131:0/384901988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1954109ac0 0x7f195410bfa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.011+0000 7f195b600700 1 --2- 172.21.15.131:0/384901988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1954073b40 0x7f1954074010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.011+0000 7f195b600700 1 -- 172.21.15.131:0/384901988 >> 172.21.15.131:0/384901988 conn(0x7f19540fd210 msgr2=0x7f19540ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.011+0000 7f195b600700 1 -- 172.21.15.131:0/384901988 shutdown_connections 2024-05-10T05:45:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.011+0000 7f195b600700 1 -- 172.21.15.131:0/384901988 wait complete. 2024-05-10T05:45:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.012+0000 7f195b600700 1 Processor -- start 2024-05-10T05:45:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.012+0000 7f195b600700 1 -- start start 2024-05-10T05:45:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f195b600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1954073b40 0x7f195419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f195b600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1954075140 0x7f195419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f195b600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1954109ac0 0x7f19541a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f195b600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19541a1b30 con 0x7f1954073b40 2024-05-10T05:45:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f195b600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19541a1ca0 con 0x7f1954109ac0 2024-05-10T05:45:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f195b600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19541a1e60 con 0x7f1954075140 2024-05-10T05:45:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f1958c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1954073b40 0x7f195419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f1958c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1954073b40 0x7f195419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57758/0 (socket says 172.21.15.131:57758) 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f1959600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1954109ac0 0x7f19541a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f1958c00700 1 -- 172.21.15.131:0/2839336273 learned_addr learned my addr 172.21.15.131:0/2839336273 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f1953e00700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1954075140 0x7f195419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f1959600700 1 -- 172.21.15.131:0/2839336273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1954075140 msgr2=0x7f195419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f1959600700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1954075140 0x7f195419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f1959600700 1 -- 172.21.15.131:0/2839336273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1954073b40 msgr2=0x7f195419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f1959600700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1954073b40 0x7f195419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.013+0000 7f1959600700 1 -- 172.21.15.131:0/2839336273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0095f0 con 0x7f1954109ac0 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.014+0000 7f1959600700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1954109ac0 0x7f19541a1520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f194c0098e0 tx=0x7f194c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.014+0000 7f1951600700 1 -- 172.21.15.131:0/2839336273 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c01d070 con 0x7f1954109ac0 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.014+0000 7f1951600700 1 -- 172.21.15.131:0/2839336273 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194c021410 con 0x7f1954109ac0 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.014+0000 7f195b600700 1 -- 172.21.15.131:0/2839336273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19541a20e0 con 0x7f1954109ac0 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.014+0000 7f1951600700 1 -- 172.21.15.131:0/2839336273 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c017650 con 0x7f1954109ac0 2024-05-10T05:45:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.014+0000 7f195b600700 1 -- 172.21.15.131:0/2839336273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19541a2500 con 0x7f1954109ac0 2024-05-10T05:45:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.015+0000 7f195b600700 1 -- 172.21.15.131:0/2839336273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19541081c0 con 0x7f1954109ac0 2024-05-10T05:45:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.017+0000 7f1951600700 1 -- 172.21.15.131:0/2839336273 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f194c00fa50 con 0x7f1954109ac0 2024-05-10T05:45:24.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.018+0000 7f1951600700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f193c0a11c0 0x7f193c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:24.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.018+0000 7f1951600700 1 -- 172.21.15.131:0/2839336273 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f194c0cde90 con 0x7f1954109ac0 2024-05-10T05:45:24.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.018+0000 7f1958c00700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f193c0a11c0 0x7f193c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:24.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.019+0000 7f1958c00700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f193c0a11c0 0x7f193c0a36d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f194800f5e0 tx=0x7f194800b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.021+0000 7f1951600700 1 -- 172.21.15.131:0/2839336273 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f194c097650 con 0x7f1954109ac0 2024-05-10T05:45:24.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:23 smithi142 ceph-mon[77667]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 365 MiB/s wr, 162 op/s 2024-05-10T05:45:24.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:23 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/884014747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:45:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.230+0000 7f195b600700 1 -- 172.21.15.131:0/2839336273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f195410f610 con 0x7f1954109ac0 2024-05-10T05:45:24.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.231+0000 7f1951600700 1 -- 172.21.15.131:0/2839336273 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f194c09ac70 con 0x7f1954109ac0 2024-05-10T05:45:24.232 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-10T05:45:24.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 -- 172.21.15.131:0/2839336273 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f193c0a11c0 msgr2=0x7f193c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:24.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f193c0a11c0 0x7f193c0a36d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f194800f5e0 tx=0x7f194800b040 comp rx=0 tx=0).stop 2024-05-10T05:45:24.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 -- 172.21.15.131:0/2839336273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1954109ac0 msgr2=0x7f19541a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:24.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1954109ac0 0x7f19541a1520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f194c0098e0 tx=0x7f194c005740 comp rx=0 tx=0).stop 2024-05-10T05:45:24.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 -- 172.21.15.131:0/2839336273 shutdown_connections 2024-05-10T05:45:24.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f193c0a11c0 0x7f193c0a36d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1954073b40 0x7f195419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1954109ac0 0x7f19541a1520 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 --2- 172.21.15.131:0/2839336273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1954075140 0x7f195419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 -- 172.21.15.131:0/2839336273 >> 172.21.15.131:0/2839336273 conn(0x7f19540fd210 msgr2=0x7f19540ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:24.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 -- 172.21.15.131:0/2839336273 shutdown_connections 2024-05-10T05:45:24.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.234+0000 7f195b600700 1 -- 172.21.15.131:0/2839336273 wait complete. 2024-05-10T05:45:24.263 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T05:45:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:23 smithi157 ceph-mon[78427]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 365 MiB/s wr, 162 op/s 2024-05-10T05:45:24.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:23 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/884014747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:45:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.531+0000 7f97c8400700 1 -- 172.21.15.131:0/360468783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c0101be0 msgr2=0x7f97c0101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.531+0000 7f97c8400700 1 --2- 172.21.15.131:0/360468783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c0101be0 0x7f97c0101ff0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f97b40099c0 tx=0x7f97b4009cd0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.531+0000 7f97c8400700 1 -- 172.21.15.131:0/360468783 shutdown_connections 2024-05-10T05:45:24.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.531+0000 7f97c8400700 1 --2- 172.21.15.131:0/360468783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97c01036a0 0x7f97c0109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.531+0000 7f97c8400700 1 --2- 172.21.15.131:0/360468783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c0101be0 0x7f97c0101ff0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.531+0000 7f97c8400700 1 --2- 172.21.15.131:0/360468783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97c0102d00 0x7f97c0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.531+0000 7f97c8400700 1 -- 172.21.15.131:0/360468783 >> 172.21.15.131:0/360468783 conn(0x7f97c00fd1d0 msgr2=0x7f97c00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:24.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.531+0000 7f97c8400700 1 -- 172.21.15.131:0/360468783 shutdown_connections 2024-05-10T05:45:24.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.531+0000 7f97c8400700 1 -- 172.21.15.131:0/360468783 wait complete. 2024-05-10T05:45:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.532+0000 7f97c8400700 1 Processor -- start 2024-05-10T05:45:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.532+0000 7f97c8400700 1 -- start start 2024-05-10T05:45:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c8400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97c0101be0 0x7f97c0197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c8400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97c0102d00 0x7f97c0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c8400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01036a0 0x7f97c019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c8400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c019fa00 con 0x7f97c0101be0 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c8400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c019fb70 con 0x7f97c01036a0 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c8400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c019fd30 con 0x7f97c0102d00 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97c0101be0 0x7f97c0197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c5a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97c0101be0 0x7f97c0197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57784/0 (socket says 172.21.15.131:57784) 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c5a00700 1 -- 172.21.15.131:0/3493316447 learned_addr learned my addr 172.21.15.131:0/3493316447 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c6400700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01036a0 0x7f97c019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c5000700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97c0102d00 0x7f97c0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c5a00700 1 -- 172.21.15.131:0/3493316447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97c0102d00 msgr2=0x7f97c0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c5a00700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97c0102d00 0x7f97c0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c5a00700 1 -- 172.21.15.131:0/3493316447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01036a0 msgr2=0x7f97c019f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.533+0000 7f97c5a00700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01036a0 0x7f97c019f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.534+0000 7f97c5a00700 1 -- 172.21.15.131:0/3493316447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b40095f0 con 0x7f97c0101be0 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.534+0000 7f97c6400700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01036a0 0x7f97c019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.534+0000 7f97c5a00700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97c0101be0 0x7f97c0197f20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009fd0 tx=0x7f97b400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.534+0000 7f97be000700 1 -- 172.21.15.131:0/3493316447 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b401d070 con 0x7f97c0101be0 2024-05-10T05:45:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.534+0000 7f97be000700 1 -- 172.21.15.131:0/3493316447 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b4005050 con 0x7f97c0101be0 2024-05-10T05:45:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.534+0000 7f97c8400700 1 -- 172.21.15.131:0/3493316447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c019ffb0 con 0x7f97c0101be0 2024-05-10T05:45:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.534+0000 7f97be000700 1 -- 172.21.15.131:0/3493316447 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b40049e0 con 0x7f97c0101be0 2024-05-10T05:45:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.534+0000 7f97c8400700 1 -- 172.21.15.131:0/3493316447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c01a0320 con 0x7f97c0101be0 2024-05-10T05:45:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.535+0000 7f97c8400700 1 -- 172.21.15.131:0/3493316447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c0108df0 con 0x7f97c0101be0 2024-05-10T05:45:24.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.537+0000 7f97be000700 1 -- 172.21.15.131:0/3493316447 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f97b4005450 con 0x7f97c0101be0 2024-05-10T05:45:24.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.539+0000 7f97be000700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97a80a11c0 0x7f97a80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:24.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.539+0000 7f97be000700 1 -- 172.21.15.131:0/3493316447 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f97b40cd610 con 0x7f97c0101be0 2024-05-10T05:45:24.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.539+0000 7f97c5000700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97a80a11c0 0x7f97a80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:24.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.539+0000 7f97c5000700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97a80a11c0 0x7f97a80a36d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f97c0198fc0 tx=0x7f97ac009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:24.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.543+0000 7f97be000700 1 -- 172.21.15.131:0/3493316447 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97b4096e30 con 0x7f97c0101be0 2024-05-10T05:45:24.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.749+0000 7f97c8400700 1 -- 172.21.15.131:0/3493316447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97c010d410 con 0x7f97c0101be0 2024-05-10T05:45:24.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.750+0000 7f97be000700 1 -- 172.21.15.131:0/3493316447 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97b409a450 con 0x7f97c0101be0 2024-05-10T05:45:24.751 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-10T05:45:24.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 -- 172.21.15.131:0/3493316447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97a80a11c0 msgr2=0x7f97a80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:24.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97a80a11c0 0x7f97a80a36d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f97c0198fc0 tx=0x7f97ac009260 comp rx=0 tx=0).stop 2024-05-10T05:45:24.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 -- 172.21.15.131:0/3493316447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97c0101be0 msgr2=0x7f97c0197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:24.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97c0101be0 0x7f97c0197f20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009fd0 tx=0x7f97b400f690 comp rx=0 tx=0).stop 2024-05-10T05:45:24.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 -- 172.21.15.131:0/3493316447 shutdown_connections 2024-05-10T05:45:24.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f97a80a11c0 0x7f97a80a36d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97c0101be0 0x7f97c0197f20 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01036a0 0x7f97c019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 --2- 172.21.15.131:0/3493316447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97c0102d00 0x7f97c0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:24.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 -- 172.21.15.131:0/3493316447 >> 172.21.15.131:0/3493316447 conn(0x7f97c00fd1d0 msgr2=0x7f97c0105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:24.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 -- 172.21.15.131:0/3493316447 shutdown_connections 2024-05-10T05:45:24.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:24.752+0000 7f97c8400700 1 -- 172.21.15.131:0/3493316447 wait complete. 2024-05-10T05:45:24.781 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-10T05:45:25.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.047+0000 7f1f3aa00700 1 -- 172.21.15.131:0/1667597011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f34073ae0 msgr2=0x7f1f34073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:25.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.047+0000 7f1f3aa00700 1 --2- 172.21.15.131:0/1667597011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f34073ae0 0x7f1f34073f30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009990 tx=0x7f1f20009ca0 comp rx=0 tx=0).stop 2024-05-10T05:45:25.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.047+0000 7f1f3aa00700 1 -- 172.21.15.131:0/1667597011 shutdown_connections 2024-05-10T05:45:25.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.047+0000 7f1f3aa00700 1 --2- 172.21.15.131:0/1667597011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f34109900 0x7f1f3410bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:25.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.047+0000 7f1f3aa00700 1 --2- 172.21.15.131:0/1667597011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f340750e0 0x7f1f340735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:25.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.047+0000 7f1f3aa00700 1 --2- 172.21.15.131:0/1667597011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f34073ae0 0x7f1f34073f30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:25.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.047+0000 7f1f3aa00700 1 -- 172.21.15.131:0/1667597011 >> 172.21.15.131:0/1667597011 conn(0x7f1f340fd010 msgr2=0x7f1f340ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:25.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.048+0000 7f1f3aa00700 1 -- 172.21.15.131:0/1667597011 shutdown_connections 2024-05-10T05:45:25.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.048+0000 7f1f3aa00700 1 -- 172.21.15.131:0/1667597011 wait complete. 2024-05-10T05:45:25.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f3aa00700 1 Processor -- start 2024-05-10T05:45:25.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f3aa00700 1 -- start start 2024-05-10T05:45:25.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f3aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f34073ae0 0x7f1f34199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f3aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f340750e0 0x7f1f3419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f3aa00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f34109900 0x7f1f341a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f3aa00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f341a19e0 con 0x7f1f340750e0 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f3aa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f341a1b20 con 0x7f1f34073ae0 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f3aa00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f341a1cb0 con 0x7f1f34109900 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f38c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f340750e0 0x7f1f3419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f39600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f34073ae0 0x7f1f34199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f38c00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f340750e0 0x7f1f3419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57796/0 (socket says 172.21.15.131:57796) 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.049+0000 7f1f38c00700 1 -- 172.21.15.131:0/4161587326 learned_addr learned my addr 172.21.15.131:0/4161587326 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.050+0000 7f1f3a000700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f34109900 0x7f1f341a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.050+0000 7f1f39600700 1 -- 172.21.15.131:0/4161587326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f34109900 msgr2=0x7f1f341a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.050+0000 7f1f39600700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f34109900 0x7f1f341a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.050+0000 7f1f39600700 1 -- 172.21.15.131:0/4161587326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f340750e0 msgr2=0x7f1f3419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:25.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.050+0000 7f1f39600700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f340750e0 0x7f1f3419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.050+0000 7f1f39600700 1 -- 172.21.15.131:0/4161587326 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200095f0 con 0x7f1f34073ae0 2024-05-10T05:45:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.050+0000 7f1f38c00700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f340750e0 0x7f1f3419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.050+0000 7f1f39600700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f34073ae0 0x7f1f34199f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800e9c0 tx=0x7f1f2800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.050+0000 7f1f32000700 1 -- 172.21.15.131:0/4161587326 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800c9c0 con 0x7f1f34073ae0 2024-05-10T05:45:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.051+0000 7f1f32000700 1 -- 172.21.15.131:0/4161587326 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f28010430 con 0x7f1f34073ae0 2024-05-10T05:45:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.051+0000 7f1f3aa00700 1 -- 172.21.15.131:0/4161587326 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f341a1f90 con 0x7f1f34073ae0 2024-05-10T05:45:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.051+0000 7f1f32000700 1 -- 172.21.15.131:0/4161587326 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28018680 con 0x7f1f34073ae0 2024-05-10T05:45:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.051+0000 7f1f3aa00700 1 -- 172.21.15.131:0/4161587326 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f341a23c0 con 0x7f1f34073ae0 2024-05-10T05:45:25.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.052+0000 7f1f3aa00700 1 -- 172.21.15.131:0/4161587326 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f34108750 con 0x7f1f34073ae0 2024-05-10T05:45:25.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.054+0000 7f1f32000700 1 -- 172.21.15.131:0/4161587326 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1f28010860 con 0x7f1f34073ae0 2024-05-10T05:45:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.055+0000 7f1f32000700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1f1c0a11c0 0x7f1f1c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.055+0000 7f1f32000700 1 -- 172.21.15.131:0/4161587326 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f1f28014070 con 0x7f1f34073ae0 2024-05-10T05:45:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.055+0000 7f1f38c00700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1f1c0a11c0 0x7f1f1c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:25.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.056+0000 7f1f38c00700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1f1c0a11c0 0x7f1f1c0a36d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000b2e0 tx=0x7f1f2001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:25.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.059+0000 7f1f32000700 1 -- 172.21.15.131:0/4161587326 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1f28052d60 con 0x7f1f34073ae0 2024-05-10T05:45:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:25 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2839336273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:45:25.303 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:25 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3493316447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:45:25.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.324+0000 7f1f3aa00700 1 -- 172.21.15.131:0/4161587326 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f1f34068110 con 0x7f1f34073ae0 2024-05-10T05:45:25.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.334+0000 7f1f32000700 1 -- 172.21.15.131:0/4161587326 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733413 (secure 0 0 0) 0x7f1f28096150 con 0x7f1f34073ae0 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: "epoch": 18, 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: "active_gid": 14154, 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: "active_name": "x", 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: "active_addrs": { 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-10T05:45:25.346 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:6800", 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2109111580 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:6801", 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2109111580 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: "active_addr": "172.21.15.131:6801/2109111580", 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: "active_change": "2024-05-10T05:26:14.753928+0000", 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: "active_mgr_features": 4540138322906710015, 2024-05-10T05:45:25.347 INFO:teuthology.orchestra.run.smithi131.stdout: "available": true, 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "standbys": [ 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "gid": 24109, 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "z", 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr_features": 4540138322906710015, 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "available_modules": [ 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "alerts", 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:25.348 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.349 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.350 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.351 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.352 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.353 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.354 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.354 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.354 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.354 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.354 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.354 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.354 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.354 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.354 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.354 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.358 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.358 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.358 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.358 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.358 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_destination": { 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_destination", 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.359 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email address to send alerts to", 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_from_name": { 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_from_name", 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "Ceph", 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email From: name", 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_host": { 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_host", 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP server", 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_password": { 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_password", 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.362 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Password to authenticate with", 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_port": { 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_port", 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "465", 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.363 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP port", 2024-05-10T05:45:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_sender": { 2024-05-10T05:45:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_sender", 2024-05-10T05:45:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.364 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP envelope sender", 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_ssl": { 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_ssl", 2024-05-10T05:45:25.365 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.366 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_user": { 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_user", 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "User to authenticate as", 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.367 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "balancer", 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "active": { 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "active", 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.368 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_time": { 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_time", 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0000", 2024-05-10T05:45:25.369 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_weekday": { 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_weekday", 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.370 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_max_iterations": { 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_max_iterations", 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.371 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "25", 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "250", 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_metrics": { 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_metrics", 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.372 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_step": { 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_step", 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.373 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.5", 2024-05-10T05:45:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0.001", 2024-05-10T05:45:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "0.999", 2024-05-10T05:45:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "aggressiveness of optimization", 2024-05-10T05:45:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T05:45:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.374 INFO:teuthology.orchestra.run.smithi131.stdout: "end_time": { 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_time", 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2359", 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.375 INFO:teuthology.orchestra.run.smithi131.stdout: "end_weekday": { 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_weekday", 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.376 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.377 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.378 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.379 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.380 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.381 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "min_score": { 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_score", 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.382 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "upmap", 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.383 INFO:teuthology.orchestra.run.smithi131.stdout: "crush-compat", 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: "none", 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap" 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Balancer mode", 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_ids": { 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_ids", 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.384 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:25.385 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_deviation": { 2024-05-10T05:45:25.386 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_deviation", 2024-05-10T05:45:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T05:45:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T05:45:25.387 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_optimizations": { 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_optimizations", 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.388 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm", 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_down_multiplier": { 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_down_multiplier", 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.389 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_refresh_rate": { 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_refresh_rate", 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.390 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "20", 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_starting_port": { 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_starting_port", 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.391 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.392 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4721", 2024-05-10T05:45:25.392 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.392 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.392 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.392 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T05:45:25.392 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.392 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.392 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_ptrace": { 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_ptrace", 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.393 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_interval": { 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_interval", 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.394 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.395 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.395 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.395 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_memory_target_ratio": { 2024-05-10T05:45:25.395 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.7", 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: "cgroups_split": { 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cgroups_split", 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.397 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "config_checks_enabled": { 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_checks_enabled", 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.398 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "config_dashboard": { 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_dashboard", 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.399 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_alertmanager": { 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_alertmanager", 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.400 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_base": { 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_base", 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Container image name, without the tag", 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.401 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_elasticsearch": { 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_elasticsearch", 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "elasticsearch container image", 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.402 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_grafana": { 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_grafana", 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.403 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_haproxy": { 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_haproxy", 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "HAproxy container image", 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.404 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_agent": { 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_agent", 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.405 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger agent container image", 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_collector": { 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_collector", 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.406 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger collector container image", 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_query": { 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_query", 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.407 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger query container image", 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_keepalived": { 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_keepalived", 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.408 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Keepalived container image", 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_loki": { 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_loki", 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.409 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Loki container image", 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_node_exporter": { 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_node_exporter", 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.410 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_nvmeof": { 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_nvmeof", 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.411 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Nvme-of container image", 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_prometheus": { 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_prometheus", 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T05:45:25.412 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_promtail": { 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_promtail", 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T05:45:25.413 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Promtail container image", 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_snmp_gateway": { 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_snmp_gateway", 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T05:45:25.414 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SNMP Gateway container image", 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "container_init": { 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_init", 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.415 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "daemon_cache_timeout": { 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "daemon_cache_timeout", 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.416 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "default_cephadm_command_timeout": { 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.417 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "900", 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "default_registry": { 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_registry", 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.418 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io", 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "device_cache_timeout": { 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_cache_timeout", 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.419 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1800", 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache device inventory", 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "device_enhanced_scan": { 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_enhanced_scan", 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.420 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "facts_cache_timeout": { 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "facts_cache_timeout", 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.421 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache host facts data", 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "host_check_interval": { 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "host_check_interval", 2024-05-10T05:45:25.422 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to perform a host check", 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "hw_monitoring": { 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hw_monitoring", 2024-05-10T05:45:25.423 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "inventory_list_all": { 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "inventory_list_all", 2024-05-10T05:45:25.424 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.425 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.426 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "log_refresh_metadata": { 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_refresh_metadata", 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.427 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.428 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.429 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.430 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.431 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.432 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "*", 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.433 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "max_count_per_host": { 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_count_per_host", 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of daemons per service per host", 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.434 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "max_osd_draining_count": { 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_osd_draining_count", 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.435 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "migration_current": { 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "migration_current", 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "internal - do not modify", 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.436 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "root", 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm-package", 2024-05-10T05:45:25.437 INFO:teuthology.orchestra.run.smithi131.stdout: "root" 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "oob_default_addr": { 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "oob_default_addr", 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "169.254.1.1", 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.438 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "prometheus_alerts_path": { 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus_alerts_path", 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.439 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_insecure": { 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_insecure", 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.440 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_password": { 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_password", 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.441 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_url": { 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_url", 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.442 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_username": { 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_username", 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.443 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "secure_monitoring_stack": { 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "secure_monitoring_stack", 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.444 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "service_discovery_port": { 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "service_discovery_port", 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.445 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8765", 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "cephadm service discovery port", 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "ssh_config_file": { 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssh_config_file", 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.446 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "use_agent": { 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_agent", 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.447 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "use_repo_digest": { 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_repo_digest", 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.448 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_failed_host_check": { 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_failed_host_check", 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.449 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_daemons": { 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_daemons", 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.450 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_hosts": { 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_hosts", 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.451 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.452 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crash", 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.453 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.454 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.455 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.456 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.457 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "retain_interval": { 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retain_interval", 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "31536000", 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.458 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_recent_interval": { 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_recent_interval", 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1209600", 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.459 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dashboard", 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.460 INFO:teuthology.orchestra.run.smithi131.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.461 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.462 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.463 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_ENABLED": { 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.464 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.465 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.466 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.467 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.468 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.469 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.470 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.471 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.472 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.473 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.474 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.475 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.476 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_URL": { 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_URL", 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.477 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.478 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.479 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.480 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.481 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.482 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.483 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.484 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.486 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.487 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.488 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.489 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.490 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.491 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.492 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.493 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.494 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.495 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8", 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.496 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "45", 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T05:45:25.497 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.500 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.501 INFO:teuthology.orchestra.run.smithi131.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.502 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.504 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.505 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "cross_origin_url": { 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cross_origin_url", 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.506 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "crt_file": { 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crt_file", 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.507 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "debug": { 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "debug", 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable/disable debug options", 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.508 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "jwt_token_ttl": { 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "jwt_token_ttl", 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "28800", 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.509 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.510 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.511 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.512 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.513 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.514 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.515 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "motd": { 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "motd", 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "The message of the day", 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.516 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect_resolve_ip_addr": { 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.517 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.518 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8080", 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.519 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.520 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl_server_port": { 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl_server_port", 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8443", 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.521 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "redirect", 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect" 2024-05-10T05:45:25.522 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-10T05:45:25.523 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "url_prefix": { 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url_prefix", 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.524 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.525 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_monitoring": { 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_monitoring", 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "monitor device health metrics", 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.526 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.527 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.528 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.529 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.530 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.531 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "mark_out_threshold": { 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mark_out_threshold", 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2419200", 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.532 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_name": { 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_name", 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "device_health_metrics", 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.533 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "retention_period": { 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retention_period", 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15552000", 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain device health metrics", 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.534 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_frequency": { 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_frequency", 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.535 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "self_heal": { 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "self_heal", 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.536 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.537 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_threshold": { 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_threshold", 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7257600", 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.538 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "diskprediction_local", 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.539 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.540 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.541 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.542 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.543 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "predict_interval": { 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predict_interval", 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.544 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "predictor_model": { 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predictor_model", 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.545 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "prophetstor", 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.546 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.547 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "influx", 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": false, 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "influxdb python module not found", 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "batch_size": { 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "batch_size", 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5000", 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T05:45:25.548 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "database": { 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "database", 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "ceph", 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.549 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": { 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hostname", 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server hostname", 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "30", 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "5", 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T05:45:25.551 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.552 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.553 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.554 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.556 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "password": { 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "password", 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "password of InfluxDB server user", 2024-05-10T05:45:25.557 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "port": { 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "port", 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8086", 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server port", 2024-05-10T05:45:25.558 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.559 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.559 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-10T05:45:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-10T05:45:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "false", 2024-05-10T05:45:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.562 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "threads": { 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threads", 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "32", 2024-05-10T05:45:25.563 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "username": { 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "username", 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.564 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "username of InfluxDB server user", 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "verify_ssl": { 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "verify_ssl", 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "true", 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.565 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "insights", 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.566 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.567 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.568 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.569 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.570 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.571 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "iostat", 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.572 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.573 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.574 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.575 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.576 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "k8sevents", 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.577 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_event_retention_days": { 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ceph_event_retention_days", 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7", 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.578 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "config_check_secs": { 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_check_secs", 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "10", 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.579 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.580 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.581 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.582 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.583 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.584 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "localpool", 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: "failure_domain": { 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "failure_domain", 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.585 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "host", 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "failure domain for any created local pool", 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.586 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.587 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "min_size": { 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_size", 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default min_size for any created local pool", 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "num_rep": { 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "num_rep", 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3", 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default replica count for any created local pool", 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_num": { 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_num", 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "128", 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "prefix": { 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prefix", 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name prefix for any created local pool", 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "subtree": { 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "subtree", 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "rack", 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mds_autoscaler", 2024-05-10T05:45:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirroring", 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.604 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "nfs", 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "fail_fs": { 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "fail_fs", 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator": { 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "rook", 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "test_orchestrator" 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Orchestrator backend", 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_perf_query", 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_support", 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_autoscaler", 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "threshold": { 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threshold", 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-10T05:45:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1.0", 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "scaling threshold", 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "progress", 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_pg_recovery_event": { 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_pg_recovery_event", 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.642 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.643 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.644 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.645 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.646 INFO:teuthology.orchestra.run.smithi131.stdout: "max_completed_events": { 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_completed_events", 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "50", 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "number of past completed events to remember", 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.647 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long the module is going to sleep", 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.648 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus", 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "cache": { 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cache", 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.649 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "exclude_perf_counters": { 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "exclude_perf_counters", 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.650 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.651 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.652 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.653 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.654 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools": { 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools", 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.656 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "300", 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.657 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_interval": { 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_interval", 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15.0", 2024-05-10T05:45:25.658 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-10T05:45:25.659 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "9283", 2024-05-10T05:45:25.660 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "stale_cache_strategy": { 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stale_cache_strategy", 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "log", 2024-05-10T05:45:25.661 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "default", 2024-05-10T05:45:25.662 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "default", 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "error" 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.663 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.664 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.665 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.666 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.667 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.668 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.669 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_snap_create": { 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_snap_create", 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.670 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "mirror_snapshot_schedule": { 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.671 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "trash_purge_schedule": { 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "trash_purge_schedule", 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.672 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "restful", 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.673 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_auth": { 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_auth", 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.674 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.675 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.676 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.677 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.678 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.681 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.682 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rgw", 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.683 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.684 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.685 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.687 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.688 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rook", 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.689 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.690 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.691 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.692 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.693 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "storage_class": { 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "storage_class", 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "local", 2024-05-10T05:45:25.694 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "selftest", 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.695 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.696 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.697 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.698 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.699 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.700 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "roption1": { 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption1", 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.701 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "roption2": { 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption2", 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "xyz", 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption1": { 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption1", 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.703 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption2": { 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption2", 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.704 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption3": { 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption3", 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption4": { 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption4", 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.706 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption5": { 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption5", 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.707 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption6": { 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption6", 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.708 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption7": { 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption7", 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "42", 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.709 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "testkey": { 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testkey", 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.710 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "testlkey": { 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testlkey", 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.711 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "testnewline": { 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testnewline", 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_m_granularity": { 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_m_granularity", 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.713 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "dump_on_update": { 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dump_on_update", 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.714 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "dump database to debug log on update", 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.715 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.716 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.717 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.718 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.719 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stats", 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.720 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.721 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.722 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.723 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.724 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.725 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "status", 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.726 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.727 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.728 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.729 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.730 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.731 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telegraf", 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "address": { 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "address", 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.732 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15", 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.733 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.734 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.735 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.736 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.737 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.738 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telemetry", 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.739 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_basic": { 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_basic", 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.740 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_crash": { 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_crash", 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.741 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_device": { 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_device", 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.742 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_ident": { 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_ident", 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.743 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_perf": { 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_perf", 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.744 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "contact": { 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "contact", 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.745 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "description": { 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "description", 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.746 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "device_url": { 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_url", 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.747 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.748 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "24", 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "8", 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.749 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "last_opt_revision": { 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "last_opt_revision", 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1", 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.750 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard": { 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard", 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.751 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard_description": { 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard_description", 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.752 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.753 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.754 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.754 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.754 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.754 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.754 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.754 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.754 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.754 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.755 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.756 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.757 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.758 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "organization": { 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "organization", 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.759 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "proxy": { 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "proxy", 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.760 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "url": { 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url", 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.761 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.762 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "test_orchestrator", 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.763 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.764 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.765 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.766 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.767 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.767 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.770 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.770 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.770 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.770 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.770 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.770 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.770 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.771 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.772 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.773 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.774 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.775 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.776 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.777 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.778 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_clones": { 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_clones", 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4", 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.779 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_delay": { 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_delay", 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.780 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_no_wait": { 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.781 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix", 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.782 INFO:teuthology.orchestra.run.smithi131.stdout: "discovery_interval": { 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "discovery_interval", 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "100", 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.783 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.784 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.784 INFO:teuthology.orchestra.run.smithi131.stdout: "identifier": { 2024-05-10T05:45:25.784 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "identifier", 2024-05-10T05:45:25.784 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.784 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.784 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.784 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.784 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.784 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.784 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:25.785 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.786 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.787 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.788 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.789 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.790 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.791 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_host": { 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_host", 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.792 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_port": { 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_port", 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.793 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10051", 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_sender": { 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_sender", 2024-05-10T05:45:25.794 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.795 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: "gid": 24116, 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "y", 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr_features": 4540138322906710015, 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: "available_modules": [ 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "alerts", 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.796 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.797 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.798 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.799 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.800 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.801 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.802 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.803 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_destination": { 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_destination", 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email address to send alerts to", 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.804 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_from_name": { 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_from_name", 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "Ceph", 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email From: name", 2024-05-10T05:45:25.805 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_host": { 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_host", 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.806 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP server", 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_password": { 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_password", 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.807 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Password to authenticate with", 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_port": { 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_port", 2024-05-10T05:45:25.808 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "465", 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP port", 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.809 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_sender": { 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_sender", 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP envelope sender", 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.810 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_ssl": { 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_ssl", 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.811 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_user": { 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_user", 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.812 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "User to authenticate as", 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "balancer", 2024-05-10T05:45:25.813 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "active": { 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "active", 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.814 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_time": { 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_time", 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0000", 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.815 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.816 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T05:45:25.816 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T05:45:25.816 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.816 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.816 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.816 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_weekday": { 2024-05-10T05:45:25.816 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_weekday", 2024-05-10T05:45:25.816 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:25.816 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.816 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_max_iterations": { 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_max_iterations", 2024-05-10T05:45:25.817 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "25", 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "250", 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.818 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_metrics": { 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_metrics", 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.819 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_step": { 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_step", 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.5", 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0.001", 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "0.999", 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "aggressiveness of optimization", 2024-05-10T05:45:25.820 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "end_time": { 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_time", 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2359", 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.821 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "end_weekday": { 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_weekday", 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:25.822 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.823 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.824 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.825 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.826 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.827 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.828 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "min_score": { 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_score", 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T05:45:25.829 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "upmap", 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.830 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: "crush-compat", 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: "none", 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap" 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Balancer mode", 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_ids": { 2024-05-10T05:45:25.831 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_ids", 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.832 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.833 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_deviation": { 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_deviation", 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T05:45:25.834 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_optimizations": { 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_optimizations", 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.835 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm", 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.836 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_down_multiplier": { 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_down_multiplier", 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.837 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_refresh_rate": { 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_refresh_rate", 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "20", 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.838 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_starting_port": { 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_starting_port", 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4721", 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_ptrace": { 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_ptrace", 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.840 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_interval": { 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_interval", 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.841 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_memory_target_ratio": { 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.842 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.7", 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "cgroups_split": { 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cgroups_split", 2024-05-10T05:45:25.843 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.844 INFO:teuthology.orchestra.run.smithi131.stdout: "config_checks_enabled": { 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_checks_enabled", 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.845 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "config_dashboard": { 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_dashboard", 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T05:45:25.846 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_alertmanager": { 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_alertmanager", 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.847 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_base": { 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_base", 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T05:45:25.848 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Container image name, without the tag", 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_elasticsearch": { 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_elasticsearch", 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.849 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "elasticsearch container image", 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_grafana": { 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_grafana", 2024-05-10T05:45:25.850 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.851 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_haproxy": { 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_haproxy", 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "HAproxy container image", 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.852 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_agent": { 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_agent", 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger agent container image", 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.853 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_collector": { 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_collector", 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.854 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger collector container image", 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_query": { 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_query", 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.855 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger query container image", 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_keepalived": { 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_keepalived", 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T05:45:25.856 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Keepalived container image", 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_loki": { 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_loki", 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.857 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Loki container image", 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_node_exporter": { 2024-05-10T05:45:25.858 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_node_exporter", 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.859 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_nvmeof": { 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_nvmeof", 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Nvme-of container image", 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_prometheus": { 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_prometheus", 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.861 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_promtail": { 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_promtail", 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T05:45:25.862 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Promtail container image", 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_snmp_gateway": { 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_snmp_gateway", 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.863 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SNMP Gateway container image", 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "container_init": { 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_init", 2024-05-10T05:45:25.864 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.865 INFO:teuthology.orchestra.run.smithi131.stdout: "daemon_cache_timeout": { 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "daemon_cache_timeout", 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.866 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "default_cephadm_command_timeout": { 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "900", 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T05:45:25.867 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "default_registry": { 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_registry", 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io", 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.868 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "device_cache_timeout": { 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_cache_timeout", 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1800", 2024-05-10T05:45:25.869 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache device inventory", 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "device_enhanced_scan": { 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_enhanced_scan", 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.870 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "facts_cache_timeout": { 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "facts_cache_timeout", 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.871 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache host facts data", 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "host_check_interval": { 2024-05-10T05:45:25.872 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "host_check_interval", 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to perform a host check", 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.873 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "hw_monitoring": { 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hw_monitoring", 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.874 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "inventory_list_all": { 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "inventory_list_all", 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.875 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.876 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.877 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "log_refresh_metadata": { 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_refresh_metadata", 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T05:45:25.878 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.879 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.880 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.882 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.883 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.884 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "*", 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "max_count_per_host": { 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_count_per_host", 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.885 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of daemons per service per host", 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "max_osd_draining_count": { 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_osd_draining_count", 2024-05-10T05:45:25.886 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.887 INFO:teuthology.orchestra.run.smithi131.stdout: "migration_current": { 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "migration_current", 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "internal - do not modify", 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.888 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "root", 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm-package", 2024-05-10T05:45:25.889 INFO:teuthology.orchestra.run.smithi131.stdout: "root" 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: "oob_default_addr": { 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "oob_default_addr", 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.890 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "169.254.1.1", 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "prometheus_alerts_path": { 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus_alerts_path", 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.891 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.892 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_insecure": { 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_insecure", 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.893 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_password": { 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_password", 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.894 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_url": { 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_url", 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.895 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_username": { 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_username", 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.896 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "secure_monitoring_stack": { 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "secure_monitoring_stack", 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.897 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "service_discovery_port": { 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "service_discovery_port", 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.898 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8765", 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "cephadm service discovery port", 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "ssh_config_file": { 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssh_config_file", 2024-05-10T05:45:25.899 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "use_agent": { 2024-05-10T05:45:25.900 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_agent", 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.901 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: "use_repo_digest": { 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_repo_digest", 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.902 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_failed_host_check": { 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_failed_host_check", 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.903 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_daemons": { 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_daemons", 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_hosts": { 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_hosts", 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.906 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crash", 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.907 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.908 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.909 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.910 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.911 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.912 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "retain_interval": { 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retain_interval", 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "31536000", 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.913 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_recent_interval": { 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_recent_interval", 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1209600", 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.914 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dashboard", 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.915 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.916 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.917 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.917 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T05:45:25.917 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T05:45:25.917 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.917 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.917 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.917 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.917 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.917 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.917 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.918 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_ENABLED": { 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.919 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T05:45:25.920 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.921 INFO:teuthology.orchestra.run.smithi131.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.922 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.923 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.924 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.925 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.926 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T05:45:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.928 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.929 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.930 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.931 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-10T05:45:25.932 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.933 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_URL": { 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_URL", 2024-05-10T05:45:25.934 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T05:45:25.935 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.936 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.937 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.938 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.939 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.940 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.941 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T05:45:25.942 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T05:45:25.943 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.944 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.945 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.946 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.947 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.948 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.949 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T05:45:25.950 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.951 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.952 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.953 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.954 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.954 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.954 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.954 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.954 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.954 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T05:45:25.954 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T05:45:25.954 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.954 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.954 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T05:45:25.955 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8", 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.956 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "45", 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.957 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.958 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.959 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T05:45:25.960 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.961 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.962 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.963 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.964 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.965 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.966 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "cross_origin_url": { 2024-05-10T05:45:25.967 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cross_origin_url", 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.968 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "crt_file": { 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crt_file", 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: "debug": { 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "debug", 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.970 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable/disable debug options", 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: "jwt_token_ttl": { 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "jwt_token_ttl", 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.971 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "28800", 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.972 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.973 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.974 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.975 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.976 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.977 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.978 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: "motd": { 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "motd", 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.979 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "The message of the day", 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect_resolve_ip_addr": { 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.980 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.981 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8080", 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.983 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.984 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl_server_port": { 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl_server_port", 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8443", 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.985 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.986 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "redirect", 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect" 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.987 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.988 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "url_prefix": { 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url_prefix", 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.989 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_monitoring": { 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_monitoring", 2024-05-10T05:45:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "monitor device health metrics", 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.992 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.993 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.994 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:25.995 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.996 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "mark_out_threshold": { 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mark_out_threshold", 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:25.997 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2419200", 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:25.998 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_name": { 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_name", 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "device_health_metrics", 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:25.999 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "retention_period": { 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retention_period", 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15552000", 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain device health metrics", 2024-05-10T05:45:26.000 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_frequency": { 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_frequency", 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.001 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: "self_heal": { 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "self_heal", 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.002 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.003 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.003 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.003 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.003 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.003 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T05:45:26.003 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.003 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.003 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.003 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.003 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.004 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_threshold": { 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_threshold", 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7257600", 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T05:45:26.005 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "diskprediction_local", 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.006 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.007 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.008 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.009 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.010 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.011 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "predict_interval": { 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predict_interval", 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "predictor_model": { 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predictor_model", 2024-05-10T05:45:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "prophetstor", 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.015 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "influx", 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": false, 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "influxdb python module not found", 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: "batch_size": { 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "batch_size", 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.016 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5000", 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "database": { 2024-05-10T05:45:26.017 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "database", 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "ceph", 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.018 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": { 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hostname", 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server hostname", 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.019 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "30", 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "5", 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.020 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.021 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.022 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.023 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.024 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.025 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.026 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "password": { 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "password", 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "password of InfluxDB server user", 2024-05-10T05:45:26.027 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: "port": { 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "port", 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8086", 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.028 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server port", 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "false", 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "threads": { 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threads", 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "32", 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "username": { 2024-05-10T05:45:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "username", 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "username of InfluxDB server user", 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "verify_ssl": { 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "verify_ssl", 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "true", 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "insights", 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "iostat", 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.042 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.043 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.044 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.045 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "k8sevents", 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_event_retention_days": { 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ceph_event_retention_days", 2024-05-10T05:45:26.047 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7", 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.048 INFO:teuthology.orchestra.run.smithi131.stdout: "config_check_secs": { 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_check_secs", 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "10", 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.049 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.050 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.051 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.052 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.053 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.055 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "localpool", 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "failure_domain": { 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "failure_domain", 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "host", 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.056 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "failure domain for any created local pool", 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.058 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.058 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.058 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.058 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.058 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.058 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.058 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.058 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.058 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.058 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.059 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.060 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.061 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.062 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "min_size": { 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_size", 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.063 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default min_size for any created local pool", 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "num_rep": { 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "num_rep", 2024-05-10T05:45:26.064 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3", 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default replica count for any created local pool", 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.065 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_num": { 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_num", 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "128", 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.066 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "prefix": { 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prefix", 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name prefix for any created local pool", 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.067 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "subtree": { 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "subtree", 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "rack", 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.068 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mds_autoscaler", 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.069 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.070 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.071 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.072 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.073 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.074 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.074 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.074 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.074 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.074 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.074 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.074 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.074 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.074 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.075 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirroring", 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.076 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.077 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.079 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.080 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.081 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "nfs", 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.082 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.083 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.084 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.085 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.086 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.087 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.088 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "fail_fs": { 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "fail_fs", 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.089 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.090 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.091 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.092 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.093 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.094 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.095 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator": { 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.096 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: "rook", 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: "test_orchestrator" 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Orchestrator backend", 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.097 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_perf_query", 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.098 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.099 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.100 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.101 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.102 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.103 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_support", 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.104 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.105 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.107 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.107 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.107 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.107 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.107 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.107 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.107 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.107 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.107 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.107 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.108 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.109 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_autoscaler", 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.111 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.112 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.113 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.114 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.115 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.116 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.117 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "threshold": { 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threshold", 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1.0", 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.118 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "scaling threshold", 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "progress", 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.119 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_pg_recovery_event": { 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_pg_recovery_event", 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.120 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.121 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.122 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.123 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.124 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.125 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.126 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.127 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "max_completed_events": { 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_completed_events", 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "50", 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "number of past completed events to remember", 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.128 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.129 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long the module is going to sleep", 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus", 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.130 INFO:teuthology.orchestra.run.smithi131.stdout: "cache": { 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cache", 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.131 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "exclude_perf_counters": { 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "exclude_perf_counters", 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T05:45:26.132 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.133 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.134 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.135 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.136 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.137 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.138 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools": { 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools", 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.139 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "300", 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.140 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_interval": { 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_interval", 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15.0", 2024-05-10T05:45:26.141 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.142 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-10T05:45:26.143 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "9283", 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.144 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "stale_cache_strategy": { 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stale_cache_strategy", 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "log", 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.145 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "default", 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.146 INFO:teuthology.orchestra.run.smithi131.stdout: "default", 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "error" 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.147 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.148 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.149 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.150 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.151 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.152 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.153 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.154 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.154 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.154 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.154 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.154 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.154 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.154 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.154 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.154 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.154 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_snap_create": { 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_snap_create", 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.155 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "mirror_snapshot_schedule": { 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.156 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "trash_purge_schedule": { 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "trash_purge_schedule", 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.157 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "restful", 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.158 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_auth": { 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_auth", 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.159 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.160 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.161 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.162 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.163 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.164 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.165 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.166 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.167 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.168 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rgw", 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.169 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.170 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.171 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.172 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.173 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.174 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.174 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.174 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.174 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.174 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.174 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.174 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.174 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.174 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.174 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.175 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rook", 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.176 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.177 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.178 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.179 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.180 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.180 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.180 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.180 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.180 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.180 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.180 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.182 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.182 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.182 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.182 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.182 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.182 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.182 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.182 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.182 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.183 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "storage_class": { 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "storage_class", 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "local", 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.184 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "selftest", 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.185 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.186 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.187 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.188 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.189 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.190 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.191 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "roption1": { 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption1", 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.192 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "roption2": { 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption2", 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "xyz", 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.193 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption1": { 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption1", 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.194 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption2": { 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption2", 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.195 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption3": { 2024-05-10T05:45:26.196 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption3", 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.197 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption4": { 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption4", 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.198 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption5": { 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption5", 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.199 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption6": { 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption6", 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.200 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption7": { 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption7", 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.201 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "42", 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "testkey": { 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testkey", 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.202 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "testlkey": { 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testlkey", 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.203 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "testnewline": { 2024-05-10T05:45:26.204 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testnewline", 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.205 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_m_granularity": { 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_m_granularity", 2024-05-10T05:45:26.206 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.207 INFO:teuthology.orchestra.run.smithi131.stdout: "dump_on_update": { 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dump_on_update", 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "dump database to debug log on update", 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.208 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.209 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.211 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.212 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.213 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.214 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stats", 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.215 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.216 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.217 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.218 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.219 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.220 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "status", 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.221 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.222 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.223 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.224 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.225 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.226 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.227 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telegraf", 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "address": { 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "address", 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T05:45:26.228 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.229 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15", 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.230 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.231 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.232 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.233 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.234 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.235 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telemetry", 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.236 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_basic": { 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_basic", 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T05:45:26.237 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_crash": { 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_crash", 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.238 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_device": { 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_device", 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.239 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_ident": { 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_ident", 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.240 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_perf": { 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_perf", 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.241 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "contact": { 2024-05-10T05:45:26.242 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "contact", 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.243 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "description": { 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "description", 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.244 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "device_url": { 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_url", 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.245 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.246 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "24", 2024-05-10T05:45:26.247 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "8", 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "last_opt_revision": { 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "last_opt_revision", 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.248 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1", 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard": { 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard", 2024-05-10T05:45:26.249 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.250 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard_description": { 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard_description", 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.251 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.252 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.253 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.254 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.255 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.256 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.257 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "organization": { 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "organization", 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.258 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "proxy": { 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "proxy", 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.259 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "url": { 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url", 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.260 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "test_orchestrator", 2024-05-10T05:45:26.261 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.262 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.263 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.264 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.265 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.266 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.267 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.268 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.269 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.270 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.271 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.272 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.273 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.274 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_clones": { 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_clones", 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4", 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.275 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_delay": { 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_delay", 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.276 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_no_wait": { 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.277 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix", 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.278 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "discovery_interval": { 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "discovery_interval", 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "100", 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.279 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "identifier": { 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "identifier", 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.280 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.281 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.282 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.283 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.284 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.285 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.286 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.286 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.286 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.286 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.286 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.286 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.286 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.286 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.286 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.286 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.287 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_host": { 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_host", 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.288 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_port": { 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_port", 2024-05-10T05:45:26.289 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10051", 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.290 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_sender": { 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_sender", 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.291 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: "modules": [ 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: "dashboard", 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: "iostat", 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: "nfs", 2024-05-10T05:45:26.292 INFO:teuthology.orchestra.run.smithi131.stdout: "restful" 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: "available_modules": [ 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "alerts", 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.293 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.294 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.295 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.296 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.297 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.298 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.299 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_destination": { 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_destination", 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.300 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email address to send alerts to", 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_from_name": { 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_from_name", 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.301 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "Ceph", 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Email From: name", 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.302 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_host": { 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_host", 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP server", 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.303 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_password": { 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_password", 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Password to authenticate with", 2024-05-10T05:45:26.304 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_port": { 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_port", 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "465", 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.305 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP port", 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_sender": { 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_sender", 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.306 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SMTP envelope sender", 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_ssl": { 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_ssl", 2024-05-10T05:45:26.307 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.308 INFO:teuthology.orchestra.run.smithi131.stdout: "smtp_user": { 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "smtp_user", 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "User to authenticate as", 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.309 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "balancer", 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: "active": { 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "active", 2024-05-10T05:45:26.310 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.311 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_time": { 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_time", 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0000", 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.312 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "begin_weekday": { 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "begin_weekday", 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T05:45:26.313 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_max_iterations": { 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_max_iterations", 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "25", 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "250", 2024-05-10T05:45:26.314 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_metrics": { 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_metrics", 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.315 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "crush_compat_step": { 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crush_compat_step", 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.316 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.5", 2024-05-10T05:45:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0.001", 2024-05-10T05:45:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "0.999", 2024-05-10T05:45:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "aggressiveness of optimization", 2024-05-10T05:45:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T05:45:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: "end_time": { 2024-05-10T05:45:26.317 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_time", 2024-05-10T05:45:26.318 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.318 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.318 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.318 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2359", 2024-05-10T05:45:26.318 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.318 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.318 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.318 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T05:45:26.318 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T05:45:26.318 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: "end_weekday": { 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "end_weekday", 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "0", 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "6", 2024-05-10T05:45:26.319 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T05:45:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T05:45:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.320 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.321 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.322 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.323 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.324 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.325 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "min_score": { 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_score", 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.326 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.327 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "upmap", 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "crush-compat", 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "none", 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap" 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Balancer mode", 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.328 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_ids": { 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_ids", 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T05:45:26.329 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.330 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_deviation": { 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_deviation", 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:26.331 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "upmap_max_optimizations": { 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "upmap_max_optimizations", 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.332 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.333 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephadm", 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_down_multiplier": { 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_down_multiplier", 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.334 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_refresh_rate": { 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_refresh_rate", 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "20", 2024-05-10T05:45:26.335 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "agent_starting_port": { 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "agent_starting_port", 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.336 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4721", 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_ptrace": { 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_ptrace", 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.337 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_interval": { 2024-05-10T05:45:26.338 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_interval", 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.339 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "autotune_memory_target_ratio": { 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0.7", 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.340 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "cgroups_split": { 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cgroups_split", 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T05:45:26.341 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "config_checks_enabled": { 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_checks_enabled", 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.342 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "config_dashboard": { 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_dashboard", 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.343 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_alertmanager": { 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_alertmanager", 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.344 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_base": { 2024-05-10T05:45:26.345 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_base", 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Container image name, without the tag", 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.346 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_elasticsearch": { 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_elasticsearch", 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "elasticsearch container image", 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.347 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_grafana": { 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_grafana", 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.348 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_haproxy": { 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_haproxy", 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.349 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "HAproxy container image", 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_agent": { 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_agent", 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.350 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger agent container image", 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_collector": { 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_collector", 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.351 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger collector container image", 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_jaeger_query": { 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_jaeger_query", 2024-05-10T05:45:26.352 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Jaeger query container image", 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.353 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_keepalived": { 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_keepalived", 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Keepalived container image", 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.354 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_loki": { 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_loki", 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Loki container image", 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.355 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_node_exporter": { 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_node_exporter", 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.356 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_nvmeof": { 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_nvmeof", 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.357 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Nvme-of container image", 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_prometheus": { 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_prometheus", 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.358 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Prometheus container image", 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_promtail": { 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_promtail", 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.359 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Promtail container image", 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "container_image_snmp_gateway": { 2024-05-10T05:45:26.360 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_image_snmp_gateway", 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "SNMP Gateway container image", 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.361 INFO:teuthology.orchestra.run.smithi131.stdout: "container_init": { 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "container_init", 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.362 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "daemon_cache_timeout": { 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "daemon_cache_timeout", 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.363 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "default_cephadm_command_timeout": { 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "900", 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.364 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "default_registry": { 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "default_registry", 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "docker.io", 2024-05-10T05:45:26.365 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: "device_cache_timeout": { 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_cache_timeout", 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.366 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1800", 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache device inventory", 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "device_enhanced_scan": { 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_enhanced_scan", 2024-05-10T05:45:26.367 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.368 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "facts_cache_timeout": { 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "facts_cache_timeout", 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "seconds to cache host facts data", 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.369 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "host_check_interval": { 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "host_check_interval", 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to perform a host check", 2024-05-10T05:45:26.370 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "hw_monitoring": { 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hw_monitoring", 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "inventory_list_all": { 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "inventory_list_all", 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.372 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.373 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.374 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: "log_refresh_metadata": { 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_refresh_metadata", 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.375 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.376 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.377 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.378 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.379 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T05:45:26.380 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.381 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "*", 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.382 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "max_count_per_host": { 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_count_per_host", 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of daemons per service per host", 2024-05-10T05:45:26.383 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "max_osd_draining_count": { 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_osd_draining_count", 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.384 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "migration_current": { 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "migration_current", 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.385 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "internal - do not modify", 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": { 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mode", 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.386 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "root", 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm-package", 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: "root" 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.387 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "oob_default_addr": { 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "oob_default_addr", 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "169.254.1.1", 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.388 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "prometheus_alerts_path": { 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus_alerts_path", 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T05:45:26.389 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_insecure": { 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_insecure", 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.390 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_password": { 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_password", 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.391 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_url": { 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_url", 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.392 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.393 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.393 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.393 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.393 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.393 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T05:45:26.393 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.393 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.393 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.393 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.394 INFO:teuthology.orchestra.run.smithi131.stdout: "registry_username": { 2024-05-10T05:45:26.394 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "registry_username", 2024-05-10T05:45:26.394 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.394 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.394 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.394 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.394 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.395 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.395 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.395 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T05:45:26.395 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.395 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.395 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.395 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.396 INFO:teuthology.orchestra.run.smithi131.stdout: "secure_monitoring_stack": { 2024-05-10T05:45:26.396 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "secure_monitoring_stack", 2024-05-10T05:45:26.396 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.396 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.396 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.396 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.396 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.397 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.397 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.397 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T05:45:26.397 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.397 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.398 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.398 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.398 INFO:teuthology.orchestra.run.smithi131.stdout: "service_discovery_port": { 2024-05-10T05:45:26.398 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "service_discovery_port", 2024-05-10T05:45:26.398 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.398 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.399 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.399 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8765", 2024-05-10T05:45:26.399 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.399 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.399 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.399 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "cephadm service discovery port", 2024-05-10T05:45:26.400 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.400 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.400 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.400 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.400 INFO:teuthology.orchestra.run.smithi131.stdout: "ssh_config_file": { 2024-05-10T05:45:26.400 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssh_config_file", 2024-05-10T05:45:26.400 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.401 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.401 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.401 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.401 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.401 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.401 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.401 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T05:45:26.402 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.402 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.402 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.402 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.402 INFO:teuthology.orchestra.run.smithi131.stdout: "use_agent": { 2024-05-10T05:45:26.402 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_agent", 2024-05-10T05:45:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T05:45:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.403 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "use_repo_digest": { 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "use_repo_digest", 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.404 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_failed_host_check": { 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_failed_host_check", 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.405 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_daemons": { 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_daemons", 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.406 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_on_stray_hosts": { 2024-05-10T05:45:26.407 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_on_stray_hosts", 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.408 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crash", 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.409 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.410 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.411 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.412 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.413 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.414 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "retain_interval": { 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retain_interval", 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "31536000", 2024-05-10T05:45:26.415 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_recent_interval": { 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_recent_interval", 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.416 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1209600", 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.417 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dashboard", 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:26.418 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.419 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T05:45:26.420 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.421 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_ENABLED": { 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.422 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.423 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.423 INFO:teuthology.orchestra.run.smithi131.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T05:45:26.423 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T05:45:26.423 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.423 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.423 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.423 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.423 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.424 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.425 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.425 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.425 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.425 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.425 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.425 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.425 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.425 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.425 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.426 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.427 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.427 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.427 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.427 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.427 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T05:45:26.427 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T05:45:26.427 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.427 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.427 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.427 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T05:45:26.428 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T05:45:26.429 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.429 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.429 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.429 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.429 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.429 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.429 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.429 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.429 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.430 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.431 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.431 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.431 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.431 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.431 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.431 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.431 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T05:45:26.431 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T05:45:26.432 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.432 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.432 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.432 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.432 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.432 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.432 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.432 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.432 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.433 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.433 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.433 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.433 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T05:45:26.433 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T05:45:26.433 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.433 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.433 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.433 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.433 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.434 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.434 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.434 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.434 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.434 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.434 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.434 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.434 INFO:teuthology.orchestra.run.smithi131.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T05:45:26.435 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T05:45:26.435 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.435 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.435 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.435 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.435 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.435 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.435 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.435 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.436 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.437 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.438 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.439 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_URL": { 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_URL", 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.440 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.441 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.442 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.443 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T05:45:26.444 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T05:45:26.445 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.446 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.447 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.448 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.449 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.450 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.451 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T05:45:26.452 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T05:45:26.453 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.454 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.455 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.456 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.457 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.458 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.459 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T05:45:26.460 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8", 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.461 INFO:teuthology.orchestra.run.smithi131.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "45", 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.462 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.463 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "admin", 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.464 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.465 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.466 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T05:45:26.467 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.468 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.469 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.470 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.471 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: "cross_origin_url": { 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cross_origin_url", 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.472 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "crt_file": { 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "crt_file", 2024-05-10T05:45:26.473 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.474 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "debug": { 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "debug", 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Enable/disable debug options", 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.475 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "jwt_token_ttl": { 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "jwt_token_ttl", 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "28800", 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.478 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.479 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.480 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.481 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.482 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.483 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.484 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.484 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.484 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.484 INFO:teuthology.orchestra.run.smithi131.stdout: "motd": { 2024-05-10T05:45:26.484 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "motd", 2024-05-10T05:45:26.484 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.484 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.484 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.484 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.484 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "The message of the day", 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect_resolve_ip_addr": { 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.485 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-10T05:45:26.486 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-10T05:45:26.487 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8080", 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.488 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.489 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl_server_port": { 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl_server_port", 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8443", 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.490 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "redirect", 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.491 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: "redirect" 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.492 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.493 INFO:teuthology.orchestra.run.smithi131.stdout: "url_prefix": { 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url_prefix", 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.494 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_monitoring": { 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_monitoring", 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.495 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "monitor device health metrics", 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.496 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.497 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.498 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.500 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.501 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: "mark_out_threshold": { 2024-05-10T05:45:26.502 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mark_out_threshold", 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "2419200", 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.503 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "pool_name": { 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pool_name", 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "device_health_metrics", 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.504 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "retention_period": { 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "retention_period", 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15552000", 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.505 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long to retain device health metrics", 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_frequency": { 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_frequency", 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.506 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "self_heal": { 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "self_heal", 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.507 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.508 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "warn_threshold": { 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "warn_threshold", 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.509 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7257600", 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.510 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "diskprediction_local", 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.511 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.512 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.513 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.514 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.515 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.516 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "predict_interval": { 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predict_interval", 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "86400", 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.517 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "predictor_model": { 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "predictor_model", 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.518 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "prophetstor", 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:26.519 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "600", 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.520 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "influx", 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": false, 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "influxdb python module not found", 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "batch_size": { 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "batch_size", 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.521 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5000", 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "database": { 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "database", 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.522 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "ceph", 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": { 2024-05-10T05:45:26.523 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "hostname", 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server hostname", 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.524 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "30", 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "5", 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.525 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.526 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.527 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.528 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.530 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.531 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.532 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.532 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.532 INFO:teuthology.orchestra.run.smithi131.stdout: "password": { 2024-05-10T05:45:26.532 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "password", 2024-05-10T05:45:26.532 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.532 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.532 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.532 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "password of InfluxDB server user", 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "port": { 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "port", 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "8086", 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "InfluxDB server port", 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "ssl": { 2024-05-10T05:45:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ssl", 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "false", 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.535 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "threads": { 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threads", 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "32", 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.536 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "username": { 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "username", 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.537 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "username of InfluxDB server user", 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "verify_ssl": { 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "verify_ssl", 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "true", 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "insights", 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.540 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.541 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.542 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.543 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.544 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.545 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "iostat", 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.546 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.547 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.548 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.549 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.550 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.551 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "k8sevents", 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_event_retention_days": { 2024-05-10T05:45:26.552 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "ceph_event_retention_days", 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "7", 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.553 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "config_check_secs": { 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "config_check_secs", 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "10", 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.554 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.555 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.556 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.557 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.558 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.559 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.560 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "localpool", 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: "failure_domain": { 2024-05-10T05:45:26.561 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "failure_domain", 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "host", 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "failure domain for any created local pool", 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.562 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.563 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.564 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.565 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.566 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.567 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.568 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "min_size": { 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "min_size", 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default min_size for any created local pool", 2024-05-10T05:45:26.569 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "num_rep": { 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "num_rep", 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3", 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.570 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default replica count for any created local pool", 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_num": { 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_num", 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.571 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "128", 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "prefix": { 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prefix", 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.572 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "name prefix for any created local pool", 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "subtree": { 2024-05-10T05:45:26.573 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "subtree", 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "rack", 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.574 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mds_autoscaler", 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.575 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.576 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.577 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.578 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.579 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.580 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirroring", 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.581 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.582 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.583 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.584 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.585 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.586 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.587 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.587 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.587 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.587 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.587 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.587 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.587 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.587 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.587 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.590 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.590 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.590 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "nfs", 2024-05-10T05:45:26.590 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.590 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.590 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.590 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.590 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.591 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.592 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.593 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.594 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.595 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "fail_fs": { 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "fail_fs", 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.596 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.597 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.598 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.599 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.600 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.601 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator": { 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "orchestrator", 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.602 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: "cephadm", 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: "rook", 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: "test_orchestrator" 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Orchestrator backend", 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.603 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_perf_query", 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.604 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.605 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.606 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.607 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.608 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "osd_support", 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.609 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.610 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.611 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.612 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.613 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "pg_autoscaler", 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.614 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.615 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.616 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.617 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.619 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "threshold": { 2024-05-10T05:45:26.620 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "threshold", 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "3.0", 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1.0", 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "scaling threshold", 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.621 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "progress", 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_pg_recovery_event": { 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_pg_recovery_event", 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.622 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.623 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.624 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.625 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.626 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.627 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.628 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "max_completed_events": { 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_completed_events", 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "50", 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "number of past completed events to remember", 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.629 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "sleep_interval": { 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "sleep_interval", 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "5", 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "how long the module is going to sleep", 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.630 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "prometheus", 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: "cache": { 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cache", 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.631 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "exclude_perf_counters": { 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "exclude_perf_counters", 2024-05-10T05:45:26.632 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.633 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.633 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.633 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.633 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.633 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.633 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.633 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T05:45:26.633 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T05:45:26.634 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.634 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.634 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.634 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.634 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.634 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.634 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.635 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.635 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.635 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.635 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.635 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.635 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.635 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.636 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.636 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.636 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.636 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.636 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.636 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.637 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.637 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.637 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.637 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.637 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.637 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.637 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.638 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.638 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.638 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.638 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.638 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.638 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.638 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.639 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.639 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.639 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.639 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.639 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.639 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.640 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.640 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.640 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.640 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.640 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.640 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.640 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.641 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.641 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.641 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.641 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.641 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.641 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.641 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.642 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools": { 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools", 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.643 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "300", 2024-05-10T05:45:26.644 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "scrape_interval": { 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "scrape_interval", 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15.0", 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.645 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "::", 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.646 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "9283", 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.647 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "stale_cache_strategy": { 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stale_cache_strategy", 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "log", 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.648 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_behaviour": { 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_behaviour", 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "default", 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.649 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "default", 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "error" 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "standby_error_status_code": { 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "standby_error_status_code", 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.650 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "500", 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "400", 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "599", 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.651 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.652 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.653 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.654 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.655 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.656 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_snap_create": { 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_snap_create", 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10", 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.657 INFO:teuthology.orchestra.run.smithi131.stdout: "mirror_snapshot_schedule": { 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.658 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "trash_purge_schedule": { 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "trash_purge_schedule", 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.659 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "restful", 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "enable_auth": { 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enable_auth", 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.660 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "key_file": { 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "key_file", 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.661 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.662 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.663 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.664 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.665 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.666 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "server_addr": { 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_addr", 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.667 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "server_port": { 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "server_port", 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.668 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rgw", 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.669 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.670 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.671 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.672 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.673 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rook", 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.675 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.676 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.677 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.678 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.679 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "storage_class": { 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "storage_class", 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "local", 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.680 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "selftest", 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.681 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.682 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.683 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.684 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.685 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.686 INFO:teuthology.orchestra.run.smithi131.stdout: "roption1": { 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption1", 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.687 INFO:teuthology.orchestra.run.smithi131.stdout: "roption2": { 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "roption2", 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "xyz", 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.688 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption1": { 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption1", 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption2": { 2024-05-10T05:45:26.689 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption2", 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption3": { 2024-05-10T05:45:26.690 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption3", 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "float", 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption4": { 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption4", 2024-05-10T05:45:26.691 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption5": { 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption5", 2024-05-10T05:45:26.692 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption6": { 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption6", 2024-05-10T05:45:26.693 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "rwoption7": { 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rwoption7", 2024-05-10T05:45:26.694 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "1", 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "42", 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "testkey": { 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testkey", 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.695 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "testlkey": { 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testlkey", 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.696 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "testnewline": { 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "testnewline", 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.698 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snap_schedule", 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "allow_m_granularity": { 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "allow_m_granularity", 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T05:45:26.699 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "dump_on_update": { 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "dump_on_update", 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.700 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "dump database to debug log on update", 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.701 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.702 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.704 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.705 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "stats", 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.706 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.707 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.708 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.709 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.710 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.711 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "status", 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.712 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.713 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.714 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.715 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telegraf", 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "address": { 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "address", 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.717 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.718 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "15", 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.719 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.720 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.721 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.722 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.723 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "telemetry", 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.724 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_basic": { 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_basic", 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.725 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_crash": { 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_crash", 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.726 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_device": { 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_device", 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.727 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_ident": { 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_ident", 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.728 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "channel_perf": { 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "channel_perf", 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.729 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "contact": { 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "contact", 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.730 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "description": { 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "description", 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.731 INFO:teuthology.orchestra.run.smithi131.stdout: "device_url": { 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "device_url", 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.732 INFO:teuthology.orchestra.run.smithi131.stdout: "enabled": { 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "enabled", 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.733 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "24", 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "8", 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.734 INFO:teuthology.orchestra.run.smithi131.stdout: "last_opt_revision": { 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "last_opt_revision", 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "1", 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.735 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard": { 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard", 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.736 INFO:teuthology.orchestra.run.smithi131.stdout: "leaderboard_description": { 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "leaderboard_description", 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.737 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.738 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.739 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.740 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.741 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.742 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "organization": { 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "organization", 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.743 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "proxy": { 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "proxy", 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.744 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "url": { 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "url", 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.745 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "test_orchestrator", 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.746 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.747 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.748 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.749 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.750 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.751 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.752 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.753 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.754 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.755 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.756 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "max_concurrent_clones": { 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "max_concurrent_clones", 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "4", 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_delay": { 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_delay", 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "0", 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.758 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "snapshot_clone_no_wait": { 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "True", 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.759 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix", 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "can_run": true, 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "error_string": "", 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "module_options": { 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "discovery_interval": { 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "discovery_interval", 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "uint", 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.760 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "100", 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "identifier": { 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "identifier", 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.761 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "interval": { 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "interval", 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "secs", 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.762 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "60", 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "log_level": { 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_level", 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.763 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.764 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster": { 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster", 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.765 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_cluster_level": { 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_cluster_level", 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "info", 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [ 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "", 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "critical", 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "debug", 2024-05-10T05:45:26.766 INFO:teuthology.orchestra.run.smithi131.stdout: "error", 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "info", 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "warning" 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "log_to_file": { 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "log_to_file", 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "bool", 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 1, 2024-05-10T05:45:26.767 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "False", 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_host": { 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_host", 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.768 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "", 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_port": { 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_port", 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "int", 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.769 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "10051", 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "zabbix_sender": { 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "zabbix_sender", 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "str", 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "level": "advanced", 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "flags": 0, 2024-05-10T05:45:26.770 INFO:teuthology.orchestra.run.smithi131.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "min": "", 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "max": "", 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "enum_allowed": [], 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "desc": "", 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "long_desc": "", 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "tags": [], 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "see_also": [] 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "services": { 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: "dashboard": "https://172.21.15.131:8443/" 2024-05-10T05:45:26.771 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "always_on_modules": { 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "octopus": [ 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.772 INFO:teuthology.orchestra.run.smithi131.stdout: "pacific": [ 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "quincy": [ 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-05-10T05:45:26.773 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "reef": [ 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "balancer", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "crash", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "devicehealth", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "orchestrator", 2024-05-10T05:45:26.774 INFO:teuthology.orchestra.run.smithi131.stdout: "pg_autoscaler", 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "progress", 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "rbd_support", 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "status", 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "telemetry", 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "volumes" 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "last_failure_osd_epoch": 3, 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "active_clients": [ 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "devicehealth", 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-10T05:45:26.775 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 19029692 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "libcephsqlite", 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 50255390 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-10T05:45:26.776 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "rbd_support", 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 980393774 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "volumes", 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "addrvec": [ 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-10T05:45:26.777 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v2", 2024-05-10T05:45:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "172.21.15.131:0", 2024-05-10T05:45:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1960720190 2024-05-10T05:45:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-10T05:45:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-10T05:45:26.778 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 -- 172.21.15.131:0/4161587326 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1f1c0a11c0 msgr2=0x7f1f1c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1f1c0a11c0 0x7f1f1c0a36d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000b2e0 tx=0x7f1f2001a040 comp rx=0 tx=0).stop 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 -- 172.21.15.131:0/4161587326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f34073ae0 msgr2=0x7f1f34199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f34073ae0 0x7f1f34199f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800e9c0 tx=0x7f1f2800ecd0 comp rx=0 tx=0).stop 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 -- 172.21.15.131:0/4161587326 shutdown_connections 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f1f1c0a11c0 0x7f1f1c0a36d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f34109900 0x7f1f341a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f34073ae0 0x7f1f34199f40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 --2- 172.21.15.131:0/4161587326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f340750e0 0x7f1f3419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 -- 172.21.15.131:0/4161587326 >> 172.21.15.131:0/4161587326 conn(0x7f1f340fd010 msgr2=0x7f1f34105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:26.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 -- 172.21.15.131:0/4161587326 shutdown_connections 2024-05-10T05:45:26.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:25.341+0000 7f1f3aa00700 1 -- 172.21.15.131:0/4161587326 wait complete. 2024-05-10T05:45:26.780 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:25 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2839336273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:45:26.780 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:25 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3493316447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:45:26.781 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:26 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/4161587326' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T05:45:26.781 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:26 smithi157 ceph-mon[78427]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 377 MiB/s wr, 197 op/s 2024-05-10T05:45:26.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:25 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2839336273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:45:26.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:25 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3493316447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:45:26.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:26 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/4161587326' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T05:45:26.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:26 smithi142 ceph-mon[77667]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 377 MiB/s wr, 197 op/s 2024-05-10T05:45:26.822 DEBUG:tasks.check_counter:Getting stats from a 2024-05-10T05:45:26.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-10T05:45:26.823 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:26 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/4161587326' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T05:45:26.823 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:26 smithi131 ceph-mon[37264]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 377 MiB/s wr, 197 op/s 2024-05-10T05:45:27.196 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:45:27.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:27 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:45:27.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:27 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:45:27.802 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:27 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T05:45:28.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:28 smithi142 ceph-mon[77667]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 319 MiB/s wr, 191 op/s 2024-05-10T05:45:28.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:28 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:45:28.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:28 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:45:28.645 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:28 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:45:28.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:28 smithi131 ceph-mon[37264]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 319 MiB/s wr, 191 op/s 2024-05-10T05:45:28.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:45:28.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:45:28.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:28 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:45:28.714 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 459, 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 630, 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 147928, 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 172484, 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.146380706, 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.061614833, 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.064596677, 2024-05-10T05:45:28.715 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.007349990, 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 632, 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.262081768, 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000414686 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1022, 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000478734, 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000468 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.716 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 77, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 243, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 46360, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 439835, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 10, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.056422680, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.041097141, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.011804527, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000422599, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 242, 2024-05-10T05:45:28.717 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.152638154, 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000630736 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 110, 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000026696, 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000242 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 2768, 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 369056 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 369, 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 361, 2024-05-10T05:45:28.718 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 578282, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 540756, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 47, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 8, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.153713861, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.028651543, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.094461289, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.022101098, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 356, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.129009307, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000362385 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 153, 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000056638, 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000370 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 532224, 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 132624 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-10T05:45:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 36, 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.002468601, 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000068572 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000026088, 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000013044 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.721 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 11, 2024-05-10T05:45:28.722 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 10, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 10, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 10, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-10T05:45:28.723 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 22, 2024-05-10T05:45:28.724 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 10738148731, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T05:45:28.725 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 1, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-10T05:45:28.726 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-10T05:45:28.727 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 3, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 3, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 1, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-10T05:45:28.728 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 1, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4195378, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.006500525, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.002166841 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.729 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 13, 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 13, 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 11, 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 12, 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 1, 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 10, 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 10, 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-05-10T05:45:28.730 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 118144, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 199060 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 4, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.731 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.732 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-10T05:45:28.733 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.734 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.735 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.736 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-10T05:45:28.737 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.738 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.739 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-10T05:45:28.740 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.741 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 0, 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 1, 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 0, 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 76, 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-10T05:45:28.743 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:28.744 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:28.745 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 222023, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 72, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-10T05:45:28.746 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7544, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 160, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 50040, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 198, 2024-05-10T05:45:28.747 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 38, 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 11286, 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 27, 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 38, 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-10T05:45:28.748 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 38, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.293994089, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.007736686 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 38, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 60 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 38, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 0, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 38, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-10T05:45:28.749 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 0, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 3, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 6, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 6, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-05-10T05:45:28.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-10T05:45:28.751 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-05-10T05:45:28.752 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 2, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 8, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 8, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 14, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 1 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-10T05:45:28.753 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:28.754 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 902, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 675174, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 902, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 902, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 675174, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.755 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 38, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 11286, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 38, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 11286, 2024-05-10T05:45:28.756 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 38, 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 38, 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 38, 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 38, 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:28.758 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x5575de99a4e0": { 2024-05-10T05:45:28.759 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 3, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 1074, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 3, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1074, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:28.760 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:28.761 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:28.761 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:28.761 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:45:28.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:28 smithi157 ceph-mon[78427]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 319 MiB/s wr, 191 op/s 2024-05-10T05:45:28.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:28 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T05:45:28.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:28 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T05:45:28.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:28 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' 2024-05-10T05:45:29.247 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-10T05:45:29.247 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-10T05:45:29.247 DEBUG:tasks.check_counter:Getting stats from d 2024-05-10T05:45:29.248 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-10T05:45:29.973 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:45:30.740 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:30 smithi131 ceph-mon[37264]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 301 MiB/s wr, 196 op/s 2024-05-10T05:45:30.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:30 smithi142 ceph-mon[77667]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 301 MiB/s wr, 196 op/s 2024-05-10T05:45:31.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:30 smithi157 ceph-mon[78427]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 301 MiB/s wr, 196 op/s 2024-05-10T05:45:32.153 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:45:32.153 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:32.153 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1631, 2024-05-10T05:45:32.153 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1633, 2024-05-10T05:45:32.153 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 513878, 2024-05-10T05:45:32.153 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 507242, 2024-05-10T05:45:32.153 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-05-10T05:45:32.153 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-05-10T05:45:32.153 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.405936892, 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.078880334, 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.066543666, 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.198045148, 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1631, 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.569723980, 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000349309 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.154 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 845, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 838, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 664224, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 318358, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.260744830, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.040906123, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.057032887, 2024-05-10T05:45:32.155 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.142052914, 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 837, 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.313263015, 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000374268 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 514496, 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 129824 2024-05-10T05:45:32.156 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 6, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 175, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 3372, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 371639, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.024005690, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.019133579, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.002800279, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000020562, 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:32.157 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 174, 2024-05-10T05:45:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.074533597, 2024-05-10T05:45:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000428354 2024-05-10T05:45:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.158 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 371328 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3557, 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.307180307, 2024-05-10T05:45:32.159 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000086359 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-05-10T05:45:32.160 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 3, 2024-05-10T05:45:32.161 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 3, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 3, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-05-10T05:45:32.162 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 0, 2024-05-10T05:45:32.163 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T05:45:32.164 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-10T05:45:32.165 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T05:45:32.166 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-10T05:45:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-10T05:45:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-10T05:45:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-10T05:45:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-10T05:45:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-10T05:45:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 0, 2024-05-10T05:45:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-10T05:45:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-10T05:45:32.167 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 15, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 1, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4220955, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 4219304, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.168 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 15 2024-05-10T05:45:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-10T05:45:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 4, 2024-05-10T05:45:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 16, 2024-05-10T05:45:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 12, 2024-05-10T05:45:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 2, 2024-05-10T05:45:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 13, 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 11, 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 3, 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 15, 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 12, 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 116396, 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 207252 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-10T05:45:32.170 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-10T05:45:32.171 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.172 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.173 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.174 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-10T05:45:32.174 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.174 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.174 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.174 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.174 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-10T05:45:32.174 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.174 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.174 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.174 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.175 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-10T05:45:32.175 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.175 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.175 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.175 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.175 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-10T05:45:32.175 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.175 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.175 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.175 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.176 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-10T05:45:32.177 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.178 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.178 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.178 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.178 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-10T05:45:32.178 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.178 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.178 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.178 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.178 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.179 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-10T05:45:32.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-10T05:45:32.182 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.182 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.182 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.182 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.182 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-10T05:45:32.182 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.182 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.182 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.182 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-10T05:45:32.184 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 2319, 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:32.185 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:32.186 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:32.187 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 189400, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 51, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:32.188 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7544, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 160, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 13904, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 40, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.189 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 2238, 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 21, 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 3, 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 2238, 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2238, 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 2.899942201, 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001295773 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-10T05:45:32.190 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2238, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 2239 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 2238, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 2238, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 0, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 1414, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 822, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 0, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 0, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-10T05:45:32.191 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 1, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-10T05:45:32.192 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-10T05:45:32.193 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 2, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 5, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 5, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 0, 2024-05-10T05:45:32.194 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 2, 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 0 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T05:45:32.195 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2478, 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 943216, 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:32.196 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2478, 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2478, 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 943216, 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-10T05:45:32.197 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 2238, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 26656, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2238, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 26656, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.198 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 2238, 2024-05-10T05:45:32.199 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 2238, 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2238, 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 2238, 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-10T05:45:32.200 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x563f7c1628f0": { 2024-05-10T05:45:32.201 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:32.202 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:32.203 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:32.203 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:45:32.628 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-05-10T05:45:32.628 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-05-10T05:45:32.628 DEBUG:tasks.check_counter:Getting stats from g 2024-05-10T05:45:32.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-10T05:45:32.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:32 smithi142 ceph-mon[77667]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 231 MiB/s wr, 195 op/s 2024-05-10T05:45:32.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:32 smithi131 ceph-mon[37264]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 231 MiB/s wr, 195 op/s 2024-05-10T05:45:33.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:32 smithi157 ceph-mon[78427]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 231 MiB/s wr, 195 op/s 2024-05-10T05:45:33.284 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:45:34.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:34 smithi142 ceph-mon[77667]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 213 MiB/s wr, 262 op/s 2024-05-10T05:45:34.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:34 smithi131 ceph-mon[37264]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 213 MiB/s wr, 262 op/s 2024-05-10T05:45:35.052 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:34 smithi157 ceph-mon[78427]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 213 MiB/s wr, 262 op/s 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 728, 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 890, 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 682357, 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 423753, 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.189008712, 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.062851397, 2024-05-10T05:45:35.993 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.091893204, 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.019830748, 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 889, 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.399215683, 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000449061 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1041, 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000519931, 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000499 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 535232, 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 132912 2024-05-10T05:45:35.994 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 90, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 257, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 49874, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 438488, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 13, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.082858046, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.058488480, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.020528313, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000307902, 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:35.995 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 256, 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.163254429, 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000637712 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 120, 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000072232, 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000601 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 4032, 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 373280 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 111, 2024-05-10T05:45:35.996 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 113, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 49409, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 418315, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 46, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 7, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.095864816, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.013377175, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.107712372, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000336610, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 111, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.106848546, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000962599 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.997 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 148, 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000046093, 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000311 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-10T05:45:35.998 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 42, 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.002808984, 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000066880 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000218977, 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000109488 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:35.999 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-10T05:45:36.000 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 1, 2024-05-10T05:45:36.000 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-05-10T05:45:36.000 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-10T05:45:36.000 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.000 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.000 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.000 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.000 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-10T05:45:36.000 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 1, 2024-05-10T05:45:36.000 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-10T05:45:36.001 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-10T05:45:36.001 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 11, 2024-05-10T05:45:36.001 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-10T05:45:36.001 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-10T05:45:36.002 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 10, 2024-05-10T05:45:36.002 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-10T05:45:36.002 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 10, 2024-05-10T05:45:36.002 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-10T05:45:36.002 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 10, 2024-05-10T05:45:36.002 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-10T05:45:36.002 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-10T05:45:36.002 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-10T05:45:36.002 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 1000, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 1, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-10T05:45:36.003 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-10T05:45:36.004 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-05-10T05:45:36.005 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-05-10T05:45:36.005 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-10T05:45:36.006 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-10T05:45:36.006 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 1, 2024-05-10T05:45:36.006 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 22, 2024-05-10T05:45:36.006 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 10738148731, 2024-05-10T05:45:36.006 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-10T05:45:36.007 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T05:45:36.007 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-10T05:45:36.007 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T05:45:36.007 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-10T05:45:36.007 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-10T05:45:36.007 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T05:45:36.008 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T05:45:36.008 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-10T05:45:36.008 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-10T05:45:36.008 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T05:45:36.008 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-05-10T05:45:36.008 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T05:45:36.008 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T05:45:36.009 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T05:45:36.009 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-05-10T05:45:36.009 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-10T05:45:36.009 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-05-10T05:45:36.009 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-05-10T05:45:36.009 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.010 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-10T05:45:36.010 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-10T05:45:36.010 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-10T05:45:36.011 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-10T05:45:36.011 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 1, 2024-05-10T05:45:36.011 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-10T05:45:36.011 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-10T05:45:36.011 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-10T05:45:36.012 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-10T05:45:36.012 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-10T05:45:36.012 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-10T05:45:36.012 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-10T05:45:36.012 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-10T05:45:36.012 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-10T05:45:36.013 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-10T05:45:36.013 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-10T05:45:36.013 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-10T05:45:36.013 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-10T05:45:36.013 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T05:45:36.014 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-10T05:45:36.014 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-10T05:45:36.014 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-10T05:45:36.014 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-10T05:45:36.014 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-10T05:45:36.015 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.015 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-10T05:45:36.015 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 4, 2024-05-10T05:45:36.016 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-10T05:45:36.016 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-10T05:45:36.016 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 4, 2024-05-10T05:45:36.016 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-10T05:45:36.016 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-10T05:45:36.016 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 1, 2024-05-10T05:45:36.017 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-10T05:45:36.017 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-10T05:45:36.017 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 1, 2024-05-10T05:45:36.017 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-10T05:45:36.017 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-10T05:45:36.018 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-10T05:45:36.018 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4195688, 2024-05-10T05:45:36.018 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-10T05:45:36.018 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-10T05:45:36.018 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4, 2024-05-10T05:45:36.018 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.008057637, 2024-05-10T05:45:36.019 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.002014409 2024-05-10T05:45:36.019 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.019 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-10T05:45:36.019 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.019 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-10T05:45:36.019 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 13, 2024-05-10T05:45:36.019 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 13, 2024-05-10T05:45:36.019 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-05-10T05:45:36.020 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 11, 2024-05-10T05:45:36.020 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 12, 2024-05-10T05:45:36.020 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 1, 2024-05-10T05:45:36.020 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 10, 2024-05-10T05:45:36.020 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 10, 2024-05-10T05:45:36.020 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-05-10T05:45:36.020 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-05-10T05:45:36.020 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-05-10T05:45:36.020 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-05-10T05:45:36.021 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 118016, 2024-05-10T05:45:36.021 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 199060 2024-05-10T05:45:36.021 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.021 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-10T05:45:36.021 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 1, 2024-05-10T05:45:36.021 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-10T05:45:36.021 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 1, 2024-05-10T05:45:36.021 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 14, 2024-05-10T05:45:36.021 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-10T05:45:36.022 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-10T05:45:36.022 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.022 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.022 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.022 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.022 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-10T05:45:36.022 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.022 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.022 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-10T05:45:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-10T05:45:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.024 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.024 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-10T05:45:36.024 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.024 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.024 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.024 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.024 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-10T05:45:36.024 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.024 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.025 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.025 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.025 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-10T05:45:36.025 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.025 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.025 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.025 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.026 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-10T05:45:36.026 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.026 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.026 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.026 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.026 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-10T05:45:36.026 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.026 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.026 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.027 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.027 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-10T05:45:36.027 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.027 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.027 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.027 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.027 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-10T05:45:36.027 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.027 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.028 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.028 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.028 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-10T05:45:36.028 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.028 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.028 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.028 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.028 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-10T05:45:36.028 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.029 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.029 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.029 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.029 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-10T05:45:36.029 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.029 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.029 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.029 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.029 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-10T05:45:36.030 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.030 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.030 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.030 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.030 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-10T05:45:36.030 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.030 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.030 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.030 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.031 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-10T05:45:36.031 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.031 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.031 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.031 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.031 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-10T05:45:36.031 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.031 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.032 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.032 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.032 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-10T05:45:36.032 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.032 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.032 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.033 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.033 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-10T05:45:36.033 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.033 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.033 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.033 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.034 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-10T05:45:36.034 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.034 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.034 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.034 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.035 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-10T05:45:36.035 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.035 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.035 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.035 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.035 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-10T05:45:36.036 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.036 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.036 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.036 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.036 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-10T05:45:36.036 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.037 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.037 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.037 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.037 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-10T05:45:36.037 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.037 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.038 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.038 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.038 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-10T05:45:36.038 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.038 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.038 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.039 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.039 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-10T05:45:36.039 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.040 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.040 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.040 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.040 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-10T05:45:36.041 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.041 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.041 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.041 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.042 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-10T05:45:36.042 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.042 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.042 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.043 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.043 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-10T05:45:36.043 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-10T05:45:36.043 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-10T05:45:36.043 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.044 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.044 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.044 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:36.044 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.044 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-10T05:45:36.044 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-10T05:45:36.044 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 2, 2024-05-10T05:45:36.044 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 1, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 716, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:36.045 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:36.046 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-10T05:45:36.047 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 254725, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 89, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:36.048 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7544, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 160, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 50040, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 198, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-10T05:45:36.049 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 40, 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 11686, 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 26, 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 40, 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 40, 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.233995282, 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.005849882 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 40, 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 62 2024-05-10T05:45:36.050 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 40, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 0, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 40, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 0, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 4, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 7, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-10T05:45:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 6, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-10T05:45:36.052 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-10T05:45:36.053 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 2, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 9, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 9, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 14, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-05-10T05:45:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 1 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-10T05:45:36.055 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 925, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 681874, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 925, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-10T05:45:36.056 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 925, 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 681874, 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-10T05:45:36.057 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 40, 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 11686, 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 40, 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 11686, 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:36.058 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 40, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 40, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 40, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 40, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:36.059 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:36.060 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.061 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.061 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.061 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:36.061 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:36.061 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x5617097e24e0": { 2024-05-10T05:45:36.061 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:36.061 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-10T05:45:36.061 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:36.061 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 4, 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 1384, 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 4, 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 4, 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1384, 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:36.062 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:36.063 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:45:36.450 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-10T05:45:36.450 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-10T05:45:36.450 DEBUG:tasks.check_counter:Getting stats from j 2024-05-10T05:45:36.450 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-10T05:45:36.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:36 smithi131 ceph-mon[37264]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 216 MiB/s wr, 398 op/s 2024-05-10T05:45:36.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:36 smithi142 ceph-mon[77667]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 216 MiB/s wr, 398 op/s 2024-05-10T05:45:37.013 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T05:45:37.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:36 smithi157 ceph-mon[78427]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 216 MiB/s wr, 398 op/s 2024-05-10T05:45:38.503 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-10T05:45:38.503 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:38.503 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 0, 2024-05-10T05:45:38.503 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 2, 2024-05-10T05:45:38.503 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 0, 2024-05-10T05:45:38.503 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 0, 2024-05-10T05:45:38.503 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-05-10T05:45:38.503 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 0, 2024-05-10T05:45:38.503 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.003907681, 2024-05-10T05:45:38.503 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.000878379, 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:38.504 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 174, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 160, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 106704, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 3, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.032386400, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.027585331, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.002517190, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:38.505 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 172, 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.053904529, 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000313398 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 106704 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:38.506 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 234, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 229, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 478128, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 130272, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 2, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 1, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.079536881, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.036756833, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.024295627, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.011053712, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 229, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.080015519, 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000349412 2024-05-10T05:45:38.507 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 478128, 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 130272 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 0, 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.508 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:38.509 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:38.510 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 149998, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 30, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-10T05:45:38.511 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 928, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 2, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:38.512 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 231, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 442658, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 231, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-10T05:45:38.513 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 231, 2024-05-10T05:45:38.514 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 442658, 2024-05-10T05:45:38.514 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-10T05:45:38.514 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-10T05:45:38.514 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-10T05:45:38.514 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-10T05:45:38.514 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:38.514 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-10T05:45:38.514 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-10T05:45:38.848 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:38 smithi131 ceph-mon[37264]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 182 MiB/s wr, 475 op/s 2024-05-10T05:45:38.874 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-10T05:45:38.874 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-10T05:45:38.874 DEBUG:tasks.check_counter:Getting stats from b 2024-05-10T05:45:38.874 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-10T05:45:38.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:38 smithi142 ceph-mon[77667]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 182 MiB/s wr, 475 op/s 2024-05-10T05:45:39.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:38 smithi157 ceph-mon[78427]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 182 MiB/s wr, 475 op/s 2024-05-10T05:45:39.126 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:45:40.086 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-10T05:45:40.086 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:40.086 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2199, 2024-05-10T05:45:40.086 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2371, 2024-05-10T05:45:40.086 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 1190641, 2024-05-10T05:45:40.086 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 2751744, 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 9, 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.222584252, 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.066701119, 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.089351729, 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.054315541, 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2375, 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.421821561, 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000177609 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.087 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2951, 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000701984, 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000237 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 548336, 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 155744 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 3218, 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2570, 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 1035836, 2024-05-10T05:45:40.088 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 1202441, 2024-05-10T05:45:40.089 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 13, 2024-05-10T05:45:40.089 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-05-10T05:45:40.089 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.287304036, 2024-05-10T05:45:40.089 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.076722392, 2024-05-10T05:45:40.089 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.165825062, 2024-05-10T05:45:40.089 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.068186110, 2024-05-10T05:45:40.089 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:40.089 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2569, 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.890382426, 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000346587 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2101, 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000780946, 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000371 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 55584, 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 410608 2024-05-10T05:45:40.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.091 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:40.091 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1634, 2024-05-10T05:45:40.091 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1464, 2024-05-10T05:45:40.091 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 389789, 2024-05-10T05:45:40.091 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 6017853, 2024-05-10T05:45:40.091 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 157, 2024-05-10T05:45:40.091 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-05-10T05:45:40.091 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.183254177, 2024-05-10T05:45:40.091 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.048894652, 2024-05-10T05:45:40.091 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.087693228, 2024-05-10T05:45:40.092 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.036663674, 2024-05-10T05:45:40.092 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:40.092 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1457, 2024-05-10T05:45:40.092 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.519064730, 2024-05-10T05:45:40.092 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000356255 2024-05-10T05:45:40.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.092 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:40.092 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1065, 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000264955, 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000248 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-05-10T05:45:40.093 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T05:45:40.094 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T05:45:40.094 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1186, 2024-05-10T05:45:40.094 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.842393409, 2024-05-10T05:45:40.094 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000710281 2024-05-10T05:45:40.094 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:40.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.094 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-05-10T05:45:40.094 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T05:45:40.094 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 52, 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.002216244, 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000042620 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 87, 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 86, 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 86, 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.177821048, 2024-05-10T05:45:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002067686 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 1, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 12, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 30, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 30, 2024-05-10T05:45:40.096 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 29, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 2, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 8, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 8, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 2, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 123, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 113, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-05-10T05:45:40.097 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 6, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 1, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 2, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 1, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 503, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-05-10T05:45:40.098 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 234, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 503, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 189, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 18, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 171, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 314, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 2, 2024-05-10T05:45:40.099 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 22, 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 151, 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 1353, 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 1311, 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 8, 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 45 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-05-10T05:45:40.100 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 5, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 6, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 20, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 14, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-05-10T05:45:40.101 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 18, 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 1, 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 1696, 2024-05-10T05:45:40.102 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 1696, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 3, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 3, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-05-10T05:45:40.103 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 10418313, 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 541, 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.241069115, 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004142456 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 29, 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 63, 2024-05-10T05:45:40.104 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 34, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 19, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 22, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 3, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 30, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 65, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 35, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 3, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 36, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 33, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 123128, 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207252 2024-05-10T05:45:40.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 124, 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 2, 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 87, 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 17, 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 6, 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 22, 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.038954206, 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001770645 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.106 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 17, 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.003731957, 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000219526 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.107 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2, 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000411672, 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000205836 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-05-10T05:45:40.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 10, 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.014798428, 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001479842 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.111 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000269367, 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000269367 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4, 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.011398128, 2024-05-10T05:45:40.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002849532 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2, 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.003034876, 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001517438 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.113 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4, 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.061081576, 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.015270394 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.114 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000971593, 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000971593 2024-05-10T05:45:40.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-05-10T05:45:40.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 7, 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.019298626, 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002756946 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.117 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 16, 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.023870619, 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001491913 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.118 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 22, 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 22, 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 731, 2024-05-10T05:45:40.119 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:40.120 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:40.121 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-10T05:45:40.122 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 307118, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 125, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7544, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 160, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 115898, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 1442, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:40.124 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 1716, 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 6237523, 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 37, 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 1716, 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1716, 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 21.814603256, 2024-05-10T05:45:40.125 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.012712472 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1716, 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1758 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 1716, 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 544, 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 1172, 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-05-10T05:45:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 11, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 23, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 546, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 58, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 527, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-05-10T05:45:40.127 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 521, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 22, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-05-10T05:45:40.128 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-05-10T05:45:40.129 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 82, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 3, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-05-10T05:45:40.130 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 41, 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 0, 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 9 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.131 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 6, 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 6, 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 14, 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:40.132 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 7047, 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1991871, 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 7047, 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 7047, 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1991871, 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.133 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 1716, 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 6238937, 2024-05-10T05:45:40.134 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1716, 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 6238937, 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:40.135 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 1716, 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 1716, 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1716, 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1716, 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:40.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 14, 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1414, 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 14, 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:40.137 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 5, 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1414, 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55ae1b5144e0": { 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 11505, 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:40.138 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 1696, 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 6224009, 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 1696, 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 541, 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 6212504, 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:40.139 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:40.140 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-10T05:45:40.437 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-05-10T05:45:40.437 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-05-10T05:45:40.438 DEBUG:tasks.check_counter:Getting stats from e 2024-05-10T05:45:40.438 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-10T05:45:40.700 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:45:40.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:40 smithi142 ceph-mon[77667]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 215 MiB/s wr, 791 op/s 2024-05-10T05:45:40.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:40 smithi131 ceph-mon[37264]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 215 MiB/s wr, 791 op/s 2024-05-10T05:45:41.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:40 smithi157 ceph-mon[78427]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 215 MiB/s wr, 791 op/s 2024-05-10T05:45:41.796 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 824, 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 826, 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 217569, 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 256328, 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.066962253, 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.024421214, 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.016524051, 2024-05-10T05:45:41.797 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.018723308, 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 824, 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.029108960, 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000035326 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:41.798 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 749, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 919, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 186150, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 604461, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.070479950, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.021070169, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.016454540, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.026329520, 2024-05-10T05:45:41.799 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 918, 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.028825245, 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000031400 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 373072 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.800 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1060, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1053, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 790224, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 385717, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 3, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.114947723, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.034635969, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.039069235, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.033832349, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1052, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.042228270, 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000040140 2024-05-10T05:45:41.801 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 515040, 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 131008 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.802 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4006, 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.108300022, 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000027034 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.803 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-05-10T05:45:41.804 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 1, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-05-10T05:45:41.805 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-05-10T05:45:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-05-10T05:45:41.807 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-05-10T05:45:41.808 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-05-10T05:45:41.809 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-05-10T05:45:41.809 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-05-10T05:45:41.809 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-05-10T05:45:41.809 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-05-10T05:45:41.809 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-05-10T05:45:41.809 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-05-10T05:45:41.809 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-05-10T05:45:41.809 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-05-10T05:45:41.809 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-05-10T05:45:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 3, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 1, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4195378, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 4195237, 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-05-10T05:45:41.811 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 3 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 3, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 3, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 0, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 1, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 1, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 0, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 0, 2024-05-10T05:45:41.812 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 0, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 0, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 119832, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207252 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-05-10T05:45:41.813 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.815 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.816 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.817 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-05-10T05:45:41.818 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.819 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.820 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.821 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-05-10T05:45:41.822 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.823 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-05-10T05:45:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 0, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 1, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 2006, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:41.825 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:41.826 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-10T05:45:41.827 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 190431, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 52, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:41.828 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7544, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 160, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 8320, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 13, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-05-10T05:45:41.829 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 2387, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 2387, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2387, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4.940909948, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002069924 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2387, 2024-05-10T05:45:41.830 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2388 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 2387, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 2387, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 1561, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 824, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-05-10T05:45:41.831 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-05-10T05:45:41.832 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-05-10T05:45:41.833 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 82, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 2, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 5, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 5, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-05-10T05:45:41.834 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-05-10T05:45:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 2629, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 941991, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 2629, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:41.836 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2629, 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 941991, 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:41.837 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 2387, 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 1079, 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2387, 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1079, 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:41.838 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 2387, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 2387, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2387, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 2387, 2024-05-10T05:45:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:41.840 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x555d543628f0": { 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T05:45:41.841 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:41.842 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:41.843 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:41.843 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-10T05:45:42.195 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-10T05:45:42.195 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-10T05:45:42.196 DEBUG:tasks.check_counter:Getting stats from h 2024-05-10T05:45:42.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-10T05:45:42.481 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:45:42.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:42 smithi142 ceph-mon[77667]: pgmap v655: 97 pgs: 97 active+clean; 9.8 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 215 MiB/s wr, 900 op/s 2024-05-10T05:45:42.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:42 smithi131 ceph-mon[37264]: pgmap v655: 97 pgs: 97 active+clean; 9.8 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 215 MiB/s wr, 900 op/s 2024-05-10T05:45:43.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:42 smithi157 ceph-mon[78427]: pgmap v655: 97 pgs: 97 active+clean; 9.8 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 215 MiB/s wr, 900 op/s 2024-05-10T05:45:43.467 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1786, 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1779, 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 5173744, 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 611075, 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 1, 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.164494092, 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.056392998, 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.050931653, 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.045688216, 2024-05-10T05:45:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1778, 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.056725591, 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000031904 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 514928, 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 130576 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 227, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 397, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 58388, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 443748, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.025584161, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.009867467, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.005775319, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.007173582, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 396, 2024-05-10T05:45:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.018627885, 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000047040 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 374640 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 863, 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 865, 2024-05-10T05:45:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 2245837, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 268400, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.065420198, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.016999918, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.017719442, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.024958313, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 863, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.024871399, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000028819 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T05:45:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4397, 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.109643438, 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000024935 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-05-10T05:45:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 50, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 50, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 50, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 2, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-05-10T05:45:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 234, 2024-05-10T05:45:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T05:45:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-05-10T05:45:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-05-10T05:45:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 1696, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 3, 2024-05-10T05:45:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 10418313, 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 10416246, 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 1696 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 20, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 36, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 16, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 10, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 13, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 3, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 50, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 55, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 5, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 118200, 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207252 2024-05-10T05:45:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-05-10T05:45:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-05-10T05:45:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-05-10T05:45:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 2757, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 189604, 2024-05-10T05:45:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 57, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7544, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 160, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 92527, 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 1173, 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-05-10T05:45:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 2630, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 26, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 4, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 2630, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2630, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.089980198, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000414441 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2630, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2631 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 2630, 2024-05-10T05:45:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 2630, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 1636, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 991, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-05-10T05:45:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 2, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-05-10T05:45:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-05-10T05:45:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 82, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 2, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 5, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 5, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-05-10T05:45:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-05-10T05:45:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 2872, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 7204153, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 2872, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2872, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 7204153, 2024-05-10T05:45:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 2630, 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 6224019, 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2630, 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 6224019, 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 2630, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 2630, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2630, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 2630, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-05-10T05:45:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x5646e08ac8f0": { 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-10T05:45:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:43.513 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-10T05:45:43.829 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-10T05:45:43.829 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-10T05:45:43.829 DEBUG:tasks.check_counter:Getting stats from k 2024-05-10T05:45:43.829 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-10T05:45:44.054 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T05:45:44.802 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:44 smithi142 ceph-mon[77667]: pgmap v656: 97 pgs: 97 active+clean; 6.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 194 MiB/s wr, 939 op/s 2024-05-10T05:45:44.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:44 smithi131 ceph-mon[37264]: pgmap v656: 97 pgs: 97 active+clean; 6.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 194 MiB/s wr, 939 op/s 2024-05-10T05:45:44.987 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-10T05:45:44.987 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 826, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 828, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 277481, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 256887, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.085568312, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.023844854, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.024163004, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.028763047, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 826, 2024-05-10T05:45:44.988 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.036799356, 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000044551 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1075, 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1066, 2024-05-10T05:45:44.989 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 736046, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 390486, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.097713818, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.034580943, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.033559037, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.024414916, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1066, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.037625274, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000035295 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.990 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 518800, 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 133232 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 4, 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 178, 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 1508, 2024-05-10T05:45:44.991 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 375488, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.009993912, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.006403925, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.001879683, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000093802, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 176, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.009821410, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000055803 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:44.992 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 374544 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3305, 2024-05-10T05:45:44.993 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.092292267, 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000027925 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-05-10T05:45:44.994 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-05-10T05:45:44.995 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 1, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-05-10T05:45:44.996 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-05-10T05:45:44.997 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T05:45:44.998 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-05-10T05:45:44.999 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-05-10T05:45:45.000 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 1, 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-05-10T05:45:45.001 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 1, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4194844, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 4194304, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.002 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 1 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 3, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 3, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 0, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 1, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 1, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 0, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 0, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 0, 2024-05-10T05:45:45.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 0, 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 118868, 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207252 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-05-10T05:45:45.004 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-05-10T05:45:45.005 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-05-10T05:45:45.005 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.005 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.005 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.005 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-05-10T05:45:45.005 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.005 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-05-10T05:45:45.006 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.008 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.009 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.009 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-05-10T05:45:45.009 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.009 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.009 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-05-10T05:45:45.011 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.012 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.014 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 0, 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 0, 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-05-10T05:45:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 0, 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 0, 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:45.018 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:45.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 197441, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 54, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7544, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 160, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 8320, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 13, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 1656, 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 1656, 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1656, 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.231959411, 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001347801 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.023 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-05-10T05:45:45.024 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-05-10T05:45:45.025 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1656, 2024-05-10T05:45:45.025 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1657 2024-05-10T05:45:45.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.025 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 1656, 2024-05-10T05:45:45.025 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 1656, 2024-05-10T05:45:45.025 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-05-10T05:45:45.025 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-05-10T05:45:45.025 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 825, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 829, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-05-10T05:45:45.026 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-05-10T05:45:45.027 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-05-10T05:45:45.028 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 82, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 2, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 6, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 6, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-05-10T05:45:45.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-05-10T05:45:45.030 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 1901, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 828447, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 1901, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:45.031 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1901, 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 828447, 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:45.032 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 1656, 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 545, 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1656, 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 545, 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-05-10T05:45:45.033 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 1656, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 1656, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1656, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1656, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.034 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:45.035 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x564e7a98c8f0": { 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T05:45:45.036 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T05:45:45.037 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-10T05:45:45.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:44 smithi157 ceph-mon[78427]: pgmap v656: 97 pgs: 97 active+clean; 6.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 194 MiB/s wr, 939 op/s 2024-05-10T05:45:45.355 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-10T05:45:45.355 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-10T05:45:45.355 DEBUG:tasks.check_counter:Getting stats from c 2024-05-10T05:45:45.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-10T05:45:45.605 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:45:46.629 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-10T05:45:46.629 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:46.629 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 51, 2024-05-10T05:45:46.629 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 59, 2024-05-10T05:45:46.629 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 8732, 2024-05-10T05:45:46.629 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 391511, 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 51, 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.022430618, 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.001199244, 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.016106283, 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000251307, 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 51, 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.009182734, 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000180053 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.630 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 40, 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000008268, 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000206 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 722, 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 892, 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 664018, 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 292935, 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 7, 2024-05-10T05:45:46.631 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.086203888, 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.031330618, 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.045894733, 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.010085953, 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 895, 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.051261901, 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000057275 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1030, 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000305971, 2024-05-10T05:45:46.632 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000297 2024-05-10T05:45:46.633 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.633 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 523520, 2024-05-10T05:45:46.633 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 136384 2024-05-10T05:45:46.633 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.633 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:46.633 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 138, 2024-05-10T05:45:46.633 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 307, 2024-05-10T05:45:46.633 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 87025, 2024-05-10T05:45:46.633 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 463140, 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.019085926, 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.009606156, 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.009086650, 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000353052, 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 308, 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.030771729, 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000099908 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:46.634 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 201, 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000047161, 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000234 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 376384 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-10T05:45:46.635 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 21, 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001047012, 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000049857 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2, 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000015730, 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000007865 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:46.636 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 11, 2024-05-10T05:45:46.637 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 10, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 10, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 10, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-05-10T05:45:46.638 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-05-10T05:45:46.639 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 22, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 10738148731, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T05:45:46.640 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 1, 2024-05-10T05:45:46.641 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-05-10T05:45:46.642 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 1, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 1, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 1, 2024-05-10T05:45:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 1, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4194844, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001362957, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001362957 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-05-10T05:45:46.644 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 13, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 13, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 11, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 12, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 1, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 10, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 10, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 0, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-05-10T05:45:46.645 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 117540, 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 199060 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.646 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.647 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.648 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-05-10T05:45:46.649 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.650 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.651 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-05-10T05:45:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.653 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.653 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.653 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.653 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-05-10T05:45:46.653 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.653 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.653 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-05-10T05:45:46.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.655 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.655 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.655 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.655 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-05-10T05:45:46.655 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.655 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.655 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.655 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.655 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-05-10T05:45:46.655 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.656 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.657 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 0, 2024-05-10T05:45:46.658 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 0, 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 0, 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 0, 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 0, 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:46.659 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:46.661 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 237991, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 76, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-10T05:45:46.662 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 7544, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 160, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 50040, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 198, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:46.663 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 35, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 10662, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 33, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 35, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 35, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.220995914, 2024-05-10T05:45:46.664 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.006314168 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 35, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 57 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 35, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 0, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 35, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 0, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 10, 2024-05-10T05:45:46.665 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 0, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 1, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 5, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 6, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 0, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 20, 2024-05-10T05:45:46.666 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-05-10T05:45:46.667 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 82, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-05-10T05:45:46.668 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 2, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 9, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 9, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 14, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 1 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.669 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:46.670 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 907, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 661333, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 907, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 907, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 661333, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 35, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 10662, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 35, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 10662, 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:46.672 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:46.673 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 35, 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 35, 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 35, 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 35, 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-10T05:45:46.674 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:46.675 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x561f934dc410": { 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 1, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 540, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 1, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 1, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 540, 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:46.676 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:46.677 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:46.677 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:46.677 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:46.677 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:46.677 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-10T05:45:46.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:46 smithi157 ceph-mon[78427]: pgmap v657: 97 pgs: 97 active+clean; 1.7 GiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 149 MiB/s wr, 939 op/s 2024-05-10T05:45:46.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:46 smithi142 ceph-mon[77667]: pgmap v657: 97 pgs: 97 active+clean; 1.7 GiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 149 MiB/s wr, 939 op/s 2024-05-10T05:45:46.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:46 smithi131 ceph-mon[37264]: pgmap v657: 97 pgs: 97 active+clean; 1.7 GiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 149 MiB/s wr, 939 op/s 2024-05-10T05:45:46.999 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-10T05:45:46.999 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-10T05:45:46.999 DEBUG:tasks.check_counter:Getting stats from f 2024-05-10T05:45:46.999 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-10T05:45:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:45:48.184 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-10T05:45:48.185 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:48.185 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2415, 2024-05-10T05:45:48.185 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2417, 2024-05-10T05:45:48.185 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 683971, 2024-05-10T05:45:48.185 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 751072, 2024-05-10T05:45:48.185 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-05-10T05:45:48.185 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-05-10T05:45:48.186 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.179847224, 2024-05-10T05:45:48.186 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.053187056, 2024-05-10T05:45:48.186 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.046797172, 2024-05-10T05:45:48.186 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.065891408, 2024-05-10T05:45:48.186 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:48.186 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2415, 2024-05-10T05:45:48.186 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.066378902, 2024-05-10T05:45:48.186 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000027486 2024-05-10T05:45:48.187 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.187 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:48.187 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.187 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.187 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.187 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.187 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 3, 2024-05-10T05:45:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 176, 2024-05-10T05:45:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 716, 2024-05-10T05:45:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 376620, 2024-05-10T05:45:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-05-10T05:45:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-05-10T05:45:48.188 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.009465913, 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.006600123, 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.001230846, 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000082341, 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 174, 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.008140618, 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000046785 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 375936 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 247, 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 238, 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 518524, 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 132343, 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 3, 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-05-10T05:45:48.190 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.034465556, 2024-05-10T05:45:48.191 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.011811939, 2024-05-10T05:45:48.191 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.016814069, 2024-05-10T05:45:48.191 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.005766114, 2024-05-10T05:45:48.191 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:48.191 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 238, 2024-05-10T05:45:48.191 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.011589548, 2024-05-10T05:45:48.191 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000048695 2024-05-10T05:45:48.191 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.191 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:48.191 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.192 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.192 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.192 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.192 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 518256, 2024-05-10T05:45:48.192 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 132032 2024-05-10T05:45:48.192 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.192 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-10T05:45:48.192 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4073, 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.105606875, 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000025928 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-10T05:45:48.193 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.194 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-05-10T05:45:48.195 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-05-10T05:45:48.196 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-05-10T05:45:48.196 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-05-10T05:45:48.196 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-05-10T05:45:48.196 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-05-10T05:45:48.196 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-05-10T05:45:48.196 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-05-10T05:45:48.196 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-05-10T05:45:48.196 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-05-10T05:45:48.196 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-05-10T05:45:48.196 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T05:45:48.197 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-05-10T05:45:48.198 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-05-10T05:45:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-05-10T05:45:48.200 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 4, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 1, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4195688, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 4195378, 2024-05-10T05:45:48.201 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 4 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 3, 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 3, 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 1, 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 1, 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2024-05-10T05:45:48.202 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 117112, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 207252 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-05-10T05:45:48.203 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.204 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.205 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.206 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-05-10T05:45:48.206 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.206 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.206 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.206 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.206 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-05-10T05:45:48.206 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.206 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.206 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.207 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.207 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-05-10T05:45:48.207 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.207 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.207 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.207 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.207 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-05-10T05:45:48.207 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.208 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.209 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.210 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.211 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.211 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.211 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-05-10T05:45:48.211 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.211 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.211 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-05-10T05:45:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-05-10T05:45:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.213 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.214 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.214 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-05-10T05:45:48.214 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.214 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.214 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.214 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.214 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-05-10T05:45:48.214 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.214 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.214 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.215 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.215 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-05-10T05:45:48.215 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.215 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.215 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.215 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.215 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-05-10T05:45:48.215 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-05-10T05:45:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-05-10T05:45:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-05-10T05:45:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-05-10T05:45:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-05-10T05:45:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-05-10T05:45:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 2, 2024-05-10T05:45:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 1, 2024-05-10T05:45:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2024-05-10T05:45:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-05-10T05:45:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-05-10T05:45:48.219 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-05-10T05:45:48.219 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 2701, 2024-05-10T05:45:48.219 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-05-10T05:45:48.219 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.219 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-10T05:45:48.219 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:48.220 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:48.220 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:48.220 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:48.220 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:48.220 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:48.220 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:48.221 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:48.221 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:48.221 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:48.221 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:48.221 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:48.221 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:48.221 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:48.222 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:48.222 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:48.222 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:48.222 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:48.222 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:48.222 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:48.222 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:48.223 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:48.223 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:48.223 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:48.223 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:48.223 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:48.223 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:48.223 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:48.223 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:48.223 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:48.224 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:48.224 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-10T05:45:48.224 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:48.224 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:48.224 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:48.224 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:48.225 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 181303, 2024-05-10T05:45:48.225 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 46, 2024-05-10T05:45:48.225 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:48.225 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:48.225 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-10T05:45:48.225 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-10T05:45:48.225 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:48.226 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:48.226 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:48.226 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:48.226 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 7544, 2024-05-10T05:45:48.226 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 160, 2024-05-10T05:45:48.226 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:48.227 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:48.227 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:48.227 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-10T05:45:48.227 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 8320, 2024-05-10T05:45:48.227 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 13, 2024-05-10T05:45:48.227 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:48.227 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-10T05:45:48.227 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:48.227 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-10T05:45:48.228 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.228 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-05-10T05:45:48.228 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-05-10T05:45:48.228 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-05-10T05:45:48.228 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 2418, 2024-05-10T05:45:48.228 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2024-05-10T05:45:48.228 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2024-05-10T05:45:48.229 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 2418, 2024-05-10T05:45:48.229 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-05-10T05:45:48.229 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2418, 2024-05-10T05:45:48.229 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.146978664, 2024-05-10T05:45:48.229 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000474350 2024-05-10T05:45:48.229 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.230 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-05-10T05:45:48.230 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-05-10T05:45:48.230 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2418, 2024-05-10T05:45:48.230 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2419 2024-05-10T05:45:48.230 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.230 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 2418, 2024-05-10T05:45:48.230 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 2418, 2024-05-10T05:45:48.230 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-05-10T05:45:48.230 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 1581, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 835, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-05-10T05:45:48.231 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2024-05-10T05:45:48.232 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-05-10T05:45:48.232 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-05-10T05:45:48.232 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-05-10T05:45:48.232 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-05-10T05:45:48.232 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-05-10T05:45:48.232 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-05-10T05:45:48.232 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-05-10T05:45:48.232 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T05:45:48.232 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-05-10T05:45:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-05-10T05:45:48.234 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-05-10T05:45:48.234 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-05-10T05:45:48.234 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-05-10T05:45:48.234 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 82, 2024-05-10T05:45:48.234 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-05-10T05:45:48.234 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 2, 2024-05-10T05:45:48.234 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 4, 2024-05-10T05:45:48.234 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 4, 2024-05-10T05:45:48.234 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-05-10T05:45:48.235 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-10T05:45:48.236 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2661, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 948327, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2661, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2661, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 948327, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:48.237 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.238 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-05-10T05:45:48.238 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:48.238 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-10T05:45:48.238 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:48.238 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-10T05:45:48.238 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-10T05:45:48.238 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:48.238 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:48.238 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2418, 2024-05-10T05:45:48.239 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 1389, 2024-05-10T05:45:48.239 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2418, 2024-05-10T05:45:48.239 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1389, 2024-05-10T05:45:48.239 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:48.239 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.239 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.239 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.239 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:48.240 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.240 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-05-10T05:45:48.240 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:48.240 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-05-10T05:45:48.240 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:48.240 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-10T05:45:48.240 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-10T05:45:48.240 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:48.240 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:48.240 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2418, 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2418, 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2418, 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2418, 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:48.241 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:48.242 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x5563535ce4e0": { 2024-05-10T05:45:48.243 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:48.243 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-10T05:45:48.243 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:48.243 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-10T05:45:48.243 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-10T05:45:48.243 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:48.243 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:48.243 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-10T05:45:48.244 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-10T05:45:48.244 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-05-10T05:45:48.244 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-05-10T05:45:48.244 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:48.244 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:48.244 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:48.244 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:48.244 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:48.244 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:48.244 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-10T05:45:48.549 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-10T05:45:48.550 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-10T05:45:48.550 DEBUG:tasks.check_counter:Getting stats from i 2024-05-10T05:45:48.550 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-10T05:45:48.775 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:45:48.868 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:48 smithi157 ceph-mon[78427]: pgmap v658: 97 pgs: 97 active+clean; 6.5 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 92 MiB/s wr, 802 op/s 2024-05-10T05:45:48.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:48 smithi142 ceph-mon[77667]: pgmap v658: 97 pgs: 97 active+clean; 6.5 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 92 MiB/s wr, 802 op/s 2024-05-10T05:45:48.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:48 smithi131 ceph-mon[37264]: pgmap v658: 97 pgs: 97 active+clean; 6.5 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 92 MiB/s wr, 802 op/s 2024-05-10T05:45:49.705 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-10T05:45:49.705 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:49.705 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 435, 2024-05-10T05:45:49.705 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 433, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 589251, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 598082, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 48, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.069148126, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.018707375, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.040521931, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.006087728, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 432, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.028944509, 2024-05-10T05:45:49.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000067001 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 145, 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000043340, 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000298 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 532768, 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 147264 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 98, 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 269, 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 51859, 2024-05-10T05:45:49.707 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 433036, 2024-05-10T05:45:49.708 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 9, 2024-05-10T05:45:49.708 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-05-10T05:45:49.708 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.017434348, 2024-05-10T05:45:49.708 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.007959913, 2024-05-10T05:45:49.708 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.007610610, 2024-05-10T05:45:49.708 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000949867, 2024-05-10T05:45:49.708 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:49.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 268, 2024-05-10T05:45:49.708 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.031029631, 2024-05-10T05:45:49.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000115782 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 97, 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000027915, 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000287 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 378816 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1369, 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1002, 2024-05-10T05:45:49.709 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 384860, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 373854, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 11, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.087945741, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.031619635, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.060681953, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.005818207, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1000, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.049171378, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000049171 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.710 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2216, 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000499959, 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000225 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-05-10T05:45:49.711 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-10T05:45:49.712 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-10T05:45:49.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 92, 2024-05-10T05:45:49.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.006797909, 2024-05-10T05:45:49.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000073890 2024-05-10T05:45:49.712 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:49.712 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.712 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-05-10T05:45:49.712 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 17, 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000305820, 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000017989 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 4, 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 4, 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4, 2024-05-10T05:45:49.713 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.003464970, 2024-05-10T05:45:49.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000866242 2024-05-10T05:45:49.714 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.714 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-05-10T05:45:49.714 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-05-10T05:45:49.714 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-05-10T05:45:49.714 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-05-10T05:45:49.714 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 11, 2024-05-10T05:45:49.714 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-05-10T05:45:49.714 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 15, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 1, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 14, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 13, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 8, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 3, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 10, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 6, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-05-10T05:45:49.715 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 1, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 1, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 1, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 2, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-05-10T05:45:49.716 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 28, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 22, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 10738148731, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 28, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 28, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 2, 2024-05-10T05:45:49.717 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 3, 2024-05-10T05:45:49.718 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T05:45:49.718 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T05:45:49.718 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T05:45:49.718 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 1, 2024-05-10T05:45:49.718 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-05-10T05:45:49.718 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 2, 2024-05-10T05:45:49.718 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 4 2024-05-10T05:45:49.718 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.719 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-05-10T05:45:49.719 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-05-10T05:45:49.719 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-05-10T05:45:49.719 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-05-10T05:45:49.719 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 2, 2024-05-10T05:45:49.719 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-05-10T05:45:49.719 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-05-10T05:45:49.719 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-05-10T05:45:49.720 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-05-10T05:45:49.720 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 2, 2024-05-10T05:45:49.720 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 2, 2024-05-10T05:45:49.720 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-05-10T05:45:49.720 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-05-10T05:45:49.720 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-05-10T05:45:49.721 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-05-10T05:45:49.721 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-05-10T05:45:49.721 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-05-10T05:45:49.721 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-05-10T05:45:49.721 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T05:45:49.722 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 1, 2024-05-10T05:45:49.722 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 15, 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 15, 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-05-10T05:45:49.723 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 1, 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 1, 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4220955, 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11, 2024-05-10T05:45:49.724 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.039681100, 2024-05-10T05:45:49.725 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003607372 2024-05-10T05:45:49.725 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.725 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-05-10T05:45:49.725 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.725 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-05-10T05:45:49.725 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 19, 2024-05-10T05:45:49.725 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 25, 2024-05-10T05:45:49.725 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 6, 2024-05-10T05:45:49.725 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 18, 2024-05-10T05:45:49.725 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 23, 2024-05-10T05:45:49.726 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 5, 2024-05-10T05:45:49.726 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 15, 2024-05-10T05:45:49.726 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 27, 2024-05-10T05:45:49.726 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 12, 2024-05-10T05:45:49.726 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-05-10T05:45:49.726 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 5, 2024-05-10T05:45:49.726 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 5, 2024-05-10T05:45:49.726 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 118364, 2024-05-10T05:45:49.726 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 199060 2024-05-10T05:45:49.727 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.727 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-05-10T05:45:49.727 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 7, 2024-05-10T05:45:49.727 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 1, 2024-05-10T05:45:49.727 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 4, 2024-05-10T05:45:49.727 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 9, 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 4, 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.728 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000272503, 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000272503 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.729 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.730 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-05-10T05:45:49.731 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.732 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-05-10T05:45:49.733 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000479484, 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000479484 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.734 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.735 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-05-10T05:45:49.736 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.737 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.737 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.737 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.737 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-05-10T05:45:49.737 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.737 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.737 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.737 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.738 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-05-10T05:45:49.738 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.738 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.738 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.738 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.738 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-05-10T05:45:49.738 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.738 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.738 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.738 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.739 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-05-10T05:45:49.739 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.739 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.739 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.739 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.739 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-05-10T05:45:49.739 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2, 2024-05-10T05:45:49.739 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.002712983, 2024-05-10T05:45:49.739 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001356491 2024-05-10T05:45:49.739 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 1, 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 0, 2024-05-10T05:45:49.740 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 3, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 3, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 2707, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:49.741 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:49.742 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:49.743 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:49.743 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:49.743 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:49.743 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:49.743 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:49.743 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:49.743 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:49.743 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 300265, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 105, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:49.744 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-10T05:45:49.745 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-10T05:45:49.745 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:49.745 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:49.745 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:49.745 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:49.745 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 7544, 2024-05-10T05:45:49.745 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 160, 2024-05-10T05:45:49.745 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 75565, 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 315, 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-05-10T05:45:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 92, 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 38920, 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 27, 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 92, 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 92, 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.438991878, 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004771650 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-05-10T05:45:49.747 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 92, 2024-05-10T05:45:49.748 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 118 2024-05-10T05:45:49.748 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.748 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 92, 2024-05-10T05:45:49.748 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 29, 2024-05-10T05:45:49.748 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 63, 2024-05-10T05:45:49.748 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-05-10T05:45:49.748 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-05-10T05:45:49.748 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 0, 2024-05-10T05:45:49.748 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 10, 2024-05-10T05:45:49.749 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 1, 2024-05-10T05:45:49.749 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 12, 2024-05-10T05:45:49.749 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 10, 2024-05-10T05:45:49.749 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-05-10T05:45:49.749 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-05-10T05:45:49.749 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-05-10T05:45:49.749 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-05-10T05:45:49.749 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 14, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 28, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 20, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-05-10T05:45:49.750 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-05-10T05:45:49.751 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 82, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 2, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 10, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 10, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 20, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2024-05-10T05:45:49.752 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 3 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.753 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-05-10T05:45:49.754 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-05-10T05:45:49.754 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 2, 2024-05-10T05:45:49.754 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-05-10T05:45:49.754 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 2, 2024-05-10T05:45:49.754 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 2, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 1898, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 837616, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 1898, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 1898, 2024-05-10T05:45:49.755 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 837616, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:49.756 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 92, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 39122, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 92, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 39122, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-10T05:45:49.757 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 92, 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 92, 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 92, 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 92, 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-05-10T05:45:49.758 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 202, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 1, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 202, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.759 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x557709c264e0": { 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 15, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 26651, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 15, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 11, 2024-05-10T05:45:49.760 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 26651, 2024-05-10T05:45:49.761 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:49.761 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:49.761 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:49.761 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:49.761 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:49.761 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:49.761 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-10T05:45:50.081 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-05-10T05:45:50.082 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-05-10T05:45:50.082 DEBUG:tasks.check_counter:Getting stats from l 2024-05-10T05:45:50.082 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-10T05:45:50.306 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T05:45:50.803 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:50 smithi157 ceph-mon[78427]: pgmap v659: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 64 MiB/s wr, 676 op/s 2024-05-10T05:45:50.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:50 smithi142 ceph-mon[77667]: pgmap v659: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 64 MiB/s wr, 676 op/s 2024-05-10T05:45:50.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:50 smithi131 ceph-mon[37264]: pgmap v659: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 64 MiB/s wr, 676 op/s 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 0, 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2, 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 0, 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 0, 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 0, 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.001219660, 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T05:45:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.000337705, 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T05:45:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 235, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 232, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 476544, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 132784, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 2, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.034715705, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.012625179, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.016025205, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.005560084, 2024-05-10T05:45:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 231, 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.009814938, 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000042488 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 476544, 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 132784 2024-05-10T05:45:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 5, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 179, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 3104, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 107680, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 3, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.009464221, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.006399719, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.001408981, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 178, 2024-05-10T05:45:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.012750904, 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000071634 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 107680 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 0, 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-10T05:45:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-10T05:45:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-10T05:45:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T05:45:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 148984, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 30, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 0, 2024-05-10T05:45:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 0, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 928, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 2, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-10T05:45:51.289 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 236, 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 443308, 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 236, 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 236, 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 443308, 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-10T05:45:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-10T05:45:51.291 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-10T05:45:51.291 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:51.291 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-10T05:45:51.291 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-10T05:45:51.632 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-10T05:45:51.633 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-10T05:45:51.633 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-10T05:45:51.643 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-10T05:45:51.643 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:45:51.643 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T05:45:51.694 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-10T05:45:51.694 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T05:45:51.781 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:45:52.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.093+0000 7fc4dce00700 1 -- 172.21.15.131:0/3686362986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d8101be0 msgr2=0x7fc4d8101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:52.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.094+0000 7fc4dce00700 1 --2- 172.21.15.131:0/3686362986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d8101be0 0x7fc4d8101ff0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009990 tx=0x7fc4c8009ca0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.094+0000 7fc4dce00700 1 -- 172.21.15.131:0/3686362986 shutdown_connections 2024-05-10T05:45:52.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.094+0000 7fc4dce00700 1 --2- 172.21.15.131:0/3686362986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d8101be0 0x7fc4d8101ff0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.094+0000 7fc4dce00700 1 --2- 172.21.15.131:0/3686362986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d81036a0 0x7fc4d8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.094+0000 7fc4dce00700 1 --2- 172.21.15.131:0/3686362986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8102d00 0x7fc4d8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.094+0000 7fc4dce00700 1 -- 172.21.15.131:0/3686362986 >> 172.21.15.131:0/3686362986 conn(0x7fc4d80fd1d0 msgr2=0x7fc4d80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:52.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.094+0000 7fc4dce00700 1 -- 172.21.15.131:0/3686362986 shutdown_connections 2024-05-10T05:45:52.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.094+0000 7fc4dce00700 1 -- 172.21.15.131:0/3686362986 wait complete. 2024-05-10T05:45:52.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.095+0000 7fc4dce00700 1 Processor -- start 2024-05-10T05:45:52.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.095+0000 7fc4dce00700 1 -- start start 2024-05-10T05:45:52.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.095+0000 7fc4dce00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d8101be0 0x7fc4d819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:52.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.095+0000 7fc4dce00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8102d00 0x7fc4d819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:52.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.095+0000 7fc4dce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d81036a0 0x7fc4d81a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:52.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.095+0000 7fc4dce00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d81a1cf0 con 0x7fc4d8101be0 2024-05-10T05:45:52.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.095+0000 7fc4dce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d81a1e60 con 0x7fc4d81036a0 2024-05-10T05:45:52.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.095+0000 7fc4dce00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d81a2020 con 0x7fc4d8102d00 2024-05-10T05:45:52.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d8101be0 0x7fc4d819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:52.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8102d00 0x7fc4d819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:52.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8102d00 0x7fc4d819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:55960/0 (socket says 172.21.15.131:55960) 2024-05-10T05:45:52.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d81036a0 0x7fc4d81a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:52.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5400700 1 -- 172.21.15.131:0/3119684328 learned_addr learned my addr 172.21.15.131:0/3119684328 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:52.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5400700 1 -- 172.21.15.131:0/3119684328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d81036a0 msgr2=0x7fc4d81a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:52.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5400700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d81036a0 0x7fc4d81a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5400700 1 -- 172.21.15.131:0/3119684328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d8101be0 msgr2=0x7fc4d819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:52.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5400700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d8101be0 0x7fc4d819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5400700 1 -- 172.21.15.131:0/3119684328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c80095f0 con 0x7fc4d8102d00 2024-05-10T05:45:52.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5e00700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d8101be0 0x7fc4d819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:52.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.096+0000 7fc4d5400700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8102d00 0x7fc4d819a750 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400e9a0 tx=0x7fc4c400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:52.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.097+0000 7fc4cea00700 1 -- 172.21.15.131:0/3119684328 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4009790 con 0x7fc4d8102d00 2024-05-10T05:45:52.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.097+0000 7fc4cea00700 1 -- 172.21.15.131:0/3119684328 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4c4010430 con 0x7fc4d8102d00 2024-05-10T05:45:52.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.097+0000 7fc4dce00700 1 -- 172.21.15.131:0/3119684328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d81a2300 con 0x7fc4d8102d00 2024-05-10T05:45:52.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.097+0000 7fc4cea00700 1 -- 172.21.15.131:0/3119684328 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4018680 con 0x7fc4d8102d00 2024-05-10T05:45:52.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.097+0000 7fc4dce00700 1 -- 172.21.15.131:0/3119684328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d81a2810 con 0x7fc4d8102d00 2024-05-10T05:45:52.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.098+0000 7fc4dce00700 1 -- 172.21.15.131:0/3119684328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d80694b0 con 0x7fc4d8102d00 2024-05-10T05:45:52.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.100+0000 7fc4cea00700 1 -- 172.21.15.131:0/3119684328 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc4c40105a0 con 0x7fc4d8102d00 2024-05-10T05:45:52.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.101+0000 7fc4cea00700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4bc0a1120 0x7fc4bc0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:52.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.101+0000 7fc4cea00700 1 -- 172.21.15.131:0/3119684328 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc4c4014070 con 0x7fc4d8102d00 2024-05-10T05:45:52.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.102+0000 7fc4d5e00700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4bc0a1120 0x7fc4bc0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:52.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.102+0000 7fc4d5e00700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4bc0a1120 0x7fc4bc0a3630 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8006010 tx=0x7fc4c801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:52.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.105+0000 7fc4cea00700 1 -- 172.21.15.131:0/3119684328 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc4c4096210 con 0x7fc4d8102d00 2024-05-10T05:45:52.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.312+0000 7fc4dce00700 1 -- 172.21.15.131:0/3119684328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4d810d410 con 0x7fc4d8102d00 2024-05-10T05:45:52.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.315+0000 7fc4cea00700 1 -- 172.21.15.131:0/3119684328 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7fc4c4099830 con 0x7fc4d8102d00 2024-05-10T05:45:52.316 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:45:52.316 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:45:52.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 -- 172.21.15.131:0/3119684328 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4bc0a1120 msgr2=0x7fc4bc0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:52.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4bc0a1120 0x7fc4bc0a3630 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8006010 tx=0x7fc4c801a040 comp rx=0 tx=0).stop 2024-05-10T05:45:52.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 -- 172.21.15.131:0/3119684328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8102d00 msgr2=0x7fc4d819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:52.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8102d00 0x7fc4d819a750 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400e9a0 tx=0x7fc4c400ecb0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 -- 172.21.15.131:0/3119684328 shutdown_connections 2024-05-10T05:45:52.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc4bc0a1120 0x7fc4bc0a3630 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4d8101be0 0x7fc4d819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d81036a0 0x7fc4d81a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 --2- 172.21.15.131:0/3119684328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d8102d00 0x7fc4d819a750 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 -- 172.21.15.131:0/3119684328 >> 172.21.15.131:0/3119684328 conn(0x7fc4d80fd1d0 msgr2=0x7fc4d8073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:52.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 -- 172.21.15.131:0/3119684328 shutdown_connections 2024-05-10T05:45:52.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.318+0000 7fc4dce00700 1 -- 172.21.15.131:0/3119684328 wait complete. 2024-05-10T05:45:52.347 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T05:28:15.347512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T05:31:49.481667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T05:31:50.960751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T05:45:52.347 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T05:45:52.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.617+0000 7fc78b800700 1 -- 172.21.15.131:0/1775205036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840750e0 msgr2=0x7fc7840735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:52.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.617+0000 7fc78b800700 1 --2- 172.21.15.131:0/1775205036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840750e0 0x7fc7840735a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc778009990 tx=0x7fc778009ca0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.617+0000 7fc78b800700 1 -- 172.21.15.131:0/1775205036 shutdown_connections 2024-05-10T05:45:52.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.617+0000 7fc78b800700 1 --2- 172.21.15.131:0/1775205036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc784109b00 0x7fc78410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.617+0000 7fc78b800700 1 --2- 172.21.15.131:0/1775205036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840750e0 0x7fc7840735a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.617+0000 7fc78b800700 1 --2- 172.21.15.131:0/1775205036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc784073ae0 0x7fc784073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.617+0000 7fc78b800700 1 -- 172.21.15.131:0/1775205036 >> 172.21.15.131:0/1775205036 conn(0x7fc7840fd210 msgr2=0x7fc7840ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:52.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.617+0000 7fc78b800700 1 -- 172.21.15.131:0/1775205036 shutdown_connections 2024-05-10T05:45:52.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.617+0000 7fc78b800700 1 -- 172.21.15.131:0/1775205036 wait complete. 2024-05-10T05:45:52.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.618+0000 7fc78b800700 1 Processor -- start 2024-05-10T05:45:52.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.618+0000 7fc78b800700 1 -- start start 2024-05-10T05:45:52.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc78b800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc784073ae0 0x7fc78419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:52.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc78b800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840750e0 0x7fc78419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:52.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc78b800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc784109b00 0x7fc7841a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:52.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc78b800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841a1bb0 con 0x7fc784109b00 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc78b800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841a1d20 con 0x7fc784073ae0 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc78b800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841a1ee0 con 0x7fc7840750e0 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc788e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc784073ae0 0x7fc78419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc789800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc784109b00 0x7fc7841a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc788e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc784073ae0 0x7fc78419a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:53872/0 (socket says 172.21.15.131:53872) 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc788e00700 1 -- 172.21.15.131:0/3506781401 learned_addr learned my addr 172.21.15.131:0/3506781401 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc783e00700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840750e0 0x7fc78419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc789800700 1 -- 172.21.15.131:0/3506781401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840750e0 msgr2=0x7fc78419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc789800700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840750e0 0x7fc78419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc789800700 1 -- 172.21.15.131:0/3506781401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc784073ae0 msgr2=0x7fc78419a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc789800700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc784073ae0 0x7fc78419a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.619+0000 7fc789800700 1 -- 172.21.15.131:0/3506781401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7780095f0 con 0x7fc784109b00 2024-05-10T05:45:52.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.620+0000 7fc783e00700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840750e0 0x7fc78419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:52.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.620+0000 7fc788e00700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc784073ae0 0x7fc78419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:52.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.620+0000 7fc789800700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc784109b00 0x7fc7841a15a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00efc0 tx=0x7fc77c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:52.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.620+0000 7fc781600700 1 -- 172.21.15.131:0/3506781401 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c018070 con 0x7fc784109b00 2024-05-10T05:45:52.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.620+0000 7fc781600700 1 -- 172.21.15.131:0/3506781401 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc77c00f040 con 0x7fc784109b00 2024-05-10T05:45:52.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.620+0000 7fc78b800700 1 -- 172.21.15.131:0/3506781401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7841a2160 con 0x7fc784109b00 2024-05-10T05:45:52.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.620+0000 7fc781600700 1 -- 172.21.15.131:0/3506781401 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00cd40 con 0x7fc784109b00 2024-05-10T05:45:52.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.620+0000 7fc78b800700 1 -- 172.21.15.131:0/3506781401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7841a2670 con 0x7fc784109b00 2024-05-10T05:45:52.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.621+0000 7fc78b800700 1 -- 172.21.15.131:0/3506781401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc784108950 con 0x7fc784109b00 2024-05-10T05:45:52.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.623+0000 7fc781600700 1 -- 172.21.15.131:0/3506781401 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc77c003a00 con 0x7fc784109b00 2024-05-10T05:45:52.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.624+0000 7fc781600700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc76c0a1240 0x7fc76c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:52.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.626+0000 7fc781600700 1 -- 172.21.15.131:0/3506781401 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc77c0ccdc0 con 0x7fc784109b00 2024-05-10T05:45:52.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.626+0000 7fc788e00700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc76c0a1240 0x7fc76c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:52.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.626+0000 7fc788e00700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc76c0a1240 0x7fc76c0a3750 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b2e0 tx=0x7fc77801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:52.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.627+0000 7fc781600700 1 -- 172.21.15.131:0/3506781401 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc77c096580 con 0x7fc784109b00 2024-05-10T05:45:52.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.834+0000 7fc78b800700 1 -- 172.21.15.131:0/3506781401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc78410f650 con 0x7fc784109b00 2024-05-10T05:45:52.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.835+0000 7fc781600700 1 -- 172.21.15.131:0/3506781401 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc77c099ba0 con 0x7fc784109b00 2024-05-10T05:45:52.837 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-10T05:45:52.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.837+0000 7fc78b800700 1 -- 172.21.15.131:0/3506781401 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc76c0a1240 msgr2=0x7fc76c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:52.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.837+0000 7fc78b800700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc76c0a1240 0x7fc76c0a3750 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b2e0 tx=0x7fc77801a040 comp rx=0 tx=0).stop 2024-05-10T05:45:52.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.837+0000 7fc78b800700 1 -- 172.21.15.131:0/3506781401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc784109b00 msgr2=0x7fc7841a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:52.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.837+0000 7fc78b800700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc784109b00 0x7fc7841a15a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00efc0 tx=0x7fc77c00c3a0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.838+0000 7fc78b800700 1 -- 172.21.15.131:0/3506781401 shutdown_connections 2024-05-10T05:45:52.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.838+0000 7fc78b800700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc76c0a1240 0x7fc76c0a3750 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.838+0000 7fc78b800700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc784109b00 0x7fc7841a15a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.838+0000 7fc78b800700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc784073ae0 0x7fc78419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.838+0000 7fc78b800700 1 --2- 172.21.15.131:0/3506781401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7840750e0 0x7fc78419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:52.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.838+0000 7fc78b800700 1 -- 172.21.15.131:0/3506781401 >> 172.21.15.131:0/3506781401 conn(0x7fc7840fd210 msgr2=0x7fc784105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:52.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.838+0000 7fc78b800700 1 -- 172.21.15.131:0/3506781401 shutdown_connections 2024-05-10T05:45:52.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:52.838+0000 7fc78b800700 1 -- 172.21.15.131:0/3506781401 wait complete. 2024-05-10T05:45:52.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:52 smithi131 ceph-mon[37264]: pgmap v660: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 MiB/s wr, 343 op/s 2024-05-10T05:45:52.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:52 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3119684328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:45:52.867 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T05:45:52.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:52 smithi142 ceph-mon[77667]: pgmap v660: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 MiB/s wr, 343 op/s 2024-05-10T05:45:52.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:52 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3119684328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:45:53.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:52 smithi157 ceph-mon[78427]: pgmap v660: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 MiB/s wr, 343 op/s 2024-05-10T05:45:53.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:52 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3119684328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:45:53.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.136+0000 7fea71400700 1 -- 172.21.15.131:0/2097211672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea6c1036a0 msgr2=0x7fea6c109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:53.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.136+0000 7fea71400700 1 --2- 172.21.15.131:0/2097211672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea6c1036a0 0x7fea6c109f50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea6400b0a0 tx=0x7fea6400b3b0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.136+0000 7fea71400700 1 -- 172.21.15.131:0/2097211672 shutdown_connections 2024-05-10T05:45:53.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.136+0000 7fea71400700 1 --2- 172.21.15.131:0/2097211672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea6c1036a0 0x7fea6c109f50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.136+0000 7fea71400700 1 --2- 172.21.15.131:0/2097211672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c101be0 0x7fea6c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.136+0000 7fea71400700 1 --2- 172.21.15.131:0/2097211672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea6c102d00 0x7fea6c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.136+0000 7fea71400700 1 -- 172.21.15.131:0/2097211672 >> 172.21.15.131:0/2097211672 conn(0x7fea6c0fd1d0 msgr2=0x7fea6c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:53.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.136+0000 7fea71400700 1 -- 172.21.15.131:0/2097211672 shutdown_connections 2024-05-10T05:45:53.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.136+0000 7fea71400700 1 -- 172.21.15.131:0/2097211672 wait complete. 2024-05-10T05:45:53.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.137+0000 7fea71400700 1 Processor -- start 2024-05-10T05:45:53.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.137+0000 7fea71400700 1 -- start start 2024-05-10T05:45:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea71400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea6c101be0 0x7fea6c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea71400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c102d00 0x7fea6c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea71400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea6c1036a0 0x7fea6c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea71400700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c1a1d40 con 0x7fea6c1036a0 2024-05-10T05:45:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea71400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c1a1eb0 con 0x7fea6c101be0 2024-05-10T05:45:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea71400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c1a2070 con 0x7fea6c102d00 2024-05-10T05:45:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea6b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea6c1036a0 0x7fea6c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea6b200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea6c1036a0 0x7fea6c1a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34394/0 (socket says 172.21.15.131:34394) 2024-05-10T05:45:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea6b200700 1 -- 172.21.15.131:0/3910751984 learned_addr learned my addr 172.21.15.131:0/3910751984 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:53.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea69e00700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c102d00 0x7fea6c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:53.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.138+0000 7fea6a800700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea6c101be0 0x7fea6c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:53.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea6b200700 1 -- 172.21.15.131:0/3910751984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c102d00 msgr2=0x7fea6c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:53.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea6b200700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c102d00 0x7fea6c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea6b200700 1 -- 172.21.15.131:0/3910751984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea6c101be0 msgr2=0x7fea6c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:53.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea6b200700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea6c101be0 0x7fea6c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea6b200700 1 -- 172.21.15.131:0/3910751984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea64009d00 con 0x7fea6c1036a0 2024-05-10T05:45:53.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea6b200700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea6c1036a0 0x7fea6c1a1730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea6400b070 tx=0x7fea640078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:53.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea63400700 1 -- 172.21.15.131:0/3910751984 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6400e040 con 0x7fea6c1036a0 2024-05-10T05:45:53.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea63400700 1 -- 172.21.15.131:0/3910751984 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea6401d470 con 0x7fea6c1036a0 2024-05-10T05:45:53.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea71400700 1 -- 172.21.15.131:0/3910751984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c1a22f0 con 0x7fea6c1036a0 2024-05-10T05:45:53.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea63400700 1 -- 172.21.15.131:0/3910751984 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea640052a0 con 0x7fea6c1036a0 2024-05-10T05:45:53.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.139+0000 7fea71400700 1 -- 172.21.15.131:0/3910751984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c1a27c0 con 0x7fea6c1036a0 2024-05-10T05:45:53.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.140+0000 7fea71400700 1 -- 172.21.15.131:0/3910751984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6c0694d0 con 0x7fea6c1036a0 2024-05-10T05:45:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.142+0000 7fea63400700 1 -- 172.21.15.131:0/3910751984 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fea6401d5e0 con 0x7fea6c1036a0 2024-05-10T05:45:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.143+0000 7fea63400700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea500a1240 0x7fea500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.144+0000 7fea63400700 1 -- 172.21.15.131:0/3910751984 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fea640cdf70 con 0x7fea6c1036a0 2024-05-10T05:45:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.144+0000 7fea6a800700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea500a1240 0x7fea500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.144+0000 7fea6a800700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea500a1240 0x7fea500a3750 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fea6c102c00 tx=0x7fea5c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.146+0000 7fea63400700 1 -- 172.21.15.131:0/3910751984 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fea640977b0 con 0x7fea6c1036a0 2024-05-10T05:45:53.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.353+0000 7fea71400700 1 -- 172.21.15.131:0/3910751984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea6c1a2ae0 con 0x7fea6c1036a0 2024-05-10T05:45:53.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.353+0000 7fea63400700 1 -- 172.21.15.131:0/3910751984 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea6409add0 con 0x7fea6c1036a0 2024-05-10T05:45:53.353 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-10T05:45:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 -- 172.21.15.131:0/3910751984 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea500a1240 msgr2=0x7fea500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea500a1240 0x7fea500a3750 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fea6c102c00 tx=0x7fea5c009260 comp rx=0 tx=0).stop 2024-05-10T05:45:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 -- 172.21.15.131:0/3910751984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea6c1036a0 msgr2=0x7fea6c1a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:53.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea6c1036a0 0x7fea6c1a1730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea6400b070 tx=0x7fea640078a0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 -- 172.21.15.131:0/3910751984 shutdown_connections 2024-05-10T05:45:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fea500a1240 0x7fea500a3750 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea6c1036a0 0x7fea6c1a1730 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea6c101be0 0x7fea6c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 --2- 172.21.15.131:0/3910751984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6c102d00 0x7fea6c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 -- 172.21.15.131:0/3910751984 >> 172.21.15.131:0/3910751984 conn(0x7fea6c0fd1d0 msgr2=0x7fea6c073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 -- 172.21.15.131:0/3910751984 shutdown_connections 2024-05-10T05:45:53.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.356+0000 7fea71400700 1 -- 172.21.15.131:0/3910751984 wait complete. 2024-05-10T05:45:53.382 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T05:45:53.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.652+0000 7f06b1800700 1 -- 172.21.15.131:0/1362224214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ac102d00 msgr2=0x7f06ac103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:53.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.652+0000 7f06b1800700 1 --2- 172.21.15.131:0/1362224214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ac102d00 0x7f06ac103110 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0694009990 tx=0x7f0694009ca0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.652+0000 7f06b1800700 1 -- 172.21.15.131:0/1362224214 shutdown_connections 2024-05-10T05:45:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.652+0000 7f06b1800700 1 --2- 172.21.15.131:0/1362224214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06ac1036a0 0x7f06ac109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.652+0000 7f06b1800700 1 --2- 172.21.15.131:0/1362224214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ac101be0 0x7f06ac101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.652+0000 7f06b1800700 1 --2- 172.21.15.131:0/1362224214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ac102d00 0x7f06ac103110 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.652+0000 7f06b1800700 1 -- 172.21.15.131:0/1362224214 >> 172.21.15.131:0/1362224214 conn(0x7f06ac0fd1d0 msgr2=0x7f06ac0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.652+0000 7f06b1800700 1 -- 172.21.15.131:0/1362224214 shutdown_connections 2024-05-10T05:45:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.652+0000 7f06b1800700 1 -- 172.21.15.131:0/1362224214 wait complete. 2024-05-10T05:45:53.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.654+0000 7f06b1800700 1 Processor -- start 2024-05-10T05:45:53.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.654+0000 7f06b1800700 1 -- start start 2024-05-10T05:45:53.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.654+0000 7f06b1800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ac101be0 0x7f06ac19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:53.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06b1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06ac102d00 0x7f06ac19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:53.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06b1800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ac1036a0 0x7f06ac1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:53.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06b1800700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1a1c20 con 0x7f06ac101be0 2024-05-10T05:45:53.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06b1800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1a1d90 con 0x7f06ac102d00 2024-05-10T05:45:53.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06b1800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1a1f50 con 0x7f06ac1036a0 2024-05-10T05:45:53.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06aaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ac101be0 0x7f06ac19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:53.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06aaa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ac101be0 0x7f06ac19a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34414/0 (socket says 172.21.15.131:34414) 2024-05-10T05:45:53.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06aaa00700 1 -- 172.21.15.131:0/3313291073 learned_addr learned my addr 172.21.15.131:0/3313291073 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:53.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06a3e00700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06ac102d00 0x7f06ac19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:53.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06ab400700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ac1036a0 0x7f06ac1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:53.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06aaa00700 1 -- 172.21.15.131:0/3313291073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ac1036a0 msgr2=0x7f06ac1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:53.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06aaa00700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ac1036a0 0x7f06ac1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06aaa00700 1 -- 172.21.15.131:0/3313291073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06ac102d00 msgr2=0x7f06ac19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:53.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06aaa00700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06ac102d00 0x7f06ac19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06aaa00700 1 -- 172.21.15.131:0/3313291073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c009520 con 0x7f06ac101be0 2024-05-10T05:45:53.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06a3e00700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06ac102d00 0x7f06ac19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:45:53.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.655+0000 7f06ab400700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ac1036a0 0x7f06ac1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:45:53.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.656+0000 7f06aaa00700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ac101be0 0x7f06ac19a140 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f069c00ea10 tx=0x7f069c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:53.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.656+0000 7f06a3400700 1 -- 172.21.15.131:0/3313291073 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c00cc90 con 0x7f06ac101be0 2024-05-10T05:45:53.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.656+0000 7f06a3400700 1 -- 172.21.15.131:0/3313291073 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f069c00cdf0 con 0x7f06ac101be0 2024-05-10T05:45:53.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.656+0000 7f06a3400700 1 -- 172.21.15.131:0/3313291073 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c018710 con 0x7f06ac101be0 2024-05-10T05:45:53.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.656+0000 7f06b1800700 1 -- 172.21.15.131:0/3313291073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06940095f0 con 0x7f06ac101be0 2024-05-10T05:45:53.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.656+0000 7f06b1800700 1 -- 172.21.15.131:0/3313291073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac1a2560 con 0x7f06ac101be0 2024-05-10T05:45:53.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.657+0000 7f06b1800700 1 -- 172.21.15.131:0/3313291073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac0694b0 con 0x7f06ac101be0 2024-05-10T05:45:53.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.659+0000 7f06a3400700 1 -- 172.21.15.131:0/3313291073 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f069c0109f0 con 0x7f06ac101be0 2024-05-10T05:45:53.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.660+0000 7f06a3400700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f06900a1240 0x7f06900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:53.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.661+0000 7f06a3400700 1 -- 172.21.15.131:0/3313291073 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f069c014070 con 0x7f06ac101be0 2024-05-10T05:45:53.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.661+0000 7f06a3e00700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f06900a1240 0x7f06900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:53.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.661+0000 7f06a3e00700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f06900a1240 0x7f06900a3750 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f069400b2e0 tx=0x7f069401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:53.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.663+0000 7f06a3400700 1 -- 172.21.15.131:0/3313291073 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f069c096950 con 0x7f06ac101be0 2024-05-10T05:45:53.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:53 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3506781401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:45:53.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:53 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3910751984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:45:53.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.873+0000 7f06b1800700 1 -- 172.21.15.131:0/3313291073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06ac10d5c0 con 0x7f06ac101be0 2024-05-10T05:45:53.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.874+0000 7f06a3400700 1 -- 172.21.15.131:0/3313291073 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f069c099f70 con 0x7f06ac101be0 2024-05-10T05:45:53.874 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.876+0000 7f06b1800700 1 -- 172.21.15.131:0/3313291073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f06900a1240 msgr2=0x7f06900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f06900a1240 0x7f06900a3750 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f069400b2e0 tx=0x7f069401a040 comp rx=0 tx=0).stop 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 -- 172.21.15.131:0/3313291073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ac101be0 msgr2=0x7f06ac19a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ac101be0 0x7f06ac19a140 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f069c00ea10 tx=0x7f069c00edd0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 -- 172.21.15.131:0/3313291073 shutdown_connections 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f06900a1240 0x7f06900a3750 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ac101be0 0x7f06ac19a140 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ac1036a0 0x7f06ac1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 --2- 172.21.15.131:0/3313291073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06ac102d00 0x7f06ac19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 -- 172.21.15.131:0/3313291073 >> 172.21.15.131:0/3313291073 conn(0x7f06ac0fd1d0 msgr2=0x7f06ac073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 -- 172.21.15.131:0/3313291073 shutdown_connections 2024-05-10T05:45:53.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:53.877+0000 7f06b1800700 1 -- 172.21.15.131:0/3313291073 wait complete. 2024-05-10T05:45:53.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:53 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3506781401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:45:53.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:53 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3910751984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:45:53.902 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T05:45:54.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:53 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3506781401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T05:45:54.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:53 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3910751984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T05:45:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.166+0000 7f326d600700 1 -- 172.21.15.131:0/2361605498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3268101be0 msgr2=0x7f3268101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.166+0000 7f326d600700 1 --2- 172.21.15.131:0/2361605498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3268101be0 0x7f3268101ff0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3258009990 tx=0x7f3258009ca0 comp rx=0 tx=0).stop 2024-05-10T05:45:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.166+0000 7f326d600700 1 -- 172.21.15.131:0/2361605498 shutdown_connections 2024-05-10T05:45:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.166+0000 7f326d600700 1 --2- 172.21.15.131:0/2361605498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3268101be0 0x7f3268101ff0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.166+0000 7f326d600700 1 --2- 172.21.15.131:0/2361605498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32681036a0 0x7f3268109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.166+0000 7f326d600700 1 --2- 172.21.15.131:0/2361605498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3268102d00 0x7f3268103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.166+0000 7f326d600700 1 -- 172.21.15.131:0/2361605498 >> 172.21.15.131:0/2361605498 conn(0x7f32680fd1d0 msgr2=0x7f32680ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.166+0000 7f326d600700 1 -- 172.21.15.131:0/2361605498 shutdown_connections 2024-05-10T05:45:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.166+0000 7f326d600700 1 -- 172.21.15.131:0/2361605498 wait complete. 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.167+0000 7f326d600700 1 Processor -- start 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.167+0000 7f326d600700 1 -- start start 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.167+0000 7f326d600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3268101be0 0x7f326819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.167+0000 7f326d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268102d00 0x7f326819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.167+0000 7f326d600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32681036a0 0x7f32681a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.167+0000 7f326d600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681a1c60 con 0x7f3268101be0 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.167+0000 7f326d600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681a1dd0 con 0x7f3268102d00 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.167+0000 7f326d600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681a1f90 con 0x7f32681036a0 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3266a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3268101be0 0x7f326819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3266a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3268101be0 0x7f326819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34442/0 (socket says 172.21.15.131:34442) 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3266a00700 1 -- 172.21.15.131:0/1985594830 learned_addr learned my addr 172.21.15.131:0/1985594830 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3266000700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268102d00 0x7f326819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3267400700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32681036a0 0x7f32681a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3266a00700 1 -- 172.21.15.131:0/1985594830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32681036a0 msgr2=0x7f32681a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3266a00700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32681036a0 0x7f32681a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3266a00700 1 -- 172.21.15.131:0/1985594830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268102d00 msgr2=0x7f326819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3266a00700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268102d00 0x7f326819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3266a00700 1 -- 172.21.15.131:0/1985594830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580095f0 con 0x7f3268101be0 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.168+0000 7f3266a00700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3268101be0 0x7f326819a180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3258000c00 tx=0x7f3258005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.169+0000 7f325f400700 1 -- 172.21.15.131:0/1985594830 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325801d070 con 0x7f3268101be0 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.169+0000 7f325f400700 1 -- 172.21.15.131:0/1985594830 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3258021410 con 0x7f3268101be0 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.169+0000 7f326d600700 1 -- 172.21.15.131:0/1985594830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32681a2210 con 0x7f3268101be0 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.169+0000 7f325f400700 1 -- 172.21.15.131:0/1985594830 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258017650 con 0x7f3268101be0 2024-05-10T05:45:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.169+0000 7f326d600700 1 -- 172.21.15.131:0/1985594830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32681a2690 con 0x7f3268101be0 2024-05-10T05:45:54.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.170+0000 7f326d600700 1 -- 172.21.15.131:0/1985594830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32680694b0 con 0x7f3268101be0 2024-05-10T05:45:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.173+0000 7f325f400700 1 -- 172.21.15.131:0/1985594830 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3258021580 con 0x7f3268101be0 2024-05-10T05:45:54.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.175+0000 7f325f400700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f324c0a1240 0x7f324c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.175+0000 7f325f400700 1 -- 172.21.15.131:0/1985594830 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f32580ce130 con 0x7f3268101be0 2024-05-10T05:45:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.175+0000 7f3266000700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f324c0a1240 0x7f324c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.176+0000 7f3266000700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f324c0a1240 0x7f324c0a3750 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f326819b220 tx=0x7f3254009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:54.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.178+0000 7f325f400700 1 -- 172.21.15.131:0/1985594830 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f32580978f0 con 0x7f3268101be0 2024-05-10T05:45:54.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.399+0000 7f326d600700 1 -- 172.21.15.131:0/1985594830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f326810d410 con 0x7f3268101be0 2024-05-10T05:45:54.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.399+0000 7f325f400700 1 -- 172.21.15.131:0/1985594830 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) v1 ==== 81+0+602 (secure 0 0 0) 0x7f325809af10 con 0x7f3268101be0 2024-05-10T05:45:54.399 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/3754674139 2024-05-11T05:26:14.753805+0000 2024-05-10T05:45:54.399 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1436248426 2024-05-11T05:26:14.753805+0000 2024-05-10T05:45:54.399 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/2071559003 2024-05-11T05:26:14.753805+0000 2024-05-10T05:45:54.399 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/3754674139 2024-05-11T05:26:14.753805+0000 2024-05-10T05:45:54.400 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3122376975 2024-05-11T05:25:49.184015+0000 2024-05-10T05:45:54.400 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6800/2225571046 2024-05-11T05:25:49.184015+0000 2024-05-10T05:45:54.400 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6801/2225571046 2024-05-11T05:25:49.184015+0000 2024-05-10T05:45:54.400 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/1953348051 2024-05-11T05:25:49.184015+0000 2024-05-10T05:45:54.400 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/2872216821 2024-05-11T05:25:49.184015+0000 2024-05-10T05:45:54.400 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:0/3302654208 2024-05-11T05:26:14.753805+0000 2024-05-10T05:45:54.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 -- 172.21.15.131:0/1985594830 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f324c0a1240 msgr2=0x7f324c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f324c0a1240 0x7f324c0a3750 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f326819b220 tx=0x7f3254009260 comp rx=0 tx=0).stop 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 -- 172.21.15.131:0/1985594830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3268101be0 msgr2=0x7f326819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3268101be0 0x7f326819a180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3258000c00 tx=0x7f3258005b00 comp rx=0 tx=0).stop 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 -- 172.21.15.131:0/1985594830 shutdown_connections 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f324c0a1240 0x7f324c0a3750 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3268101be0 0x7f326819a180 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32681036a0 0x7f32681a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 --2- 172.21.15.131:0/1985594830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268102d00 0x7f326819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 -- 172.21.15.131:0/1985594830 >> 172.21.15.131:0/1985594830 conn(0x7f32680fd1d0 msgr2=0x7f3268073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 -- 172.21.15.131:0/1985594830 shutdown_connections 2024-05-10T05:45:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:54.402+0000 7f326d600700 1 -- 172.21.15.131:0/1985594830 wait complete. 2024-05-10T05:45:54.403 INFO:teuthology.orchestra.run.smithi131.stderr:listed 10 entries 2024-05-10T05:45:54.427 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T05:45:54.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:45:54.428 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T05:45:54.532 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-10T05:45:54.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:45:54.532 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T05:45:54.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:45:54.596 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-10T05:45:54.640 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T05:45:54.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:45:54.640 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T05:45:54.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:54 smithi131 ceph-mon[37264]: pgmap v661: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 219 op/s 2024-05-10T05:45:54.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:54 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3313291073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:45:54.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:54 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1985594830' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T05:45:54.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T05:45:54.749 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-05-10T05:45:54.883 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-10T05:45:54.894 INFO:tasks.cephadm:Teardown begin 2024-05-10T05:45:54.895 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T05:45:54.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:45:54.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:54 smithi142 ceph-mon[77667]: pgmap v661: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 219 op/s 2024-05-10T05:45:54.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:54 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3313291073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:45:54.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:54 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1985594830' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T05:45:55.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:54 smithi157 ceph-mon[78427]: pgmap v661: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 219 op/s 2024-05-10T05:45:55.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:54 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3313291073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T05:45:55.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:54 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1985594830' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T05:45:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.164+0000 7f669cc00700 1 -- 172.21.15.131:0/250011324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66981036a0 msgr2=0x7f6698109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.164+0000 7f669cc00700 1 --2- 172.21.15.131:0/250011324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66981036a0 0x7f6698109f50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f669000b0a0 tx=0x7f669000b3b0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.164+0000 7f669cc00700 1 -- 172.21.15.131:0/250011324 shutdown_connections 2024-05-10T05:45:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.164+0000 7f669cc00700 1 --2- 172.21.15.131:0/250011324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66981036a0 0x7f6698109f50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.164+0000 7f669cc00700 1 --2- 172.21.15.131:0/250011324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6698101be0 0x7f6698101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.164+0000 7f669cc00700 1 --2- 172.21.15.131:0/250011324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698102d00 0x7f6698103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.164+0000 7f669cc00700 1 -- 172.21.15.131:0/250011324 >> 172.21.15.131:0/250011324 conn(0x7f66980fd1d0 msgr2=0x7f66980ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.164+0000 7f669cc00700 1 -- 172.21.15.131:0/250011324 shutdown_connections 2024-05-10T05:45:55.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.164+0000 7f669cc00700 1 -- 172.21.15.131:0/250011324 wait complete. 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.165+0000 7f669cc00700 1 Processor -- start 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.165+0000 7f669cc00700 1 -- start start 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.165+0000 7f669cc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698101be0 0x7f669819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.165+0000 7f669cc00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698102d00 0x7f669819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f669cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66981036a0 0x7f66981a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f669cc00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66981a1e00 con 0x7f6698102d00 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f669cc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66981a1f70 con 0x7f66981036a0 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f669cc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66981a2130 con 0x7f6698101be0 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6695400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698102d00 0x7f669819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6695400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698102d00 0x7f669819a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34456/0 (socket says 172.21.15.131:34456) 2024-05-10T05:45:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6695400700 1 -- 172.21.15.131:0/1051826573 learned_addr learned my addr 172.21.15.131:0/1051826573 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6695e00700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698101be0 0x7f669819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6696800700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66981036a0 0x7f66981a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6695400700 1 -- 172.21.15.131:0/1051826573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698101be0 msgr2=0x7f669819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6695400700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698101be0 0x7f669819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6695400700 1 -- 172.21.15.131:0/1051826573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66981036a0 msgr2=0x7f66981a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6695400700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66981036a0 0x7f66981a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6695400700 1 -- 172.21.15.131:0/1051826573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6690009d00 con 0x7f6698102d00 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.166+0000 7f6695400700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698102d00 0x7f669819a860 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f668400e9a0 tx=0x7f668400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.167+0000 7f668ea00700 1 -- 172.21.15.131:0/1051826573 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684009790 con 0x7f6698102d00 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.167+0000 7f668ea00700 1 -- 172.21.15.131:0/1051826573 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6684010430 con 0x7f6698102d00 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.167+0000 7f669cc00700 1 -- 172.21.15.131:0/1051826573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66981a2410 con 0x7f6698102d00 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.167+0000 7f668ea00700 1 -- 172.21.15.131:0/1051826573 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684018680 con 0x7f6698102d00 2024-05-10T05:45:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.167+0000 7f669cc00700 1 -- 172.21.15.131:0/1051826573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66981a2920 con 0x7f6698102d00 2024-05-10T05:45:55.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.168+0000 7f669cc00700 1 -- 172.21.15.131:0/1051826573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66980694b0 con 0x7f6698102d00 2024-05-10T05:45:55.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.170+0000 7f668ea00700 1 -- 172.21.15.131:0/1051826573 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f66840105a0 con 0x7f6698102d00 2024-05-10T05:45:55.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.171+0000 7f668ea00700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f667c0a1120 0x7f667c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:55.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.171+0000 7f668ea00700 1 -- 172.21.15.131:0/1051826573 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f6684014070 con 0x7f6698102d00 2024-05-10T05:45:55.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.171+0000 7f6695e00700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f667c0a1120 0x7f667c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:55.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.172+0000 7f6695e00700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f667c0a1120 0x7f667c0a3630 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6698102c00 tx=0x7f6688007400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:55.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.176+0000 7f668ea00700 1 -- 172.21.15.131:0/1051826573 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6684052d60 con 0x7f6698102d00 2024-05-10T05:45:55.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.428+0000 7f669cc00700 1 -- 172.21.15.131:0/1051826573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f669810d440 con 0x7f6698102d00 2024-05-10T05:45:55.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.429+0000 7f668ea00700 1 -- 172.21.15.131:0/1051826573 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10931 (secure 0 0 0) 0x7f66840961d0 con 0x7f6698102d00 2024-05-10T05:45:55.429 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:45:55.429 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:45:08.699705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:45:55.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.431+0000 7f669cc00700 1 -- 172.21.15.131:0/1051826573 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f667c0a1120 msgr2=0x7f667c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:55.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.431+0000 7f669cc00700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f667c0a1120 0x7f667c0a3630 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6698102c00 tx=0x7f6688007400 comp rx=0 tx=0).stop 2024-05-10T05:45:55.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.431+0000 7f669cc00700 1 -- 172.21.15.131:0/1051826573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698102d00 msgr2=0x7f669819a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:55.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.431+0000 7f669cc00700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698102d00 0x7f669819a860 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f668400e9a0 tx=0x7f668400ecb0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.432+0000 7f669cc00700 1 -- 172.21.15.131:0/1051826573 shutdown_connections 2024-05-10T05:45:55.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.432+0000 7f669cc00700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f667c0a1120 0x7f667c0a3630 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.432+0000 7f669cc00700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66981036a0 0x7f66981a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.432+0000 7f669cc00700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6698101be0 0x7f669819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.432+0000 7f669cc00700 1 --2- 172.21.15.131:0/1051826573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6698102d00 0x7f669819a860 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.432+0000 7f669cc00700 1 -- 172.21.15.131:0/1051826573 >> 172.21.15.131:0/1051826573 conn(0x7f66980fd1d0 msgr2=0x7f6698073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:55.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.432+0000 7f669cc00700 1 -- 172.21.15.131:0/1051826573 shutdown_connections 2024-05-10T05:45:55.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.432+0000 7f669cc00700 1 -- 172.21.15.131:0/1051826573 wait complete. 2024-05-10T05:45:55.433 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 30 2024-05-10T05:45:55.457 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T05:45:55.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:55 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1051826573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:45:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.721+0000 7f40cc600700 1 -- 172.21.15.131:0/1452899978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40c4101be0 msgr2=0x7f40c4101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.721+0000 7f40cc600700 1 --2- 172.21.15.131:0/1452899978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40c4101be0 0x7f40c4101ff0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009990 tx=0x7f40b8009ca0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.721+0000 7f40cc600700 1 -- 172.21.15.131:0/1452899978 shutdown_connections 2024-05-10T05:45:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.721+0000 7f40cc600700 1 --2- 172.21.15.131:0/1452899978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40c4101be0 0x7f40c4101ff0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.721+0000 7f40cc600700 1 --2- 172.21.15.131:0/1452899978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c41036a0 0x7f40c4109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.721+0000 7f40cc600700 1 --2- 172.21.15.131:0/1452899978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c4102d00 0x7f40c4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.721+0000 7f40cc600700 1 -- 172.21.15.131:0/1452899978 >> 172.21.15.131:0/1452899978 conn(0x7f40c40fd210 msgr2=0x7f40c40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:55.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.721+0000 7f40cc600700 1 -- 172.21.15.131:0/1452899978 shutdown_connections 2024-05-10T05:45:55.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.721+0000 7f40cc600700 1 -- 172.21.15.131:0/1452899978 wait complete. 2024-05-10T05:45:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.722+0000 7f40cc600700 1 Processor -- start 2024-05-10T05:45:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.722+0000 7f40cc600700 1 -- start start 2024-05-10T05:45:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40cc600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c4101be0 0x7f40c4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40cc600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40c4102d00 0x7f40c4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40cc600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c41036a0 0x7f40c419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40cc600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c419fac0 con 0x7f40c4102d00 2024-05-10T05:45:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40cc600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c419fc30 con 0x7f40c41036a0 2024-05-10T05:45:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40cc600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c419fdf0 con 0x7f40c4101be0 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40c9200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40c4102d00 0x7f40c4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40c9200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40c4102d00 0x7f40c4198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34464/0 (socket says 172.21.15.131:34464) 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40c9200700 1 -- 172.21.15.131:0/3708770246 learned_addr learned my addr 172.21.15.131:0/3708770246 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40c9c00700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c4101be0 0x7f40c4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40ca600700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c41036a0 0x7f40c419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40c9200700 1 -- 172.21.15.131:0/3708770246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c4101be0 msgr2=0x7f40c4197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40c9200700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c4101be0 0x7f40c4197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40c9200700 1 -- 172.21.15.131:0/3708770246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c41036a0 msgr2=0x7f40c419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.723+0000 7f40c9200700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c41036a0 0x7f40c419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.724+0000 7f40c9200700 1 -- 172.21.15.131:0/3708770246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b80095f0 con 0x7f40c4102d00 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.724+0000 7f40c9200700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40c4102d00 0x7f40c4198520 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f40b000e9a0 tx=0x7f40b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.724+0000 7f40c2000700 1 -- 172.21.15.131:0/3708770246 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0009790 con 0x7f40c4102d00 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.724+0000 7f40cc600700 1 -- 172.21.15.131:0/3708770246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c41a00d0 con 0x7f40c4102d00 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.724+0000 7f40c2000700 1 -- 172.21.15.131:0/3708770246 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b0010430 con 0x7f40c4102d00 2024-05-10T05:45:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.724+0000 7f40cc600700 1 -- 172.21.15.131:0/3708770246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c41a0630 con 0x7f40c4102d00 2024-05-10T05:45:55.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.724+0000 7f40c2000700 1 -- 172.21.15.131:0/3708770246 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0018620 con 0x7f40c4102d00 2024-05-10T05:45:55.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.725+0000 7f40cc600700 1 -- 172.21.15.131:0/3708770246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c4108df0 con 0x7f40c4102d00 2024-05-10T05:45:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.727+0000 7f40c2000700 1 -- 172.21.15.131:0/3708770246 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f40b00105a0 con 0x7f40c4102d00 2024-05-10T05:45:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.728+0000 7f40c2000700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40ac0a1120 0x7f40ac0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.729+0000 7f40c2000700 1 -- 172.21.15.131:0/3708770246 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f40b0014070 con 0x7f40c4102d00 2024-05-10T05:45:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.729+0000 7f40c9c00700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40ac0a1120 0x7f40ac0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.730+0000 7f40c9c00700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40ac0a1120 0x7f40ac0a3630 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40b800b7b0 tx=0x7f40b801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:55.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.731+0000 7f40c2000700 1 -- 172.21.15.131:0/3708770246 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f40b0096440 con 0x7f40c4102d00 2024-05-10T05:45:55.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:55 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1051826573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:45:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.981+0000 7f40cc600700 1 -- 172.21.15.131:0/3708770246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40c40694b0 con 0x7f40c4102d00 2024-05-10T05:45:55.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.982+0000 7f40c2000700 1 -- 172.21.15.131:0/3708770246 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10931 (secure 0 0 0) 0x7f40b0099a60 con 0x7f40c4102d00 2024-05-10T05:45:55.982 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:45:55.983 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2077120233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2077120233},{"type":"v1","addr":"172.21.15.131:6835","nonce":2077120233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3274595788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3274595788},{"type":"v1","addr":"172.21.15.157:6833","nonce":3274595788}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T05:31:51.547495+0000","modified":"2024-05-10T05:45:08.699705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24571,"mds_2":24569,"mds_3":24565,"mds_4":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/3035299290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3035299290},{"type":"v1","addr":"172.21.15.142:6833","nonce":3035299290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/3356790716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3356790716},{"type":"v1","addr":"172.21.15.131:6837","nonce":3356790716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3168061272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3168061272},{"type":"v1","addr":"172.21.15.142:6835","nonce":3168061272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/72787128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":72787128},{"type":"v1","addr":"172.21.15.157:6835","nonce":72787128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/2645029376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2645029376},{"type":"v1","addr":"172.21.15.142:6837","nonce":2645029376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6837/707351222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":707351222},{"type":"v1","addr":"172.21.15.157:6837","nonce":707351222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6839/1794088540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1794088540},{"type":"v1","addr":"172.21.15.157:6839","nonce":1794088540}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24569":{"gid":24569,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6841/2396760252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6840","nonce":2396760252},{"type":"v1","addr":"172.21.15.131:6841","nonce":2396760252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24571":{"gid":24571,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6839/2980869223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2980869223},{"type":"v1","addr":"172.21.15.131:6839","nonce":2980869223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2348955278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2348955278},{"type":"v1","addr":"172.21.15.142:6839","nonce":2348955278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T05:45:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 -- 172.21.15.131:0/3708770246 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40ac0a1120 msgr2=0x7f40ac0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40ac0a1120 0x7f40ac0a3630 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40b800b7b0 tx=0x7f40b801a040 comp rx=0 tx=0).stop 2024-05-10T05:45:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 -- 172.21.15.131:0/3708770246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40c4102d00 msgr2=0x7f40c4198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40c4102d00 0x7f40c4198520 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f40b000e9a0 tx=0x7f40b000ecb0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 -- 172.21.15.131:0/3708770246 shutdown_connections 2024-05-10T05:45:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f40ac0a1120 0x7f40ac0a3630 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c41036a0 0x7f40c419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c4101be0 0x7f40c4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 --2- 172.21.15.131:0/3708770246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40c4102d00 0x7f40c4198520 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:55.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 -- 172.21.15.131:0/3708770246 >> 172.21.15.131:0/3708770246 conn(0x7f40c40fd210 msgr2=0x7f40c4105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:55.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 -- 172.21.15.131:0/3708770246 shutdown_connections 2024-05-10T05:45:55.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:55.985+0000 7f40cc600700 1 -- 172.21.15.131:0/3708770246 wait complete. 2024-05-10T05:45:55.986 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 30 2024-05-10T05:45:56.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T05:45:56.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:55 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1051826573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:45:56.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.275+0000 7f1138200700 1 -- 172.21.15.131:0/3873977328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1130101be0 msgr2=0x7f1130101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:56.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.275+0000 7f1138200700 1 --2- 172.21.15.131:0/3873977328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1130101be0 0x7f1130101ff0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f11240099c0 tx=0x7f1124009cd0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.275+0000 7f1138200700 1 -- 172.21.15.131:0/3873977328 shutdown_connections 2024-05-10T05:45:56.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.275+0000 7f1138200700 1 --2- 172.21.15.131:0/3873977328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11301036a0 0x7f1130109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.275+0000 7f1138200700 1 --2- 172.21.15.131:0/3873977328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1130101be0 0x7f1130101ff0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.275+0000 7f1138200700 1 --2- 172.21.15.131:0/3873977328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1130102d00 0x7f1130103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.275+0000 7f1138200700 1 -- 172.21.15.131:0/3873977328 >> 172.21.15.131:0/3873977328 conn(0x7f11300fd210 msgr2=0x7f11300ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:56.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.275+0000 7f1138200700 1 -- 172.21.15.131:0/3873977328 shutdown_connections 2024-05-10T05:45:56.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.275+0000 7f1138200700 1 -- 172.21.15.131:0/3873977328 wait complete. 2024-05-10T05:45:56.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.276+0000 7f1138200700 1 Processor -- start 2024-05-10T05:45:56.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.276+0000 7f1138200700 1 -- start start 2024-05-10T05:45:56.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1138200700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1130101be0 0x7f1130197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:56.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1138200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1130102d00 0x7f1130198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:56.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1138200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11301036a0 0x7f113019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1138200700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113019fad0 con 0x7f1130101be0 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1138200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113019fc40 con 0x7f11301036a0 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1138200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113019fe00 con 0x7f1130102d00 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1135800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1130101be0 0x7f1130197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1135800700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1130101be0 0x7f1130197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34490/0 (socket says 172.21.15.131:34490) 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1135800700 1 -- 172.21.15.131:0/228849379 learned_addr learned my addr 172.21.15.131:0/228849379 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1134e00700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1130102d00 0x7f1130198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1136200700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11301036a0 0x7f113019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1135800700 1 -- 172.21.15.131:0/228849379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1130102d00 msgr2=0x7f1130198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1135800700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1130102d00 0x7f1130198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1135800700 1 -- 172.21.15.131:0/228849379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11301036a0 msgr2=0x7f113019f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:56.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1135800700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11301036a0 0x7f113019f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.277+0000 7f1135800700 1 -- 172.21.15.131:0/228849379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11240095f0 con 0x7f1130101be0 2024-05-10T05:45:56.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.278+0000 7f1135800700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1130101be0 0x7f1130197ff0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1124009fd0 tx=0x7f112400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:56.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.278+0000 7f112e000700 1 -- 172.21.15.131:0/228849379 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112401d070 con 0x7f1130101be0 2024-05-10T05:45:56.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.278+0000 7f112e000700 1 -- 172.21.15.131:0/228849379 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1124005050 con 0x7f1130101be0 2024-05-10T05:45:56.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.278+0000 7f112e000700 1 -- 172.21.15.131:0/228849379 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124004970 con 0x7f1130101be0 2024-05-10T05:45:56.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.278+0000 7f1138200700 1 -- 172.21.15.131:0/228849379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11301a0080 con 0x7f1130101be0 2024-05-10T05:45:56.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.278+0000 7f1138200700 1 -- 172.21.15.131:0/228849379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11301a0450 con 0x7f1130101be0 2024-05-10T05:45:56.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.279+0000 7f1138200700 1 -- 172.21.15.131:0/228849379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1130108df0 con 0x7f1130101be0 2024-05-10T05:45:56.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.281+0000 7f112e000700 1 -- 172.21.15.131:0/228849379 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1124005450 con 0x7f1130101be0 2024-05-10T05:45:56.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.282+0000 7f112e000700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11180a1120 0x7f11180a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:56.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.283+0000 7f112e000700 1 -- 172.21.15.131:0/228849379 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7f11240cd5b0 con 0x7f1130101be0 2024-05-10T05:45:56.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.283+0000 7f1134e00700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11180a1120 0x7f11180a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:56.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.283+0000 7f1134e00700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11180a1120 0x7f11180a3630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1130199090 tx=0x7f111c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:56.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.287+0000 7f112e000700 1 -- 172.21.15.131:0/228849379 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1124096dd0 con 0x7f1130101be0 2024-05-10T05:45:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.494+0000 7f1138200700 1 -- 172.21.15.131:0/228849379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f113010d410 con 0x7f1130101be0 2024-05-10T05:45:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.496+0000 7f112e000700 1 -- 172.21.15.131:0/228849379 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22604 (secure 0 0 0) 0x7f112409a3f0 con 0x7f1130101be0 2024-05-10T05:45:56.496 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-10T05:45:56.496 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"6e8de474-0e8d-11ef-bc98-c7b262605968","created":"2024-05-10T05:25:18.809652+0000","modified":"2024-05-10T05:32:15.903245+0000","last_up_change":"2024-05-10T05:31:13.892125+0000","last_in_change":"2024-05-10T05:30:57.338719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T05:28:15.347512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T05:31:49.481667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T05:31:50.960751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ad2b260-d7ae-4270-ab31-8855bf4b056a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6803","nonce":3101739550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6805","nonce":3101739550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6809","nonce":3101739550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3101739550},{"type":"v1","addr":"172.21.15.131:6807","nonce":3101739550}]},"public_addr":"172.21.15.131:6803/3101739550","cluster_addr":"172.21.15.131:6805/3101739550","heartbeat_back_addr":"172.21.15.131:6809/3101739550","heartbeat_front_addr":"172.21.15.131:6807/3101739550","state":["exists","up"]},{"osd":1,"uuid":"908f4583-6e05-4832-add9-714bf647e807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6801","nonce":451743952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6803","nonce":451743952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6807","nonce":451743952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":451743952},{"type":"v1","addr":"172.21.15.142:6805","nonce":451743952}]},"public_addr":"172.21.15.142:6801/451743952","cluster_addr":"172.21.15.142:6803/451743952","heartbeat_back_addr":"172.21.15.142:6807/451743952","heartbeat_front_addr":"172.21.15.142:6805/451743952","state":["exists","up"]},{"osd":2,"uuid":"c2da9e21-2747-4101-8566-feb07d70e809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6801","nonce":3652465947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6803","nonce":3652465947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6807","nonce":3652465947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3652465947},{"type":"v1","addr":"172.21.15.157:6805","nonce":3652465947}]},"public_addr":"172.21.15.157:6801/3652465947","cluster_addr":"172.21.15.157:6803/3652465947","heartbeat_back_addr":"172.21.15.157:6807/3652465947","heartbeat_front_addr":"172.21.15.157:6805/3652465947","state":["exists","up"]},{"osd":3,"uuid":"665394bb-71dc-422c-bebc-5b0557311dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6811","nonce":2743036600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6813","nonce":2743036600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6817","nonce":2743036600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2743036600},{"type":"v1","addr":"172.21.15.131:6815","nonce":2743036600}]},"public_addr":"172.21.15.131:6811/2743036600","cluster_addr":"172.21.15.131:6813/2743036600","heartbeat_back_addr":"172.21.15.131:6817/2743036600","heartbeat_front_addr":"172.21.15.131:6815/2743036600","state":["exists","up"]},{"osd":4,"uuid":"a346957d-a63e-4865-b454-204ba17320f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6809","nonce":1165225030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6811","nonce":1165225030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6815","nonce":1165225030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1165225030},{"type":"v1","addr":"172.21.15.142:6813","nonce":1165225030}]},"public_addr":"172.21.15.142:6809/1165225030","cluster_addr":"172.21.15.142:6811/1165225030","heartbeat_back_addr":"172.21.15.142:6815/1165225030","heartbeat_front_addr":"172.21.15.142:6813/1165225030","state":["exists","up"]},{"osd":5,"uuid":"33496762-ef3a-4ac9-99a6-0b85e46fd5df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6809","nonce":1157754257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6811","nonce":1157754257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6815","nonce":1157754257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1157754257},{"type":"v1","addr":"172.21.15.157:6813","nonce":1157754257}]},"public_addr":"172.21.15.157:6809/1157754257","cluster_addr":"172.21.15.157:6811/1157754257","heartbeat_back_addr":"172.21.15.157:6815/1157754257","heartbeat_front_addr":"172.21.15.157:6813/1157754257","state":["exists","up"]},{"osd":6,"uuid":"4f5e4557-11d2-40d7-9cee-aa6e36659940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6819","nonce":1462313866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6821","nonce":1462313866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6825","nonce":1462313866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":1462313866},{"type":"v1","addr":"172.21.15.131:6823","nonce":1462313866}]},"public_addr":"172.21.15.131:6819/1462313866","cluster_addr":"172.21.15.131:6821/1462313866","heartbeat_back_addr":"172.21.15.131:6825/1462313866","heartbeat_front_addr":"172.21.15.131:6823/1462313866","state":["exists","up"]},{"osd":7,"uuid":"bfcb0014-f452-4449-98aa-2f041be39249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6817","nonce":1027002285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6819","nonce":1027002285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6823","nonce":1027002285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1027002285},{"type":"v1","addr":"172.21.15.142:6821","nonce":1027002285}]},"public_addr":"172.21.15.142:6817/1027002285","cluster_addr":"172.21.15.142:6819/1027002285","heartbeat_back_addr":"172.21.15.142:6823/1027002285","heartbeat_front_addr":"172.21.15.142:6821/1027002285","state":["exists","up"]},{"osd":8,"uuid":"1fb1c9d3-d3b2-47db-9e17-16b16513a568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6817","nonce":2904635811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6819","nonce":2904635811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6823","nonce":2904635811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2904635811},{"type":"v1","addr":"172.21.15.157:6821","nonce":2904635811}]},"public_addr":"172.21.15.157:6817/2904635811","cluster_addr":"172.21.15.157:6819/2904635811","heartbeat_back_addr":"172.21.15.157:6823/2904635811","heartbeat_front_addr":"172.21.15.157:6821/2904635811","state":["exists","up"]},{"osd":9,"uuid":"31e98635-37d4-43eb-a609-a43df9f89afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6827","nonce":558554742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6829","nonce":558554742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6833","nonce":558554742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":558554742},{"type":"v1","addr":"172.21.15.131:6831","nonce":558554742}]},"public_addr":"172.21.15.131:6827/558554742","cluster_addr":"172.21.15.131:6829/558554742","heartbeat_back_addr":"172.21.15.131:6833/558554742","heartbeat_front_addr":"172.21.15.131:6831/558554742","state":["exists","up"]},{"osd":10,"uuid":"a073aaf2-1cff-4d5e-8cf7-20f41deeab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6825","nonce":2763887998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6827","nonce":2763887998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6831","nonce":2763887998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2763887998},{"type":"v1","addr":"172.21.15.142:6829","nonce":2763887998}]},"public_addr":"172.21.15.142:6825/2763887998","cluster_addr":"172.21.15.142:6827/2763887998","heartbeat_back_addr":"172.21.15.142:6831/2763887998","heartbeat_front_addr":"172.21.15.142:6829/2763887998","state":["exists","up"]},{"osd":11,"uuid":"6d1207f7-17c7-4e22-85e7-a589fd87f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6825","nonce":1016688620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6827","nonce":1016688620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6831","nonce":1016688620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1016688620},{"type":"v1","addr":"172.21.15.157:6829","nonce":1016688620}]},"public_addr":"172.21.15.157:6825/1016688620","cluster_addr":"172.21.15.157:6827/1016688620","heartbeat_back_addr":"172.21.15.157:6831/1016688620","heartbeat_front_addr":"172.21.15.157:6829/1016688620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:36.707059+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:27:55.267823+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:12.720023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:36.829563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:28:55.365706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:13.298499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:35.817641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:29:54.834072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:12.129838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:35.603497+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:30:54.462865+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T05:31:12.018346+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6801/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/1436248426":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/2071559003":"2024-05-11T05:26:14.753805+0000","172.21.15.131:6800/3754674139":"2024-05-11T05:26:14.753805+0000","172.21.15.131:0/3122376975":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6800/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:6801/2225571046":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/1953348051":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/2872216821":"2024-05-11T05:25:49.184015+0000","172.21.15.131:0/3302654208":"2024-05-11T05:26:14.753805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T05:45:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 -- 172.21.15.131:0/228849379 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11180a1120 msgr2=0x7f11180a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11180a1120 0x7f11180a3630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1130199090 tx=0x7f111c009260 comp rx=0 tx=0).stop 2024-05-10T05:45:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 -- 172.21.15.131:0/228849379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1130101be0 msgr2=0x7f1130197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1130101be0 0x7f1130197ff0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1124009fd0 tx=0x7f112400f690 comp rx=0 tx=0).stop 2024-05-10T05:45:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 -- 172.21.15.131:0/228849379 shutdown_connections 2024-05-10T05:45:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f11180a1120 0x7f11180a3630 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1130101be0 0x7f1130197ff0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11301036a0 0x7f113019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 --2- 172.21.15.131:0/228849379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1130102d00 0x7f1130198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 -- 172.21.15.131:0/228849379 >> 172.21.15.131:0/228849379 conn(0x7f11300fd210 msgr2=0x7f1130105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.498+0000 7f1138200700 1 -- 172.21.15.131:0/228849379 shutdown_connections 2024-05-10T05:45:56.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.499+0000 7f1138200700 1 -- 172.21.15.131:0/228849379 wait complete. 2024-05-10T05:45:56.527 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T05:45:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.792+0000 7fc6eec00700 1 -- 172.21.15.131:0/2289836257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8101be0 msgr2=0x7fc6e8101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.792+0000 7fc6eec00700 1 --2- 172.21.15.131:0/2289836257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8101be0 0x7fc6e8101ff0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009990 tx=0x7fc6d8009ca0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.792+0000 7fc6eec00700 1 -- 172.21.15.131:0/2289836257 shutdown_connections 2024-05-10T05:45:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.792+0000 7fc6eec00700 1 --2- 172.21.15.131:0/2289836257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8101be0 0x7fc6e8101ff0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.792+0000 7fc6eec00700 1 --2- 172.21.15.131:0/2289836257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e81036a0 0x7fc6e8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.792+0000 7fc6eec00700 1 --2- 172.21.15.131:0/2289836257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6e8102d00 0x7fc6e8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.792+0000 7fc6eec00700 1 -- 172.21.15.131:0/2289836257 >> 172.21.15.131:0/2289836257 conn(0x7fc6e80fd210 msgr2=0x7fc6e80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.792+0000 7fc6eec00700 1 -- 172.21.15.131:0/2289836257 shutdown_connections 2024-05-10T05:45:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.792+0000 7fc6eec00700 1 -- 172.21.15.131:0/2289836257 wait complete. 2024-05-10T05:45:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.793+0000 7fc6eec00700 1 Processor -- start 2024-05-10T05:45:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.793+0000 7fc6eec00700 1 -- start start 2024-05-10T05:45:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.793+0000 7fc6eec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e8101be0 0x7fc6e819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.793+0000 7fc6eec00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8102d00 0x7fc6e819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6eec00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6e81036a0 0x7fc6e81a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6eec00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81a1c60 con 0x7fc6e8102d00 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6eec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81a1dd0 con 0x7fc6e8101be0 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6eec00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81a1f90 con 0x7fc6e81036a0 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6e7400700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8102d00 0x7fc6e819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6e7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e8101be0 0x7fc6e819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6e7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e8101be0 0x7fc6e819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:54806/0 (socket says 172.21.15.131:54806) 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6e7e00700 1 -- 172.21.15.131:0/2512148324 learned_addr learned my addr 172.21.15.131:0/2512148324 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6ecc00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6e81036a0 0x7fc6e81a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6e7400700 1 -- 172.21.15.131:0/2512148324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6e81036a0 msgr2=0x7fc6e81a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6e7400700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6e81036a0 0x7fc6e81a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6e7400700 1 -- 172.21.15.131:0/2512148324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e8101be0 msgr2=0x7fc6e819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6e7400700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e8101be0 0x7fc6e819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6e7400700 1 -- 172.21.15.131:0/2512148324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80095f0 con 0x7fc6e8102d00 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6ecc00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6e81036a0 0x7fc6e81a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T05:45:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.794+0000 7fc6e7e00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e8101be0 0x7fc6e819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.795+0000 7fc6e7400700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8102d00 0x7fc6e819a6c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400e9f0 tx=0x7fc6d400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.795+0000 7fc6e4c00700 1 -- 172.21.15.131:0/2512148324 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d400eee0 con 0x7fc6e8102d00 2024-05-10T05:45:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.795+0000 7fc6e4c00700 1 -- 172.21.15.131:0/2512148324 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6d400cbd0 con 0x7fc6e8102d00 2024-05-10T05:45:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.795+0000 7fc6eec00700 1 -- 172.21.15.131:0/2512148324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e81a2270 con 0x7fc6e8102d00 2024-05-10T05:45:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.795+0000 7fc6e4c00700 1 -- 172.21.15.131:0/2512148324 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4018680 con 0x7fc6e8102d00 2024-05-10T05:45:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.795+0000 7fc6eec00700 1 -- 172.21.15.131:0/2512148324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e81a27d0 con 0x7fc6e8102d00 2024-05-10T05:45:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.796+0000 7fc6eec00700 1 -- 172.21.15.131:0/2512148324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e80694b0 con 0x7fc6e8102d00 2024-05-10T05:45:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.798+0000 7fc6e4c00700 1 -- 172.21.15.131:0/2512148324 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc6d400cd40 con 0x7fc6e8102d00 2024-05-10T05:45:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.799+0000 7fc6e4c00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc6cc0a1240 0x7fc6cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.800+0000 7fc6e4c00700 1 -- 172.21.15.131:0/2512148324 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8227+0+0 (secure 0 0 0) 0x7fc6d4014070 con 0x7fc6e8102d00 2024-05-10T05:45:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.800+0000 7fc6e7e00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc6cc0a1240 0x7fc6cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.800+0000 7fc6e7e00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc6cc0a1240 0x7fc6cc0a3750 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800b2e0 tx=0x7fc6d801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:56.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:56.802+0000 7fc6e4c00700 1 -- 172.21.15.131:0/2512148324 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc6d4096470 con 0x7fc6e8102d00 2024-05-10T05:45:56.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:56 smithi142 ceph-mon[77667]: pgmap v662: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 146 op/s 2024-05-10T05:45:56.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:56 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/3708770246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:45:56.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:56 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/228849379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:45:56.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:56 smithi131 ceph-mon[37264]: pgmap v662: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 146 op/s 2024-05-10T05:45:56.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:56 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/3708770246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:45:56.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:56 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/228849379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:45:57.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:56 smithi157 ceph-mon[78427]: pgmap v662: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 146 op/s 2024-05-10T05:45:57.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:56 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/3708770246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T05:45:57.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:56 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/228849379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T05:45:57.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.056+0000 7fc6eec00700 1 -- 172.21.15.131:0/2512148324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc6e810d410 con 0x7fc6e8102d00 2024-05-10T05:45:57.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.069+0000 7fc6e4c00700 1 -- 172.21.15.131:0/2512148324 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc6d4026090 con 0x7fc6e8102d00 2024-05-10T05:45:57.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 -- 172.21.15.131:0/2512148324 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc6cc0a1240 msgr2=0x7fc6cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:57.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc6cc0a1240 0x7fc6cc0a3750 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800b2e0 tx=0x7fc6d801a040 comp rx=0 tx=0).stop 2024-05-10T05:45:57.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 -- 172.21.15.131:0/2512148324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8102d00 msgr2=0x7fc6e819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:57.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8102d00 0x7fc6e819a6c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400e9f0 tx=0x7fc6d400ed00 comp rx=0 tx=0).stop 2024-05-10T05:45:57.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 -- 172.21.15.131:0/2512148324 shutdown_connections 2024-05-10T05:45:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fc6cc0a1240 0x7fc6cc0a3750 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6e81036a0 0x7fc6e81a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e8101be0 0x7fc6e819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 --2- 172.21.15.131:0/2512148324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8102d00 0x7fc6e819a6c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 -- 172.21.15.131:0/2512148324 >> 172.21.15.131:0/2512148324 conn(0x7fc6e80fd210 msgr2=0x7fc6e8073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 -- 172.21.15.131:0/2512148324 shutdown_connections 2024-05-10T05:45:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.073+0000 7fc6eec00700 1 -- 172.21.15.131:0/2512148324 wait complete. 2024-05-10T05:45:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T05:45:57.108 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T05:45:57.394 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-b[115203]: ignoring --setuser ceph since I am not root 2024-05-10T05:45:57.394 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-b[115203]: ignoring --setgroup ceph since I am not root 2024-05-10T05:45:57.394 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-b[115203]: starting mds.b at 2024-05-10T05:45:57.395 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-h[114852]: ignoring --setuser ceph since I am not root 2024-05-10T05:45:57.395 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-h[114852]: ignoring --setgroup ceph since I am not root 2024-05-10T05:45:57.395 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:45:57 smithi142 ceph-mds[114875]: -- 172.21.15.142:0/779613267 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556b171201a0 con 0x556b17180880 2024-05-10T05:45:57.395 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-h[114852]: starting mds.h at 2024-05-10T05:45:57.395 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-k[114170]: ignoring --setuser ceph since I am not root 2024-05-10T05:45:57.395 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-k[114170]: ignoring --setgroup ceph since I am not root 2024-05-10T05:45:57.395 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:45:57 smithi142 ceph-mds[114193]: -- 172.21.15.142:0/507008826 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5654fa7821a0 con 0x5654fa7e2880 2024-05-10T05:45:57.396 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-k[114170]: starting mds.k at 2024-05-10T05:45:57.396 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-e[114513]: ignoring --setuser ceph since I am not root 2024-05-10T05:45:57.396 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-e[114513]: ignoring --setgroup ceph since I am not root 2024-05-10T05:45:57.396 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:45:57 smithi142 ceph-mds[114536]: -- 172.21.15.142:0/3976381956 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c0980dc1a0 con 0x55c09813c880 2024-05-10T05:45:57.396 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:45:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-e[114513]: starting mds.e at 2024-05-10T05:45:57.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.405+0000 7f45cf600700 1 -- 172.21.15.131:0/3019459282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c8102d00 msgr2=0x7f45c8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:57.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.405+0000 7f45cf600700 1 --2- 172.21.15.131:0/3019459282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c8102d00 0x7f45c8103110 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009920 tx=0x7f45b4009c30 comp rx=0 tx=0).stop 2024-05-10T05:45:57.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.405+0000 7f45cf600700 1 -- 172.21.15.131:0/3019459282 shutdown_connections 2024-05-10T05:45:57.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.405+0000 7f45cf600700 1 --2- 172.21.15.131:0/3019459282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45c81036a0 0x7f45c8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:57.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.405+0000 7f45cf600700 1 --2- 172.21.15.131:0/3019459282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c8101be0 0x7f45c8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:57.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.405+0000 7f45cf600700 1 --2- 172.21.15.131:0/3019459282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c8102d00 0x7f45c8103110 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:57.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.405+0000 7f45cf600700 1 -- 172.21.15.131:0/3019459282 >> 172.21.15.131:0/3019459282 conn(0x7f45c80fd1d0 msgr2=0x7f45c80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:57.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.405+0000 7f45cf600700 1 -- 172.21.15.131:0/3019459282 shutdown_connections 2024-05-10T05:45:57.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.405+0000 7f45cf600700 1 -- 172.21.15.131:0/3019459282 wait complete. 2024-05-10T05:45:57.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.406+0000 7f45cf600700 1 Processor -- start 2024-05-10T05:45:57.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45cf600700 1 -- start start 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45cf600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c8101be0 0x7f45c8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45cf600700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45c8102d00 0x7f45c8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45cf600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c81036a0 0x7f45c819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45cf600700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c819fab0 con 0x7f45c8102d00 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45cf600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c819fc20 con 0x7f45c81036a0 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45cf600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c819fde0 con 0x7f45c8101be0 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45c7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45c8102d00 0x7f45c8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45ccc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c8101be0 0x7f45c8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45c7e00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45c8102d00 0x7f45c8198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34598/0 (socket says 172.21.15.131:34598) 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45c7e00700 1 -- 172.21.15.131:0/1305610805 learned_addr learned my addr 172.21.15.131:0/1305610805 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.407+0000 7f45cd600700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c81036a0 0x7f45c819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45c7e00700 1 -- 172.21.15.131:0/1305610805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c8101be0 msgr2=0x7f45c8197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:57.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45c7e00700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c8101be0 0x7f45c8197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:57.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45c7e00700 1 -- 172.21.15.131:0/1305610805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c81036a0 msgr2=0x7f45c819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:57.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45c7e00700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c81036a0 0x7f45c819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:57.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45c7e00700 1 -- 172.21.15.131:0/1305610805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0009d00 con 0x7f45c8102d00 2024-05-10T05:45:57.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45cd600700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c81036a0 0x7f45c819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T05:45:57.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45c7e00700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45c8102d00 0x7f45c8198510 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009230 tx=0x7f45b400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:57.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45c5600700 1 -- 172.21.15.131:0/1305610805 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b401c070 con 0x7f45c8102d00 2024-05-10T05:45:57.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45c5600700 1 -- 172.21.15.131:0/1305610805 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45b400f7d0 con 0x7f45c8102d00 2024-05-10T05:45:57.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45cf600700 1 -- 172.21.15.131:0/1305610805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b4009520 con 0x7f45c8102d00 2024-05-10T05:45:57.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45c5600700 1 -- 172.21.15.131:0/1305610805 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b40038e0 con 0x7f45c8102d00 2024-05-10T05:45:57.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.408+0000 7f45cf600700 1 -- 172.21.15.131:0/1305610805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c81a02a0 con 0x7f45c8102d00 2024-05-10T05:45:57.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.409+0000 7f45cf600700 1 -- 172.21.15.131:0/1305610805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c8108df0 con 0x7f45c8102d00 2024-05-10T05:45:57.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.412+0000 7f45c5600700 1 -- 172.21.15.131:0/1305610805 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f45b4020030 con 0x7f45c8102d00 2024-05-10T05:45:57.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.413+0000 7f45c5600700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f45b00a11c0 0x7f45b00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:57.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.413+0000 7f45c5600700 1 -- 172.21.15.131:0/1305610805 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8657+0+0 (secure 0 0 0) 0x7f45b40cdbb0 con 0x7f45c8102d00 2024-05-10T05:45:57.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.413+0000 7f45ccc00700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f45b00a11c0 0x7f45b00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:57.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.414+0000 7f45ccc00700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f45b00a11c0 0x7f45b00a36d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f45c8102c00 tx=0x7f45bc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:57.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.415+0000 7f45c5600700 1 -- 172.21.15.131:0/1305610805 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f45b40971c0 con 0x7f45c8102d00 2024-05-10T05:45:57.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-a[37241]: 2024-05-10T05:45:57.057+0000 7f58c6600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T05:45:57.460 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:45:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-g[88209]: ignoring --setuser ceph since I am not root 2024-05-10T05:45:57.460 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:45:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-g[88209]: ignoring --setgroup ceph since I am not root 2024-05-10T05:45:57.460 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:45:57 smithi131 ceph-mds[88251]: -- 172.21.15.131:0/442770928 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f65b3821a0 con 0x55f65b3e4880 2024-05-10T05:45:57.460 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:45:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-g[88209]: starting mds.g at 2024-05-10T05:45:57.461 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-a[88598]: ignoring --setuser ceph since I am not root 2024-05-10T05:45:57.461 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-a[88598]: ignoring --setgroup ceph since I am not root 2024-05-10T05:45:57.461 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-mds[88630]: -- 172.21.15.131:0/3554089571 <== mon.0 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5612e86341a0 con 0x5612e8696400 2024-05-10T05:45:57.461 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-a[88598]: starting mds.a at 2024-05-10T05:45:57.461 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:45:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-d[87678]: ignoring --setuser ceph since I am not root 2024-05-10T05:45:57.461 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:45:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-d[87678]: ignoring --setgroup ceph since I am not root 2024-05-10T05:45:57.462 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:45:57 smithi131 ceph-mds[87701]: -- 172.21.15.131:0/1841663194 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ff2064a1a0 con 0x55ff206ac880 2024-05-10T05:45:57.462 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:45:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-d[87678]: starting mds.d at 2024-05-10T05:45:57.553 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:45:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-f[114596]: ignoring --setuser ceph since I am not root 2024-05-10T05:45:57.553 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:45:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-f[114596]: ignoring --setgroup ceph since I am not root 2024-05-10T05:45:57.553 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:45:57 smithi157 ceph-mds[114619]: -- 172.21.15.157:0/459529672 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a7cc3081a0 con 0x55a7cc368d00 2024-05-10T05:45:57.553 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:45:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-f[114596]: starting mds.f at 2024-05-10T05:45:57.553 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:45:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-c[114939]: ignoring --setuser ceph since I am not root 2024-05-10T05:45:57.553 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:45:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-c[114939]: ignoring --setgroup ceph since I am not root 2024-05-10T05:45:57.553 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:45:57 smithi157 ceph-mds[114962]: -- 172.21.15.157:0/3691192604 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563b326661a0 con 0x563b326c6400 2024-05-10T05:45:57.554 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:45:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-c[114939]: starting mds.c at 2024-05-10T05:45:57.554 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:45:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-i[115279]: ignoring --setuser ceph since I am not root 2024-05-10T05:45:57.554 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:45:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-i[115279]: ignoring --setgroup ceph since I am not root 2024-05-10T05:45:57.554 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:45:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-i[115279]: starting mds.i at 2024-05-10T05:45:57.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:57.671+0000 7f45cf600700 1 -- 172.21.15.131:0/1305610805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f45c81087d0 con 0x7f45c8102d00 2024-05-10T05:45:57.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:57 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2512148324' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T05:45:57.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:57 smithi142 ceph-mon[77667]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T05:45:57.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:57 smithi142 ceph-mon[77667]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T05:45:57.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:57 smithi142 ceph-mon[77667]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T05:45:57.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:57 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2512148324' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T05:45:57.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:57 smithi142 ceph-mon[77667]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-10T05:45:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2512148324' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T05:45:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-mon[37264]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T05:45:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-mon[37264]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T05:45:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-mon[37264]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T05:45:57.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2512148324' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T05:45:57.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:57 smithi131 ceph-mon[37264]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-10T05:45:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:57 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2512148324' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T05:45:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:57 smithi157 ceph-mon[78427]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T05:45:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:57 smithi157 ceph-mon[78427]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T05:45:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:57 smithi157 ceph-mon[78427]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T05:45:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:57 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2512148324' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T05:45:58.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:57 smithi157 ceph-mon[78427]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-10T05:45:58.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.076+0000 7f45c5600700 1 -- 172.21.15.131:0/1305610805 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f45b409a7e0 con 0x7f45c8102d00 2024-05-10T05:45:58.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.079+0000 7f45cf600700 1 -- 172.21.15.131:0/1305610805 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f45b00a11c0 msgr2=0x7f45b00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:58.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.079+0000 7f45cf600700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f45b00a11c0 0x7f45b00a36d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f45c8102c00 tx=0x7f45bc009260 comp rx=0 tx=0).stop 2024-05-10T05:45:58.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.079+0000 7f45cf600700 1 -- 172.21.15.131:0/1305610805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45c8102d00 msgr2=0x7f45c8198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:58.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.079+0000 7f45cf600700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45c8102d00 0x7f45c8198510 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009230 tx=0x7f45b400b790 comp rx=0 tx=0).stop 2024-05-10T05:45:58.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.080+0000 7f45cf600700 1 -- 172.21.15.131:0/1305610805 shutdown_connections 2024-05-10T05:45:58.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.080+0000 7f45cf600700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f45b00a11c0 0x7f45b00a36d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:58.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.080+0000 7f45cf600700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c81036a0 0x7f45c819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:58.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.080+0000 7f45cf600700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c8101be0 0x7f45c8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:58.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.080+0000 7f45cf600700 1 --2- 172.21.15.131:0/1305610805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45c8102d00 0x7f45c8198510 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:58.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.080+0000 7f45cf600700 1 -- 172.21.15.131:0/1305610805 >> 172.21.15.131:0/1305610805 conn(0x7f45c80fd1d0 msgr2=0x7f45c8105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:58.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.081+0000 7f45cf600700 1 -- 172.21.15.131:0/1305610805 shutdown_connections 2024-05-10T05:45:58.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.081+0000 7f45cf600700 1 -- 172.21.15.131:0/1305610805 wait complete. 2024-05-10T05:45:58.110 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T05:45:58.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.382+0000 7f9251a00700 1 -- 172.21.15.131:0/113062226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f924c101bd0 msgr2=0x7f924c101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:58.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.382+0000 7f9251a00700 1 --2- 172.21.15.131:0/113062226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f924c101bd0 0x7f924c101fe0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f923c0098f0 tx=0x7f923c009c00 comp rx=0 tx=0).stop 2024-05-10T05:45:58.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.383+0000 7f9251a00700 1 -- 172.21.15.131:0/113062226 shutdown_connections 2024-05-10T05:45:58.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.383+0000 7f9251a00700 1 --2- 172.21.15.131:0/113062226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f924c101bd0 0x7f924c101fe0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:58.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.383+0000 7f9251a00700 1 --2- 172.21.15.131:0/113062226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f924c103690 0x7f924c109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:58.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.383+0000 7f9251a00700 1 --2- 172.21.15.131:0/113062226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c102cf0 0x7f924c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:58.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.383+0000 7f9251a00700 1 -- 172.21.15.131:0/113062226 >> 172.21.15.131:0/113062226 conn(0x7f924c0fd1e0 msgr2=0x7f924c0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:58.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.383+0000 7f9251a00700 1 -- 172.21.15.131:0/113062226 shutdown_connections 2024-05-10T05:45:58.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.383+0000 7f9251a00700 1 -- 172.21.15.131:0/113062226 wait complete. 2024-05-10T05:45:58.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f9251a00700 1 Processor -- start 2024-05-10T05:45:58.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f9251a00700 1 -- start start 2024-05-10T05:45:58.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f9251a00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f924c101bd0 0x7f924c107740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f9251a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c102cf0 0x7f924c107c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f9251a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f924c103690 0x7f924c104350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f9251a00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c104a50 con 0x7f924c101bd0 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f9251a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c104bc0 con 0x7f924c102cf0 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f9251a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c104d80 con 0x7f924c103690 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f924aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f924c101bd0 0x7f924c107740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f924aa00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f924c101bd0 0x7f924c107740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34614/0 (socket says 172.21.15.131:34614) 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f924aa00700 1 -- 172.21.15.131:0/2418803507 learned_addr learned my addr 172.21.15.131:0/2418803507 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.384+0000 7f9243e00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c102cf0 0x7f924c107c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f924b400700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f924c103690 0x7f924c104350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f924aa00700 1 -- 172.21.15.131:0/2418803507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f924c103690 msgr2=0x7f924c104350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f924aa00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f924c103690 0x7f924c104350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:58.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f924aa00700 1 -- 172.21.15.131:0/2418803507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c102cf0 msgr2=0x7f924c107c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:58.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f924aa00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c102cf0 0x7f924c107c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:58.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f924aa00700 1 -- 172.21.15.131:0/2418803507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c009520 con 0x7f924c101bd0 2024-05-10T05:45:58.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f924aa00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f924c101bd0 0x7f924c107740 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f923c000c00 tx=0x7f923c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:58.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f9243400700 1 -- 172.21.15.131:0/2418803507 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c01c070 con 0x7f924c101bd0 2024-05-10T05:45:58.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f9243400700 1 -- 172.21.15.131:0/2418803507 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f923c005500 con 0x7f924c101bd0 2024-05-10T05:45:58.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f9251a00700 1 -- 172.21.15.131:0/2418803507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c1a6620 con 0x7f924c101bd0 2024-05-10T05:45:58.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f9243400700 1 -- 172.21.15.131:0/2418803507 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c003a40 con 0x7f924c101bd0 2024-05-10T05:45:58.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.385+0000 7f9251a00700 1 -- 172.21.15.131:0/2418803507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c1a6a40 con 0x7f924c101bd0 2024-05-10T05:45:58.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.386+0000 7f9251a00700 1 -- 172.21.15.131:0/2418803507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924c0678b0 con 0x7f924c101bd0 2024-05-10T05:45:58.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.388+0000 7f9243400700 1 -- 172.21.15.131:0/2418803507 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f923c003c00 con 0x7f924c101bd0 2024-05-10T05:45:58.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.390+0000 7f9243400700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92300a11c0 0x7f92300a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:58.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.390+0000 7f9243400700 1 -- 172.21.15.131:0/2418803507 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8657+0+0 (secure 0 0 0) 0x7f923c0cd730 con 0x7f924c101bd0 2024-05-10T05:45:58.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.390+0000 7f9243e00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92300a11c0 0x7f92300a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:58.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.390+0000 7f9243e00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92300a11c0 0x7f92300a36d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f924c105330 tx=0x7f9234009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:58.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.394+0000 7f9243400700 1 -- 172.21.15.131:0/2418803507 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f923c096da0 con 0x7f924c101bd0 2024-05-10T05:45:58.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:58.605+0000 7f9251a00700 1 -- 172.21.15.131:0/2418803507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f924c1a6d30 con 0x7f924c101bd0 2024-05-10T05:45:58.894 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: pgmap v664: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1305610805' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: Cluster is now healthy 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.157:6834/862470093,v1:172.21.15.157:6835/862470093] up:boot 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.142:6833/1079295971,v1:172.21.15.142:6835/1079295971] up:boot 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.157:6837/3784037621,v1:172.21.15.157:6838/3784037621] up:boot 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.142:6836/1012185753,v1:172.21.15.142:6838/1012185753] up:boot 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.142:6837/2172126355,v1:172.21.15.142:6839/2172126355] up:boot 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.142:6832/3967792740,v1:172.21.15.142:6834/3967792740] up:boot 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.157:6836/3863032330,v1:172.21.15.157:6839/3863032330] up:boot 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.131:6836/2430005207,v1:172.21.15.131:6838/2430005207] up:boot 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.131:6837/1775943015,v1:172.21.15.131:6839/1775943015] up:boot 2024-05-10T05:45:58.895 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: mds.? [v2:172.21.15.131:6840/1643781580,v1:172.21.15.131:6841/1643781580] up:boot 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1305610805' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: fsmap 12 up:standby 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T05:45:58.896 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:45:58 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2418803507' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T05:45:58.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: pgmap v664: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-10T05:45:58.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1305610805' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T05:45:58.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T05:45:58.960 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: Cluster is now healthy 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.157:6834/862470093,v1:172.21.15.157:6835/862470093] up:boot 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.142:6833/1079295971,v1:172.21.15.142:6835/1079295971] up:boot 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.157:6837/3784037621,v1:172.21.15.157:6838/3784037621] up:boot 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.142:6836/1012185753,v1:172.21.15.142:6838/1012185753] up:boot 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.142:6837/2172126355,v1:172.21.15.142:6839/2172126355] up:boot 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.142:6832/3967792740,v1:172.21.15.142:6834/3967792740] up:boot 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.157:6836/3863032330,v1:172.21.15.157:6839/3863032330] up:boot 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.131:6836/2430005207,v1:172.21.15.131:6838/2430005207] up:boot 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.131:6837/1775943015,v1:172.21.15.131:6839/1775943015] up:boot 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: mds.? [v2:172.21.15.131:6840/1643781580,v1:172.21.15.131:6841/1643781580] up:boot 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1305610805' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: fsmap 12 up:standby 2024-05-10T05:45:58.961 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T05:45:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T05:45:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T05:45:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T05:45:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T05:45:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T05:45:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T05:45:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T05:45:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T05:45:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T05:45:58.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:45:58 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2418803507' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T05:45:59.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: pgmap v664: 97 pgs: 97 active+clean; 6.5 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-10T05:45:59.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1305610805' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T05:45:59.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T05:45:59.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T05:45:59.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: Cluster is now healthy 2024-05-10T05:45:59.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.157:6834/862470093,v1:172.21.15.157:6835/862470093] up:boot 2024-05-10T05:45:59.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.142:6833/1079295971,v1:172.21.15.142:6835/1079295971] up:boot 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.157:6837/3784037621,v1:172.21.15.157:6838/3784037621] up:boot 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.142:6836/1012185753,v1:172.21.15.142:6838/1012185753] up:boot 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.142:6837/2172126355,v1:172.21.15.142:6839/2172126355] up:boot 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.142:6832/3967792740,v1:172.21.15.142:6834/3967792740] up:boot 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.157:6836/3863032330,v1:172.21.15.157:6839/3863032330] up:boot 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.131:6836/2430005207,v1:172.21.15.131:6838/2430005207] up:boot 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.131:6837/1775943015,v1:172.21.15.131:6839/1775943015] up:boot 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: mds.? [v2:172.21.15.131:6840/1643781580,v1:172.21.15.131:6841/1643781580] up:boot 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1305610805' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: fsmap 12 up:standby 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T05:45:59.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T05:45:59.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T05:45:59.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T05:45:59.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T05:45:59.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T05:45:59.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T05:45:59.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='mgr.14154 172.21.15.131:0/1452159079' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T05:45:59.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:45:58 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2418803507' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T05:45:59.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.080+0000 7f9243400700 1 -- 172.21.15.131:0/2418803507 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) v1 ==== 181+0+0 (secure 0 0 0) 0x7f923c09a3c0 con 0x7f924c101bd0 2024-05-10T05:45:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 -- 172.21.15.131:0/2418803507 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92300a11c0 msgr2=0x7f92300a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92300a11c0 0x7f92300a36d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f924c105330 tx=0x7f9234009310 comp rx=0 tx=0).stop 2024-05-10T05:45:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 -- 172.21.15.131:0/2418803507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f924c101bd0 msgr2=0x7f924c107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f924c101bd0 0x7f924c107740 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f923c000c00 tx=0x7f923c00bf60 comp rx=0 tx=0).stop 2024-05-10T05:45:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 -- 172.21.15.131:0/2418803507 shutdown_connections 2024-05-10T05:45:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7f92300a11c0 0x7f92300a36d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f924c101bd0 0x7f924c107740 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f924c103690 0x7f924c104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:59.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 --2- 172.21.15.131:0/2418803507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f924c102cf0 0x7f924c107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:59.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 -- 172.21.15.131:0/2418803507 >> 172.21.15.131:0/2418803507 conn(0x7f924c0fd1e0 msgr2=0x7f924c073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:59.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 -- 172.21.15.131:0/2418803507 shutdown_connections 2024-05-10T05:45:59.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.084+0000 7f9251a00700 1 -- 172.21.15.131:0/2418803507 wait complete. 2024-05-10T05:45:59.087 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-05-10T05:45:59.113 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T05:45:59.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.388+0000 7fca6da00700 1 -- 172.21.15.131:0/98553594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca68069090 msgr2=0x7fca68104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.388+0000 7fca6da00700 1 --2- 172.21.15.131:0/98553594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca68069090 0x7fca68104720 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fca580098f0 tx=0x7fca58009c00 comp rx=0 tx=0).stop 2024-05-10T05:45:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.388+0000 7fca6da00700 1 -- 172.21.15.131:0/98553594 shutdown_connections 2024-05-10T05:45:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.388+0000 7fca6da00700 1 --2- 172.21.15.131:0/98553594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca68107880 0x7fca68109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.388+0000 7fca6da00700 1 --2- 172.21.15.131:0/98553594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca68069090 0x7fca68104720 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.388+0000 7fca6da00700 1 --2- 172.21.15.131:0/98553594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca68104d10 0x7fca681071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.388+0000 7fca6da00700 1 -- 172.21.15.131:0/98553594 >> 172.21.15.131:0/98553594 conn(0x7fca680fc0d0 msgr2=0x7fca680fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:45:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.388+0000 7fca6da00700 1 -- 172.21.15.131:0/98553594 shutdown_connections 2024-05-10T05:45:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.388+0000 7fca6da00700 1 -- 172.21.15.131:0/98553594 wait complete. 2024-05-10T05:45:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.389+0000 7fca6da00700 1 Processor -- start 2024-05-10T05:45:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca6da00700 1 -- start start 2024-05-10T05:45:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca6da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca68069090 0x7fca6819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca6da00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca68104d10 0x7fca6819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca6da00700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca68107880 0x7fca681a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca6da00700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca681a1cc0 con 0x7fca68107880 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca6da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca681a1e30 con 0x7fca68069090 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca6da00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca681a1ff0 con 0x7fca68104d10 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca5fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca68104d10 0x7fca6819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca66a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca68069090 0x7fca6819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca66a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca68069090 0x7fca6819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.131:54900/0 (socket says 172.21.15.131:54900) 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca66a00700 1 -- 172.21.15.131:0/1319020033 learned_addr learned my addr 172.21.15.131:0/1319020033 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca5fe00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca68104d10 0x7fca6819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53364/0 (socket says 172.21.15.131:53364) 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.390+0000 7fca67400700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca68107880 0x7fca681a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:59.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca66a00700 1 -- 172.21.15.131:0/1319020033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca68104d10 msgr2=0x7fca6819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca66a00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca68104d10 0x7fca6819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca66a00700 1 -- 172.21.15.131:0/1319020033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca68107880 msgr2=0x7fca681a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca66a00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca68107880 0x7fca681a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca66a00700 1 -- 172.21.15.131:0/1319020033 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca60009d00 con 0x7fca68069090 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca66a00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca68069090 0x7fca6819a1e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fca58009fd0 tx=0x7fca5800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca5f400700 1 -- 172.21.15.131:0/1319020033 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5801c070 con 0x7fca68069090 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca6da00700 1 -- 172.21.15.131:0/1319020033 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca58009520 con 0x7fca68069090 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca5f400700 1 -- 172.21.15.131:0/1319020033 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca5800f7d0 con 0x7fca68069090 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca5f400700 1 -- 172.21.15.131:0/1319020033 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca58003aa0 con 0x7fca68069090 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.391+0000 7fca6da00700 1 -- 172.21.15.131:0/1319020033 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca681a2560 con 0x7fca68069090 2024-05-10T05:45:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.392+0000 7fca6da00700 1 -- 172.21.15.131:0/1319020033 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca680694d0 con 0x7fca68069090 2024-05-10T05:45:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.395+0000 7fca5f400700 1 -- 172.21.15.131:0/1319020033 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fca58003c00 con 0x7fca68069090 2024-05-10T05:45:59.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.396+0000 7fca5f400700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca4c0a11c0 0x7fca4c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T05:45:59.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.396+0000 7fca5f400700 1 -- 172.21.15.131:0/1319020033 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8161+0+0 (secure 0 0 0) 0x7fca580cd910 con 0x7fca68069090 2024-05-10T05:45:59.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.396+0000 7fca5fe00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca4c0a11c0 0x7fca4c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T05:45:59.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.397+0000 7fca5fe00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca4c0a11c0 0x7fca4c0a36d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fca6819b280 tx=0x7fca50009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T05:45:59.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.399+0000 7fca5f400700 1 -- 172.21.15.131:0/1319020033 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fca58097240 con 0x7fca68069090 2024-05-10T05:45:59.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:45:59.613+0000 7fca6da00700 1 -- 172.21.15.131:0/1319020033 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fca6810d3e0 con 0x7fca68069090 2024-05-10T05:46:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.095+0000 7fca5f400700 1 -- 172.21.15.131:0/1319020033 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) v1 ==== 169+0+0 (secure 0 0 0) 0x7fca580250d0 con 0x7fca68069090 2024-05-10T05:46:00.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 -- 172.21.15.131:0/1319020033 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca4c0a11c0 msgr2=0x7fca4c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:46:00.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca4c0a11c0 0x7fca4c0a36d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fca6819b280 tx=0x7fca50009260 comp rx=0 tx=0).stop 2024-05-10T05:46:00.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 -- 172.21.15.131:0/1319020033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca68069090 msgr2=0x7fca6819a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T05:46:00.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca68069090 0x7fca6819a1e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fca58009fd0 tx=0x7fca5800b790 comp rx=0 tx=0).stop 2024-05-10T05:46:00.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 -- 172.21.15.131:0/1319020033 shutdown_connections 2024-05-10T05:46:00.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.131:6800/2109111580,v1:172.21.15.131:6801/2109111580] conn(0x7fca4c0a11c0 0x7fca4c0a36d0 secure :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fca6819b280 tx=0x7fca50009260 comp rx=0 tx=0).stop 2024-05-10T05:46:00.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca68107880 0x7fca681a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:46:00.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca68069090 0x7fca6819a1e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:46:00.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 --2- 172.21.15.131:0/1319020033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca68104d10 0x7fca6819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T05:46:00.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 -- 172.21.15.131:0/1319020033 >> 172.21.15.131:0/1319020033 conn(0x7fca680fc0d0 msgr2=0x7fca680fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T05:46:00.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.100+0000 7fca6da00700 1 -- 172.21.15.131:0/1319020033 shutdown_connections 2024-05-10T05:46:00.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-10T05:46:00.101+0000 7fca6da00700 1 -- 172.21.15.131:0/1319020033 wait complete. 2024-05-10T05:46:00.102 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-05-10T05:46:00.143 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T05:46:00.209 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T05:46:00.245 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T05:46:00.286 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-10T05:46:00.286 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T05:46:00.328 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T05:46:00.347 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T05:46:00.368 INFO:tasks.cephadm:Stopping all daemons... 2024-05-10T05:46:00.368 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-10T05:46:00.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.a 2024-05-10T05:46:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:46:00 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/2418803507' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T05:46:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:46:00 smithi142 ceph-mon[77667]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T05:46:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:46:00 smithi142 ceph-mon[77667]: pgmap v666: 65 pgs: 65 active+clean; 577 KiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:46:00.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:46:00 smithi142 ceph-mon[77667]: from='client.? 172.21.15.131:0/1319020033' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T05:46:00.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:46:00 smithi142 ceph-mon[77667]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T05:46:00.425 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:46:00 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/2418803507' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T05:46:00.425 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:46:00 smithi131 ceph-mon[37264]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T05:46:00.426 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:46:00 smithi131 ceph-mon[37264]: pgmap v666: 65 pgs: 65 active+clean; 577 KiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:46:00.426 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:46:00 smithi131 ceph-mon[37264]: from='client.? 172.21.15.131:0/1319020033' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T05:46:00.426 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:46:00 smithi131 ceph-mon[37264]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T05:46:00.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:46:00 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/2418803507' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T05:46:00.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:46:00 smithi157 ceph-mon[78427]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T05:46:00.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:46:00 smithi157 ceph-mon[78427]: pgmap v666: 65 pgs: 65 active+clean; 577 KiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T05:46:00.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:46:00 smithi157 ceph-mon[78427]: from='client.? 172.21.15.131:0/1319020033' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T05:46:00.553 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:46:00 smithi157 ceph-mon[78427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T05:46:00.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:46:00 smithi131 systemd[1]: Stopping Ceph mon.a for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:01.153 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:46:00 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-a[37241]: 2024-05-10T05:46:00.743+0000 7f58cd400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:01.153 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:46:00 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-a[37241]: 2024-05-10T05:46:00.743+0000 7f58cd400700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-10T05:46:01.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:May 10 05:46:01 smithi131 bash[117852]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-a 2024-05-10T05:46:01.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.a.service' 2024-05-10T05:46:01.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:01.675 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-10T05:46:01.675 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-10T05:46:01.675 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.b 2024-05-10T05:46:02.122 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:46:01 smithi142 systemd[1]: Stopping Ceph mon.b for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:02.122 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:46:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-b[77644]: 2024-05-10T05:46:01.908+0000 7fe2bf3f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:02.122 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:46:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-b[77644]: 2024-05-10T05:46:01.908+0000 7fe2bf3f0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-10T05:46:02.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 10 05:46:02 smithi142 bash[120080]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-b 2024-05-10T05:46:02.479 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.b.service' 2024-05-10T05:46:02.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:02.531 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-10T05:46:02.531 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-10T05:46:02.532 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.c 2024-05-10T05:46:02.917 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:46:02 smithi157 systemd[1]: Stopping Ceph mon.c for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:02.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:46:02 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-c[78404]: 2024-05-10T05:46:02.770+0000 7f2395fe2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:02.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:46:02 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-c[78404]: 2024-05-10T05:46:02.770+0000 7f2395fe2700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-10T05:46:03.279 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 10 05:46:02 smithi157 bash[119966]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mon-c 2024-05-10T05:46:03.285 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mon.c.service' 2024-05-10T05:46:03.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:03.332 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-10T05:46:03.332 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-10T05:46:03.332 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.x 2024-05-10T05:46:03.702 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:46:03 smithi131 systemd[1]: Stopping Ceph mgr.x for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:03.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:46:03 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:46:03.700+0000 7fd0f7400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:03.960 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:46:03 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x[37514]: 2024-05-10T05:46:03.700+0000 7fd0f7400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-10T05:46:04.408 INFO:journalctl@ceph.mgr.x.smithi131.stdout:May 10 05:46:04 smithi131 bash[117991]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-x 2024-05-10T05:46:04.433 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-10T05:46:04.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:04.517 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-10T05:46:04.517 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-10T05:46:04.517 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.y 2024-05-10T05:46:04.894 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 10 05:46:04 smithi142 systemd[1]: Stopping Ceph mgr.y for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:06.469 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-10T05:46:06.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:06.507 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-10T05:46:06.507 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-10T05:46:06.507 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.z 2024-05-10T05:46:06.803 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:46:06 smithi157 systemd[1]: Stopping Ceph mgr.z for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:07.553 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 10 05:46:07 smithi157 bash[120114]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mgr-z 2024-05-10T05:46:08.376 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-10T05:46:08.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:08.414 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-10T05:46:08.414 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-10T05:46:08.414 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.0 2024-05-10T05:46:08.810 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:46:08 smithi131 systemd[1]: Stopping Ceph osd.0 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:09.210 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:46:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-0[55073]: 2024-05-10T05:46:08.808+0000 7f284e400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:09.210 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:46:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-0[55073]: 2024-05-10T05:46:08.808+0000 7f284e400700 -1 osd.0 85 *** Got signal Terminated *** 2024-05-10T05:46:09.210 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:46:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-0[55073]: 2024-05-10T05:46:08.808+0000 7f284e400700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:46:14.710 INFO:journalctl@ceph.osd.0.smithi131.stdout:May 10 05:46:14 smithi131 bash[118132]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-0 2024-05-10T05:46:16.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.0.service' 2024-05-10T05:46:16.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:16.253 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-10T05:46:16.253 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-10T05:46:16.254 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.1 2024-05-10T05:46:16.632 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:46:16 smithi142 systemd[1]: Stopping Ceph osd.1 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:16.894 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:46:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-1[82760]: 2024-05-10T05:46:16.630+0000 7fec0d923700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:16.894 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:46:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-1[82760]: 2024-05-10T05:46:16.630+0000 7fec0d923700 -1 osd.1 85 *** Got signal Terminated *** 2024-05-10T05:46:16.894 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:46:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-1[82760]: 2024-05-10T05:46:16.630+0000 7fec0d923700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:46:22.394 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 10 05:46:22 smithi142 bash[120370]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-1 2024-05-10T05:46:23.269 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.1.service' 2024-05-10T05:46:23.310 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:23.311 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-10T05:46:23.311 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-10T05:46:23.311 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.2 2024-05-10T05:46:23.803 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:46:23 smithi157 systemd[1]: Stopping Ceph osd.2 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:23.803 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:46:23 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-2[84228]: 2024-05-10T05:46:23.566+0000 7fb1a06d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:23.803 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:46:23 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-2[84228]: 2024-05-10T05:46:23.566+0000 7fb1a06d9700 -1 osd.2 85 *** Got signal Terminated *** 2024-05-10T05:46:23.803 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:46:23 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-2[84228]: 2024-05-10T05:46:23.566+0000 7fb1a06d9700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:46:29.552 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 10 05:46:29 smithi157 bash[120267]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-2 2024-05-10T05:46:30.388 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.2.service' 2024-05-10T05:46:30.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:30.429 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-10T05:46:30.429 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-10T05:46:30.429 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.3 2024-05-10T05:46:30.806 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 10 05:46:30 smithi131 systemd[1]: Stopping Ceph osd.3 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:31.210 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 10 05:46:30 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-3[63713]: 2024-05-10T05:46:30.804+0000 7f32e9a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:31.210 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 10 05:46:30 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-3[63713]: 2024-05-10T05:46:30.804+0000 7f32e9a00700 -1 osd.3 85 *** Got signal Terminated *** 2024-05-10T05:46:31.210 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 10 05:46:30 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-3[63713]: 2024-05-10T05:46:30.804+0000 7f32e9a00700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:46:34.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:34 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:34.424+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:35.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:34 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:34.688+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:35.460 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 10 05:46:35 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-3[63713]: 2024-05-10T05:46:35.056+0000 7f32e5200700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:13.246766+0000 front 2024-05-10T05:46:13.246795+0000 (oldest deadline 2024-05-10T05:46:34.347012+0000) 2024-05-10T05:46:35.460 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:35 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:35.028+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:35.553 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:35 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:35.224+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:35.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:35 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:35.419+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:36.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:35 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:35.735+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:36.372 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:36 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:36.009+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:36.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:36 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:36.370+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:36.553 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:36 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:36.233+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:36.710 INFO:journalctl@ceph.osd.3.smithi131.stdout:May 10 05:46:36 smithi131 bash[118410]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-3 2024-05-10T05:46:37.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:36 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:36.725+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:37.289 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:37 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:37.007+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:37.552 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:37 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:37.188+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:37.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:37 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:37.381+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:37.685 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:37 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:37.287+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:38.127 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.3.service' 2024-05-10T05:46:38.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:37 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:37.764+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:38.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:38.219 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-10T05:46:38.219 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-10T05:46:38.219 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.4 2024-05-10T05:46:38.324 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:38 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:38.015+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:38.553 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:38 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:38.184+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:38.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:38 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:38.417+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:38.644 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 10 05:46:38 smithi142 systemd[1]: Stopping Ceph osd.4 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:38.644 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 10 05:46:38 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-4[91367]: 2024-05-10T05:46:38.443+0000 7f74eb1b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:38.644 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 10 05:46:38 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-4[91367]: 2024-05-10T05:46:38.443+0000 7f74eb1b5700 -1 osd.4 85 *** Got signal Terminated *** 2024-05-10T05:46:38.644 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 10 05:46:38 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-4[91367]: 2024-05-10T05:46:38.443+0000 7f74eb1b5700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:46:38.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:38 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:38.323+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:39.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:38 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:38.802+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:39.371 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:39 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:39.002+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:39.458 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:39 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:39.135+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:39.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:39 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:39.371+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:39.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:39 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:39.458+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:40.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:39 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:39.846+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:40.333 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:39 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:39.978+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:40.458 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:40 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:40.121+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:40.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:40 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:40.333+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:40.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:40 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:40.458+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:41.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:40 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:40.803+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:41.330 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:40 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:40.975+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:41.506 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:41 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:41.139+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:41.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:41 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:41.329+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:41.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:41 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:41.506+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:42.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:41 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:41.843+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:42.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:41 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:41.949+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:42.525 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:42 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:42.179+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:42.644 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 10 05:46:42 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-4[91367]: 2024-05-10T05:46:42.345+0000 7f74e3765700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:18.785424+0000 front 2024-05-10T05:46:18.785453+0000 (oldest deadline 2024-05-10T05:46:42.186239+0000) 2024-05-10T05:46:42.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:42 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:42.302+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:42.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:42 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:42.525+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:43.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:42 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:42.838+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:43.299 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:42 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:42.962+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:43.519 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:43 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:43.174+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:43.644 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 10 05:46:43 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-4[91367]: 2024-05-10T05:46:43.344+0000 7f74e3765700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:18.785424+0000 front 2024-05-10T05:46:18.785453+0000 (oldest deadline 2024-05-10T05:46:42.186239+0000) 2024-05-10T05:46:43.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:43 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:43.299+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:43.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:43 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:43.518+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:43.974 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:43 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:43.667+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:43.974 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:43 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:43.834+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:44.322 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:44 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:44.009+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:44.323 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:44 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:44.009+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:44.394 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 10 05:46:44 smithi142 bash[120625]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-4 2024-05-10T05:46:44.503 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:44 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:44.160+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:44.503 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:44 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:44.160+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:18.443235+0000 front 2024-05-10T05:46:18.443310+0000 (oldest deadline 2024-05-10T05:46:43.743435+0000) 2024-05-10T05:46:44.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:44 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:44.322+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:44.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:44 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:44.502+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:44.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:44 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:44.502+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:44.837 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:44 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:44.660+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:45.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:44 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:44.836+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:45.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:44 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:44.836+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:45.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.4.service' 2024-05-10T05:46:45.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:45.237 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-10T05:46:45.237 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-10T05:46:45.238 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.5 2024-05-10T05:46:45.315 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:45 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:45.043+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:45.315 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:45 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:45.043+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:45.479 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:45 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:45.133+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:45.480 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:45 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:45.133+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:18.443235+0000 front 2024-05-10T05:46:18.443310+0000 (oldest deadline 2024-05-10T05:46:43.743435+0000) 2024-05-10T05:46:45.480 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:45 smithi157 systemd[1]: Stopping Ceph osd.5 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:45.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:45 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:45.314+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:45.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:45 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:45.479+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:45.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:45 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:45.479+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:45.803 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:45 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:45.504+0000 7f5a6cc9e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:45.803 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:45 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:45.504+0000 7f5a6cc9e700 -1 osd.5 85 *** Got signal Terminated *** 2024-05-10T05:46:45.805 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:45 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:45.504+0000 7f5a6cc9e700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:46:46.144 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:45 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:45.665+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:46.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:45 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:45.796+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:46.145 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:45 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:45.796+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:46.354 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:46 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:46.058+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:46.354 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:46 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:46.058+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:46.495 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:46 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:46.180+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:46.495 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:46 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:46.180+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:18.443235+0000 front 2024-05-10T05:46:18.443310+0000 (oldest deadline 2024-05-10T05:46:43.743435+0000) 2024-05-10T05:46:46.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:46 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:46.354+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:46.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:46 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:46.354+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:46.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:46 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:46.494+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:46.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:46 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:46.494+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:47.144 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:46 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:46.677+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:47.145 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:46 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:46.827+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:47.145 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:46 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:46.827+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:47.460 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:47 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:47.338+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:47.460 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:47 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:47.338+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:47.460 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:47 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:47.088+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:47.460 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:47 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:47.088+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:47.522 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:47 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:47.212+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:47.522 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:47 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:47.212+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:18.443235+0000 front 2024-05-10T05:46:18.443310+0000 (oldest deadline 2024-05-10T05:46:43.743435+0000) 2024-05-10T05:46:47.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:47 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:47.545+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:47.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:47 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:47.522+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:47.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:47 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:47.522+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:48.144 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:47 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:47.687+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:48.145 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:47 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:47.861+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:48.145 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:47 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:47.861+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:48.362 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:48 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:48.097+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:48.362 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:48 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:48.097+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:48.538 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:48 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:48.182+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:48.538 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:48 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:48.182+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:18.443235+0000 front 2024-05-10T05:46:18.443310+0000 (oldest deadline 2024-05-10T05:46:43.743435+0000) 2024-05-10T05:46:48.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:48 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:48.362+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:48.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:48 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:48.362+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:48.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:48 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:48.544+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:48.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:48 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:48.544+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:48.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:48 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:48.537+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:49.144 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:48 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:48.648+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:49.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:48 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:48.877+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:49.145 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:48 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:48.877+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:49.408 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:49 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:49.078+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:49.408 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:49 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:49.078+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:49.528 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:49 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:49.166+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:49.528 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:49 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:49.166+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:18.443235+0000 front 2024-05-10T05:46:18.443310+0000 (oldest deadline 2024-05-10T05:46:43.743435+0000) 2024-05-10T05:46:49.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:49 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:49.408+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:49.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:49 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:49.408+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:49.802 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:49 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:49.527+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:49.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:49 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:49.529+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:49.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:49 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:49.529+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:50.144 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:49 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:49.653+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:50.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:49 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:49.882+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:50.145 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:49 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:49.882+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:50.145 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:49 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:49.882+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:46:50.426 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:50 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:50.056+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:50.426 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:50 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:50.056+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:50.481 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:50 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:50.164+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.941342+0000 front 2024-05-10T05:46:10.941432+0000 (oldest deadline 2024-05-10T05:46:34.342035+0000) 2024-05-10T05:46:50.481 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:50 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5[92136]: 2024-05-10T05:46:50.164+0000 7f5a6524e700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:18.443235+0000 front 2024-05-10T05:46:18.443310+0000 (oldest deadline 2024-05-10T05:46:43.743435+0000) 2024-05-10T05:46:50.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:50 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:50.426+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:50.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:50 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:50.426+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:50.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:50 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:50.577+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:50.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:50 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:50.577+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:46:50.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:50 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:50.480+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:50.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:50 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:50.480+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:50.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:50 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:50.480+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:46:50.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:50 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:50.620+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:51.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:50 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:50.894+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:51.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:50 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:50.894+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:51.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:50 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:50.894+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:46:51.448 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:51 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:51.030+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:51.448 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:51 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:51.030+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:51.525 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 10 05:46:51 smithi157 bash[120523]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-5 2024-05-10T05:46:51.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:51 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:51.448+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:51.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:51 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:51.448+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:51.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:51 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:51.448+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:46:51.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:51 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:51.600+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:51.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:51 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:51.600+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:46:51.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:51 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:51.524+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:51.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:51 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:51.524+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:51.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:51 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:51.524+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:46:51.865 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:51 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:51.584+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:52.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:51 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:51.864+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:52.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:51 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:51.864+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:52.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:51 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:51.864+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:46:52.317 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.5.service' 2024-05-10T05:46:52.358 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:46:52.358 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-10T05:46:52.358 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-10T05:46:52.359 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.6 2024-05-10T05:46:52.415 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:52 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:52.032+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:52.416 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:52 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:52.032+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:52.416 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:52 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:52.032+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.411906+0000 front 2024-05-10T05:46:26.411675+0000 (oldest deadline 2024-05-10T05:46:51.612444+0000) 2024-05-10T05:46:52.565 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:52 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:52.546+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:52.565 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:52 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:52.546+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:52.565 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:52 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:52.546+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:46:52.710 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:52 smithi131 systemd[1]: Stopping Ceph osd.6 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:46:52.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:52 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:52.415+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:52.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:52 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:52.415+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:52.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:52 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:52.415+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:46:52.885 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:52 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:52.550+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:53.004 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:52 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:52.727+0000 7f64da200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:46:53.005 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:52 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:52.727+0000 7f64da200700 -1 osd.6 85 *** Got signal Terminated *** 2024-05-10T05:46:53.005 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:52 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:52.727+0000 7f64da200700 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:46:53.005 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:53.004+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:53.052 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:52 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:52.565+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:53.053 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:52 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:52.565+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:46:53.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:52 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:52.884+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:53.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:52 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:52.884+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:53.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:52 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:52.884+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:46:53.393 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:53.004+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:53.394 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:53.004+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.411906+0000 front 2024-05-10T05:46:26.411675+0000 (oldest deadline 2024-05-10T05:46:51.612444+0000) 2024-05-10T05:46:53.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:53.393+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:53.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:53.393+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:53.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:53.393+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:46:53.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:53 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:53.589+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:53.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:53 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:53.589+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:46:53.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:53 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:53.516+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:53.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:53 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:53.516+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:53.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:53 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:53.516+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:46:53.852 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:53 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:53.520+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:54.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:53 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:53.852+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:54.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:53 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:53.852+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:54.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:53 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:53.852+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:46:54.367 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:53.973+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:54.367 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:53.973+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:54.367 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:53 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:53.973+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.411906+0000 front 2024-05-10T05:46:26.411675+0000 (oldest deadline 2024-05-10T05:46:51.612444+0000) 2024-05-10T05:46:54.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:54 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:54.367+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:54.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:54 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:54.367+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:54.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:54 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:54.367+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:46:54.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:54 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:54.621+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:54.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:54 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:54.621+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:46:54.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:54 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:54.526+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:54.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:54 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:54.526+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:54.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:54 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:54.526+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:46:54.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:54 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:54.477+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:54.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:54 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:54.477+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:46:55.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:54 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:54.946+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:55.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:54 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:54.946+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:55.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:54 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:54.946+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.411906+0000 front 2024-05-10T05:46:26.411675+0000 (oldest deadline 2024-05-10T05:46:51.612444+0000) 2024-05-10T05:46:55.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:54 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:54.899+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:55.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:54 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:54.899+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:55.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:54 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:54.899+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:46:55.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:55 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:55.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:55.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:55 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:55.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:55.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:55 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:55.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:46:55.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:55.432+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:55.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:55.432+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:46:56.053 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:55 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:55.573+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:56.053 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:55 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:55.573+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:46:56.053 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:55 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:55.557+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:56.053 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:55 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:55.557+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:56.053 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:55 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:55.558+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:46:56.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:55 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:55.926+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:56.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:55 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:55.926+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:56.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:55 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:55.926+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.411906+0000 front 2024-05-10T05:46:26.411675+0000 (oldest deadline 2024-05-10T05:46:51.612444+0000) 2024-05-10T05:46:56.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:55.935+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:56.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:55.935+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:56.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:55 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:55.935+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:46:56.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:56 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:56.420+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:56.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:56 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:56.420+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:56.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:56 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:56.420+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:46:56.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:56 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:56.590+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:56.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:56 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:56.590+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:46:56.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:56 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:56.508+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:56.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:56 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:56.508+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:56.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:56 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:56.508+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:46:56.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:56 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:56.405+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:56.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:56 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:56.405+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:46:57.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:56 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:56.915+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:10.306822+0000 front 2024-05-10T05:46:10.306855+0000 (oldest deadline 2024-05-10T05:46:34.406977+0000) 2024-05-10T05:46:57.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:56 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:56.915+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.309303+0000 front 2024-05-10T05:46:21.309373+0000 (oldest deadline 2024-05-10T05:46:43.510284+0000) 2024-05-10T05:46:57.210 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:56 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6[72201]: 2024-05-10T05:46:56.915+0000 7f64d0a00700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.411906+0000 front 2024-05-10T05:46:26.411675+0000 (oldest deadline 2024-05-10T05:46:51.612444+0000) 2024-05-10T05:46:57.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:56 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:56.967+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:57.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:56 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:56.967+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:57.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:56 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:56.967+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:46:57.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:57.416+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:57.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:57.416+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:57.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:57 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:57.416+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:46:57.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:57.572+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:57.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:57.572+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:46:57.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:57.508+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:57.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:57.508+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:57.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:57.508+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:46:57.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:57 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:57.508+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:46:57.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:57.435+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:57.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:57.435+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:46:57.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:57.435+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.122824+0000 front 2024-05-10T05:46:34.122919+0000 (oldest deadline 2024-05-10T05:46:56.422853+0000) 2024-05-10T05:46:58.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:57.996+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:58.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:57.996+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:58.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:57.996+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:46:58.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:57.996+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:46:58.710 INFO:journalctl@ceph.osd.6.smithi131.stdout:May 10 05:46:58 smithi131 bash[118676]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-6 2024-05-10T05:46:58.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:58 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:58.385+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:58.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:58 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:58.385+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:58.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:58 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:58.385+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:46:58.802 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:58 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:58.545+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:58.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:58 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:58.545+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:46:58.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:58 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:58.472+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:58.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:58 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:58.472+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:58.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:58 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:58.472+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:46:58.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:58 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:58.472+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:46:58.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:58 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:58.444+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:58.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:58 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:58.444+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:46:58.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:58 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:58.444+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.122824+0000 front 2024-05-10T05:46:34.122919+0000 (oldest deadline 2024-05-10T05:46:56.422853+0000) 2024-05-10T05:46:59.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:58 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:58.946+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:46:59.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:58 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:58.946+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:46:59.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:58 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:58.946+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:46:59.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:58 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:58.946+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:46:59.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:59 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:59.416+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:46:59.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:59 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:59.416+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:46:59.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:46:59 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:46:59.416+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:46:59.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:59 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:59.527+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:46:59.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:46:59 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:46:59.527+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:46:59.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:59 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:59.432+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:46:59.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:59 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:59.432+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:46:59.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:59 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:59.432+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:46:59.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:46:59 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:46:59.432+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:46:59.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:59 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:59.482+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:46:59.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:59 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:59.482+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:46:59.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:46:59 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:46:59.482+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.122824+0000 front 2024-05-10T05:46:34.122919+0000 (oldest deadline 2024-05-10T05:46:56.422853+0000) 2024-05-10T05:47:00.108 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.6.service' 2024-05-10T05:47:00.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:00.195 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-10T05:47:00.195 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-10T05:47:00.195 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.7 2024-05-10T05:47:00.227 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:59 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:59.921+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:00.227 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:59 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:59.921+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:00.227 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:59 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:59.921+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:00.227 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:46:59 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:46:59.921+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:00.517 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:00 smithi142 systemd[1]: Stopping Ceph osd.7 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:00.517 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:00.431+0000 7f8a48ae2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:00.517 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:00.431+0000 7f8a48ae2700 -1 osd.7 85 *** Got signal Terminated *** 2024-05-10T05:47:00.517 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:00.431+0000 7f8a48ae2700 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:47:00.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:00 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:00.410+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:00.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:00 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:00.410+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:00.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:00 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:00.410+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:00.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:00 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:00.410+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:00.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:00 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:00.491+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:00.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:00 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:00.491+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:00.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:00 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:00.425+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:00.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:00 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:00.425+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:00.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:00 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:00.425+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:00.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:00 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:00.425+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:00.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:00.516+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:47:00.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:00.516+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:47:00.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:00.516+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.122824+0000 front 2024-05-10T05:46:34.122919+0000 (oldest deadline 2024-05-10T05:46:56.422853+0000) 2024-05-10T05:47:01.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:00.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:01.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:00.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:01.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:00.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:01.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:00 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:00.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:01.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:01 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:01.414+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:01.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:01 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:01.414+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:01.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:01 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:01.414+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:01.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:01 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:01.414+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:01.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:01 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:01.534+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:01.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:01 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:01.534+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:01.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:01 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:01.534+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:01.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:01 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:01.401+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:01.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:01 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:01.401+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:01.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:01 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:01.401+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:01.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:01 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:01.401+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:01.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:01.502+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:47:01.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:01.502+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:47:01.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:01.502+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.122824+0000 front 2024-05-10T05:46:34.122919+0000 (oldest deadline 2024-05-10T05:46:56.422853+0000) 2024-05-10T05:47:02.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:01.914+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:02.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:01.914+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:02.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:01.914+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:02.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:01.914+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:02.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:02 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:02.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:02.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:02 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:02.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:02.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:02 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:02.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:02.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:02 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:02.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:02.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:02 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:02.506+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:02.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:02 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:02.506+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:02.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:02 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:02.506+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:02.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:02 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:02.436+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:02.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:02 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:02.436+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:02.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:02 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:02.436+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:02.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:02 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:02.436+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:02.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:02 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:02.520+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:47:02.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:02 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:02.520+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:47:02.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:02 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:02.520+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.122824+0000 front 2024-05-10T05:46:34.122919+0000 (oldest deadline 2024-05-10T05:46:56.422853+0000) 2024-05-10T05:47:03.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:02 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:02.946+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:03.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:02 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:02.946+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:03.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:02 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:02.946+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:03.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:02 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:02.947+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:03.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:03 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:03.394+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:03.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:03 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:03.394+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:03.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:03 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:03.394+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:03.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:03 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:03.394+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:03.802 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:03 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:03.473+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:03.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:03 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:03.473+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:03.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:03 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:03.473+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:03.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:03 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:03.425+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:03.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:03 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:03.425+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:03.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:03 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:03.425+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:03.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:03 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:03.425+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:03.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:03 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:03.504+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:47:03.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:03 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:03.504+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:47:03.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:03 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:03.504+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.122824+0000 front 2024-05-10T05:46:34.122919+0000 (oldest deadline 2024-05-10T05:46:56.422853+0000) 2024-05-10T05:47:04.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:03 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:03.968+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:04.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:03 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:03.968+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:04.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:03 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:03.968+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:04.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:03 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:03.968+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:04.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:04 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:04.388+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:04.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:04 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:04.388+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:04.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:04 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:04.388+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:04.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:04 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:04.388+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:04.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:04 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:04.388+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:04.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:04 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:04.467+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:04.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:04 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:04.467+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:04.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:04 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:04.467+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:04.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:04 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:04.389+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:04.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:04 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:04.389+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:04.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:04 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:04.389+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:04.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:04 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:04.389+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:04.894 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:04 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:04.530+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:19.520479+0000 front 2024-05-10T05:46:19.520503+0000 (oldest deadline 2024-05-10T05:46:43.620688+0000) 2024-05-10T05:47:04.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:04 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:04.530+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.321703+0000 front 2024-05-10T05:46:25.321657+0000 (oldest deadline 2024-05-10T05:46:54.122347+0000) 2024-05-10T05:47:04.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:04 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:04.530+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.122824+0000 front 2024-05-10T05:46:34.122919+0000 (oldest deadline 2024-05-10T05:46:56.422853+0000) 2024-05-10T05:47:04.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:04 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7[100107]: 2024-05-10T05:47:04.530+0000 7f8a41092700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.523667+0000 front 2024-05-10T05:46:40.523714+0000 (oldest deadline 2024-05-10T05:47:04.023815+0000) 2024-05-10T05:47:05.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:04 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:04.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:05.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:04 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:04.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:05.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:04 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:04.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:05.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:04 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:04.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:05.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:04 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:04.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:05.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:05 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:05.373+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:05.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:05 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:05.373+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:05.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:05 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:05.373+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:05.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:05 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:05.373+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:05.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:05 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:05.373+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:05.805 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:05 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:05.460+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:05.805 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:05 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:05.460+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:05.805 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:05 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:05.460+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:05.805 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:05 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:05.460+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.541191+0000 front 2024-05-10T05:46:42.541239+0000 (oldest deadline 2024-05-10T05:47:05.441272+0000) 2024-05-10T05:47:05.805 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:05 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:05.429+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:05.805 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:05 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:05.429+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:05.805 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:05 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:05.429+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:05.805 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:05 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:05.429+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:05.805 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:05 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:05.429+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:06.394 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 10 05:47:05 smithi142 bash[120887]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-7 2024-05-10T05:47:06.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:05 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:05.926+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:06.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:05 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:05.926+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:06.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:05 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:05.926+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:06.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:05 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:05.926+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:06.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:05 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:05.926+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:06.714 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:06 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:06.342+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:06.714 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:06 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:06.342+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:06.714 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:06 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:06.342+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:06.714 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:06 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:06.342+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:06.714 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:06 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:06.342+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:06.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:06 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:06.423+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:06.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:06 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:06.423+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:06.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:06 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:06.423+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:06.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:06 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:06.423+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.541191+0000 front 2024-05-10T05:46:42.541239+0000 (oldest deadline 2024-05-10T05:47:05.441272+0000) 2024-05-10T05:47:06.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:06 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:06.386+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:06.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:06 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:06.386+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:06.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:06 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:06.386+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:06.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:06 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:06.386+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:06.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:06 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:06.386+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:07.136 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:06 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:06.935+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:07.137 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:06 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:06.935+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:07.137 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:06 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:06.935+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:07.137 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:06 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:06.935+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:07.137 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:06 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:06.935+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:07.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.7.service' 2024-05-10T05:47:07.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:07.226 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-10T05:47:07.226 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-10T05:47:07.226 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.8 2024-05-10T05:47:07.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:07.357+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:07.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:07.357+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:07.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:07.357+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:07.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:07.357+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:07.553 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:07.357+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:07.554 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:07 smithi157 systemd[1]: Stopping Ceph osd.8 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:07.554 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:07.459+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:07.554 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:07.459+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:07.554 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:07.459+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:07.554 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:07.459+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.541191+0000 front 2024-05-10T05:46:42.541239+0000 (oldest deadline 2024-05-10T05:47:05.441272+0000) 2024-05-10T05:47:07.554 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:07.475+0000 7fe3092d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:07.554 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:07.475+0000 7fe3092d1700 -1 osd.8 85 *** Got signal Terminated *** 2024-05-10T05:47:07.554 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:07 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:07.475+0000 7fe3092d1700 -1 osd.8 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:47:07.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:07 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:07.330+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:07.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:07 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:07.330+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:07.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:07 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:07.330+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:07.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:07 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:07.330+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:07.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:07 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:07.330+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:08.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:07 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:07.972+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:08.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:07 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:07.972+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:08.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:07 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:07.972+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:08.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:07 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:07.972+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:08.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:07 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:07.972+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:08.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:08.371+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:08.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:08.371+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:08.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:08.371+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:08.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:08.371+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:08.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:08 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:08.371+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:08.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:08 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:08.415+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:08.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:08 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:08.415+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:08.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:08 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:08.415+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:08.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:08 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:08.415+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.541191+0000 front 2024-05-10T05:46:42.541239+0000 (oldest deadline 2024-05-10T05:47:05.441272+0000) 2024-05-10T05:47:08.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:08 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:08.361+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:08.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:08 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:08.361+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:08.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:08 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:08.361+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:08.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:08 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:08.361+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:08.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:08 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:08.361+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:09.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:08 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:08.967+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:09.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:08 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:08.967+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:09.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:08 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:08.967+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:09.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:08 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:08.967+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:09.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:08 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:08.967+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:09.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:09 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:09.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:09.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:09 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:09.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:09.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:09 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:09.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:09.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:09 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:09.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:09.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:09 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:09.413+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:09.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:09.449+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:09.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:09.449+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:09.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:09.449+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:09.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:09.449+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.541191+0000 front 2024-05-10T05:46:42.541239+0000 (oldest deadline 2024-05-10T05:47:05.441272+0000) 2024-05-10T05:47:09.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:09.403+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:09.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:09.403+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:09.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:09.403+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:09.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:09.403+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:09.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:09.403+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:10.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:09 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:09.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:10.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:09 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:09.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:10.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:09 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:09.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:10.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:09 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:09.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:10.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:09 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:09.957+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:10.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:10 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:10.444+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:10.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:10 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:10.444+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:10.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:10 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:10.444+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:10.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:10 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:10.444+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:10.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:10 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:10.444+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:10.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:10 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:10.425+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:10.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:10 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:10.425+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:10.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:10 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:10.425+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:10.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:10 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:10.425+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.541191+0000 front 2024-05-10T05:46:42.541239+0000 (oldest deadline 2024-05-10T05:47:05.441272+0000) 2024-05-10T05:47:10.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:10 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:10.430+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:10.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:10 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:10.430+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:10.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:10 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:10.430+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:10.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:10 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:10.430+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:10.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:10 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:10.430+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:11.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:10 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:10.989+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:11.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:10 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:10.989+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:11.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:10 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:10.989+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:11.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:10 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:10.989+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:11.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:10 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:10.989+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:11.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:11.428+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:11.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:11.428+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:11.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:11.428+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:11.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:11.428+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.541191+0000 front 2024-05-10T05:46:42.541239+0000 (oldest deadline 2024-05-10T05:47:05.441272+0000) 2024-05-10T05:47:11.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:11.467+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:11.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:11.467+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:11.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:11.467+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:11.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:11.467+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:11.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:11 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:11.467+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:11.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:11 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:11.481+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:11.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:11 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:11.481+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:11.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:11 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:11.481+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:11.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:11 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:11.481+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:11.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:11 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:11.481+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:11.961 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:11 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:11.481+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:48.968251+0000 front 2024-05-10T05:46:48.968202+0000 (oldest deadline 2024-05-10T05:47:11.168634+0000) 2024-05-10T05:47:12.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:11 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:11.978+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:12.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:11 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:11.978+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:12.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:11 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:11.978+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:12.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:11 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:11.978+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:12.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:11 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:11.978+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:12.803 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:12.415+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:21.037452+0000 front 2024-05-10T05:46:21.037414+0000 (oldest deadline 2024-05-10T05:46:46.937499+0000) 2024-05-10T05:47:12.804 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:12.415+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.937923+0000 front 2024-05-10T05:46:26.938026+0000 (oldest deadline 2024-05-10T05:46:50.438063+0000) 2024-05-10T05:47:12.804 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:12.415+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.239722+0000 front 2024-05-10T05:46:33.239761+0000 (oldest deadline 2024-05-10T05:47:01.440156+0000) 2024-05-10T05:47:12.804 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8[100790]: 2024-05-10T05:47:12.415+0000 7fe305889700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.541191+0000 front 2024-05-10T05:46:42.541239+0000 (oldest deadline 2024-05-10T05:47:05.441272+0000) 2024-05-10T05:47:12.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:12.432+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:12.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:12.432+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:12.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:12.432+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:12.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:12.432+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:12.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:12.432+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:12.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:12 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:12.432+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:12.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:12 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:12.491+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:12.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:12 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:12.491+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:12.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:12 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:12.491+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:12.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:12 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:12.491+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:12.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:12 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:12.491+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:12.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:12 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:12.491+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:48.968251+0000 front 2024-05-10T05:46:48.968202+0000 (oldest deadline 2024-05-10T05:47:11.168634+0000) 2024-05-10T05:47:13.302 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 10 05:47:12 smithi157 bash[120778]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-8 2024-05-10T05:47:13.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:12 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:12.986+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:13.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:12 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:12.986+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:13.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:12 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:12.986+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:13.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:12 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:12.986+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:13.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:12 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:12.986+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:13.783 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:13.449+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:13.783 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:13.450+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:13.783 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:13.450+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:13.783 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:13.450+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:13.783 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:13.450+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:13.783 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:13.450+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:13.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:13 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:13.529+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:13.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:13 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:13.529+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:13.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:13 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:13.529+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:13.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:13 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:13.529+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:13.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:13 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:13.529+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:13.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:13 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:13.529+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:48.968251+0000 front 2024-05-10T05:46:48.968202+0000 (oldest deadline 2024-05-10T05:47:11.168634+0000) 2024-05-10T05:47:14.121 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.8.service' 2024-05-10T05:47:14.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:14.163 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-10T05:47:14.163 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-10T05:47:14.163 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.9 2024-05-10T05:47:14.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:13 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:13.966+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:14.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:13 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:13.966+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:14.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:13 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:13.966+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:14.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:13 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:13.966+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:14.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:13 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:13.966+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:14.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:13 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:13.966+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:14.496 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:14 smithi131 systemd[1]: Stopping Ceph osd.9 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:14.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:14 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:14.463+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:14.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:14 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:14.463+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:14.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:14 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:14.463+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:14.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:14 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:14.463+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:14.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:14 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:14.463+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:14.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:14 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:14.463+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:14.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:14.495+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:14.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:14.495+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:14.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:14.495+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:14.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:14.495+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:14.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:14.495+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:14.961 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:14.495+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:48.968251+0000 front 2024-05-10T05:46:48.968202+0000 (oldest deadline 2024-05-10T05:47:11.168634+0000) 2024-05-10T05:47:14.961 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:14.547+0000 7fe9b5e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:14.961 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:14.547+0000 7fe9b5e00700 -1 osd.9 85 *** Got signal Terminated *** 2024-05-10T05:47:14.961 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:14 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:14.547+0000 7fe9b5e00700 -1 osd.9 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:47:15.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:14 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:14.998+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:15.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:14 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:14.998+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:15.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:14 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:14.998+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:15.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:14 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:14.998+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:15.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:14 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:14.998+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:15.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:14 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:14.998+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:15.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:15 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:15.456+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:15.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:15 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:15.456+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:15.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:15 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:15.456+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:15.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:15 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:15.456+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:15.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:15 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:15.456+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:15.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:15 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:15.456+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:48.968251+0000 front 2024-05-10T05:46:48.968202+0000 (oldest deadline 2024-05-10T05:47:11.168634+0000) 2024-05-10T05:47:15.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:15 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:15.498+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:15.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:15 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:15.498+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:15.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:15 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:15.498+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:15.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:15 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:15.498+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:15.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:15 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:15.498+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:15.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:15 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:15.498+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:16.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:16.003+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:16.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:16.003+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:16.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:16.003+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:16.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:16.003+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:16.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:16.003+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:16.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:16 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:16.003+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:16.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:16 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:16.438+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:16.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:16 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:16.438+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:16.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:16 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:16.438+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:16.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:16 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:16.438+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:16.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:16 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:16.438+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:16.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:16 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:16.438+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:48.968251+0000 front 2024-05-10T05:46:48.968202+0000 (oldest deadline 2024-05-10T05:47:11.168634+0000) 2024-05-10T05:47:16.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:16 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:16.497+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:16.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:16 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:16.497+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:16.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:16 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:16.497+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:16.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:16 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:16.497+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:16.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:16 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:16.497+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:16.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:16 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:16.497+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:17.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:17 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:17.052+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:17.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:17 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:17.052+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:17.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:17 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:17.052+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:17.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:17 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:17.052+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:17.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:17 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:17.052+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:17.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:17 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:17.052+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:17.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:17 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:17.409+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:17.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:17 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:17.409+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:17.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:17 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:17.409+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:17.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:17 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:17.409+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:17.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:17 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:17.409+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:17.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:17 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:17.409+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:48.968251+0000 front 2024-05-10T05:46:48.968202+0000 (oldest deadline 2024-05-10T05:47:11.168634+0000) 2024-05-10T05:47:17.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:17 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:17.540+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:17.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:17 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:17.540+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:17.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:17 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:17.540+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:17.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:17 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:17.540+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:17.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:17 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:17.540+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:17.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:17 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:17.540+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:18.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.005+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:18.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.005+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:18.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.005+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:18.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.005+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:18.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.005+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:18.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.005+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:18.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:18 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:18.419+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:18.710 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:18 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:18.419+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:18.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:18 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:18.419+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:18.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:18 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:18.419+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:18.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:18 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:18.419+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:18.711 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:18 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:18.419+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:48.968251+0000 front 2024-05-10T05:46:48.968202+0000 (oldest deadline 2024-05-10T05:47:11.168634+0000) 2024-05-10T05:47:18.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:18 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:18.518+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:18.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:18 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:18.518+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:18.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:18 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:18.518+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:18.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:18 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:18.518+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:18.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:18 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:18.518+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:18.805 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:18 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:18.518+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:19.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.983+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:19.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.983+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:19.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.983+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:19.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.983+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:19.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.983+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:19.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:18 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:18.983+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:19.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:19 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:19.494+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:19.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:19 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:19.494+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:19.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:19 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:19.494+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:19.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:19 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:19.494+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:19.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:19 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:19.494+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:19.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:19 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:19.494+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:19.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:19 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:19.466+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:11.962635+0000 front 2024-05-10T05:46:11.962535+0000 (oldest deadline 2024-05-10T05:46:36.662780+0000) 2024-05-10T05:47:19.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:19 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:19.466+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.764365+0000 front 2024-05-10T05:46:20.764346+0000 (oldest deadline 2024-05-10T05:46:45.464225+0000) 2024-05-10T05:47:19.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:19 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:19.466+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:25.465056+0000 front 2024-05-10T05:46:25.464870+0000 (oldest deadline 2024-05-10T05:46:50.765012+0000) 2024-05-10T05:47:19.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:19 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:19.466+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.266322+0000 front 2024-05-10T05:46:34.266367+0000 (oldest deadline 2024-05-10T05:47:00.166614+0000) 2024-05-10T05:47:19.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:19 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:19.466+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:40.167300+0000 front 2024-05-10T05:46:40.167431+0000 (oldest deadline 2024-05-10T05:47:04.267072+0000) 2024-05-10T05:47:19.960 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:19 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9[80914]: 2024-05-10T05:47:19.466+0000 7fe9ac600700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:48.968251+0000 front 2024-05-10T05:46:48.968202+0000 (oldest deadline 2024-05-10T05:47:11.168634+0000) 2024-05-10T05:47:20.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.021+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:20.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.021+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:20.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.021+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:20.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.021+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:20.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.021+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:20.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.021+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:20.460 INFO:journalctl@ceph.osd.9.smithi131.stdout:May 10 05:47:20 smithi131 bash[118934]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-9 2024-05-10T05:47:20.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:20 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:20.510+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:20.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:20 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:20.510+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:20.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:20 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:20.510+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:20.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:20 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:20.510+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:20.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:20 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:20.510+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:20.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:20 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:20.510+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:21.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.992+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:21.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.992+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:21.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.992+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:21.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.992+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:21.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.992+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:21.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:20 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:20.992+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:21.777 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.9.service' 2024-05-10T05:47:21.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:21 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:21.480+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:21.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:21 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:21.480+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:21.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:21 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:21.480+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:21.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:21 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:21.480+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:21.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:21 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:21.480+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:21.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:21 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:21.480+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:21.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:21.867 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-10T05:47:21.867 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-10T05:47:21.867 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.10 2024-05-10T05:47:22.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:21 smithi142 systemd[1]: Stopping Ceph osd.10 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:22.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:21 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:21.995+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:22.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:21 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:21.995+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:22.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:21 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:21.995+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:22.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:21 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:21.995+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:22.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:21 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:21.995+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:22.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:21 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:21.995+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:22.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:22 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:22.103+0000 7f38de2f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:22.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:22 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:22.103+0000 7f38de2f7700 -1 osd.10 85 *** Got signal Terminated *** 2024-05-10T05:47:22.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:22 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:22.103+0000 7f38de2f7700 -1 osd.10 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:47:22.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:22 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:22.524+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:22.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:22 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:22.524+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:22.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:22 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:22.524+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:22.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:22 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:22.524+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:22.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:22 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:22.524+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:22.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:22 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:22.524+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:23.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:22 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:22.973+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:23.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:22 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:22.973+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:23.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:22 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:22.973+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:23.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:22 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:22.973+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:23.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:22 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:22.973+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:23.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:22 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:22.973+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:23.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:22 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:22.973+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-10T05:46:57.458559+0000 front 2024-05-10T05:46:57.458525+0000 (oldest deadline 2024-05-10T05:47:22.158480+0000) 2024-05-10T05:47:23.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:23 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:23.484+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:23.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:23 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:23.484+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:23.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:23 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:23.484+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:23.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:23 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:23.484+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:23.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:23 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:23.484+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:23.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:23 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:23.484+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:24.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:23 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:23.959+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:24.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:23 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:23.959+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:24.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:23 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:23.959+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:24.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:23 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:23.959+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:24.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:23 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:23.959+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:24.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:23 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:23.959+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:24.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:23 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:23.959+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-10T05:46:57.458559+0000 front 2024-05-10T05:46:57.458525+0000 (oldest deadline 2024-05-10T05:47:22.158480+0000) 2024-05-10T05:47:24.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:24 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:24.441+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:24.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:24 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:24.441+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:24.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:24 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:24.441+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:24.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:24 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:24.441+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:24.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:24 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:24.441+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:24.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:24 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:24.441+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:25.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:24 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:24.976+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:25.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:24 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:24.976+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:25.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:24 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:24.976+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:25.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:24 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:24.976+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:25.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:24 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:24.976+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:25.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:24 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:24.976+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:25.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:24 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:24.976+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-10T05:46:57.458559+0000 front 2024-05-10T05:46:57.458525+0000 (oldest deadline 2024-05-10T05:47:22.158480+0000) 2024-05-10T05:47:25.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:25 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:25.488+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:25.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:25 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:25.488+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:25.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:25 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:25.488+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:25.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:25 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:25.488+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:25.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:25 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:25.488+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:25.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:25 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:25.488+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:26.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:25 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:25.954+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:26.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:25 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:25.954+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:26.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:25 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:25.954+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:26.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:25 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:25.954+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:26.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:25 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:25.954+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:26.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:25 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:25.954+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:26.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:25 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:25.954+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-10T05:46:57.458559+0000 front 2024-05-10T05:46:57.458525+0000 (oldest deadline 2024-05-10T05:47:22.158480+0000) 2024-05-10T05:47:26.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:26 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:26.454+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:26.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:26 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:26.454+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:26.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:26 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:26.454+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:26.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:26 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:26.454+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:26.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:26 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:26.454+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:26.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:26 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:26.454+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:27.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:26 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:26.917+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:12.050780+0000 front 2024-05-10T05:46:12.050743+0000 (oldest deadline 2024-05-10T05:46:34.350738+0000) 2024-05-10T05:47:27.394 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:26 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:26.917+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.251915+0000 front 2024-05-10T05:46:20.252055+0000 (oldest deadline 2024-05-10T05:46:44.352133+0000) 2024-05-10T05:47:27.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:26 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:26.917+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:28.453383+0000 front 2024-05-10T05:46:28.453546+0000 (oldest deadline 2024-05-10T05:46:49.453635+0000) 2024-05-10T05:47:27.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:26 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:26.917+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:34.154859+0000 front 2024-05-10T05:46:34.154797+0000 (oldest deadline 2024-05-10T05:46:57.054798+0000) 2024-05-10T05:47:27.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:26 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:26.917+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:39.355800+0000 front 2024-05-10T05:46:39.355776+0000 (oldest deadline 2024-05-10T05:47:04.655915+0000) 2024-05-10T05:47:27.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:26 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:26.917+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:49.357336+0000 front 2024-05-10T05:46:49.357255+0000 (oldest deadline 2024-05-10T05:47:13.457037+0000) 2024-05-10T05:47:27.395 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:26 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10[108095]: 2024-05-10T05:47:26.917+0000 7f38da8af700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6822 osd.6 since back 2024-05-10T05:46:57.458559+0000 front 2024-05-10T05:46:57.458525+0000 (oldest deadline 2024-05-10T05:47:22.158480+0000) 2024-05-10T05:47:27.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:27 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:27.495+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:27.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:27 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:27.495+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:27.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:27 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:27.495+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:27.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:27 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:27.495+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:27.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:27 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:27.495+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:27.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:27 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:27.495+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:28.144 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 10 05:47:27 smithi142 bash[121142]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-10 2024-05-10T05:47:28.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:28 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:28.458+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:28.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:28 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:28.458+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:28.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:28 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:28.458+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:28.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:28 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:28.458+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:28.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:28 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:28.458+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:28.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:28 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:28.458+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:28.934 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.10.service' 2024-05-10T05:47:28.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:28.976 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-10T05:47:28.976 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-10T05:47:28.976 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.11 2024-05-10T05:47:29.303 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:29 smithi157 systemd[1]: Stopping Ceph osd.11 for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:29.303 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:29 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:29.213+0000 7fa9ad0b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:29.303 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:29 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:29.213+0000 7fa9ad0b8700 -1 osd.11 85 *** Got signal Terminated *** 2024-05-10T05:47:29.303 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:29 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:29.213+0000 7fa9ad0b8700 -1 osd.11 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T05:47:29.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:29 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:29.418+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:29.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:29 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:29.418+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:29.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:29 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:29.418+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:29.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:29 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:29.418+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:29.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:29 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:29.418+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:29.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:29 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:29.418+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:30.802 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:30 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:30.461+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:30.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:30 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:30.461+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:30.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:30 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:30.461+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:30.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:30 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:30.461+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:30.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:30 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:30.461+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:30.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:30 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:30.461+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:31.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:31 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:31.414+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:31.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:31 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:31.414+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:31.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:31 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:31.414+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:31.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:31 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:31.414+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:31.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:31 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:31.414+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:31.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:31 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:31.414+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:31.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:31 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:31.414+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-10T05:47:05.185983+0000 front 2024-05-10T05:47:05.185853+0000 (oldest deadline 2024-05-10T05:47:30.485717+0000) 2024-05-10T05:47:32.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:32 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:32.385+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:32.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:32 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:32.385+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:32.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:32 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:32.385+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:32.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:32 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:32.385+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:32.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:32 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:32.385+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:32.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:32 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:32.385+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:32.804 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:32 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:32.385+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-10T05:47:05.185983+0000 front 2024-05-10T05:47:05.185853+0000 (oldest deadline 2024-05-10T05:47:30.485717+0000) 2024-05-10T05:47:33.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:33 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:33.384+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6806 osd.0 since back 2024-05-10T05:46:08.974867+0000 front 2024-05-10T05:46:08.975052+0000 (oldest deadline 2024-05-10T05:46:34.274985+0000) 2024-05-10T05:47:33.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:33 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:33.384+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-10T05:46:20.676803+0000 front 2024-05-10T05:46:20.676776+0000 (oldest deadline 2024-05-10T05:46:44.176909+0000) 2024-05-10T05:47:33.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:33 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:33.384+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-10T05:46:26.478013+0000 front 2024-05-10T05:46:26.477952+0000 (oldest deadline 2024-05-10T05:46:49.878564+0000) 2024-05-10T05:47:33.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:33 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:33.384+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6814 osd.3 since back 2024-05-10T05:46:33.379900+0000 front 2024-05-10T05:46:33.379825+0000 (oldest deadline 2024-05-10T05:46:57.479774+0000) 2024-05-10T05:47:33.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:33 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:33.384+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-10T05:46:42.681719+0000 front 2024-05-10T05:46:42.681769+0000 (oldest deadline 2024-05-10T05:47:04.981535+0000) 2024-05-10T05:47:33.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:33 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:33.384+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-10T05:46:50.282521+0000 front 2024-05-10T05:46:50.282571+0000 (oldest deadline 2024-05-10T05:47:11.982586+0000) 2024-05-10T05:47:33.803 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:33 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11[108768]: 2024-05-10T05:47:33.384+0000 7fa9a9670700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-10T05:47:05.185983+0000 front 2024-05-10T05:47:05.185853+0000 (oldest deadline 2024-05-10T05:47:30.485717+0000) 2024-05-10T05:47:35.053 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 10 05:47:34 smithi157 bash[121046]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-osd-11 2024-05-10T05:47:36.034 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@osd.11.service' 2024-05-10T05:47:36.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:36.074 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-10T05:47:36.075 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-10T05:47:36.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.a 2024-05-10T05:47:36.460 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:47:36 smithi131 systemd[1]: Stopping Ceph mds.a for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:36.960 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:47:36 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-a[88598]: 2024-05-10T05:47:36.477+0000 7fc383e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:36.960 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:47:36 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-a[88598]: 2024-05-10T05:47:36.477+0000 7fc383e00700 -1 mds.a *** got signal Terminated *** 2024-05-10T05:47:40.960 INFO:journalctl@ceph.mds.a.smithi131.stdout:May 10 05:47:40 smithi131 bash[119201]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-a 2024-05-10T05:47:41.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.a.service' 2024-05-10T05:47:41.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:41.189 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-10T05:47:41.189 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-10T05:47:41.189 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.d 2024-05-10T05:47:41.460 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:47:41 smithi131 systemd[1]: Stopping Ceph mds.d for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:41.960 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:47:41 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-d[87678]: 2024-05-10T05:47:41.594+0000 7fdc19000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:41.960 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:47:41 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-d[87678]: 2024-05-10T05:47:41.594+0000 7fdc19000700 -1 mds.d *** got signal Terminated *** 2024-05-10T05:47:45.710 INFO:journalctl@ceph.mds.d.smithi131.stdout:May 10 05:47:45 smithi131 bash[119339]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-d 2024-05-10T05:47:45.919 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.d.service' 2024-05-10T05:47:45.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:45.999 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-10T05:47:46.000 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-10T05:47:46.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.g 2024-05-10T05:47:46.210 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:47:46 smithi131 systemd[1]: Stopping Ceph mds.g for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:46.710 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:47:46 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-g[88209]: 2024-05-10T05:47:46.410+0000 7f08f9a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:46.710 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:47:46 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-g[88209]: 2024-05-10T05:47:46.410+0000 7f08f9a00700 -1 mds.g *** got signal Terminated *** 2024-05-10T05:47:49.710 INFO:journalctl@ceph.mds.g.smithi131.stdout:May 10 05:47:49 smithi131 bash[119478]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-g 2024-05-10T05:47:49.929 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.g.service' 2024-05-10T05:47:50.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:50.010 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-10T05:47:50.010 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-10T05:47:50.010 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.j 2024-05-10T05:47:50.210 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 10 05:47:50 smithi131 systemd[1]: Stopping Ceph mds.j for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:50.710 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 10 05:47:50 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-j[87290]: 2024-05-10T05:47:50.366+0000 7f90e6600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:50.711 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 10 05:47:50 smithi131 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-j[87290]: 2024-05-10T05:47:50.366+0000 7f90e6600700 -1 mds.j *** got signal Terminated *** 2024-05-10T05:47:51.960 INFO:journalctl@ceph.mds.j.smithi131.stdout:May 10 05:47:51 smithi131 bash[119617]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-j 2024-05-10T05:47:52.050 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.j.service' 2024-05-10T05:47:52.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:52.129 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-10T05:47:52.129 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-10T05:47:52.129 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.b 2024-05-10T05:47:52.450 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:47:52 smithi142 systemd[1]: Stopping Ceph mds.b for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:52.894 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:47:52 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-b[115203]: 2024-05-10T05:47:52.448+0000 7fe4279ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:52.894 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:47:52 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-b[115203]: 2024-05-10T05:47:52.448+0000 7fe4279ad700 -1 mds.b *** got signal Terminated *** 2024-05-10T05:47:56.894 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 10 05:47:56 smithi142 bash[121404]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-b 2024-05-10T05:47:56.999 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.b.service' 2024-05-10T05:47:57.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:47:57.037 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-10T05:47:57.037 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-10T05:47:57.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.e 2024-05-10T05:47:57.298 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:47:57 smithi142 systemd[1]: Stopping Ceph mds.e for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:47:57.644 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:47:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-e[114513]: 2024-05-10T05:47:57.297+0000 7f501a11a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:47:57.644 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:47:57 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-e[114513]: 2024-05-10T05:47:57.297+0000 7f501a11a700 -1 mds.e *** got signal Terminated *** 2024-05-10T05:48:01.573 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 10 05:48:01 smithi142 bash[121543]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-e 2024-05-10T05:48:01.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.e.service' 2024-05-10T05:48:01.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:48:01.613 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-10T05:48:01.614 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-10T05:48:01.614 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.h 2024-05-10T05:48:01.864 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:48:01 smithi142 systemd[1]: Stopping Ceph mds.h for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:48:02.144 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:48:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-h[114852]: 2024-05-10T05:48:01.863+0000 7fd73c6a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:48:02.144 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:48:01 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-h[114852]: 2024-05-10T05:48:01.863+0000 7fd73c6a3700 -1 mds.h *** got signal Terminated *** 2024-05-10T05:48:05.554 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 10 05:48:05 smithi142 bash[121683]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-h 2024-05-10T05:48:05.559 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.h.service' 2024-05-10T05:48:05.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:48:05.593 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-10T05:48:05.593 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-10T05:48:05.593 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.k 2024-05-10T05:48:05.846 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:48:05 smithi142 systemd[1]: Stopping Ceph mds.k for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:48:06.144 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:48:05 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-k[114170]: 2024-05-10T05:48:05.845+0000 7f1408273700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:48:06.144 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:48:05 smithi142 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-k[114170]: 2024-05-10T05:48:05.846+0000 7f1408273700 -1 mds.k *** got signal Terminated *** 2024-05-10T05:48:09.586 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 10 05:48:09 smithi142 bash[121822]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-k 2024-05-10T05:48:09.594 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.k.service' 2024-05-10T05:48:09.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:48:09.628 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-10T05:48:09.629 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-10T05:48:09.629 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.c 2024-05-10T05:48:10.053 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:48:09 smithi157 systemd[1]: Stopping Ceph mds.c for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:48:10.053 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:48:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-c[114939]: 2024-05-10T05:48:09.865+0000 7f862fded700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:48:10.053 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:48:09 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-c[114939]: 2024-05-10T05:48:09.865+0000 7f862fded700 -1 mds.c *** got signal Terminated *** 2024-05-10T05:48:13.670 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 10 05:48:13 smithi157 bash[121301]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-c 2024-05-10T05:48:13.676 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.c.service' 2024-05-10T05:48:13.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:48:13.714 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-10T05:48:13.715 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-10T05:48:13.715 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.f 2024-05-10T05:48:13.973 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:48:13 smithi157 systemd[1]: Stopping Ceph mds.f for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:48:14.302 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:48:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-f[114596]: 2024-05-10T05:48:13.972+0000 7fd658137700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:48:14.303 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:48:13 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-f[114596]: 2024-05-10T05:48:13.972+0000 7fd658137700 -1 mds.f *** got signal Terminated *** 2024-05-10T05:48:17.552 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 10 05:48:17 smithi157 bash[121440]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-f 2024-05-10T05:48:17.660 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.f.service' 2024-05-10T05:48:17.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:48:17.695 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-10T05:48:17.695 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-10T05:48:17.696 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.i 2024-05-10T05:48:17.955 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:48:17 smithi157 systemd[1]: Stopping Ceph mds.i for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:48:18.302 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:48:17 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-i[115279]: 2024-05-10T05:48:17.954+0000 7fe915bd2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:48:18.303 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:48:17 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-i[115279]: 2024-05-10T05:48:17.954+0000 7fe915bd2700 -1 mds.i *** got signal Terminated *** 2024-05-10T05:48:21.553 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 10 05:48:21 smithi157 bash[121580]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-i 2024-05-10T05:48:21.619 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.i.service' 2024-05-10T05:48:21.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:48:21.655 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-10T05:48:21.655 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-10T05:48:21.655 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.l 2024-05-10T05:48:21.922 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 10 05:48:21 smithi157 systemd[1]: Stopping Ceph mds.l for 6e8de474-0e8d-11ef-bc98-c7b262605968... 2024-05-10T05:48:22.303 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 10 05:48:21 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-l[114254]: 2024-05-10T05:48:21.920+0000 7f1bf1d36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T05:48:22.303 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 10 05:48:21 smithi157 ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-l[114254]: 2024-05-10T05:48:21.920+0000 7f1bf1d36700 -1 mds.l *** got signal Terminated *** 2024-05-10T05:48:23.484 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 10 05:48:23 smithi157 bash[121730]: ceph-6e8de474-0e8d-11ef-bc98-c7b262605968-mds-l 2024-05-10T05:48:23.490 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e8de474-0e8d-11ef-bc98-c7b262605968@mds.l.service' 2024-05-10T05:48:23.525 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T05:48:23.525 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-10T05:48:23.526 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T05:48:23.786 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:48:26.730 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T05:48:26.917 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:48:28.320 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T05:48:28.505 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:48:29.889 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T05:48:29.966 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T05:48:30.003 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T05:48:30.034 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-10T05:48:30.036 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605/remote/smithi131/crash 2024-05-10T05:48:30.037 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/crash -- . 2024-05-10T05:48:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T05:48:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2024-05-10T05:48:30.107 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605/remote/smithi142/crash 2024-05-10T05:48:30.107 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/crash -- . 2024-05-10T05:48:30.145 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T05:48:30.145 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-05-10T05:48:30.147 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605/remote/smithi157/crash 2024-05-10T05:48:30.148 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/crash -- . 2024-05-10T05:48:30.188 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T05:48:30.188 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-05-10T05:48:30.190 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-10T05:48:30.190 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T05:48:30.266 INFO:teuthology.orchestra.run.smithi131.stderr:grep: /var/log/ceph/6e8de474-0e8d-11ef-bc98-c7b262605968/ceph.log: No such file or directory 2024-05-10T05:48:30.272 INFO:tasks.cephadm:Compressing logs... 2024-05-10T05:48:30.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T05:48:30.276 DEBUG:teuthology.orchestra.run.smithi142:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T05:48:30.278 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T05:48:30.305 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T05:48:30.307 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T05:48:30.336 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T05:50:08.867 INFO:tasks.cephadm:Archiving logs... 2024-05-10T05:50:08.867 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605/remote/smithi131/log 2024-05-10T05:50:08.868 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T05:50:19.813 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605/remote/smithi142/log 2024-05-10T05:50:19.814 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T05:50:31.891 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605/remote/smithi157/log 2024-05-10T05:50:31.893 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T05:50:43.194 INFO:tasks.cephadm:Removing cluster... 2024-05-10T05:50:43.195 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 --force 2024-05-10T05:50:43.459 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:50:44.147 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 --force 2024-05-10T05:50:44.350 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:50:44.681 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6e8de474-0e8d-11ef-bc98-c7b262605968 --force 2024-05-10T05:50:44.869 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 6e8de474-0e8d-11ef-bc98-c7b262605968 2024-05-10T05:50:45.214 INFO:tasks.cephadm:Removing cephadm ... 2024-05-10T05:50:45.215 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T05:50:45.262 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T05:50:45.283 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T05:50:45.302 INFO:tasks.cephadm:Teardown complete 2024-05-10T05:50:45.302 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-10T05:50:45.313 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-10T05:50:45.314 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T05:50:45.318 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T05:50:45.327 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T05:50:45.460 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-10T05:50:45.471 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-10T05:50:45.471 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T05:50:45.475 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T05:50:45.478 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T05:50:45.493 INFO:teuthology.orchestra.run.smithi142.stderr:bash: ntpq: command not found 2024-05-10T05:50:45.494 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2024-05-10T05:50:45.498 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T05:50:45.498 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-05-10T05:50:45.498 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 +1735us[+1655us] +/- 51ms 2024-05-10T05:50:45.498 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -1548us[-1548us] +/- 70ms 2024-05-10T05:50:45.498 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -743us[ -824us] +/- 75ms 2024-05-10T05:50:45.498 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:50:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T05:50:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-05-10T05:50:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +1704us[+1699us] +/- 51ms 2024-05-10T05:50:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -1592us[-1597us] +/- 70ms 2024-05-10T05:50:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -786us[ -791us] +/- 75ms 2024-05-10T05:50:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:50:45.512 INFO:teuthology.orchestra.run.smithi131.stderr:bash: ntpq: command not found 2024-05-10T05:50:45.519 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T05:50:45.519 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-05-10T05:50:45.519 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 +1678us[+1728us] +/- 51ms 2024-05-10T05:50:45.520 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -1612us[-1562us] +/- 70ms 2024-05-10T05:50:45.520 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -808us[ -759us] +/- 75ms 2024-05-10T05:50:45.520 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T05:50:45.521 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-10T05:50:45.574 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-10T05:50:45.574 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-10T05:50:45.610 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T05:50:45.748 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T05:50:45.824 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T05:50:45.891 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T05:50:45.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:50:45.963 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T05:50:45.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:50:45.996 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T05:50:46.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:50:46.040 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-10T05:50:46.051 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-10T05:50:46.092 INFO:teuthology.task.internal:Duration was 2603.702702 seconds 2024-05-10T05:50:46.092 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-10T05:50:46.133 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-10T05:50:46.133 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T05:50:46.138 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T05:50:46.141 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T05:50:46.188 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T05:50:46.188 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T05:50:46.246 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T05:50:46.747 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-10T05:50:46.747 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-05-10T05:50:46.748 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T05:50:46.805 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-05-10T05:50:46.805 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T05:50:46.831 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-05-10T05:50:46.832 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T05:50:46.854 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-10T05:50:46.854 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T05:50:46.858 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T05:50:46.873 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T05:50:46.954 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-10T05:50:46.954 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T05:50:46.958 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T05:50:46.961 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T05:50:47.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-10T05:50:47.128 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-10T05:50:47.128 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T05:50:47.194 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T05:50:47.223 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T05:50:47.250 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-10T05:50:47.262 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T05:50:47.267 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T05:50:47.270 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T05:50:47.294 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-05-10T05:50:47.319 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-05-10T05:50:47.326 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-05-10T05:50:47.361 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T05:50:47.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:50:47.402 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T05:50:47.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:50:47.418 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T05:50:47.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T05:50:47.434 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-10T05:50:47.447 INFO:teuthology.task.internal:Transferring archived files... 2024-05-10T05:50:47.448 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605/remote/smithi131 2024-05-10T05:50:47.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T05:50:47.532 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605/remote/smithi142 2024-05-10T05:50:47.533 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T05:50:47.577 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700605/remote/smithi157 2024-05-10T05:50:47.577 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T05:50:47.618 INFO:teuthology.task.internal:Removing archive directory... 2024-05-10T05:50:47.618 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T05:50:47.622 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T05:50:47.625 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T05:50:47.668 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-10T05:50:47.684 INFO:teuthology.task.internal:Not uploading archives. 2024-05-10T05:50:47.684 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-10T05:50:47.718 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-10T05:50:47.718 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T05:50:47.723 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T05:50:47.726 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T05:50:47.740 INFO:teuthology.orchestra.run.smithi142.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 05:50 /home/ubuntu/cephtest 2024-05-10T05:50:47.743 INFO:teuthology.orchestra.run.smithi157.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 05:50 /home/ubuntu/cephtest 2024-05-10T05:50:47.761 INFO:teuthology.orchestra.run.smithi131.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 05:50 /home/ubuntu/cephtest 2024-05-10T05:50:47.765 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-10T05:50:47.778 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-10T05:50:47.871 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} duration: 2603.7027015686035 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-10T05:50:47.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T05:50:47.958 INFO:teuthology.run:pass